Catalog of Python Patterns? [LONG!]

[Jim Dennis]

 GoF Design Patterns in Python

 [Warning LONG POST AHEAD!]

 I'd like to see a canonical catalog of OODP in Python.

 At a minimum it should list each of the 23 GoF patterns, and
 at least one minimal Python implementation of that pattern.
 It should also point out the "native" use of each pattern in
 the Python core, standard modules or among popular publicly 
 available third-party modules, packages and applications.  
 Naturally it should also discuss alternatives (Borg vs. Singleton)
 and issues (particularly cases where Pythonic idioms make 
 patterns trivial, for example).

 I realize that there used to be a "Python-Patterns" SIG (now
 "retired") and a Google search will reveal a number of articles
 on the topic (such as the one by Vespe Savikko).  Unfortunately 
 none of these sources is comprehensive.  Not one of these articles
 or web sites shows a list of all 23 GoF patterns with a minimal
 Python implementation.  Also many of them are somewhat dated.  
 I'd like to focus on Python 2.2 features (nods and caveats regarding
 older versions can be added later).

 I'll be the first to admit that I'm not a "Patterns" expert.
 That's why I'd like to see *simple* (as it brain-dead, dirt,
 simple) examples.  I'm also not an experienced professional 
 OO programmer.  I consider myself to be a student in this field.
 So all of my comments should be take with a large lump of salt,
 perhaps with a whole 5 lb. salt lick (for you equestrians).

 Here's a start.  (All I've done is copy the GoF 23 from the 
 inside cover of their book, and inserted my questions, comments 
 and suggestions.  Note that many of my comments are laced with
 the phrase "I imagine" or qualifiers like "maybe" or "could" ---
 like I said, I'm just studying this stuff and I have little
 practical experience with it).

 Who wants to contribute "text book" sample implementations?

 Creational::
   Abstract Factory:
    I've never understood the difference between an "abstract factory"
    and a "factory method."  Since Python doesn't really have 
    "abstract" versus "concrete" classes, perhaps the distinction is
    meaningless.

    Every Python class acts like a factory.  anObj = Foo() returns
    a reference to a new "Foo" object.  Additionally Python allows
    us to define normal functions which return object references.
    Indeed there is no way to tell from this assignment whether Foo
    is a class or a function.  It could easily be a function that 
    returns an object from a pool or one that returns a reference to
    a singleton. (Of course having a function masquerading as a 
    "class" in this way would inelegant and might violate the 
    principle of least surprise for some of the programmers using
    or extending this code).

    In Python2.2 it's possible to create/over-ride a class method
    with the reserved __new__ name so that we have greater control
    over the new instance creation (instantiation).  Guido shows
    how __new__ can be used to implement a Singleton. It should be
    easy to use __new__ to implement object pools (is that a 
    non-GoF pattern?), or any of creational patterns. (see below)

	Clearly, for a number of patterns, one object will be acting
	as some form of dynamic interface for one or more objects 
	"behind it."  (Decorator, Adapter, Bridge, Proxy, Facade, 
	etc).  The __init__ method (constructor) may have to instantiate
	these back-end objects, storing references to them as instance
	members.  The resulting "front-end" object must then dispatch
	method invocations (and attempted member accesses) to the 
	appropriate back-end object(s) according to the required semantics.

	Indeed this model seems to be a unifying theme among many of the
	GoF patterns.  A Decorator enhances a back-end object by adding
	it's to members or methods to it (like a wrapper) and dispatching
	"base" methods back into it's instance of the underlying object.
	Bridges and Adapters reformulate method calls as the pass them
	back.  A Proxy selectively rejects some method invocations, or
	over-rides them, to "protect" the underlying object in some way.
	A Facade allows a group of objects to appear as a single object;
	it dispatches methods to the appropriate back-end instances.
	(In any of these cases the "front-end" might also wrapper or
	over-ride the return values). 

	Quite a bit of the motivation and inspiration for the GoF 
	patterns seems to be in recognition of the limitations of 
	inheritance and class hierarchies.  They talk quit a bit about
	how these patterns allow OO programmers to avoid common
	pitfalls that result from attempts to use inheritance for 
	"has a" rather than "is a" relationships, for cases where 
	objects need to interact with other objects or in ways OTHER
	than inheritance.
	
	[Perhaps I'm wrong about this, but this is my take on the GoF
	patterns. BTW: the term "delegation" might be useful here.]

   Builder:
     I guess that "Builders" would often be used for "Composite" 
	 objects.  

   Factory Method:
     Are there any cases in Python 2.2 where this would be implemented
	 through something other than some combination of code in __new__
	 and/or __init__?

   Prototype:
     Are there any examples of this in the Python distribution?
	 I guess the implementation would usually be to create a 
	 _clone() or __clone() method which would instantiate a blank
	 object (clone = self.__class__()?) passing it appropriate 
	 constructor arguments (which it must have cached in its own
	 __init__) and calling on whatever methods are necessary to 
	 put the clone into the desired (mostly identical) state as
	 the prototypical instance.  Finally the clone() function 
	 would return a reference to the newly cloned instance.

	 I suppose one could have a Prototype (possibly some
	 selected instances of a class) define the __call__ method to
	 perform the cloning.  This would make the prototype look like
	 a factory or a normal class object throughout the rest of the
	 program.  I could envision a "Prototype" function that would
	 convert (promote?) a "normal" instance variable into a 
	 Prototype by causing it to bind its __call__ method to the
	 necessary cloning operations.  This might look a little like
	 the way that one defines a class or static method and "promotes" 
	 using syntax of the form:  mymethod = staticmethod(mymethod).
	 (Obviously the same function could be used to create a 
	 new Prototype from any extent instance: fooProto = Prototype(inst)
	 instead of myproto = Prototype(myproto); the only difference
	 is whether you bind the result to the old name or a new one).

	 One question that naturally comes up with regard to Prototypes
	 is this: How does Python's introspection features facilitate
	 the implementation of Prototypes.  How much of the cloning 
	 operation can be standardized and encapsulated in our 
	 (hypothetical) "Prototype()" function (or mix-in class?)?
	 [I can imagine all sorts of clever fiddling with self.__dict__
	 et al --- possibly using the deepcopy module/functions; but
	 the actual fiddling is beyond me at this point, so it's 
	 best left to the imagination].
	 
   Singleton:
     Alex Martelli has long been a proponent of the Borg pattern
     as an alternative to the Singleton.  While his point is well
	 taken (the focus should be on behavior, not object identity)
	 I think of the Borg as a way to work around limitation in the
	 earlier versions of Python.  With the introduction of new 
	 classes and the ability to create static methods and over-ride
	 __new__ there is no need to use the Borg. 

	 Incidentally the Borg is yet another example of the delegation
	 theme that I discussed before.  It redirects all access to 
	 any member of the collective to the "queen" (back-end) instance.

 Structural::
   Adapter:
   Bridge: 
     Frankly I don't see much difference between these two patterns.
	 I guess anygui might use Adapters to provide one uniform
	 interfacee to GUI widgets while adapting method calls into the
	 various semantics of back-end GUIs and widget collections.
	 A Bridge does the same thing.  GoF makes the distinction on 
	 "intent" (Adapters being used to access legacy or external 
	 classes through a common API, Bridges being a design decision
	 to decouple one's own classes from their APIs; it sounds like
	 hair splitting to me -- the same implementation techniques
	 should work in either case.

	 I can imagine cases where an Adapter or Bridge might have to
	 also be a bit of a Decorator or a Facade. If the semantics of 
	 one API differ from another, for example, one might have to 
	 instantiatiate additional backend objects to supply enough
	 functionality to support the desired common interface.  Otherwise
	 your interface is constrained to a least common denominator
	 among the possible back-end classes.

	 In these (and all of the other "delegation" patterns) I can
	 imagine lots of tricky, clever, subtle (and "too" extremes
	 of each: as in "too tricky," "too clever," and "too subtle")
	 ways of using __getattr__ and __setattr__ methods and the 
	 hasattr(), to avoid explicit enumeration of the back-end
	 methods for those cases where the backend does have the 
	 same interface as the Adapter/Bridge.  In other words it
	 may be possible through __getattr__ and __setattr__ or
	 similar special Python methods to allow any of these delegate
	 Patterns to only over-ride the necessary functions while
	 transparently passing along any other methods (and transparently
	 passing back results or raising exceptions?).

	 What are some pointers and caveats about this sort of "dynamic 
	 delegation" (sort of "inheritance from outside of the family 
	 tree")?
	 
   Composite:
     The GoF refers to Composites as "tree structures."  This strikes
	 me as odd since I can imagine many composites of objects that
	 aren't trees.  I can imagine chains, directed graphs, networks, 
	 and pools of objects that are interconnected through instance
	 members.  I can also see where one might have class/static 
	 members which track all of the extant instances (__new__ might
	 append to "__class__.__instancelist" and __del__ might perform
	 the remove.  Could __del__ "divert" the destruction, blank the
	 instance and add it to a pool?  Might one define a wrapper around
	 the del() built-in that would do something like: 

		def del(arg):
			if hasattr(arg,'__pool__'): arg.__pool__()
			else:                     _del(arg)

	But I gather (from reading the GoF) that the object here
	(to excuse the pun) is to allow for a method to propagate
	transparently to any subset of the composite regardless
	of whether that subset effects the whole tree (invoke it
	on the root node) or an individual leaf (the degenerate
	case).  So I guess the core of a Python Composite with
	support for a given method's propagation might be:

		Class CompositeFoo:
			def __init__(self):
				self.children = [] 
			def propagated_method(self, *args, **kwargs):
				# do stuff to self
				if supposed_to_propagate:
					for i in self.children:
						i.propagated_method(args, kwargs)

	[Does that make sense?  BTW: I'm envisioning some cases
	where the propagation is intended to be limited or conditional]

	So, what are the caveats?  Are there any examples in Python
	itself?  Are there any coding conventions that might be
    encapsulated into an abstract base class or a mix-in? 

   Decorator:
   Facade:
     As I've already said, Decorators and Facades, Adapters, Proxies
	 and Bridges (and Mediators and possibly even State objects all 
	 seem to be forms of delegation (or collaboration?).  In all 
	 cases there is a delegate (front end) object which is "used" 
	 in our code and it instantiates one or more of the underlying 
	 (back end) objects as its own member(s).  The delegate then 
	 dispatches or relays method invocations back to the underlying 
	 object(s) according the the delegate's semantics.  

	 Am I missing something here?  Is this a gross 
	 oversimplification?  Are there any differences among these
	 patterns that require special implementation (for Python)
	 or to the same language features and techniques apply to 
	 all of these "delegations" (my term).

   Flyweight:
     I've read innumerable times that the Python interpreter 
	 implements a set of static immutable objects for a small 
	 range of integers (-1 or -2 through 100 or so?).  I guess
	 this is an example of Python's native use of the Flyweight
	 pattern.  

	 Are there any others?  Would it be possible to have a 
	 sort of "copy on write" (COW) Flyweight?  These would behave
	 like Singletons for read-only purposes, but then be promoted
	 to separate/new objects on any setattr?  What would be a 
	 use for such a beast.

   Proxy:
     See my comments for Decorators, Facades, Adapters, and Bridges.


 Behavioral::
   Chain of Responsibility:
     A cascaded of Decorators or a Chain of responsibility.  
	 What's the difference.

   Command:
     Some of the GoF's comments on this sound like it would
	 be ideal for supporting AOP (aspect oriented programming).

   Interpreter:
     I just don't get this pattern.  I'll have to mull on it
	 for a bit.  Other the trite observation that Python *is*
	 an interpreter, are there any examples of the Interpreter
	 pattern in Python?

   Iterator:
     Iterators seem to be evolving into one of Pyton's most 
	 visible and commonly used features.  In Python2.2 we 
	 see automatic iteration over files, and dictionaries as
	 well as the traditional iteration over built-in sequences
	 (lists, tuples, strings).  We also see support for defining
	 iterators for our own classes.

	 It seems like iterators and generators are intertwined in
	 someway, but I don't understand it.  [So maybe I'm just
	 wrong and they only seem to be related because they're 
	 described in the same docs, or something].

   Mediator:
     This seems like a sort of bi-directional Facade to me.
	
   Memento:
     Here's where Python's "encapsulation by courtesy" and lack of
	 enforced data hiding might make a GoF pattern particularly 
	 easy; almost unecessary.  Alternatively I can imagine that 
	 some implementation of the "Memento" pattern might involve a
	 coding convention where each class that supports it implements
	 an __memoize() method.  Might that return a pickle of self?

   Observer:
	 I can imagine where some objects, including composite
	 objects might "register" with some sort of "registry" (sorry
	 for the redundancy).  I'm just not sure what sort of applications
	 one my find for this sort of "registry" even though I'm sure
	 it would be easy (almost trivial) to implement in Python.

	 I suppose such a "registry" makes quite a bit of sense in the 
	 Observer pattern.  Although I gather that this would usually be
	 a instance registry in each of the "observer/dispatchers"

   State:
     I see this as similar to an Adapter except that it may 
	 change which back end object(s) it dispatches to (or calls
	 upon or delegates for) might change dynamically during the
	 lifetime of a given instance.  For example I can imagine an
	 object that implements a simple list for some of its members
	 until the list gets beyond a given size, then it might 
	 change that to a dictionary, or some sort of AVL tree or
	 more complex data structure at the object needs to scale 
	 upwards.

   Strategy:
   Template Method:
     Would some of the mix-ins from SocketServer count as 
	 examples of the Strategy or Template Method patterns?
     
   Visitor:
     os.path.walk is an obvious example of a standard Python
	 module/function that uses Visitor.  (Obvious because it's
	 documented as the "visitor" function).

     However, this is also one of the most confusing functions
	 in the standard Python library.  I seem to see  befuddled 
	 posts about it in the c.l.py newsgroup at least once a month.
	 As the GoF point out, Visitors are often used on Composites.

	 It seems like generators might often be alternatives to 
	 Visitors.  Rather than passing a function to each object
	 and saying "do this to yourself" it seems natural to 
	 iterator over a set of object references and invoke the 
	 methods (or even, horrors! play with its attributes) directly.
	 
	 However, Python's ability to pass functions, objects and 
	 closures around as first order data types seems to make 
	 Visitors easy to implement --- if you can just understand
	 the scoping well enough for the Visitor to know what to
	 do (and who to do it to) when it gets there.