Catalog of Python Patterns? [LONG!]

[Brett Cannon]

Have you looked at the Python Cookbook
(http://aspn.activestate.com/ASPN/Cookbook/Python)?  There are several GoF
patterns already there and done by some of the top Python programmers out
there.  There are even some Python-skewed version of some of these
patterns (the Borg pattern by Alex Martelli comes to mind).

-Brett C.



On 4 Apr 2002, Jim Dennis wrote:

>
>  GoF Design Patterns in Python
>
>  [Warning LONG POST AHEAD!]
>
>  I'd like to see a canonical catalog of OODP in Python.
>
>  At a minimum it should list each of the 23 GoF patterns, and
>  at least one minimal Python implementation of that pattern.
>  It should also point out the "native" use of each pattern in
>  the Python core, standard modules or among popular publicly
>  available third-party modules, packages and applications.
>  Naturally it should also discuss alternatives (Borg vs. Singleton)
>  and issues (particularly cases where Pythonic idioms make
>  patterns trivial, for example).
>
>  I realize that there used to be a "Python-Patterns" SIG (now
>  "retired") and a Google search will reveal a number of articles
>  on the topic (such as the one by Vespe Savikko).  Unfortunately
>  none of these sources is comprehensive.  Not one of these articles
>  or web sites shows a list of all 23 GoF patterns with a minimal
>  Python implementation.  Also many of them are somewhat dated.
>  I'd like to focus on Python 2.2 features (nods and caveats regarding
>  older versions can be added later).
>
>  I'll be the first to admit that I'm not a "Patterns" expert.
>  That's why I'd like to see *simple* (as it brain-dead, dirt,
>  simple) examples.  I'm also not an experienced professional
>  OO programmer.  I consider myself to be a student in this field.
>  So all of my comments should be take with a large lump of salt,
>  perhaps with a whole 5 lb. salt lick (for you equestrians).
>
>  Here's a start.  (All I've done is copy the GoF 23 from the
>  inside cover of their book, and inserted my questions, comments
>  and suggestions.  Note that many of my comments are laced with
>  the phrase "I imagine" or qualifiers like "maybe" or "could" ---
>  like I said, I'm just studying this stuff and I have little
>  practical experience with it).
>
>  Who wants to contribute "text book" sample implementations?
>
>  Creational::
>    Abstract Factory:
>     I've never understood the difference between an "abstract factory"
>     and a "factory method."  Since Python doesn't really have
>     "abstract" versus "concrete" classes, perhaps the distinction is
>     meaningless.
>
>     Every Python class acts like a factory.  anObj = Foo() returns
>     a reference to a new "Foo" object.  Additionally Python allows
>     us to define normal functions which return object references.
>     Indeed there is no way to tell from this assignment whether Foo
>     is a class or a function.  It could easily be a function that
>     returns an object from a pool or one that returns a reference to
>     a singleton. (Of course having a function masquerading as a
>     "class" in this way would inelegant and might violate the
>     principle of least surprise for some of the programmers using
>     or extending this code).
>
>     In Python2.2 it's possible to create/over-ride a class method
>     with the reserved __new__ name so that we have greater control
>     over the new instance creation (instantiation).  Guido shows
>     how __new__ can be used to implement a Singleton. It should be
>     easy to use __new__ to implement object pools (is that a
>     non-GoF pattern?), or any of creational patterns. (see below)
>
> 	Clearly, for a number of patterns, one object will be acting
> 	as some form of dynamic interface for one or more objects
> 	"behind it."  (Decorator, Adapter, Bridge, Proxy, Facade,
> 	etc).  The __init__ method (constructor) may have to instantiate
> 	these back-end objects, storing references to them as instance
> 	members.  The resulting "front-end" object must then dispatch
> 	method invocations (and attempted member accesses) to the
> 	appropriate back-end object(s) according to the required semantics.
>
> 	Indeed this model seems to be a unifying theme among many of the
> 	GoF patterns.  A Decorator enhances a back-end object by adding
> 	it's to members or methods to it (like a wrapper) and dispatching
> 	"base" methods back into it's instance of the underlying object.
> 	Bridges and Adapters reformulate method calls as the pass them
> 	back.  A Proxy selectively rejects some method invocations, or
> 	over-rides them, to "protect" the underlying object in some way.
> 	A Facade allows a group of objects to appear as a single object;
> 	it dispatches methods to the appropriate back-end instances.
> 	(In any of these cases the "front-end" might also wrapper or
> 	over-ride the return values).
>
> 	Quite a bit of the motivation and inspiration for the GoF
> 	patterns seems to be in recognition of the limitations of
> 	inheritance and class hierarchies.  They talk quit a bit about
> 	how these patterns allow OO programmers to avoid common
> 	pitfalls that result from attempts to use inheritance for
> 	"has a" rather than "is a" relationships, for cases where
> 	objects need to interact with other objects or in ways OTHER
> 	than inheritance.
>
> 	[Perhaps I'm wrong about this, but this is my take on the GoF
> 	patterns. BTW: the term "delegation" might be useful here.]
>
>    Builder:
>      I guess that "Builders" would often be used for "Composite"
> 	 objects.
>
>    Factory Method:
>      Are there any cases in Python 2.2 where this would be implemented
> 	 through something other than some combination of code in __new__
> 	 and/or __init__?
>
>    Prototype:
>      Are there any examples of this in the Python distribution?
> 	 I guess the implementation would usually be to create a
> 	 _clone() or __clone() method which would instantiate a blank
> 	 object (clone = self.__class__()?) passing it appropriate
> 	 constructor arguments (which it must have cached in its own
> 	 __init__) and calling on whatever methods are necessary to
> 	 put the clone into the desired (mostly identical) state as
> 	 the prototypical instance.  Finally the clone() function
> 	 would return a reference to the newly cloned instance.
>
> 	 I suppose one could have a Prototype (possibly some
> 	 selected instances of a class) define the __call__ method to
> 	 perform the cloning.  This would make the prototype look like
> 	 a factory or a normal class object throughout the rest of the
> 	 program.  I could envision a "Prototype" function that would
> 	 convert (promote?) a "normal" instance variable into a
> 	 Prototype by causing it to bind its __call__ method to the
> 	 necessary cloning operations.  This might look a little like
> 	 the way that one defines a class or static method and "promotes"
> 	 using syntax of the form:  mymethod = staticmethod(mymethod).
> 	 (Obviously the same function could be used to create a
> 	 new Prototype from any extent instance: fooProto = Prototype(inst)
> 	 instead of myproto = Prototype(myproto); the only difference
> 	 is whether you bind the result to the old name or a new one).
>
> 	 One question that naturally comes up with regard to Prototypes
> 	 is this: How does Python's introspection features facilitate
> 	 the implementation of Prototypes.  How much of the cloning
> 	 operation can be standardized and encapsulated in our
> 	 (hypothetical) "Prototype()" function (or mix-in class?)?
> 	 [I can imagine all sorts of clever fiddling with self.__dict__
> 	 et al --- possibly using the deepcopy module/functions; but
> 	 the actual fiddling is beyond me at this point, so it's
> 	 best left to the imagination].
>
>    Singleton:
>      Alex Martelli has long been a proponent of the Borg pattern
>      as an alternative to the Singleton.  While his point is well
> 	 taken (the focus should be on behavior, not object identity)
> 	 I think of the Borg as a way to work around limitation in the
> 	 earlier versions of Python.  With the introduction of new
> 	 classes and the ability to create static methods and over-ride
> 	 __new__ there is no need to use the Borg.
>
> 	 Incidentally the Borg is yet another example of the delegation
> 	 theme that I discussed before.  It redirects all access to
> 	 any member of the collective to the "queen" (back-end) instance.
>
>  Structural::
>    Adapter:
>    Bridge:
>      Frankly I don't see much difference between these two patterns.
> 	 I guess anygui might use Adapters to provide one uniform
> 	 interfacee to GUI widgets while adapting method calls into the
> 	 various semantics of back-end GUIs and widget collections.
> 	 A Bridge does the same thing.  GoF makes the distinction on
> 	 "intent" (Adapters being used to access legacy or external
> 	 classes through a common API, Bridges being a design decision
> 	 to decouple one's own classes from their APIs; it sounds like
> 	 hair splitting to me -- the same implementation techniques
> 	 should work in either case.
>
> 	 I can imagine cases where an Adapter or Bridge might have to
> 	 also be a bit of a Decorator or a Facade. If the semantics of
> 	 one API differ from another, for example, one might have to
> 	 instantiatiate additional backend objects to supply enough
> 	 functionality to support the desired common interface.  Otherwise
> 	 your interface is constrained to a least common denominator
> 	 among the possible back-end classes.
>
> 	 In these (and all of the other "delegation" patterns) I can
> 	 imagine lots of tricky, clever, subtle (and "too" extremes
> 	 of each: as in "too tricky," "too clever," and "too subtle")
> 	 ways of using __getattr__ and __setattr__ methods and the
> 	 hasattr(), to avoid explicit enumeration of the back-end
> 	 methods for those cases where the backend does have the
> 	 same interface as the Adapter/Bridge.  In other words it
> 	 may be possible through __getattr__ and __setattr__ or
> 	 similar special Python methods to allow any of these delegate
> 	 Patterns to only over-ride the necessary functions while
> 	 transparently passing along any other methods (and transparently
> 	 passing back results or raising exceptions?).
>
> 	 What are some pointers and caveats about this sort of "dynamic
> 	 delegation" (sort of "inheritance from outside of the family
> 	 tree")?
>
>    Composite:
>      The GoF refers to Composites as "tree structures."  This strikes
> 	 me as odd since I can imagine many composites of objects that
> 	 aren't trees.  I can imagine chains, directed graphs, networks,
> 	 and pools of objects that are interconnected through instance
> 	 members.  I can also see where one might have class/static
> 	 members which track all of the extant instances (__new__ might
> 	 append to "__class__.__instancelist" and __del__ might perform
> 	 the remove.  Could __del__ "divert" the destruction, blank the
> 	 instance and add it to a pool?  Might one define a wrapper around
> 	 the del() built-in that would do something like:
>
> 		def del(arg):
> 			if hasattr(arg,'__pool__'): arg.__pool__()
> 			else:                     _del(arg)
>
> 	But I gather (from reading the GoF) that the object here
> 	(to excuse the pun) is to allow for a method to propagate
> 	transparently to any subset of the composite regardless
> 	of whether that subset effects the whole tree (invoke it
> 	on the root node) or an individual leaf (the degenerate
> 	case).  So I guess the core of a Python Composite with
> 	support for a given method's propagation might be:
>
> 		Class CompositeFoo:
> 			def __init__(self):
> 				self.children = []
> 			def propagated_method(self, *args, **kwargs):
> 				# do stuff to self
> 				if supposed_to_propagate:
> 					for i in self.children:
> 						i.propagated_method(args, kwargs)
>
> 	[Does that make sense?  BTW: I'm envisioning some cases
> 	where the propagation is intended to be limited or conditional]
>
> 	So, what are the caveats?  Are there any examples in Python
> 	itself?  Are there any coding conventions that might be
>     encapsulated into an abstract base class or a mix-in?
>
>    Decorator:
>    Facade:
>      As I've already said, Decorators and Facades, Adapters, Proxies
> 	 and Bridges (and Mediators and possibly even State objects all
> 	 seem to be forms of delegation (or collaboration?).  In all
> 	 cases there is a delegate (front end) object which is "used"
> 	 in our code and it instantiates one or more of the underlying
> 	 (back end) objects as its own member(s).  The delegate then
> 	 dispatches or relays method invocations back to the underlying
> 	 object(s) according the the delegate's semantics.
>
> 	 Am I missing something here?  Is this a gross
> 	 oversimplification?  Are there any differences among these
> 	 patterns that require special implementation (for Python)
> 	 or to the same language features and techniques apply to
> 	 all of these "delegations" (my term).
>
>    Flyweight:
>      I've read innumerable times that the Python interpreter
> 	 implements a set of static immutable objects for a small
> 	 range of integers (-1 or -2 through 100 or so?).  I guess
> 	 this is an example of Python's native use of the Flyweight
> 	 pattern.
>
> 	 Are there any others?  Would it be possible to have a
> 	 sort of "copy on write" (COW) Flyweight?  These would behave
> 	 like Singletons for read-only purposes, but then be promoted
> 	 to separate/new objects on any setattr?  What would be a
> 	 use for such a beast.
>
>    Proxy:
>      See my comments for Decorators, Facades, Adapters, and Bridges.
>
>
>  Behavioral::
>    Chain of Responsibility:
>      A cascaded of Decorators or a Chain of responsibility.
> 	 What's the difference.
>
>    Command:
>      Some of the GoF's comments on this sound like it would
> 	 be ideal for supporting AOP (aspect oriented programming).
>
>    Interpreter:
>      I just don't get this pattern.  I'll have to mull on it
> 	 for a bit.  Other the trite observation that Python *is*
> 	 an interpreter, are there any examples of the Interpreter
> 	 pattern in Python?
>
>    Iterator:
>      Iterators seem to be evolving into one of Pyton's most
> 	 visible and commonly used features.  In Python2.2 we
> 	 see automatic iteration over files, and dictionaries as
> 	 well as the traditional iteration over built-in sequences
> 	 (lists, tuples, strings).  We also see support for defining
> 	 iterators for our own classes.
>
> 	 It seems like iterators and generators are intertwined in
> 	 someway, but I don't understand it.  [So maybe I'm just
> 	 wrong and they only seem to be related because they're
> 	 described in the same docs, or something].
>
>    Mediator:
>      This seems like a sort of bi-directional Facade to me.
>
>    Memento:
>      Here's where Python's "encapsulation by courtesy" and lack of
> 	 enforced data hiding might make a GoF pattern particularly
> 	 easy; almost unecessary.  Alternatively I can imagine that
> 	 some implementation of the "Memento" pattern might involve a
> 	 coding convention where each class that supports it implements
> 	 an __memoize() method.  Might that return a pickle of self?
>
>    Observer:
> 	 I can imagine where some objects, including composite
> 	 objects might "register" with some sort of "registry" (sorry
> 	 for the redundancy).  I'm just not sure what sort of applications
> 	 one my find for this sort of "registry" even though I'm sure
> 	 it would be easy (almost trivial) to implement in Python.
>
> 	 I suppose such a "registry" makes quite a bit of sense in the
> 	 Observer pattern.  Although I gather that this would usually be
> 	 a instance registry in each of the "observer/dispatchers"
>
>    State:
>      I see this as similar to an Adapter except that it may
> 	 change which back end object(s) it dispatches to (or calls
> 	 upon or delegates for) might change dynamically during the
> 	 lifetime of a given instance.  For example I can imagine an
> 	 object that implements a simple list for some of its members
> 	 until the list gets beyond a given size, then it might
> 	 change that to a dictionary, or some sort of AVL tree or
> 	 more complex data structure at the object needs to scale
> 	 upwards.
>
>    Strategy:
>    Template Method:
>      Would some of the mix-ins from SocketServer count as
> 	 examples of the Strategy or Template Method patterns?
>
>    Visitor:
>      os.path.walk is an obvious example of a standard Python
> 	 module/function that uses Visitor.  (Obvious because it's
> 	 documented as the "visitor" function).
>
>      However, this is also one of the most confusing functions
> 	 in the standard Python library.  I seem to see  befuddled
> 	 posts about it in the c.l.py newsgroup at least once a month.
> 	 As the GoF point out, Visitors are often used on Composites.
>
> 	 It seems like generators might often be alternatives to
> 	 Visitors.  Rather than passing a function to each object
> 	 and saying "do this to yourself" it seems natural to
> 	 iterator over a set of object references and invoke the
> 	 methods (or even, horrors! play with its attributes) directly.
>
> 	 However, Python's ability to pass functions, objects and
> 	 closures around as first order data types seems to make
> 	 Visitors easy to implement --- if you can just understand
> 	 the scoping well enough for the Visitor to know what to
> 	 do (and who to do it to) when it gets there.
>
>