3rd argument to os.path.walk(): Why?

[Alex Martelli]

"Rob Hooft" <rob at hooft.net> wrote in message
news:m2vgsqw1k2.fsf at hooft.net...
>
>  AM> E.g., say you want to _count_ the files:
>
>  AM> import os
>
>  AM> class Counter:
>  AM>   def __init__(self):
>  AM>     self.count = 0
>  AM>   def inc(self, n=1):
>  AM>     self.count += n
>
>  AM> def count_names(count, dir, flist):
>  AM>   count.inc(len(flist))
>
>  AM> count = Counter()
>  AM> os.path.walk("/", count_names, count)
>
>  AM> print count.count,"files in all"
>
> If you'd do this, why not make "count_names" a method of the Counter
> class?

As things stand here, "Counter" is a completely general-purpose
class, which has no knowledge whatsoever of files, their names,
and path-walking -- it's completely decoupled from all of these
concerns, a low-level "utility class" that is reusable in all
sorts of contexts.

Strictly coupling a generic counter-class to specific things it
could be counting doesn't seem like an elegant design at all.

If anything, one might *derive* from Counter a NamesCounter
subclass, or *contain and delegate-to* a Counter from a wrapper
class -- assuming os.path.walk takes any callable (specifically,
a bound method is what we would pass here) even though the docs
specifically say it takes a *function* instead (the docs are
often too-restrictive regarding object-types expected by core
Python functions...).

Once that is feasible, then both idioms for setting callbacks
are equally elegant:
1. the classical procedural set-callback idiom, where you
    provide a function (typically not carrying state) and
    the state to be passed to it as an argument when the
    calling-back does happen;
2. the strongly object-oriented set-callback idiom, where
    you provide a callable object which entirely takes care
    of its own state issues.

The tradeoffs between these two design approaches are,
more or less, 'the usual suspects'.  One way to frame [1]
is strictly object-oriented terms is as an application
of the "Memento" design-pattern -- the 'function' might
still be a more-general 'callable', but some part of its
state is objectified as a separate 'memento' object that
is explicitly handled back to it as needed.

"Memento" is one of many important ways in which issues
of object-identity and object-state can be decoupled, and
thereby handled more explicitly, with finer-grained
control.  (One doesn't always _want_ to control things
on a fine-grained level, which is why it's nice to have
different idioms available in the language/framework:-).


Alex