Update locals()

[Alex Martelli]

On Sunday 28 April 2002 22:57, holger krekel wrote:

	[trying to compact this down again]

[about locals() and globals()]

> Are you sure that this holds?

You're right in your quotes from the library manual and I can't easily
find out the various spots from where, collectively, I formed a much
stronger opinion (could be python-dev messages, comments in code,
etc).  If this matters to you I suggest you ask for official clarification
(e.g. on sourceforge) -- e.g., is it just a current implementation accident 
that locals()['x']='y' either changes the local namespace or doesn't but 
never raises exceptions, or is it a guarantee for future language versions. 
I believe the former holds.

> scope. But anyway we certainly agree that "exec" without optional
> arguments *does* execute in current scopes (no matter what behaviour the
> objects returned by locals()/globals() have). We are only arguing whether
> there is some good use for execing in current scopes.

Yes.  You brought up the irrelevant aside that locals and globals could be
passed anyway, even if one did require always passing a dictionary, and
I wanted to point out that this wasn't the case (under semantic 
equivalence).


> Hmmm. certainly a possible update-path.
> I really like about python that you can refer to classes,
> object's attributes/methods, the global/local bindings
> as objects and work with it. This is (one of) the goal

I agree _except_ for the local and global namespaces -- because [a] you 
can't (see above), [b] it's best that you can't.

> that many other current languages are missing. Nobody says that you don't
> have to care about any side effects this may and will have.

Sorry, but the last sentences is totally mysterious to me.


> > def addMethods(class_, filters, statefulList):
> >     for name in statefulList:
> >         delegatee = getattr(filters, name)
> >         def method(self, *a, **k): return delegatee(*a, **k)(self)
> >         method.__doc__ = getattr(filters, name, undoc).__doc__.strip()
> > + '\n' setattr(class_, name, method)
>
> Thanks for this transformation. But of course a 'but' follows :-)
> As soon as
>
> - you want to adapt the definition to a filter's method signature
>
> - have templated or parametrized code in the function body
>
> - want to see what actually gets defined
>   (with my version i can easily write or print the
>    actual function's definition and look at a real definition)
>
> it might get difficult or impossible to do it your way.

It might, but I asked you to provide an example, you did, and I recoded
it in a way that I consider preferable (without any exec or eval).  If you
want to continue this exercise, despite its proving rather sterile, please
keep providing examples for me to recode, rather than handwaving
generalizations as in those first two points.

Point three is murky to me.  "The actual function definition" you can't
either see or print with your method -- it's hidden somewhere in
filters, and all your method is doing is delegating to it.  (Maybe you
mean something other than 'method' when you say 'method', which
is after all a specific technical term in Python, but then you'll have to
rephrase, because even mentally translating e.g. 'method' into
'approach' this is still quite unclear).

Source code can be seen or printed at runtime without needing it to
come from a string exec'd with default dictionaries, as is pretty
obvious.  So, if this point has any importance, you should restate it.


> Writing a reusable helper function i could say something like
>
>     exec make_adapters(filters, statefulList, '''
>     def %name (self, %args):
>         """%__doc__"""
>         return %sourcescope.%name (%args) (self)
>     ''')
>
> or even better make the templated adapter definition a defaulted
> argument. This provides more flexibility for the *caller* of the helper
> function without ever having to modify the helper function.

I _thought_ I knew what the term "unmaintainable" meant.  This, however, 
takes this important term to levels I had never dreamed of.  Any bug at
all in this complicated, tightly-coupled nest of function and data, any 
little discrepancy, will be hell to find out and fix.

Even if I did need to have a function take a string template and make
code from it, I would never have it return strings to be exec'd in such
a way.  Rather, I'd return a dictionary (or sequence of pairs) with
names and function objects, and install them as needed caller-side
(again in a metaclass or after the class statement), either inline or in
an auxiliary function -- e.g. supposing inline, and a defaulted string
template:

    for name, function in make_adapters(filters, statefulList):
        setattr(filename, name, function)

THIS gives flexibility to the caller (which may easily install the 
functions in different objects and/or with changed names as need be)
AND to function make_adapters (which may generate the needed
functions in any of several different ways, rather than being constrained 
to returning a string -- strings of sourcecode are not the best ways to
sling code around in a language where functions and classes are first-class 
objects).  I suspect this might also be slightly more maintainable (although
not GOOD in this way, maybe not quite as horrible as the version based
on exec-in-local-scope).


> Also i use the template 'exec in current scope' method to allow
>
> resultfilters = "isfile and nolink and fnmatch('data*')"
>
> to be compiled at run time into a function and then be
> applied to - say - 15000 files. This technique is similar
> to what the regular expressions module does with re.compile.

*NO*!  re.compile NEVER mucks with your local namespace.
Any "similarity" may be with functions such as the built-in compile
(see below), never with exec-in-local-scope, which *does*
have uncontrollable alterations of namespace as its only reason to exist.

> This may be a better example than the original one?!
> At least i don't see how you can apply your techniques
> to it without using slow 'evals' or a slow interpretation

You seem to be unaware that built-in function 'compile' is able to make 
code objects, that eval is not slow at all when passed a code object, 
and/or that the functions of module new also let you build functions.
These are not "my" technique, btw, but Python's.


> None.strip() in
>
> ...__dict__.get(name).__doc__.strip()
>
> would fail at run time if a filters's doc has not been defined.
> Same with your code.

Yes, but later you access filter.name unconditionally, and filter
is what is at the point of the ellipsis.  The 'get', with or without a
default 2nd argument, has nothing to do with whether a specific
named filter has or lacks a docstring, but only with whether a
filter of that name exists in the filter module.  And if no filter of
that name exists, the method is going to fail at runtime anyway.
So I still don't see why you're using this peculiar construct.


> I appreciate your explanations and code very much.
>
> The only better thing i can imagine is if you tried a tiny bit
> harder to see some *valid* points rather than only pin down invalid
> points. Otherwise you give me the feeling that i am completly on the
> wrong path. (or is this intended?!)

I do believe you are "completely on the wrong path" in wanting to
use exec in local scope.  I am certainly open to being convinced,
should some example be brought that I don't find a way to recode
which seems preferable to me; it just appears to me to be highly
unlikely for such an example to emerge.

> I just try to communicate a use for "exec" where it is preferrable
> to not use explicit dictionaries.

I think "exec in local scope" is invariably overkill, and the more highly
disciplined tools that Python provides in abundance afford better,
faster and more maintainable ways to perform even tasks so dynamic
that their very wisdom is dubious.

'GOTO' (in languages that have it) may no doubt let you construct
your own data structures, finely tuned to your tastes and what you
see as the task at hand.  However, the more highly disciplined
control-flow tools that Python provides in abundance afford better,
faster, and more maintainable ways to control your program's flow.
I think the analogy is pretty strict, and long for a Python whose
exec was a function taking mandatory dict arguments.  Oh well.


I think one good language, quite close to Python in most respects, that may
support better your specific quest of runtime compilation of source in
order to alter namespaces, is Ruby.  In Python, you're fighting against
the grain of the language when you do this.  In Ruby, you are closer
to the mainstream of the language -- you may also alter in such ways
built-in types akin to Python's object, string, dict, etc, while Python does
not let you do that.  I think Python's approach is wiser, and Ruby's leads
to a programming style damaging the ability to understand and maintain
large programs (but I haven't tackled 'big' projects in Ruby, so this is
only a working hypothesis for me so far).  But I suspect you might disagree,
and perhaps find yourself more at home in Ruby.  Do give it a look.


Alex