Update locals()

[holger krekel]

On Mon, Apr 29, 2002 at 12:25:56AM +0200, Alex Martelli wrote:
> > Thanks for this transformation. But of course a 'but' follows :-)
> > As soon as
> >
> > - you want to adapt the definition to a filter's method signature
> >
> > - have templated or parametrized code in the function body
> >
> > - want to see what actually gets defined
> >   (with my version i can easily write or print the
> >    actual function's definition and look at a real definition)
> >
> > it might get difficult or impossible to do it your way.
> 
> It might, but I asked you to provide an example, you did, and I recoded
> it in a way that I consider preferable (without any exec or eval).  If you
> want to continue this exercise, despite its proving rather sterile, please
> keep providing examples for me to recode, rather than handwaving
> generalizations as in those first two points.

Why is "adapting to the definition of a methods signature" handwaving?
Previously we were both using '*args,**kwargs' while
the adapted function definition ('access') does not have any elipsis.

ok. so the code would *basically* look like this:

   from inspect import formatargspec, getargspec
   for name in statefulList:
        sig = apply(formatargspec, getargspec(getattr(filters,name))) 
        sig = '(self,'+sig[1:]
        exec """
            def %(name)s %(sig)s:
                    ...
        """

Please note that i said *basically*. This does not work if 
filters.name is a callable class in which case the __init__ signature
is the reference.

Point two (templated/parametrized code) is clarified below.

> Point three is murky to me.  "The actual function definition" you can't
> either see or print with your method -- 
> ...
> Source code can be seen or printed at runtime without needing it to
> come from a string exec'd with default dictionaries, as is pretty
> obvious.  So, if this point has any importance, you should restate it.

Oh well. (Of course) i mean the templated function definition 
which i want to see expanded just as the parser&compiler sees it. 
Not the function which it delegates to (which i already showed
in source code...). Strange misunderstanding.

> > Writing a reusable helper function i could say something like
> >
> >     exec make_adapters(filters, statefulList, '''
> >     def %name (self, %args):
> >         """%__doc__"""
> >         return %sourcescope.%name (%args) (self)
> >     ''')
> >
> > or even better make the templated adapter definition a defaulted
> > argument. This provides more flexibility for the *caller* of the helper
> > function without ever having to modify the helper function.
> 
> I _thought_ I knew what the term "unmaintainable" meant.  This, however, 
> takes this important term to levels I had never dreamed of.  Any bug at
> all in this complicated, tightly-coupled nest of function and data, any 
> little discrepancy, will be hell to find out and fix.

1. It's seems at least as maintainable as - say - the various html-templating
   engines and embedded code stuff. Or DTML for that matter :-)
   (Agreed, it's not pretty)

2. I can write out (in make_adapters) the actual (expanded) definitions 
   to stdout or a file. There i can look at it as if i had written an 
   ordinary function (read: no templating/weird constructions). Easy to debug.

3. it's not that my whole program consists of these things. It's kept
   only to very few places. 

> ...
> strings of sourcecode are not the best ways to
> sling code around in a language where functions and classes are first-class 
> objects).  

What hinders make_adapters to return code objects or in other words:
since when is exec restricted to strings? Sorry, i don't get it.
exec can execute code objects as well as strings. 

> > Also i use the template 'exec in current scope' method to allow
> >
> > resultfilters = "isfile and nolink and fnmatch('data*')"
> >
> > to be compiled at run time into a function and then be
> > applied to - say - 15000 files. This technique is similar
> > to what the regular expressions module does with re.compile.
> 
> *NO*!  re.compile NEVER mucks with your local namespace.
> Any "similarity" may be with functions such as the built-in compile
> (see below), never with exec-in-local-scope, which *does*
> have uncontrollable alterations of namespace as its only reason to exist.

[ i must, i must, i must be much more careful with respect to 
  possible (evil) misinterpretations ]

OF COURSE i didn't mean that re.compile mucks with the local namespace.
This thread started (and still has this as the subject line !!!)
about "updating locals" with a re-pattern's match.groupdict(). 
Remember that we both came up with a 3-liner to achive the goal
AND that i AGREED that this is better than updating locals.

re.compile compiles a pattern into a pattern object which
can be used for searches. In the above case the helper
uses exec to dynamically construct a function object from
a pattern. That's why i said 'similar' technique. Why are
you often assuming the worst of all interpretations? 

The code that gets "isfile and nolink and fnmatch('data')" 
makes a function object using 'exec' in local scope.
It could use a dedicated dictionary, agreed. But why should it?

> > This may be a better example than the original one?!
> > At least i don't see how you can apply your techniques
> > to it without using slow 'evals' or a slow interpretation
> 
> You seem to be unaware that built-in function 'compile' is able to make 
> code objects, that eval is not slow at all when passed a code object, 
> and/or that the functions of module new also let you build functions.
> These are not "my" technique, btw, but Python's.

i am not unaware of this fact. Not at all. I just thought that providing
strings is more readable and didn't bother to send it through a 
'compile' statement.  

> > None.strip() in
> >
> > ...__dict__.get(name).__doc__.strip()
> >
> > would fail at run time if a filters's doc has not been defined.
> > Same with your code.
> 
> Yes, but later you access filter.name unconditionally, and filter
> is what is at the point of the ellipsis.  The 'get', with or without a
> default 2nd argument, has nothing to do with whether a specific
> named filter has or lacks a docstring, but only with whether a
> filter of that name exists in the filter module.  And if no filter of
> that name exists, the method is going to fail at runtime anyway.
> So I still don't see why you're using this peculiar construct.

sorry. my mistake. Wrong translation from thought to code.
i meant to say 

getattr(filters.__dict__.get(name), '__doc__', undoc).strip()

the name will be there and a runtime-error is ok otherwise. 
It's just the undefined __doc__ which i didn't want to
cause a runtime error. Let's drop this issue.

> > The only better thing i can imagine is if you tried a tiny bit
> > harder to see some *valid* points rather than only pin down invalid
> > points. Otherwise you give me the feeling that i am completly on the
> > wrong path. (or is this intended?!)
> 
> I do believe you are "completely on the wrong path" in wanting to
> use exec in local scope.  I am certainly open to being convinced,
> should some example be brought that I don't find a way to recode
> which seems preferable to me; it just appears to me to be highly
> unlikely for such an example to emerge.

You may be right. 

> > I just try to communicate a use for "exec" where it is preferrable
> > to not use explicit dictionaries.
> 
> I think "exec in local scope" is invariably overkill, and the more highly
> disciplined tools that Python provides in abundance afford better,
> faster and more maintainable ways to perform even tasks so dynamic
> that their very wisdom is dubious.

yes, let's hope so. 

> I think one good language, quite close to Python in most respects, that may
> support better your specific quest of runtime compilation of source in
> order to alter namespaces, is Ruby.  In Python, you're fighting against
> the grain of the language when you do this.  In Ruby, you are closer
> to the mainstream of the language -- you may also alter in such ways
> built-in types akin to Python's object, string, dict, etc, while Python does
> not let you do that.  I think Python's approach is wiser, and Ruby's leads
> to a programming style damaging the ability to understand and maintain
> large programs (but I haven't tackled 'big' projects in Ruby, so this is
> only a working hypothesis for me so far).  But I suspect you might disagree,
> and perhaps find yourself more at home in Ruby.  Do give it a look.

sorry. i am not going away from python :-/ Like i said earlier
it's not that altering namespaces is my prefered technique everywhere. 
I just wanted (and am obviously failing) to discuss a possible use of 
execing in current scope. 

If you are coming to EuroPython maybe we find half an hour where we 
can discuss it on the screen? Doing this in mail makes it hard to not 
misunderstand each other. 

It's especially frustrating if you continue to assume that i would think 
're.compile modifies the callers namespace' or 'exec only takes strings' 
or 'i always want to alter namespaces from everywhere' or
'source code cannot be seen at runtime' or ...
and finally try to get me to Ruby. I am not a speaking japanese :-)

regards,

    holger