[Python-Dev] Proposed PEP: Optimizing Global Variable and Attribute Access

[Aahz Maruch]

[sorry, excessive quoting ahead]

Skip Montanaro wrote:
> 
> 
>     Aahz> Skip Montanaro wrote:
> 
>     >> The number of tracked objects should be relatively small.  All active
>     >> frames of all active threads could conceivably be tracking objects,
>     >> but this seems small compared to the number of functions defined in a
>     >> given application.
> 
>     Aahz> Hmmm?  That doesn't seem quite right to me.  That's just a
>     Aahz> knee-jerk reaction, I don't have any real evidence.
> 
> Here's my rationale.  Perhaps I've goofed somewhere.  TRACK_OBJECT and
> UNTRACK_OBJECT opcodes will only be inserted in the bodies of functions.
> Functions are only considered active when there is an active (but possibly
> suspended) eval_frame call on the C stack that is executing that function's
> byte code.  Suppose my module is
> 
>     def a():
>       print "a"
> 
>     def b():
>       print "b"
>       a()
> 
>     def c():
>       print "c"
> 
>     b()
> 
> At the point where a's print statement is executed, functions a and b are
> active.  Function c is not.
>
> This reasoning applies on a per-thread basis.  So the number of active
> functions by my definition is the number of times eval_frame appears on the
> call stack for all active threads.  In most situations that will be far less
> (barring deep recursion) than the number of functions available to the
> application.  All I guess I'm trying to communicate is that object tracking
> is a dynamic thing, not a static thing.

That's true.  My point was more that, in my knee-jerk opinion, if one
uses, say, math.sin() at all in a module, it will be active in a large
number of the callframes.  Therefore, the number of tracked objects
could easily get quite large.  It's a relative thing, of course, and I
still think your statistical analysis about cost/benefit is accurate.

>     Aahz> As a side note, I'm not sure whether some mention should be made
>     Aahz> that TRACK_OBJECT specifically does not work any differently from
>     Aahz> current Python when it comes to threads.  That is, if it's a
>     Aahz> shared variable of any sort, you need a mutex if you're going to
>     Aahz> perform multiple operations on it.
> 
> That's true.  If a global object is shared among multiple threads, access to
> it must still be protected.  I haven't added anything to the mix in that
> regard.  I'll add a short section on threads.
> 
> Hmm...  What about this (dumb) code?
> 
>     l = []
>     lock = threading.Lock()
>     ...
>     def fill_l():
>         for i in range(1000):
>             lock.acquire()
>             l.append(math.sin(i))
>             lock.release()
>     ...
>     def consume_l():
>         while 1:
>             lock.acquire()
>             if l:
>                 elt = l.pop()
>             lock.release()
>             fiddle(elt)
> 
> It's not clear from a static analysis of the code what the lock is
> protecting.  (You can't tell at compile-time that threads are even involved
> can you?)  Would or should it affect attempts to track "l.append" or
> "math.sin" in the fill_l function?

Oooooo.  Ouch.

No, on third thought, it still doesn't matter.  If someone is playing
games with module globals and not using locks, that code is broken in
current Python.  We don't need your track() and untrack() functions, but
that should be made clear in the PEP, I think.
-- 
                      --- Aahz (@pobox.com)

Hugs and backrubs -- I break Rule 6       <*>       http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista

I don't really mind a person having the last whine, but I do mind someone 
else having the last self-righteous whine.