[Python-Dev] -z, -i and -m, maybe bug in runpy?

[Nick Coghlan]

Nick Coghlan wrote:
> Phillip J. Eby wrote:
>> While trying to get my -z replacement patch to work, I stumbled across 
>> a bug in the -m implementation (and in runpy).  It seems that when you 
>> run the code of a -m module, it is *not* run in the __main__ module 
>> namespace!
>>
>> So even though __name__=='__main__', globals() is not 
>> sys.modules['__main__'].__dict__.  This seems wrong to me.  Does 
>> anybody know why runpy doesn't actually run the code in the target 
>> module?
> 
> After implementing the runpy explicit relative import tests over the 
> last couple of days, it actually occurred to me earlier today that I 
> didn't have a test for this scenario. When I thought of the test, I was 
> also pretty sure it would fail - it appears I was right :)

OK, I've now had a closer look, and the problem isn't what I initially 
thought when I read your message (the test which I expected to fail 
actually passed without changing the current implementation). It turns 
out that while the module is actually executing it does the right thing 
- the problem only arises when the run_module function attempts to clean 
up after itself by reverting some of the changes it makes to the sys module.

The specific problem is this sentence from the run_module docs:
   "Both sys.argv[0] and sys.modules[__name__] are restored to their 
original values before the function returns."

It looks like those semantics are a mistake - the changes to the sys 
module should persist after the function terminates, leaving it to the 
calling code to decide whether or not it wants to restore the original 
state.

>> One consequence of this is that the -i option is much less useful when 
>> you use -m, because the script's globals have disappeared before you 
>> get to the interpreter prompt.

See above - the problem is that the function is cleaning up after itself 
and deleting things that may still be of interest when -i is also specified.

>> At this point, I've successfully gotten a -z replacement patch, except 
>> that it inherits this apparent bug from -m, which for a while led me 
>> to believe my patch was broken (when in fact it works fine, apart from 
>> inheriting the -m behavior).
>>
>> Does anybody know if this behavior is intended, and if so, why?  And 
>> what are the consequences of changing/fixing it?

It was intended enough to be documented that way, but I don't recall 
putting any significant thought into that aspect of the implementation, 
and nor do I remember anyone else questioning it.

The fact that it completely breaks the -i switch seems more than enough 
reason to consider it a bug, though.

I've changed the behaviour in r56520 to simply leave the alterations to 
sys in place when the function terminates. While this is a definite 
change to the interface (and hence not a candidate for direct 
backporting), I think the difference is small enough for the 2.5 to 2.6 
transition.

If enough people prefer, I can switch the code to an approach which 
fixes -m while leaving the semantics of runpy.run_module alone. This 
would involve renaming the version of run_module I just checked into SVN 
have -m invoke that version directly. run_module would be changed to 
wrap the function used by -m in the necessary code to restore the sys 
module to something more closely resembling its original state. That 
would also be the approach to take if we decided we wanted to backport 
this fix to the 2.5 maintenance branch.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org