[pypy-dev] Python Execution Contexts

[anatoly techtonik]

On Wed, Mar 20, 2013 at 8:21 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>
>> On Wed, Mar 20, 2013 at 7:36 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com
>> > wrote:
>>
>>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>>
>>>> On Wed, Mar 20, 2013 at 5:28 PM, Amaury Forgeot d'Arc <
>>>> amauryfa at gmail.com> wrote:
>>>>
>>>>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I've created a module to dump function trace during execution of
>>>>>> Python script. You can see session example in attachment. The format is of
>>>>>> PHP Xdebug tool [2] just because I had some scripts from the past to
>>>>>> analyze it.
>>>>>>
>>>>>> The module uses sys.settrace() to analyse frames in 'call' events
>>>>>> with callback(frame, event, arg).
>>>>>>
>>>>>> Recently I've got an anonymous report that some frame misses filename
>>>>>> information when run under IDE:
>>>>>>
>>>>>> https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes
>>>>>>
>>>>>> There is unlikely to be any more feedback from the user to make me
>>>>>> understand and reproduce the behavior, nor there is a sufficient
>>>>>> documentation in sys.trace description [1]. So I am on my own. But I can
>>>>>> not read C code the CPython is written in, so I ask here.
>>>>>>
>>>>>> What are possible execution contexts for Python?
>>>>>> How each execution context affects data structure that is passed to
>>>>>> trace function (frame and arg)?
>>>>>>
>>>>>
>>>>>  It's simpler than that: when running from an interactive session,
>>>>> f_back is empty...
>>>>>
>>>>> >>> def trace(frame, event, arg):
>>>>> ...     funcname = frame.f_code.co_name
>>>>> ...     if funcname == '<module>':
>>>>> ...         print frame.f_back.f_code.co_filename
>>>>> ...
>>>>> >>> import sys
>>>>> >>> sys.settrace(trace)
>>>>> >>> 1
>>>>> Traceback (most recent call last):
>>>>>   File "<stdin>", line 1, in <module>
>>>>>   File "<stdin>", line 4, in trace
>>>>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>>>>
>>>>
>>>> Thanks a lot. =)
>>>>
>>>> User said he was using module from the IDE and at first I thought it
>>>> wasn't possible to use module without previous stack frame (f_back), but I
>>>> was wrong:
>>>>
>>>> > py -2 -s -S
>>>> Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit
>>>> (Intel)] on win32
>>>> >>> import xtrace
>>>> >>> xtrace.start()
>>>> TRACE START [2013-03-20 18:49:27]
>>>> >>> z = 0
>>>>   -> decode() C:\Python27\lib\encodings\cp437.py:14
>>>> Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>>   File "xtrace.py", line 66, in function_trace_xdebug
>>>>     filename = self._strip_cwd(frame.f_back.f_code.co_filename)
>>>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>>>
>>>>
>>>> On the way I've discovered that my Python is hacked. Without -s -S key
>>>> it executes modules from local temp directory.
>>>> ...
>>>>           -> ensure_unicode()
>>>> c:\users\user\appdata\local\temp\easy_install-k8gvbp\pyreadline-1.7.1-py2.7-win32.egg.tmp\pyre
>>>> dline\unicode_helper.py:20
>>>> ...
>>>>
>>>>
>>>> So there are absolutely no differences in running code in these
>>>> execution contexts?
>>>> - code runs from a file
>>>> - code runs from a interactive console
>>>>
>>>
>>> Of course there is a difference; the code below works run from a file,
>>> but not when pasted into the interactive console:
>>>
>>> def trace(frame, event, arg):
>>>     funcname = frame.f_code.co_name
>>>     print frame.f_code, repr(frame.f_code.co_code), event, arg
>>>     if funcname == '<module>':
>>>         print "    module", repr(frame.f_back.f_code.co_code)
>>>
>>> import sys
>>> sys.settrace(trace)
>>> exec '1 + 1'
>>>
>>
>>  For console it pretty clear why no previous frame exists - stack is
>> resolved at >>> prompt. But what is this top frame that appears inside file
>> context? The following code gives:
>> <module> 7
>> <module> 1
>>
>> def trace(frame, event, arg):
>>     print frame.f_back.f_code.co_name, frame.f_back.f_lineno
>>     print frame.f_code.co_name, frame.f_lineno
>>
>> import sys
>> sys.settrace(trace)
>> exec '1 + 1'
>>
>> How to identify the frames here? How come that 7th line of module is
>> executed before 1 st? I expected to have module name available somewhere
>> for <module> object. All right, this kind of tricky - '1 +1' a separate
>> module, right? It is anonymous and doesn't have a name. Still is it
>> possible to identify it somehow? For printing in function traces.
>>
>
> An exception traceback gives you the answer; with the script foo.py below:
>
> def f():
>   exec 'x + 1'
> f()
>
> I get:
>
> Traceback (most recent call last):
>   File "/tmp/foo.py", line 4, in <module>
>     f()
>   File "/tmp/foo.py", line 2, in f
>     exec 'x + 1'
>   File "<string>", line 1, in <module>
> NameError: name 'x' is not defined
>
> Do you see all the information you need here?
> If yes, it's possible to extract it from the current frames.
>


Not really. It got too far from original user story about execution
contexts, so I
need to choose a different entry point to continue discussion. What I need
is the
graph of state machine of Python execution. I want to inspect possible
starting
points, from which select which are visible to the user in which cases.

Still, thanks to the pointer. I fixed the .f_back exception in version 0.3
of xtrace,
and added a separate function - beep() that allows extending xtrace with
your
own stuff for inspecting execution process.

This module opened a Pandora box of Python internals. Version 0.4 still
fails to
trace files specified on command line, and I am lost in internal details of
execfile
+ locals()/globals()/namespacing/scoping. Python tracker doesn't help here.
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130325/4034f373/attachment-0001.html>