PEP idea: Instrumented Python

[Christopher Nelson]

I was looking at adding dtrace-like dynamic tracing to Python.  Note that
this isn't dtrace itself.  The basic rationale:

1. A lot of enterprise-level software is written in Python.  It is difficult
to impossible to reproduce the customer environment in a test lab.
Sometimes applications hang mysteriously, and we are unable to duplicate the
problems in our environment.
2. dtrace is a good design.  It has little to no overhead for collection
points that are not collecting data.  Emulating that behavior allows you to
test what you fly and fly what you test.
3. Having the ability to only get trace data you want (or need) instead of
trace data for every instruction executed is very useful.  It also reduces
bandwidth requirements when archiving the traces, or shipping them out over
a network connection.
4. The ability to dynamically expand and contract the tracing scope during a
trace is vital to getting the right data.

The general plan would be to provide collectors in all of the public python
API functions.  The filtering code would also be in the local system, for
speed and bandwidth-conserving purposes.  The filters should generally be
native code.  The data should be writeable to a file or to a network
socket.  In addition, there should be a network socket listening for
commands from a tracing client.  The client would be able to upload
additional filter scripts and start or stop tracing transparently to the
running source.

For security there should be an authentication mechanism that allows the
application to decide whether or not a connection to the trace facility
should be allowed, and a way to disable the trace listening port on
startup.  (Or perhaps it should be disabled on startup by default and
enabled if so desired.)

Finally, there should, of course, be Pythonic access to the trace system.

The essential differences between the current tracing system and the
proposal involve the selectable granularity, the transparent remote access,
and automatic triggering of collection points.

Comments? Please make sure to reply to all, I am not subscribed to the list.

-={C}=-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20071012/d41d9001/attachment.html>