[Python-Dev] Instruction count frequency server

[Skip Montanaro]

Last night during the midst of a lengthy reply to a post by Alex Martelli,
it occurred to me that it might be useful for people fiddling around with
the Python VM to have access to a large body of instruction count
information.  Marc-Andre Lemburg posted a note a year or so ago that
summarized dynamic instruction count frequencies gathered over 100 million
PyVM instruction cycles.  While looking at his table I noticed that he
reported a very high frequency of LOAD_NAME instructions.  I took a quick
peek at static instruction counts in the core library and found that
LOAD_NAME almost never appears in that particular chunk of Python bytecode.
So, although MAL collected data over a large number of instruction cycles,
it's not obvious that his programs were "typical".

This morning I did something about that.  I wrote a little XML-RPC server
that allows people to post or retrieve instruction count frequencies.  It's
at manatee.mojam.com, port 7304.  To get a quick synopsis of the methods the
server implements, try this:

    dxp = xmlrpclib.Server("http://manatee.mojam.com:7304")
    print dxp.synopsis()

More detail can be had with

    print dxp.usage()

The usage response contains details about the individual methods, run-length
encode/decode functions, and hints on how to send sys.getdxp() output to the
server automatically whenever a program exits normally.

Server response isn't terribly speedy, mostly because when sending or
receiving instruction count data a fair number of bytes stream one way or
the other, even though the data is run-length encoded and the server will
gzip encode output when it can.  Even when queried from the same machine as
the server it takes about 5 seconds to return non-trivial output.  Oh well,
XML-RPC isn't perfect for everything.  Hopefully the load won't be too
heavy.

Feedback is welcome.

Skip