Improved struct module

[Tim Peters]

[comparing Robin Boerdijk's xstruct C extension module
     http://www.sis.nl/python/xstruct/xstruct.html
 with Just van Rossum's sstruct Python module
     http://starship.python.net/crew/just/code/sstruct.py
]

[Robin Boerdijk]
> I have done a comparison on how fast sstruct is compared to xstruct
> using the Xsdp example that comes with xstruct. The test scripts are
> at the end of this post. They both do 10000 iterations of creating
> and packing an XsdpMessage. On my computer, the sstruct script takes
> 2.4 seconds, the xstruct script takes 0.33 seconds.

The ratio on my machine is about the same as you got.  You should qualify
this with an indication of what it's measuring, though!  The inner loops
are:

[for xstruct]
for i in xrange(0, 10000):
    msg = XsdpMessage()
    msg.correl_id = 0x01020304
    msg.message_type = 1
    msg.data = "Hello, World !"

[for sstruct]
for i in xrange(0, 10000):
    msg = copy.copy(default_msg)
    msg['correl_id'] = 0x01020304
    msg['message_type'] = 1
    msg['data'] = "Hello, World !"
    sstruct.pack(XsdpMessageFormat, msg)

That may (or may not -- beats me) be typical of apps that access each struct
member once and are then done with the struct.  In an app that accesses (or
modifies) struct members multiple times, Just's approach is much faster;
e.g., xstruct takes 3+ times longer (on my machine) than sstruct if the
inner loops are changed to:

[for xstruct]
for i in xrange(0, 10000):
    msg = XsdpMessage()
    for j in xrange(100):
        msg.correl_id = 0x01020304
        msg.message_type = 1
        msg.data = "Hello, World !"

[for sstruct]
for i in xrange(0, 10000):
    msg = copy.copy(default_msg)
    for j in xrange(100):
        msg['correl_id'] = 0x01020304
        msg['message_type'] = 1
        msg['data'] = "Hello, World !"
    sstruct.pack(XsdpMessageFormat, msg)

That may (or may not -- beats me <wink>) be typical of apps that access each
struct member many times.

My guess (with nothing to back it up) is that xstruct would be faster in
more apps than sstruct.  People making their own guesses must realize there
are tradeoffs here, though:  almost nothing is a pure win, and this is no
exception -- which one runs faster depends on what you're doing.

[about the docs]
> ...
> So far, I have only documented up to the point that I thought that people
> who need to interact intimately with C structs from Python (and have been
> struggling with struct.pack and struct.unpack to do that) would get
> enthousiastic about this new module.

Curiously enough, I hardly ever muck with C structs.  I too would like to
hear from people who do!  I only jumped in here because I thought you had
written a very interesting module, and your announcement was followed by
days of silence.  You may not like the approach I've taken here, but it has
gotten the module some much-needed publicity <wink>.

[about the buffer interface]
>> It's not ready for prime time (e.g., it's easy to crash the interpreter
>> with the little that's already there, staying within pure Python).

> Can you give me an example ?

Yes, but I won't.  You can create your own by thinking about the
implementation:  a buffer object contains a pointer to a base object and a
separate pointer to a chunk of untyped memory.  If the base object happens
to stop using that memory, there's no way for the buffer object to know
that.  And if that memory happens to get reused by some other object for
something else (or even unmapped by the OS), anything at all can happen --
it's a hole so big even Perl could fit into it <wink>.  Your new objects are
safe because they hold on to their buffer memory until they die (and so long
as a buffer object refers to them, they can't die).  Some object types
aren't so nicely behaved.

any-more-hints-and-even-guido-could-create-an-example<wink>-ly y'rs  - tim