struct: type registration?

[John Machin]

On 1/06/2006 10:50 AM, Giovanni Bajo wrote:
> Hello,
> 
> given the ongoing work on struct (which I thought was a dead module), I was
> wondering if it would be possible to add an API to register custom parsing
> codes for struct. Whenever I use it for non-trivial tasks, I always happen to
> write small wrapper functions to adjust the values returned by struct.
> 
> An example API would be the following:
> 
> ============================================
> def mystring_len():
>     return 20
> 
> def mystring_pack(s):
>     if len(s) > 20:
>         raise ValueError, "a mystring can be at max 20 chars"
>     s = (s + "\0"*20)[:20]

Have you considered s.ljust(20, "\0") ?

>     s = struct.pack("20s", s)
>     return s

I am an idiot, so please be gentle with me: I don't understand why you 
are using struct.pack at all:

|>>> import struct
|>>> x = ("abcde" + "\0" * 20)[:20]
|>>> x
'abcde\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
|>>> len(x)
20
|>>> y = struct.pack("20s", x)
|>>> y == x
True
|>>>

Looks like a big fat no-op to me; you've done all the heavy lifting 
yourself.

> 
> def mystring_unpack(s):
>     assert len(s) == 20
>     s = struct.unpack("20s", s)[0]

Errrm, g'day, it's that pesky idiot again:

|>>> z = struct.unpack("20s", y)[0]
|>>> z
'abcde\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
|>>> z == y == x
True

>     idx = s.find("\0")
>     if idx >= 0:
>         s = s[:idx]
>     return s

Have you considered this:

|>>> z.rstrip("\0")
'abcde'
|>>> ("\0" * 20).rstrip("\0")
''
|>>> ("x" * 20).rstrip("\0")
'xxxxxxxxxxxxxxxxxxxx'

> 
> struct.register("S", mystring_pack, mystring_unpack, mystring_len)
> 
> # then later
> foo = struct.unpack("iilS", data)
> ============================================
> 
> This is only an example, any similar API which might fit better struct
> internals would do as well.
> 
> As shown, the custom packer/unpacker can call the original pack/unpack as a
> basis for their work. I guess an issue with this could be the endianess
> problem: it would make sense if, when called recursively, struct.pack/unpack
> used by the default the endianess specified by the external format string.