[Python-Dev] PEP: Adding data-type objects to Python

[Paul Moore]

On 10/31/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> In order to make sense of the data-format object that I'm proposing you
> have to see the need to share information about data-format through an
> extended buffer protocol (which I will be proposing soon).  I'm not
> going to try to argue that right now because there are a lot of people
> who can do that.
>
> So, I'm going to assume that you see the need for it.  If you don't,
> then just suspend concern about that for the moment.  There are a lot of
> us who really see the need for it.

[...]

> Again, my real purpose is the extended buffer protocol.  These
> data-format type is a means to that end.  If the consensus is that
> nobody sees a greater use of the data-format type beyond the buffer
> protocol, then I will just write 1 PEP for the extended buffer protocol.

While I don't personally use NumPy, I can see where an extended buffer
protocol like you describe could be advantageous, and so I'm happy to
concede that benefit.

I can also vaguely see that a unified "block of memory description"
would be useful. My interest would be in the area of the struct module
(unpacking and packing data for dumping to byte streams - whether this
happens in place or not is not too important to this use case).
However, I cannot see how your proposal would help here in practice -
does it include the functionality of the struct module (or should it?)
If so, then I'd like to see examples of equivalent constructs. If not,
then isn't it yet another variation on the theme, adding to the
problem of multiple approaches rather than helping?

I can also see the parallels with ctypes. Here I feel a little less
sure that keeping the two approaches is wrong. I don't know why I feel
like that - maybe nothing more than familiarity with ctypes - but I
don't have the same reluctance to have both the ctypes data definition
stuff and the new datatype proposal.

Enough of the abstract. As a concrete example, suppose I have a (byte)
string in my program containing some binary data - an ID3 header, or a
TCP packet, or whatever. It doesn't really matter. Does your proposal
offer anything to me in how I might manipulate that data (assuming I'm
not using NumPy)? (I'm not insisting that it should, I'm just trying
to understand the scope of the PEP).

Paul.