[Python-Dev] PEP: Adding data-type objects to Python

["Martin v. Löwis"]

Travis Oliphant schrieb:
> Function pointers are "supported" with the void data-type and could be 
> more specifically supported if it were important.   People typically 
> don't use the buffer protocol to send function-pointers around in a way 
> that the void description wouldn't be enough.

As I said before, I can't tell whether it's important, as I still don't
know what the purpose of this PEP is. If it is to support a unification
of memory layout specifications, and if that unifications is also to
include ctypes, then yes, it is important. If it is to describe array
elements in NumArray arrays, then it might not be important.

For the usage of ctypes, the PEP void type is insufficient to describe
function pointers: you also need a specification of the signature of
the function pointer (parameter types and return type), or else you
can't use the function pointer (i.e. you can't call the function).

> Pointers are also "supported" with the void data-type.  If pointers to 
> other data-types were an important feature to support, then this could 
> be added in many ways (a flag on the data-type object for example is how 
> this is done is NumPy).

For ctypes, (I think) you need "true" pointers to other layouts, or
else you couldn't set up the memory correctly.

I don't understand how this could work with some extended buffer
protocol, though: would a buffer still have to be a contiguous piece
of memory? If you have structures with pointers in them, they
rarely point to contiguous memory.

> Unions are actually supported (just define two fields with the same 
> offset).

Ah, ok. What's the string syntax for it?

> I don't know what you mean by "packed structs" (unless you are talking 
> about alignment issues in which case there is support for it).

Yes, this is indeed about alignment; I missed it. What's the string
syntax for it?

> I'm not sure I understand what you mean by "incomplete / recursive" 
> types unless you are referring to something like a node where an element 
> of the structure is a pointer to another structure of the same kind 
> (like used in linked-lists or trees).  If that is the case, then it's 
> easily supported once support for pointers is added.

That's what I mean, yes. I'm not sure how it can easily be added,
though. Suppose you want to describe

struct item{
  int key;
  char* value;
  struct item *next;
};

How would you do that? Something like

item = datatype([('key', 'i4'), ('value', 'S*'), ('next',
'what_to_put_here*')]

can't work: item hasn't been assigned, yet, so you can't
use it as the field type.

> I also don't know what you mean by "open-ended arrays."  The data-format 
> is meant to describe a fixed-size chunk of data.

I see. In C (and thus in ctypes), you sometimes have what C99 calls
"flexible array member":

struct PyString{
  Py_ssize_t ob_refcnt;
  PyObject *ob_type;
  Py_ssize_t ob_len;
  char ob_sval[];
};

where the ob_sval field can extend arbitrarily, as it is the last
member of the struct. Of course, this will give you dynamically-sized
objects (objects in C cannot really be "variable-sized", since the
size of a memory block has to be defined at allocation time, and
can't really change afterwards).

> String syntax is not needed to support all of these things.

Ok. That's confusing in the PEP: it's not clear whether all these
forms are meant to be equivalent, and, if not, which one is the most
generic one, and what aspects are missing in what forms. Also,
if you have a datatype which cannot be expressed in the string
syntax, what is its "str" attribute?

Regards,
Martin