[pypy-dev] Builtin types

[Bengt Richter]

At 09:01 2003-01-24 +0100, Thomas Heller wrote:
>Bengt Richter <bokr at oz.net> writes:
>
>> I have some thoughts for decribing C structures (and anything else
>> digital) in a canonical abstract way. I'll call such a decription
>> a meta-representation, and a function that produces it from a Python
>> object is meta_repr(python_object) => meta_rep_obj. The reverse
>> operation is meta_eval ;-)
>> 
>> The basis for meta_repr is that it capture and meta-represent
>> (type, id, value) for an object in a pure abstract form.
>> The abstract basis for storing the information is a bitvector
>> object with attribute names that specify slices of the bitvector.
>> I.e., there is an effective dict like {'slicename':(lo,hi)} and
>> __getattr__ uses it so that bvec.slicename refers to bvec[lo:hi] etc.
>> This is pure and abstract data, but you can see that interpreted
>> little-endianly you can very straightforwardly lay out a C struct
>> with whatever field widths and alignments you want.
>> 
>> I also want to make a way to create subclasses of the base bitvector
>> class that can have specified size and named slices as class variable
>> info shared by instances. This is very analogous to a C struct
>> definition, and BTW also permits union by just defining overlapping
>> named slices.
>
>This all sounds very similar to what is already implemented in ctypes ;-)
>
>This is a the structure which stores to information for one field of a
>C structure or union:
>
>typedef struct {
>        PyObject_HEAD
>        int offset;
>        int size;
>        int index;                      /* Index into CDataObject's
>                                           object array */
>        PyObject *proto;                /* a type or NULL */
>        GETFUNC getfunc;                /* getter function if proto is NULL */
>        SETFUNC setfunc;                /* setter function if proto is NULL */
>} CFieldObject;
>
>'offset' is what you call 'lo', and 'offset + size' is your 'hi'

It does sound like a lot of similar ground is covered. Are your
offset and size values in bits or bytes? (I intended bits).

>attribute.  'proto' (I should probably have chosen a better name) is a
>Python object holding information about the field type (such as
>alignment requirements and storage size), and 'getfunc' and 'setfunc'
>are pointers to functions which are able to convert the data from
>Python to C and vice versa.
>
>Instances of these CFieldObjects populate the class dict of ctypes
>Structure and Union subclasses, they are created by their respective
>metaclass from the _fields_ attribute, and are used as attribute
>descriptors to expose the fields to Python.
This part sounds very close to the same, except I was going to specify
fields by parameters in keyword arg tuples, with the keywords as field
names. And I was basing getfunc and setfunc on slice operations operating
on regions of a bit list. And there are some things I haven't worked out
yet -- but you are right, there are similarities. There would have to
be ;-) But I'm kind of partial to the notion of pure abstract bit
sequences and being able to build purely abstract composites of those.
I realize you can view the C entities as abstract data elements too, but
I'd think the pure based abstractions could survive C going away, and
getting retargeted to something else perhaps a little more readily. It
is speculation at this point for me though.  I will explore a little
further see how it looks.

The thing I think would be cool is if one could write python to build
meta_repr objects in python and have Psyco compile Python code
manipulating those representations and wind up with machine code
effectively equivalent to what the C code in your ctypes module does
when given the same implicit abstract info defining fields and accessors
etc., and code using those.

The thing about the latter situation is that Psyco would still see code
accessing a foreign interface to getfunc/setfunc, whereas if it sees
Python code actually accessing the data meta-representations behind getfunc/setfunc, it has a chance to bypass function calls and generate
inline machine code instead of using your canned functions. Probably
a loss at first, but eventually it could be a gain, depending on Psyco?

Regards,
Bengt