Need more comments from scientific community on python-dev

[Travis Oliphant]

Sasha wrote:

>On 10/31/06, Travis Oliphant <oliphant at ee.byu.edu> wrote:
>
>  
>
>>Please read my posts about the Python type-object verses normal Python
>>Object situation.  That really is the crux of the matter.
>>
>>    
>>
>I read the whole python-dev thread before replying.  I may be a little
>biased because I never liked somewhat cryptic letter codes in Numeric
>and the change of codes from Numeric to numpy did not contribute to my
>sympathy particularly when the convert script changed all unrelated
>instances of 'b' in my code to something else.  
>

The letter codes are not data-type objects.  The letter codes are only 
there for historical reasons (and they are in the struct and array 
modules too so blame Python...)

>I am also not a big
>fan of record arrays.  I believe that numeric data should be stored in
>"inverted tables," where columns of homogeneous data are stored
>contiguously. 
>
Sure.  I understand this.  This was my argument too when Numarray was 
first proposed.  But, how do you memory-map a record where the data is 
actually stored differently?  Numeric's answer is "you don't" but that 
is un-acceptable.  This is why record-arrays were created and the whole 
pandora's box of data-types was opened.

>  
>
>>Ctypes uses a Python type object for every data-format.
>>NumPy uses an instance of a data-type object for every data-format.
>>
>>    
>>
>Yes, but AFAIK this is a recent innovation.  Numarray used type
>objects and Numeric simply used letter codes.
>
>  
>
Recent if 1 year is recent, and only if you worry precisely about when 
they were Python objects. Numeric always used a PyArray_Descr * 
structure (which was a PyObject_HEAD away from being a Python object) to 
describe data.   The letter codes were just simple ways to "represent" 
those underlying structures (which could have been exposed to Python as 
"data-types" from the very beginning.

>>What advantage do we gain by making every instance of a data-type object
>>*also* a Python type object?
>>    
>>
>
>I think the main advantage is that you can have instances:
>  
>
>>>>c_int(42)
>>>>        
>>>>
>c_int(42)
>
>Of course, numpy has scalars for that, but ctypes also has
>fixed-length arrays, that are somewhat different from ndarrays:
>  
>

Sure, but you don't "need" an instance for every memory-layout 
description.   If you want one, then great, ctypes gives it to you.  But 
"requiring" data-type to be encoded in a type object is over-kill.


>> We get a lot of head-ache.  Have you seen
>>what ctypes had to do?  It had to define a new Dictionary object so it
>>could attach it to the tp_dict parameter because you can't just inherit
>>from the PyTypeObject and add the fields you want to the structure.
>>This is my argument.
>>
>>    
>>
>
>But, isn't this someone else's head-ache?  Someone has already gone
>through all these contortions, why not reuse the effort?  
>

No, it's not that simple.  We have a headache whenever we want to do 
something like I just did and separate out the concepts of what makes a 
Python Object a Python object.  Now, we don't just modify a simple 
C-structure (PyArray_Descr *), we have to modify a "meta-type" or a 
altered dictionary and get that change put in to ctypes.

Inheriting from Python type objects is harder.   People who have 
apparently never tried seem to think it's not but it is. 

>Others on
>python-dev described some specific needs of ctypes that your datatype
>object does not address.  Your point seems to be that numpy does not
>share these needs and could use a much simpler approach.
>  
>
No, that's not the point.  The data-type object could easily be extended 
to accomodate those needs.  The point is using data-types as instances 
of a regular Python object or data-types as instances of a "type" object.

>>I for one am not going to put any effort in that direction.  People are
>>free to do it, if they want, of course.  But, it's no small change.
>>    
>>
>
>Can you present some use cases that illustrate the advantage of your
>datatype approach best?
>  
>
>I will try to implement them with ctypes.
>  
>
This is missing the point.  I have no doubt ctypes "could" be used on 
the Python side to describe a data-type.  But, we need the PyArray_Descr 
* structure in NumPy.  What are you going to replace the PyArray_Descr * 
structure with?  That is the point. 

>> I would, however, put effort into "undertstanding ctypes objects" as
>>data-type objects.
>>    
>>
>
>Yes, this is the key.  I think we should better understand ctypes
>limitations before proposing an alternative.  
>
Please understand what I meant.  I meant putting effort into getting 
PyArray_DescrConverter to allow ctypes inputs and convert to the 
appropriate PyArray_Descr * structure.   I already understand ctypes 
objects.  I want the dtype() command to also understand them.

-Travis




-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642