Need more comments from scientific community on python-dev

[Lisandro Dalcin]

On 10/31/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> I'm recruiting more comments on python-dev regarding my two proposals
> for improving Python's native ability to share ndarray-like information.
>

I believe there are another really important reason to support
Travis's proposal.

Python should have a general, simple, powerful and standard way to
associate type metadata to binary streams. This enables efficient
share or memory, but also can help a lot to achieve inter-language
capabilities.

As an example, I would to make some comments about MPI, the "de facto"
standard for message-passing in parallel computing. When MPI
specification was being developed, there were many different,
incompatible libraries for message passing, and many of them (ie. PVM)
usually communicated binary data following a pack/unpack approach. The
MPI Forum decided to add an alternative approach, adding a class
'MPI_Datatype', and many predefined instances of this class (MPI_INT,
MPI_FLOAT, etc.) in order to represent the basic datatypes (integers,
floats, strings, complex numbers) in C, Fortran, and C++. Predefined
MPI datatypes can be used as building blocks for describing very
complicated memory layouts and derived,nested structures. This way,
any implementations of this standard can support interprocess
communication in a multi-language, muti-architecture approach. MPI
datatypes can also be "decoded" in order to get the basic, predefined
types contained in a user-defined datatype.

I think the MPI example, despite being taken from a very specific
domain with particular needs, has a strong connection with Travis's
proposal. Pack/unpack degrades performance, but if you want to
efficiently share binary data (with other languages/architecture),
then a general and extensible mechanism for attaching type metadata to
binary streams must me defined.

I would like to know your opinions about this. Travis's datatypes in
Python should be more that something to share data, they should be
able to **describe** data. And I am not sure if ctypes can be be the
way. Can Jython take advantage of ctypes??? Or in the near future
CPython/Jython will have a 'jtypes' module? I forget! Travis/Pearu,
why don't you develop a 'ftypes' for Fortran binary data, and next ask
Python core developers to include it in standard library (a joke, of
course)?

Travis, if you think my previous example about MPI can help, I can
resend this post to Python-Dev. I ask you this because I really want
to help and not to make noise.

Regars,


-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642