[PYTHON MATRIX-SIG] More on Less Copying

[Konrad Hinsen]

> (a) One of my needs is to read large arrays of arbitrary rank from
> binary files. The size of the array is known in advance but need not
> be equal to the remaining size of the file.

You are adressing an important topic that has largely been neglected
so far: file I/O, both in binary and in text form, suitable for
interchanging data with non-Python programs (note that pickle works
fine as long as the format doesn't matter).

> Two ways to do this might be:
> 
>     a = Numeric.new(shape, typecode)    # create uninitialized new array
>     a.read(file)                        # fill array; copy once
> 
> and:
> 
>     a = Numeric.read(file, shape, typecode)

The first version is more general, since it allows to overwrite an
existing array by reading from a file. What I don't like about this
approach is that there is no control over the file format. You must
have a file that uses the same memory layout as NumPy arrays, which
in principle is not documented. In practice, you would know the file
format and want NumPy to do whatever necessary, including perhaps
number format conversion. You might also want to be able to read/write
a subarray.

> (b) I've heard mutterings about in-place operations, which avoid a
> copy, but they aren't implemented yet. Are these important to others
> here? I'm going to assume that they are.
> 
> No operators are currently defined in the base language for in-place
> operations. That means we have to explicity use methods. I think I
> would be willing to live with:
> 
>     a.add_inplace(b)

Try
  Numeric.add(a, b, a)
This will add a and b and put the result into a without creating a
copy.

> (c) However, there is a non-zero chance that at some point in-place
> operations on scalars are going to appear in the base language. It
> would be nice to be in a position to be upwardly compatible with that
> change.

I doubt this will happen. The assumption that numbers are immutable is
pretty basic in Python, and a lot of code would break if it were
violated. At best, operators like += would be added for mutable types
(e.g. lists) and classes. And even that doesn't look like a
high-priority item for future versions.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                          | E-Mail: hinsen@ibs.ibs.fr
Laboratoire de Dynamique Moleculaire   | Tel.: +33-4.76.88.99.28
Institut de Biologie Structurale       | Fax:  +33-4.76.88.54.94
41, av. des Martyrs                    | Deutsch/Esperanto/English/
38027 Grenoble Cedex 1, France         | Nederlands/Francais
-------------------------------------------------------------------------------

_______________
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
_______________