[Numpy-discussion] copy on demand

[Konrad Hinsen]

> Wouldn't an (almost) automatic solution be to simply replace (almost) all
> instances of a[b:c] with a.view[b:c] in your legacy code? Even for unusual

That would convert all slicing operations, even those working on
strings, lists, and user-defined sequence-type objects.

> cases (like if you heavily mix arrays and lists) you could still

I do, and I don't consider it that unusual. Anyway, even if some
function gets called only with array arguments, I don't see how a code
analyzer could detect that. So it would be...

> autoconvert by inserting ``if type(foo) == ArrayType:...``, although

typechecks for every slicing or indexing operation (a[0] generates a
view as well for a multidimensional array). Guaranteed to render most
code unreadable, and of course slow down execution.

A further challenge for your code convertor:

    f(a[0], b[2:3], c[-1, 1])

That makes eight type combination cases.

> Well, AFAIK there are actually three mutable sequence types in
> python core and all have copy-slicing behavior: list, UserList and
> array:

UserList is not an independent type, it is merely a subclassable
wrapper around lists. As for the array module, I haven't seen any code
that uses it.

> I would suppose that in the grand scheme of things numarray.array is intended
> as an eventual replacement for array.array, or not?

In the interest of those who rely on the current array module, I hope not.

> much "lets make it really good (where good is what *I* say) then loads of
> people will adopt it", it was more: "Numeric has a good chance to grow
> considerably in popularity over the next years, so it will be much easier to
> fix things now than later" (for slicing behavior, now is likely to be the last
> chance).

I agree - except that I think it is already too late.

> The fact that matlab users are used to copy-on-demand and the fact that many
> people, (including you if I understand you correctly) think that copy-slicing
> semantics as such (without backward compatibility concerns) are preferable,

Yes, assuming that views are somehow available. But my preference is
not so strong that I consider it a sufficient reason to break lots of
code. View semantics is not a catastrophe. All of us continue to use
NumPy in spite of it, and I suspect none of use loses any sleep over
it. I have spent perhaps a few hours in total (over six years of using
NumPy) to track down view-related bugs, which makes it a minor problem
on my personal scale.

> I don't think matlab or similar alternatives make legally binding promises
> about backwards compatibility, or do they? It guess it is actually more

Of course not, software providers for the mass market take great care
not to promise anything. But if Matlab did anything as drastic as what
we are discussing, they would loose lots of paying customers.

> But reliability to me also includes the ability for growth -- I not only want
> my old code to work in a couple of years, I also want the tool I wrote it in
> to remain competitive and this can conflict with backwards-compatibility. I

In what way does the current slicing behaviour render your code
non-competitive?

> like the balance python strikes here so far -- the language has

Me too. But there haven't been any incompatible changes in the
documented core language, and only very few in the standard library
(the to-be-abandoned re module comes to mind - anything else?).

For a bad example, see the Python XML package(s). Lots of changes,
incompatibilities between parsers, etc. The one decision I really
regret is to have chosen an XML-based solution for documentation. Now
I spend two days at every new release of my stuff to adapt the XML
code to the fashion of the day.

It is almost ironic that I appear here as the great anti-change
advocate, since in many other occasions I have argued for improvement
over excessive compatiblity. Basically I favour motivated incompatible
changes, but under the condition that updating of existing code is
manageable. Changing the semantics of a type is about the worst I can
imagine in this respect.

Konrad.
-- 
-------------------------------------------------------------------------------
Konrad Hinsen                            | E-Mail: hinsen at cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------------