[Numpy-discussion] proposing a "beware of [as]matrix()" warning

[Robert Kern]

On Wed, Apr 28, 2010 at 15:50, Travis Oliphant <oliphant at enthought.com> wrote:
>
> On Apr 28, 2010, at 11:19 AM, Robert Kern wrote:
>
>> On Wed, Apr 28, 2010 at 11:05, Travis Oliphant
>> <oliphant at enthought.com> wrote:
>>>
>>> On Apr 26, 2010, at 7:19 PM, David Warde-Farley wrote:
>>>
>>>> Trying to debug code written by an undergrad working for a colleague
>>>> of
>>>> mine who ported code over from MATLAB, I am seeing an ugly melange
>>>> of
>>>> matrix objects and ndarrays that are interacting poorly with each
>>>> other
>>>> and various functions in SciPy/other libraries. In particular there
>>>> was
>>>> a custom minimizer function that contained a line "a * b", that was
>>>> receiving an Nx1 "matrix" and a N-length array and computing an
>>>> outer
>>>> product. Hence the unexpected 6 GB of memory usage and weird
>>>> results...
>>>
>>> Overloading '*' and '**' while convenient does have consequences.
>>> It
>>> would be nice if we could have a few more infix operators in Python
>>> to
>>> allow separation of  element-by-element calculations and "dot-
>>> product"
>>> calculations.
>>>
>>> A proposal was made to allow "calling a NumPy array" to infer dot
>>> product:
>>>
>>> a(b) is equivalent to dot(a,b)
>>>
>>> a(b)(c) would be equivalent to dot(dot(a,b),c)
>>>
>>> This seems rather reasonable.
>>>
>>>
>>> While I don't have any spare cycles to push it forward and we are
>>> already far along on the NumPy to 3.0, I had wondered if we couldn't
>>> use the leverage of Python core developers wanting NumPy to be ported
>>> to Python 3 to actually add a few more infix operators to the
>>> language.
>>>
>>> One of the problems of moving to Python 3.0 for many people is that
>>> there are not  "new features" to outweigh the hassle of moving.
>>> Having a few more infix operators would be a huge incentive to the
>>> NumPy community to move to Python 3.
>>>
>>> Anybody willing to lead the charge with the Python developers?
>>
>> There is currently a moratorium on language changes. This will have
>> to wait.
>
> Exceptions can always be made for the right reasons.    I don't think
> this particular question has received sufficient audience with Python
> core developers.

It received plenty of audience on python-dev in 2008. But no one from
our community cared enough to actually implement it.

  http://fperez.org/py4science/numpy-pep225/numpy-pep225.html

> The reason they want the moratorium is for
> stability, but they also want Python 3k to be adopted.

This is not something that will justify an exception. Things like "oh
crap, this old feature has a lurking flaw that we've never noticed
before and needs a language change to fix" are possible exceptions to
the moratorium, not something like this. PEP 3003 quite clearly lays
out the possible exceptions:

"""
Case-by-Case Exemptions

  New methods on built-ins

    The case for adding a method to a built-in object can be made.

  Incorrect language semantics

    If the language semantics turn out to be ambiguous or improperly
implemented based on the intention of the original design then the
semantics may change.

  Language semantics that are difficult to implement

    Because other VMs have not begun implementing Python 3.x semantics
there is a possibility that certain semantics are too difficult to
replicate. In those cases they can be changed to ease adoption of
Python 3.x by the other VMs.
"""

This feature falls into none of these categories. It does fall into this one:

"""
Cannot Change
  ...
  Language syntax

    The grammar file essentially becomes immutable apart from ambiguity fixes.
"""

Guido is taking a hard line on this.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco