Discussion: new operators for numerical computation

[Huaiyu Zhu]

On Thu, 20 Jul 2000 00:58:06 GMT, Tim Hochberg <tim.hochberg at ieee.org> wrote:
>
>>    (2) Which is matrixwise or elementwise
>
>I can't see a good answer here. NumPy has been around for a long time
>and I believe it has a fairly large user base which is a strong
>argument in favor of the ".x" operators being matrixwise. However,

We may need also to consider future userbase.  I would predict (but
predictions are unreliable!) that a large portion of future userbase from
current matlab/octave users.  (See below)

My own experience and bias: NumPy does not orient toward people using arrays
as objects in linear space, but rather like a holder of a large amount of
homogeneous data.  I'd already tried to use it for a month but almost gave
up. I was looking for a python-octave pathway when Konrad Hinsen suggested
that a simple wrapper would achieve the goal.  I would imagine the majority
of people who would prefer this never actually crossed the line from matlab.

One indicator of this is that when I asked about matrixwise interfaces most
answers came from people who are perfectly comfortable with multiarray. But
when dealing with multidiemnsional arrays the infix notation is not quite
necessary nor convenient.

So my point is that the portion of NumPy users who do linear algebra instead
of multilinear algebra may not be that great.  (You can see my bias
throughout this post.  Be warned.)

>other operators within python (*,+) operate "objectwise" on sequences
>(concatenation and replication). I think it would be suprising if
>these operators were elementwise for matrices and objectwise for
>other, similar, objects (sequences). This tends to favor ".x" being
>elementwise.

This is by far the most important point, imho.  It is good for an OO
language to set the default behavior as objectwise.  Even if the additional
operators are not be implemented on other objects, the conceptual
inconsistency is still worth considering.


>>    (3) Compatibility: NumPy uses * for elementwise, MatPy uses * for
>>        matrixwise.
>
>Do you have an idea of what the relative sizes of the userbases of
>these two packages are? Honest question -- I have no idea, but I've
>been operating under the assumption that userbase of MatPy was
>relatively small.

I don't know how to gauge NumPy.  Any ideas?

For MatPy the statitics of downloading is quite small.  Just over 50 for the
latest version in ten days and several hundred total for all version in the
one month it has been in existence.  The number is largely restricted to
those who read c.l.p at the time I annouce a new release, but it has got to
Vault of Parnassus recently.  It did not work on Windows until over a week
ago.  I would say that those who use MatPy even at this alpha stage are
those who really need this interface, and represent a tip of iceberg of the
potential users.

>>    (4) Similarity with Matlab (but not really compatibility). 
>
>I think it's a mistake to look just at Matlab here. If were going to
>look to matrix languages for inspriation we should look to
>several. Unfortunately, I'm no help here.

Well, I just did a little experiment and found that it is surprisingly easy
to translate Matlab to python.  The other way is of course almost
impossible. After all, Matlab is such a simple language compared with
Python.  (I wonder why people wwould compare it to Perl. <0.8wink>)

So I would consider the similarity quite important.  If a matlab program
consists mostly of formulas the transition is almost effortness.  Once we
have these operators I do not see any great hurdle for mass immigration.

Any examples of how other numerical languages treat this issue?

There is anothe issue.  If we decide on * for elementwise operations it is
best to use @* for matrix usage.  Having both * and .* and let them mean
exactly the oposite of matlab is really too confusing for the targeted
users, several have pointed out already.

>
>> 3. Their behavior on other objects:
>> 
>Note that if ".x" ends up being matrixwise, things will become
>horribly inconsistent if it is later decided to extend these operators
>to other sequences. This IMO is probably the strongest argument for
>making ".x" elementwise.

Agreed. I think these operators will not have any default meaning on objects
other than numbers. However, is it allowed for non-numerical classes to use
them through __atmul__?  After all, there is no syntactic distinction
between these classes.  So what if a subclass of UserList want to have both
objectwise and elementwise add?

>> 5. Other bits and pieces:
>> 
>>    - How to deal with left division?
>> 
>>     (1) Use sol(a,b)
>>     (2) use /@ or /.
>>     (3) use %
>
>'%' is already a valid elementwise operator and to have "%" and ".%"
>mean completely different things is not good, so (3) is out. 
>
>I think (2) only makes sense if the ".x" operators are matrixwise. It
>seems just too weird to have "/" be right division "/." be left
>division and "./" be elementwise. It's much more sensible to have "/"
>be elementwise, "/." be left division, and "./" be right division for
>instance.
>
>If possible I would like to avoid (1). 'sol' would almost certainly be
>in the a seperate module and this is potentially painful.

This seems to be the best reason to use @/ and /@.  But sol does not live in
a different module in MatPy.  Using % or | involves changing the meaning of
currntly used operators.  Even though the actual usage might be very small,
that's still a change.  If we are not going to grab ^ for pow, we probably
shouldn't consider grabing % and | for sol.

>>    - Do we want a copy operator := which is different from =?  It allows
>>      efficient code.  The following code does not change A
>>          B := A
>>          B += C
>>      This definitely will be usable for other classes. Call it __copy__?
>
>This seems to be a general request and not particularly numeric
>specific. And I don't think it adds much in the way of
>functionality. I would drop this. 

Agreed.  As someone mentioned, it's no difference from B=A.copy()

>
>>    - Since augmented assignment almost certainly will get in the core, we
>>      need to consider combining them.  A few possible opinions
>> 
>>      (1) Just use prefix . or @ to mean e or m, use suffix = to mean in
>>          place assignment, and all are well defined.
>> 
>>      (2) a .**= 3 looks too long.  Don't combine them.
>> 
>>      (3) allow .+=, etc but ban .**=.
>
>Here I'd go with (1), consistency over beauty.

Agreed.  I was only worrying that someone might charge its ugly.  But the
fault was really in using ** for pow in the first place.

>I'm not yet convinced as yet that all of these operators are needed;
>the +, - and .+, .- operators are very similar in behaviour. The main
>reason they are present seems to be orthogonality. I'm sure this has
>been brought up before, but does MatLab have both broadcasting and non
>broadcasting versions of its operators? Is it worth considering adding
>only the matrix operators .*, ./, /., and(?) .** and letting NumPy/
>MatPy choose whether they wish to broadcast or not? It seems that the
>difference between .* and * is different in character than the
>proposed difference between .+ and +; in one it's between elementwise
>and matrixwise operations, and in the second it's between broadcasting
>and nonbroadcasting operations. Yes, you can think of the second in
>terms of the first, but it still feels a bit forced to me.

I have a different opinion.  The main reason they are needed is because they
are conceptually consistant.  The definition of + - * / adhere strictly in
accordance with linear algebra, so they will balk at the wrong dimensions.
But .+ .- .* ./ are elementwise, for which the current NumPy rules make
perfect sense to me: make elementwise operations, extending missing
dimensions whenever necessary.  Unless we change the current NumPy defaults
it is consistent to have them all.  And why would we change?  I find

a .+ b

so much more reasable (and probably faster) than

a*ones(size(b)) + ones(size(a))*b


Just for some ideas, I have a set of operators that would satisfy most of
the above need:

NumPy matrix  NumPy element  MatPy matrix  Matpy element
  @+              +              +            .+
  @-              -              -            .-
  @*              *              *            .*
  @/              /              /            ./
  /@             none           what          none
  @**             **             **           .**

This provides the smoothest migration path for all the users. Note that the
cause of NumPy/MatPy inconsistency was the lack of binary operators.  As
this already happened, it is difficult to integrate them until both got two
versions of operators.  Problems:

- This introduces 6+5 new operators.  Ironically the previous lack of
  operators actually results in more operators.

- Having standard + - * / mean different things in NumPy and MatPy.  I
  personally find this no problem.  In certain portions of my programs I need
  elementwise operations exclusively, and I found it convenient to cast
  everything into NumPy without trouble.

- There's still no left division for MatPy.

Huaiyu