[PYTHON MATRIX-SIG] Yorick, Python and Array.py

[Chris Chase S1A]

>>>>> "JH" == James Hugunin <jjh@mama-bear.lcs.mit.edu> writes:

JH> I'm not sure that this is true.  I've found that for my matrix object (on  
JH> which I have received almost entirely good reviews from the testers) that  
JH> this sort of indexing can be added very easily to python using mapping  
JH> semantics (as Dave mentioned):

JH> ie. a[[All,Empty,All]] <--> a(,-,)

I guess I am not clear on this.  Are All and Empty special objects
defined by your matrix module?

Aside: Actually, if the additions I had in mind were made (like slice
notation) I would write the righthand side as a[:,-,:] or a[:,None,:],
avoiding a new syntactic element for an empty argument (not seen
elsewhere in the language).

If you are concerned about too many extras added to the language, I
would suggest that None be equivalent to "-", removing the requirement
for your Empty object.

JH> I do think that it would be nice to remove that extra set of brackets,  
JH> which Guido is also in favor of; however, I'm just not convinced that any  
JH> further modifications are needed.

As you have indicated, there are clever workarounds that can
accomplish the Yorick type of additions without internal language
changes.  However, I do not think that we should rule out internal
Python additions if they are beneficial but don't break, limit,
or violate the spirit/flavor of the language.

JH> a[[Star,2,3]] <--> a(*,2,3)

Okay.  This is good.  Would you use something like Dot for the Yorick
".." pseudo index?

How do you keep the definitions of Star and Dot from being hidden
inadvertently by other definitions in the users code (like an imported
star catalog module that defines a Star object)?

JH> a[[Slice(1,10,2)]] <--> a(1:10:2)

Would the following also work?

a[::2,:-4,::-1]  <--> a[[Slice(None,None,2),Slice(None,-4),Slice(None,None,-1)]]

This is product indexing on a, i.e. the indexes of the resulting
elements are taken from the Cartesian product of the index vectors.
This example takes every other on the first dimension, up to the fourth
from the last on the second dimension, and reverses the last dimension.

Of course one may want to mix scalars, slices, and matrices as indexes
in the same subscripting expression:

a[(1,2,3),5,::-1,b] <--> a[[[1,2,3],5,Slice(None,None,-1),b]]

Are you also considering supporting indexing a multi-dimensional array
in flattened form when only a single index vector is given?  E.g. for
the rank 3 a:

a[[Slice(1,None,2)]] accessing every other element of a in flattened
form.  This is useful and common in other array languages.

The danger here is that a[[[1,2,3]]] could be mistaken for a[[1,2,3]],
where the first is a one-dimensional 3 element vector index into a
flattened array and the second is a multi-dimensional index using a
scalar for each dimension.  (Dropping the extra grouping, the
difference a[(1,2,3)] and a[1,2,3] is slightly more evident).

JH> I realize that the syntax in Yorick is a little bit more consise, but I'm  
JH> just not convinced that such fundamental changes to the python core are in  
JH> fact necessary.

Necessary, perhaps not.  Worthwhile, I would definitely say
yes. (Especially if someone volunteers the work for the changes).  

At the least, I feel the following would be worthwhile:

1) drop the extra grouping inside [] for multi-dimensional subscripts.
2) support slice expressions with strides wherever a 'test'
   syntactical element is possible. This internalizes your Slice()
   functionality.  It is a natural extension of slice syntax already
   available.  But requires a special check for calling the current
   __slice__ functionality for sequences.

Mabye:
3?) '-' for pseudo index [not a big benefit but it only takes 3 lines
    of code, one in Grammar and 2 in compile.c] 

I have already started implementing these.  I think that your
workaround for the rubber indexes is just as good as what I had
started to implement (probably better in the how it fits into the
language philosophy).

Aside: It would have been useful if the Python methods for sequence
and dictionary types (e.g. __slice__) had originally been defined to
take variable length argument lists instead of defined length and type
argument lists.  Then additions for slices with strides and
multi-dimensional indexing would have been easier to add without
breaking existing code (argument checking would be up to the method.
If more arguments then desired were passed, e.g. 3 instead of 2 for a
slice, then either the extra arguments would be ignored or an
exception signaled.  Similarly for subscripting.).

Chris Chase

=================
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================