[SciPy-dev] Nested arrays

[Fernando Perez]

Travis Oliphant wrote:
> Fernando Perez wrote:

>>But having scalar indexing return an opaque proxy breaks almost completely 
>>this kind of usage, because you can't get back, in a straightforward manner, 
>>the things you put in the container to begin with.  This should illustrate my 
>>point:
>> 
>>
> 
> 
> Frankly, I'm willing to get rid of them.  I suggested that a couple of 
> months ago, but a few people said that having the attributes even on 
> object scalars was going to be useful, so I capitulated and just tried 
> to improve the proxying...
> 
> I suppose a more thorough discussion is warranted.  I think with some 
> effort one could figure out how to make it always appear that the object 
> array-scalar type is the actual underlying object but now with the 
> attributes of arrays.
> 
> Repeat your example with a new-style class, to see what you get however.

Well, let me put a mean twist on this:

In [10]: class foo(object):
    ....:     flags = 'they come in all kinds of pretty colors'
    ....:     def hi(self):
    ....:         return 'hi'
    ....:
    ....:

In [11]: x = empty(2, object)

In [12]: x[0] = foo()

In [13]: x[0].hi()
Out[13]: 'hi'

In [14]: x[0].shape
Out[14]: ()

In [15]: x[0].flags
Out[15]: 'they come in all kinds of pretty colors'

Again, I think there's too strong of a clash between the abstraction of array 
scalar proxies and whatever object is being boxed in.  For the regular 
datatypes I think this is OK, as you normally don't go around calling 
3.dosomething() in Python.  But for a container meant to accept arbitrary 
objects, I'm not convinced that the cost of this kind of shadowing and 
blending of internal and proxy attributes is any good.

I really don't claim to understand one tenth of the (often contradictory) 
constraints that have led you to all these design decisions, and I trust your 
instinct a lot.  But in this case, my (perhaps naive) vote would be for x[0] 
to return the raw, un-proxied object always.

I can also see the flip side of the argument (not having to special-case code, 
so you can always assume that x[0].shape exists).  However, I think that my 
example above shows that this 'guarantee' is so weak as to be useless:

In [17]: if x[0].flags['ALIGNED']:
    ....:     print 'the stars are smiling'
    ....:
---------------------------------------------------------------------------
exceptions.TypeError                                 Traceback (most recent 
call last)

/home/fperez/<ipython console>

TypeError: string indices must be integers

Basically, if you get fed an 'O' array, today all bets are off as to what's 
going to come out of indexing a scalar element.

So you might as well, at least, have the guarantee that you get what you put 
in there to begin with, instead of the bizarre hybrid that comes out today.

OK, I've managed to convince myself that this _is_ the right thing to do (TM), 
and that 'O' arrays should return un-proxied values.  I vote +1 on that until 
proven wrong (in 3 minutes, I'm sure :)

> So, I'm still undecided.  Perhaps we can figure out why old-style 
> classes aren't working this way...

(I think) it's because only new-style classes implement the full descriptor 
protocol.  I imagine you are using property() for all these special 
attributes, and property() fails in silent, mysterious ways with old-style 
classes (I know because it was precisely this issue that forced me to make the 
main ipython class new-style a week ago).

Cheers,

f