[Numpy-discussion] numpy all unexpected result (generator)

[Neal Becker]

Dag Sverre Seljebotn wrote:

> On 01/31/2012 03:07 PM, Robert Kern wrote:
>> On Tue, Jan 31, 2012 at 13:26, Neal Becker<ndbecker2 at gmail.com>  wrote:
>>> I was just bitten by this unexpected behavior:
>>>
>>> In [24]: all ([i>    0 for i in xrange (10)])
>>> Out[24]: False
>>>
>>> In [25]: all (i>    0 for i in xrange (10))
>>> Out[25]: True
>>>
>>> Turns out:
>>> In [31]: all is numpy.all
>>> Out[31]: True
>>>
>>> So numpy.all doesn't seem to do what I would expect when given a generator.
>>> Bug?
>>
>> Expected behavior. numpy.all(), like nearly all numpy functions,
>> converts the input to an array using numpy.asarray(). numpy.asarray()
>> knows nothing special about generators and other iterables that are
>> not sequences, so it thinks it's a single scalar object. This scalar
>> object happens to have a __nonzero__() method that returns True like
>> most Python objects that don't override this.
>>
>> In order to use generic iterators that are not sequences, you need to
>> explicitly use numpy.fromiter() to convert them to ndarrays. asarray()
>> and array() can't do it in general because they need to autodiscover
>> the shape and dtype all at the same time.
> 
> Perhaps np.asarray could specifically check for a generator argument and
> raise an exception? I imagine that would save people some time when
> running into this...
> 
> If you really want
> 
> In [7]: x = np.asarray(None)
> 
> In [8]: x[()] = (i for i in range(10))
> 
> In [9]: x
> Out[9]: array(<generator object <genexpr> at 0x4553fa0>, dtype=object)
> 
> ...then one can type it out?
> 
> Dag

The reason it surprised me, is that python 'all' doesn't behave as numpy 'all' 
in this respect - and using ipython, I didn't even notice that 'all' was 
numpy.all rather than standard python all.  All in all, rather unfortunate :)