Assigning generator expressions to ctype arrays

[Steven D'Aprano]

On Thu, 27 Oct 2011 13:34:28 -0700, Patrick Maupin wrote:

> Bug or misunderstanding?
> 
> Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) [GCC 4.5.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> x = 32 * [0]
>>>> x[:] = (x for x in xrange(32))
>>>> from ctypes import c_uint
>>>> x = (32 * c_uint)()
>>>> x[:] = xrange(32)
>>>> x[:] = (x for x in xrange(32))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: Can only assign sequence of same size

>From the outside, you can't tell how big a generator expression is. It 
has no length:

>>> g = (x for x in xrange(32))
>>> len(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'generator' has no len()

Since the array object has no way of telling whether the generator will 
have the correct size, it refuses to guess. I would argue that it should 
raise a TypeError with a less misleading error message, rather than a 
ValueError, so "bug".

The simple solution is to use a list comp instead of a generator 
expression. If you have an arbitrary generator passed to you from the 
outside, and you don't know how long it is yourself, you can use 
itertools.islice to extract just the number of elements you want. Given g 
some generator expression, rather than doing this:

# risky, if g is huge, the temporary list will also be huge
x[:] = list(g)[:32]

do this instead:

# use lazy slices guaranteed not to be unexpectedly huge
x[:] = list(itertools.islice(g, 32))


-- 
Steven