Generator (re-)definition within a loop

[Pierre Reinbold]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/18/2010 11:48 PM, Terry Reedy wrote:
> On 6/18/2010 3:57 PM, Pierre Reinbold wrote:
>
>> def genexp_product(*args):
>>      pools = map(tuple, args)
>>      result = [[]]
>>      for pool in pools:
>>          result = (x+[y] for x in result for y in pool)
> 
> The name binding in the first for-clause in a genexp is handled slightly
> differently from that in subsequent for-clauses. I suspect that this is
> relevant here. This code rebinds 'result' to a new generator *without*
> running running the previously bound 'result' generator.
> 
>>      for prod in result:
>>          yield tuple(prod)
> 
> This runs N nested generators, but which?
> 
>> but this do not work as expected:
>>
>>>>> print list(product("ABC", "xy"))
>> [('A', 'x'), ('A', 'y'), ('B', 'x'), ('B', 'y'), ('C', 'x'), ('C', 'y')]
>>>>> print list(genexp_product("ABC", "xy"))
>> [('x', 'x'), ('x', 'y'), ('y', 'x'), ('y', 'y')]
>>
>> My question is why ? What is the final generator "result" at the end of
>> the main loop in genexp_product ? How is it build exactly ? Is this an
>> effet of some sort of "lazy evaluation" ?
> 
> That and/or a namespace issue.
> 
> Let's apply Reedy's Rule: when you have trouble understanding a function
> expression, replace it with the (near) equivalent def statement. (Among
> other advantages, one can insert print calls!)
> 
> Genexps, like lambdas, are specialized function expressions.
> 
> def augment(s_of_s, s):
>      for x in s_of_s:
>           for y in s:
>                yield x+[y]
> 
> def gen_product(*args):
>     pools = map(tuple, args)
>     result = [[]]
>     for pool in pools:
>         result = augment(result,pool)
>     for prod in result:
>         yield tuple(prod)
> 
> print(list(gen_product("ABC", "xy")))
> 
>>>> #3.1
> [('A', 'x'), ('A', 'y'), ('B', 'x'), ('B', 'y'), ('C', 'x'), ('C', 'y')]

Very instructive post ! Thank you !

Just trying to understand, I have apply the Reedy's Rule in an attempt
to reproduce the same behaviour as the generator expression. The idea, I
guess, is to (re-)define the generator function inside the loop.

My first try gives this:

def badgen_product1(*args, **kwds):
    pools = map(tuple, args)
    result = [[]]
    for pool in pools:
        def result():
            for x in result():
                for y in pool:
                    yield x+[y]
    for prod in result():
        yield tuple(prod)

But this does not reproduce the generator expression, it leads naturally
to an infinite recursion (which is what I expected first for the
generator expression btw)

    for x in result():
RuntimeError: maximum recursion depth exceeded

Another try to avoid infinite recursion:

def badgen_product2(*args, **kwds):
    pools = map(tuple, args)
    result = [[]]
    for pool in pools:
        def augments():
            for x in result:
                for y in pool:
                    yield x+[y]
        result = augments()
    for prod in result:
        yield tuple(prod)

And this one gives the infamous:

    for x in result:
ValueError: generator already executing

Which seems to indicate that the lazy evaluation leads to eventually
bind everything to the same generator.

So, neither of my attempts reproduce the behaviour of the generator
expression. What do I miss here ?

Thank for your help,


3.14r
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkwfFNIACgkQ9D25xYOIvisX3ACgu3BjwvepFDcbuUbtvaUc10eD
pJkAnA3cqPEM60kYfrcNLI6qmFzHvtRe
=LxCp
-----END PGP SIGNATURE-----