Sending changed parameters into nested generators

[cbrown at cbrownsystems.com]

On Nov 12, 10:52 pm, "John O'Hagan" <resea... at johnohagan.com> wrote:
> On Sat, 13 Nov 2010, Steven D'Aprano wrote:
> > On Fri, 12 Nov 2010 09:47:26 +0000, John O'Hagan wrote:
> > > I have a generator function which takes as arguments another generator
> > > and a dictionary of other generators like this:
>
> > > def modgen(gen, gendict):
> > >     for item in gen():
> > >       for k, v in gendict:
> > >               do_something_called_k(item, v.next())
>
> > >       yield item
>
> > [snip]
>
> > > If anyone's still reading :) , how can I send new values to arbitrary
> > > sub- generators?
>
> > I have a headache after reading your problem :(
>
> > I think it's a good time to point you at the Zen, particularly these five
> > maxims:
>
> > Beautiful is better than ugly.
> > Simple is better than complex.
> > Complex is better than complicated.
> > Flat is better than nested.
> > If the implementation is hard to explain, it's a bad idea.
>
> > I'm afraid that your nested generators inside another generator idea
> > fails all of those... it's not elegant (beautiful), it's complicated,
> > it's nested, and the implementation is hard to explain.
>
> > You could probably replace generators with full-blown iterators, but I
> > wonder what you're trying to accomplish that is so complicated that it
> > needs such complexity to solve it. What are you actually trying to
> > accomplish? Can you give a simple example of what practical task you hope
> > to perform? I suspect there's probably a more elegant way to solve the
> > problem.
>
> I hope there is!
>
> The project not practical but artistic; it's a real-time musical composition
> program.
>
> A (simplified) description: one module contains number-list generating
> functions, others contain functions designed to filter and modify the number
> lists produced, according to various parameters. Each such stream of number
> lists is assigned a musical meaning (e.g. pitch, rhythm, volume, etc) and they
> are combined to produce representations of musical phrases, which are sent to
> a backend which plays the music as it is produced, and makes PDF scores. Each
> such "instrument" runs as a separate thread, so several can play together in a
> coordinated fashion.
>
> All the compositional interest lies in the selection of number-list generators
> and how their output is modified. For example, if I say "Play every third note
> up an octave" it's not very interesting, compared to "Play every nth note up
> an interval of m", where n and m vary according to some pattern. It gets even
> more interesting when that pattern is a function of x and y, which also vary
> according to another pattern, and so on.
>
> To that end, I have each parameter of each modifier set by another generator,
> such that the value may change with each iteration. This may continue
> recursively, until at some level we give a parameter a simple value.
>
> That's all working, but I also want it to be interactive. Each thread opens a
> terminal where new options can be entered, but so far it only works, as I
> mentioned, for changing the values in a top-level mutable object.
>

I might first suggest this, although I have some caveats to add:

def genfilter(evaluator, **param_sources):
    while True:
        params = {}
        for param, gen in param_sources.iteritems():
            params[param] = gen.next()
        yield evaluator(**params)

You can then do things like:

>>> def concat(in1, in2):
>>>     return str(in1)+"|"+str(in2)
>>> a = (i for i in range(1,5))      # generator based on a list
>>> b = (2*i for i in xrange(1,5))   # 'pure' generator
>>> c = genfilter(concat, in1=a, in2=b)
>>> for x in c:
>>>     print x
1|2
2|4
3}6
4|8

or, more relevant to your original question regarding modifying things
mid-stream:

>>> class Mult():
>>>     def __init__(self, multiplier):
>>>         self.mulitplier = multiplier
>>>     def multi(self, val):
>>>         return val*self.multiplier
>>>
>>> m = Mult(2)
>>> a = (i for i in range(1,10))
>>> b = (i for i in range(1,10))
>>> c = genfilter(m.multi, val=b)
>>> d = genfilter(concat, in1=a, in2=c)
>>> d.next()
1|2
>>> d.next()
2|4
>>> m.multiplier = 3
>>> d.next()
3|9

Obviously more complex and useful examples could be made, but
hopefully you get the idea: don't focus on modifying the generators,
instead modify the action of the functions which take the generators
as inputs.

But a real problem with this whole strategy is that a generator's
next() function is called every time it is evaluated. If the
relationship between your various generators forms a rooted tree,
that's not a problem, but I would think the relationships form a
directed acyclic graph, and in that case, you end up 'double
incrementing' nodes in a way you don't want:

>>> m = Mult(2)
>>> a = (i for i in range(1,10))
>>> b = genfilter(m.multi, val=a)
>>> c = genfilter(concat, in1=a, in2=b)
>>> for x in c:
>>>     print x
1|4
3|8
5|12
7|16

And that's not an unnatural situation for your application: you might
have a generator that produces a note, and then three other distinct
genfilters that you want to use to create three instruments which form
a chord based on that note. With the above pattern (and also your
original pattern) you would end up invoking the note generator three
times; and I think you don't really want that.

To solve that problem, you need a somewhat more complex solution: a
class that ensures that each previous stage is only invoked once per
'pass'. I've got an idea for that, if that is of interest.

Cheers - Chas