Reducing "yield from" overhead in recursive generators

[Greg Ewing]

On 19/03/22 9:40 am, Rathmann wrote:
> The other challenge/question would be to see if there is a way to implement
> this basic approach with lower overhead.

My original implementation of yield-from didn't have this problem,
because it didn't enter any of the intermediate frames -- it just
ran down a chain of C pointers to find the currently active one.

At some point this was changed, I think so that all the frames
would show up in the traceback in the event of an exception.
This was evidently seen as more important than having efficient
resumption of nested generators.

I'm not sure exactly how it works now, but I believe it involves
re-executing the YIELD_FROM bytecode in each generator down the
chain whenever a resumption occurs. This is nice and simple, but
not very efficient.

Maybe another way could be found of preserving the traceback,
such as temporarily splicing the chain of frames into the call
stack and then resuming the last one.

-- 
Greg