Reducing "yield from" overhead in recursive generators

[Barry]

> On 19 Mar 2022, at 03:07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
> On 19/03/22 9:40 am, Rathmann wrote:
>> The other challenge/question would be to see if there is a way to implement
>> this basic approach with lower overhead.
> 
> My original implementation of yield-from didn't have this problem,
> because it didn't enter any of the intermediate frames -- it just
> ran down a chain of C pointers to find the currently active one.
> 
> At some point this was changed, I think so that all the frames
> would show up in the traceback in the event of an exception.
> This was evidently seen as more important than having efficient
> resumption of nested generators.

I would have thought that it would be possible to do both sets of pointer/accounting.
One that supports your original fast implementation and add to that the info to do the
trace back.

Barry

> 
> I'm not sure exactly how it works now, but I believe it involves
> re-executing the YIELD_FROM bytecode in each generator down the
> chain whenever a resumption occurs. This is nice and simple, but
> not very efficient.
> 
> Maybe another way could be found of preserving the traceback,
> such as temporarily splicing the chain of frames into the call
> stack and then resuming the last one.
> 
> -- 
> Greg
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>