[Patches] [ python-Patches-1223381 ] PEP 342 Generator enhancements

Mon Aug 1 12:10:29 CEST 2005

Patches item #1223381, was opened at 2005-06-19 00:08
Message generated for change (Comment added) made by mwh
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1223381&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core (C code)
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Phillip J. Eby (pje)
Assigned to: Phillip J. Eby (pje)
Summary: PEP 342 Generator enhancements

Initial Comment:
The attached patch implements code and test changes for:

* yield expressions
* bare yield (short for yield None)
* yield in try/finally
* generator.send(value) (send value into generator;
substituted for PEP 342's next(arg))
* generator.throw(typ[,val[,tb]]) (raise error in
generator)
* generator.close() (per PEP 343)
* GeneratorExit built-in exception type
* generator.__del__ (well, the C equivalent)
* All necessary mods to the compiler, parser module,
and Python 'compiler' package to support these changes.

It was necessary to change a small part of the eval
loop (well, the initialization, not the loop) and the
gc module's has_finalizer() logic in order to support a
C equivalent to __del__.  Specialists in these areas
should probably scrutinize this patch!

There is one additional implementation detail that was
not contemplated in either PEP. in order to prevent
used-up generators from retaining unnecessary
references to their frame's contents, I set the
generator's gi_frame member to None whenever the
generator finishes normally or with an error.  Thus, an
exhausted generator cannot be part of a cycle, and it
releases its frame object sooner than in previous
Python versions.  For generators used only in a direct
"for" loop, this makes no difference, but for
generators used with the iterator protocol (i.e.
"gen.next()") from Python, this avoids stranding the
generator's frame in a traceback cycle.

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2005-08-01 11:10

Message:
Logged In: YES 
user_id=6656

I think so, yes :)

----------------------------------------------------------------------

Comment By: Phillip J. Eby (pje)
Date: 2005-07-31 22:10

Message:
Logged In: YES 
user_id=56214

mwh, is the only difference in your patch the addition of
the incref?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-07-07 15:14

Message:
Logged In: YES 
user_id=6656

You missed an incref in gen_throw (symptom was

Fatal Python error: ../Objects/tupleobject.c:169 object at
0x4013807c has negative ref count -606348326

on exit after running test_generators).  New patch attached.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2005-06-28 08:10

Message:
Logged In: YES 
user_id=6380

Since I've accepted PEP 342+343 last night in my Europython
keynote, you can check this in now.

----------------------------------------------------------------------

Comment By: Phillip J. Eby (pje)
Date: 2005-06-27 16:40

Message:
Logged In: YES 
user_id=56214

I had hoped to be able to avoid dealing with string
exceptions, but I guess it makes sense that if somebody has
thrown one, it should be able to propagate. So I guess I'll
take care of that along with the rest.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-06-27 16:31

Message:
Logged In: YES 
user_id=6656

Heh heh heh.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2005-06-27 16:22

Message:
Logged In: YES 
user_id=6380

(Removing 343 from the subject since all this is now in PEP
342.)

Playing with the code a bit, I notice that throw() doesn't
like string exceptions. That could cause some problems.

Also, I think the tb argument should be allowed to be None.

Otherwise, great! In a few hours you can check it in. :-)

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-06-24 00:20

Message:
Logged In: YES 
user_id=6656

Fair enough.  I want to use with statements in a presentation at 
EuroPython next week, so I've hacked together a truly horrible 
implementation that will do for now and will see about a real one some time 
after...

----------------------------------------------------------------------

Comment By: Phillip J. Eby (pje)
Date: 2005-06-24 00:07

Message:
Logged In: YES 
user_id=56214

No, I haven't, and I never really planned to be the person
implementing "with".  Doesn't mean I might not end up doing
it eventually, but it's not currently in my plans.  I'm
going to have my hands full just updating PEP 342 (Guido has
asked me to take it over), and getting docs written to go
with this patch.  Between all that and my distutils-related
work, my pro-bono Python project time is booked up solid for
maybe the next 2 months or so.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-06-22 21:59

Message:
Logged In: YES 
user_id=6656

Argh, I'd hoped you'd done with, too :)

Have you done any work on that yet?

----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-06-19 08:37

Message:
Logged In: YES 
user_id=1188172

Minor nit: on line 642, 'yield_stmt' should be changed to
'yield_expr'.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1223381&group_id=5470