[Python-Dev] why _PyGen_Finalize(gen) propagates close() to _PyGen_yf() ?

[Oleg Nesterov]

Hello,

Let me first clarify, I do not claim this is a bug, I am trying to learn
python and now I trying to understand yield-from.

This simple test-case

	g = (x for x in range(10))

	def fg():
		for x in g:
			yield x

	print(next(fg()))
	print(next(g))

works as expected and prints:

	0
	1

However, if I change fg() to use yield-from

	g = (x for x in range(10))

	def fg():
		yield from g

	print(next(fg()))
	print(next(g))

then next(g) raises StopIteration:

	0
	Traceback (most recent call last):
	  File "/tmp/T.py", line 10, in <module>
	    print(next(g))
	StopIteration

because g.close() is called by destructor of the object returned by fg().

To me this looks strange and confusing. I tried to google, and found
https://docs.python.org/3/whatsnew/3.3.html#pep-380 but it doesn't document
this behaviour. I understand that yield-from should propagate .close(), but
why _PyGen_Finalize() should send close() to the gi_yieldfrom object?

I applied the patch below just to verify that I actually understand what's
going on, and with this patch the 2nd test-case works as I'd expect. But
since I am very new to python I'd suspect that the code is fine and I simply
do not understand why it works this way.

So. could someone please explain the rationale behind this behaviour? And
probably update the docs should be updated?

Thanks in advance.

Oleg.
---

diff --git a/Objects/genobject.c b/Objects/genobject.c
index 24a1da6..d5152eb 100644
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -6,6 +6,7 @@
 #include "opcode.h"
 
 static PyObject *gen_close(PyGenObject *, PyObject *);
+static PyObject *do_gen_close(PyGenObject *, PyObject *);
 static PyObject *async_gen_asend_new(PyAsyncGenObject *, PyObject *);
 static PyObject *async_gen_athrow_new(PyAsyncGenObject *, PyObject *);
 
@@ -71,7 +72,7 @@ _PyGen_Finalize(PyObject *self)
         }
     }
     else {
-        res = gen_close(gen, NULL);
+        res = do_gen_close(gen, NULL);
     }
 
     if (res == NULL) {
@@ -373,10 +374,9 @@ _PyGen_yf(PyGenObject *gen)
 }
 
 static PyObject *
-gen_close(PyGenObject *gen, PyObject *args)
+do_gen_close(PyGenObject *gen, PyObject *yf)
 {
     PyObject *retval;
-    PyObject *yf = _PyGen_yf(gen);
     int err = 0;
 
     if (yf) {
@@ -407,6 +407,11 @@ gen_close(PyGenObject *gen, PyObject *args)
     return NULL;
 }
 
+static PyObject *
+gen_close(PyGenObject *gen, PyObject *args)
+{
+    return do_gen_close(gen, _PyGen_yf(gen));
+}
 
 PyDoc_STRVAR(throw_doc,
 "throw(typ[,val[,tb]]) -> raise exception in generator,\n\