[Python-checkins] r46060 - sandbox/trunk/Doc/functional.rst
andrew.kuchling
python-checkins at python.org
Sat May 20 21:54:13 CEST 2006
Author: andrew.kuchling
Date: Sat May 20 21:54:13 2006
New Revision: 46060
Modified:
sandbox/trunk/Doc/functional.rst
Log:
Add generator section; various markup fixes
Modified: sandbox/trunk/Doc/functional.rst
==============================================================================
--- sandbox/trunk/Doc/functional.rst (original)
+++ sandbox/trunk/Doc/functional.rst Sat May 20 21:54:13 2006
@@ -60,9 +60,9 @@
discourage the use of assignment statements such as ``a=3`` or ``c = a
+ b``.
-Functional programming may seem like an odd constraint to work under,
-a pointless skill like writing a BASIC interpreter in {\TeX}. There
-are theoretical and practical advantages to the functional style:
+Functional programming may seem like an odd constraint to work under.
+There are theoretical and practical advantages to the functional
+style:
* Formal provability.
* Modularity.
@@ -224,7 +224,7 @@
Note that you can only go forward in an iterator; there's no way to
get the previous element, reset the iterator, or make a copy of it.
Iterator objects can optionally provide these additional capabilities,
-but the iterator protocol only specifies the \method{next()} method.
+but the iterator protocol only specifies the ``next()`` method.
Functions may therefore consume all of the iterator's output, and if
you need to do something different with the same stream, you'll have
to create a new iterator.
@@ -260,7 +260,7 @@
This is just the default behaviour. If you want to iterate over keys,
values, or key/value pairs, you can explicitly call the
-\method{iterkeys()}, \method{itervalues()}, or \method{iteritems()}
+``iterkeys()``, ``itervalues()``, or ``iteritems()``
methods to get an appropriate iterator.
The ``dict()`` constructor can accept an iterator that returns a
@@ -270,15 +270,13 @@
>>> dict(iter(L))
{'Italy': 'Rome', 'US': 'Washington DC', 'France': 'Paris'}
-Files also support iteration by calling the \method{readline()}
+Files also support iteration by calling the ``readline()``
method until there are no more lines in the file. This means you can
-read each line of a file like this:
+read each line of a file like this::
-\begin{verbatim}
-for line in file:
- # do something for each line
- ...
-\end{verbatim}
+ for line in file:
+ # do something for each line
+ ...
XXX sets
@@ -295,7 +293,7 @@
List comprehensions (or "listcomps") are a concise notation for such
operations, borrowed from the functional programming language Haskell
-(\url{http://www.haskell.org}). You can strip all the whitespace from
+(http://www.haskell.org). You can strip all the whitespace from
a stream of strings with the following list comprehension::
line_list = [' line 1\n', 'line 2 \n', ...]
@@ -320,9 +318,9 @@
if condition ]
The elements of the generated list will be the successive
-values of \var{expression}. The final \keyword{if} clause is
-optional; if present, \var{expression} is only evaluated and added to
-the result when \var{condition} is true.
+values of ``expression``. The final ``if`` clause is
+optional; if present, ``expression`` is only evaluated and added to
+the result when ``condition`` is true.
The ``for...in`` clauses contain the sequences to be iterated over.
The sequences do not have to be the same length, because they are
@@ -332,23 +330,21 @@
resulting pair of elements from ``sequence1`` and ``sequence2``.
Put another way, a list comprehension is equivalent to the following
-Python code:
+Python code::
-\begin{verbatim}
-for expr1 in sequence1:
- for expr2 in sequence2:
- ...
- for exprN in sequenceN:
- if (condition):
- # Append the value of
- # the expression to the
- # resulting list.
-\end{verbatim}
+ for expr1 in sequence1:
+ for expr2 in sequence2:
+ ...
+ for exprN in sequenceN:
+ if (condition):
+ # Append the value of
+ # the expression to the
+ # resulting list.
-This means that when there are multiple \keyword{for}...\keyword{in}
+This means that when there are multiple ``for...in``
clauses, the resulting list will be equal to the product of the
lengths of all the sequences. If you have two lists of length 3, the
-output list is 9 elements long:
+output list is 9 elements long::
seq1 = 'abc'
seq2 = (1,2,3)
@@ -371,12 +367,202 @@
Generators
-----------------------
-Generators are a special class of functions that simplify
-the task of writing certain kinds of iterators.
+Generators are a special class of functions that simplify the task of
+writing iterators. Regular functions compute a value and return it,
+but generators return an iterator that returns a stream of values.
+
+You're doubtless familiar with how regular function calls work in
+Python or C. When you call a function, it gets a private namespace
+where its local variables are created. When the function reaches a
+``return`` statement, the local variables are destroyed and the
+value is returned to the caller. A later call to the same function
+creates a new private namespace and a fresh set of local
+variables. But, what if the local variables weren't thrown away on
+exiting a function? What if you could later resume the function where
+it left off? This is what generators provide; they can be thought of
+as resumable functions.
+
+Here's the simplest example of a generator function::
+
+ def generate_ints(N):
+ for i in range(N):
+ yield i
+
+Any function containing a ``yield`` keyword is a generator function;
+this is detected by Python's bytecode compiler which compiles the
+function specially as a result.
+
+When you call a generator function, it doesn't return a single value;
+instead it returns a generator object that supports the iterator
+protocol. On executing the ``yield`` expression, the generator
+outputs the value of ``i``, similar to a ``return``
+statement. The big difference between ``yield`` and a
+``return`` statement is that on reaching a ``yield`` the
+generator's state of execution is suspended and local variables are
+preserved. On the next call to the generator's ``.next()`` method,
+the function will resume executing.
+
+Here's a sample usage of the ``generate_ints()`` generator::
+
+ >>> gen = generate_ints(3)
+ >>> gen
+ <generator object at 0x8117f90>
+ >>> gen.next()
+ 0
+ >>> gen.next()
+ 1
+ >>> gen.next()
+ 2
+ >>> gen.next()
+ Traceback (most recent call last):
+ File "stdin", line 1, in ?
+ File "stdin", line 2, in generate_ints
+ StopIteration
-XXX {Copy bits of what's new}
+You could equally write ``for i in generate_ints(5)``, or
+``a,b,c = generate_ints(3)``.
+
+Inside a generator function, the ``return`` statement can only be used
+without a value, and signals the end of the procession of values;
+after executing a ``return`` the generator cannot return any further
+values. ``return`` with a value, such as ``return 5``, is a syntax
+error inside a generator function. The end of the generator's results
+can also be indicated by raising ``StopIteration`` manually, or by
+just letting the flow of execution fall off the bottom of the
+function.
+
+You could achieve the effect of generators manually by writing your
+own class and storing all the local variables of the generator as
+instance variables. For example, returning a list of integers could
+be done by setting ``self.count`` to 0, and having the
+``next()`` method increment ``self.count`` and return it.
+However, for a moderately complicated generator, writing a
+corresponding class can be much messier.
+
+The test suite included with Python's library, ``test_generators.py``,
+contains a number of more interesting examples. Here's one generator
+that implements an in-order traversal of a tree using generators
+recursively.
+
+::
+
+ # A recursive generator that generates Tree leaves in in-order.
+ def inorder(t):
+ if t:
+ for x in inorder(t.left):
+ yield x
+ yield t.label
+ for x in inorder(t.right):
+ yield x
+
+Two other examples in ``test_generators.py`` produce
+solutions for the N-Queens problem (placing N queens on an NxN
+chess board so that no queen threatens another) and the Knight's Tour
+(finding a route that takes a knight to every square of an NxN chessboard
+without visiting any square twice).
+
+
+
+New generator features in Python 2.5
+''''''''''''''''''''''''''''''''''''''''''''''
+
+In Python 2.4 and earlier, generators only produced output. Once a
+generator's code was invoked to create an iterator, there's no way to
+pass any new information into the function when its execution is
+resumed. (You could hack together this ability by making the
+generator look at a global variable or passing in some mutable object
+that callers then modify, but these approaches are messy.)
+In Python 2.5 there's a simple way to pass values into a generator.
+
+In Python 2.5 ``yield`` became an expression, returning a value that
+can be assigned to a variable or otherwise operated on::
+
+ val = (yield i)
+
+I recommend that you **always** put parentheses around a ``yield``
+expression when you're doing something with the returned value, as in
+the above example. The parentheses aren't always necessary, but it's
+easier to always add them instead of having to remember when they're
+needed.
+
+(PEP 342 explains the exact rules, which are that a
+``yield``-expression must always be parenthesized except when it
+occurs at the top-level expression on the right-hand side of an
+assignment. This means you can write ``val = yield i`` but have to
+use parentheses when there's an operation, as in ``val = (yield i)
++ 12``.)
+
+Values are sent into a generator by calling its
+``send(value})`` method. This method resumes the
+generator's code and the ``yield`` expression returns the specified
+value. If the regular ``next()`` method is called, the
+``yield`` returns ``None``.
+
+Here's a simple counter that increments by 1 and allows changing the
+value of the internal counter.
+
+::
+
+ def counter (maximum):
+ i = 0
+ while i < maximum:
+ val = (yield i)
+ # If value provided, change counter
+ if val is not None:
+ i = val
+ else:
+ i += 1
+
+And here's an example of changing the counter:
+
+ >>> it = counter(10)
+ >>> print it.next()
+ 0
+ >>> print it.next()
+ 1
+ >>> print it.send(8)
+ 8
+ >>> print it.next()
+ 9
+ >>> print it.next()
+ Traceback (most recent call last):
+ File ``t.py'', line 15, in ?
+ print it.next()
+ StopIteration
-Mention 2.5 additions but don't describe them.
+Because ``yield`` will often be returning ``None``, you
+should always check for this case. Don't just use its value in
+expressions unless you're sure that the ``send()`` method
+will be the only method used resume your generator function.
+
+In addition to ``send()``, there are two other new methods on
+generators:
+
+* ``throw(type, value=None, traceback=None)`` is used to raise an exception inside the
+ generator; the exception is raised by the ``yield`` expression
+ where the generator's execution is paused.
+
+* ``close()`` raises a ``GeneratorExit``
+ exception inside the generator to terminate the iteration.
+ On receiving this
+ exception, the generator's code must either raise
+ ``GeneratorExit`` or ``StopIteration``; catching the
+ exception and doing anything else is illegal and will trigger
+ a ``RuntimeError``. ``close()`` will also be called by
+ Python's garbage collector when the generator is garbage-collected.
+
+ If you need to run cleanup code when a ``GeneratorExit`` occurs,
+ I suggest using a ``try: ... finally:`` suite instead of
+ catching ``GeneratorExit``.
+
+The cumulative effect of these changes is to turn generators from
+one-way producers of information into both producers and consumers.
+
+Generators also become **coroutines**, a more generalized form of
+subroutines. Subroutines are entered at one point and exited at
+another point (the top of the function, and a ``return``
+statement), but coroutines can be entered, exited, and resumed at
+many different points (the ``yield`` statements).
The itertools module
More information about the Python-checkins
mailing list