[Python-Dev] PEP 3142: Add a "while" clause to generator expressions

Calvin Spealman ironfroggy at gmail.com
Mon Jan 19 17:47:07 CET 2009 

On Mon, Jan 19, 2009 at 11:41 AM, Gerald Britton
<gerald.britton at gmail.com> wrote:
> Thanks Calvin,
>
> Could you please expand on your thoughts about possible confusion?
> That is, how do you see a programmer becoming confused if this option
> were added to the syntax.

I think that the difference between these two lines is too easy to
miss among other code:

    prime = (p for p in sieve() while p < 1000)

and:

    prime = (p for p in sieve() if p < 1000)

The very distinctly different behavior is two important to look so
similar at a glance. And, again, I just don't see takewhile being used
all that often. Not nearly often enough to warrant replacing it with a
special syntax! Only 178 results from Google CodeSearch, while chain,
groupby, and repeat get 4000, 3000, and 1000 respectively. Should
those be given their own syntax?

> On Mon, Jan 19, 2009 at 11:29 AM, Calvin Spealman <ironfroggy at gmail.com> wrote:
>> I am really unconvinced of the utility of this proposal and quite
>> convinced of the confusing factor it may well add to the current
>> syntax. I would like to see more applicable examples. It would replace
>> uses of takewhile, but that isn't a really often used function. So, is
>> there any evidence to support that making this a new syntax would find
>> so many more uses of the construct to be worth it? I believe not.
>>
>> On Mon, Jan 19, 2009 at 10:10 AM, Gerald Britton
>> <gerald.britton at gmail.com> wrote:
>>> Please find below PEP 3142: Add a "while" clause to generator
>>> expressions.  I'm looking for feedback and discussion.
>>>
>>>
>>> PEP: 3142
>>> Title: Add a "while" clause to generator expressions
>>> Version: $Revision: 68715 $
>>> Last-Modified: $Date: 2009-01-18 11:28:20 +0100 (So, 18. Jan 2009) $
>>> Author: Gerald Britton <gerald.britton at gmail.com>
>>> Status: Draft
>>> Type: Standards Track
>>> Content-Type: text/plain
>>> Created: 12-Jan-2009
>>> Python-Version: 3.0
>>> Post-History:
>>>
>>>
>>> Abstract
>>>
>>>  This PEP proposes an enhancement to generator expressions, adding a
>>>  "while" clause to complement the existing "if" clause.
>>>
>>>
>>> Rationale
>>>
>>>  A generator expression (PEP 289 [1]) is a concise method to serve
>>>  dynamically-generated objects to list comprehensions (PEP 202 [2]).
>>>  Current generator expressions allow for an "if" clause to filter
>>>  the objects that are returned to those meeting some set of
>>>  criteria.  However, since the "if" clause is evaluated for every
>>>  object that may be returned, in some cases it is possible that all
>>>  objects would be rejected after a certain point.  For example:
>>>
>>>      g = (n for n in range(100) if n*n < 50)
>>>
>>>  which is equivalent to the using a generator function
>>>  (PEP 255 [3]):
>>>
>>>      def __gen(exp):
>>>          for n in exp:
>>>              if n*n < 50:
>>>                  yield n
>>>      g = __gen(iter(range(10)))
>>>
>>>  would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also consider
>>>  the numbers from 8 to 99 and reject them all since n*n >= 50 for
>>>  numbers in that range.  Allowing for a "while" clause would allow
>>>  the redundant tests to be short-circuited:
>>>
>>>      g = (n for n in range(100) while n*n < 50)
>>>
>>>  would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8
>>>  since the condition (n*n < 50) is no longer true.  This would be
>>>  equivalent to the generator function:
>>>
>>>      def __gen(exp):
>>>          for n in exp:
>>>              if n*n < 50:
>>>                  yield n
>>>              else:
>>>                  break
>>>      g = __gen(iter(range(100)))
>>>
>>>  Currently, in order to achieve the same result, one would need to
>>>  either write a generator function such as the one above or use the
>>>  takewhile function from itertools:
>>>
>>>      from itertools import takewhile
>>>      g = takewhile(lambda n: n*n < 50, range(100))
>>>
>>>  The takewhile code achieves the same result as the proposed syntax,
>>>  albeit in a longer (some would say "less-elegant") fashion.  Also,
>>>  the takewhile version requires an extra function call (the lambda
>>>  in the example above) with the associated performance penalty.
>>>  A simple test shows that:
>>>
>>>      for n in (n for n in range(100) if 1): pass
>>>
>>>  performs about 10% better than:
>>>
>>>      for n in takewhile(lambda n: 1, range(100)): pass
>>>
>>>  though they achieve similar results.  (The first example uses a
>>>  generator; takewhile is an iterator).  If similarly implemented,
>>>  a "while" clause should perform about the same as the "if" clause
>>>  does today.
>>>
>>>  The reader may ask if the "if" and "while" clauses should be
>>>  mutually exclusive.  There are good examples that show that there
>>>  are times when both may be used to good advantage. For example:
>>>
>>>      p = (p for p in primes() if p > 100 while p < 1000)
>>>
>>>  should return prime numbers found between 100 and 1000, assuming
>>>  I have a primes() generator that yields prime numbers.  Of course, this
>>>  could also be achieved like this:
>>>
>>>      p = (p for p in (p for p in primes() if p > 100) while p < 1000)
>>>
>>>  which is syntactically simpler.  Some may also ask if it is possible
>>>  to cover dropwhile() functionality in a similar way.  I initially thought
>>>  of:
>>>
>>>      p = (p for p in primes() not while p < 100)
>>>
>>>  but I am not sure that I like it since it uses "not" in a non-pythonic
>>>  fashion, I think.
>>>
>>>  Adding a "while" clause to generator expressions maintains the
>>>  compact form while adding a useful facility for short-circuiting
>>>  the expression.
>>>
>>> Implementation:
>>>
>>>  I am willing to assist in the implementation of this feature, although I have
>>>  not contributed to Python thus far and would definitely need mentoring. (At
>>>  this point I am not quite sure where to begin.)  Presently though, I would
>>>  find it challenging to fit this work into my existing workload.
>>>
>>>
>>> Acknowledgements
>>>
>>>  Raymond Hettinger first proposed the concept of generator
>>>  expressions in January 2002.
>>>
>>>
>>> References
>>>
>>>  [1] PEP 289: Generator Expressions
>>>      http://www.python.org/dev/peps/pep-0289/
>>>
>>>  [2] PEP 202: List Comprehensions
>>>      http://www.python.org/dev/peps/pep-0202/
>>>
>>>  [3] PEP 255: Simple Generators
>>>      http://www.python.org/dev/peps/pep-0255/
>>>
>>>
>>> Copyright
>>>
>>>  This document has been placed in the public domain.
>>>
>>>
>>> Local Variables:
>>> mode: indented-text
>>> indent-tabs-mode: nil
>>> sentence-end-double-space: t
>>> fill-column: 70
>>> coding: utf-8
>>> End:
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
>>>
>>>
>>
>>
>>
>> --
>> Read my blog! I depend on your acceptance of my opinion! I am interesting!
>> http://techblog.ironfroggy.com/
>> Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
>>
>



-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

More information about the Python-Dev mailing list