From george.sakkis at gmail.com Fri May 1 01:27:13 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 30 Apr 2009 19:27:13 -0400 Subject: [Python-ideas] caching properties In-Reply-To: <20090430221900.6e4c0181@o> References: <20090430090004.57c2e51c@o> <9e754ef50904300536n35009806w4509e4f17da0ebc3@mail.gmail.com> <200904302356.48819.steve@pearwood.info> <20090430221900.6e4c0181@o> Message-ID: <91ad5bf80904301627s512a918cuaac35bc61c6947d8@mail.gmail.com> On Thu, Apr 30, 2009 at 4:19 PM, spir wrote: > (Aside from the hashable issue pointed by Arnaud) > I wonder about having the whole parameter tuple as key for caching. > In packrat parsing, you may have more than one parameter (including the source, indeed) but only one is relevant for memoizing (the position). Cache has to be reset anyway when starting a new parse, so that having the source in keys is irrelevant. > Typically, if not a single value, I guess saved results form a simple array depending on an ordinal. I think It's clear by now that caching in the general case is not trivial, both in terms of API and implementation. That makes the original request - caching properties only - more appealing since most problems go away if there are no parameters. In the simplest case where cache expiration is not supported, cachedproperty can be a 7-line decorator: def cachedproperty(fget): def fget_wrapper(self): try: return fget_wrapper._cached except AttributeError: fget_wrapper._cached = value = fget(self) return value return property(fget_wrapper, doc=fget.__doc__) Cache expiration can be exposed by (ab)using the deleter; "del obj.prop" looks much better than "ObjType.prop.fget.expire(obj)": def cachedproperty(fget): def fget_wrapper(self): try: return fget_wrapper._cached except AttributeError: fget_wrapper._cached = value = fget(self) return value def fdel(self): try: del fget_wrapper._cached except AttributeError: pass return property(fget_wrapper, fdel=fdel, doc=fget.__doc__) And finally here's a general version that supports properly the new in 2.6 getter()/setter()/deleter() methods (note that setting a property expires the cache, just like delete): class cachedproperty(property): def __init__(self, fget=None, fset=None, fdel=None, doc=None): if fget is not None: def fget_wrapper(obj): try: return self._cached except AttributeError: self._cached = value = fget(obj) return value else: fget_wrapper = None if fset is not None: def fset_wrapper(obj, value): fset(obj,value) try: del self._cached except AttributeError: pass else: fset_wrapper = None if fdel is not None: def fdel_wrapper(obj): fdel(obj) try: del self._cached except AttributeError: pass else: def fdel_wrapper(obj): try: del self._cached except AttributeError: pass super(cachedproperty,self).__init__(fget_wrapper, fset_wrapper, fdel_wrapper, doc) self.__doc__ = doc or getattr(fget, '__doc__', None) # store for getter() / setter() / deleter() self._fget, self._fset, self._fdel = fget, fset, fdel def getter(self, getter): return self.__class__(getter, self._fset, self._fdel, self.__doc__) def setter(self, setter): return self.__class__(self._fget, setter, self._fdel, self.__doc__) def deleter(self, deleter): return self.__class__(self._fget, self._fset, deleter, self.__doc__) George From ben+python at benfinney.id.au Fri May 1 01:37:37 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 01 May 2009 09:37:37 +1000 Subject: [Python-ideas] caching properties References: <20090430090004.57c2e51c@o> <9e754ef50904300536n35009806w4509e4f17da0ebc3@mail.gmail.com> <200904302356.48819.steve@pearwood.info> <20090430221900.6e4c0181@o> <91ad5bf80904301627s512a918cuaac35bc61c6947d8@mail.gmail.com> Message-ID: <87r5z9ogqm.fsf@benfinney.id.au> George Sakkis writes: > I think It's clear by now that caching in the general case is not > trivial, both in terms of API and implementation. That makes the > original request - caching properties only - more appealing since most > problems go away if there are no parameters. Nevertheless, the ?memoize? pattern *is* well-understood, and already implemented for Python as discussed earlier. It covers the ?cached property? as a degenerate (i.e. simpler) case. Making a specific decorator that *only* addresses ?cached property? is too narrow, IMO. I think a ?memoize? decorator (by whatever name) is the right level of generality to address this use case, and has the advantage of existing implementations. -- \ ?As I bit into the nectarine, it had a crisp juiciness about it | `\ that was very pleasurable - until I realized it wasn't a | _o__) nectarine at all, but A HUMAN HEAD!!? ?Jack Handey | Ben Finney From george.sakkis at gmail.com Fri May 1 02:16:11 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 30 Apr 2009 20:16:11 -0400 Subject: [Python-ideas] caching properties In-Reply-To: <87r5z9ogqm.fsf@benfinney.id.au> References: <20090430090004.57c2e51c@o> <9e754ef50904300536n35009806w4509e4f17da0ebc3@mail.gmail.com> <200904302356.48819.steve@pearwood.info> <20090430221900.6e4c0181@o> <91ad5bf80904301627s512a918cuaac35bc61c6947d8@mail.gmail.com> <87r5z9ogqm.fsf@benfinney.id.au> Message-ID: <91ad5bf80904301716x7a77e3dct9b15552526311068@mail.gmail.com> On Thu, Apr 30, 2009 at 7:37 PM, Ben Finney wrote: > George Sakkis > writes: > >> I think It's clear by now that caching in the general case is not >> trivial, both in terms of API and implementation. That makes the >> original request - caching properties only - more appealing since most >> problems go away if there are no parameters. > > Nevertheless, the ?memoize? pattern *is* well-understood, and already > implemented for Python as discussed earlier. It covers the ?cached > property? as a degenerate (i.e. simpler) case. Making a specific > decorator that *only* addresses ?cached property? is too narrow, IMO. > > I think a ?memoize? decorator (by whatever name) is the right level of > generality to address this use case, and has the advantage of existing > implementations. IMHO no existing implementation is good enough for addition to the standard library. Off the top of my head, some of the issues a general and robust implementation should address are: - Non hashable parameters. - Caching keys based on a subset of the provided parameters (or perhaps more general a key function). - Bounded-length caches. - Different expiration policies for bounded-length caches. - As good performance as possible given the previous constraints. Of course one can write a vanilla version in a few lines, and indeed searching for "memoize" in code.activestate.com returns no less than 125 results (including one I posted 4.5 years ago). I think the sheer number of the posted recipes is an indication that none of them has "solved" the problem for good, otherwise people would just reuse it instead of keep posting alternatives. George From aahz at pythoncraft.com Fri May 1 03:42:00 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 30 Apr 2009 18:42:00 -0700 Subject: [Python-ideas] caching properties In-Reply-To: <91ad5bf80904301716x7a77e3dct9b15552526311068@mail.gmail.com> References: <20090430090004.57c2e51c@o> <9e754ef50904300536n35009806w4509e4f17da0ebc3@mail.gmail.com> <200904302356.48819.steve@pearwood.info> <20090430221900.6e4c0181@o> <91ad5bf80904301627s512a918cuaac35bc61c6947d8@mail.gmail.com> <87r5z9ogqm.fsf@benfinney.id.au> <91ad5bf80904301716x7a77e3dct9b15552526311068@mail.gmail.com> Message-ID: <20090501014200.GA10335@panix.com> On Thu, Apr 30, 2009, George Sakkis wrote: > On Thu, Apr 30, 2009 at 7:37 PM, Ben Finney wrote: >> >> I think a memoize decorator (by whatever name) is the right level of >> generality to address this use case, and has the advantage of existing >> implementations. > > IMHO no existing implementation is good enough for addition to the > standard library. Off the top of my head, some of the issues a general > and robust implementation should address are: > - Non hashable parameters. > - Caching keys based on a subset of the provided parameters (or > perhaps more general a key function). > - Bounded-length caches. > - Different expiration policies for bounded-length caches. > - As good performance as possible given the previous constraints. > > Of course one can write a vanilla version in a few lines, and indeed > searching for "memoize" in code.activestate.com returns no less than > 125 results (including one I posted 4.5 years ago). I think the sheer > number of the posted recipes is an indication that none of them has > "solved" the problem for good, otherwise people would just reuse it > instead of keep posting alternatives. Then a full-blown PEP is required. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair From gagsl-py2 at yahoo.com.ar Fri May 1 03:42:20 2009 From: gagsl-py2 at yahoo.com.ar (gagsl-py2 at yahoo.com.ar) Date: Thu, 30 Apr 2009 18:42:20 -0700 (PDT) Subject: [Python-ideas] caching properties Message-ID: <615377.15553.qm@web32806.mail.mud.yahoo.com> En Thu, 30 Apr 2009 17:59:55 -0300, Jared Grubb escribi?: > The only thing I dislike is how many dictionary lookups are required in > order to return the value after it's been cached. I count 4 lookups > (object.prop, prop.__call__, self._cache, and self._cache[args]). These > add up, especially if object.prop could have returned the value > immediately without having to go through so much indirection (but this > is not currently possible) It is not possible if one insist on using property. But Oleg Broytmann's CachedAttribute class works fine -- it seems to me it didn't get the deserved attention. I'll repost a slightly modified version: class CachedAttribute(object): def __init__(self, method, name=None): self.method = method self.name = name or method.__name__ self.__doc__ = method.__doc__ def __get__(self, inst, cls): if inst is None: return self result = self.method(inst) setattr(inst, self.name, result) return result class Foo(object): @CachedAttribute def expensive(self): "docstring for the expensive property" print("computing value...") return 42 py> f = Foo() py> f.expensive computing value... 42 py> f.expensive 42 py> f.expensive = 43 py> f.expensive 43 py> del f.expensive py> f.expensive computing value... 42 py> f.expensive 42 py> Note that once the value is computed, it becomes an instance attribute; later accesses retrieve it directly, no indirection is involved. -- Gabriel Genellina Yahoo! Cocina Recetas pr?cticas y comida saludable http://ar.mujer.yahoo.com/cocina/ From steve at pearwood.info Fri May 1 04:18:05 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 1 May 2009 12:18:05 +1000 Subject: [Python-ideas] caching properties In-Reply-To: <91ad5bf80904301627s512a918cuaac35bc61c6947d8@mail.gmail.com> References: <20090430221900.6e4c0181@o> <91ad5bf80904301627s512a918cuaac35bc61c6947d8@mail.gmail.com> Message-ID: <200905011218.06007.steve@pearwood.info> On Fri, 1 May 2009 09:27:13 am George Sakkis wrote: > I think It's clear by now that caching in the general case is not > trivial, both in terms of API and implementation. That makes the > original request - caching properties only - more appealing since > most problems go away if there are no parameters. > > In the simplest case where cache expiration is not supported, > cachedproperty can be a 7-line decorator: [snip] In my opinion, properties aren't special enough to need a special cachedproperty() decorator. Especially not get-only properties that never expire, since they're just a function taking a single argument that returns a constant result. A generic cache that runs on any function should be sufficient, and more useful than one that is specific to properties. Properties are not the only application for caching expensive function calls that never change. -- Steven D'Aprano From steve at pearwood.info Fri May 1 04:20:10 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 1 May 2009 12:20:10 +1000 Subject: [Python-ideas] caching properties In-Reply-To: <91ad5bf80904301716x7a77e3dct9b15552526311068@mail.gmail.com> References: <87r5z9ogqm.fsf@benfinney.id.au> <91ad5bf80904301716x7a77e3dct9b15552526311068@mail.gmail.com> Message-ID: <200905011220.10467.steve@pearwood.info> On Fri, 1 May 2009 10:16:11 am George Sakkis wrote: > IMHO no existing implementation is good enough for addition to the > standard library. Off the top of my head, some of the issues a > general and robust implementation should address are: > - Non hashable parameters. > - Caching keys based on a subset of the provided parameters (or > perhaps more general a key function). > - Bounded-length caches. > - Different expiration policies for bounded-length caches. > - As good performance as possible given the previous constraints. > > Of course one can write a vanilla version in a few lines, and indeed > searching for "memoize" in code.activestate.com returns no less than > 125 results (including one I posted 4.5 years ago). I think the sheer > number of the posted recipes is an indication that none of them has > "solved" the problem for good, otherwise people would just reuse it > instead of keep posting alternatives. Or people just like re-inventing the wheel. By all means go ahead and write a PEP and develop an heavyweight "deal with everything including the kitchen sink" solution. But in the meantime, the standard library could do with a couple of nice, simple, lightweight memoize decorators. -- Steven D'Aprano From steve at pearwood.info Fri May 1 04:45:17 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 1 May 2009 12:45:17 +1000 Subject: [Python-ideas] caching properties In-Reply-To: References: Message-ID: <200905011245.17681.steve@pearwood.info> On Fri, 1 May 2009 06:59:55 am Jared Grubb wrote: > On 30 Apr 2009, at 12:22, Scott David Daniels wrote: > > This is slightly better (name change as in Antoine Pitrou's > > comment): > > > > class cached(object): > > > > def __init__(self, function): > > self._function = function > > self._cache = {} > > > > def __call__(self, *args): > > try: > > return self._cache[args] > > except KeyError: > > self._cache[args] = self._function(*args) > > return self._cache[args] > > > > def expire(self, *args): > > del self._cache[args] > > The only thing I dislike is how many dictionary lookups are required > in order to return the value after it's been cached. I count 4 > lookups (object.prop, prop.__call__, self._cache, and > self._cache[args]). These add up, especially if object.prop could > have returned the value immediately without having to go through so > much indirection (but this is not currently possible) But you shouldn't be comparing a cheap attribute requiring one key lookup to a cache requiring four lookups. You should be comparing an expensive function call to four lookups. If the function isn't expensive, there's no value in caching it. -- Steven D'Aprano From arnodel at googlemail.com Fri May 1 16:49:12 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 1 May 2009 15:49:12 +0100 Subject: [Python-ideas] caching properties In-Reply-To: References: <200904301357.08083.steve@pearwood.info> <20090430165309.4404f5d7@o> Message-ID: <9bfc700a0905010749n440d3758l4c7e4b8e8d669d5c@mail.gmail.com> 2009/4/30 Antoine Pitrou : > spir writes: >> >> now I wonder whether it's really worth it. Because usually it's a very simple >> thing to implement -- often two obvious lines of code -- and easy to > understand, even for someone who does >> not know the principle yet. > > Well, for one, a generic implementation may have to be thread-safe. > Also, while it's easy to implement, it's the kind of useful primitive - like > enumerate() and others - which is nice to have builtin in the language or the > standard library. Except that the implementation of enumerate() is not in debate (nor its interface). For caching it will probably be a different story. -- Arnaud From denis.spir at free.fr Sat May 2 14:18:44 2009 From: denis.spir at free.fr (spir) Date: Sat, 2 May 2009 14:18:44 +0200 Subject: [Python-ideas] =?utf-8?q?PL=C2=A0internationalisation?= Message-ID: <20090502141844.122350e4@o> Hello, There was thread about internationalisation of python (the language itself) in the case specific case of brasilian portuguese. I stepped on the following paper about a Perl dialect in latin! http://www.csse.monash.edu.au/~damian/papers/HTML/Perligata.html which I found great, especially because they explicitely address both question of why and how (very rare): Abstract This paper describes a Perl module -- Lingua::Romana::Perligata -- that makes it possible to write Perl programs in Latin. A plausible rationale for wanting to do such a thing is provided, along with a comprehensive overview of the syntax and semantics of Latinized Perl. The paper also explains the special source filtering and parsing techniques required to efficiently interpret a programming language in which the syntax is (largely) non-positional. Denis PS: This paper comes from a site mainly devoted to teaching of programming that also holds the following perl (sic): "Seven Deadly Sins of Introductory Programming Language Design" Abstract We discuss seven undesirable features common to many programming languages used to teach first-time programmers, and illustrate typical pedagogical difficulties which stem from them with examples drawn from the programming languages ABC, Ada, C, C++, Eiffel, Haskell, LISP, Modula 3, Pascal, Prolog, Scheme, and Turing. We propose seven language design (or selection) principles which may reduce the incidence of such undesirable features. http://www.csse.monash.edu.au/~damian/papers/PDF/SevenDeadlySins.pdf Actually the whole site is a golden mine for anybody, I guess, interested in PL design and/or pedagogy. ------ la vita e estrany From john.a.graham at gmail.com Sat May 2 22:27:47 2009 From: john.a.graham at gmail.com (John Graham) Date: Sat, 2 May 2009 15:27:47 -0500 Subject: [Python-ideas] A Continuations Compromise in Python Message-ID: Hi all It was suggested I post this idea to the python-ideas list. I apologize in advance for its length, I tried to be thorough :) Recently there's been some debate over the Tail-Call-Optimization and its use in Python. Some advocates coming from languages that support this optimization say that it makes recursive code far more efficient, and indeed allows some things to happen that could not otherwise occur without running out of stack space. The argument against, made most faithfully by the BDFL himself, is that looping behavior is in general easier to understand than recursion, and that implicitly optimizing code behind the scenes may confuse newcomers to the language. This would especially affect stack traces. Furthermore, Python is already an expressive language and there appear to be few use cases that truly demand TCO. Many of these arguments are strong, however, their strength can be used against them. Python is a teaching language, and proudly so, which is one of the reasons to keep it clean and simple. At the same time, recursion is an important part of Computer Science, software development and programming. Indeed, many people's first recursive functions nowadays are written in Python! Python is the perfect language to introduce these different kinds of constructs as the 'functional' paradigm is further absorbed by other mainstream languages in industry and academia. Other than it's potential to teach, Python's first class support of recursion, through an explicit mechanism to indicate 'continuation' style functions, is also undervalued in the use cases it more simply carries out compared to iterative implementations (never mind the performance benefit). The Actor model and message passing, state machine logic, and green thread schedulers are all potential uses. Similarly, many multi-threaded/multi-processor programs simplify their logic when passing continuations. This proposal is to extend the use of the keyword 'continue' as a way to explicitly invoke continuation style function calls, and through that mechanism, tail-call optimized recursion. The Zen of Python states that explicit is better than implicit, and this is the most major hang up with many proposed 'optimizations' that change system behavior. In many TCO proposals, the compiler is responsible for identifying recursive calls that can be optimized. In this proposal, continuations are achieved explicitly, and it is up to the programmer to ensure that they are proper tail calls, keeping all of the choice and power in the programmer's hands. This is best shown with a few examples. Currently, a 'continuation' style function might be written like this: def func(continuationFunc, *args): return continuationFunc(*args) The function is passed in as a first class object, then applied to some arguments and immidately returned. This is an important point about true tail-calls ? their values must immediately be returned. The proposed syntax extension would change the above function to the following similar function: def func(continuationFunc, *args): continue continuationFunc(*args) In this case, instead of the 'return' keyword, the 'continue' keyword is used to indicate the programmer wishes to transfer full control to the continuationFunction. Tail-call purity would be enforced, meaning that non-tail calls def func(continuationFunc, *args): continue 1 + continuationFunc(*args) would either cause a syntax error or throw a runtime exception. This serves as a learning opportunity for those new to programming, just as many of Python's uniquely elegant implementations of programming constructs have, yet also provide a powerful and expressive way to solve certain kinds of recursive problems. The double use of the 'continue' keyword is no accident. The current use of the 'continue' keyword makes a prime candidate for extension for a few reasons. First there are theoretical reasons. For one, the use of the English word 'continue' in the above constructs is intuitive and would not take much explaining to a newcomer as to what it means versus 'return'. More importantly, since it has been pointed out that many recursive structures can be implemented as imperative loops, it seems appropriate that the same keyword would mean equivalent/parallel things in the two styles. Using continue inside a loop ? bounce back up to the top of the loop and start with the next iteration, is synonymous with the proposed recursive use, to bounce back up the stack and immediately call the 'next' function. Indeed, trampoline style schedulers would use the continue keyword in its current form to implement continuations! These parallels between recursive and imperative loops help understanding the new proposed use. There are also some practical concerns. 'continue' as currently used is not that popular of a construct. When one needs to 'continue', one is glad it's there, but due to Python's expressive generator expressions, for-each style loops and inner functions, many use cases for 'continue' become more elegantly expressed in other ways. This means that, either way, a newcomers introduction to the uses of 'continue' aren't going to happen until he or she has a firm grasp of other parts of the language. Moreover, there's no rule saying that a student has to learn about all the uses of a keyword upon being first introduced to the keyword. A student would only inadvertently need to learn about continuation passing via 'continue' if they misunderstood the 'continue' statement while learning about loops, and somehow triggered the interpreter's wraith that they didn't define their continuation properly. There's also no hard and fast rule that a student would learn about complex loop control before they'd be interested in continuation passing, so it works both ways. 'continue' also currently has a very well defined, strict syntax. This is very important for any potential extensions, as it guarantees that this proposal is completely backwards compatible. The uses of the keyword in the examples are currently syntax errors, meaning they can't be in any current running code. The reuse of a keyword already in the language also guarantees there will be no naming collisions with older code. Using 'continue' in this way seems very 'Pythonic'. It flows naturally from the meaning of the keywords, and the extended syntax is still rather simple ? it must be a single callable. There are no unique rules or corner cases. It also mimics other return-control flow additions to the language, most notably the 'yield' construct, used for generators and co-routines. Yield is overloaded, in a sense, to have two different yet similar purposes. Overloading 'continue' in this way seems to naturally flow from the evolution of the language. Likewise, users are still free to intermix yields, returns and continues in functions and still have a very well defined, easy to understand behavior. Some have voiced concerns. One instance is the appearance of a 'continue' in a try/catch/finally block. try: continue func(*args) catch: #error handle This appears to be an implementation problem, but only until you were to expand the same block into an equivalent, old-style 'return code' error handling scheme. err = continue func(*args) if(err): #error handle else: return err In this form, it becomes apparent that this is an illegal use of continue, as 'func' is not a tail-call! Checking the result of a tail-call and controlling logic via that doesn't make any sense, and as such, we can without remorse or regret say 'no tail calls in try/catch/finally' blocks. Tail-calls are functional calls where there is absolutely no more processing to be done in the current stack frame. 'Clean up' logic or error handling in catch and finally blocks indicates there is more processing to be done, and as such, there simply is no well defined 'tail-call' in these circumstances. They ought to remain normal stack returns. 'Continue' can certainly propagate exceptions as well as return values, but those would be caught at whichever stack level initiated the tail-call loop. Another concern is the fact that stack traces disappear with these types of constructs. The counter argument is that the construct that they replace (in some instances) are loops, which themselves only have limited 'stack' trace information. If you crash in a loop, you don't necessarily get the iteration of the loop you were on, or any information from previous iterations. That is all lost. Likewise, a naive implementation of the proposed 'continue' function would also lose stack trace information ? however, it will do so in a way one can expect. Just as if one were to implement a catch block that swallowed every exception and reported nothing, one should be familiar with the behavior changes in the stack that 'continue' would produce. Thankfully, unlike some behind the scenes optimization, a 'continue' keyword clearly shows where tail-calls are made in this proposal. In a more advanced implementation, the clear use of 'continue' would allow the interpreter to annotate the stack trace, such that some information could easily be provided about current and historical program flow. There are some alternative designs for this problems that have been proposed. SJBrown has proposed a similar syntax, using the two keywords 'continue as' instead of simply 'continue'. This would further make clear to users unfamiliar with continuations that behavior is expected to be different here, and the English still flows elegantly. It does, however, reuse the 'as' keyword in an unfamiliar way. Other proposals followed the same logic, but introduced new keywords such as 'recurse' or 'tail'. These avoid any potential ambiguity with the 'continue' keyword, but are not backwards compatible. Furthermore, this proposal sees the actual English of the word 'continue' to be a strength in this case. Continuations and tail-calls are different, sometimes clearer, ways to implement advanced control flow. They allow the programmer a lot of power in a small expressive and elegant package. They are too frequently hidden behind the scenes, though, as the programmer relies on compiler identified 'optimization' spots. Introducing explicit continuations to Python would allow students learning programming and advanced users both an expressive, yet very easy to learn and familiar, construct. Extending the 'continue' keyword is the best candidate for this change, as its backwards compatible, parallels other similar extensions like yield, mimics the current use of the keyword, and is, in this author's opinion, 'Pythonic' ;) From lists at janc.be Sat May 2 23:35:29 2009 From: lists at janc.be (Jan Claeys) Date: Sat, 02 May 2009 23:35:29 +0200 Subject: [Python-ideas] keywording prohibited In-Reply-To: <20090423125354.GA59@panix.com> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> Message-ID: <1241300129.30081.38.camel@saeko.local> Op donderdag 23-04-2009 om 05:53 uur [tijdzone -0700], schreef Aahz: > How do non-keyword builtins bite? You can accidentally "re-define" or mask them (if you don't use a tool that warns for that). -- Jan Claeys From cmjohnson.mailinglist at gmail.com Sat May 2 23:47:35 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 2 May 2009 11:47:35 -1000 Subject: [Python-ideas] "yield from" -> "yield as"? Message-ID: <3bdda690905021447v736915c4g4647b5e99b158336@mail.gmail.com> John Graham wrote: > There are some alternative designs for this problems that have been > proposed. ?SJBrown has proposed a similar syntax, using the two > keywords 'continue as' instead of simply 'continue'. ?This would > further make clear to users unfamiliar with continuations that > behavior is expected to be different here, and the English still flows > elegantly. ?It does, however, reuse the 'as' keyword in an unfamiliar > way. I'm not sure what I think about redefining the continue keyword, but has anyone already proposed doing "yield as" instead of "yield from"? To me it make that wording makes a little more sense, "OK, you're not yielding as f anymore, now you've silently replaced yourself with g and it's 'as if' g is doing the yielding." "Yield from" makes it clear that all of the values of g will be yielded, but "yield as" makes it clear that things like .send(), etc. will be sent to g and not intercepted by f in any real way. My 2-cents on the bikeshed, -- Carl From gerald.britton at gmail.com Sat May 2 23:57:00 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 2 May 2009 17:57:00 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: Message-ID: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> First off, I think that you misrepresent Python by your characterization of it as a teaching language. To be sure, many CS programs _do_ teach Python as part of their curricula and use it to teach core topics. But Python is much more than that -- it is a serious language for serious applications. (If you're not sure about this, just ask Google, which has built large parts of its infrastructure on Python). More importantly, while I appreciate your idea from a theoretical perspective, I believe that your proposal would garner more attention if you would provide some real-world examples. If the expanded role for _continue_ were already implemented, what real problems would be easier to solve, or result in more elegant source code? I, for one, would like to see some "before" and "after" examples using the syntax on typical applications. On Sat, May 2, 2009 at 4:27 PM, John Graham wrote: > Hi all > > It was suggested I post this idea to the python-ideas list. ?I > apologize in advance for its length, I tried to be thorough :) > > Recently there's been some debate over the Tail-Call-Optimization and > its use in Python. ?Some advocates coming from languages that support > this optimization say that it makes recursive code far more efficient, > and indeed allows some things to happen that could not otherwise occur > without running out of stack space. > > The argument against, made most faithfully by the BDFL himself, is > that looping behavior is in general easier to understand than > recursion, and that implicitly optimizing code behind the scenes may > confuse newcomers to the language. ?This would especially affect stack > traces. ?Furthermore, Python is already an expressive language and > there appear to be few use cases that truly demand TCO. > > Many of these arguments are strong, however, their strength can be > used against them. ?Python is a teaching language, and proudly so, > which is one of the reasons to keep it clean and simple. ?At the same > time, recursion is an important part of Computer Science, software > development and programming. ?Indeed, many people's first recursive > functions nowadays are written in Python! ?Python is the perfect > language to introduce these different kinds of constructs as the > 'functional' paradigm is further absorbed by other mainstream > languages in industry and academia. > > Other than it's potential to teach, Python's first class support of > recursion, through an explicit mechanism to indicate 'continuation' > style functions, is also undervalued in the use cases it more simply > carries out compared to iterative implementations (never mind the > performance benefit). ?The Actor model and message passing, state > machine logic, and green thread schedulers are all potential uses. > Similarly, many multi-threaded/multi-processor programs simplify their > logic when passing continuations. > > This proposal is to extend the use of the keyword 'continue' as a way > to explicitly invoke continuation style function calls, and through > that mechanism, tail-call optimized recursion. ?The Zen of Python > states that explicit is better than implicit, and this is the most > major hang up with many proposed 'optimizations' that change system > behavior. ?In many TCO proposals, the compiler is responsible for > identifying recursive calls that can be optimized. ?In this proposal, > continuations are achieved explicitly, and it is up to the programmer > to ensure that they are proper tail calls, keeping all of the choice > and power in the programmer's hands. ?This is best shown with a few > examples. > > Currently, a 'continuation' style function might be written like this: > > def func(continuationFunc, *args): > ? ? ? ?return continuationFunc(*args) > > The function is passed in as a first class object, then applied to > some arguments and immidately returned. ?This is an important point > about true tail-calls ? their values must immediately be returned. > The proposed syntax extension would change the above function to the > following similar function: > > def func(continuationFunc, *args): > ? ? ? ?continue continuationFunc(*args) > > In this case, instead of the 'return' keyword, the 'continue' keyword > is used to indicate the programmer wishes to transfer full control to > the continuationFunction. ?Tail-call purity would be enforced, meaning > that non-tail calls > > def func(continuationFunc, *args): > ? ? ? ?continue 1 + continuationFunc(*args) > > would either cause a syntax error or throw a runtime exception. > > This serves as a learning opportunity for those new to programming, > just as many of Python's uniquely elegant implementations of > programming constructs have, yet also provide a powerful and > expressive way to solve certain kinds of recursive problems. > > The double use of the 'continue' keyword is no accident. ?The current > use of the 'continue' keyword makes a prime candidate for extension > for a few reasons. ?First there are theoretical reasons. ?For one, the > use of the English word 'continue' in the above constructs is > intuitive and would not take much explaining to a newcomer as to what > it means versus 'return'. > > More importantly, since it has been pointed out that many recursive > structures can be implemented as imperative loops, it seems > appropriate that the same keyword would mean equivalent/parallel > things in the two styles. ?Using continue inside a loop ? bounce back > up to the top of the loop and start with the next iteration, is > synonymous with the proposed recursive use, to bounce back up the > stack and immediately call the 'next' function. ?Indeed, trampoline > style schedulers would use the continue keyword in its current form to > implement continuations! ?These parallels between recursive and > imperative loops help understanding the new proposed use. > > There are also some practical concerns. ?'continue' as currently used > is not that popular of a construct. ?When one needs to 'continue', one > is glad it's there, but due to Python's expressive generator > expressions, for-each style loops and inner functions, many use cases > for 'continue' become more elegantly expressed in other ways. ?This > means that, either way, a newcomers introduction to the uses of > 'continue' aren't going to happen until he or she has a firm grasp of > other parts of the language. ?Moreover, there's no rule saying that a > student has to learn about all the uses of a keyword upon being first > introduced to the keyword. ?A student would only inadvertently need to > learn about continuation passing via 'continue' if they misunderstood > the 'continue' statement while learning about loops, and somehow > triggered the interpreter's wraith that they didn't define their > continuation properly. ?There's also no hard and fast rule that a > student would learn about complex loop control before they'd be > interested in continuation passing, so it works both ways. ?'continue' > also currently has a very well defined, strict syntax. ?This is very > important for any potential extensions, as it guarantees that this > proposal is completely backwards compatible. ?The uses of the keyword > in the examples are currently syntax errors, meaning they can't be in > any current running code. ?The reuse of a keyword already in the > language also guarantees there will be no naming collisions with older > code. > > Using 'continue' in this way seems very 'Pythonic'. ?It flows > naturally from the meaning of the keywords, and the extended syntax is > still rather simple ? it must be a single callable. ?There are no > unique rules or corner cases. ?It also mimics other return-control > flow additions to the language, most notably the 'yield' construct, > used for generators and co-routines. ?Yield is overloaded, in a sense, > to have two different yet similar purposes. ?Overloading 'continue' in > this way seems to naturally flow from the evolution of the language. > Likewise, users are still free to intermix yields, returns and > continues in functions and still have a very well defined, easy to > understand behavior. > > Some have voiced concerns. ?One instance is the appearance of a > 'continue' in a try/catch/finally block. > > ? ? ? ?try: > ? ? ? ? ? ? ? ?continue func(*args) > ? ? ? ?catch: > ? ? ? ? ? ? ? ?#error handle > > This appears to be an implementation problem, but only until you were > to expand the same block into an equivalent, old-style 'return code' > error handling scheme. > > ? ? ? ?err = continue func(*args) > ? ? ? ?if(err): > ? ? ? ? ? ? ? ?#error handle > ? ? ? ?else: > ? ? ? ? ? ? ? ?return err > > In this form, it becomes apparent that this is an illegal use of > continue, as 'func' is not a tail-call! ?Checking the result of a > tail-call and controlling logic via that doesn't make any sense, and > as such, we can without remorse or regret say 'no tail calls in > try/catch/finally' blocks. ?Tail-calls are functional calls where > there is absolutely no more processing to be done in the current stack > frame. ?'Clean up' logic or error handling in catch and finally blocks > indicates there is more processing to be done, and as such, there > simply is no well defined 'tail-call' in these circumstances. ?They > ought to remain normal stack returns. > > 'Continue' can certainly propagate exceptions as well as return > values, but those would be caught at whichever stack level initiated > the tail-call loop. > > Another concern is the fact that stack traces disappear with these > types of constructs. ?The counter argument is that the construct that > they replace (in some instances) are loops, which themselves only have > limited 'stack' trace information. ?If you crash in a loop, you don't > necessarily get the iteration of the loop you were on, or any > information from previous iterations. ?That is all lost. ?Likewise, a > naive implementation of the proposed 'continue' function would also > lose stack trace information ? however, it will do so in a way one can > expect. ?Just as if one were to implement a catch block that swallowed > every exception and reported nothing, one should be familiar with the > behavior changes in the stack that 'continue' would produce. > Thankfully, unlike some behind the scenes optimization, a 'continue' > keyword clearly shows where tail-calls are made in this proposal. ?In > a more advanced implementation, the clear use of 'continue' would > allow the interpreter to annotate the stack trace, such that some > information could easily be provided about current and historical > program flow. > > There are some alternative designs for this problems that have been > proposed. ?SJBrown has proposed a similar syntax, using the two > keywords 'continue as' instead of simply 'continue'. ?This would > further make clear to users unfamiliar with continuations that > behavior is expected to be different here, and the English still flows > elegantly. ?It does, however, reuse the 'as' keyword in an unfamiliar > way. ?Other proposals followed the same logic, but introduced new > keywords such as 'recurse' or 'tail'. ?These avoid any potential > ambiguity with the 'continue' keyword, but are not backwards > compatible. ?Furthermore, this proposal sees the actual English of the > word 'continue' to be a strength in this case. > > Continuations and tail-calls are different, sometimes clearer, ways to > implement advanced control flow. ?They allow the programmer a lot of > power in a small expressive and elegant package. ?They are too > frequently hidden behind the scenes, though, as the programmer relies > on compiler identified 'optimization' spots. ?Introducing explicit > continuations to Python would allow students learning programming and > advanced users both an expressive, yet very easy to learn and > familiar, construct. ?Extending the 'continue' keyword is the best > candidate for this change, as its backwards compatible, parallels > other similar extensions like yield, mimics the current use of the > keyword, and is, in this author's opinion, 'Pythonic' ;) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From aahz at pythoncraft.com Sun May 3 00:51:53 2009 From: aahz at pythoncraft.com (Aahz) Date: Sat, 2 May 2009 15:51:53 -0700 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> Message-ID: <20090502225153.GA5084@panix.com> On Sat, May 02, 2009, Gerald Britton wrote: > > First off, I think that you misrepresent Python by your > characterization of it as a teaching language. To be sure, many CS > programs _do_ teach Python as part of their curricula and use it to > teach core topics. But Python is much more than that -- it is a > serious language for serious applications. (If you're not sure about > this, just ask Google, which has built large parts of its > infrastructure on Python). I'm reasonaly sure John Graham had no intention of characterizing Python as "only" a teaching language; it surely is the case that Python *is* intended as a teaching language: http://www.python.org/doc/essays/cp4e.html -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Typing is cheap. Thinking is expensive." --Roy Smith From aahz at pythoncraft.com Sun May 3 00:57:35 2009 From: aahz at pythoncraft.com (Aahz) Date: Sat, 2 May 2009 15:57:35 -0700 Subject: [Python-ideas] keywording prohibited In-Reply-To: <1241300129.30081.38.camel@saeko.local> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> Message-ID: <20090502225735.GB5084@panix.com> On Sat, May 02, 2009, Jan Claeys wrote: > Op donderdag 23-04-2009 om 05:53 uur [tijdzone -0700], schreef Aahz: >> >> How do non-keyword builtins bite? > > You can accidentally "re-define" or mask them (if you don't use a tool > that warns for that). Given how long it has been since the original post, I think it's unfair of you to cut this much context. What Denis Spir originally wrote was, "On the contrary: non-keyword builtins bite painfully!" and my rejoinder was written because of the extreme language. Your issue is a valid one, but I have seen no cases where I would call it "painful" contrasted with the pain of being unable to use a keyword anywhere as a name or attribute. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Typing is cheap. Thinking is expensive." --Roy Smith From rhamph at gmail.com Sun May 3 01:22:17 2009 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 2 May 2009 17:22:17 -0600 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: Message-ID: On Sat, May 2, 2009 at 2:27 PM, John Graham wrote: > Hi all > > It was suggested I post this idea to the python-ideas list. ?I > apologize in advance for its length, I tried to be thorough :) > > Recently there's been some debate over the Tail-Call-Optimization and > its use in Python. ?Some advocates coming from languages that support > this optimization say that it makes recursive code far more efficient, Premature optimization, irrelevant. > and indeed allows some things to happen that could not otherwise occur > without running out of stack space. Use a trampoline. It's the same thing you propose, only it uses existing syntax. It's got a few rough edges, but they're quite manageable, and in no way justify the vast bikeshed this issue has garnered. -- Adam Olsen, aka Rhamphoryncus From john.a.graham at gmail.com Sun May 3 01:55:12 2009 From: john.a.graham at gmail.com (John Graham) Date: Sat, 2 May 2009 18:55:12 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> Message-ID: Thanks Gerald. I did not mean to imply Python was not an industrial strength language at all. In fact, I wasn't introduced to it at school, but at work. I just know, like you pointed out, that Python's simple, elegant syntax is there for a reason. Namely, because it's easy to teach. One of the main criticisms of TCO, and continuations that might take advantage of it, was that it added too much complexity in its 'hidden' form, which would confuse newcomers to the language. This is an important criticism, which I hope using an explicit 'keyword' would ameliorate. Here's an example an anecdote (since the example code would take to much space...) that has been asked for. Both show the general pattern (which is now solved with trampoline functions) exists in multiple contexts, and can be simplified via the use of TCO, which can be made explicit and correct via the introduction of a new control keyword ('continue') to invoke it. This was a state machine implementation proposed by someone playing with the syntax def state1(self): do stuff on self if cond: continue self.state2() else: continue self.state1() def state2(self): ... The response from someone critical of the idea was to just use a trampoline and this style: def state1(self): do stuff on self if cond: return state2 else: return state1 For a second example, I can only offer an anecdote. I got interested in this style after coding up an 'active object' in C++, which is more or less all the stuff you have to 'bolt on' to a language without the Actor model to simulate actors. In python, you would get a little help from class decorators, but (and here's why I leave out the code because it'd just be too much) for the most part, I had to involve parts from threads/processes, locks/shared memory, message queues, and method wrappers. All just to get message passing. Basically speaking, the threads were used as a main loop - again, trampolining - across anything that came into the threadsafe message queue. With 'continue' implying a continuation + TCO, actors boil down to little more than class Actor: def method1(self, messageBack): continue messageBack() The pattern here, basically, that continue eliminates, is the constant referral to 'just use a trampoline function'. To me, language constructs exist to codify certain patterns, similar to the way list comprehensions captured a lot of what was previously done in for loops. Insofar as complexity is concerned, I feel like I'd have an easier time explaining to someone what a continuation was versus a normal return, rather than what a trampoline function was. All of the criticisms that have been levied against TCO can also be levied against trampolines, as they're going to just as likely not keep stack debug information. And while I've tried to stay away from the efficiency use case, that can't be ignored. Recursive data structures, mutually recursive functions and the messages I described above all would benefit from the efficiency of the optimized tail call. Guido implied one of the main criticisms of the tail call optimization was that it was done without the users knowledge, behind the scenes, by the compiler. I think many have always acknowledged that TCO was, in fact, an optimization, it was just the complexity that went with a 'hidden' optimization. Making it explicit using the 'continue' keyword acknowledges that criticism. So, I suppose if you wanted more examples of where 'continue' would be better than the current implementation, they are all the same examples of when the hidden TCO was applied in other languages. The use case is there, its just that in Python, we can make sure the User is aware - not to mention, we can enforce correctness. If a recursive call isn't tail recursive in another language that implements TCO, it just gets silently converted to a normal call which might blow the stack. Differentiating between continue and return in Python will enforce proper tail calls, not silently 'default' to something the user didn't want. On Sat, May 2, 2009 at 4:57 PM, Gerald Britton wrote: > First off, I think that you misrepresent Python by your > characterization of it as a teaching language. ?To be sure, many CS > programs _do_ teach Python as part of their curricula and use it to > teach core topics. ?But Python is much more than that -- it is a > serious language for serious applications. ?(If you're not sure about > this, just ask Google, which has built large parts of its > infrastructure on Python). > > More importantly, while I appreciate your idea from a theoretical > perspective, I believe that your proposal would garner more attention > if you would provide some real-world examples. ?If the expanded role > for _continue_ were already implemented, what real problems would be > easier to solve, or result in more elegant source code? ?I, for one, > would like to see some "before" and "after" examples using the syntax > on typical applications. > > On Sat, May 2, 2009 at 4:27 PM, John Graham wrote: >> Hi all >> >> It was suggested I post this idea to the python-ideas list. ?I >> apologize in advance for its length, I tried to be thorough :) >> >> Recently there's been some debate over the Tail-Call-Optimization and >> its use in Python. ?Some advocates coming from languages that support >> this optimization say that it makes recursive code far more efficient, >> and indeed allows some things to happen that could not otherwise occur >> without running out of stack space. >> >> The argument against, made most faithfully by the BDFL himself, is >> that looping behavior is in general easier to understand than >> recursion, and that implicitly optimizing code behind the scenes may >> confuse newcomers to the language. ?This would especially affect stack >> traces. ?Furthermore, Python is already an expressive language and >> there appear to be few use cases that truly demand TCO. >> >> Many of these arguments are strong, however, their strength can be >> used against them. ?Python is a teaching language, and proudly so, >> which is one of the reasons to keep it clean and simple. ?At the same >> time, recursion is an important part of Computer Science, software >> development and programming. ?Indeed, many people's first recursive >> functions nowadays are written in Python! ?Python is the perfect >> language to introduce these different kinds of constructs as the >> 'functional' paradigm is further absorbed by other mainstream >> languages in industry and academia. >> >> Other than it's potential to teach, Python's first class support of >> recursion, through an explicit mechanism to indicate 'continuation' >> style functions, is also undervalued in the use cases it more simply >> carries out compared to iterative implementations (never mind the >> performance benefit). ?The Actor model and message passing, state >> machine logic, and green thread schedulers are all potential uses. >> Similarly, many multi-threaded/multi-processor programs simplify their >> logic when passing continuations. >> >> This proposal is to extend the use of the keyword 'continue' as a way >> to explicitly invoke continuation style function calls, and through >> that mechanism, tail-call optimized recursion. ?The Zen of Python >> states that explicit is better than implicit, and this is the most >> major hang up with many proposed 'optimizations' that change system >> behavior. ?In many TCO proposals, the compiler is responsible for >> identifying recursive calls that can be optimized. ?In this proposal, >> continuations are achieved explicitly, and it is up to the programmer >> to ensure that they are proper tail calls, keeping all of the choice >> and power in the programmer's hands. ?This is best shown with a few >> examples. >> >> Currently, a 'continuation' style function might be written like this: >> >> def func(continuationFunc, *args): >> ? ? ? ?return continuationFunc(*args) >> >> The function is passed in as a first class object, then applied to >> some arguments and immidately returned. ?This is an important point >> about true tail-calls ? their values must immediately be returned. >> The proposed syntax extension would change the above function to the >> following similar function: >> >> def func(continuationFunc, *args): >> ? ? ? ?continue continuationFunc(*args) >> >> In this case, instead of the 'return' keyword, the 'continue' keyword >> is used to indicate the programmer wishes to transfer full control to >> the continuationFunction. ?Tail-call purity would be enforced, meaning >> that non-tail calls >> >> def func(continuationFunc, *args): >> ? ? ? ?continue 1 + continuationFunc(*args) >> >> would either cause a syntax error or throw a runtime exception. >> >> This serves as a learning opportunity for those new to programming, >> just as many of Python's uniquely elegant implementations of >> programming constructs have, yet also provide a powerful and >> expressive way to solve certain kinds of recursive problems. >> >> The double use of the 'continue' keyword is no accident. ?The current >> use of the 'continue' keyword makes a prime candidate for extension >> for a few reasons. ?First there are theoretical reasons. ?For one, the >> use of the English word 'continue' in the above constructs is >> intuitive and would not take much explaining to a newcomer as to what >> it means versus 'return'. >> >> More importantly, since it has been pointed out that many recursive >> structures can be implemented as imperative loops, it seems >> appropriate that the same keyword would mean equivalent/parallel >> things in the two styles. ?Using continue inside a loop ? bounce back >> up to the top of the loop and start with the next iteration, is >> synonymous with the proposed recursive use, to bounce back up the >> stack and immediately call the 'next' function. ?Indeed, trampoline >> style schedulers would use the continue keyword in its current form to >> implement continuations! ?These parallels between recursive and >> imperative loops help understanding the new proposed use. >> >> There are also some practical concerns. ?'continue' as currently used >> is not that popular of a construct. ?When one needs to 'continue', one >> is glad it's there, but due to Python's expressive generator >> expressions, for-each style loops and inner functions, many use cases >> for 'continue' become more elegantly expressed in other ways. ?This >> means that, either way, a newcomers introduction to the uses of >> 'continue' aren't going to happen until he or she has a firm grasp of >> other parts of the language. ?Moreover, there's no rule saying that a >> student has to learn about all the uses of a keyword upon being first >> introduced to the keyword. ?A student would only inadvertently need to >> learn about continuation passing via 'continue' if they misunderstood >> the 'continue' statement while learning about loops, and somehow >> triggered the interpreter's wraith that they didn't define their >> continuation properly. ?There's also no hard and fast rule that a >> student would learn about complex loop control before they'd be >> interested in continuation passing, so it works both ways. ?'continue' >> also currently has a very well defined, strict syntax. ?This is very >> important for any potential extensions, as it guarantees that this >> proposal is completely backwards compatible. ?The uses of the keyword >> in the examples are currently syntax errors, meaning they can't be in >> any current running code. ?The reuse of a keyword already in the >> language also guarantees there will be no naming collisions with older >> code. >> >> Using 'continue' in this way seems very 'Pythonic'. ?It flows >> naturally from the meaning of the keywords, and the extended syntax is >> still rather simple ? it must be a single callable. ?There are no >> unique rules or corner cases. ?It also mimics other return-control >> flow additions to the language, most notably the 'yield' construct, >> used for generators and co-routines. ?Yield is overloaded, in a sense, >> to have two different yet similar purposes. ?Overloading 'continue' in >> this way seems to naturally flow from the evolution of the language. >> Likewise, users are still free to intermix yields, returns and >> continues in functions and still have a very well defined, easy to >> understand behavior. >> >> Some have voiced concerns. ?One instance is the appearance of a >> 'continue' in a try/catch/finally block. >> >> ? ? ? ?try: >> ? ? ? ? ? ? ? ?continue func(*args) >> ? ? ? ?catch: >> ? ? ? ? ? ? ? ?#error handle >> >> This appears to be an implementation problem, but only until you were >> to expand the same block into an equivalent, old-style 'return code' >> error handling scheme. >> >> ? ? ? ?err = continue func(*args) >> ? ? ? ?if(err): >> ? ? ? ? ? ? ? ?#error handle >> ? ? ? ?else: >> ? ? ? ? ? ? ? ?return err >> >> In this form, it becomes apparent that this is an illegal use of >> continue, as 'func' is not a tail-call! ?Checking the result of a >> tail-call and controlling logic via that doesn't make any sense, and >> as such, we can without remorse or regret say 'no tail calls in >> try/catch/finally' blocks. ?Tail-calls are functional calls where >> there is absolutely no more processing to be done in the current stack >> frame. ?'Clean up' logic or error handling in catch and finally blocks >> indicates there is more processing to be done, and as such, there >> simply is no well defined 'tail-call' in these circumstances. ?They >> ought to remain normal stack returns. >> >> 'Continue' can certainly propagate exceptions as well as return >> values, but those would be caught at whichever stack level initiated >> the tail-call loop. >> >> Another concern is the fact that stack traces disappear with these >> types of constructs. ?The counter argument is that the construct that >> they replace (in some instances) are loops, which themselves only have >> limited 'stack' trace information. ?If you crash in a loop, you don't >> necessarily get the iteration of the loop you were on, or any >> information from previous iterations. ?That is all lost. ?Likewise, a >> naive implementation of the proposed 'continue' function would also >> lose stack trace information ? however, it will do so in a way one can >> expect. ?Just as if one were to implement a catch block that swallowed >> every exception and reported nothing, one should be familiar with the >> behavior changes in the stack that 'continue' would produce. >> Thankfully, unlike some behind the scenes optimization, a 'continue' >> keyword clearly shows where tail-calls are made in this proposal. ?In >> a more advanced implementation, the clear use of 'continue' would >> allow the interpreter to annotate the stack trace, such that some >> information could easily be provided about current and historical >> program flow. >> >> There are some alternative designs for this problems that have been >> proposed. ?SJBrown has proposed a similar syntax, using the two >> keywords 'continue as' instead of simply 'continue'. ?This would >> further make clear to users unfamiliar with continuations that >> behavior is expected to be different here, and the English still flows >> elegantly. ?It does, however, reuse the 'as' keyword in an unfamiliar >> way. ?Other proposals followed the same logic, but introduced new >> keywords such as 'recurse' or 'tail'. ?These avoid any potential >> ambiguity with the 'continue' keyword, but are not backwards >> compatible. ?Furthermore, this proposal sees the actual English of the >> word 'continue' to be a strength in this case. >> >> Continuations and tail-calls are different, sometimes clearer, ways to >> implement advanced control flow. ?They allow the programmer a lot of >> power in a small expressive and elegant package. ?They are too >> frequently hidden behind the scenes, though, as the programmer relies >> on compiler identified 'optimization' spots. ?Introducing explicit >> continuations to Python would allow students learning programming and >> advanced users both an expressive, yet very easy to learn and >> familiar, construct. ?Extending the 'continue' keyword is the best >> candidate for this change, as its backwards compatible, parallels >> other similar extensions like yield, mimics the current use of the >> keyword, and is, in this author's opinion, 'Pythonic' ;) >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > > > -- > Gerald Britton > From john.a.graham at gmail.com Sun May 3 02:02:13 2009 From: john.a.graham at gmail.com (John Graham) Date: Sat, 2 May 2009 19:02:13 -0500 Subject: [Python-ideas] "yield from" -> "yield as"? In-Reply-To: <3bdda690905021447v736915c4g4647b5e99b158336@mail.gmail.com> References: <3bdda690905021447v736915c4g4647b5e99b158336@mail.gmail.com> Message-ID: Just wanted to politically come out as in favor of painting the bikeshed any damn color, so long as we build it :) In other words, I'm just as supportive of another 'yield' construct as I am 'continue', if it were to serve the same role that's proposed. The main thrust of the 'continue' argument was that many of the criticisms of allowing TCO into the language disappear if we make it explicit, so the user doesn't have to guess. On Sat, May 2, 2009 at 4:47 PM, Carl Johnson wrote: > John Graham wrote: > >> There are some alternative designs for this problems that have been >> proposed. ?SJBrown has proposed a similar syntax, using the two >> keywords 'continue as' instead of simply 'continue'. ?This would >> further make clear to users unfamiliar with continuations that >> behavior is expected to be different here, and the English still flows >> elegantly. ?It does, however, reuse the 'as' keyword in an unfamiliar >> way. > > I'm not sure what I think about redefining the continue keyword, but > has anyone already proposed doing "yield as" instead of "yield from"? > To me it make that wording makes a little more sense, "OK, you're not > yielding as f anymore, now you've silently replaced yourself with g > and it's 'as if' g is doing the yielding." "Yield from" makes it clear > that all of the values of g will be yielded, but "yield as" makes it > clear that things like .send(), etc. will be sent to g and not > intercepted by f in any real way. > > My 2-cents on the bikeshed, > > -- Carl > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From ben+python at benfinney.id.au Sun May 3 02:07:41 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sun, 03 May 2009 10:07:41 +1000 Subject: [Python-ideas] =?utf-8?q?PL=C2=A0internationalisation?= References: <20090502141844.122350e4@o> Message-ID: <87ws8zkq0i.fsf@benfinney.id.au> spir writes: > There was thread about internationalisation of python (the language > itself) in the case specific case of brasilian portuguese. > > I stepped on the following paper about a Perl dialect in latin! [?] Denis, this seems less suited for ?python-ideas? than for the main Python discussion forum. Could you please consider taking topics like this to in future? -- \ ?Instead of a trap door, what about a trap window? The guy | `\ looks out it, and if he leans too far, he falls out. Wait. I | _o__) guess that's like a regular window.? ?Jack Handey | Ben Finney From cs at zip.com.au Sun May 3 02:46:31 2009 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 3 May 2009 10:46:31 +1000 Subject: [Python-ideas] "yield from" -> "yield as"? In-Reply-To: <3bdda690905021447v736915c4g4647b5e99b158336@mail.gmail.com> Message-ID: <20090503004631.GA31730@cskk.homeip.net> On 02May2009 11:47, Carl Johnson wrote: | John Graham wrote: | > There are some alternative designs for this problems that have been | > proposed. ?SJBrown has proposed a similar syntax, using the two | > keywords 'continue as' instead of simply 'continue'. ?This would | > further make clear to users unfamiliar with continuations that | > behavior is expected to be different here, and the English still flows | > elegantly. ?It does, however, reuse the 'as' keyword in an unfamiliar | > way. | | I'm not sure what I think about redefining the continue keyword, but | has anyone already proposed doing "yield as" instead of "yield from"? | To me it make that wording makes a little more sense, "OK, you're not | yielding as f anymore, now you've silently replaced yourself with g | and it's 'as if' g is doing the yielding." "Yield from" makes it clear | that all of the values of g will be yielded, but "yield as" makes it | clear that things like .send(), etc. will be sent to g and not | intercepted by f in any real way. I'm a little uncomfortable with yield as. We've got: import foo as bar from foo import zot as bah with fred() as bah: ... All of these place a new name "bah" into the namespace for further use. "yield as" doesn't, but reads to me like it should. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ From the back of the research lab, the sound of a large metallic pile of loose objects collapsing, accompanied by a loud "Aaaaiiieeyyrrgghhh!!" from George. A few seconds of silence, then: "I'll have to call you back." From denis.spir at free.fr Sun May 3 03:16:33 2009 From: denis.spir at free.fr (spir) Date: Sun, 3 May 2009 03:16:33 +0200 Subject: [Python-ideas] keywording prohibited In-Reply-To: <20090502225735.GB5084@panix.com> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> Message-ID: <20090503031633.3d1663a9@o> Le Sat, 2 May 2009 15:57:35 -0700, Aahz s'exprima ainsi: > >> How do non-keyword builtins bite? > > > > You can accidentally "re-define" or mask them (if you don't use a tool > > that warns for that). > > Given how long it has been since the original post, I think it's unfair > of you to cut this much context. Sorry for this. > What Denis Spir originally wrote was, > "On the contrary: non-keyword builtins bite painfully!" and my rejoinder > was written because of the extreme language. Still, Jan's reply is the right issue I had in mind. > Your issue is a valid one, but I have seen no cases where I would call > it "painful" contrasted with the pain of being unable to use a keyword > anywhere as a name or attribute. Depends on your pov. I consider the _possibility_ of using 'list', 'range' or 'type' as a name for totally custom thing, without even a warning, an issue, not a wishable feature. It _invents_ very hard to diagnose bugs. Painful they are. I cannot even figure out a debugging process that would point to the real source of the problem -- except for a sudden "eureka". Actually, the issue is that precisely the non-protected build-in words are those so obvious variable names for data or funcs. I have (painfully) learnt a reflex to use 'ranj' for range, 'Seq' for list, 'typ' for type, etc... But maybe its me and I'm the only one ;-) [There is an (unsatisfying) solution with some editors: just add all those word to the keyword set so that they will jump at your eyes when you pretend use them... But they also look like keywords when safely used as attributes. I would prefere a python warning mode switch.] Strangely enough, I cannot imagine a use case for 'for' or 'in' or 'while' as names. while these ones are protected (indeed, I don't mean they shouldn't). Denis ------ la vita e estrany From adam at atlas.st Sun May 3 04:28:47 2009 From: adam at atlas.st (Adam Atlas) Date: Sat, 2 May 2009 22:28:47 -0400 Subject: [Python-ideas] keywording prohibited In-Reply-To: <20090503031633.3d1663a9@o> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> Message-ID: <83CDA82B-E353-4A0C-B8D0-E99AB1A525F7@atlas.st> On 2 May 2009, at 21:16, spir wrote: > I have (painfully) learnt a reflex to use 'ranj' for range, 'Seq' > for list, 'typ' for type, etc... > But maybe its me and I'm the only one ;-) Isn't the convention to suffix a name with an underscore when it would clash with a builtin? (range_, list_, type_, etc.) I'm not sure how much I like that stylistically, but I've seen it used a lot. (I think in some cases there are better alternatives -- e.g. instead of naming a variable "seq" or "list_", I'd have the name specify what it's a list *of*.) From stephen at xemacs.org Sun May 3 07:45:08 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 03 May 2009 14:45:08 +0900 Subject: [Python-ideas] keywording prohibited In-Reply-To: <20090503031633.3d1663a9@o> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> Message-ID: <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> spir writes: > Depends on your pov. I consider the _possibility_ of using 'list', > 'range' or 'type' as a name for totally custom thing, without even > a warning, an issue, not a wishable feature. It _invents_ very hard > to diagnose bugs. If you want warnings, use pylint. It catches all of those. On the other hand, Python's even-handed treatment of builtins, standard library identifiers, and user-defined identifiers makes for a great simplification in the rules for the language, and allows for introspective usage: # Here's something you can do for fun on a lazy Sunday afternoon. the_artist_formerly_known_as_list = list def list(thing): print "Somebody called list!" return the_artist_formerly_known_as_list(thing) For most of us, the gain in simplicity and power is well worth the pain. > Strangely enough, I cannot imagine a use case for 'for' or 'in' or > 'while' as names. while these ones are protected (indeed, I don't > mean they shouldn't). Dunno about the other two, but in C I used to use the equivalent of in = open("mind", "r") text = in.readlines() all the time. From cmjohnson.mailinglist at gmail.com Sun May 3 08:44:27 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 2 May 2009 20:44:27 -1000 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> Message-ID: <3bdda690905022344g5329cfb0ld8952c03f0cecf34@mail.gmail.com> I have a question about the implementation of "yield from." I recall hearing some talk about optimizing the stack in "yield from," but I didn't catch all of the details. I take it that there will be some steps taken to ensure that yield from's yield from their inner most yielder without having to touch base at all the objects in between whoever is asking for the value and whoever is giving it. That being the case, will this example implementation of a linked list still blow the stack for lists bigger than 1,000 items or not? class LinkedList: def __iter__(self): yield self.value if self.link: yield from self.link If it does still blow up the stack, then it's no big deal, but if this won't blow the stack up anymore, then it seems like there's very little difference between the kind of recursive programming invited by "yield from" and the kind of recursive programming invited by "continue object". If you hate reading TCO code (and I take this to be the #1 objection to adding TCO to Python, though there are also more technical reasons), you're still going to get it, only using "yield from" instead of "continue". So in that case, "continue" and "yield from" should be thought of as a pair of stack optimizers, one for functions and one for generators. -- Carl From steve at pearwood.info Sun May 3 08:59:35 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 3 May 2009 16:59:35 +1000 Subject: [Python-ideas] keywording prohibited In-Reply-To: <1241300129.30081.38.camel@saeko.local> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> Message-ID: <200905031659.36155.steve@pearwood.info> On Sun, 3 May 2009 07:35:29 am Jan Claeys wrote: > Op donderdag 23-04-2009 om 05:53 uur [tijdzone -0700], schreef Aahz: > > How do non-keyword builtins bite? > > You can accidentally "re-define" or mask them (if you don't use a > tool that warns for that). Cross out "accidentally", and the ability to shadow built-ins is a feature, not a bug. Anything that you do accidentally is a problem. You can accidentally shadow a module, or one of your own functions or names. And while Python has a rich set of built-ins, it isn't so many that learning them is a burden. I would expect that the average newbie would accidentally shadow a built-in once or twice, then never do it again. (Especially if they hang around comp.lang.python and get yelled at every time they do it.) -- Steven D'Aprano From steve at pearwood.info Sun May 3 09:02:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 3 May 2009 17:02:03 +1000 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: Message-ID: <200905031702.03363.steve@pearwood.info> On Sun, 3 May 2009 09:22:17 am Adam Olsen wrote: > On Sat, May 2, 2009 at 2:27 PM, John Graham wrote: > > Hi all > > > > It was suggested I post this idea to the python-ideas list. ?I > > apologize in advance for its length, I tried to be thorough :) > > > > Recently there's been some debate over the Tail-Call-Optimization > > and its use in Python. ?Some advocates coming from languages that > > support this optimization say that it makes recursive code far more > > efficient, > > Premature optimization, irrelevant. It's hardly "premature" to notice that recursive code in Python is significantly slower and less efficient than in other languages. This is a known problem, or at least issue, since some people consider that the solution (tail-recursion optimization) is worse that the problem. > > and indeed allows some things to happen that could not otherwise > > occur without running out of stack space. > > Use a trampoline. Up until a month or so ago, I'd never heard of the term "trampoline" (apart from the thing you jump up and down on), and I still don't know what it means. Checking Wikipedia, I see that in computing, trampoline has *eleven* definitions. Which one do you mean? > It's the same thing you propose, only it uses > existing syntax. It's got a few rough edges, but they're quite > manageable, and in no way justify the vast bikeshed this issue has > garnered. For the record, I find the OP's idea dubious. I don't like having yet another way of spelling "exit this function and return this result": return, yield, and now (proposed) continue. So a *very* tentative -0 on the suggestion. Maybe +0. Just tossing a wild idea out there... is it conceivable to build a decorator that optimizes a tail-call recursive function? Then it becomes a matter of explicit programmer choice whether or not to do so, with no new syntax. You would have the choice of: * write a function as tail-recursion because it is most expressive and simple algorithm, and live with the inefficiency (which as Adam points out, may not be a problem in practice); * manually re-write it as a for-loop, exchanging simplicity for runtime efficiency; or * decorate the recursive function, giving up some debugging information for efficiency, but keeping the simplicity of the tail-recursive form. I don't know whether such a thing is even possible, let alone how to go about doing such a thing, but if it did exist, I'd use it. -- Steven D'Aprano From steve at pearwood.info Sun May 3 09:03:48 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 3 May 2009 17:03:48 +1000 Subject: [Python-ideas] keywording prohibited In-Reply-To: <83CDA82B-E353-4A0C-B8D0-E99AB1A525F7@atlas.st> References: <20090423094835.19fdf337@o> <20090503031633.3d1663a9@o> <83CDA82B-E353-4A0C-B8D0-E99AB1A525F7@atlas.st> Message-ID: <200905031703.48677.steve@pearwood.info> On Sun, 3 May 2009 12:28:47 pm Adam Atlas wrote: > Isn't the convention to suffix a name with an underscore when it > would clash with a builtin? (range_, list_, type_, etc.) That's one convention. Another is to name things alist, atype, astr,. etc. If you need two of them, the obvious extension is blist, btype... When writing *small* generic methods or functions, I'm also fond of using L for list, x and y for floats, n or i for ints, and similar. This needs to be handled with care, for obvious reasons. > I'm not sure how much I like that stylistically, but I've seen it > used a lot. (I think in some cases there are better alternatives -- > e.g. instead of naming a variable "seq" or "list_", I'd have the name > specify what it's a list *of*.) Well, this is Python, and we use duck-typing, so everything would need to be list_of_string_like_instances or similar :) But seriously, I tend to use plurals for that. If I have an argument that takes a collection of widgets, say, I call it "widgets", and write code like this: for widget in widgets: whatever() I don't think there's much to be gained by calling it "list_of_widgets" unless it really needs to be a list, and not any other collection type. -- Steven D'Aprano From arnodel at googlemail.com Sun May 3 09:10:26 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 3 May 2009 08:10:26 +0100 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <3bdda690905022344g5329cfb0ld8952c03f0cecf34@mail.gmail.com> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <3bdda690905022344g5329cfb0ld8952c03f0cecf34@mail.gmail.com> Message-ID: <4BD79B6B-259D-44F8-BB98-C2932D7A8C80@googlemail.com> On 3 May 2009, at 07:44, Carl Johnson wrote: > I have a question about the implementation of "yield from." I recall > hearing some talk about optimizing the stack in "yield from," but I > didn't catch all of the details. I take it that there will be some > steps taken to ensure that yield from's yield from their inner most > yielder without having to touch base at all the objects in between > whoever is asking for the value and whoever is giving it. That being > the case, will this example implementation of a linked list still blow > the stack for lists bigger than 1,000 items or not? > I haven't kept up with recent threads about yield-from but in its early stages at least the patch did still 'touch base' all objects in between, but at C-speed rather than at Python-speed. However I'm pretty sure it would be possible to short-ciruit this, but a trace of the stack of call still needs to be kept if the yield-from is followed by more processing in the generator function (IOW, if it's not a tail yield-from). > class LinkedList: > def __iter__(self): > yield self.value > if self.link: yield from self.link > > If it does still blow up the stack, then it's no big deal, but if this > won't blow the stack up anymore, then it seems like there's very > little difference between the kind of recursive programming invited by > "yield from" and the kind of recursive programming invited by > "continue object". If you hate reading TCO code (and I take this to be > the #1 objection to adding TCO to Python, though there are also more > technical reasons), you're still going to get it, only using "yield > from" instead of "continue". So in that case, "continue" and "yield > from" should be thought of as a pair of stack optimizers, one for > functions and one for generators. So 'continue f(x)' could be spelt 'return from f(x)'? But it's not the same anyway, because yield-from can be a tail-call or not, so the parallel would be more something like this: functions: return f(x) # normal call continue f(x) # optimized tail call generators: yield from gen # normal yield-from continue from gen # optimized tail yield-from In fact, 'continue from gen' would make sense even if it is not a tail yield-from. It would yield control to gen and never come back. OK I have to stop writing whatever comes to my mind now :) -- Arnaud -- Arnaud From greg.ewing at canterbury.ac.nz Sun May 3 10:14:35 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 May 2009 20:14:35 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: Message-ID: <49FD526B.3090809@canterbury.ac.nz> John Graham wrote: > There are some alternative designs for this problems that have been > proposed. SJBrown has proposed a similar syntax, using the two > keywords 'continue as' instead of simply 'continue'. I think it should be called "goto". :-) -- Greg From greg.ewing at canterbury.ac.nz Sun May 3 10:26:51 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 May 2009 20:26:51 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <3bdda690905022344g5329cfb0ld8952c03f0cecf34@mail.gmail.com> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <3bdda690905022344g5329cfb0ld8952c03f0cecf34@mail.gmail.com> Message-ID: <49FD554B.4090803@canterbury.ac.nz> Carl Johnson wrote: > I take it that there will be some > steps taken to ensure that yield from's yield from their inner most > yielder without having to touch base at all the objects in between Not in my current implementation. It still makes a nested sequence of calls to get to the innermost iterator -- it's just that, at least in the case of generators, they're C calls rather than Python ones, so they're much faster. You can still blow the stack, though. -- Greg From greg.ewing at canterbury.ac.nz Sun May 3 10:30:54 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 May 2009 20:30:54 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905031702.03363.steve@pearwood.info> References: <200905031702.03363.steve@pearwood.info> Message-ID: <49FD563E.50404@canterbury.ac.nz> Steven D'Aprano wrote: > It's hardly "premature" to notice that recursive code in Python is > significantly slower and less efficient than in other languages. This > is a known problem, or at least issue, since some people consider that > the solution (tail-recursion optimization) is worse that the problem. It's also by no means certain that TCO would provide the kind of speed benefit that people imagine. A lot of the overhead of making a Python function call is concerned with packing up the arguments and unpacking them again, which you still need to do even if you're reusing the stack frame. -- Greg From denis.spir at free.fr Sun May 3 11:36:17 2009 From: denis.spir at free.fr (spir) Date: Sun, 3 May 2009 11:36:17 +0200 Subject: [Python-ideas] keywording prohibited In-Reply-To: <200905031703.48677.steve@pearwood.info> References: <20090423094835.19fdf337@o> <20090503031633.3d1663a9@o> <83CDA82B-E353-4A0C-B8D0-E99AB1A525F7@atlas.st> <200905031703.48677.steve@pearwood.info> Message-ID: <20090503113617.05b9f3ca@o> Le Sun, 3 May 2009 17:03:48 +1000, Steven D'Aprano s'exprima ainsi: > e.g. instead of naming a variable "seq" or "list_", I'd have the name > > specify what it's a list *of*.) > > Well, this is Python, and we use duck-typing, so everything would need > to be list_of_string_like_instances or similar :) > > But seriously, I tend to use plurals for that. If I have an argument > that takes a collection of widgets, say, I call it "widgets", and write > code like this: > > for widget in widgets: > whatever() I use exactly the same convention. And 100% agree with the fact that the Good Way is to use specific names that denote what use you have for the value, ie the concept the name carries /in your prog/. But precisely sometimes the meaning is generic. Or so obvious from the context or application that it really looks stupid to over-specify (while I often do it anyway). Also, humans are lazy.?[One may require a python programmer not to be lazy, but it will not buy much, and we still have to cope with the fact -- including for ourselves.] In fact, I think issues about reusing builtin name are similar to the ones when bad-naming in general, including the fact that the developper is the first one trapped. But the consequences are potentially much worse and difficulty of debugging usually very great. Denis ------ la vita e estrany From stephen at xemacs.org Sun May 3 13:59:32 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 03 May 2009 20:59:32 +0900 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <49FD563E.50404@canterbury.ac.nz> References: <200905031702.03363.steve@pearwood.info> <49FD563E.50404@canterbury.ac.nz> Message-ID: <871vr6iehn.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > It's also by no means certain that TCO would provide the > kind of speed benefit that people imagine. A lot of the > overhead of making a Python function call is concerned > with packing up the arguments and unpacking them again, > which you still need to do even if you're reusing the > stack frame. I thought that was a good part of the appeal of TCO, though, that the compiler can often arrange to do data manipulations in such a way that the stack frame (or even data-in-register) is just there, ready to go when control is transferred. Ie, the packing/unpacking that is purely related to function calling is avoided. Is this a difficulty in implementing for Python, or did I misunderstand the concept? Ie. that's my fuzzy recollection of a head-splitting conversation with a Schemer that started, "what does call-with-current-continuation *do*?" and part I of his lecture finished with "well, why don't we start with something a little easier like tail call optimization?" Incomprehension-is-my-middle-name-ly y'rs, From solipsis at pitrou.net Sun May 3 14:30:55 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 3 May 2009 12:30:55 +0000 (UTC) Subject: [Python-ideas] A Continuations Compromise in Python References: <200905031702.03363.steve@pearwood.info> Message-ID: Steven D'Aprano writes: > > It's hardly "premature" to notice that recursive code in Python is > significantly slower and less efficient than in other languages. This > is a known problem, or at least issue, since some people consider that > the solution (tail-recursion optimization) is worse that the problem. Is there some evidence that this "known issue" has been popping up in real-world production Python code (I am not talking about Lisp, C, or any other language), rather than academic discussions? Premature optimization is trying to optimize something which is not a significant contributor to execution time. Regards Antoine. From gerald.britton at gmail.com Sun May 3 15:28:30 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sun, 3 May 2009 09:28:30 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <200905031702.03363.steve@pearwood.info> Message-ID: <5d1a32000905030628u6fefd12fwaaaff1255581c8fd@mail.gmail.com> I think its about more than optimization. It's also about being able to code a solution using recursion without worrying about the stack. The execution time may or may not change but the Python source may be easier to read using recursion instead of iteration. At the same time, I feel that some of the hoops one has to jump through (like passing state in function arguments) can adversely affect the readability and elegance of a recursive solution that is trying to take advantage of TCO. In general, I don't like coding source a certain way because "I know" that the compiler will do something "special" if I do. What is "special" today may be not so special (or even worse) tomorrow. I suppose I would rather do the work of translating my recursive solution into an iterative one rather than trying to code my recursive function just right so that the compiler will do TCO for me, if the stack is a concern. With this approach, I can wind up with faster execution and lower amortized memory costs as well. On Sun, May 3, 2009 at 8:30 AM, Antoine Pitrou wrote: > Steven D'Aprano writes: >> >> It's hardly "premature" to notice that recursive code in Python is >> significantly slower and less efficient than in other languages. This >> is a known problem, or at least issue, since some people consider that >> the solution (tail-recursion optimization) is worse that the problem. > > Is there some evidence that this "known issue" has been popping up in real-world > production Python code (I am not talking about Lisp, C, or any other language), > rather than academic discussions? > > Premature optimization is trying to optimize something which is not a > significant contributor to execution time. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From steve at pearwood.info Sun May 3 16:34:53 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 4 May 2009 00:34:53 +1000 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <200905031702.03363.steve@pearwood.info> Message-ID: <200905040034.55415.steve@pearwood.info> On Sun, 3 May 2009 10:30:55 pm Antoine Pitrou wrote: > Steven D'Aprano writes: > > It's hardly "premature" to notice that recursive code in Python is > > significantly slower and less efficient than in other languages. > > This is a known problem, or at least issue, since some people > > consider that the solution (tail-recursion optimization) is worse > > that the problem. > > Is there some evidence that this "known issue" has been popping up in > real-world production Python code (I am not talking about Lisp, C, or > any other language), rather than academic discussions? > > Premature optimization is trying to optimize something which is not a > significant contributor to execution time. People spend time trying to speed up general purpose code in the standard library, to save 1% or 2% on benchmarks. I'm sure I don't need to google for examples -- you've been around the Python-Dev list long enough to see plenty of examples, and I for one haven't seen anyone objecting to improving Python's general execution speed. In this case, the speed-up could (in principle) be by up to a factor of five or so, not a mere couple of percent. Recursion in Python is quite slow compared to iteration. Here's a real (albeit trivial) example: the Towers of Hanoi. def move(n, a, b, c): # Version using tail recursion. "Move n disks from needle a to b using c as temporary storage." if n > 0: for t in move(n-1, a, c, b): yield t yield (a, b) for t in move(n-1, c, b, a): yield t def move2(n, a, b, c): # Non-tail recursive version. while n > 0: for t in move2(n-1, a, c, b): yield t yield (a, b) n -= 1 a, c = c, a And in use: >>> assert list(move(15, 1, 2, 3)) == list(move2(15, 1, 2, 3)) >>> from time import time >>> t = time(); L = list(move(15, 1, 2, 3)); t = time() - t; print t 0.40623497963 >>> t = time(); L = list(move2(15, 1, 2, 3)); t = time() - t; print t 0.261477947235 Okay, this toy program isn't exactly a mission-critical application, but it does demonstrate a genuine performance penalty when using recursion. In this case, two recursive calls (one of which is a tail-call) takes nearly 60% more time than a single recursive call in a while loop. But more importantly, execution time is not the only resource that needs to be optimized. Programmer effort is an important resource that many people wish to optimize. Removing tail-call recursion is a simple algorithm quite well suited to be done by the machine, but tedious and sometimes tricky for the average programmer to do correctly. Another is stack space -- hence the relatively low default recursion limit. Anyone who has ever seen "maximum recursion depth exceeded" in real-world code has a real problem which (if the function is tail-recursive) could be fixed by a hypothetical optimizer. -- Steven D'Aprano From george.sakkis at gmail.com Sun May 3 16:46:00 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 3 May 2009 10:46:00 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905031702.03363.steve@pearwood.info> References: <200905031702.03363.steve@pearwood.info> Message-ID: <91ad5bf80905030746o5f089ec4i7e9190f7ce039a4f@mail.gmail.com> On Sun, May 3, 2009 at 3:02 AM, Steven D'Aprano wrote: > Just tossing a wild idea out there... is it conceivable to build a > decorator that optimizes a tail-call recursive function? Yes, check http://code.activestate.com/recipes/496691/. The caveat is that for "small" values it actually pessimizes rather than optimizes the function ("Note also that these decorators are not optimizing and for small argument values they are actually far slower.") George From solipsis at pitrou.net Sun May 3 17:16:10 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 3 May 2009 15:16:10 +0000 (UTC) Subject: [Python-ideas] A Continuations Compromise in Python References: <200905031702.03363.steve@pearwood.info> <200905040034.55415.steve@pearwood.info> Message-ID: Steven D'Aprano writes: > > People spend time trying to speed up general purpose code in the > standard library, to save 1% or 2% on benchmarks. Actually, it's quite rare that these patches (those which only yield a 1 or 2% improvement) are accepted, except when they don't make the implementation more complex. Lots of other patches with more significant speedups on micro-benchmarks are refused, too. You can find some of them in the bug tracker. > and I for one haven't seen anyone > objecting to improving Python's general execution speed. Not in the absolute, sure. If someone produces a patch speeding up recursive calls it will be considered with the same criteria as any patch claiming to improve performance (see above). This doesn't mean it will be accepted for sure. > In this case, two recursive calls (one of which is a tail-call) takes > nearly 60% more time than a single recursive call in a while loop. Why do you think recursion has anything to do with it, rather than simply the fact that there are twice more function calls? Besides, I object to the claim that solving the Towers of Hano? problem is a real-world example ;) That said, it's true that recursive calls are probably costlier than non-recursive ones, due to the fact that only one frame object is cached for each code objects. But removing this limitation shouldn't make the common (non-recursive) case slower, and it shouldn't increase memory consumption too much, so it's not as easy as it seems. Regards Antoine. From robertc at robertcollins.net Sun May 3 17:29:05 2009 From: robertc at robertcollins.net (Robert Collins) Date: Mon, 04 May 2009 01:29:05 +1000 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905040034.55415.steve@pearwood.info> References: <200905031702.03363.steve@pearwood.info> <200905040034.55415.steve@pearwood.info> Message-ID: <1241364545.20116.14.camel@lifeless-64> On Mon, 2009-05-04 at 00:34 +1000, Steven D'Aprano wrote: > Okay, this toy program isn't exactly a mission-critical application, but > it does demonstrate a genuine performance penalty when using recursion. > In this case, two recursive calls (one of which is a tail-call) takes > nearly 60% more time than a single recursive call in a while loop. This doesn't demonstrate where the issue is. Is it function calls? The tail recursion call makes (eyeball count) twice as many python function calls (not calls into generators) per call to move(2)? Or perhaps something else? Sure, the TCO one is slower, but is the handing of result from generator to generator really the issue? If it is rather something else, it may be that the nested yielding while costing, is not costing disproportionately - and is still a fraction of the cost. python -m timeit 'from foo import do_move, do_move2' 'do_move()' 65535 10 loops, best of 3: 164 msec per loop python -m timeit 'from foo import do_move, do_move2' 'do_move2()' 32768 10 loops, best of 3: 108 msec per loop Half as many function calls, 2/3rds the time. *Much* better measurement than these sketches is needed to say where the issue is. -Rob -------------- next part -------------- A non-text attachment was scrubbed... Name: foo.py Type: text/x-python Size: 796 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From steve at pearwood.info Sun May 3 17:29:28 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 4 May 2009 01:29:28 +1000 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905040034.55415.steve@pearwood.info> References: <200905040034.55415.steve@pearwood.info> Message-ID: <200905040129.29010.steve@pearwood.info> On Mon, 4 May 2009 12:34:53 am Steven D'Aprano wrote about removing tail-recursion: > In this case, the speed-up could (in principle) be by up to a factor > of five or so, not a mere couple of percent. Sorry, that "factor of five" is probably bogus. I got that for some comparisons between recursion and iteration, but the speed difference is probably not relevant to the sort of tail call optimizations we're discussing. I will *not* defend my suggestion of 5x faster, however, on the basis of my Towers of Hanoi test, I think 2x faster is conceivable. I found Guido's recent blog post of tail call optimization: http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html Worthwhile reading. Also read the commenters: they make some interesting points, such as that tail call optimization is a general technique applicable to more than just recursion. Guido's largest objection to TCO is that it ruins nice stack traces when you get an exception. I must admit I've never understood this argument. Perhaps I'm missing something, but I've never considered the stack trace you get in recursive functions useful. Here's an example: >>> def spam(n=0): ... if n == 10: raise ValueError( ... 'Nobody expects the Spanish Inquisition!') ... spam(n+1) ... >>> spam() Traceback (most recent call last): File "", line 1, in File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 3, in spam File "", line 2, in spam ValueError: Nobody expects the Spanish Inquisition! To me, all those identical "line 3, in spam" lines are just noise. They get in the way of a nice stack trace! What is Guido seeing that I'm not? Hopefully he isn't counting them by hand to see how deep he got into the recursion! I wish there was a way to tell Python to just throw that white noise away and give me the sort of stack trace I get from a loop function. (It's not so bad when there only ten lines, but when there's 1000, you might very well fill your xterm's buffer and lose valuable history.) >>> def ham(n=0): ... while n < 0: ... n += 1 ... raise ValueError('Nobody expects the Spanish Inquisition!') ... >>> ham() Traceback (most recent call last): File "", line 1, in File "", line 4, in ham ValueError: Nobody expects the Spanish Inquisition! Which of course illustrates the point that Guido's recommendation to re-write the recursion as an iterative loop by hand will have the same effect on the stack trace as iteration already does. I haven't heard anyone argue that stack traces in a while or for loop should show you the entire history of the loop, so I wonder why recursive calls should be treated as sacrosanct? (I have nothing to say about Guido's other arguments against TCO at this point, but my silence should not be interpreted as agreement.) -- Steven D'Aprano From john.a.graham at gmail.com Sun May 3 18:00:25 2009 From: john.a.graham at gmail.com (John Graham) Date: Sun, 3 May 2009 11:00:25 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905040129.29010.steve@pearwood.info> References: <200905040034.55415.steve@pearwood.info> <200905040129.29010.steve@pearwood.info> Message-ID: Your comment on the iterative solution also erasing the stack trace is a good one. Insofar as the specific problem, I've always thought maybe ... notation could capture recursive stack traces easier... Traceback (most recent call last): File "", line 1, in File "", line 3, in spam ... File "", line 2, in spam ValueError: Nobody expects the Spanish Inquisition! although I believe that particular syntax is used in Doctest (although would it be incorrect in this case?). The same could be used in a 'continue' statement, indicating the first one or two functions and the last one or two functions, similar to sequences described in math (1,2,...,n-1,n). Anywho, I'm no expert on stack traces so if someone else had a better idea I'm all ears. I'd also like to agree with your statement that TCO is not just about execution speed. In fact, to clear up this whole argument, lets go ahead and claim that TCO code will never be faster than the iterative version of things. TCO will prevent things from blowing the stack. And despite many of the cases presented, there are some things that CAN'T be done iteratively, simply, as the commenters to the post you linked to pointed out. Mutual recursion is one, as the iterative solution to that begins to grow in size and use some more complex looping logic. Actual continuation style, where you're making tail-calls to OTHER functions, can only be represented by a trampoline. Which, as I've stated before, is hardly a simpler solution. The complexity of the iterative solution simply goes up from there. I want to clarify that the original suggestion was never to implement TCO 'implicitly'. I don't want to have to 'write my function just right' to get TCO to work. This is why the keyword was suggested, as it'd be an explicit way to tell the interpreter 'this is a tail-call'. If it's not, then throw an exception. Otherwise, there's no optimizations going on behind the scenes that I'm not aware of, which is the case in languages that just turn tail calls optimized behind the scenes. I agree completely with Guido that this can confuse newcomers. So to clarify (and don't get me wrong, this is a very interesting conversation :), the proposal is to add explicit and unambiguous support for the elimination of tail calls (via the use of a keyword), rather than an implicit, confusing optimization behind the scenes, and to do so not to increase runtime performance (per say) but instead to allow techniques which currently blow the stack to work. On Sun, May 3, 2009 at 10:29 AM, Steven D'Aprano wrote: > On Mon, 4 May 2009 12:34:53 am Steven D'Aprano wrote about removing > tail-recursion: > >> In this case, the speed-up could (in principle) be by up to a factor >> of five or so, not a mere couple of percent. > > Sorry, that "factor of five" is probably bogus. I got that for some > comparisons between recursion and iteration, but the speed difference > is probably not relevant to the sort of tail call optimizations we're > discussing. I will *not* defend my suggestion of 5x faster, however, on > the basis of my Towers of Hanoi test, I think 2x faster is conceivable. > > > I found Guido's recent blog post of tail call optimization: > > http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html > > Worthwhile reading. Also read the commenters: they make some interesting > points, such as that tail call optimization is a general technique > applicable to more than just recursion. > > Guido's largest objection to TCO is that it ruins nice stack traces when > you get an exception. I must admit I've never understood this argument. > Perhaps I'm missing something, but I've never considered the stack > trace you get in recursive functions useful. Here's an example: > >>>> def spam(n=0): > ... ? ? if n == 10: raise ValueError( > ... ? ? 'Nobody expects the Spanish Inquisition!') > ... ? ? spam(n+1) > ... >>>> spam() > Traceback (most recent call last): > ?File "", line 1, in > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 2, in spam > ValueError: Nobody expects the Spanish Inquisition! > > To me, all those identical "line 3, in spam" lines are just noise. They > get in the way of a nice stack trace! What is Guido seeing that I'm > not? Hopefully he isn't counting them by hand to see how deep he got > into the recursion! > > I wish there was a way to tell Python to just throw that white noise > away and give me the sort of stack trace I get from a loop function. > (It's not so bad when there only ten lines, but when there's 1000, you > might very well fill your xterm's buffer and lose valuable history.) > >>>> def ham(n=0): > ... ? ? while n < 0: > ... ? ? ? ? ? ? n += 1 > ... ? ? raise ValueError('Nobody expects the Spanish Inquisition!') > ... >>>> ham() > Traceback (most recent call last): > ?File "", line 1, in > ?File "", line 4, in ham > ValueError: Nobody expects the Spanish Inquisition! > > Which of course illustrates the point that Guido's recommendation to > re-write the recursion as an iterative loop by hand will have the same > effect on the stack trace as iteration already does. I haven't heard > anyone argue that stack traces in a while or for loop should show you > the entire history of the loop, so I wonder why recursive calls should > be treated as sacrosanct? > > (I have nothing to say about Guido's other arguments against TCO at this > point, but my silence should not be interpreted as agreement.) > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From steve at pearwood.info Sun May 3 18:03:28 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 4 May 2009 02:03:28 +1000 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <200905040034.55415.steve@pearwood.info> Message-ID: <200905040203.28672.steve@pearwood.info> On Mon, 4 May 2009 01:16:10 am Antoine Pitrou wrote: > > In this case, two recursive calls (one of which is a tail-call) > > takes nearly 60% more time than a single recursive call in a while > > loop. > > Why do you think recursion has anything to do with it, rather than > simply the fact that there are twice more function calls? But surely that's the point of removing tail-recursion? Each function call has overhead, and by reducing the number of function calls, you reduce the amount of overhead. Perhaps we're not discussing the same thing? After reading your reply, there does seem to be some confusion (at least in my head) as to what precisely we're talking about. I've seen tail-call optimization described as programatically converting a tail-call recursive function into an equivalent iterative function. I've also seen it referring to a process of minimizing the depth of the function call stack while still making the same number of function calls. > Besides, I object to the claim that solving the Towers of Hano? > problem is a real-world example ;) Well, I did admit it was a toy :) > That said, it's true that recursive calls are probably costlier than > non-recursive ones, due to the fact that only one frame object is > cached for each code objects. But removing this limitation shouldn't > make the common (non-recursive) case slower, and it shouldn't > increase memory consumption too much, so it's not as easy as it > seems. I don't think anyone really believes it is easy. Even the people on various blog sites saying it's "easy" are actually saying that a solution which is either fragile, or slow, or both, is easy, and that shouldn't surprise anyone. -- Steven D'Aprano From steve at pearwood.info Sun May 3 18:06:26 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 4 May 2009 02:06:26 +1000 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <200905040129.29010.steve@pearwood.info> Message-ID: <200905040206.26458.steve@pearwood.info> On Mon, 4 May 2009 02:00:25 am John Graham wrote: > I want to clarify that the original suggestion was never to implement > TCO 'implicitly'. ?I don't want to have to 'write my function just > right' to get TCO to work. ?This is why the keyword was suggested, as > it'd be an explicit way to tell the interpreter 'this is a > tail-call'. If it's not, then throw an exception. ?Otherwise, there's > no optimizations going on behind the scenes that I'm not aware of, > which is the case in languages that just turn tail calls optimized > behind the scenes. Hmmm... that puts a whole new light on your proposal, at least in my mind. I'm still dubious about using the keyword continue, but less so than before. Let me think about it. -- Steven D'Aprano From john.a.graham at gmail.com Sun May 3 18:31:13 2009 From: john.a.graham at gmail.com (John Graham) Date: Sun, 3 May 2009 11:31:13 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905040206.26458.steve@pearwood.info> References: <200905040129.29010.steve@pearwood.info> <200905040206.26458.steve@pearwood.info> Message-ID: Back to the bike shed argument, the use of 'continue' is incidental. I personally like it, but I totally can see the arguments brought against it. My main thrust is for explicit TCO in Python, to allow for different models of computation in an explicit, Pythonic way. I believe others have suggested making 'yield from' require tail calls, or introducing 'yield as'. Both have their ups and downs. I'd suggest we debate whether or not the original idea, a TCO keyword in python, separate from what that keyword ought to be. It's no use talking about keywords if the original idea is bunk, and I believe the second argument can be won far more pragmatically than the first, which requires some careful thought on all of our parts. On Sun, May 3, 2009 at 11:06 AM, Steven D'Aprano wrote: > On Mon, 4 May 2009 02:00:25 am John Graham wrote: >> I want to clarify that the original suggestion was never to implement >> TCO 'implicitly'. ?I don't want to have to 'write my function just >> right' to get TCO to work. ?This is why the keyword was suggested, as >> it'd be an explicit way to tell the interpreter 'this is a >> tail-call'. If it's not, then throw an exception. ?Otherwise, there's >> no optimizations going on behind the scenes that I'm not aware of, >> which is the case in languages that just turn tail calls optimized >> behind the scenes. > > Hmmm... that puts a whole new light on your proposal, at least in my > mind. I'm still dubious about using the keyword continue, but less so > than before. Let me think about it. > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From qrczak at knm.org.pl Sun May 3 18:42:11 2009 From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Sun, 3 May 2009 18:42:11 +0200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905040129.29010.steve@pearwood.info> References: <200905040034.55415.steve@pearwood.info> <200905040129.29010.steve@pearwood.info> Message-ID: <3f4107910905030942v756ac54meb9081831e743581@mail.gmail.com> 2009/5/3 Steven D'Aprano : > Guido's largest objection to TCO is that it ruins nice stack traces when > you get an exception. I must admit I've never understood this argument. Tail call optimization is more general than tail recursion optimization and it can indeed eliminates useful context from the call stack if the caller and callee are different functions. -- Marcin Kowalczyk qrczak at knm.org.pl http://qrnik.knm.org.pl/~qrczak/ From solipsis at pitrou.net Sun May 3 19:31:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 3 May 2009 17:31:32 +0000 (UTC) Subject: [Python-ideas] A Continuations Compromise in Python References: <200905040034.55415.steve@pearwood.info> <200905040203.28672.steve@pearwood.info> Message-ID: Hello again, Steven D'Aprano writes: > > But surely that's the point of removing tail-recursion? Each function > call has overhead, and by reducing the number of function calls, you > reduce the amount of overhead. Well, it's not that obvious. Tail call optimization does not totally remove function calls, it replaces them with a slightly lighter kind of control transfer. It would not save the cost of setting up a new frame object (the function you transfer control to probably has differing constants and local variables), of passing parameters around (Python's rich function call conventions cost some CPU time) and various other bookkeeping. So, TCO would make the cost of the tail call slightly lighter (how much exactly is unknown), but it won't *suppress* it like e.g. inlining would do. > I've seen tail-call optimization > described as programatically converting a tail-call recursive function > into an equivalent iterative function. I've also seen it referring to a > process of minimizing the depth of the function call stack while still > making the same number of function calls. The former sentence is the formulation in theoretical (algorithmic) terms. The latter sentence is how I believe it gets implemented in practice. Basically, at the assembler level, you replace a "CALL + RETURN" sequence with a single "JUMP" instruction. But that JUMP instruction still must do all the work of setting up parameters, initializing a new frame etc. It's not anything like a lightweight intra-function JUMP. > Even the people on > various blog sites saying it's "easy" are actually saying that a > solution which is either fragile, or slow, or both, is easy, and that > shouldn't surprise anyone. Well, in the sentence you were responding to I was talking about optimizing recursive calls in Python *without* introducing TCO. ;) Regards Antoine. From arnodel at googlemail.com Sun May 3 19:42:20 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 3 May 2009 18:42:20 +0100 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <871vr6iehn.fsf@uwakimon.sk.tsukuba.ac.jp> References: <200905031702.03363.steve@pearwood.info> <49FD563E.50404@canterbury.ac.nz> <871vr6iehn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <9bfc700a0905031042l16b6da0ah2c75529402b2e6d6@mail.gmail.com> 2009/5/3 Stephen J. Turnbull : > Greg Ewing writes: > > ?> It's also by no means certain that TCO would provide the > ?> kind of speed benefit that people imagine. A lot of the > ?> overhead of making a Python function call is concerned > ?> with packing up the arguments and unpacking them again, > ?> which you still need to do even if you're reusing the > ?> stack frame. > > I thought that was a good part of the appeal of TCO, though, that the > compiler can often arrange to do data manipulations in such a way that > the stack frame (or even data-in-register) is just there, ready to go > when control is transferred. ?Ie, the packing/unpacking that is purely > related to function calling is avoided. ?Is this a difficulty in > implementing for Python, or did I misunderstand the concept? This can be done with recursive tail calls, but I guess that in Python, in general, the compiler is not able to know whether a tail call is actually calling the function currently being executed, as the compiler only knows it calls a function with a given name (which is only bound to an object at runtime). However, I think that the interpreter would be able to decide this at run time - and even to short-circuit the packing/unpacking sequence of parameters. -- Arnaud From arnodel at googlemail.com Sun May 3 20:58:59 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 3 May 2009 19:58:59 +0100 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905040129.29010.steve@pearwood.info> References: <200905040034.55415.steve@pearwood.info> <200905040129.29010.steve@pearwood.info> Message-ID: <9bfc700a0905031158w1dfa981aq1539dff57cb1493c@mail.gmail.com> 2009/5/3 Steven D'Aprano : > Guido's largest objection to TCO is that it ruins nice stack traces when > you get an exception. I must admit I've never understood this argument. > Perhaps I'm missing something, but I've never considered the stack > trace you get in recursive functions useful. Here's an example: > >>>> def spam(n=0): > ... ? ? if n == 10: raise ValueError( > ... ? ? 'Nobody expects the Spanish Inquisition!') > ... ? ? spam(n+1) > ... >>>> spam() > Traceback (most recent call last): > ?File "", line 1, in > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 3, in spam > ?File "", line 2, in spam > ValueError: Nobody expects the Spanish Inquisition! > > To me, all those identical "line 3, in spam" lines are just noise. They > get in the way of a nice stack trace! What is Guido seeing that I'm > not? Hopefully he isn't counting them by hand to see how deep he got > into the recursion! Well if you have >>> def foo(): ... bar() ... >>> def bar(): ... baz() ... >>> def baz(): ... raise ValueError('wrong value :S') ... >>> foo() Traceback (most recent call last): File "", line 1, in File "", line 2, in foo File "", line 2, in bar File "", line 2, in baz ValueError: wrong value :S With TCO, I guess you would get something like: >>> foo() Traceback (most recent call last): File "", line 1, in File "", line 2, in baz ValueError: wrong value :S -- Arnaud From denis.spir at free.fr Sun May 3 21:02:09 2009 From: denis.spir at free.fr (spir) Date: Sun, 3 May 2009 21:02:09 +0200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <9bfc700a0905031042l16b6da0ah2c75529402b2e6d6@mail.gmail.com> References: <200905031702.03363.steve@pearwood.info> <49FD563E.50404@canterbury.ac.nz> <871vr6iehn.fsf@uwakimon.sk.tsukuba.ac.jp> <9bfc700a0905031042l16b6da0ah2c75529402b2e6d6@mail.gmail.com> Message-ID: <20090503210209.1ddf6f53@o> Le Sun, 3 May 2009 18:42:20 +0100, Arnaud Delobelle s'exprima ainsi: > This can be done with recursive tail calls, but I guess that in > Python, in general, the compiler is not able to know whether a tail > call is actually calling the function currently being executed, as the > compiler only knows it calls a function with a given name (which is > only bound to an object at runtime). However, I think that the > interpreter would be able to decide this at run time - and even to > short-circuit the packing/unpacking sequence of parameters. The explicit keyword solution proposed is a big help, in this case. Then the interpreter can (try and) proceed to optimization as required -- and possibly fail and protest against wrong requirement. Instead of collecting all needed information and running complicated algorithms _just to know_ whether optimization is simply possible -- before any process. Denis ------ la vita e estrany From rhamph at gmail.com Sun May 3 22:54:03 2009 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 3 May 2009 14:54:03 -0600 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> Message-ID: On Sat, May 2, 2009 at 5:55 PM, John Graham wrote: > The pattern here, basically, that continue eliminates, is the constant > referral to 'just use a trampoline function'. ?To me, language > constructs exist to codify certain patterns, similar to the way list > comprehensions captured a lot of what was previously done in for > loops. No, it's not about codifying. It's about having a *significantly better* solution by modifying the language than working within the language. List comprehensions are significantly better than a full for-loop. Adding a keyword is not significantly better than returning your next function; it's actually worse. It's a non-solution to a non-problem. If you actually *had* a problem you could do it with trampolines. They do exactly what's needed, they just don't put a bow on it. Ho hum. -- Adam Olsen, aka Rhamphoryncus From john.a.graham at gmail.com Sun May 3 22:56:09 2009 From: john.a.graham at gmail.com (John Graham) Date: Sun, 3 May 2009 15:56:09 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> Message-ID: Could you be more elaborate on what attributes you're looking for that make something "significantly better"? On Sun, May 3, 2009 at 3:54 PM, Adam Olsen wrote: > On Sat, May 2, 2009 at 5:55 PM, John Graham wrote: >> The pattern here, basically, that continue eliminates, is the constant >> referral to 'just use a trampoline function'. ?To me, language >> constructs exist to codify certain patterns, similar to the way list >> comprehensions captured a lot of what was previously done in for >> loops. > > No, it's not about codifying. ?It's about having a *significantly > better* solution by modifying the language than working within the > language. ?List comprehensions are significantly better than a full > for-loop. ?Adding a keyword is not significantly better than returning > your next function; it's actually worse. > > It's a non-solution to a non-problem. ?If you actually *had* a problem > you could do it with trampolines. ?They do exactly what's needed, they > just don't put a bow on it. ?Ho hum. > > > -- > Adam Olsen, aka Rhamphoryncus > From rhamph at gmail.com Sun May 3 23:11:39 2009 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 3 May 2009 15:11:39 -0600 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <200905031702.03363.steve@pearwood.info> References: <200905031702.03363.steve@pearwood.info> Message-ID: On Sun, May 3, 2009 at 1:02 AM, Steven D'Aprano wrote: > On Sun, 3 May 2009 09:22:17 am Adam Olsen wrote: >> Premature optimization, irrelevant. > > It's hardly "premature" to notice that recursive code in Python is > significantly slower and less efficient than in other languages. This > is a known problem, or at least issue, since some people consider that > the solution (tail-recursion optimization) is worse that the problem. If you need a tail recursive algorithm, you need it to have O(1) space consumption. Constant overhead from each call is minor in comparison. We wouldn't even have this discussion if constant overhead was given by itself, but we would have it if O(1) space consumption was given by itself. > Up until a month or so ago, I'd never heard of the term "trampoline" > (apart from the thing you jump up and down on), and I still don't know > what it means. Checking Wikipedia, I see that in computing, trampoline > has *eleven* definitions. Which one do you mean? * Used in some LISP implementations, a trampoline is a loop that iteratively invokes thunk-returning functions. A single trampoline is sufficient to express all control transfers of a program; a program so expressed is trampolined or in "trampolined style"; converting a program to trampolined style is trampolining. Trampolined functions can be used to implement tail recursive function calls in stack-oriented languages.[1] For example, something like this (but obviously with arguments and the ability to finish looping): def a(): return b def b(): return a def trampoline(func): while True: func = func() > Just tossing a wild idea out there... is it conceivable to build a > decorator that optimizes a tail-call recursive function? Then it > becomes a matter of explicit programmer choice whether or not to do so, > with no new syntax. You would have the choice of: Yes, there's a lot of weird bytecode hacks out there. > * write a function as tail-recursion because it is most expressive and > simple algorithm, and live with the inefficiency (which as Adam points > out, may not be a problem in practice); > > * manually re-write it as a for-loop, exchanging simplicity for runtime > efficiency; or > > * decorate the recursive function, giving up some debugging information > for efficiency, but keeping the simplicity of the tail-recursive form. One unfortunate point of confusion is that giving up debugging information is only an issue if you want to do it *by default*, applying it to the entire language. In contrast, any infinite loop should use O(1) space, and therefor cannot retain any debugging information. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Sun May 3 23:22:55 2009 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 3 May 2009 15:22:55 -0600 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> Message-ID: On Sun, May 3, 2009 at 2:56 PM, John Graham wrote: > On Sun, May 3, 2009 at 3:54 PM, Adam Olsen wrote: >> On Sat, May 2, 2009 at 5:55 PM, John Graham wrote: >>> The pattern here, basically, that continue eliminates, is the constant >>> referral to 'just use a trampoline function'. To me, language >>> constructs exist to codify certain patterns, similar to the way list >>> comprehensions captured a lot of what was previously done in for >>> loops. >> >> No, it's not about codifying. It's about having a *significantly >> better* solution by modifying the language than working within the >> language. List comprehensions are significantly better than a full >> for-loop. Adding a keyword is not significantly better than returning >> your next function; it's actually worse. >> >> It's a non-solution to a non-problem. If you actually *had* a problem >> you could do it with trampolines. They do exactly what's needed, they >> just don't put a bow on it. Ho hum. > > Could you be more elaborate on what attributes you're looking for that > make something "significantly better"? It's fairly subjective, but lists vs genexps make a good example: data = [] for i in range(50): data.append[i**2] use_result(data) vs use_result(i**2 for i in range(50)) Of course a listcomp would be a more direct comparison, but I think the argument is stronger because it's *not* a direct translation. You'd have to write a generator function, which is enough clumsier that nobody would even consider it, before genexps were added. In contrast, we have the before with trampolines: def a(): return (b, arg1, arg2) and the after: def a(): continue b(arg1, arg2) Sure, it's prettier to look like a function call, but you're disguising the fact that you're not calling it as a function right now. You're also disguising that you're returning from this function. If explaining the trampoline to others is the issue, why not call it a "pop and call" operation? Pop off the old stack frame, then do your call. Or call it "return and call", if they don't understand stack frames. -- Adam Olsen, aka Rhamphoryncus From Scott.Daniels at Acm.Org Mon May 4 00:21:19 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun, 03 May 2009 15:21:19 -0700 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <9bfc700a0905031158w1dfa981aq1539dff57cb1493c@mail.gmail.com> References: <200905040034.55415.steve@pearwood.info> <200905040129.29010.steve@pearwood.info> <9bfc700a0905031158w1dfa981aq1539dff57cb1493c@mail.gmail.com> Message-ID: Arnaud Delobelle wrote: > 2009/5/3 Steven D'Aprano : > >> Guido's largest objection to TCO is that it ruins nice stack traces when >> you get an exception. I must admit I've never understood this argument. >> Perhaps I'm missing something, but I've never considered the stack >> trace you get in recursive functions useful. Here's an example: >> >>>>> def spam(n=0): >> ... if n == 10: raise ValueError( >> ... 'Nobody expects the Spanish Inquisition!') >> ... spam(n+1) Right, but if you have more than one tail call, you lose the history of _which_ recursive call was taken. If you are saying, "Only a single tail-recursive call is allowed per function," then you are eliminating solution by cases. Think: def recurse(source): if branching(source): if goleft(source): return recurse(source[0]) else: return recurse(source[1:]) return source --Scott David Daniels Scott.Daniels at Acm.Org From tjreedy at udel.edu Mon May 4 00:20:50 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 03 May 2009 18:20:50 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <200905031702.03363.steve@pearwood.info> Message-ID: Adam Olsen wrote: > On Sun, May 3, 2009 at 1:02 AM, Steven D'Aprano wrote: >> Up until a month or so ago, I'd never heard of the term "trampoline" >> (apart from the thing you jump up and down on), and I still don't know >> what it means. Checking Wikipedia, I see that in computing, trampoline >> has *eleven* definitions. Which one do you mean? I was pretty much in the same position also. > * Used in some LISP implementations, a trampoline is a loop that > iteratively invokes thunk-returning functions. A single trampoline is > sufficient to express all control transfers of a program; Aha. This is a restatement of the decades-old fact that a while loop and goto are sufficient to implement any flow chart. > For example, something like this (but obviously with arguments Or a global state structure. > and the ability to finish looping): Not sure what you mean, unless a way to exit the mainloop. > def a(): > return b > > def b(): > return a > > def trampoline(func): > while True: > func = func() So the modern version is functions returning the next function to call instead of code-chunks setting the next label to jump to. Of course, that means that one can match the worst spaghetti-code mess with this mechanism also ;-). Thanks for a clear explanation. tjr From john.a.graham at gmail.com Mon May 4 00:46:18 2009 From: john.a.graham at gmail.com (John Graham) Date: Sun, 3 May 2009 17:46:18 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090503181307.226f095f@bhuda.mired.org> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> Message-ID: One use case in the back of my mind dealing with the Actor model was with the web. Actor's communicate via message passing only, and in many cases this is asynchronous message passing. This is basically the same thing as sending and receiving information via HTTP. Although I admit I haven't done much web work in awhile... I think the Seaside framework uses continuations in this way. Anyway, when a user is navigating a website, if they have a few forms to fill in, it can possibly become hair keeping track of that user's session. I'd imagine the standard way is whenever the user inserts information into a form and transfers to another page in the site, some sort of session object or state object is updated with this information, and any further interactions with them are tracked this way. This can cause problems when they use their back button, or just copy the web address to open a new tab (perhaps to navigate to a different part of the site.) Message passing/continuation passing style, in this case, would basically - as I understand it - code up each specific page as an island, expecting all user information to be given in its arguments and not making any look ups to any sort of session or state object for the user. Jumping to a new web page then is nothing more than 'continue' to another function call, whilst passing all needed and gathered information through function arguments alone. The backbutton and clones work in a simpler fashion because there is no more persistent state between page visits, all state is transferred at the boundary right where the 'continue' happens. Like many have mentioned, you can probably do the same thing, once again by having some sort of trampoline outside of each page handler figuring out where to send the use next, and transferring some sort of global state id along with each user, I just imagine that since web programming is one of the major uses of Python, this sort of thing has to get pretty tiring. I realize I'm still speaking a little too high level and in generalities, but if someone would like I could take some time and try and drive the above example into actual code. On Sun, May 3, 2009 at 5:13 PM, Mike Meyer wrote: > On Sun, 3 May 2009 15:22:55 -0600 > Adam Olsen wrote: >> In contrast, we have the before with trampolines: >> >> def a(): >> ? ? return (b, arg1, arg2) >> >> and the after: >> >> def a(): >> ? ? continue b(arg1, arg2) > > But that's only one end of the code involved in the trampoline. The > other end goes from: > > ? ? ?func = a > ? ? ?results = args > ? ? ?while func: > ? ? ? ? results = func(*results) > ? ? ? ? func = results[0] > ? ? ? ? results = results[1] > > or (by tweaking your return code to "return b, (arg1, arg2)") the less > transparent: > > ? ? ?results = a, args > ? ? ?while results[0]: > ? ? ? ? results = apply(*results) > ? ? ?results = results[1] > > to: > > ? ? ?results = a(args) > > > That looks every bit as significant as the change you get from a list > comprehension to me - and these are simplified by ignoring keyword > args. > > But we still need a use case. > >> Sure, it's prettier to look like a function call, but you're >> disguising the fact that you're not calling it as a function right >> now. ?You're also disguising that you're returning from this function. > > Actually, I consider that a problem with the proposed syntax. I'd > prefer to make the distinction explicit. > > ? ? ? -- > Mike Meyer ? ? ? ? ? ? ?http://www.mired.org/consulting.html > Independent Network/Unix/Perforce consultant, email for more information. > > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org > From greg.ewing at canterbury.ac.nz Mon May 4 02:22:11 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 04 May 2009 12:22:11 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <871vr6iehn.fsf@uwakimon.sk.tsukuba.ac.jp> References: <200905031702.03363.steve@pearwood.info> <49FD563E.50404@canterbury.ac.nz> <871vr6iehn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <49FE3533.2060700@canterbury.ac.nz> Stephen J. Turnbull wrote: > I thought that was a good part of the appeal of TCO, though, that the > compiler can often arrange to do data manipulations in such a way that > the stack frame (or even data-in-register) is just there, ready to go > when control is transferred. Possibly that could be done in the case of self-recursion, but it would be trickier when the function being tail-called is a different one. In that case you also have the problem that the called function can have a different stack size, number of local variables, list of constants, etc. You can't just re-use the same stack frame for a different function without making various adjustments. -- Greg From greg.ewing at canterbury.ac.nz Mon May 4 02:30:25 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 04 May 2009 12:30:25 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <5d1a32000905030628u6fefd12fwaaaff1255581c8fd@mail.gmail.com> References: <200905031702.03363.steve@pearwood.info> <5d1a32000905030628u6fefd12fwaaaff1255581c8fd@mail.gmail.com> Message-ID: <49FE3721.50908@canterbury.ac.nz> Gerald Britton wrote: > It's also about being able > to code a solution using recursion without worrying about the stack. > The execution time may or may not change but the Python source may be > easier to read using recursion instead of iteration. Stack space is only an issue if you're using recursion to traverse a linear data structure. If the structure is tree-shaped, you'll run out of memory for the tree long before it's deep enough to cause stack problems. I've yet to see a Python function operating on a linear structure that was easier to follow using recursion instead of iteration. -- Greg From rhamph at gmail.com Mon May 4 07:24:44 2009 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 3 May 2009 23:24:44 -0600 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090503181307.226f095f@bhuda.mired.org> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> Message-ID: On Sun, May 3, 2009 at 4:13 PM, Mike Meyer wrote: > That looks every bit as significant as the change you get from a list > comprehension to me - and these are simplified by ignoring keyword > args. Yet you only need to do that once. At best (with a significant userbase) it'd go in the stdlib. Otherwise you can put it in the cookbook. -- Adam Olsen, aka Rhamphoryncus From solipsis at pitrou.net Mon May 4 13:53:36 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 May 2009 11:53:36 +0000 (UTC) Subject: [Python-ideas] A Continuations Compromise in Python References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> Message-ID: John Graham writes: > > I just imagine that since web > programming is one of the major uses of Python, this sort of thing has > to get pretty tiring. That's why Web frameworks exist, you know. Just try one of them, one day, out of curiousity. ;) Regards Antoine. From john.a.graham at gmail.com Mon May 4 14:37:40 2009 From: john.a.graham at gmail.com (John Graham) Date: Mon, 4 May 2009 07:37:40 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> Message-ID: True, but similar to ABC's, if each of your major users (i.e. frameworks) ends up reinventing a incompatible wheel, when is it time to try and come up with some small standard they can all share? On Mon, May 4, 2009 at 6:53 AM, Antoine Pitrou wrote: > John Graham writes: >> >> I just imagine that since web >> programming is one of the major uses of Python, this sort of thing has >> to get pretty tiring. > > That's why Web frameworks exist, you know. Just try one of them, one day, out of > curiousity. ;) > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From john.a.graham at gmail.com Mon May 4 14:47:25 2009 From: john.a.graham at gmail.com (John Graham) Date: Mon, 4 May 2009 07:47:25 -0500 Subject: [Python-ideas] keywording prohibited In-Reply-To: <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sun, May 3, 2009 at 12:45 AM, Stephen J. Turnbull wrote: > spir writes: > > ?> Depends on your pov. I consider the _possibility_ of using 'list', > ?> 'range' or 'type' as a name for totally custom thing, without even > ?> a warning, an issue, not a wishable feature. It _invents_ very hard > ?> to diagnose bugs. > > If you want warnings, use pylint. ?It catches all of those. Has anyone ever suggested a standard pylint, something that might be distributed with the interpreter/libraries? As important as the 'look-and-feel' is to Python, it seems like an automated style-checker / 'you probably didn't meant to do that' engine would be a good idea to introduce even newbies to the language instead of them having to search it out. -jg From solipsis at pitrou.net Mon May 4 14:53:02 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 4 May 2009 12:53:02 +0000 (UTC) Subject: [Python-ideas] A Continuations Compromise in Python References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> Message-ID: John Graham writes: > > True, but similar to ABC's, if each of your major users (i.e. > frameworks) ends up reinventing a incompatible wheel, when is it time > to try and come up with some small standard they can all share? Well, feel free to propose a PEP to lighten the load of Web framework implementors :) But I'm not convinced continuations have anything to do with it. Regards Antoine. From aahz at pythoncraft.com Mon May 4 15:10:24 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 4 May 2009 06:10:24 -0700 Subject: [Python-ideas] keywording prohibited In-Reply-To: References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20090504131024.GA25783@panix.com> On Mon, May 04, 2009, John Graham wrote: > > Has anyone ever suggested a standard pylint, something that might be > distributed with the interpreter/libraries? As important as the > 'look-and-feel' is to Python, it seems like an automated style-checker > / 'you probably didn't meant to do that' engine would be a good idea > to introduce even newbies to the language instead of them having to > search it out. It's been mentioned; I don't remember why it never got any traction, whether because the authors didn't want to deal with it or nobody bothered submitting a PEP (this definitely needs a PEP). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From debatem1 at gmail.com Mon May 4 22:05:21 2009 From: debatem1 at gmail.com (CTO) Date: Mon, 4 May 2009 13:05:21 -0700 (PDT) Subject: [Python-ideas] keywording prohibited In-Reply-To: <20090504131024.GA25783@panix.com> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> Message-ID: > It's been mentioned; I don't remember why it never got any traction, > whether because the authors didn't want to deal with it or nobody > bothered submitting a PEP (this definitely needs a PEP). I'll volunteer to put one together if somebody's willing to walk me through the PEP process. Geremy Condra From aahz at pythoncraft.com Mon May 4 22:23:48 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 4 May 2009 13:23:48 -0700 Subject: [Python-ideas] keywording prohibited In-Reply-To: References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> Message-ID: <20090504202348.GA5402@panix.com> On Mon, May 04, 2009, CTO wrote: > >> It's been mentioned; I don't remember why it never got any traction, >> whether because the authors didn't want to deal with it or nobody >> bothered submitting a PEP (this definitely needs a PEP). > > I'll volunteer to put one together if somebody's willing to walk me > through the PEP process. Um, I think you trimmed too much of the quote -- what are you talking about? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From debatem1 at gmail.com Mon May 4 22:31:39 2009 From: debatem1 at gmail.com (CTO) Date: Mon, 4 May 2009 13:31:39 -0700 (PDT) Subject: [Python-ideas] keywording prohibited In-Reply-To: <20090504202348.GA5402@panix.com> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> Message-ID: <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> Now with full quote: > On Mon, May 04, 2009, John Graham wrote: >> Has anyone ever suggested a standard pylint, something that might be >> distributed with the interpreter/libraries? As important as the >> 'look-and-feel' is to Python, it seems like an automated style-checker >> / 'you probably didn't meant to do that' engine would be a good idea >> to introduce even newbies to the language instead of them having to >> search it out. > > It's been mentioned; I don't remember why it never got any traction, > whether because the authors didn't want to deal with it or nobody > bothered submitting a PEP (this definitely needs a PEP). > -- > Aahz (a... at pythoncraft.com) <*> http://www.pythoncraft.com/ > > "It is easier to optimize correct code than to correct optimized code." > --Bill Harlan I'll volunteer to put one together if somebody's willing to walk me through the PEP process. Geremy Condra From debatem1 at gmail.com Mon May 4 23:07:40 2009 From: debatem1 at gmail.com (CTO) Date: Mon, 4 May 2009 14:07:40 -0700 (PDT) Subject: [Python-ideas] keywording prohibited In-Reply-To: <20090504202348.GA5402@panix.com> References: <20090423094835.19fdf337@o> <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> Message-ID: <1365fd1b-31e7-4da3-8a8a-a89a07a52d8a@o27g2000vbd.googlegroups.com> On May 4, 4:23?pm, Aahz wrote: > On Mon, May 04, 2009, CTO wrote: > > >> It's been mentioned; I don't remember why it never got any traction, > >> whether because the authors didn't want to deal with it or nobody > >> bothered submitting a PEP (this definitely needs a PEP). > > > I'll volunteer to put one together if somebody's willing to walk me > > through the PEP process. > > Um, I think you trimmed too much of the quote -- what are you talking > about? Apologies- and apparently the list has eaten my earlier reply. This was in reference to the discussion about distributing an automated style checker with python. Geremy Condra From jjb5 at cornell.edu Mon May 4 22:48:08 2009 From: jjb5 at cornell.edu (Joel Bender) Date: Mon, 04 May 2009 16:48:08 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <200905031702.03363.steve@pearwood.info> Message-ID: <49FF5488.5040002@cornell.edu> Adam Olsen wrote: > def a(): > return b > > def b(): > return a > > def trampoline(func): > while True: > func = func() Every once in a while (usually with actor and message passing algorithms) I get the urge to have something like this, I didn't realize it had a formal name: def a(n): trampoline b(n-1) print "never gets printed" def b(n): if cond(n): return n else: trampoline a(n * 2) print "never gets printed" So a trampolined call bails out of the current call frame, and whatever the trampolined call returns, it returns. >>> a(5) == b(4) True >>> The same thing would happen with generators and try/except handlers: def a(n): try: yield n trampoline b(n-1) except Exception, err: print "a() error:", err def b(n): yield n raise RuntimeError, "snort" >>> for i in a(2): ... print i ... 2 1 Traceback (most recent call last): File "", line 1, in File "", line 3, in b RuntimeError: snort >>> Joel From aahz at pythoncraft.com Tue May 5 01:15:33 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 4 May 2009 16:15:33 -0700 Subject: [Python-ideas] lint in stdlib In-Reply-To: <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> References: <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> Message-ID: <20090504231533.GB6215@panix.com> On Mon, May 04, 2009, CTO wrote: > Aahz: >> On Mon, May 04, 2009, John Graham wrote: >>> >>> Has anyone ever suggested a standard pylint, something that might be >>> distributed with the interpreter/libraries? As important as the >>> 'look-and-feel' is to Python, it seems like an automated style-checker >>> / 'you probably didn't meant to do that' engine would be a good idea >>> to introduce even newbies to the language instead of them having to >>> search it out. >> >> It's been mentioned; I don't remember why it never got any traction, >> whether because the authors didn't want to deal with it or nobody >> bothered submitting a PEP (this definitely needs a PEP). > > I'll volunteer to put one together if somebody's willing to walk me > through the PEP process. Sure! I suggest that you start by checking with the maintainers of pychecker and pylint to find out how they feel about it. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From greg.ewing at canterbury.ac.nz Tue May 5 01:24:03 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 05 May 2009 11:24:03 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> Message-ID: <49FF7913.5080406@canterbury.ac.nz> John Graham wrote: > True, but similar to ABC's, if each of your major users (i.e. > frameworks) ends up reinventing a incompatible wheel, when is it time > to try and come up with some small standard they can all share? Maybe I'm missing something, but I don't see how a stateless web server would make use of any kind of continuation-invoking mechanism. The main loop of such a server just gets a request, maps the url to a function, and then calls that function with parameters from the query string. The function does its thing and returns some html to send back to the browser. It has no idea which function should be called next, because there's no "next" in a stateless server -- it depends entirely on what the user of the browser does from there. Another way of thinking about this is that the continuation is encoded in the html form that the function returns. So the main loop of the server, together with the browser+user, can be thought of as a kind of trampoline, where the "function" you return isn't a Python function at all, but a form whose "submit" button sends a query that results in the desired function being called. It seems to me that the existing Python language is perfectly adequate for expressing this. -- Greg From pyideas at rebertia.com Tue May 5 02:25:24 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Mon, 4 May 2009 17:25:24 -0700 Subject: [Python-ideas] lint in stdlib In-Reply-To: <20090504231533.GB6215@panix.com> References: <20090423125354.GA59@panix.com> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> Message-ID: <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> On Mon, May 4, 2009 at 4:15 PM, Aahz wrote: > On Mon, May 04, 2009, CTO wrote: >> Aahz: >>> On Mon, May 04, 2009, John Graham wrote: >>>> >>>> Has anyone ever suggested a standard pylint, something that might be >>>> distributed with the interpreter/libraries? ?As important as the Wouldn't it be be better placed in the Tools/ directory rather than as part of the stdlib? Cheers, Chris -- http://blog.rebertia.com From john.a.graham at gmail.com Tue May 5 02:26:18 2009 From: john.a.graham at gmail.com (John Graham) Date: Mon, 4 May 2009 19:26:18 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090504191646.375875cd@bhuda.mired.org> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> Message-ID: On Mon, May 4, 2009 at 6:16 PM, Mike Meyer wrote: > On Sun, 3 May 2009 23:24:44 -0600 > Adam Olsen wrote: > >> On Sun, May 3, 2009 at 4:13 PM, Mike Meyer wrote: >> > That looks every bit as significant as the change you get from a list >> > comprehension to me - and these are simplified by ignoring keyword >> > args. >> >> Yet you only need to do that once. > > Well, once every time you call a callable that needs a > trampoline. That would include any recursive calls that weren't > properly tail recursive. That could get ugly pretty fast. > >> At best (with a significant userbase) it'd go in the stdlib. >> Otherwise you can put it in the cookbook. > > That still applies, though. I think a decorator could do the job > (only the usage example has been tested): > > ? ?class trampoline(object): > ? ? ? ?"""Provides a trampoline for functions that need one. > > ? ? ? ?Such functions *must* return a triplet of (nextfunc, args, > ? ? ? ?kwdargs) to pass control to nextfunc (which must be also be a > ? ? ? ?trampoline'd function with the given args & kwdargs; or a pair > ? ? ? ?(None, results) to return to their caller. > > ? ? ? ?Usage: > ? ? ? ?@trampoline > ? ? ? ?def tramploop(count): > ? ? ? ? ? if count: > ? ? ? ? ? ? ?print count > ? ? ? ? ? ? ?return tramploop, (count - 1,), {} > ? ? ? ? ? else: > ? ? ? ? ? ? ?return None, None > > ? ? ? ?@trampoline > ? ? ? ?def ack(m, n): > ? ? ? ? ? if m == 0: > ? ? ? ? ? ? ?return None, n + 1 > ? ? ? ? ? elif n == 0: > ? ? ? ? ? ? ?return ack, (m - 1, 1), {} > ? ? ? ? ? else: > ? ? ? ? ? ? ?return ack, (m - 1, ack(m, n - 1)), {} > > ? ? ? ?""" > > ? ? ? ?def __init__(self, func): > ? ? ? ? ? ?self.func = func > > ? ? ? ?def __call__(self, *args, **kwds): > ? ? ? ? ? ?next = self > ? ? ? ? ? ?while True: > ? ? ? ? ? ? ? ?res = next.func(*args, **kwds) > ? ? ? ? ? ? ? ?if not res[0]: > ? ? ? ? ? ? ? ? ? ?return res[1] > ? ? ? ? ? ? ? ?next, args, kwds = res > > Assuming this (or more likely something like it) works, there's not > much use for any way to make the system do TCO for you, other than > performance. And IIRC, that wasn't the OP's issue. > > ? ? ? -- > Mike Meyer ? ? ? ? ? ? ?http://www.mired.org/consulting.html > Independent Network/Unix/Perforce consultant, email for more information. > > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org > As I recall, a few people have advocated a TCO decorator, which I imagine would look a lot like your trampoline decorator, and it usually falls apart in corner cases. I don't have any examples with me though. I do have a question though, if recursive algorithms are generally frowned upon (and I'd say, without TCO, anything too complex hits the stack quick) why is recursion even supported? What is the 'Pythonic' use of recursion? From john.a.graham at gmail.com Tue May 5 02:29:24 2009 From: john.a.graham at gmail.com (John Graham) Date: Mon, 4 May 2009 19:29:24 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <49FF7913.5080406@canterbury.ac.nz> References: <20090503181307.226f095f@bhuda.mired.org> <49FF7913.5080406@canterbury.ac.nz> Message-ID: On Mon, May 4, 2009 at 6:24 PM, Greg Ewing wrote: > John Graham wrote: >> >> True, but similar to ABC's, if each of your major users (i.e. >> frameworks) ends up reinventing a incompatible wheel, when is it time >> to try and come up with some small standard they can all share? > > Maybe I'm missing something, but I don't see how a > stateless web server would make use of any kind of > continuation-invoking mechanism. > > The main loop of such a server just gets a request, > maps the url to a function, and then calls that > function with parameters from the query string. > The function does its thing and returns some > html to send back to the browser. > > It has no idea which function should be called > next, because there's no "next" in a stateless > server -- it depends entirely on what the user > of the browser does from there. > > Another way of thinking about this is that the > continuation is encoded in the html form that the > function returns. So the main loop of the server, > together with the browser+user, can be thought of > as a kind of trampoline, where the "function" you > return isn't a Python function at all, but a form > whose "submit" button sends a query that results > in the desired function being called. > > It seems to me that the existing Python language > is perfectly adequate for expressing this. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > I believe it would be used for state that is not transferred via the URL, such as a user's session with the server should some sort of persistence take place. From john.a.graham at gmail.com Tue May 5 02:30:03 2009 From: john.a.graham at gmail.com (John Graham) Date: Mon, 4 May 2009 19:30:03 -0500 Subject: [Python-ideas] lint in stdlib In-Reply-To: <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> References: <20090423125354.GA59@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> Message-ID: On Mon, May 4, 2009 at 7:25 PM, Chris Rebert wrote: > On Mon, May 4, 2009 at 4:15 PM, Aahz wrote: >> On Mon, May 04, 2009, CTO wrote: >>> Aahz: >>>> On Mon, May 04, 2009, John Graham wrote: >>>>> >>>>> Has anyone ever suggested a standard pylint, something that might be >>>>> distributed with the interpreter/libraries? ?As important as the > > Wouldn't it be be better placed in the Tools/ directory rather than as > part of the stdlib? > > Cheers, > Chris > -- > http://blog.rebertia.com > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > Touche', salesman. -Jg From ben+python at benfinney.id.au Tue May 5 02:57:08 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Tue, 05 May 2009 10:57:08 +1000 Subject: [Python-ideas] lint in stdlib References: <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> Message-ID: <87r5z4icyj.fsf@benfinney.id.au> Aahz writes: > >> On Mon, May 04, 2009, John Graham wrote: > >>> Has anyone ever suggested a standard pylint, something that might > >>> be distributed with the interpreter/libraries? As important as the > >>> 'look-and-feel' is to Python, it seems like an automated > >>> style-checker / 'you probably didn't meant to do that' engine > >>> would be a good idea to introduce even newbies to the language > >>> instead of them having to search it out. > > Sure! I suggest that you start by checking with the maintainers of > pychecker and pylint to find out how they feel about it. The popularity of ?pyflakes? suggests it would be good to coordinate with that project too. -- \ ?When in doubt tell the truth. It will confound your enemies | `\ and astound your friends.? ?Mark Twain, _Following the Equator_ | _o__) | Ben Finney From greg.ewing at canterbury.ac.nz Tue May 5 03:11:20 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 05 May 2009 13:11:20 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <20090503181307.226f095f@bhuda.mired.org> <49FF7913.5080406@canterbury.ac.nz> Message-ID: <49FF9238.7000608@canterbury.ac.nz> John Graham wrote: > I believe it would be used for state that is not transferred via the > URL, such as a user's session with the server should some sort of > persistence take place. In that case you have a stateful server rather than a stateless one. But I still don't see how the proposed "continue" statement would help, because you don't want to call the continuation *now*, you want to defer it until later. So you still need to store it somewhere or return it to a trampoline. Seems to me a better structure for this would be to represent the session by an instance of a class, with methods corresponding to the various queries that can be sent. Then you've got much the same situation as with the stateless server, except that the mapping from query string to function goes through an intermediates step of locating the appropriate session object, based on a session id in the query. Still no need for continuations anywhere. -- Greg From aahz at pythoncraft.com Tue May 5 03:25:27 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 4 May 2009 18:25:27 -0700 Subject: [Python-ideas] lint in stdlib In-Reply-To: <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> References: <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> Message-ID: <20090505012527.GA1358@panix.com> On Mon, May 04, 2009, Chris Rebert wrote: >>>> On Mon, May 04, 2009, John Graham wrote: >>>>> >>>>> Has anyone ever suggested a standard pylint, something that might be >>>>> distributed with the interpreter/libraries? ?As important as the > > Wouldn't it be be better placed in the Tools/ directory rather than as > part of the stdlib? Perhaps, but I don't think so -- you need a source tree to get Tools/ AFAIK, and if some form of lint gets blessed by incorporation into Python, then it needs to be available for all standard installs. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From aahz at pythoncraft.com Tue May 5 03:48:26 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 4 May 2009 18:48:26 -0700 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> Message-ID: <20090505014826.GA21593@panix.com> On Mon, May 04, 2009, John Graham wrote: > > I do have a question though, if recursive algorithms are generally > frowned upon (and I'd say, without TCO, anything too complex hits the > stack quick) why is recursion even supported? What is the 'Pythonic' > use of recursion? Who has said that recursive algorithms are "generally" frowned upon? What people have said is that Python is not focused on recursive algorithms the way TCO-languages are, but that is also true of e.g. functional programming style. Recursive algorithms are natural for nested datastructures, but you will almost never find a nested datastructure that is a thousand levels deep. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From john.a.graham at gmail.com Tue May 5 04:01:58 2009 From: john.a.graham at gmail.com (John Graham) Date: Mon, 4 May 2009 21:01:58 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <49FF9238.7000608@canterbury.ac.nz> References: <20090503181307.226f095f@bhuda.mired.org> <49FF7913.5080406@canterbury.ac.nz> <49FF9238.7000608@canterbury.ac.nz> Message-ID: On Mon, May 4, 2009 at 8:11 PM, Greg Ewing wrote: > John Graham wrote: > >> I believe it would be used for state that is not transferred via the >> URL, such as a user's session with the server should some sort of >> persistence take place. > > In that case you have a stateful server rather > than a stateless one. > > But I still don't see how the proposed "continue" > statement would help, because you don't want to > call the continuation *now*, you want to defer > it until later. So you still need to store it > somewhere or return it to a trampoline. > > Seems to me a better structure for this would > be to represent the session by an instance of a > class, with methods corresponding to the various > queries that can be sent. Then you've got much > the same situation as with the stateless server, > except that the mapping from query string to > function goes through an intermediates step of > locating the appropriate session object, based > on a session id in the query. Still no need > for continuations anywhere. > > -- > Greg > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > In this case, continuation-passing-style would force capturing of this state in the same way many 'functional-style' languages capture state, immutable state in closures? I can see how immutable state from closures might have a benefit when back button and clone-tab buttons are pressed, since there is no one grand object or class to refer to per session, but more like little breadcrumbs of immutable state between function calls. From debatem1 at gmail.com Tue May 5 09:45:07 2009 From: debatem1 at gmail.com (CTO) Date: Tue, 5 May 2009 00:45:07 -0700 (PDT) Subject: [Python-ideas] lint in stdlib In-Reply-To: <87r5z4icyj.fsf@benfinney.id.au> References: <20090423125354.GA59@panix.com> <1241300129.30081.38.camel@saeko.local> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <87r5z4icyj.fsf@benfinney.id.au> Message-ID: On May 4, 8:57?pm, Ben Finney wrote: > Aahz writes: > > >> On Mon, May 04, 2009, John Graham wrote: > > >>> Has anyone ever suggested a standard pylint, something that might > > >>> be distributed with the interpreter/libraries? As important as the > > >>> 'look-and-feel' is to Python, it seems like an automated > > >>> style-checker / 'you probably didn't meant to do that' engine > > >>> would be a good idea to introduce even newbies to the language > > >>> instead of them having to search it out. > > > Sure! ?I suggest that you start by checking with the maintainers of > > pychecker and pylint to find out how they feel about it. > > The popularity of ?pyflakes? suggests it would be good to coordinate > with that project too. > I've sent emails to pylint, pychecker, pyflakes, and pep8.py. We'll see what comes out of that, but if anybody is aware of any alternative emails for any of the above, I should probably send something to those too- the one for pylint in particular doesn't look like it's been checked in a while. Geremy Condra From tjreedy at udel.edu Tue May 5 10:55:24 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 05 May 2009 04:55:24 -0400 Subject: [Python-ideas] lint in stdlib In-Reply-To: <20090505012527.GA1358@panix.com> References: <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> Message-ID: Aahz wrote: > On Mon, May 04, 2009, Chris Rebert wrote: >>>>> On Mon, May 04, 2009, John Graham wrote: >>>>>> Has anyone ever suggested a standard pylint, something that might be >>>>>> distributed with the interpreter/libraries? ?As important as the >> Wouldn't it be be better placed in the Tools/ directory rather than as >> part of the stdlib? > > Perhaps, but I don't think so -- you need a source tree to get Tools/ > AFAIK, and if some form of lint gets blessed by incorporation into > Python, then it needs to be available for all standard installs. Regardless, the stdlib is for modules to import, tools for programs that run. From jh at improva.dk Tue May 5 10:56:53 2009 From: jh at improva.dk (Jacob Holm) Date: Tue, 05 May 2009 10:56:53 +0200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090505014826.GA21593@panix.com> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505014826.GA21593@panix.com> Message-ID: <49FFFF55.5080602@improva.dk> Aahz wrote: > On Mon, May 04, 2009, John Graham wrote: >> I do have a question though, if recursive algorithms are generally >> frowned upon (and I'd say, without TCO, anything too complex hits the >> stack quick) why is recursion even supported? What is the 'Pythonic' >> use of recursion? > > Who has said that recursive algorithms are "generally" frowned upon? > What people have said is that Python is not focused on recursive > algorithms the way TCO-languages are, but that is also true of e.g. > functional programming style. An unfortunate result of "Python is not focused on recursive algorithms" is that such algorithms become much harder to work with. Yes, there is always a way to write it without recursion, and often the non-recursive version will run faster because of the slow function calls. However, the non-recursive version is usually much less readable/maintainable, which is really a shame in a language designed for readability. > Recursive algorithms are natural for > nested datastructures, but you will almost never find a nested > datastructure that is a thousand levels deep. Balanced trees are not the only data structures in the world. Depth-first traversal of a graph is most easily expressed using recursion, and graphs with thousands (or even millions) of vertices are not uncommon. Also, there are many other types of problems where a recursive algorithm is the most natural. Most discrete optimization problems fall in this category. For toy examples, look at almost any type of puzzle-solving. The proposed "continue " feature doesn't help these cases, because they don't use tail calls. For that you need something like Stackless. I'm -1 on the spelling anyway because it would kill the idea of using "continue " in a for-loop as a way to send the value of to the generator being iterated over. I'm +0 on the idea of using syntax to explicitly request tail call optimization. I'm +1 on doing implicit tail-call optimization, even if it does make the stack trace less useful. - Jacob From fuzzyman at gmail.com Tue May 5 11:29:45 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 5 May 2009 10:29:45 +0100 Subject: [Python-ideas] lint in stdlib In-Reply-To: <20090504231533.GB6215@panix.com> References: <20090423125354.GA59@panix.com> <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> Message-ID: <6f4025010905050229o74591a9fod7a6d9255b726a8f@mail.gmail.com> 2009/5/5 Aahz > On Mon, May 04, 2009, CTO wrote: > > Aahz: > >> On Mon, May 04, 2009, John Graham wrote: > >>> > >>> Has anyone ever suggested a standard pylint, something that might be > >>> distributed with the interpreter/libraries? As important as the > >>> 'look-and-feel' is to Python, it seems like an automated style-checker > >>> / 'you probably didn't meant to do that' engine would be a good idea > >>> to introduce even newbies to the language instead of them having to > >>> search it out. > >> > >> It's been mentioned; I don't remember why it never got any traction, > >> whether because the authors didn't want to deal with it or nobody > >> bothered submitting a PEP (this definitely needs a PEP). > > > > I'll volunteer to put one together if somebody's willing to walk me > > through the PEP process. > > Sure! I suggest that you start by checking with the maintainers of > pychecker and pylint to find out how they feel about it. PyLint is huge and depends on a whole ream of support libraries. I think the logilab guys would probably rather maintain it themselves and I'm not sure it is really suitable for the standard library anyway. PyFlakes and PyChecker are both very good; although neither does as much as PyLint. I'm not sure if either of the two does enough to enforce full PEP-8 compliance. Michael Foord > > -- > Aahz (aahz at pythoncraft.com) <*> > http://www.pythoncraft.com/ > > "It is easier to optimize correct code than to correct optimized code." > --Bill Harlan > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhamph at gmail.com Tue May 5 12:15:29 2009 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 5 May 2009 04:15:29 -0600 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090504191646.375875cd@bhuda.mired.org> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> Message-ID: On Mon, May 4, 2009 at 5:16 PM, Mike Meyer wrote: > On Sun, 3 May 2009 23:24:44 -0600 > Adam Olsen wrote: > >> On Sun, May 3, 2009 at 4:13 PM, Mike Meyer wrote: >> > That looks every bit as significant as the change you get from a list >> > comprehension to me - and these are simplified by ignoring keyword >> > args. >> >> Yet you only need to do that once. > > Well, once every time you call a callable that needs a > trampoline. That would include any recursive calls that weren't > properly tail recursive. That could get ugly pretty fast. You only need to *define* it once. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Tue May 5 12:21:05 2009 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 5 May 2009 04:21:05 -0600 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> Message-ID: On Mon, May 4, 2009 at 6:26 PM, John Graham wrote: > As I recall, a few people have advocated a TCO decorator, which I > imagine would look a lot like your trampoline decorator, and it > usually falls apart in corner cases. ?I don't have any examples with > me though. ?I do have a question though, if recursive algorithms are > generally frowned upon (and I'd say, without TCO, anything too complex > hits the stack quick) why is recursion even supported? ?What is the > 'Pythonic' use of recursion? There's many forms of recursion that do not involve unbounded recursion. Consider a call to len() that calls len() on another object. -- Adam Olsen, aka Rhamphoryncus From google at mrabarnett.plus.com Tue May 5 14:18:07 2009 From: google at mrabarnett.plus.com (MRAB) Date: Tue, 05 May 2009 13:18:07 +0100 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <49FFFF55.5080602@improva.dk> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505014826.GA21593@panix.com> <49FFFF55.5080602@improva.dk> Message-ID: <4A002E7F.9060405@mrabarnett.plus.com> Jacob Holm wrote: > Aahz wrote: >> On Mon, May 04, 2009, John Graham wrote: >>> I do have a question though, if recursive algorithms are generally >>> frowned upon (and I'd say, without TCO, anything too complex hits the >>> stack quick) why is recursion even supported? What is the 'Pythonic' >>> use of recursion? >> >> Who has said that recursive algorithms are "generally" frowned upon? >> What people have said is that Python is not focused on recursive >> algorithms the way TCO-languages are, but that is also true of e.g. >> functional programming style. > > An unfortunate result of "Python is not focused on recursive algorithms" > is that such algorithms become much harder to work with. Yes, there is > always a way to write it without recursion, and often the non-recursive > version will run faster because of the slow function calls. However, > the non-recursive version is usually much less readable/maintainable, > which is really a shame in a language designed for readability. > > >> Recursive algorithms are natural for >> nested datastructures, but you will almost never find a nested >> datastructure that is a thousand levels deep. > > Balanced trees are not the only data structures in the world. > > Depth-first traversal of a graph is most easily expressed using > recursion, and graphs with thousands (or even millions) of vertices are > not uncommon. > > Also, there are many other types of problems where a recursive algorithm > is the most natural. Most discrete optimization problems fall in this > category. For toy examples, look at almost any type of puzzle-solving. > > > The proposed "continue " feature doesn't help these cases, > because they don't use tail calls. For that you need something like > Stackless. I'm -1 on the spelling anyway because it would kill the idea > of using "continue " in a for-loop as a way to send the > value of to the generator being iterated over. > > I'm +0 on the idea of using syntax to explicitly request tail call > optimization. > > I'm +1 on doing implicit tail-call optimization, even if it does make > the stack trace less useful. > Could Python implement tail recursion optimisation and somehow 'fake' the stack trace? I'm thinking that the stack trace would contain a repeating pattern of calls, so it would store and show the pattern of calls and the count of how many times the sequence was repeated. From john.a.graham at gmail.com Tue May 5 14:29:22 2009 From: john.a.graham at gmail.com (John Graham) Date: Tue, 5 May 2009 07:29:22 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090505001653.6196a7b5@bhuda.mired.org> References: <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505001653.6196a7b5@bhuda.mired.org> Message-ID: > > The version above avoids all that by doing what the OP suggested, and > leaving it up to the programmer to decide what is and what is not a > tail call. This mostly just demonstrates that this can be done with > existing language constructs without the need for changes to the > language. So you think a tail-call decorator approach would work, so long as the programmer enforces themselves that it is a tail call? This seems like a good compromise, if there is an implementation in the cookbook that does this sort of simple optimization. What might be a good approach is to watch how popular that sort of decorator is, and if it turns out to be more popular than is expected, perhaps putting it in the language would sound like a better idea. I'd rather the trampoline that implements tail calls be written in C than in Python, if it is heavily used. > Of course, we still don't have a solid use case for this, which is why > I haven't bothered to really flesh it out by catching programmer > errors, making the API a bit cleaner, etc. > >> I don't have any examples with >> me though. ?I do have a question though, if recursive algorithms are >> generally frowned upon (and I'd say, without TCO, anything too complex >> hits the stack quick) why is recursion even supported? ?What is the >> 'Pythonic' use of recursion? > > Because it's a very powerful, practical approach to a very large set > of problems. Anyone whose had more than a smattering of education in > math will have seen recursively defined functions or sequences of some > sort, and will intuitively try writing them - and then wonder why it > doesn't work in non-recursive languages (yes, I've seen that > happen). Powerful, practical and intuitive sounds pythonic to me. > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org > Sorry, that was a sneaky trick question. I just wanted to make sure we're all on the same page that recursion *CAN* be a good way to write some things, even clearer than loops. The problem is that there are a whole host of things, like the graph traversal Jacob has pointed out, that can't work with such a low stack limit. I just find it annoying that the theoretically 'correct' and admittedly 'elegant' solution to a few pretty standard graph problems can't be expressed in Python because we hit the stack. (I'm excluding those that Jacob makes note can't use tail calls anyway) I also pointed out the continuation-passing-style web server, the actor model and messages as well as the generic trampoline replacement as all use cases. None of these are all that wide-spread, but I guess my point is that if there were a language feature that could help in all of these areas, that must count for something. There is no one 'killer' use case, but there's a good half dozen or so we've pointed out. Surely that puts the proposal on the same footing, at least, as co-routines. I also don't want to discount the performance benefit. I know I said above, my main thrust is towards allowing 'correct' implementations of algorithms to run in Python without hitting the stack limit, but it is a completely valid sub-issue if performance can noticeably increase, as Arnauld just linked to and I'll be reading here in a moment. Some have also mentioned that they're not fans of the actual keyword use. I wanted to try and split those two debates apart, whether or not it is warranted, which I believe it is, and how it should be implemented. I had another proposal on the actual keyword front, "return from", which looks like it would kind of provide some symmetry to the 'return' and 'yield' constructs and also reads pretty intuitively, in my opinion. Thanks everyone for your input so far. -JG From john.a.graham at gmail.com Tue May 5 14:33:07 2009 From: john.a.graham at gmail.com (John Graham) Date: Tue, 5 May 2009 07:33:07 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <4A002E7F.9060405@mrabarnett.plus.com> References: <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505014826.GA21593@panix.com> <49FFFF55.5080602@improva.dk> <4A002E7F.9060405@mrabarnett.plus.com> Message-ID: On Tue, May 5, 2009 at 7:18 AM, MRAB wrote: > Jacob Holm wrote: >> >> Aahz wrote: >>> >>> On Mon, May 04, 2009, John Graham wrote: >>>> >>>> I do have a question though, if recursive algorithms are generally >>>> frowned upon (and I'd say, without TCO, anything too complex hits the >>>> stack quick) why is recursion even supported? ?What is the 'Pythonic' >>>> use of recursion? >>> >>> Who has said that recursive algorithms are "generally" frowned upon? >>> What people have said is that Python is not focused on recursive >>> algorithms the way TCO-languages are, but that is also true of e.g. >>> functional programming style. >> >> An unfortunate result of "Python is not focused on recursive algorithms" >> is that such algorithms become much harder to work with. ?Yes, there is >> always a way to write it without recursion, and often the non-recursive >> version will run faster because of the slow function calls. ?However, the >> non-recursive version is usually much less readable/maintainable, which is >> really a shame in a language designed for readability. >> >> >>> Recursive algorithms are natural for >>> nested datastructures, but you will almost never find a nested >>> datastructure that is a thousand levels deep. >> >> Balanced trees are not the only data structures in the world. >> >> Depth-first traversal of a graph is most easily expressed using recursion, >> and graphs with thousands (or even millions) of vertices are not uncommon. >> >> Also, there are many other types of problems where a recursive algorithm >> is the most natural. ?Most discrete optimization problems fall in this >> category. ?For toy examples, look at almost any type of puzzle-solving. >> >> >> The proposed "continue " feature doesn't help these cases, >> because they don't use tail calls. ?For that you need something like >> Stackless. ?I'm -1 on the spelling anyway because it would kill the idea of >> using "continue " in a for-loop as a way to send the value of >> to the generator being iterated over. >> >> I'm +0 on the idea of using syntax to explicitly request tail call >> optimization. >> >> I'm +1 on doing implicit tail-call optimization, even if it does make the >> stack trace less useful. >> > Could Python implement tail recursion optimisation and somehow 'fake' > the stack trace? I'm thinking that the stack trace would contain a > repeating pattern of calls, so it would store and show the pattern of > calls and the count of how many times the sequence was repeated. Plain recursive tail calls might be faked by just saying "You called this a whole bunch". But message passing sorts of tail calls, where you are calling other methods and functions, could really use a stack trace. The way this sort of thing is dealt with in other languages is to put the stack trace in one part of a 'history' object, and then the tail-call history in another. They show up the same way, but the stack part of the history is not limited (except to the size of the stack) while the tail call part is put on a large circular buffer. You can potentially lose information in the trace this way too, but it's a pragmatic solution and it's as least as 'pretty' as setting the stack limit so low anyway. From jeremy at alum.mit.edu Tue May 5 14:42:44 2009 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 5 May 2009 08:42:44 -0400 Subject: [Python-ideas] lint in stdlib In-Reply-To: <6f4025010905050229o74591a9fod7a6d9255b726a8f@mail.gmail.com> References: <20090423125354.GA59@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <6f4025010905050229o74591a9fod7a6d9255b726a8f@mail.gmail.com> Message-ID: On Tue, May 5, 2009 at 5:29 AM, Michael Foord wrote: > > > 2009/5/5 Aahz >> >> On Mon, May 04, 2009, CTO wrote: >> > Aahz: >> >> On Mon, May 04, 2009, John Graham wrote: >> >>> >> >>> Has anyone ever suggested a standard pylint, something that might be >> >>> distributed with the interpreter/libraries? ?As important as the >> >>> 'look-and-feel' is to Python, it seems like an automated style-checker >> >>> / 'you probably didn't meant to do that' engine would be a good idea >> >>> to introduce even newbies to the language instead of them having to >> >>> search it out. >> >> >> >> It's been mentioned; I don't remember why it never got any traction, >> >> whether because the authors didn't want to deal with it or nobody >> >> bothered submitting a PEP (this definitely needs a PEP). >> > >> > I'll volunteer to put one together if somebody's willing to walk me >> > through the PEP process. >> >> Sure! ?I suggest that you start by checking with the maintainers of >> pychecker and pylint to find out how they feel about it. > > PyLint is huge and depends on a whole ream of support libraries. I think the > logilab guys would probably rather maintain it themselves and I'm not sure > it is really suitable for the standard library anyway. We use pylint at work for our own lint tool. It's been useful to us. I'm not sure that I see much value in adding it or something like it to the standard distribution. If there is a good lint tool that supports PEP 8 and is available for download, I expect most python developers would install it. Jeremy > PyFlakes and PyChecker are both very good; although neither does as much as > PyLint. I'm not sure if either of the two does enough to enforce full PEP-8 > compliance. > > Michael Foord > > >> >> -- >> Aahz (aahz at pythoncraft.com) ? ? ? ? ? <*> >> http://www.pythoncraft.com/ >> >> "It is easier to optimize correct code than to correct optimized code." >> --Bill Harlan >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas > > > > -- > http://www.ironpythoninaction.com/ > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > From denis.spir at free.fr Tue May 5 15:41:02 2009 From: denis.spir at free.fr (spir) Date: Tue, 5 May 2009 15:41:02 +0200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505001653.6196a7b5@bhuda.mired.org> Message-ID: <20090505154102.4ddcba4d@o> Le Tue, 5 May 2009 07:29:22 -0500, John Graham s'exprima ainsi: > I had another proposal on the actual keyword front, > "return from", which looks like it would kind of provide some symmetry > to the 'return' and 'yield' constructs and also reads pretty > intuitively, in my opinion. Sounds good imo. return --> the process does not go farther in the curent func from --> there a detour Continue (as used in loops) implies instead starting again. Denis ------ la vita e estrany From ncoghlan at gmail.com Tue May 5 16:46:46 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 06 May 2009 00:46:46 +1000 Subject: [Python-ideas] lint in stdlib In-Reply-To: References: <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> Message-ID: <4A005156.9010900@gmail.com> Terry Reedy wrote: > Aahz wrote: >> On Mon, May 04, 2009, Chris Rebert wrote: >>>>>> On Mon, May 04, 2009, John Graham wrote: >>>>>>> Has anyone ever suggested a standard pylint, something that might be >>>>>>> distributed with the interpreter/libraries? ?As important as the >>> Wouldn't it be be better placed in the Tools/ directory rather than as >>> part of the stdlib? >> >> Perhaps, but I don't think so -- you need a source tree to get Tools/ >> AFAIK, and if some form of lint gets blessed by incorporation into >> Python, then it needs to be available for all standard installs. > > Regardless, the stdlib is for modules to import, tools for programs that > run. We blur that line all the time though (cf. timeit, pdb, pydoc, webbrowser, runpy, probably others). So I'd agree with Aahz that if python-dev is going to bless something along these lines, it should be something worthy of inclusion in the standard lib itself rather than just being dropped into the Tools directory. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From zac256 at gmail.com Tue May 5 18:25:16 2009 From: zac256 at gmail.com (Zac Burns) Date: Tue, 5 May 2009 09:25:16 -0700 Subject: [Python-ideas] New list methods Message-ID: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> I would like to see the following methods added to lists... rindex : Return first index of value from the right rremove: Return first occurence of value from the right. -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer (Digital Overlord) Zindagi Games From grosser.meister.morti at gmx.net Tue May 5 18:45:39 2009 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Tue, 05 May 2009 18:45:39 +0200 Subject: [Python-ideas] New list methods In-Reply-To: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> Message-ID: <4A006D33.3030202@gmx.net> Zac Burns wrote: > I would like to see the following methods added to lists... > > rindex : Return first index of value from the right > rremove: Return first occurence of value from the right. > +1 From phd at phd.pp.ru Tue May 5 18:49:02 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 5 May 2009 20:49:02 +0400 Subject: [Python-ideas] New list methods In-Reply-To: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> Message-ID: <20090505164902.GA4843@phd.pp.ru> On Tue, May 05, 2009 at 09:25:16AM -0700, Zac Burns wrote: > I would like to see the following methods added to lists... > > rindex : Return first index of value from the right > rremove: Return first occurence of value from the right. l.rremove() is the same as l.pop() or l.pop(-1) Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From jared.grubb at gmail.com Tue May 5 19:10:17 2009 From: jared.grubb at gmail.com (Jared Grubb) Date: Tue, 5 May 2009 10:10:17 -0700 Subject: [Python-ideas] New list methods In-Reply-To: <20090505164902.GA4843@phd.pp.ru> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <20090505164902.GA4843@phd.pp.ru> Message-ID: <2130C1FE-8AFC-41A8-87E7-998E77BC92F1@gmail.com> On 5 May 2009, at 09:49, Oleg Broytmann wrote: > On Tue, May 05, 2009 at 09:25:16AM -0700, Zac Burns wrote: >> I would like to see the following methods added to lists... >> >> rindex : Return first index of value from the right >> rremove: Return first occurence of value from the right. > > l.rremove() is the same as l.pop() or l.pop(-1) No, pop removes an element; rremove removes an element of a given value. Jared From jeremy at jeremybanks.ca Tue May 5 19:11:44 2009 From: jeremy at jeremybanks.ca (Jeremy Banks) Date: Tue, 5 May 2009 14:11:44 -0300 Subject: [Python-ideas] New list methods In-Reply-To: <20090505164902.GA4843@phd.pp.ru> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <20090505164902.GA4843@phd.pp.ru> Message-ID: <9e754ef50905051011i19c7d8f1p57624e03b6de06b5@mail.gmail.com> I believe the proposal is to have l.rremove() take a value parameter, and remove the last occurrence of that value from the list, not just the last value in the list. My assumpting is that their basic functionality would work like this. l = [1, 2, 3, 2, 1] l.rindex(2) == 3 l.rremove(2) l == [1, 2, 3, 1] I'd say +1. On 2009-05-05, Oleg Broytmann wrote: > On Tue, May 05, 2009 at 09:25:16AM -0700, Zac Burns wrote: >> I would like to see the following methods added to lists... >> >> rindex : Return first index of value from the right >> rremove: Return first occurence of value from the right. > > l.rremove() is the same as l.pop() or l.pop(-1) > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From zac256 at gmail.com Tue May 5 19:17:51 2009 From: zac256 at gmail.com (Zac Burns) Date: Tue, 5 May 2009 10:17:51 -0700 Subject: [Python-ideas] New list methods In-Reply-To: <9e754ef50905051011i19c7d8f1p57624e03b6de06b5@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <20090505164902.GA4843@phd.pp.ru> <9e754ef50905051011i19c7d8f1p57624e03b6de06b5@mail.gmail.com> Message-ID: <333edbe80905051017w194209f9i236e8e17c6f19f5f@mail.gmail.com> Jeremy is correct... Sorry, there was a typo in the proposal. This: "rremove: Return first occurence of value from the right." Should read: "rremove: Remove first occurence of value from the right." The methods are 'inspired' by the similar 'r' methods of strings. -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer (Digital Overlord) Zindagi Games On Tue, May 5, 2009 at 10:11 AM, Jeremy Banks wrote: > I believe the proposal is to have l.rremove() take a value parameter, > and remove the last occurrence of that value from the list, not just > the last value in the list. > > My assumpting is that their basic functionality would work like this. > > ? ?l = [1, 2, 3, 2, 1] > ? ?l.rindex(2) == 3 > ? ?l.rremove(2) > ? ?l == [1, 2, 3, 1] > > I'd say +1. > > On 2009-05-05, Oleg Broytmann wrote: >> On Tue, May 05, 2009 at 09:25:16AM -0700, Zac Burns wrote: >>> I would like to see the following methods added to lists... >>> >>> rindex : Return first index of value from the right >>> rremove: Return first occurence of value from the right. >> >> ? ?l.rremove() is the same as l.pop() or l.pop(-1) >> >> Oleg. >> -- >> ? ? ?Oleg Broytmann ? ? ? ? ? ?http://phd.pp.ru/ ? ? ? ? ? ?phd at phd.pp.ru >> ? ? ? ? ? ?Programmers don't die, they just GOSUB without RETURN. >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From debatem1 at gmail.com Tue May 5 19:25:48 2009 From: debatem1 at gmail.com (CTO) Date: Tue, 5 May 2009 10:25:48 -0700 (PDT) Subject: [Python-ideas] New list methods In-Reply-To: <333edbe80905051017w194209f9i236e8e17c6f19f5f@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <20090505164902.GA4843@phd.pp.ru> <9e754ef50905051011i19c7d8f1p57624e03b6de06b5@mail.gmail.com> <333edbe80905051017w194209f9i236e8e17c6f19f5f@mail.gmail.com> Message-ID: <7f81bc89-1e9b-448d-a9ce-4816dc5df5bb@n8g2000vbb.googlegroups.com> On May 5, 1:17?pm, Zac Burns wrote: > Jeremy is correct... > > Sorry, there was a typo in the proposal. > > This: > ? ? "rremove: Return first occurence of value from the right." > > Should read: > ? ? "rremove: ?Remove first occurence of value from the right." > > The methods are 'inspired' by the similar 'r' methods of strings. Sounds handy. Would collections.Sequence support them? Geremy Condra From george.sakkis at gmail.com Tue May 5 19:46:57 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Tue, 5 May 2009 13:46:57 -0400 Subject: [Python-ideas] New list methods In-Reply-To: <9e754ef50905051011i19c7d8f1p57624e03b6de06b5@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <20090505164902.GA4843@phd.pp.ru> <9e754ef50905051011i19c7d8f1p57624e03b6de06b5@mail.gmail.com> Message-ID: <91ad5bf80905051046p3e8b5541wdc8f32126a50b5ba@mail.gmail.com> On Tue, May 5, 2009 at 1:11 PM, Jeremy Banks wrote: > I believe the proposal is to have l.rremove() take a value parameter, > and remove the last occurrence of that value from the list, not just > the last value in the list. > > My assumpting is that their basic functionality would work like this. > > ? ?l = [1, 2, 3, 2, 1] > ? ?l.rindex(2) == 3 > ? ?l.rremove(2) > ? ?l == [1, 2, 3, 1] > > I'd say +1. The functionality would be useful but instead of polluting the API with various "r*" methods, I'd prefer a `reversed=False` boolean parameter wherever it makes sense, just like sort(ed). I.e. instead of list.rindex -> list.index(reversed=True) and list.rremove -> list.remove(reversed=True). George From google at mrabarnett.plus.com Tue May 5 19:48:53 2009 From: google at mrabarnett.plus.com (MRAB) Date: Tue, 05 May 2009 18:48:53 +0100 Subject: [Python-ideas] New list methods In-Reply-To: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> Message-ID: <4A007C05.7090702@mrabarnett.plus.com> Zac Burns wrote: > I would like to see the following methods added to lists... > > rindex : Return first index of value from the right > rremove: Return first occurence of value from the right. > index() has start and stop arguments. Would anything be gained if remove() had them too? From zac256 at gmail.com Tue May 5 19:52:54 2009 From: zac256 at gmail.com (Zac Burns) Date: Tue, 5 May 2009 10:52:54 -0700 Subject: [Python-ideas] New list methods In-Reply-To: <4A007C05.7090702@mrabarnett.plus.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <4A007C05.7090702@mrabarnett.plus.com> Message-ID: <333edbe80905051052p2f67e8ebv6aad5245ae5c0763@mail.gmail.com> I've tried to 'r' search the list using the start/stop parameters to no avail. l = [1, 2, 3, 2, 1] l.index(2, 0, 4) == 1 # True l.index(2, 4, 0) # ValueError l.index(2, -1, 0) # ValueError -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer (Digital Overlord) Zindagi Games On Tue, May 5, 2009 at 10:48 AM, MRAB wrote: > Zac Burns wrote: >> >> I would like to see the following methods added to lists... >> >> rindex : Return first index of value from the right >> rremove: Return first occurence of value from the right. >> > index() has start and stop arguments. Would anything be gained > if remove() had them too? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From zac256 at gmail.com Tue May 5 19:54:20 2009 From: zac256 at gmail.com (Zac Burns) Date: Tue, 5 May 2009 10:54:20 -0700 Subject: [Python-ideas] New list methods In-Reply-To: <333edbe80905051052p2f67e8ebv6aad5245ae5c0763@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <4A007C05.7090702@mrabarnett.plus.com> <333edbe80905051052p2f67e8ebv6aad5245ae5c0763@mail.gmail.com> Message-ID: <333edbe80905051054h1970fbf3v89f64c348274835a@mail.gmail.com> Sorry for previously top-posting, something I need to get used to. I'm +1 for the reversed parameter over 'r' methods. -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer (Digital Overlord) Zindagi Games From cmjohnson.mailinglist at gmail.com Tue May 5 22:20:31 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Tue, 5 May 2009 10:20:31 -1000 Subject: [Python-ideas] New list methods In-Reply-To: <333edbe80905051054h1970fbf3v89f64c348274835a@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <4A007C05.7090702@mrabarnett.plus.com> <333edbe80905051052p2f67e8ebv6aad5245ae5c0763@mail.gmail.com> <333edbe80905051054h1970fbf3v89f64c348274835a@mail.gmail.com> Message-ID: <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> Would there be any interest in adding a 'reversed' kwarg to the relevant string methods, deprecating the r-methods in Python 3.2, and removing them in Python 3.3? It might make things a little simpler and unclutter the dir for strings a bit? From google at mrabarnett.plus.com Tue May 5 22:40:56 2009 From: google at mrabarnett.plus.com (MRAB) Date: Tue, 05 May 2009 21:40:56 +0100 Subject: [Python-ideas] New list methods In-Reply-To: <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <4A007C05.7090702@mrabarnett.plus.com> <333edbe80905051052p2f67e8ebv6aad5245ae5c0763@mail.gmail.com> <333edbe80905051054h1970fbf3v89f64c348274835a@mail.gmail.com> <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> Message-ID: <4A00A458.3030001@mrabarnett.plus.com> Carl Johnson wrote: > Would there be any interest in adding a 'reversed' kwarg to the > relevant string methods, deprecating the r-methods in Python 3.2, and > removing them in Python 3.3? It might make things a little simpler and > unclutter the dir for strings a bit? > The keyword is 'reverse', not 'reversed'. From greg.ewing at canterbury.ac.nz Wed May 6 01:16:54 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 06 May 2009 11:16:54 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <20090503181307.226f095f@bhuda.mired.org> <49FF7913.5080406@canterbury.ac.nz> <49FF9238.7000608@canterbury.ac.nz> Message-ID: <4A00C8E6.5050305@canterbury.ac.nz> John Graham wrote: > In this case, continuation-passing-style would force capturing of this > state in the same way many 'functional-style' languages capture state, > immutable state in closures? Even if it is, I still don't see how the proposed TCO feature would help here, because it calls the supplied function *immediately*, which is not what you want to do. What you're talking about amounts to running the session as a coroutine, and what you want to do is suspend the coroutine until a particular query comes back from the browser. TCO won't help you with that. -- Greg From john.a.graham at gmail.com Wed May 6 01:28:03 2009 From: john.a.graham at gmail.com (John Graham) Date: Tue, 5 May 2009 18:28:03 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090505190805.544f4bff@bhuda.mired.org> References: <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505001653.6196a7b5@bhuda.mired.org> <20090505190805.544f4bff@bhuda.mired.org> Message-ID: > I think that a trampoline decorator is an acceptable substitute for > new syntax that indicates that a call is a tail call and should be > optimized as such. > > I'm still thinking about whether or not such a syntax is an acceptable > replacement for real TCO. Part of me says that explicit is better than > implicit, so having a programmer say "I believe this is a tail call" > and having the compiler flag them is better than having a system where > adding a comma can cause non-trivial changes in the semantics of the > code. On the other hand, another part believes that the benefits of > using an compiler are that the programmer shouldn't have to worry > about such optimizations, and should always get the benefit of them. I > think the former is going to win out, because if you're in a situation > where you have to have TCO, you have to take care to make sure you > write proper tail calls anyway. > I think it's best not to even think of it as an 'optimization', but rather just a different model of computing that the programmer would obviously need to know about. It changes too much behind the scenes to be 'just' an optimization in many people's eyes. -JG ps. work & school but i'll give you some raw code use cases eventually. From steve at pearwood.info Wed May 6 01:31:46 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 6 May 2009 09:31:46 +1000 Subject: [Python-ideas] New list methods In-Reply-To: <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <333edbe80905051054h1970fbf3v89f64c348274835a@mail.gmail.com> <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> Message-ID: <200905060931.46193.steve@pearwood.info> On Wed, 6 May 2009 06:20:31 am Carl Johnson wrote: > Would there be any interest in adding a 'reversed' kwarg to the > relevant string methods, deprecating the r-methods in Python 3.2, and > removing them in Python 3.3? It might make things a little simpler > and unclutter the dir for strings a bit? -0.5 from me. I think a "reversed" argument clutters up the signature of the various find() methods (etc) for little or no benefit, and certainly not enough benefit to make up for the hassle of going through the depreciation process and breaking compatibility with existing code. A "reversed" kwarg would only be useful when you don't know which direction you want to search in until runtime. How often does that happen? In practice, I believe you will nearly always know whether you want to search from the left or the right when you're writing the code. so I expect the "reversed" kwarg will nearly always be given as a constant: astring.find("spam", reversed=True) as opposed to: astring.find("spam", reversed=(today==Wednesday)) A slightly whimsical example, but I hope it illustrates the point. On the rare occasion that you do need a flag to distinguish "operate from the left" from "operate from the right", I don't believe it is a hardship to use something like the following idiom: def func(astring, arg, ... , reversed=False): if reversed: find = astring.find else: find = astring.rfind ... n = find(arg) -- Steven D'Aprano From greg.ewing at canterbury.ac.nz Wed May 6 01:35:04 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 06 May 2009 11:35:04 +1200 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <49FFFF55.5080602@improva.dk> References: <5d1a32000905021457y2a38d196o547426cde43a2965@mail.gmail.com> <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505014826.GA21593@panix.com> <49FFFF55.5080602@improva.dk> Message-ID: <4A00CD28.9070704@canterbury.ac.nz> Jacob Holm wrote: > Depth-first traversal of a graph is most easily expressed using > recursion... > > The proposed "continue " feature doesn't help these cases, > because they don't use tail calls. And that's exactly the point -- the algorithms to which TCO *can* be applied are precisely the ones that are not typically expressed using recursion in Python. -- Greg From george.sakkis at gmail.com Wed May 6 02:45:32 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Tue, 5 May 2009 20:45:32 -0400 Subject: [Python-ideas] New list methods In-Reply-To: <200905060931.46193.steve@pearwood.info> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <333edbe80905051054h1970fbf3v89f64c348274835a@mail.gmail.com> <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> <200905060931.46193.steve@pearwood.info> Message-ID: <91ad5bf80905051745m46d7a188nf3bacc18764a93f9@mail.gmail.com> On Tue, May 5, 2009 at 7:31 PM, Steven D'Aprano wrote: > On Wed, 6 May 2009 06:20:31 am Carl Johnson wrote: >> Would there be any interest in adding a 'reversed' kwarg to the >> relevant string methods, deprecating the r-methods in Python 3.2, and >> removing them in Python 3.3? It might make things a little simpler >> and unclutter the dir for strings a bit? > > A "reversed" kwarg would only be useful when you don't know which > direction you want to search in until runtime. How often does that > happen? In practice, I believe you will nearly always know whether you > want to search from the left or the right when you're writing the code. > so I expect the "reversed" kwarg will nearly always be given as a > constant: > > astring.find("spam", reversed=True) > > as opposed to: > > astring.find("spam", reversed=(today==Wednesday)) > > A slightly whimsical example, but I hope it illustrates the point. Sorry, that doesn't make much sense. Not only list.sort()/sorted() use exactly this API and we (thankfully) don't have list.rsort()/rsorted(), but that argument could be used against almost all functions with boolean parameters, since they are typically called with a constant True or False [1]. George [1] http://www.google.com/codesearch?hl=en&start=10&sa=N&filter=0p&q=lang:python+package:svn.python.org/projects/python/trunk+%3D(True|False) From jimjjewett at gmail.com Wed May 6 05:26:43 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 5 May 2009 23:26:43 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <200905031702.03363.steve@pearwood.info> Message-ID: On 5/3/09, Antoine Pitrou wrote: > Steven D'Aprano writes: >> It's hardly "premature" to notice that recursive code in Python is >> significantly slower and less efficient than in other languages. This >> is a known problem, or at least issue, since some people consider that >> the solution (tail-recursion optimization) is worse that the problem. > Is there some evidence that this "known issue" has been popping up in > real-world production Python code (1) The overhead of function calls (which are more common in a recursive style) is a real problem. That said, it isn't clear that Tail Call Optimization would make enough of a dent on the cost of calling a function. (2) The direct efficiency losses (particularly in space) show up every time an exception is thrown for excessive recursion depth. You could argue that this isn't common in production code, but that is because people learn to distort their code to avoid that limit. The counter-argument is that sometimes excessive depth is caused by a bug, and finding out sooner is a good thing. Guido (and Greg, earlier in this thread) have said that hitting the recursion limit is *usually* a bug. If that were true (and it might be), then tail call optimization *in python* would usually just mean "When I write an infinite loop, take out the breakpoints and debugging information." -jJ From jimjjewett at gmail.com Wed May 6 05:31:40 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 5 May 2009 23:31:40 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: <20090505154102.4ddcba4d@o> References: <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505001653.6196a7b5@bhuda.mired.org> <20090505154102.4ddcba4d@o> Message-ID: On 5/5/09, spir wrote: > Le Tue, 5 May 2009 07:29:22 -0500, > John Graham s'exprima ainsi: >> I had another proposal on the actual keyword front, >> "return from", which looks like it would kind of provide some symmetry >> to the 'return' and 'yield' constructs and also reads pretty >> intuitively, in my opinion. > Sounds good imo. > return --> the process does not go farther in the curent func > from --> there a detour Starting with a clean slate, I would agree. In practice, "return from" may already have too strong an association with its use in lisp-like languages. The real meaning here is closer to GOTO; I would expect a "return from" to pop several layers from the stack (rather that simply replacing the current one). -jJ From tjreedy at udel.edu Wed May 6 06:23:28 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 06 May 2009 00:23:28 -0400 Subject: [Python-ideas] lint in stdlib In-Reply-To: <4A005156.9010900@gmail.com> References: <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> Message-ID: Nick Coghlan wrote: > Terry Reedy wrote: >> Aahz wrote: >>> On Mon, May 04, 2009, Chris Rebert wrote: >>>>>>> On Mon, May 04, 2009, John Graham wrote: >>>>>>>> Has anyone ever suggested a standard pylint, something that might be >>>>>>>> distributed with the interpreter/libraries? ?As important as the >>>> Wouldn't it be be better placed in the Tools/ directory rather than as >>>> part of the stdlib? >>> Perhaps, but I don't think so -- you need a source tree to get Tools/ My standard Windows 3.0.1 installation has Tools without the source tree. It has 5 directories, including Scripts/, which has about 50 .py files, including 2to3. If *that* can go there, I see no reason why a syntax checker could not also. >>> AFAIK, and if some form of lint gets blessed by incorporation into >>> Python, then it needs to be available for all standard installs. >> Regardless, the stdlib is for modules to import, tools for programs that >> run. > > We blur that line all the time though (cf. timeit, pdb, pydoc, > webbrowser, runpy, probably others). > > So I'd agree with Aahz that if python-dev is going to bless something > along these lines, it should be something worthy of inclusion in the > standard lib itself rather than just being dropped into the Tools directory. timit, pdb, webbrowser, amd runpy are importable modules that *also* have a command-line interface via "if __name__ == '__main__': ...". Pydoc might be also, but I could not be sure from the doc. If the checker were written similarly, so that it could be imported and used from within a program to check another file, then I would agree that stdlib would be a place for it. tjr From tjreedy at udel.edu Wed May 6 06:27:18 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 06 May 2009 00:27:18 -0400 Subject: [Python-ideas] New list methods In-Reply-To: <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <4A007C05.7090702@mrabarnett.plus.com> <333edbe80905051052p2f67e8ebv6aad5245ae5c0763@mail.gmail.com> <333edbe80905051054h1970fbf3v89f64c348274835a@mail.gmail.com> <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> Message-ID: Carl Johnson wrote: > Would there be any interest in adding a 'reversed' kwarg to the > relevant string methods, deprecating the r-methods in Python 3.2, and > removing them in Python 3.3? It might make things a little simpler and > unclutter the dir for strings a bit? I personally would strongly prefer a reverse keyward and would not mind de-cluttering the current set of methods too. From debatem1 at gmail.com Wed May 6 06:48:46 2009 From: debatem1 at gmail.com (CTO) Date: Tue, 5 May 2009 21:48:46 -0700 (PDT) Subject: [Python-ideas] lint in stdlib In-Reply-To: References: <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> Message-ID: <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> [snip] > My standard Windows 3.0.1 installation has Tools without the source > tree. ?It has 5 directories, including Scripts/, which has about 50 .py > files, including 2to3. If *that* can go there, I see no reason why a > syntax checker could not also. > > >>> AFAIK, and if some form of lint gets blessed by incorporation into > >>> Python, then it needs to be available for all standard installs. > >> Regardless, the stdlib is for modules to import, tools for programs that > >> ?run. > > > We blur that line all the time though (cf. timeit, pdb, pydoc, > > webbrowser, runpy, probably others). > > > So I'd agree with Aahz that if python-dev is going to bless something > > along these lines, it should be something worthy of inclusion in the > > standard lib itself rather than just being dropped into the Tools directory. > > timit, pdb, webbrowser, amd runpy are importable modules that *also* > have a command-line interface via "if __name__ == '__main__': ...". > Pydoc might be also, but I could not be sure from the doc. ?If the > checker were written similarly, so that it could be imported and used > from within a program to check another file, then I would agree that > stdlib would be a place for it. > > tjr [snip] Skipping over some of the wrangling about where it goes for a minute, is there any firm consensus on which tool to put in? I've only heard back from pylint, and there seems to be a general feeling that that's the strictest (and therefore best), but that its external dependencies constitute a liability. Does anybody have a firm objection on that score? Geremy Condra From steve at pearwood.info Wed May 6 08:00:36 2009 From: steve at pearwood.info (steve at pearwood.info) Date: Wed, 6 May 2009 16:00:36 +1000 Subject: [Python-ideas] New list methods In-Reply-To: <91ad5bf80905051745m46d7a188nf3bacc18764a93f9@mail.gmail.com> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <200905060931.46193.steve@pearwood.info> <91ad5bf80905051745m46d7a188nf3bacc18764a93f9@mail.gmail.com> Message-ID: <200905061600.37083.steve@pearwood.info> On Wed, 6 May 2009 10:45:32 am George Sakkis wrote: > > A "reversed" kwarg would only be useful when you don't know which > > direction you want to search in until runtime. How often does > > that happen? In practice, I believe you will nearly always know > > whether you want to search from the left or the right when you're > > writing the code. so I expect the "reversed" kwarg will nearly > > always be given as a constant: > > > > astring.find("spam", reversed=True) > > > > as opposed to: > > > > astring.find("spam", reversed=(today==Wednesday)) > > > > A slightly whimsical example, but I hope it illustrates the > > point. > > Sorry, that doesn't make much sense. Not only list.sort()/sorted() > use exactly this API and we (thankfully) don't have > list.rsort()/rsorted(), list.sort() and sorted() are a very different situation. There was no depreciated reversesort() function to worry about when the reverse param was added. What we used to have is alist.sort() alist.reverse() For obvious reasons, that can't be written as alist.sort().reverse() Personally, I don't dislike the two-line version, but given that sorted() has a reverse flag, I accept that sort() should get one too for consistency. As for sorted(), remember that reversed() returns an iterator, not a list. So without a reverse param, you would need to write: list(reversed(sorted(alist))) but that's nasty for something as common as reversing a sorted list. > but that argument could be used against > almost all functions with boolean parameters, since they are > typically called with a constant True or False [1]. And I would exactly make that argument. What other built-in functions take a bool param to select between different functionality? sorted() is a special case. Anything else? I can't think of anything off the top of my head, but maybe that's just me. While I won't put words into Guido's mouth, I believe he has made that argument in the past. If your function takes a bool param to switch between two different behaviours, that's a good sign that it should be two functions rather than one. -- Steven D'Aprano From steve at pearwood.info Wed May 6 08:03:50 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 6 May 2009 16:03:50 +1000 Subject: [Python-ideas] New list methods In-Reply-To: References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> Message-ID: <200905061603.50567.steve@pearwood.info> On Wed, 6 May 2009 02:27:18 pm Terry Reedy wrote: > Carl Johnson wrote: > > Would there be any interest in adding a 'reversed' kwarg to the > > relevant string methods, deprecating the r-methods in Python 3.2, > > and removing them in Python 3.3? It might make things a little > > simpler and unclutter the dir for strings a bit? > > I personally would strongly prefer a reverse keyward and would not > mind de-cluttering the current set of methods too. Do you have any use-cases where you don't know whether you want forward or reverse search until runtime? That is, where you currently write something like: if some_var: n = astring.find(target) else: n = astring.rfind(target) or equivalent? -- Steven D'Aprano From ben+python at benfinney.id.au Wed May 6 09:14:08 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 06 May 2009 17:14:08 +1000 Subject: [Python-ideas] lint in stdlib References: <20090502225735.GB5084@panix.com> <20090503031633.3d1663a9@o> <87eiv6ivtn.fsf@uwakimon.sk.tsukuba.ac.jp> <20090504131024.GA25783@panix.com> <20090504202348.GA5402@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> Message-ID: <8763geitz3.fsf@benfinney.id.au> CTO writes: > Skipping over some of the wrangling about where it goes for a minute, > is there any firm consensus on which tool to put in? I've only heard > back from pylint, and there seems to be a general feeling that that's > the strictest (and therefore best), but that its external dependencies > constitute a liability. Does anybody have a firm objection on that > score? Not a firm objection: pylint is also the most complex (AFAICT). Perhaps a simpler one that still does a good job would be best for stdlib consideration. -- \ ?To save the world requires faith and courage: faith in reason, | `\ and courage to proclaim what reason shows to be true.? | _o__) ?Bertrand Russell | Ben Finney From john.a.graham at gmail.com Wed May 6 14:27:02 2009 From: john.a.graham at gmail.com (John Graham) Date: Wed, 6 May 2009 07:27:02 -0500 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505001653.6196a7b5@bhuda.mired.org> <20090505154102.4ddcba4d@o> Message-ID: On Tue, May 5, 2009 at 10:31 PM, Jim Jewett wrote: > On 5/5/09, spir wrote: >> Le Tue, 5 May 2009 07:29:22 -0500, >> John Graham s'exprima ainsi: > >>> I had another proposal on the actual keyword front, >>> "return from", which looks like it would kind of provide some symmetry >>> to the 'return' and 'yield' constructs and also reads pretty >>> intuitively, in my opinion. > >> Sounds good imo. >> return --> the process does not go farther in the curent func >> from ? --> there a detour > > Starting with a clean slate, I would agree. > > In practice, "return from" may already have too strong an association > with its use in lisp-like languages. ?The real meaning here is closer > to GOTO; I would expect a "return from" to pop several layers from the > stack (rather that simply replacing the current one). > > -jJ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > Embrace, extend, destroy. It's not enough we take tail calls from the schemers, we'll teach a new generation of kids to think Lisp is even weirder than it is by reusing their terms! :) From john.a.graham at gmail.com Wed May 6 14:32:50 2009 From: john.a.graham at gmail.com (John Graham) Date: Wed, 6 May 2009 07:32:50 -0500 Subject: [Python-ideas] New list methods In-Reply-To: <200905061603.50567.steve@pearwood.info> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> <200905061603.50567.steve@pearwood.info> Message-ID: On Wed, May 6, 2009 at 1:03 AM, Steven D'Aprano wrote: > On Wed, 6 May 2009 02:27:18 pm Terry Reedy wrote: >> Carl Johnson wrote: >> > Would there be any interest in adding a 'reversed' kwarg to the >> > relevant string methods, deprecating the r-methods in Python 3.2, >> > and removing them in Python 3.3? It might make things a little >> > simpler and unclutter the dir for strings a bit? >> >> I personally would strongly prefer a reverse keyward and would not >> mind de-cluttering the current set of methods too. > > Do you have any use-cases where you don't know whether you want > forward or reverse search until runtime? > > That is, where you currently write something like: > > if some_var: > ? ?n = astring.find(target) > else: > ? ?n = astring.rfind(target) > > or equivalent? > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > I may be over architecting, but the combination of a separate function to distinguish between two different behaviors, especially something as cross cutting as doing something in reverse, seems a lot like a builder pattern. Something like the following, though I doubt the following syntax would be seen at all as pretty: lst.reversed().find(target) #rfind lst.find(target) #left find you can already do this with the following reversed(lst).find(target) but this is pretty inefficient. the purpose of a builder pattern like series of functions to 'configure' whether you want to search from the left or right would be that the 'reversed()' function in the first example wouldn't actually reverse the list, but instead reverse which side the next function that was called operated from. I don't know if that's really that possible within the language. -JG From jnoller at gmail.com Wed May 6 14:50:21 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 6 May 2009 08:50:21 -0400 Subject: [Python-ideas] lint in stdlib In-Reply-To: <8763geitz3.fsf@benfinney.id.au> References: <20090502225735.GB5084@panix.com> <90ed0f8b-d5ca-43f7-8928-13c0e6b7143d@g20g2000vba.googlegroups.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> <8763geitz3.fsf@benfinney.id.au> Message-ID: <4222a8490905060550g45df67d2s9c4e258f7e9bbb86@mail.gmail.com> On Wed, May 6, 2009 at 3:14 AM, Ben Finney wrote: > CTO writes: > >> Skipping over some of the wrangling about where it goes for a minute, >> is there any firm consensus on which tool to put in? I've only heard >> back from pylint, and there seems to be a general feeling that that's >> the strictest (and therefore best), but that its external dependencies >> constitute a liability. Does anybody have a firm objection on that >> score? > > Not a firm objection: pylint is also the most complex (AFAICT). Perhaps > a simpler one that still does a good job would be best for stdlib > consideration. +1, something which does simple linting, and pep 8 checking would be a big enough benefit over what we have now. Ideally, it would support extensibility for additional rules. jesse From google at mrabarnett.plus.com Wed May 6 15:00:00 2009 From: google at mrabarnett.plus.com (MRAB) Date: Wed, 06 May 2009 14:00:00 +0100 Subject: [Python-ideas] New list methods In-Reply-To: References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <3bdda690905051320kf1a4766r8c290534439a162e@mail.gmail.com> <200905061603.50567.steve@pearwood.info> Message-ID: <4A0189D0.90303@mrabarnett.plus.com> John Graham wrote: > On Wed, May 6, 2009 at 1:03 AM, Steven D'Aprano wrote: >> On Wed, 6 May 2009 02:27:18 pm Terry Reedy wrote: >>> Carl Johnson wrote: >>>> Would there be any interest in adding a 'reversed' kwarg to the >>>> relevant string methods, deprecating the r-methods in Python 3.2, >>>> and removing them in Python 3.3? It might make things a little >>>> simpler and unclutter the dir for strings a bit? >>> I personally would strongly prefer a reverse keyward and would not >>> mind de-cluttering the current set of methods too. >> Do you have any use-cases where you don't know whether you want >> forward or reverse search until runtime? >> >> That is, where you currently write something like: >> >> if some_var: >> n = astring.find(target) >> else: >> n = astring.rfind(target) >> >> or equivalent? >> > > I may be over architecting, but the combination of a separate function > to distinguish between two different behaviors, especially something > as cross cutting as doing something in reverse, seems a lot like a > builder pattern. Something like the following, though I doubt the > following syntax would be seen at all as pretty: > > lst.reversed().find(target) #rfind > lst.find(target) #left find > > you can already do this with the following > > reversed(lst).find(target) > > but this is pretty inefficient. the purpose of a builder pattern like > series of functions to 'configure' whether you want to search from the > left or right would be that the 'reversed()' function in the first > example wouldn't actually reverse the list, but instead reverse which > side the next function that was called operated from. > > I don't know if that's really that possible within the language. > Wouldn't it be like using 'find' on an enumerated iterator, returning the index where a match was found? From gerald.britton at gmail.com Wed May 6 15:38:29 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Wed, 6 May 2009 09:38:29 -0400 Subject: [Python-ideas] A Continuations Compromise in Python In-Reply-To: References: <20090503181307.226f095f@bhuda.mired.org> <20090504191646.375875cd@bhuda.mired.org> <20090505001653.6196a7b5@bhuda.mired.org> <20090505154102.4ddcba4d@o> Message-ID: <5d1a32000905060638g7920a5eek1f9f38fe2dc0f569@mail.gmail.com> Perhaps we should implement "come from" and "go to" while we're at it. Oh, let's not leave out "alter" (for those of you old enough to have used COBOL) as well! On Tue, May 5, 2009 at 11:31 PM, Jim Jewett wrote: > On 5/5/09, spir wrote: >> Le Tue, 5 May 2009 07:29:22 -0500, >> John Graham s'exprima ainsi: > >>> I had another proposal on the actual keyword front, >>> "return from", which looks like it would kind of provide some symmetry >>> to the 'return' and 'yield' constructs and also reads pretty >>> intuitively, in my opinion. > >> Sounds good imo. >> return --> the process does not go farther in the curent func >> from ? --> there a detour > > Starting with a clean slate, I would agree. > > In practice, "return from" may already have too strong an association > with its use in lisp-like languages. ?The real meaning here is closer > to GOTO; I would expect a "return from" to pop several layers from the > stack (rather that simply replacing the current one). > > -jJ > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From konryd at gmail.com Wed May 6 15:55:17 2009 From: konryd at gmail.com (Konrad Delong) Date: Wed, 6 May 2009 15:55:17 +0200 Subject: [Python-ideas] Coloured documentation Message-ID: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> The documentation under http://docs.python.org/ could have different colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way one would know on sight which docs one's reading. It would also cause a lot of bikeshed talk for every release :) Konrad From jeremy at jeremybanks.ca Wed May 6 16:01:43 2009 From: jeremy at jeremybanks.ca (Jeremy Banks) Date: Wed, 6 May 2009 11:01:43 -0300 Subject: [Python-ideas] Coloured documentation In-Reply-To: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> Message-ID: <9e754ef50905060701h5c99a39cvd328693fcecaa112@mail.gmail.com> I'm +1 if it could be implemented without much hassle. Perhaps the bikeshedding could be minimized by deciding in advance on how the colors would be calculated. Something like... R = (MAJOR + MINOR / 16) / 5 G = 1 - ((MAJOR + MINOR / 16) / 5) B = REVISION / 16 So the change would be more dramatic between larger changes. On 2009-05-06, Konrad Delong wrote: > The documentation under http://docs.python.org/ could have different > colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way > one would know on sight which docs one's reading. > It would also cause a lot of bikeshed talk for every release :) > > Konrad > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From sridharr at activestate.com Wed May 6 19:22:50 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Wed, 06 May 2009 10:22:50 -0700 Subject: [Python-ideas] Coloured documentation In-Reply-To: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> Message-ID: <4A01C76A.7030809@activestate.com> On 09-05-06 06:55 AM, Konrad Delong wrote: > The documentation underhttp://docs.python.org/ could have different > colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way > one would know on sight which docs one's reading. Good idea, although I'd rather have a vertical bar instead of color schemes. For example, the vertical bar here, http://www.w3.org/TR/2009/PER-xpath-functions-20090421/ that reads "W3V Proposed Edited Recommendation" could read "2.6" in slightly bigger font for the Python docs. From debatem1 at gmail.com Wed May 6 20:03:27 2009 From: debatem1 at gmail.com (CTO) Date: Wed, 6 May 2009 11:03:27 -0700 (PDT) Subject: [Python-ideas] Coloured documentation In-Reply-To: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> Message-ID: <149c566b-f888-4a83-afb4-077046a21642@h23g2000vbc.googlegroups.com> On May 6, 9:55?am, Konrad Delong wrote: > The documentation underhttp://docs.python.org/could have different > colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way > one would know on sight which docs one's reading. > It would also cause a lot of bikeshed talk for every release :) > > Konrad Great idea, I get bitten by this all the time. Also looks like its just a simple tweak- . Geremy Condra From jeremy at alum.mit.edu Wed May 6 20:43:14 2009 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Wed, 6 May 2009 14:43:14 -0400 Subject: [Python-ideas] lint in stdlib In-Reply-To: <4222a8490905060550g45df67d2s9c4e258f7e9bbb86@mail.gmail.com> References: <20090502225735.GB5084@panix.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> <8763geitz3.fsf@benfinney.id.au> <4222a8490905060550g45df67d2s9c4e258f7e9bbb86@mail.gmail.com> Message-ID: On Wed, May 6, 2009 at 8:50 AM, Jesse Noller wrote: > On Wed, May 6, 2009 at 3:14 AM, Ben Finney wrote: >> CTO writes: >> >>> Skipping over some of the wrangling about where it goes for a minute, >>> is there any firm consensus on which tool to put in? I've only heard >>> back from pylint, and there seems to be a general feeling that that's >>> the strictest (and therefore best), but that its external dependencies >>> constitute a liability. Does anybody have a firm objection on that >>> score? >> >> Not a firm objection: pylint is also the most complex (AFAICT). Perhaps >> a simpler one that still does a good job would be best for stdlib >> consideration. > > +1, something which does simple linting, and pep 8 checking would be a > big enough benefit over what we have now. Ideally, it would support > extensibility for additional rules. I don't see much value in building a new lint tool. I might like it if pylint were less complex than it is, but I don't understand the design fully enough to have justification for that. If we can get a good PEP 8 checker out of pylint, it would be silly to write something from scratch. We went through a similar exercise at work two summers ago and concluded that we'd it would be faster to build on pylint than to write something from scratch. I don't think there are any special considerations for python stdlib development that would lead me to reach a different conclusion. Certainly, it would be convenient if the lint tool was part of the source tree, but our editors, compilers, debuggers, and source control tools aren't part of the source tree either. I don't think the desire to put it in tools or stdlib outweighs the cost of building and maintaining a new tool when a perfectly good one already exists. Jeremy From santagada at gmail.com Wed May 6 21:30:36 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 6 May 2009 16:30:36 -0300 Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A01C76A.7030809@activestate.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> Message-ID: <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> On May 6, 2009, at 2:22 PM, Sridhar Ratnakumar wrote: > On 09-05-06 06:55 AM, Konrad Delong wrote: >> The documentation underhttp://docs.python.org/ could have different >> colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way >> one would know on sight which docs one's reading. > > Good idea, although I'd rather have a vertical bar instead of color > schemes. For example, the vertical bar here, > > http://www.w3.org/TR/2009/PER-xpath-functions-20090421/ > > that reads "W3V Proposed Edited Recommendation" could read "2.6" in > slightly bigger font for the Python docs. I'm +1 for the color bar idea like w3c and also we should just choose a color palette and then jus choose a color for each new release of python (and of course the back ones). No color formulas based on versions please (this would just end up in ugly colors and no one knowing how to decode the version from the color anyway). -- Leonardo Santagada santagada at gmail.com From debatem1 at gmail.com Wed May 6 21:44:46 2009 From: debatem1 at gmail.com (CTO) Date: Wed, 6 May 2009 12:44:46 -0700 (PDT) Subject: [Python-ideas] lint in stdlib In-Reply-To: References: <20090502225735.GB5084@panix.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> <8763geitz3.fsf@benfinney.id.au> <4222a8490905060550g45df67d2s9c4e258f7e9bbb86@mail.gmail.com> Message-ID: On May 6, 2:43?pm, Jeremy Hylton wrote: > On Wed, May 6, 2009 at 8:50 AM, Jesse Noller wrote: > > On Wed, May 6, 2009 at 3:14 AM, Ben Finney wrote: > >> CTO writes: > > >>> Skipping over some of the wrangling about where it goes for a minute, > >>> is there any firm consensus on which tool to put in? I've only heard > >>> back from pylint, and there seems to be a general feeling that that's > >>> the strictest (and therefore best), but that its external dependencies > >>> constitute a liability. Does anybody have a firm objection on that > >>> score? > > >> Not a firm objection: pylint is also the most complex (AFAICT). Perhaps > >> a simpler one that still does a good job would be best for stdlib > >> consideration. > > > +1, something which does simple linting, and pep 8 checking would be a > > big enough benefit over what we have now. Ideally, it would support > > extensibility for additional rules. > > I don't see much value in building a new lint tool. ?I might like it > if pylint were less complex than it is, but I don't understand the > design fully enough to have justification for that. ?If we can get a > good PEP 8 checker out of pylint, it would be silly to write something > from scratch. ?We went through a similar exercise at work two summers > ago and concluded that we'd it would be faster to build on pylint than > to write something from scratch. ?I don't think there are any special > considerations for python stdlib development that would lead me to > reach a different conclusion. ?Certainly, it would be convenient if > the lint tool was part of the source tree, but our editors, compilers, > debuggers, and source control tools aren't part of the source tree > either. ?I don't think the desire to put it in tools or stdlib > outweighs the cost of building and maintaining a new tool when a > perfectly good one already exists. Should we give another look to something like pep8.py, then? Less complex, certainly, no dependencies and it runs cleanly under python3 after running it through 2to3. On the minus side, it is *much* less rigorous. Geremy Condra From g.brandl at gmx.net Wed May 6 21:44:58 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 06 May 2009 21:44:58 +0200 Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A01C76A.7030809@activestate.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> Message-ID: Sridhar Ratnakumar schrieb: > On 09-05-06 06:55 AM, Konrad Delong wrote: >> The documentation underhttp://docs.python.org/ could have different >> colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way >> one would know on sight which docs one's reading. > > Good idea, although I'd rather have a vertical bar instead of color > schemes. For example, the vertical bar here, > > http://www.w3.org/TR/2009/PER-xpath-functions-20090421/ > > that reads "W3V Proposed Edited Recommendation" could read "2.6" in > slightly bigger font for the Python docs. First, it's not like the docs' version is not displayed on every page at the top and bottom :) Still, I can see that it would be helpful if it was kept visible all the time. I've experimented with some CSS to make the top navigation bar sticky; however I could not find a way to do this without the heading being hidden when jumping to an anchor. If someone has a patch, please step forward, but please let's put this discussion on the doc-SIG. cheers, Georg From google at mrabarnett.plus.com Wed May 6 21:51:21 2009 From: google at mrabarnett.plus.com (MRAB) Date: Wed, 06 May 2009 20:51:21 +0100 Subject: [Python-ideas] Coloured documentation In-Reply-To: <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> Message-ID: <4A01EA39.9010408@mrabarnett.plus.com> Leonardo Santagada wrote: > > On May 6, 2009, at 2:22 PM, Sridhar Ratnakumar wrote: > >> On 09-05-06 06:55 AM, Konrad Delong wrote: >>> The documentation underhttp://docs.python.org/ could have different >>> colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way >>> one would know on sight which docs one's reading. >> >> Good idea, although I'd rather have a vertical bar instead of color >> schemes. For example, the vertical bar here, >> >> http://www.w3.org/TR/2009/PER-xpath-functions-20090421/ >> >> that reads "W3V Proposed Edited Recommendation" could read "2.6" in >> slightly bigger font for the Python docs. > > I'm +1 for the color bar idea like w3c and also we should just choose a > color palette and then jus choose a color for each new release of python > (and of course the back ones). No color formulas based on versions > please (this would just end up in ugly colors and no one knowing how to > decode the version from the color anyway). > I was thinking of running through the spectrum from Python 1.0 red through Python 2.0 green to Python 3.0 blue. Not sure what happens when we get to ultraviolet. New monitors? :-) From g.brandl at gmx.net Wed May 6 21:56:17 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 06 May 2009 21:56:17 +0200 Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A01EA39.9010408@mrabarnett.plus.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> <4A01EA39.9010408@mrabarnett.plus.com> Message-ID: MRAB schrieb: > Leonardo Santagada wrote: >> >> On May 6, 2009, at 2:22 PM, Sridhar Ratnakumar wrote: >> >>> On 09-05-06 06:55 AM, Konrad Delong wrote: >>>> The documentation underhttp://docs.python.org/ could have different >>>> colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way >>>> one would know on sight which docs one's reading. >>> >>> Good idea, although I'd rather have a vertical bar instead of color >>> schemes. For example, the vertical bar here, >>> >>> http://www.w3.org/TR/2009/PER-xpath-functions-20090421/ >>> >>> that reads "W3V Proposed Edited Recommendation" could read "2.6" in >>> slightly bigger font for the Python docs. >> >> I'm +1 for the color bar idea like w3c and also we should just choose a >> color palette and then jus choose a color for each new release of python >> (and of course the back ones). No color formulas based on versions >> please (this would just end up in ugly colors and no one knowing how to >> decode the version from the color anyway). >> > I was thinking of running through the spectrum from Python 1.0 red > through Python 2.0 green to Python 3.0 blue. Not sure what happens when > we get to ultraviolet. New monitors? :-) Please no. The docs are already considered a bit too colorful by some people. (Also, Python 1.0 is green and Python 2.0 is blue.) Georg From yoavglazner at gmail.com Wed May 6 22:08:56 2009 From: yoavglazner at gmail.com (yoav glazner) Date: Wed, 6 May 2009 23:08:56 +0300 Subject: [Python-ideas] [Value Returning Threard] Message-ID: Hi all, My Idea is to have a value returning Thread. I'll explain by example. def foo(): time.sleep(20) return 'bar' value = thread.startValueReturningThread(foo) #i need a better name for the function...) #here we do some work mg = moonGravity() mg.disable() #now we need the value that foo returned print value #this would be blocking untill foo is done! This feature should provide a way to increase performance when possible with simple syntax. What do you think? -------------- next part -------------- An HTML attachment was scrubbed... URL: From pyideas at rebertia.com Wed May 6 22:41:09 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Wed, 6 May 2009 13:41:09 -0700 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: References: Message-ID: <50697b2c0905061341g230a96bar2440319bdf197c52@mail.gmail.com> On Wed, May 6, 2009 at 1:08 PM, yoav glazner wrote: > Hi all, > My Idea is to have a value returning Thread. > I'll?explain?by example. > > def foo(): > ?? ?time.sleep(20) > ?? ?return 'bar' > value = thread.startValueReturningThread(foo) #i need a better name for the > function...) > #here we do some work > mg = moonGravity() > mg.disable() > #now we need the value that foo returned > print value #this would be blocking untill foo is done! > > This feature should provide a way to increase?performance?when possible with > simple syntax. > What do you think? Sounds like you haven't heard of Futures - http://en.wikipedia.org/wiki/Future_(programming) One of probably many recipes for them in Python: http://code.activestate.com/recipes/84317/ Cheers, Chris -- http://blog.rebertia.com From debatem1 at gmail.com Wed May 6 22:43:23 2009 From: debatem1 at gmail.com (CTO) Date: Wed, 6 May 2009 13:43:23 -0700 (PDT) Subject: [Python-ideas] lint in stdlib In-Reply-To: References: <20090502225735.GB5084@panix.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> <8763geitz3.fsf@benfinney.id.au> <4222a8490905060550g45df67d2s9c4e258f7e9bbb86@mail.gmail.com> Message-ID: <6c94696d-9d21-4444-afc6-ef59016a6f6b@s31g2000vbp.googlegroups.com> > Should we give another look to something like pep8.py, then? Less > complex, certainly, no dependencies and it runs cleanly under python3 > after running it through 2to3. On the minus side, it is *much* less > rigorous. > > Geremy Condra I've also just heard back from the maintainer of pyflakes, and he seems pretty happy with the idea. Here's the email: Geremy, I maintain Pyflakes and am an employee of the company responsible for its development. I think it's a good idea; Pyflakes is definitely an important part of my development process. There are pretty diverse opinions about how thorough and punctilious Python code checkers should try to be, so there are a bunch of relatively popular programs with vaguely similar functionality. I'm a fan of Pyflakes in particular because it's quick, passive and not overly ambitious or presumptious. Let me know how things work out. Take care, Moe So, are there any particular objections to pyflakes? Does anybody have a strong preference for pylint, pep8.py, or pychecker over pyflakes? Geremy Condra From george.sakkis at gmail.com Wed May 6 22:49:45 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 6 May 2009 16:49:45 -0400 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: References: Message-ID: <91ad5bf80905061349w6d68c527lc3c46df9b5e115dd@mail.gmail.com> On Wed, May 6, 2009 at 4:08 PM, yoav glazner wrote: > Hi all, > My Idea is to have a value returning Thread. > I'll?explain?by example. > > def foo(): > ?? ?time.sleep(20) > ?? ?return 'bar' > value = thread.startValueReturningThread(foo) #i need a better name for the > function...) > #here we do some work > mg = moonGravity() > mg.disable() > #now we need the value that foo returned > print value #this would be blocking untill foo is done! > > This feature should provide a way to increase?performance?when possible with > simple syntax. > What do you think? You're looking for the threadmethod decorator [1]. I'm not sure it's robust and useful enough to be included in the standard library though. George [1] http://code.activestate.com/recipes/440569/ From google at mrabarnett.plus.com Wed May 6 22:51:47 2009 From: google at mrabarnett.plus.com (MRAB) Date: Wed, 06 May 2009 21:51:47 +0100 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: References: Message-ID: <4A01F863.1030004@mrabarnett.plus.com> yoav glazner wrote: > Hi all, > > My Idea is to have a value returning Thread. > I'll explain by example. > > def foo(): > time.sleep(20) > return 'bar' > > value = thread.startValueReturningThread(foo) #i need a better name for > the function...) > > #here we do some work > mg = moonGravity() > mg.disable() > > #now we need the value that foo returned > print value #this would be blocking untill foo is done! > > > This feature should provide a way to increase performance when possible > with simple syntax. > > What do you think? > I think it would be better if startValueReturningThread() returned what we could call a 'deferred result' object: result = thread.startValueReturningThread(foo) If you requested the actual result before it was available then it would wait: print result.value() # or 'result.value'? It could be implemented using a thread and a queue. From solipsis at pitrou.net Wed May 6 23:02:02 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 6 May 2009 21:02:02 +0000 (UTC) Subject: [Python-ideas] [Value Returning Threard] References: <4A01F863.1030004@mrabarnett.plus.com> Message-ID: MRAB writes: > > I think it would be better if startValueReturningThread() returned what > we could call a 'deferred result' object: Or perhaps you just want to use Twisted. Regards Antoine. From aahz at pythoncraft.com Wed May 6 23:02:21 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 6 May 2009 14:02:21 -0700 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: References: Message-ID: <20090506210221.GA25517@panix.com> On Wed, May 06, 2009, yoav glazner wrote: > > #now we need the value that foo returned > print value #this would be blocking untill foo is done! > > > This feature should provide a way to increase performance when possible with > simple syntax. Please provide more explanation for why the currently available features are insufficient. Note that what you're asking for would change the semantics of Python at a very deep level and will almost certainly be rejected in its present form. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From ben+python at benfinney.id.au Thu May 7 01:47:42 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 May 2009 09:47:42 +1000 Subject: [Python-ideas] lint in stdlib References: <20090502225735.GB5084@panix.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> <8763geitz3.fsf@benfinney.id.au> <4222a8490905060550g45df67d2s9c4e258f7e9bbb86@mail.gmail.com> Message-ID: <87r5z1hjz5.fsf@benfinney.id.au> Jeremy Hylton writes: > On Wed, May 6, 2009 at 8:50 AM, Jesse Noller wrote: > > On Wed, May 6, 2009 at 3:14 AM, Ben Finney wrote: > >> Not a firm objection: pylint is also the most complex (AFAICT). > >> Perhaps a simpler one that still does a good job would be best for > >> stdlib consideration. > > > > +1, something which does simple linting, and pep 8 checking would be > > a big enough benefit over what we have now. Ideally, it would > > support extensibility for additional rules. > > I don't see much value in building a new lint tool. I might like it > if pylint were less complex than it is, but I don't understand the > design fully enough to have justification for that. If we can get a > good PEP 8 checker out of pylint, it would be silly to write something > from scratch. AFAIK, nobody is suggesting writing one from scratch; earlier in this thread we explored the fact that there are several already written and working. -- \ ?Always code as if the guy who ends up maintaining your code | `\ will be a violent psychopath who knows where you live.? ?John | _o__) F. Woods | Ben Finney From ben+python at benfinney.id.au Thu May 7 01:49:31 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 07 May 2009 09:49:31 +1000 Subject: [Python-ideas] lint in stdlib References: <20090502225735.GB5084@panix.com> <20090504231533.GB6215@panix.com> <50697b2c0905041725ib0454bfr75fa41b3f3240d69@mail.gmail.com> <20090505012527.GA1358@panix.com> <4A005156.9010900@gmail.com> <15aff1eb-e68a-4a34-9466-8567e63fe805@j12g2000vbl.googlegroups.com> <8763geitz3.fsf@benfinney.id.au> <4222a8490905060550g45df67d2s9c4e258f7e9bbb86@mail.gmail.com> <6c94696d-9d21-4444-afc6-ef59016a6f6b@s31g2000vbp.googlegroups.com> Message-ID: <87my9phjw4.fsf@benfinney.id.au> CTO writes: > So, are there any particular objections to pyflakes? Does anybody have > a strong > preference for pylint, pep8.py, or pychecker over pyflakes? I'm happy with focussing on pyflakes; it's the one I use most often. I've tried to love pylint, but configuring it so it's useful per project was beyond me. -- \ ?If you wish to strive for peace of soul, then believe; if you | `\ wish to be a devotee of truth, then inquire.? ?Friedrich | _o__) Nietzsche | Ben Finney From john.a.graham at gmail.com Thu May 7 02:23:27 2009 From: john.a.graham at gmail.com (John Graham) Date: Wed, 6 May 2009 19:23:27 -0500 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: <20090506210221.GA25517@panix.com> References: <20090506210221.GA25517@panix.com> Message-ID: On Wed, May 6, 2009 at 4:02 PM, Aahz wrote: > On Wed, May 06, 2009, yoav glazner wrote: >> >> #now we need the value that foo returned >> print value #this would be blocking untill foo is done! >> >> >> This feature should provide a way to increase performance when possible with >> simple syntax. > > Please provide more explanation for why the currently available features > are insufficient. ?Note that what you're asking for would change the > semantics of Python at a very deep level and will almost certainly be > rejected in its present form. > -- > Aahz (aahz at pythoncraft.com) ? ? ? ? ? <*> ? ? ? ? http://www.pythoncraft.com/ > > "It is easier to optimize correct code than to correct optimized code." > --Bill Harlan > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > If the OP is asking for futures, couldn't that be implemented in a library without touching the language? It's a powerful enough feature to consider putting it in a library. It's going to be one of the 'primitives' of C++0x if one wanted a precedent (although I'm not suggesting everything C++ does is a good thing. Or even most of what C++ is a good thing). From greg.ewing at canterbury.ac.nz Thu May 7 02:37:33 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 07 May 2009 12:37:33 +1200 Subject: [Python-ideas] New list methods In-Reply-To: <200905061600.37083.steve@pearwood.info> References: <333edbe80905050925h7205ad19gfde591afc30701b5@mail.gmail.com> <200905060931.46193.steve@pearwood.info> <91ad5bf80905051745m46d7a188nf3bacc18764a93f9@mail.gmail.com> <200905061600.37083.steve@pearwood.info> Message-ID: <4A022D4D.8000506@canterbury.ac.nz> steve at pearwood.info wrote: > What other built-in functions take a bool param to select between > different functionality? sorted() is a special case. Another thing about sort() is that reversed=True is equivalent to passing a key function that results in reversed ordering, so it's easy to see it as a parameterization of the existing behaviour. That's not the case with index() and remove() -- there's nothing you can pass to the existing methods that will cause them to operate in reverse. -- Greg From greg.ewing at canterbury.ac.nz Thu May 7 03:32:08 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 07 May 2009 13:32:08 +1200 Subject: [Python-ideas] Coloured documentation In-Reply-To: <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> Message-ID: <4A023A18.9080103@canterbury.ac.nz> Leonardo Santagada wrote: > No color formulas based on > versions please (this would just end up in ugly colors and no one > knowing how to decode the version from the color anyway). Instead of just one colour, parts of the page could be coloured according to different parts of the version number, e.g. header from the major version and sidebar from the minor version, with maybe a stripe somewhere for the revision. The digits could be encoded using the standard resistor colour code: 0 - black 1 - brown 2 - red 3 - orange 4 - yellow 5 - green 6 - blue 7 - purple 8 - grey 9 - white Then there would be no problem recovering the version number from the colour scheme. -- Greg From greg.ewing at canterbury.ac.nz Thu May 7 03:37:46 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 07 May 2009 13:37:46 +1200 Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A01EA39.9010408@mrabarnett.plus.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> <4A01EA39.9010408@mrabarnett.plus.com> Message-ID: <4A023B6A.50303@canterbury.ac.nz> MRAB wrote: > I was thinking of running through the spectrum from Python 1.0 red > through Python 2.0 green to Python 3.0 blue. Not sure what happens when > we get to ultraviolet. New monitors? :-) So for 3.2 we'll need sunblock, for and 3.5 we'll need welding goggles... X-ray shielding for 4.0? -- Greg From santagada at gmail.com Thu May 7 04:38:50 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 6 May 2009 23:38:50 -0300 Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A023A18.9080103@canterbury.ac.nz> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> <4A023A18.9080103@canterbury.ac.nz> Message-ID: <6BD3A635-9CD4-4716-BB25-1587849BE193@gmail.com> On May 6, 2009, at 10:32 PM, Greg Ewing wrote: > Leonardo Santagada wrote: > >> No color formulas based on versions please (this would just end up >> in ugly colors and no one knowing how to decode the version from >> the color anyway). > > Instead of just one colour, parts of the page > could be coloured according to different parts of > the version number, e.g. header from the major > version and sidebar from the minor version, with > maybe a stripe somewhere for the revision. > > The digits could be encoded using the standard > resistor colour code: > > 0 - black > 1 - brown > 2 - red > 3 - orange > 4 - yellow > 5 - green > 6 - blue > 7 - purple > 8 - grey > 9 - white > > Then there would be no problem recovering the > version number from the colour scheme. Without an emoticon I can't know for certain if this is a joke (but I'm pretty sure it is). (maybe an even funnier one would be to put a sidebar with the version written all over it vertically, forming a texture). The thing is that having a border (maybe just on the right side) on the online version of the docs help to give people guidance, if they are used to the python 2.6 docs they will know when something is changed when looking at the 3.1 docs. But maybe one color for each version is too much, maybe just one color for the 2.x series and another for the 3.x series, maybe people who had this problem more than I did can say if they got confused around different 2.x versions or only 3 to 2 diffs. -- Leonardo Santagada santagada at gmail.com From yoavglazner at gmail.com Thu May 7 08:10:40 2009 From: yoavglazner at gmail.com (yoav glazner) Date: Thu, 7 May 2009 09:10:40 +0300 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: References: <20090506210221.GA25517@panix.com> Message-ID: > > > >Please provide more explanation for why the currently available features > >are insufficient. Note that what you're asking for would change the > >semantics of Python at a very deep level and will almost certainly be > >rejected in its present form. > -- > >Aahz (aahz at pythoncraft.com) <*> > http://www.pythoncraft.com/ > I'm no python expert but i had to do a stress test on a libary. (checking thread safety) here is the code (we soe absraction): def testMultiThread(): global resultDict resultDict = {} def testFuncThread(func): global resultDict if func(): resultDict[func] = 'Passed' else: resultDict[func] = 'Falied' thread.start_new(testFuncThread,(Test1,)) thread.start_new(testFuncThread,(Test2,)) thread.start_new(testFuncThread,(Test3,)) while (len(resultDict) < 3): time.sleep(5) return resultDict #here i did something else such as logging and returning True or False Maybe there is a better way for doing this? If we had something like i suggested (with .value or not) def testMultiThread(): resultDict = {} resultDict[Test1]= thread.startValueReturningThread(Test1) resultDict[Test1]= thread.startValueReturningThread(Test2) resultDict[Test1]= thread.startValueReturningThread(Test3) return resultDict #or again, pharsing the result... -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu May 7 08:31:45 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 7 May 2009 06:31:45 +0000 (UTC) Subject: [Python-ideas] [Value Returning Threard] References: <20090506210221.GA25517@panix.com> Message-ID: John Graham writes: > > If the OP is asking for futures, couldn't that be implemented in a > library without touching the language? It's a powerful enough feature > to consider putting it in a library. I don't know what C++ futures are but you should take a look at Twisted Deferred objects. The official implementation is in Python but a C implementation has been lingering on for years, you may be able to give them some help: http://twistedmatrix.com/trac/ticket/2245 Regards Antoine. From talin at acm.org Thu May 7 08:28:24 2009 From: talin at acm.org (Talin) Date: Wed, 06 May 2009 23:28:24 -0700 Subject: [Python-ideas] Coloured documentation In-Reply-To: <9e754ef50905060701h5c99a39cvd328693fcecaa112@mail.gmail.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <9e754ef50905060701h5c99a39cvd328693fcecaa112@mail.gmail.com> Message-ID: <4A027F88.8070307@acm.org> I've found through experimentation that Georg's default CSS color scheme looks pretty good when you apply a hue rotation. (I.e. convert to HSB, add a delta to H, then convert back to RGB.) In fact, if you are using Sphinx and you want something that looks (a) pretty, and (b) different than the default, but you don't want to spend a lot of time tweaking all of the colors, a global hue rotation is the easiest way to go. The net result is that light colors stay light, dark colors dark, and complementary colors stay complementary, and the whole thing stays readable. A 30 degree rotation for each release would allow for adjacent releases to be visually distinct, and would allow for 12 releases before the color scheme repeats. If more schemes are needed, you can generate additional permutations by swapping color components. -- Talin Jeremy Banks wrote: > I'm +1 if it could be implemented without much hassle. > > Perhaps the bikeshedding could be minimized by deciding in advance on > how the colors would be calculated. > > Something like... > R = (MAJOR + MINOR / 16) / 5 > G = 1 - ((MAJOR + MINOR / 16) / 5) > B = REVISION / 16 > > So the change would be more dramatic between larger changes. > > On 2009-05-06, Konrad Delong wrote: >> The documentation under http://docs.python.org/ could have different >> colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way >> one would know on sight which docs one's reading. >> It would also cause a lot of bikeshed talk for every release :) >> >> Konrad >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From robertc at robertcollins.net Thu May 7 08:43:44 2009 From: robertc at robertcollins.net (Robert Collins) Date: Thu, 07 May 2009 16:43:44 +1000 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: References: <20090506210221.GA25517@panix.com> Message-ID: <1241678624.20116.324.camel@lifeless-64> On Thu, 2009-05-07 at 06:31 +0000, Antoine Pitrou wrote: > John Graham writes: > > > > If the OP is asking for futures, couldn't that be implemented in a > > library without touching the language? It's a powerful enough feature > > to consider putting it in a library. > > I don't know what C++ futures are but you should take a look at Twisted Deferred > objects. The official implementation is in Python but a C implementation has > been lingering on for years, you may be able to give them some help: > http://twistedmatrix.com/trac/ticket/2245 Futures block, Defereds call forward. foo = something_returning_a_future() # does not block foo.get_value() # blocks def something_returning_a_future(): def worker(): [some code here that will does expensive work] return Future(worker) class Future: def __init__(callable): # imagine thread-safe code here to run callable in a thread def get_value(self): if not self.result: self.result = self.queue.pop() return self.result This is approximately a Future implementation for python. The difference - and its quite a big one - to Defereds is that defereds are a call-forward approach, whereas Futures are proxy objects that are waited on to get their results. twisted depends on never blocking as a core part of the design, Futures and Promises don't work well here. -Rob -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: From denis.spir at free.fr Thu May 7 09:39:28 2009 From: denis.spir at free.fr (spir) Date: Thu, 7 May 2009 09:39:28 +0200 Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A023A18.9080103@canterbury.ac.nz> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> <28337B8E-44CE-4A81-AB50-3B29B4B7D679@gmail.com> <4A023A18.9080103@canterbury.ac.nz> Message-ID: <20090507093928.248602aa@o> Le Thu, 07 May 2009 13:32:08 +1200, Greg Ewing s'exprima ainsi: > Leonardo Santagada wrote: > > > No color formulas based on > > versions please (this would just end up in ugly colors and no one > > knowing how to decode the version from the color anyway). > > Instead of just one colour, parts of the page > could be coloured according to different parts of > the version number, e.g. header from the major > version and sidebar from the minor version, with > maybe a stripe somewhere for the revision. > > The digits could be encoded using the standard > resistor colour code: > > 0 - black > 1 - brown > 2 - red > 3 - orange > 4 - yellow > 5 - green > 6 - blue > 7 - purple > 8 - grey > 9 - white > > Then there would be no problem recovering the > version number from the colour scheme. > I like the idea. We could hardly find a simpler encoding colour scheme for the version. Also, the resistor colour sequence include the common rainbow sequence (2..7). About layout, I don't think we need big colour bars or such invasive things. A little symbol at the lower left or right corner could do the job: ##### major @ @ minor If you know what the symbol means, you don't need it big. If you don't, having it big does not help. It's easier to add it to the current layout, too. The issue is that python's doc holds many very long pages split into sections and sub-sections, etc.. so that it's common to land somewhere inside in a page (either from a toc or external link). The colour mark should then be static if we want it be really useful, but it's a much more difficult thing and does not fit at all with the present layout (no static top or side bar). Another possibility is to use the version color as title backgroud in place of the present gray. I wouldn't find this agressive, as long as the colors are rather pastel than bright, because the same colour would be used on the whole page. But surely there would be much resistance to this idea. Denis ------ la vita e estrany From aahz at pythoncraft.com Thu May 7 14:16:01 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 7 May 2009 05:16:01 -0700 Subject: [Python-ideas] [Value Returning Threard] In-Reply-To: References: <20090506210221.GA25517@panix.com> Message-ID: <20090507121601.GB29699@panix.com> On Thu, May 07, 2009, yoav glazner wrote: > > I'm no python expert but i had to do a stress test on a libary. > [...] > Maybe there is a better way for doing this? For questions about using Python, please post to comp.lang.python -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From mrts.pydev at gmail.com Fri May 8 09:58:10 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Fri, 8 May 2009 10:58:10 +0300 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: Message-ID: On Sun, Apr 12, 2009 at 1:22 AM, Mart S?mermaa wrote: > There was a somewhat ancient discussion on OrderedDict and OrderedSet > before: http://mail.python.org/pipermail/python-dev/2005-March/051915.html > > The resolution seemed to be that neither of them should be in stdlib. Now > that OrderedDict is in and Raymond Hettinger has created a solid OrderedSet > implementation: http://code.activestate.com/recipes/576694/ , could the > latter also be included in collections? So, let's review what we have in terms of data structures: Structure Stable Unique Python type ------------------------------------------- Multiset no no - Set no yes set Map no yes dict List yes no list, tuple Ordered set yes yes - Ordered map yes yes collections.OrderedDict where "stable" means that input order is retained. As Multiset is arguably quite useless, only Ordered set is missing from "total" coverage of data structures. And it is practical as well. Am I really the only one who would like to see this in stdlib? From mrts.pydev at gmail.com Fri May 8 10:05:18 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Fri, 8 May 2009 11:05:18 +0300 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: Message-ID: On Fri, May 8, 2009 at 10:58 AM, Mart S?mermaa wrote: > So, let's review what we have in terms of data structures: s/data structures/container data structures/ > from "total" coverage of data structures. And it is practical as well. s/"total" coverage of data structures/"total" coverage of container data structures based on ordering and uniqueness criteria/ From arnodel at googlemail.com Fri May 8 15:14:34 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 8 May 2009 14:14:34 +0100 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: Message-ID: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> 2009/5/8 Mart S?mermaa : > As Multiset is arguably quite useless[...] Arguably, it is far from useless. -- Arnaud From mrts.pydev at gmail.com Fri May 8 16:24:41 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Fri, 8 May 2009 17:24:41 +0300 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> Message-ID: On Fri, May 8, 2009 at 4:14 PM, Arnaud Delobelle wrote: > 2009/5/8 Mart S?mermaa : >> As Multiset is arguably quite useless[...] > > Arguably, it is far from useless. > > -- > Arnaud > Can you perhaps provide a use case where a list (an "ordered multiset") does not suffice? From google at mrabarnett.plus.com Fri May 8 17:00:57 2009 From: google at mrabarnett.plus.com (MRAB) Date: Fri, 08 May 2009 16:00:57 +0100 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> Message-ID: <4A044929.1060404@mrabarnett.plus.com> Mart S?mermaa wrote: > On Fri, May 8, 2009 at 4:14 PM, Arnaud Delobelle wrote: >> 2009/5/8 Mart S?mermaa : >>> As Multiset is arguably quite useless[...] >> Arguably, it is far from useless. >> > Can you perhaps provide a use case where a list (an "ordered > multiset") does not suffice? > Raymond Hettinger has created a Counter class instead of a strict multiset class. It's useful for efficiently counting how many times things occur. Without it you would have to build a dict of counts (I've done it myself). Practicality beats purity, etc. From arnodel at googlemail.com Fri May 8 17:06:58 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 8 May 2009 16:06:58 +0100 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> Message-ID: <0DBA1EB2-1FF9-482C-BAF3-BC60F73CA475@googlemail.com> On 8 May 2009, at 15:23, Mart S?mermaa wrote: > On Fri, May 8, 2009 at 4:14 PM, Arnaud Delobelle > wrote: >> 2009/5/8 Mart S?mermaa : >>> As Multiset is arguably quite useless[...] >> >> Arguably, it is far from useless. > > Can you perhaps provide a use case where a list (an "ordered > multiset") does not suffice? I implemented a unification algorithm that needed multisets to work in O(n^2). -- Arnaud From mrts.pydev at gmail.com Fri May 8 17:56:44 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Fri, 8 May 2009 18:56:44 +0300 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: <0DBA1EB2-1FF9-482C-BAF3-BC60F73CA475@googlemail.com> References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> <0DBA1EB2-1FF9-482C-BAF3-BC60F73CA475@googlemail.com> Message-ID: On Fri, May 8, 2009 at 6:06 PM, Arnaud Delobelle wrote: > > I implemented a unification algorithm that needed multisets to work in > O(n^2). Complexity is a totally different theme and best reserved for extensions. Supporting special-case containers that have various space/speed and best/worst case tradeoffs have been discussed before and the consensus seems to be that supporting them in stdlib is infeasible -- with what I entirely agree. Supporting multiset only for O(n^2) performance would violate that consensus. However, ordered set and ordered map are not special-case in that sense. It's not the O-complexity but general utility -- i.e. the plugging of the holes in the ordering and uniqueness table and helping people with the corresponding use cases -- that vouch for their inclusion in the stdlib. Now that ordered map is in, ordered set could follow. From tjreedy at udel.edu Fri May 8 20:12:02 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 May 2009 14:12:02 -0400 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: Message-ID: Mart S?mermaa wrote: > On Sun, Apr 12, 2009 at 1:22 AM, Mart S?mermaa wrote: >> There was a somewhat ancient discussion on OrderedDict and OrderedSet >> before: http://mail.python.org/pipermail/python-dev/2005-March/051915.html >> >> The resolution seemed to be that neither of them should be in stdlib. Now >> that OrderedDict is in and Raymond Hettinger has created a solid OrderedSet >> implementation: http://code.activestate.com/recipes/576694/ , could the >> latter also be included in collections? > > So, let's review what we have in terms of data structures: > > Structure Stable Unique Python type > ------------------------------------------- > Multiset no no - > Set no yes set > Map no yes dict > List yes no list, tuple > Ordered set yes yes - > Ordered map yes yes collections.OrderedDict > > where "stable" means that input order is retained. > > As Multiset is arguably quite useless, only Ordered set is missing > from "total" coverage of data structures. And it is practical as well. > > Am I really the only one who would like to see this in stdlib? What are the use cases? An 'ordered set' is basically a list + set. From mrts.pydev at gmail.com Fri May 8 21:00:56 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Fri, 8 May 2009 22:00:56 +0300 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: Message-ID: On Fri, May 8, 2009 at 9:12 PM, Terry Reedy wrote: > What are the use cases? ?An 'ordered set' is basically a list + set. A list with no duplicate values (can be alternatively called unique list) is a common problem judging on the following: http://www.google.com/codesearch?q=lang%3Apython+list+unique http://www.google.com/codesearch?q=lang%3Apython+list+duplicates http://www.peterbe.com/plog/uniqifiers-benchmark/ http://code.activestate.com/recipes/52560/ http://www.google.ee/search?q=python+unique+list My use case stems from http://bugs.python.org/issue5877 , see http://github.com/mrts/qparams/blob/bf1b29ad46f9d848d5609de6de0bfac1200da310/qparams.py#L365 From arnodel at googlemail.com Fri May 8 21:16:41 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 8 May 2009 20:16:41 +0100 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> <0DBA1EB2-1FF9-482C-BAF3-BC60F73CA475@googlemail.com> Message-ID: <9FE328C6-9B56-4CBE-A1B2-53E9E1B44AF8@googlemail.com> On 8 May 2009, at 16:56, Mart S?mermaa wrote: > On Fri, May 8, 2009 at 6:06 PM, Arnaud Delobelle > wrote: >> >> I implemented a unification algorithm that needed multisets to work >> in >> O(n^2). > > Complexity is a totally different theme and best reserved for > extensions. Supporting special-case containers that have various > space/speed and best/worst case tradeoffs have been discussed before > and the consensus seems to be that supporting them in stdlib is > infeasible -- with what I entirely agree. Supporting multiset only for > O(n^2) performance would violate that consensus. > You misunderstand me. My task was to implement this algorithm. Multisets were the natural data structure to implement this algorithm. That it was O(n^2) (rather than the exponential complexity of the naive unification algorithm) is an issue attached to the algorithm, not to whether I use multisets or not to implement it. -- Arnaud From mrts.pydev at gmail.com Fri May 8 21:25:25 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Fri, 8 May 2009 22:25:25 +0300 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: <9FE328C6-9B56-4CBE-A1B2-53E9E1B44AF8@googlemail.com> References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> <0DBA1EB2-1FF9-482C-BAF3-BC60F73CA475@googlemail.com> <9FE328C6-9B56-4CBE-A1B2-53E9E1B44AF8@googlemail.com> Message-ID: On Fri, May 8, 2009 at 10:16 PM, Arnaud Delobelle wrote: > You misunderstand me. ?My task was to implement this algorithm. ?Multisets > were the natural data structure to implement this algorithm. ?That it was > O(n^2) (rather than the exponential complexity of the naive unification > algorithm) is an issue attached to the algorithm, not to whether I use > multisets or not to implement it. The main question though is whether to add an ordered set or not, multiset is slightly off-topic. From chambon.pascal at wanadoo.fr Fri May 8 22:31:57 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Fri, 08 May 2009 22:31:57 +0200 Subject: [Python-ideas] Default arguments in Python - the return Message-ID: <4A0496BD.3030709@wanadoo.fr> Hello, I'm surely not original in any way there, but I'd like to put back on the table the matter of "default argument values". Or, more precisely, the "one shot" handling of default values, which makes that the same mutable objects, given once as default arguments, come back again and again at each function call. They thus become some kinds of "static variables", which get polluted by the previous calls, whereas many-many-many python users still believe that they get a fresh new value at each function call. I think I understand how default arguments are currently implemented (and so, "why" - technically - it does behave this way), but I'm still unsure of "why" - semantically - this must be so. I've browsed lots of google entries on that subject, but as far as I'm concerned, I've found nothing in favor current semantic. I've rather found dozens, hundreds of posts of people complaining that they got biten by this gotcha, many of them finishing with a "Never put mutable values in default arguments, unless you're very very sure of what you're doing !". And no one seemed to enjoy the possibilities of getting "potentially static variables" this way. Static variables are imo a rather bad idea, since they create "stateful functions", that make debugging and maintenance more difficult ; but when such static variable are, furthermore, potentially non-static (i.e when the corresponding function argument is supplied), I guess they become totally useless and dangerous - a perfect way to get hard-to-debug behaviours. On the other hand, when people write "def func(mylist=[]):", they basically DO want a fresh new list at each call, be it given by the caller or the default argument system. So it's really a pity to need tricks like >/ def f(a, L=None): />/ if L is None: />/ L = [] /to get what we want (and if None was also a possible value ? what other value should we put as a placeholder for "I'd like None or a fresh new list but I can't say it directly ?"). So I'd like to know : are there other "purely intellectual" arguments for/against the current semantic of default arguments (I might have missed some discussion on this subject, feel free to point them ? Currently, this default argument handling looks, like a huge gotcha for newcomers, and, I feel, like an embarrassing wart to most pythonistas. Couldn't it be worth finding a new way of doing it ? Maybe there are strong arguments against a change at that level ; for example, performance issues (I'm not good in those matters). But I need to ensure. So here are my rough ideas on what we might do - if after having the suggestions from expert people, it looks like it's worth writting a PEP, I'll be willing to particpateon it. Basically, I'd change the python system so that, when a default argument expression is encountered, instead of being executed, it's wrapped in some kind of zero-argument lambda expression, which gets pushed in the "func_defaults" attribute of the function. And then, each time a default argument is required in a function call, this lambda expression gets evaluated and gives the expected value. I guess this will mean some overhead during function call, so this might become another issue. It's also a non retrocompatible change, so I assume we'd have to use a "from __future__ import XXX" until Python4000. But I think the change is worth the try, because it's a trap which waits for all the python beginners. So, if this matters hasn't already been marked somewhere as a no-go, I eagerly await the feedback of users and core developpers on the subject. :) By the way, I'm becoming slightly allergical to C-like languages (too much hassle for too little gain, compared to high level dynamic languages), but if that proposition goes ahead, and no one wants to handle the implementation details, I'll put the hands in the engine ^^ Regards, Pascal -------------- next part -------------- An HTML attachment was scrubbed... URL: From pyideas at rebertia.com Fri May 8 23:12:28 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Fri, 8 May 2009 14:12:28 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A0496BD.3030709@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> Message-ID: <50697b2c0905081412w74e9fcbbl16a189724c8b0e8c@mail.gmail.com> On Fri, May 8, 2009 at 1:31 PM, Pascal Chambon wrote: > Hello, > > I'm surely not original in any way there, but I'd like to put back on the > table the matter of "default argument values". > Or, more precisely, the "one shot" handling of default values, which makes > that the same mutable objects, given once as default arguments, come back > again and again at each function call. > They thus become some kinds of "static variables", which get polluted by the > previous calls, whereas many-many-many python users still believe that they > get a fresh new value at each function call. > I think I understand how default arguments are currently implemented (and > so, "why" - technically - it does behave this way), but I'm still unsure of > "why" - semantically - this must be so. > So I'd like to know : are there other "purely intellectual" arguments > for/against the current semantic of default arguments (I might have missed > some discussion on this subject, feel free to point them ? Point-point: http://mail.python.org/pipermail/python-ideas/2007-January/000121.html And see also the links below. > Currently, this default argument handling looks, like a huge gotcha for > newcomers, and, I feel, like an embarrassing wart to most pythonistas. > Couldn't it be worth finding a new way of doing it ? > Maybe there are strong arguments against a change at that level ; for > example, performance issues (I'm not good in those matters). But I need to > ensure. > > So here are my rough ideas on what we might do - if after having the > suggestions from expert people, it looks like it's worth writting a PEP, > I'll be willing to particpateon it. > Basically, I'd change the python system so that, when a default argument > expression is encountered, instead of being executed, it's wrapped in some > kind of zero-argument lambda expression, which gets pushed in the > "func_defaults" attribute of the function. > And then, each time a default argument is required in a function call, this > lambda expression gets evaluated and gives the expected value. > > I guess this will mean some overhead during function call, so this might > become another issue. > It's also a non retrocompatible change, so I assume we'd have to use a "from > __future__ import XXX" until Python4000. > But I think the change is worth the try, because it's a trap which waits for > all the python beginners. > > So, if this matters hasn't already been marked somewhere as a no-go, I > eagerly await the feedback of users and core developpers on the subject. :) It's basically been rejected. See GvR Pronouncement: http://mail.python.org/pipermail/python-3000/2007-February/005715.html regarding the pre-PEP "Default Argument Expressions": http://mail.python.org/pipermail/python-3000/2007-February/005704.html Unless your exact idea somehow differs significantly from my pre-PEP (sounds like it doesn't IMHO), it's not gonna happen. It's basically too magical. Cheers, Chris -- http://blog.rebertia.com From george.sakkis at gmail.com Sat May 9 00:06:43 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 8 May 2009 18:06:43 -0400 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <50697b2c0905081412w74e9fcbbl16a189724c8b0e8c@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <50697b2c0905081412w74e9fcbbl16a189724c8b0e8c@mail.gmail.com> Message-ID: <91ad5bf80905081506g28c5936es17c0140a8568cf6b@mail.gmail.com> On Fri, May 8, 2009 at 5:12 PM, Chris Rebert wrote: > On Fri, May 8, 2009 at 1:31 PM, Pascal Chambon >> So, if this matters hasn't already been marked somewhere as a no-go, I >> eagerly await the feedback of users and core developpers on the subject. :) > > It's basically been rejected. See GvR Pronouncement: > http://mail.python.org/pipermail/python-3000/2007-February/005715.html > regarding the pre-PEP "Default Argument Expressions": > http://mail.python.org/pipermail/python-3000/2007-February/005704.html > > Unless your exact idea somehow differs significantly from my pre-PEP > (sounds like it doesn't IMHO), it's not gonna happen. It's basically > too magical. FWIW I don't find the dual semantics, with explicit syntax for the new semantics ("def foo(bar=new baz)") mentioned in the PEP too magical. If even C, a relatively small language, affords two calling semantics, why would it be too confusing for Python ? Perhaps that PEP might had had better luck if it didn't propose replacing the current semantics with the new. George From dagmoxnes at hotmail.com Sat May 9 00:05:15 2009 From: dagmoxnes at hotmail.com (Dag Moxnes) Date: Sat, 9 May 2009 00:05:15 +0200 Subject: [Python-ideas] str.format utility function Message-ID: Hi list, I'm sorry if this has been discussed before, but I did not find any references. I've been playing abit with the new str.format function. I really like the syntax and simplicity. However, when simply printing named variables in locals() or a class, I quite common use-case, I find it a bit too complicated with the **: "Local variable var1 is %(var1)s" % locals() vs "Local variable var1 is {var1}".format(**locals()) and "Instance variable var2 is %(var2)s" % self.__dict__ vs "Instance variable var2 is {var2}" % **self.__dict__ Therefore I have made myself a utility funcion: def easyformat(s, data): try: return s.format(**data) except TypeError: return s.format(**data.__dict__) so that I can do the following: "Local variable var1 is {var1}".format(**locals()) and "Instance variable var2 is %(var2)s" % self Should a function similar to this (maybe with a better name) be included included in some standard library? -Dag _________________________________________________________________ F? mer ut av Windows Live? med Internet Explorer? 8. http://microsoft.no/ie -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerald.britton at gmail.com Sat May 9 01:27:15 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 8 May 2009 19:27:15 -0400 Subject: [Python-ideas] str.format utility function In-Reply-To: References: Message-ID: <5d1a32000905081627v718fc916p808dd4b12b3b4f34@mail.gmail.com> fwiw: one = 'slower' two = 'yours' "my str is %(one)s than %(two)s" % locals() is slower than: "my str is %(one)s than %(two)s" % {"one":faster, "two":yours} probably because of the overhead of the function call to locals(). Basically I find using locals() this way is just lazy programming, since you hope your variables are in there somewhere. If someone changes them later, your string expression may fail. I'd rather be explicit about what I'm doing. On Fri, May 8, 2009 at 6:05 PM, Dag Moxnes wrote: > Hi list, > > I'm sorry if this has been discussed before, but I did not find any > references. I've been playing abit with the new str.format function. I > really like the syntax and simplicity. > > However, when simply printing named variables in locals() or a class, I > quite common use-case, I find it a bit too complicated with the **: > > "Local variable var1 is %(var1)s" % locals() > > vs > > "Local variable var1 is {var1}".format(**locals()) > > and > > "Instance variable var2 is %(var2)s" % self.__dict__ > > vs > > "Instance variable var2 is {var2}" % **self.__dict__ > > Therefore I have made myself a utility funcion: > > def easyformat(s, data): > ??? try: > ??????? return s.format(**data) > ??? except TypeError: > ??????? return s.format(**data.__dict__) > > so that I can do the following: > > "Local variable var1 is {var1}".format(**locals()) > > and > > "Instance variable var2 is %(var2)s" % self > > Should a function similar to this (maybe with a better name) be included > included in some standard library? > > -Dag > > ________________________________ > En om gangen eller alle p? ?n gang? F? oppdateringer fra vennene dine p? ett > sted. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- Gerald Britton From tjreedy at udel.edu Sat May 9 01:34:42 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 May 2009 19:34:42 -0400 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A0496BD.3030709@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> Message-ID: Pascal Chambon wrote: > I'm surely not original in any way there, but I'd like to put back on > the table the matter of "default argument values". There have been two proposals: 1. Evaluate the expression once, store the result, and copy on each function call. - Expensive. - Nearly always not needed. - Not always possible. 2. Store the expression and evaluate on each function call (your re-proposal). - Expensive. - The result may be different for each function call, and might raise an exception. - This is the job of the suite !!!!!!!!!!!!!!!! - Which is to say, run-time code belongs in the function body, not the header. > And no one seemed to enjoy the possibilities of getting "potentially > static variables" this way. You did not search hard enough. > Static variables are imo a rather bad idea, So you want to take them away from everyone else. I think *that* is a rather bad idea ;-). No one is forcing you to use them. > On the other hand, when people write "def func(mylist=[]):", they > basically DO want a fresh new list at each call, Maybe, maybe not. > be it given by the caller or the default argument system. > So it's really a pity to need tricks like > >/ def f(a, L=None): > />/ if L is None: > />/ L = [] Or don't supply a default arg if that is not what you really want. Putting call time code in the function body is not a trick. > /to get what we want (and if None was also a possible value ? __none = object() def(par = __none): if par == __none: ... as had been posted each time this question has been asked. > I guess this will mean some overhead during function call, I absolutely guarantee that this will. Functions calls are expensive. Adding a function call for each default arg (and many functions have more than one) multiplies the calling overhead. > so this might become another issue. Is and always has been. Terry Jan Reedy From tjreedy at udel.edu Sat May 9 01:52:59 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 08 May 2009 19:52:59 -0400 Subject: [Python-ideas] str.format utility function In-Reply-To: References: Message-ID: Dag Moxnes wrote: > Hi list, > > I'm sorry if this has been discussed before, but I did not find any > references. I've been playing abit with the new str.format function. I > really like the syntax and simplicity. > > However, when simply printing named variables in locals() or a class, I > quite common use-case, I find it a bit too complicated with the **: > > "Local variable var1 is %(var1)s" % locals() > > vs > > "Local variable var1 is {var1}".format(**locals()) I see nothing compicated about **. In any case, it is a common idiom that Python progrmmers should learn and hopefully become comfortable with. I see nothing gained with s.easyformat(s, locals()) # over s.format(**locals) except a few extra chars to type ;-). If you were doing that often, you could write def lform(s): # format caller locals l = # ask on Python list or check python recipies return s.format(**l) > "Instance variable var2 is %(var2)s" % self.__dict__ > > vs > > "Instance variable var2 is {var2}" % **self.__dict__\ You meant, "Instance variable var2 is {var2}".format(**self.__dict__) Wrapping this seems like a possible method. For a constant format applicable to all instances, override .__str__(). > def easyformat(s, data): > try: > return s.format(**data) > except TypeError: > return s.format(**data.__dict__) > Should a function similar to this (maybe with a better name) be > included in some standard library? I think not. Terry Jan Reedy From ziade.tarek at gmail.com Sat May 9 10:24:44 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 9 May 2009 10:24:44 +0200 Subject: [Python-ideas] [Python-Dev] Adding a "sysconfig" module in the stdlib In-Reply-To: <5d44f72f0905072120ud43b30ft3e9d6d547afaf3f2@mail.gmail.com> References: <94bdd2610905071736wa6a86awa1a7cb30a6f6e775@mail.gmail.com> <5b8d13220905072025m522ce6e5pbfad73ebe18e3f30@mail.gmail.com> <5d44f72f0905072120ud43b30ft3e9d6d547afaf3f2@mail.gmail.com> Message-ID: <94bdd2610905090124me478477h919e4470e137ae0e@mail.gmail.com> 2009/5/8 Jeffrey Yasskin : > > I'm +1 to the idea of a sysconfig module (by whatever name). I > believe, however, that we should be able to generate the module as > part of the build process instead of having it look up values from the > Makefile. Like, constants at the top of the module, generated at built time ? Tarek -- Tarek Ziad? | http://ziade.org From denis.spir at free.fr Sat May 9 12:17:37 2009 From: denis.spir at free.fr (spir) Date: Sat, 9 May 2009 12:17:37 +0200 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> Message-ID: <20090509121737.3693797b@o> Le Fri, 8 May 2009 17:24:41 +0300, Mart S?mermaa s'exprima ainsi: > On Fri, May 8, 2009 at 4:14 PM, Arnaud Delobelle > wrote: > > 2009/5/8 Mart S?mermaa : > >> As Multiset is arguably quite useless[...] > > > > Arguably, it is far from useless. > > > > -- > > Arnaud > > > > > Can you perhaps provide a use case where a list (an "ordered > multiset") does not suffice? I feel the same. Unorder (like in dict or set) is not a feature, I guess. Set is useful for it ensures uniqueness of items, not because of unorder. A kind of "unique-item" sequence would do as well. Unorder is rather a consequence of hash implementation for performance, but I cannot find any use case where it would be a useful feature. Examples welcome. Denis ------ la vita e estrany From denis.spir at free.fr Sat May 9 12:42:56 2009 From: denis.spir at free.fr (spir) Date: Sat, 9 May 2009 12:42:56 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A0496BD.3030709@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> Message-ID: <20090509124256.42bd78ce@o> Le Fri, 08 May 2009 22:31:57 +0200, Pascal Chambon s'exprima ainsi: > And no one seemed to enjoy the possibilities of getting "potentially > static variables" this way. Static variables are imo a rather bad idea, > since they create "stateful functions", that make debugging and > maintenance more difficult ; but when such static variable are, > furthermore, potentially non-static (i.e when the corresponding function > argument is supplied), I guess they become totally useless and dangerous > - a perfect way to get hard-to-debug behaviours. If we want static vars, there are better places than default args for this. (See also the thread about memoizing). E.g. on the object when it's a method, or even on the func itself. def squares(n): square = n * n; print square if square not in squares.static_list: squares.static_list.append(n) squares.static_list = [] squares(1);squares(2);squares(1);squares(3) print squares.static_list Denis ------ la vita e estrany From chambon.pascal at wanadoo.fr Sat May 9 12:56:20 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Sat, 09 May 2009 12:56:20 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> Message-ID: <4A056154.2090103@wanadoo.fr> Thanks everyone for the feedback and the links (I was obviously too confident in Google's first pages, to miss such things >_<) Terry Reedy a ?crit : >> And no one seemed to enjoy the possibilities of getting "potentially >> static variables" this way. > > You did not search hard enough. > Well, for sure some people here and there used that semantic to have, for example, a "default cache" handling the requests for which a specific cache isn't provided. But that behavior can as easily be obtained with a much more explicit way, which furthermore lets you access your default cache easily from inside the function code, even when a specific cache is provided : class a: cache=[1,2,3] def func(self, x, newcache=cache): print "Current cache state :",y y.append(x) print "The static, default cache is ", cache So I don't see the default argument trick as a "neat feature", rather as a way of doing simple things obscure. >> Static variables are imo a rather bad idea, > > So you want to take them away from everyone else. I think *that* is a > rather bad idea ;-). No one is forcing you to use them. > I don't want to annihilate all traces of static variables :p ; I just find them ugly, because they create stateful functions whose state is hidden in them (like some do with free variables, too), and that's imo not a "robust code best practice". But what kills me with current default arguments is that those aren't even real static variables : they're "potentially static variables", and as far as I've seen, you have no easy way to check whether, for instance, the argument value that you've gotten is the default, static one, or a new one provided by the caller (of course, you can store the default value somewhere else for reference, but it's lamely redundant). If people want static variables in python, for example to avoid OO programming and still have stateful functions, we can add an explicit "static" keyword or its equivalent. But using the ambiguous value given via a default-valued argument is not pretty, imo. Unless we have a way to access, from inside a code block, the function object in which this code block belongs. Does it exist ? Do we have any way, from inside a call block, to browse the default arguments that this code block might receive ? > >> I guess this will mean some overhead during function call, > > I absolutely guarantee that this will. Functions calls are expensive. > Adding a function call for each default arg (and many functions have > more than one) multiplies the calling overhead. > > > so this might become another issue. > > Is and always has been. > Well, if, like it was proposed in previous threads, the expression is only reevaluated in particular circumstances (i.e, if the user asks it with a special syntax), it won't take more time than the usual "if myarg is None : myarg = []" ; but I agree that alternate syntaxes have led to infinite and complex discussions, and that the simpler solution I provided is likely to be too CPU intensive, more than I expected... > >> /to get what we want (and if None was also a possible value ? > > __none = object() > def(par = __none): > if par == __none: ... > > as had been posted each time this question has been asked. Well, I didn't turn my rhetorical question properly it seems ^^. I wholly agree that you can always use another object as a placeholder, but I don't quite like the idea of creating new instances just to signify "that's not a valid value that you can use, create one brand new" On the other hand, would anyone support my alternative wish, of having a builtin "NotGiven", similar to "NotImplemented", and dedicated to this somehow usual taks of "placeholder" ? There would be two major pros for this, imo : - giving programmers a handy object for all unvanted "mutable default argument" situations, without having to think "is None a value I might want to get ?" - *Important* : by appearing in the beginning of the doc near True and False, this keyword would be much more visible to beginners than the deep pages on "default argument handling" ; thus, they'd have much more chances to cross warnings on this Gotcha, than they currently have (and seeing "NotGiven" in tutorials would force them to wonder why it's so, it's imo much more explicit than seeing "None" values instead) So, since reevaluation of arguments actually *is* a no-go, and forbidding mutable arguments is obviously a no-go too, would you people support this integrating of "NotGiven" (or any other name) in the builtins ? It'd sound to me like a good practice. Regards, Pascal From chambon.pascal at wanadoo.fr Sat May 9 13:16:46 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Sat, 09 May 2009 13:16:46 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <20090509124256.42bd78ce@o> References: <4A0496BD.3030709@wanadoo.fr> <20090509124256.42bd78ce@o> Message-ID: <4A05661E.6010305@wanadoo.fr> spir a ?crit : > Le Fri, 08 May 2009 22:31:57 +0200, > Pascal Chambon s'exprima ainsi: > > >> And no one seemed to enjoy the possibilities of getting "potentially >> static variables" this way. Static variables are imo a rather bad idea, >> since they create "stateful functions", that make debugging and >> maintenance more difficult ; but when such static variable are, >> furthermore, potentially non-static (i.e when the corresponding function >> argument is supplied), I guess they become totally useless and dangerous >> - a perfect way to get hard-to-debug behaviours. >> > > If we want static vars, there are better places than default args for this. (See also the thread about memoizing). E.g. on the object when it's a method, or even on the func itself. > > def squares(n): > square = n * n; print square > if square not in squares.static_list: > squares.static_list.append(n) > squares.static_list = [] > > squares(1);squares(2);squares(1);squares(3) > print squares.static_list > > Denis > ------ > la vita e estrany > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > > Well, I've just realized I'd sent a semi-dumb question in my previous answer :p I'd never quite realized it was possible to store stuffs inside the function object, by retrieving it from inside the code object. And it works pretty well... In classes you can access your function via self, in closures they get caught in cells... it's only in global scope that there are problems : here, if you rename squares (newsquares = squares ; squares = None), you'll have an error by calling newsquares, because it searches "squares" in the global scope, without the help of "self" or closures. Still, an explicit way of targetting "the function I'm in" would be sweet imo, but retrieving it that way is not far from being as handy. Thanks for the tip that opened my eyes, regards, Pascal -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat May 9 13:32:35 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 9 May 2009 21:32:35 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A056154.2090103@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> Message-ID: <200905092132.36438.steve@pearwood.info> On Sat, 9 May 2009 08:56:20 pm Pascal Chambon wrote: > But what kills me with current default arguments is that those aren't > even real static variables : they're "potentially static variables", > and as far as I've seen, you have no easy way to check whether, for > instance, the argument value that you've gotten is the default, > static one, or a new one provided by the caller (of course, you can > store the default value somewhere else for reference, but it's lamely > redundant). I'm not really sure why you would want to do that. The whole point of default values is to avoid needing to care whether or not the caller has provided an argument or not. [...] > Does it exist ? Do we have any way, from inside a call block, to > browse the default arguments that this code block might receive ? >>> def spam(n=42): ... return "spam "*n ... >>> spam.func_defaults (42,) dir(func_object) is your friend :) [...] > but I agree that alternate syntaxes have led to infinite and complex > discussions, and that the simpler solution I provided is likely to be > too CPU intensive, more than I expected... I would support... no, that's too strong. I wouldn't oppose the suggestion that Python grow syntax for "evaluate this default argument every time the function is called (unless the argument is given by the caller)". The tricky part is coming up with good syntax and a practical mechanism. [...] > On the other hand, would anyone support my alternative wish, of > having a builtin "NotGiven", similar to "NotImplemented", and > dedicated to this somehow usual taks of "placeholder" ? There already is such a beast: None is designed to be used as a placeholder for Not Given, Nothing, No Result, etc. If None is not suitable, NotImplemented is also a perfectly good built-in singleton object which can be used as a sentinel. It's already used as a sentinel for a number of built-in functions and operators. There's no reason you can't use it as well. > There would be two major pros for this, imo : > - giving programmers a handy object for all unvanted "mutable > default argument" situations, without having to think "is None a > value I might want to get ?" But then they would need to think "Is NotGiven a value I might want to get, so I can pass it on to another function unchanged?", and you would then need to create another special value ReallyNotGiven. And so on. > - *Important* : by appearing in the beginning of the doc near > True and False, this keyword would be much more visible to beginners > than the deep pages on "default argument handling" ; thus, they'd > have much more chances to cross warnings on this Gotcha, than they > currently have (and seeing "NotGiven" in tutorials would force them > to wonder why it's so, it's imo much more explicit than seeing "None" > values instead) Heh heh heh, he thinks beginners read manuals :-) > So, since reevaluation of arguments actually *is* a no-go, and > forbidding mutable arguments is obviously a no-go too, would you > people support this integrating of "NotGiven" (or any other name) in > the builtins ? It'd sound to me like a good practice. -1 on an extra builtin. There's already two obvious ones, and if for some reason you need to accept None and NotImplemented as valid data, then you can create an unlimited number of sentinels with object(). The best advantage of using object() is that because the sentinel is unique to your module, you can guarantee that nobody can accidentally pass it, or expect to use it as valid data. -- Steven D'Aprano From steve at pearwood.info Sat May 9 13:36:12 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 9 May 2009 21:36:12 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A05661E.6010305@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <20090509124256.42bd78ce@o> <4A05661E.6010305@wanadoo.fr> Message-ID: <200905092136.12307.steve@pearwood.info> On Sat, 9 May 2009 09:16:46 pm Pascal Chambon wrote: > Still, an explicit way of targetting "the function I'm in" would be > sweet imo, but retrieving it that way is not far from being as handy. There's a rather long discussion on the comp.lang.python newsgroup at the moment about that exact question. Look for the recent thread titled "Self function". If you can't get Usenet and don't like Google Groups, the c.l.py newsgroup is also available as a python mailing list, and a gmane mailing list. -- Steven D'Aprano From steve at pearwood.info Sat May 9 13:39:53 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 9 May 2009 21:39:53 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <200905092136.12307.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <4A05661E.6010305@wanadoo.fr> <200905092136.12307.steve@pearwood.info> Message-ID: <200905092139.53685.steve@pearwood.info> On Sat, 9 May 2009 09:36:12 pm Steven D'Aprano wrote: > There's a rather long discussion on the comp.lang.python newsgroup at > the moment about that exact question Er, to be precise, by "at the moment" I actually mean "over the last few days". The thread seems to have more-or-less finished now. Of course, no Usenet thread is every *completely* finished. Please feel free to resurrect it if you have any good ideas, questions or insight into the issue. -- Steven D'Aprano From denis.spir at free.fr Sat May 9 14:34:08 2009 From: denis.spir at free.fr (spir) Date: Sat, 9 May 2009 14:34:08 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A056154.2090103@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> Message-ID: <20090509143408.6da5d2b6@o> Le Sat, 09 May 2009 12:56:20 +0200, Pascal Chambon s'exprima ainsi: > If people want static variables in python, for example to avoid OO > programming and still have stateful functions, we can add an explicit > "static" keyword or its equivalent. This is far from beeing pythonic anyway, I guess. Ditto for storing data on the func itself (as shown in another post). It provide a way of linking together data and behaviour; similar techniques are used e.g. in Lisp, so that many Lisp people find OO pretty useless. But python has OO in-built, and even as mainsteam paradigm. Data related to behaviour should be set on an object. > But using the ambiguous value given > via a default-valued argument is not pretty, imo. > Unless we have a way to access, from inside a code block, the function > object in which this code block belongs. > > Does it exist ? Do we have any way, from inside a call block, to browse > the default arguments that this code block might receive ? This is a feature of much more reflexive/meta languages like Io (or again Lisp), that were indeed designed from scratch with this capacity in mind and intended as a major programming feature. In Io you can even access the 'raw' message _before_ evaluation, so that you get the expression of argument, not only the resulting value. Denis ------ la vita e estrany From daniel at stutzbachenterprises.com Sat May 9 15:04:09 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 9 May 2009 08:04:09 -0500 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: <20090509121737.3693797b@o> References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> <20090509121737.3693797b@o> Message-ID: On Sat, May 9, 2009 at 5:17 AM, spir wrote: > I feel the same. Unorder (like in dict or set) is not a feature, I guess. > Set is useful for it ensures uniqueness of items, not because of unorder. A > kind of "unique-item" sequence would do as well. > Unorder is rather a consequence of hash implementation for performance, but > I cannot find any use case where it would be a useful feature. Examples > welcome. > Unorder is an absence of a feature, so it will never be "useful". However, unorder frees up the implementation to be more efficient since there is less information to keep track of. Also, insertion order isn't always the desired order. For example, maintaining a sorted list of unique items is often handy. In other cases, a different ordering may be useful. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From google at mrabarnett.plus.com Sat May 9 15:53:42 2009 From: google at mrabarnett.plus.com (MRAB) Date: Sat, 09 May 2009 14:53:42 +0100 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> <20090509121737.3693797b@o> Message-ID: <4A058AE6.8020901@mrabarnett.plus.com> Daniel Stutzbach wrote: > On Sat, May 9, 2009 at 5:17 AM, spir > wrote: > > I feel the same. Unorder (like in dict or set) is not a feature, I > guess. Set is useful for it ensures uniqueness of items, not because > of unorder. A kind of "unique-item" sequence would do as well. > Unorder is rather a consequence of hash implementation for > performance, but I cannot find any use case where it would be a > useful feature. Examples welcome. > > > Unorder is an absence of a feature, so it will never be "useful". > However, unorder frees up the implementation to be more efficient since > there is less information to keep track of. > > Also, insertion order isn't always the desired order. For example, > maintaining a sorted list of unique items is often handy. In other > cases, a different ordering may be useful. > There's also the question of whether the order is significant. Does a == b if a and b contain exactly the same items, but in a different order? If they are lists, then yes; if they are multisets, then no. From arnodel at googlemail.com Sat May 9 16:15:53 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sat, 9 May 2009 15:15:53 +0100 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: <20090509121737.3693797b@o> References: <9bfc700a0905080614v49186712p5a2a222139fe918f@mail.gmail.com> <20090509121737.3693797b@o> Message-ID: <0DBA5288-F73F-43D3-8E55-F65463E40E41@googlemail.com> On 9 May 2009, at 11:17, spir wrote: > I feel the same. Unorder (like in dict or set) is not a feature, I > guess. Set is useful for it ensures uniqueness of items, not because > of unorder. That is one reason why sets are useful. Another reason, which is at least as important, is that {1,2}=={2,1} is true - and that's precisely because the elements are not ordered. So absence of order is definitely a useful feature. -- Arnaud From g.brandl at gmx.net Sat May 9 17:38:12 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 09 May 2009 17:38:12 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A05661E.6010305@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <20090509124256.42bd78ce@o> <4A05661E.6010305@wanadoo.fr> Message-ID: Pascal Chambon schrieb: > Still, an explicit way of targetting "the function I'm in" would be > sweet imo, but retrieving it that way is not far from being as handy. You could use e.g. def selffunc(func): @wraps(func) def newfunc(*args, **kwds): return func(func, *args, **kwds) return newfunc @selffunc def foo(func, a): func.cache = a to avoid the "ugly" lookup of the function in the global namespace. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From steve at pearwood.info Sat May 9 17:39:47 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 10 May 2009 01:39:47 +1000 Subject: [Python-ideas] =?iso-8859-1?q?Add_OrderedSet_now_that_OrderedDict?= =?iso-8859-1?q?_is_in=09collections?= In-Reply-To: <4A058AE6.8020901@mrabarnett.plus.com> References: <4A058AE6.8020901@mrabarnett.plus.com> Message-ID: <200905100139.47370.steve@pearwood.info> On Sat, 9 May 2009 11:53:42 pm MRAB wrote: > There's also the question of whether the order is significant. Does a > == b if a and b contain exactly the same items, but in a different > order? If they are lists, then yes; if they are multisets, then no. I'm afraid you have made a mistake. List equality does not ignore order. >>> [1, 2, 3] == [2, 1, 3] False -- Steven D'Aprano From google at mrabarnett.plus.com Sat May 9 18:01:56 2009 From: google at mrabarnett.plus.com (MRAB) Date: Sat, 09 May 2009 17:01:56 +0100 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: <200905100139.47370.steve@pearwood.info> References: <4A058AE6.8020901@mrabarnett.plus.com> <200905100139.47370.steve@pearwood.info> Message-ID: <4A05A8F4.1050603@mrabarnett.plus.com> Steven D'Aprano wrote: > On Sat, 9 May 2009 11:53:42 pm MRAB wrote: > >> There's also the question of whether the order is significant. Does a >> == b if a and b contain exactly the same items, but in a different >> order? If they are lists, then yes; if they are multisets, then no. > > I'm afraid you have made a mistake. List equality does not ignore order. > >>>> [1, 2, 3] == [2, 1, 3] > False > Oops! Wrong way round. I meant: If they are multisets, then yes; if they are lists, then no. From jan.kanis at phil.uu.nl Sat May 9 23:54:03 2009 From: jan.kanis at phil.uu.nl (Jan Kanis) Date: Sat, 9 May 2009 23:54:03 +0200 Subject: [Python-ideas] Add OrderedSet now that OrderedDict is in collections In-Reply-To: References: Message-ID: <59a221a0905091454u443e346cn5fbbe00c333df39b@mail.gmail.com> > So, let's review what we have in terms of data structures: > > Structure ? ? ? Stable ?Unique ?Python type > ------------------------------------------- > Multiset ? ? ? ?no ? ? ?no ? ? ?- > Set ? ? ? ? ? ? no ? ? ?yes ? ? set > Map ? ? ? ? ? ? no ? ? ?yes ? ? dict > List ? ? ? ? ? ?yes ? ? no ? ? ?list, tuple > Ordered set ? ? yes ? ? yes ? ? - > Ordered map ? ? yes ? ? yes ? ? collections.OrderedDict > > where "stable" means that input order is retained. And where do Multimaps fit in? And Ordered Mulitmaps? Example of an Ordered Multimap in the wild: the headers of an email message, where multiple values may be attached to a single key, and the ordering of key-value pairs is important (and not all key-value pairs with the same key are necessarily adjacent). But basically it is anything that behaves like a sequence of key-value pairs. Come to think of it, 'headers' in lots of network protocols are like this. (ps: +0 on adding ordered sets) From tleeuwenburg at gmail.com Sun May 10 02:19:01 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Sun, 10 May 2009 10:19:01 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A0496BD.3030709@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> Message-ID: <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> Hi Pascal, Taking the example of def foo(bar = []): bar.append(4) print(bar) I'm totally with you in thinking that what is 'natural' is to expect to get a new, empty, list every time. However this isn't want happens. As far as I'm concerned, that should more or less be the end of the discussion in terms of what should ideally happen. The responses to the change in behaviour which I see as more natural are, to summarise, as follows: -- For all sorts of technical reasons, it's too hard -- It changes the semantics of the function definition being evaluated at compile time -- It's not what people are used to With regards to the second point, it's not like the value of arguments is set at compile time, so I don't really see that this stands up. I don't think it's intuitive, it's just that people become accustomed to it. There is indeed, *some sense* in understanding that the evaluation occurs at compile-time, but there is also a lot of sense (and in my opinion, more sense) in understanding the evaluation as happening dynamically when the function is called. With regards to the first point, I'm not sure that this is as significant as all of that, although of course I defer to the language authors here. However, it seems as though it could be no more costly than the lines of code which most frequently follow to initialise these variables. On the final point, that's only true for some people. For a whole lot of people, they stumble over it and get it wrong. It's one of the most un-Pythonic things which I have to remember about Python when programming -- a real gotcha. I don't see it as changing one way of doing things for another equally valid way of doing things, but changing something that's confusing and unexpected for something which is far more natural and, to me, Pythonic. For me, Python 3k appears to be a natural place to do this. Python 3 still appears to be regarded as a work-in-progress by most people, and I don't think that it's 'too late' to change for Python 3k. Perhaps, given the timing, the people involved, the complexity of change etc, then for pragmatic reasons this may have to be delayed, but I don't think that's a good thing. I'd much rather see it done, personally. I think that many people would feel the same way. Regards, -Tennessee On Sat, May 9, 2009 at 6:31 AM, Pascal Chambon wrote: > Hello, > > I'm surely not original in any way there, but I'd like to put back on the > table the matter of "default argument values". > Or, more precisely, the "one shot" handling of default values, which makes > that the same mutable objects, given once as default arguments, come back > again and again at each function call. > They thus become some kinds of "static variables", which get polluted by > the previous calls, whereas many-many-many python users still believe that > they get a fresh new value at each function call. > I think I understand how default arguments are currently implemented (and > so, "why" - technically - it does behave this way), but I'm still unsure of > "why" - semantically - this must be so. > > I've browsed lots of google entries on that subject, but as far as I'm > concerned, I've found nothing in favor current semantic. > I've rather found dozens, hundreds of posts of people complaining that they > got biten by this gotcha, many of them finishing with a "Never put mutable > values in default arguments, unless you're very very sure of what you're > doing !". > > And no one seemed to enjoy the possibilities of getting "potentially static > variables" this way. Static variables are imo a rather bad idea, since they > create "stateful functions", that make debugging and maintenance more > difficult ; but when such static variable are, furthermore, potentially > non-static (i.e when the corresponding function argument is supplied), I > guess they become totally useless and dangerous - a perfect way to get > hard-to-debug behaviours. > > On the other hand, when people write "def func(mylist=[]):", they basically > DO want a fresh new list at each call, be it given by the caller or the > default argument system. > So it's really a pity to need tricks like > >* def f(a, L=None): > *>* if L is None: > *>* L = [] > *to get what we want (and if None was also a possible value ? what other > value should we put as a placeholder for "I'd like None or a fresh new list > but I can't say it directly ?"). > > So I'd like to know : are there other "purely intellectual" arguments > for/against the current semantic of default arguments (I might have missed > some discussion on this subject, feel free to point them ? > > Currently, this default argument handling looks, like a huge gotcha for > newcomers, and, I feel, like an embarrassing wart to most pythonistas. > Couldn't it be worth finding a new way of doing it ? > Maybe there are strong arguments against a change at that level ; for > example, performance issues (I'm not good in those matters). But I need to > ensure. > > So here are my rough ideas on what we might do - if after having the > suggestions from expert people, it looks like it's worth writting a PEP, > I'll be willing to particpateon it. > Basically, I'd change the python system so that, when a default argument > expression is encountered, instead of being executed, it's wrapped in some > kind of zero-argument lambda expression, which gets pushed in the > "func_defaults" attribute of the function. > And then, each time a default argument is required in a function call, this > lambda expression gets evaluated and gives the expected value. > > I guess this will mean some overhead during function call, so this might > become another issue. > It's also a non retrocompatible change, so I assume we'd have to use a > "from __future__ import XXX" until Python4000. > But I think the change is worth the try, because it's a trap which waits > for all the python beginners. > > So, if this matters hasn't already been marked somewhere as a no-go, I > eagerly await the feedback of users and core developpers on the subject. :) > > By the way, I'm becoming slightly allergical to C-like languages (too much > hassle for too little gain, compared to high level dynamic languages), but > if that proposition goes ahead, and no one wants to handle the > implementation details, I'll put the hands in the engine ^^ > > Regards, > Pascal > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sun May 10 03:23:42 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 10 May 2009 13:23:42 +1200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <200905092139.53685.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <4A05661E.6010305@wanadoo.fr> <200905092136.12307.steve@pearwood.info> <200905092139.53685.steve@pearwood.info> Message-ID: <4A062C9E.2020606@canterbury.ac.nz> Steven D'Aprano wrote: > Of course, no Usenet thread is every *completely* finished. Unless Nazis have been mentioned, of course. (Oops, looks like I just ended this thread -- sorry about that!) -- Greg From steve at pearwood.info Sun May 10 03:23:36 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 10 May 2009 11:23:36 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> Message-ID: <200905101123.37344.steve@pearwood.info> On Sun, 10 May 2009 10:19:01 am Tennessee Leeuwenburg wrote: > Hi Pascal, > Taking the example of > > def foo(bar = []): > bar.append(4) > print(bar) > > I'm totally with you in thinking that what is 'natural' is to expect > to get a new, empty, list every time. That's not natural to me. I would be really, really surprised by the behaviour you claim is "natural": >>> DEFAULT = 3 >>> def func(a=DEFAULT): ... return a+1 ... >>> func() 4 >>> DEFAULT = 7 >>> func() 8 For deterministic functions, the same argument list should return the same result each time. By having default arguments be evaluated every time they are required, any function with a default argument becomes non-deterministic. Late evaluation of defaults is, essentially, equivalent to making the default value a global variable. Global variables are rightly Considered Harmful: they should be used with care, if at all. > However this isn't want > happens. As far as I'm concerned, that should more or less be the end > of the discussion in terms of what should ideally happen. As far as I'm concerned, what Python does now is the idea behaviour. Default arguments are part of the function *definition*, not part of the body of the function. The definition of the function happens *once* -- the function isn't recreated each time you call it, so default values shouldn't be recreated either. > The responses to the change in behaviour which I see as more natural > are, to summarise, as follows: > -- For all sorts of technical reasons, it's too hard > -- It changes the semantics of the function definition being > evaluated at compile time > -- It's not what people are used to And it's not what many people want. You only see the people who complain about this feature. For the multitude of people who expect it or like it, they have no reason to say anything (except in response to complaints). When was the last time you saw somebody write to the list to say "Gosh, I really love that Python uses + for addition"? Features that *just work* never or rarely get mentioned. > With regards to the second point, it's not like the value of > arguments is set at compile time, so I don't really see that this > stands up. I don't see what relevance that has. If the arguments are provided at runtime, then the default value doesn't get used. > I don't think it's intuitive, Why do you think that intuitiveness is more valuable than performance and consistency? Besides, intuitiveness is a fickle thing. Given this pair of functions: def expensive_calculation(): time.sleep(60) return 1 def useful_function(x=expensive_calculation()): return x + 1 I think people would be VERY surprised that calling useful_function() with no arguments would take a minute *every time*, and would complain that this slowness was "unintuitive". > it's just that people become > accustomed to it. There is indeed, *some sense* in understanding that > the evaluation occurs at compile-time, but there is also a lot of > sense (and in my opinion, more sense) in understanding the evaluation > as happening dynamically when the function is called. No. The body of the function is executed each time the function is called. The definition of the function is executed *once*, at compile time. Default arguments are part of the definition, not the body, so they too should only be executed once. If you want them executed every time, put them in the body: def useful_function(x=SENTINEL): if x is SENTINEL: x = expensive_calculation() return x+1 > With regards to the first point, I'm not sure that this is as > significant as all of that, although of course I defer to the > language authors here. However, it seems as though it could be no > more costly than the lines of code which most frequently follow to > initialise these variables. > > On the final point, that's only true for some people. For a whole lot > of people, they stumble over it and get it wrong. It's one of the > most un-Pythonic things which I have to remember about Python when > programming -- a real gotcha. I accept that it is a Gotcha. The trouble is, the alternative behaviour you propose is *also* a Gotcha, but it's a worse Gotcha, because it leads to degraded performance, surprising introduction of global variables where no global variables were expected, and a breakdown of the neat distinction between creating a function and executing a function. But as for it being un-Pythonic, I'm afraid that if you really think that, your understanding of Pythonic is weak. From the Zen: The Zen of Python, by Tim Peters Special cases aren't special enough to break the rules. Although practicality beats purity. If the implementation is hard to explain, it's a bad idea. (1) Assignments outside of the body of a function happen once, at compile time. Default values are outside the body of the function. You want a special case for default values so that they too happen at runtime. That's not special enough to warrant breaking the rules. (2) The potential performance degradation of re-evaluating default arguments at runtime is great. For practical reasons, it's best to evaluate them once only. (3) In order to get the behaviour you want, the Python compiler would need a more complicated implementation which would be hard to explain. > I don't see it as changing one way of > doing things for another equally valid way of doing things, but > changing something that's confusing and unexpected for something > which is far more natural and, to me, Pythonic. I'm sorry, while re-evaluation of default arguments is sometimes useful, it's more often NOT useful. Most default arguments are simple objects like small ints or None. What benefit do you gain from re-evaluating them every single time? Zero benefit. (Not much cost either, for simple cases, but no benefit.) But for more complex cases, there is great benefit to evaluating default arguments once only, and an easy work-around for those rare cases that you do want re-evaluation. > For me, Python 3k appears to be a natural place to do this. Python 3 > still appears to be regarded as a work-in-progress by most people, > and I don't think that it's 'too late' to change for Python 3k. Fortunately you're not Guido, and fortunately this isn't going to happen. I recommend you either accept that this behaviour is here to stay, or if you're *particularly* enamoured of late evaluation behaviour of defaults, that you work on some sort of syntax to make it optional. -- Steven D'Aprano From cmjohnson.mailinglist at gmail.com Sun May 10 04:06:06 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 9 May 2009 16:06:06 -1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <200905101123.37344.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> Message-ID: <3bdda690905091906y1c5cbf2dvb5e53c77cb989f06@mail.gmail.com> I think this is a case where there are pros and cons on both sides. There are a lot of pros to the current behavior (performance, flexibility, etc.), but it comes with the con of confusing newbies and making people go through the same song and dance to set a ?"sentinel value" when the want the other behavior and they can't ensure that None won't be passed. The newbie problem can't be fixed from now until Python 4000, since it would break a lot of existing uses of default values, but we could cut down on the annoyance of setting and check a sentinel value by introducing a new keyword, eg. def f(l=fresh []): ? ?... instead of __blank = object() def f(l=__blank): ? ?if l is __blank: ? ? ? ?l = [] ? ?... The pros of a new keyword are saving 3 lines and being more clear upfront about what's going on with the default value . The con is that adding a new keyword bloats the language. We could try reusing an existing keyword, but none of the current ones seem to fit: and ? ? ? ? ? ? ? ? elif ? ? ? ? ? ? ? ?import ? ? ? ? ? ? ?return as ? ? ? ? ? ? ? ? ?else ? ? ? ? ? ? ? ?in ? ? ? ? ? ? ? ? ?try assert ? ? ? ? ? ? ?except ? ? ? ? ? ? ?is ? ? ? ? ? ? ? ? ?while break ? ? ? ? ? ? ? finally ? ? ? ? ? ? lambda ? ? ? ? ? ? ?with class ? ? ? ? ? ? ? for ? ? ? ? ? ? ? ? not ? ? ? ? ? ? ? ? yield continue ? ? ? ? ? ?from ? ? ? ? ? ? ? ?or def ? ? ? ? ? ? ? ? global ? ? ? ? ? ? ?pass del ? ? ? ? ? ? ? ? if ? ? ? ? ? ? ? ? ?raise (I copied this from Python 3.0's help, but there seems to be a documentation error: nonlocal, None, True, and False are also keywords in Python 3+.) The best one on the current list it seems to me would be "else" as in def f(l else []): ? ?... But I dunno? It just not quite right, you know? So, I'm -0 on changing the current behavior, but I'm open to it if someone can find a way to do it that isn't just an ad hoc solution to this one narrow problem but has a wider general use. From tleeuwenburg at gmail.com Sun May 10 04:06:50 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Sun, 10 May 2009 12:06:50 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <200905101123.37344.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> Message-ID: <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> > > > > For me, Python 3k appears to be a natural place to do this. Python 3 > > still appears to be regarded as a work-in-progress by most people, > > and I don't think that it's 'too late' to change for Python 3k. > > Fortunately you're not Guido, and fortunately this isn't going to > happen. I recommend you either accept that this behaviour is here to > stay, or if you're *particularly* enamoured of late evaluation > behaviour of defaults, that you work on some sort of syntax to make it > optional. Thank you for the rest of the email, which was (by and large) well-considered and (mostly) stuck to the points of the matter. I will get to them in proper time when I have been able to add to the argument in a considered way after fully understanding your points. However, this last section really got under my skin. It seems completely inappropriate to devolve any well-intentioned email discussion into an appalling self-service ad-hominem attack. Your assertion of your ethical viewpoint (use of Fortunately without a backing argument), attempt to bully me out of my position (recomment you accept this behaviour is here to stay) are not appreciated. You have *your* view of what is fortunate, right and appropriate. I took every care NOT to assert my own viewpoint as universally true; you have not done so. Guido is just a person, as you are just a person, as I am just a person. Can we not please just stick to a simple, civilised discussion of the point without trying to win cheap debating points or use the "Zen" of Python to denigrate people who have either genuinely failed to grasp some aspect of a concept, or whose intuition is simply different. Without people whose intuition is different, no advancement is possible. Without debate about what constitutes the "Zen" of Python, the "Zen" of Python must always be static, unchanging, unchallenged and therefore cannot grow. I do not think that is what anyone meant when they were penning the "Zen" of Python. This list is not best-served by grandstanding. It may not even be best-served by the now effectively personal debate which you have drawn me into through your personalisation of the issue (I quote: "your understanding is weak"). Terms such as weak and strong are inherently laden with ethical and social overtones -- incomplete, misplaced, or any number of other qualifiers could have kept the debate to the factual level. Regards, -Tennessee -------------- next part -------------- An HTML attachment was scrubbed... URL: From Scott.Daniels at Acm.Org Sun May 10 07:28:13 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sat, 09 May 2009 22:28:13 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> Message-ID: Any argument for changing to a more "dynamic" default scheme had better have a definition of the behavior of the following code, and produce a good rationale for that behavior: x = 5 def function_producer(y): def inner(arg=x+y): return arg + 2 return inner x = 6 f1 = function_producer(x) x = 4.1 y = 7 f2 = function_producer(3) print x, y, f1(), f2() del y x = 45 print x, f1(), f2() del x print f1(), f2() --Scott David Daniels Scott.Daniels at Acm.Org From tjreedy at udel.edu Sun May 10 07:47:12 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 10 May 2009 01:47:12 -0400 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> Message-ID: Tennessee Leeuwenburg wrote: > > > For me, Python 3k appears to be a natural place to do this. Python 3 > > still appears to be regarded as a work-in-progress by most people, > > and I don't think that it's 'too late' to change for Python 3k. Sorry, it *is* too late. The developers have been very careful about breaking 3.0 code in 3.1 only with strong justification. 3.1 is in feature freeze as of a few days ago. > Fortunately you're not Guido, and fortunately this isn't going to > happen. I recommend you either accept that this behaviour is here to > stay, or if you're *particularly* enamoured of late evaluation > behaviour of defaults, that you work on some sort of syntax to make it > optional. > Thank you for the rest of the email, which was (by and large) > well-considered and (mostly) stuck to the points of the matter. I will > get to them in proper time when I have been able to add to the argument > in a considered way after fully understanding your points. > > However, this last section really got under my skin. It seems completely > inappropriate to devolve any well-intentioned email discussion into an > appalling self-service ad-hominem attack. I do not see any attack whatsoever, just advice which you took wrongly. > ...se of Fortunately without a backing argument), 'Fortunately' as is clear from the context, was in respect to your expressed casual attitude toward breaking code. Some people have a negative reaction to that. In any case, it is a separate issue from 'default arguments'. > attempt to bully me out of my position (recomment you accept this behaviour > is here to stay) are not appreciated. He recommended that you not beat your head against a brick wall because of a misconception about what is currently socially possible. He then suggested something that *might* be possible. If that advice offends you, so be it. Terry Jan Reedy From george.sakkis at gmail.com Sun May 10 08:32:07 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 10 May 2009 02:32:07 -0400 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> Message-ID: <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> On Sun, May 10, 2009 at 1:28 AM, Scott David Daniels wrote: > > Any argument for changing to a more "dynamic" default scheme had better > have a definition of the behavior of the following code, and produce a > good rationale for that behavior: > > ? ?x = 5 > ? ?def function_producer(y): > ? ? ? ?def inner(arg=x+y): > ? ? ? ? ? ?return arg + 2 > ? ? ? ?return inner I don't think the proposed scheme was ever accused of not being well-defined. Here's the current equivalent dynamic version: x = 5 def function_producer(y): missing = object() def inner(arg=missing): if arg is missing: arg = x+y return arg + 2 return inner -1 for changing the current semantics (too much potential breakage), +0.x for a new keyword that adds dynamic semantics (and removes the need for the sentinel kludge). George From larry at hastings.org Sun May 10 12:14:32 2009 From: larry at hastings.org (Larry Hastings) Date: Sun, 10 May 2009 03:14:32 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> Message-ID: <4A06A908.3070507@hastings.org> George Sakkis wrote: > +0.x for a new keyword that adds dynamic semantics (and removes the > need for the sentinel kludge). We don't need new syntax for it. Here's a proof-of-concept hack that you can do it with a function decorator. import copy def clone_arguments(f): default_args = list(f.func_defaults) if len(default_args) < f.func_code.co_argcount: delta = f.func_code.co_argcount - len(default_args) default_args = ([None] * delta) + default_args def fn(*args): if len(args) < default_args: args = args + tuple(copy.deepcopy(default_args[len(args):])) return f(*args) return fn @clone_arguments def thing_taking_array(a, b = []): b.append(a) return b print thing_taking_array('123') print thing_taking_array('abc') -1 on changing Python one iota for this, /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Sun May 10 12:34:32 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sun, 10 May 2009 00:34:32 -1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A06A908.3070507@hastings.org> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> <4A06A908.3070507@hastings.org> Message-ID: <3bdda690905100334j60f7f5b4ibda4f7e93e33d284@mail.gmail.com> Larry Hastings wrote: > George Sakkis wrote: > > +0.x for a new keyword that adds dynamic semantics (and removes the > need for the sentinel kludge). > > We don't need new syntax for it.? Here's a proof-of-concept hack that you > can do it with a function decorator. Your decorator only works for mutables where you just want a deep copy. It doesn't work for cases where you want a whole expression to be re-evaluated from scratch. (Maybe for the side effects or something.) That said, it couldn't be that hard to work out a similar decorator using lambda thunks instead. The internals of the decorator would be something like: for n, arg in enumerate(args): if arg is defaults[n]: #If you didn't get passed anything args[n] = defaults[n]() #Unthunk the lambda The usage might be: @dynamicdefaults def f(arg=lambda: dosomething()): It cuts 3 lines of boilerplate down to one line, but makes all your function calls a little slower. -- Carl From denis.spir at free.fr Sun May 10 13:19:29 2009 From: denis.spir at free.fr (spir) Date: Sun, 10 May 2009 13:19:29 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <3bdda690905091906y1c5cbf2dvb5e53c77cb989f06@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <3bdda690905091906y1c5cbf2dvb5e53c77cb989f06@mail.gmail.com> Message-ID: <20090510131929.2302acb9@o> Le Sat, 9 May 2009 16:06:06 -1000, Carl Johnson s'exprima ainsi: > I think this is a case where there are pros and cons on both sides. > There are a lot of pros to the current behavior (performance, > flexibility, etc.), but it comes with the con of confusing newbies and > making people go through the same song and dance to set a ?"sentinel > value" when the want the other behavior and they can't ensure that > None won't be passed. The newbie problem can't be fixed from now until > Python 4000, since it would break a lot of existing uses of default > values, but we could cut down on the annoyance of setting and check a > sentinel value by introducing a new keyword, eg. > > def f(l=fresh []): > ? ?... > > instead of > > __blank = object() > def f(l=__blank): > ? ?if l is __blank: > ? ? ? ?l = [] > ? ?... [...] Maybe the correctness of the current behaviour can be checked by a little mental experiment. ======= Just imagine python hasn't default arguments yet, and they are the object of a PEP. An implementation similar to the current one is proposed. Then, people realise that in the case where the given value happens to be mutable _and_ updated in the function body,... What do you think should/would be decided? -1- Great, we get static variables for free. It is a worthful feature we expected for a while. In numerous use cases they will allow easier and much more straightforward code. Let's go for it. -2- Well, static variables may be considered useful, in which case there should be a new PEP for them. Conceptually, they are a totally different feature, we shall certainly not mess up both, shall we? ======= I bet for n?2, for the reasoning of people stating it's a major gotcha will be hard to ignore. But may be wrong. Still, default arguments actually *are* called "default arguments", which means they should be considered as such, while they do not behave as such in all cases. Now, we must consider the concrete present situation in which their real behaviour is used as a common workaround. I do not really understand why default args are used as static vars while at least another possibility exists in python which is semantically much more consistent: ### instead of "def callSave(number, record=[])" ### just set record on the func: def callSave(value): callSave.record.append(value) return callSave.record callSave.record = [] print callSave(1) ; print callSave(2) ; print callSave(3) ==> [1] [1, 2] [1, 2, 3] Also, func attributes are an alternative for another common (mis)use of default arguments, namely the case of a function factory: def paramPower(exponent): ### instead of "def power(number, exponent=exponent)" ### just set exponent on the func: def power(number): return number**power.exponent power.exponent = exponent return power power3 = paramPower(3) ; power5 = paramPower(5) print power3(2) ; print power5(2) ==> 8 32 In both cases, tha notion of a func attribute rather well matches the variable value's meaning. As a consequence, I find this solution much nicer for a func factory as well as for a static variable. Denis ------ la vita e estrany From tleeuwenburg at gmail.com Sun May 10 14:12:25 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Sun, 10 May 2009 22:12:25 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> Message-ID: <43c8685c0905100512t4307f561qcfc842bbf6bd6c4@mail.gmail.com> > > >> However, this last section really got under my skin. It seems completely >> inappropriate to devolve any well-intentioned email discussion into an >> appalling self-service ad-hominem attack. >> > > I do not see any attack whatsoever, just advice which you took wrongly. > > ...se of Fortunately without a backing argument), >> > > 'Fortunately' as is clear from the context, was in respect to your > expressed casual attitude toward breaking code. Some people have a negative > reaction to that. In any case, it is a separate issue from 'default > arguments'. > > > attempt to bully me out of my position (recomment you accept this > behaviour > > is here to stay) are not appreciated. > > He recommended that you not beat your head against a brick wall because of > a misconception about what is currently socially possible. He then > suggested something that *might* be possible. If that advice offends you, > so be it. It's not the content of the advice (don't push stuff uphill) which got to me at all, it was the tone and manner in which it was conveyed. Much of the email was well-balanced, which I fully acknowledged. Maybe you're just more inclined to overlook a few bits of innuendo, and probably most of the time so am I. However, it's actually not okay, and the implied personal criticism was very clearly present. It wasn't severe, ,perhaps my reaction was quite forceful, but it's just not okay to be putting people down. I don't have a casual attitude towards breaking code, just an open mind towards discussions on their merits. I don't really appreciate the negative tones, and I'm sure that if anyone else is in the firing line, they wouldn't appreciate either, even if it to some extent it's all a bit of a storm in a teacup. Unless someone who is happy to cop a bit of flak stands up and says that's not on, then maintaining a "thick skin" -- i.e. putting up with people putting you down, be it through a clear and direct put-down, or through a more subtle implication -- becomes the norm. It becomes acceptable, perhaps indeed even well-regarded, to take a certain viewpoint then suggest that anyone who doesn't share it is doing something wrong. Well nuts to that. Emails are, as everyone should know, an unclear communication channel. I've found myself on the wrong side of this kind of debate before, and I've heard plenty of stories of people who were put down, pushed out or made to feel stupid -- and for what? There are just so many stories, many of which I have heard first-hand, of people who have felt alienated on online lists where prowess and insight are so highly regarded that they become means by which others are put down. It's that larger problem which people need not to put up with. However, I'm just about to go offline for 12 hours or so, and I know the US will be waking up to their emails shortly, so I just wanted to take this opportunity before the sun rotates again to say to the list and the original author that I'd really like to avoid a continued shouting contest or make anyone upset. I've obviously ruffled some feathers already, and I guess probably this email may ruffle some more, but really I just want to make clear that : (a) It's not okay to put myself or anyone else down, claiming some personal superiority (b) That attitude is all this email is about. It doesn't need to be any bigger than that. Regards, -Tennessee -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Sun May 10 16:58:30 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 10 May 2009 16:58:30 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <20090510131929.2302acb9@o> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <3bdda690905091906y1c5cbf2dvb5e53c77cb989f06@mail.gmail.com> <20090510131929.2302acb9@o> Message-ID: spir schrieb: > Also, func attributes are an alternative for another common (mis)use of default arguments, namely the case of a function factory: > > def paramPower(exponent): > ### instead of "def power(number, exponent=exponent)" > ### just set exponent on the func: > def power(number): > return number**power.exponent > power.exponent = exponent > return power > power3 = paramPower(3) ; power5 = paramPower(5) > print power3(2) ; print power5(2) > ==> > 8 > 32 You don't need a function attribute here; just "exponent" will work fine. The problem is where you define multiple functions, e.g. in a "for" loop, and function attributes don't help there: adders = [] for i in range(10): def adder(x): return x + i This will fail to do what it seems to do; you need to have some kind of binding "i" in a scope where it stays constant. You could use a "make_adder" factory function, similar to your paramPower, but that is more kludgy than necessary, because it can easily be solved by a default argument. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sun May 10 16:59:11 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 10 May 2009 16:59:11 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905100512t4307f561qcfc842bbf6bd6c4@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <43c8685c0905100512t4307f561qcfc842bbf6bd6c4@mail.gmail.com> Message-ID: Tennessee Leeuwenburg schrieb: > I don't have a casual attitude towards breaking code, just an open mind > towards discussions on their merits. I don't really appreciate the > negative tones, and I'm sure that if anyone else is in the firing line, > they wouldn't appreciate either, even if it to some extent it's all a > bit of a storm in a teacup. Unless someone who is happy to cop a bit of > flak stands up and says that's not on, then maintaining a "thick skin" > -- i.e. putting up with people putting you down, be it through a clear > and direct put-down, or through a more subtle implication -- becomes the > norm. It becomes acceptable, perhaps indeed even well-regarded, to take > a certain viewpoint then suggest that anyone who doesn't share it is > doing something wrong. The problem is that people whose proposal immediately meets negative reactions usually feel put down no matter what exactly you say to them. If there was a polite way of saying "This will not change, please don't waste more of our time with this discussion." that still gets the point across, I would be very grateful. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From arnodel at googlemail.com Sun May 10 18:10:33 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 10 May 2009 17:10:33 +0100 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A06A908.3070507@hastings.org> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> <4A06A908.3070507@hastings.org> Message-ID: <5F6AB4B5-3AC5-45B7-8F69-BB71B0593DF3@googlemail.com> On 10 May 2009, at 11:14, Larry Hastings wrote: > George Sakkis wrote: >> >> +0.x for a new keyword that adds dynamic semantics (and removes the >> need for the sentinel kludge). > > > We don't need new syntax for it. Here's a proof-of-concept hack > that you can do it with a function decorator. > import copy Not that we don't need syntax for default arguments either :) Here is a decorator that does it: def default(**defaults): defaults = defaults.items() def decorator(f): def decorated(*args, **kwargs): for name, val in defaults: kwargs.setdefault(name, val) return f(*args, **kwargs) return decorated return decorator Here it is in action: >>> z=1 >>> @default(z=z) ... def foo(a, z): ... print a + z ... >>> z=None >>> foo(3) 4 >>> >>> @default(history=[]) ... def bar(x, history): ... history.append(x) ... return list(history) ... >>> map(bar, 'spam') [['s'], ['s', 'p'], ['s', 'p', 'a'], ['s', 'p', 'a', 'm']] Let's get rid of default arguments altogether, and we will have solved the problem! Furthemore, by removing default arguments from the language, we can let people choose what semantics they want for default arguments. I.e, if they want them to be reevaluated each time, they could write the default decorator as follows (it is exactly the same as the one above except for a pair of parentheses that have been added on one line. def dynamic_default(**defaults): defaults = defaults.items() def decorator(f): def decorated(*args, **kwargs): for name, val in defaults: kwargs.setdefault(name, val()) # ^^ return f(*args, **kwargs) return decorated return decorator Example: >>> @dynamic_default(l=list) ... def baz(a, l): ... l.append(a) ... return l ... >>> baz(2) [2] >>> baz(3) [3] ;) -- Arnaud From george.sakkis at gmail.com Sun May 10 19:00:15 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 10 May 2009 13:00:15 -0400 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <5F6AB4B5-3AC5-45B7-8F69-BB71B0593DF3@googlemail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> <4A06A908.3070507@hastings.org> <5F6AB4B5-3AC5-45B7-8F69-BB71B0593DF3@googlemail.com> Message-ID: <91ad5bf80905101000j71717455y2d667227a9349822@mail.gmail.com> On Sun, May 10, 2009 at 12:10 PM, Arnaud Delobelle wrote: > Furthemore, by removing default arguments from the language, we can let > people choose what semantics they want for default arguments. ?I.e, if they > want them to be reevaluated each time, they could write the default > decorator as follows (it is exactly the same as the one above except for a > pair of parentheses that have been added on one line. Cute, but that's still a subset of what the dynamic semantics would provide; the evaluated thunks would have access to the previously defined arguments: def foo(a, b, d=(a*a+b+b)**0.5, s=1/d): return (a,b,d,s) would be equivalent to missing = object() def foo(a, b, d=missing, s=missing): if d is missing: d = (a*a+b+b)**0.5 if s is missing: s = 1/d return (a,b,d,s) George From floris.bruynooghe at gmail.com Sun May 10 19:03:22 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Sun, 10 May 2009 18:03:22 +0100 Subject: [Python-ideas] Add __nonzero__ to threading.Event Message-ID: <20090510170322.GA17770@laurie.devork> Hi I was wondering why threading.Event hasn't got the __nonzero__ special attribute? I think it would allow for code to be more pythonic if you could just do "if e: ..." instead of "if e.isSet(): ..." (or "if e.is_set(): ..." in 2.6+). The patch is trivial: Index: Lib/threading.py =================================================================== --- Lib/threading.py (revision 72551) +++ Lib/threading.py (working copy) @@ -371,6 +371,9 @@ is_set = isSet + def __nonzero__(self): + return self.__flag + def set(self): self.__cond.acquire() try: Is it worth submitting this to the tracker or is there some reason why this is a bad idea that I've missed? Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From aahz at pythoncraft.com Sun May 10 19:20:16 2009 From: aahz at pythoncraft.com (Aahz) Date: Sun, 10 May 2009 10:20:16 -0700 Subject: [Python-ideas] Add __nonzero__ to threading.Event In-Reply-To: <20090510170322.GA17770@laurie.devork> References: <20090510170322.GA17770@laurie.devork> Message-ID: <20090510172016.GA26119@panix.com> On Sun, May 10, 2009, Floris Bruynooghe wrote: > > I was wondering why threading.Event hasn't got the __nonzero__ special > attribute? I think it would allow for code to be more pythonic if you > could just do "if e: ..." instead of "if e.isSet(): ..." > (or "if e.is_set(): ..." in 2.6+). > > [...] > > Is it worth submitting this to the tracker or is there some reason why > this is a bad idea that I've missed? Please submit to the tracker so that even if it *is* a bad idea it will get recorded. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From Scott.Daniels at Acm.Org Sun May 10 20:30:49 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun, 10 May 2009 11:30:49 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> Message-ID: George Sakkis wrote: > On Sun, May 10, 2009 at 1:28 AM, Scott David Daniels > wrote: >> Any argument for changing to a more "dynamic" default scheme had better >> have a definition of the behavior of the following code, and produce a >> good rationale for that behavior: >> >> x = 5 >> def function_producer(y): >> def inner(arg=x+y): >> return arg + 2 >> return inner > > I don't think the proposed scheme was ever accused of not being > well-defined. Here's the current equivalent dynamic version: > > x = 5 > def function_producer(y): > missing = object() > def inner(arg=missing): > if arg is missing: > arg = x+y > return arg + 2 > return inner So sorry. def function_producer(y): def inner(arg=x+y): return arg + 2 y *= 10 return inner I was trying to point out that it becomes much trickier building functions with dynamic parts, and fluffed the example. --Scott David Daniels Scott.Daniels at Acm.Org From larry at hastings.org Sun May 10 21:27:15 2009 From: larry at hastings.org (Larry Hastings) Date: Sun, 10 May 2009 12:27:15 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <5F6AB4B5-3AC5-45B7-8F69-BB71B0593DF3@googlemail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> <4A06A908.3070507@hastings.org> <5F6AB4B5-3AC5-45B7-8F69-BB71B0593DF3@googlemail.com> Message-ID: <4A072A93.9060300@hastings.org> Arnaud Delobelle wrote: > Not that we don't need syntax for default arguments either :) Here is > a decorator that does it: [...] Let's get rid of default arguments > altogether, and we will have solved the problem! Furthemore, by > removing default arguments from the language, we can let people choose > what semantics they want for default arguments. [...] ;) Comedy or not, I don't support getting rid of default arguments. Nor do I support changing the semantics of default arguments so they represent code that is run on each function invocation. As I demonstrated, people can already choose what semantics they want for default arguments, by choosing whether or not to decorate their functions with clone_arguments or the like. We don't need to remove default arguments from the language for that to happen. /larry/ From george.sakkis at gmail.com Sun May 10 22:06:53 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Sun, 10 May 2009 16:06:53 -0400 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <91ad5bf80905101000j71717455y2d667227a9349822@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> <4A06A908.3070507@hastings.org> <5F6AB4B5-3AC5-45B7-8F69-BB71B0593DF3@googlemail.com> <91ad5bf80905101000j71717455y2d667227a9349822@mail.gmail.com> Message-ID: <91ad5bf80905101306y1f782488l82c086fdf5d46c22@mail.gmail.com> On Sun, May 10, 2009 at 1:00 PM, George Sakkis wrote: > On Sun, May 10, 2009 at 12:10 PM, Arnaud Delobelle > wrote: > >> Furthemore, by removing default arguments from the language, we can let >> people choose what semantics they want for default arguments. ?I.e, if they >> want them to be reevaluated each time, they could write the default >> decorator as follows (it is exactly the same as the one above except for a >> pair of parentheses that have been added on one line. > > Cute, but that's still a subset of what the dynamic semantics would > provide; the evaluated thunks would have access to the previously > defined arguments: > > def foo(a, b, d=(a*a+b+b)**0.5, s=1/d): > ? ?return (a,b,d,s) > > would be equivalent to > > missing = object() > def foo(a, b, d=missing, s=missing): > ? ?if d is missing: > ? ? ? ?d = (a*a+b+b)**0.5 > ? ?if s is missing: > ? ? ? ?s = 1/d > ? ?return (a,b,d,s) Just for kicks, here's a decorator that supports dependent dynamically computed defaults; it uses eval() to create the lambdas on the fly: @evaldefaults('s','d') def foo(a, b, d='(a*a+b*b)**0.5', t=0.1, s='(1+t)/(d+t)'): return (a,b,d,t,s) print foo(3,4) #======= import inspect import functools # from http://code.activestate.com/recipes/551779/ from getcallargs import getcallargs def evaldefaults(*eval_params): eval_params = frozenset(eval_params) def decorator(f): params,_,_,defaults = inspect.getargspec(f) param2default = dict(zip(params[-len(defaults):], defaults)) if defaults else {} param2lambda = {} for p in eval_params: argsrepr = ','.join(params[:params.index(p)]) param2lambda[p] = eval('lambda %s: %s' % (argsrepr, param2default[p]), f.func_globals) @functools.wraps(f) def wrapped(*args, **kwargs): allkwargs,missing = getcallargs(f, *args, **kwargs) missing_eval_params = eval_params.intersection(missing) f_locals = {} for i,param in enumerate(params): value = allkwargs[param] if param in missing_eval_params: allkwargs[param] = value = param2lambda[param](**f_locals) f_locals[param] = value return f(**allkwargs) return wrapped return decorator George From solipsis at pitrou.net Mon May 11 02:11:33 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 11 May 2009 00:11:33 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Add_=5F=5Fnonzero=5F=5F_to_threading=2EE?= =?utf-8?q?vent?= References: <20090510170322.GA17770@laurie.devork> Message-ID: Floris Bruynooghe writes: > > Is it worth submitting this to the tracker or is there some reason why > this is a bad idea that I've missed? First, __nonzero__ should only be used when the notion of a truth value is obvious. I'm not sure it is the case here, although I don't find it shocking. Second, adding it would break compatibility for code using "if not event" as a shorthand for "if event is None". In any case, as Aahz said, please file a bug in the tracker. Regards Antoine. From tleeuwenburg at gmail.com Mon May 11 02:45:31 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Mon, 11 May 2009 10:45:31 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <200905101123.37344.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> Message-ID: <43c8685c0905101745s50370a21j51a1a325ae887eb1@mail.gmail.com> On Sun, May 10, 2009 at 11:23 AM, Steven D'Aprano wrote: > On Sun, 10 May 2009 10:19:01 am Tennessee Leeuwenburg wrote: > > Hi Pascal, > > Taking the example of > > > > def foo(bar = []): > > bar.append(4) > > print(bar) > > > > I'm totally with you in thinking that what is 'natural' is to expect > > to get a new, empty, list every time. > > That's not natural to me. I would be really, really surprised by the > behaviour you claim is "natural": > > >>> DEFAULT = 3 > >>> def func(a=DEFAULT): > ... return a+1 > ... > >>> func() > 4 > >>> DEFAULT = 7 > >>> func() > 8 Good example! If I may translate that back into the example using a list to make sure I've got it right... default = [] def func(a=default): a.append(5) func() func() default will now be [5,5] > For deterministic functions, the same argument list should return the > same result each time. By having default arguments be evaluated every > time they are required, any function with a default argument becomes > non-deterministic. Late evaluation of defaults is, essentially, > equivalent to making the default value a global variable. Global > variables are rightly Considered Harmful: they should be used with > care, if at all. If I can just expand on that point somewhat... In the example I gave originally, I had in mind someone designing a function, whereby it could be called either with some pre-initialised term, or otherwise it would use a default value of []. I imagined a surprised designer finding that the default value of [] was a pointer to a specific list, rather than a new empty list each time. e.g. def foo(bar = []): bar.append(5) return bar The same argument list (i.e. no arguments) would result in a different result being returned every time. On the first call, bar would be [5], then [5,5], then [5,5,5]; yet the arguments passed (i.e. none, use default) would not have changed. You have come up with another example. I think it is designed to illustrate that a default argument doesn't need to specify a default value for something, but could be a default reference (such as a relatively-global variable). In that case, it is modifying something above its scope. To me, that is what you would expect under both "ways of doing things". I wonder if I am missing your point... I'm totally with you on the Global Variables Are Bad principle, however. I don't design them in myself, and where I have worked with them, usually they have just caused confusion. > However this isn't want > > happens. As far as I'm concerned, that should more or less be the end > > of the discussion in terms of what should ideally happen. > > As far as I'm concerned, what Python does now is the idea behaviour. > Default arguments are part of the function *definition*, not part of > the body of the function. The definition of the function happens > *once* -- the function isn't recreated each time you call it, so > default values shouldn't be recreated either. I agree that's how you see things, and possibly how many people see things, but I don't accept that it is a more natural way of seeing things. However, what *I* think is more natural is just one person's viewpoint... I totally see the philosophical distinction you are trying to draw, and it certainly does help to clarify why things are the way they are. However, I just don't know that it's the best way they could be. > The responses to the change in behaviour which I see as more natural > > are, to summarise, as follows: > > -- For all sorts of technical reasons, it's too hard > > -- It changes the semantics of the function definition being > > evaluated at compile time > > -- It's not what people are used to > > And it's not what many people want. > > You only see the people who complain about this feature. For the > multitude of people who expect it or like it, they have no reason to > say anything (except in response to complaints). When was the last time > you saw somebody write to the list to say "Gosh, I really love that > Python uses + for addition"? Features that *just work* never or rarely > get mentioned. :) ... well, that's basically true. Of course there are some particular aspects of Python which are frequently mentioned as being wonderful, but I see your point. However, I'm not sure we really know one way or another about what people want then -- either way. > > With regards to the second point, it's not like the value of > > arguments is set at compile time, so I don't really see that this > > stands up. > > I don't see what relevance that has. If the arguments are provided at > runtime, then the default value doesn't get used. I think this is the fundamental difference -- to me speaks worlds :) ... I think you just have a different internal analogy for programming than I do. That's fine. To me, I don't see that a line of code should not be dynamically evaluated just because it's part of the definition. I just don't see why default values shouldn't be (or be able to be) dynamically evaluated. Personally I think that doing it all the time is more natural, but I certainly don't see why allowing the syntax would be bad. I'd basically do that 100% of the time. I'm not sure I've ever used a default value other than None in a way which I wouldn't want dynamically evaluated. > > I don't think it's intuitive, > > Why do you think that intuitiveness is more valuable than performance > and consistency? Because I like Python more than C? I'm pretty sure everyone here would agree than in principle, elegance of design and intuitive syntax are good. Agreeing on what that means might involve some robust discussion, but I think everyone would like the same thing. Well, consistency is pretty hard to do without... :) > Besides, intuitiveness is a fickle thing. Given this pair of functions: > > def expensive_calculation(): > time.sleep(60) > return 1 > > def useful_function(x=expensive_calculation()): > return x + 1 > > I think people would be VERY surprised that calling useful_function() > with no arguments would take a minute *every time*, and would complain > that this slowness was "unintuitive". That seems at first like a good point. It is a good point, but I don't happen to side with you on this issue, although I do see that many people might. The code that I write is not essentially performance-bound. It's a lot more design-bound (by which I mean it's very complicated, and anything I can do to simplify it is well-worth a bit of a performance hit). However, when the design options are available (setting aside what default behaviour should be), it's almost always possible to design things how you'd like them. e.g. def speed_critical_function(x=None): if x is None: time.sleep(60: return 1 def handy_simple_function(foo=5, x=[]): or maybe (foo=5, x = new []): for i in range(5): x.append(i) return x Then, thinking about it a little more (and bringing back a discussion of default behaviour), I don't really see why the implementation of the dynamic function definition would be any slower than using None to indicate it wasn't passed in, followed by explicit default-value setting. > > it's just that people become > > accustomed to it. There is indeed, *some sense* in understanding that > > the evaluation occurs at compile-time, but there is also a lot of > > sense (and in my opinion, more sense) in understanding the evaluation > > as happening dynamically when the function is called. > > No. The body of the function is executed each time the function is > called. The definition of the function is executed *once*, at compile > time. Default arguments are part of the definition, not the body, so > they too should only be executed once. If you want them executed every > time, put them in the body: > > def useful_function(x=SENTINEL): > if x is SENTINEL: > x = expensive_calculation() > return x+1 I agree that's how things *are* done, but I just don't see why it should be that way, beyond it being what people are used to. It seems like there is no reason why it would be difficult to implement CrazyPython which does things as I suggest. Given that, it also doesn't seem like there is some inherent reason to prefer the design style of RealActualPython over CrazyPython. Except, of course that RealActualPython exists and I can use it right now (thanks developers!), versus CrazyPython which is just an idea. > With regards to the first point, I'm not sure that this is as > > significant as all of that, although of course I defer to the > > language authors here. However, it seems as though it could be no > > more costly than the lines of code which most frequently follow to > > initialise these variables. > > > > On the final point, that's only true for some people. For a whole lot > > of people, they stumble over it and get it wrong. It's one of the > > most un-Pythonic things which I have to remember about Python when > > programming -- a real gotcha. > > > I accept that it is a Gotcha. The trouble is, the alternative behaviour > you propose is *also* a Gotcha, but it's a worse Gotcha, because it > leads to degraded performance, surprising introduction of global > variables where no global variables were expected, and a breakdown of > the neat distinction between creating a function and executing a > function. > > But as for it being un-Pythonic, I'm afraid that if you really think > that, your understanding of Pythonic is weak. From the Zen: > > The Zen of Python, by Tim Peters > > Special cases aren't special enough to break the rules. > Although practicality beats purity. > If the implementation is hard to explain, it's a bad idea. > > (1) Assignments outside of the body of a function happen once, at > compile time. Default values are outside the body of the function. You > want a special case for default values so that they too happen at > runtime. That's not special enough to warrant breaking the rules. Your logic is impeccable :) ... yet, if I may continue to push my wheelbarrow uphill for a moment longer, I would argue that is an implementation detail, not a piece of design philosophy. > (2) The potential performance degradation of re-evaluating default > arguments at runtime is great. For practical reasons, it's best to > evaluate them once only. Maybe that's true. I guess I have two things to say on that point. The first is that I'm still not sure that's really true in a problematic way. Anyone wanting efficiency could continue to use sentinel values of None (which obviously don't need to be dynamically evaluated) while other cases would surely be no slower than the initialisation code would be anyway. Is the cost issue really that big a problem? The other is that while pragmatics is, of course, a critical issue, it's also true that it's well-worth implementing more elegant language features if possible. It's always a balance. The fastest languages are always less 'natural', while the more elegant and higher-level languages somewhat slower. Where a genuine design improvement is found, I think it's worth genuinely considering including that improvement, even if it is not completely pragmatic. > (3) In order to get the behaviour you want, the Python compiler would > need a more complicated implementation which would be hard to explain. Yes, that's almost certainly true. > > I don't see it as changing one way of > > doing things for another equally valid way of doing things, but > > changing something that's confusing and unexpected for something > > which is far more natural and, to me, Pythonic. > > I'm sorry, while re-evaluation of default arguments is sometimes useful, > it's more often NOT useful. Most default arguments are simple objects > like small ints or None. What benefit do you gain from re-evaluating > them every single time? Zero benefit. (Not much cost either, for simple > cases, but no benefit.) > > But for more complex cases, there is great benefit to evaluating default > arguments once only, and an easy work-around for those rare cases that > you do want re-evaluation. Small ints and None are global pointers (presumably!) so there is no need to re-evaluate them every time. The list example is particularly relevant (ditto empty dictionary) since I think that would be one of the most common cases for re-evaluation. Presumably a reasonably efficient implementation could be worked out such that dynamic evaluation of the default arguments (and indeed the entire function definition) need only occur where a dynamic default value were included. I agree that the workaround is not that big a deal once you're fully accustomed to How Things Work, but it just seems 'nicer' to allow dynamic defaults. That's all I really wanted to say in the first instance, I didn't think that position would really get anyone's back up. Regards, -Tennessee -------------- next part -------------- An HTML attachment was scrubbed... URL: From tleeuwenburg at gmail.com Mon May 11 03:09:50 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Mon, 11 May 2009 11:09:50 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <43c8685c0905100512t4307f561qcfc842bbf6bd6c4@mail.gmail.com> Message-ID: <43c8685c0905101809k705905dbq875e4fa712771153@mail.gmail.com> On Mon, May 11, 2009 at 12:59 AM, Georg Brandl wrote: > Tennessee Leeuwenburg schrieb: > > > I don't have a casual attitude towards breaking code, just an open mind > > towards discussions on their merits. I don't really appreciate the > > negative tones, and I'm sure that if anyone else is in the firing line, > > they wouldn't appreciate either, even if it to some extent it's all a > > bit of a storm in a teacup. Unless someone who is happy to cop a bit of > > flak stands up and says that's not on, then maintaining a "thick skin" > > -- i.e. putting up with people putting you down, be it through a clear > > and direct put-down, or through a more subtle implication -- becomes the > > norm. It becomes acceptable, perhaps indeed even well-regarded, to take > > a certain viewpoint then suggest that anyone who doesn't share it is > > doing something wrong. > > The problem is that people whose proposal immediately meets negative > reactions usually feel put down no matter what exactly you say to them. > If there was a polite way of saying "This will not change, please don't > waste more of our time with this discussion." that still gets the point > across, I would be very grateful. I understand that. I think there is a good way to do it. First of all, I would recognise that this is the python-ideas list, not the python-dev list, and that this is *exactly* the place to discuss ideas on their merits, and potentially put aside pragmatics to engage in a discussion of design philosophy. For bad ideas, I would suggest: "Thanks for your contribution. However, this has been discussed quite a lot before, and the groundswell of opinion is most likely that this is not going to be a good addition to Python. However, if you'd like to discuss the idea further, please consider posting it to comp.lang.py." For good/okay ideas that just won't get up, I would suggest: "Thanks for your contribution. I see your point, but I don't think it's likely to get enough traction amongst the developer community for someone else to implement it. However, if you'd like more feedback on your ideas so that you can develop a PEP or patch, please feel free to have a go. However, please don't be disappointed if it doesn't get a lot of support unless you are happy to provide some more justification for your position.". I don't really think anyone on a mailing list really needs to waste any more time than they want to -- just ignore the thread. I would definitely avoid things like: "You clearly have no idea what you are talking about" "If you only knew what I knew, you'd know differently" It's probably not possible to avoid people with an idea feeling deflated if their ideas are not popular, but on an ideas list such as this, I think that having conversations should be encouraged. Certainly that's what got under my skin. If I was chatting in person, or with friends, or in a meeting, the appropriate thing to do would be to say "Hey, that's a bit rough!" and then probably the attitude would be wound back, or the person would respond with "Oh, that's not what I meant, I just meant this..." and the misunderstanding would be quickly resolved. Unfortunately, email just *sucks* for telling the difference between someone with a chip on their shoulder, or someone who is just being helpful and made a bad choice of words. Cheers, -T -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Mon May 11 06:05:26 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 11 May 2009 00:05:26 -0400 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> Message-ID: George Sakkis wrote: > On Sun, May 10, 2009 at 1:28 AM, Scott David Daniels > wrote: >> Any argument for changing to a more "dynamic" default scheme had better >> have a definition of the behavior of the following code, and produce a >> good rationale for that behavior: >> >> x = 5 >> def function_producer(y): >> def inner(arg=x+y): >> return arg + 2 >> return inner In this version, x is resolved each time when function_producer is called which could be different each time. The x = 5 line is possible irrelevant. > I don't think the proposed scheme was ever accused of not being > well-defined. Here's the current equivalent dynamic version: > > x = 5 > def function_producer(y): > missing = object() > def inner(arg=missing): > if arg is missing: > arg = x+y > return arg + 2 > return inner In this version, x is resolved each time each of the returned inners is called, which could be different each time and which could be different from any of the times function_producer was called. Not equivalent. Given how often this issue is resuscitated, I am will consider raising my personal vote from -1 to -0. My friendly advice to anyone trying to promote a change is to avoid categorical (and false or debatable) statements like "Newbies get confused" (Some do, some do not) or "The intuitive meaning is" (to you maybe, not to me). Also avoid metaphysical issues, especially those that divide people.* Do focus on practical issues, properly qualified statements, and how a proposal would improve Python. Terry Jan Reedy * The initial argument for changing the meaning of int/int amounted to this: "integers are *merely* a subset of reals, therefore....". Those with a different philosophy dissented and the actual proposal was initially lost in the resulting fire. From tjreedy at udel.edu Mon May 11 06:56:05 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 11 May 2009 00:56:05 -0400 Subject: [Python-ideas] Add __nonzero__ to threading.Event In-Reply-To: <20090510170322.GA17770@laurie.devork> References: <20090510170322.GA17770@laurie.devork> Message-ID: Floris Bruynooghe wrote: > Hi > > I was wondering why threading.Event hasn't got the __nonzero__ special > attribute? I think it would allow for code to be more pythonic if you > could just do "if e: ..." instead of "if e.isSet(): ..." > (or "if e.is_set(): ..." in 2.6+). 2.6 and 3.1 are feature frozen. I think the proposal should be to add __bool__ to 3.2 and maybe backport to 2.7 as __nonzero__. It is possible that 3.2 will get new features not added to 2.7, but I doubt the opposite. > The patch is trivial: > > Index: Lib/threading.py > =================================================================== > --- Lib/threading.py (revision 72551) > +++ Lib/threading.py (working copy) > @@ -371,6 +371,9 @@ > > is_set = isSet > > + def __nonzero__(self): > + return self.__flag > + > def set(self): > self.__cond.acquire() > try: > > > Is it worth submitting this to the tracker or is there some reason why > this is a bad idea that I've missed? Since you have a patch that people could grab and apply, and not just an idea, I say submit too. tjr From stephen at xemacs.org Mon May 11 10:57:49 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 11 May 2009 17:57:49 +0900 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905101809k705905dbq875e4fa712771153@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <43c8685c0905100512t4307f561qcfc842bbf6bd6c4@mail.gmail.com> <43c8685c0905101809k705905dbq875e4fa712771153@mail.gmail.com> Message-ID: <87eiuwxbhu.fsf@uwakimon.sk.tsukuba.ac.jp> > On Mon, May 11, 2009 at 12:59 AM, Georg Brandl wrote: > > The problem is that people whose proposal immediately meets > > negative reactions usually feel put down no matter what exactly > > you say to them. If there was a polite way of saying "This will > > not change, please don't waste more of our time with this > > discussion." that still gets the point across, I would be very > > grateful. I thought that the way to do that is to say "This proposal is un-Pythonic".[1] Now, a claim that something is "un-Pythonic" is basically the end of discussion, since in the end it's a claim that the BDFL would reject it, not something based entirely on "objective criteria". Even the BDFL sometimes doesn't know what's Pythonic until he sees it! So a bare claim of "un-Pythonic" is borrowing the BDFL's authority, which should be done very cautiously. IMO, the "problem" arose here because Tennessee went outside of discussing the idea on its merits. Rather, he wrote "the current situation seems un-Pythonic," but didn't point to violations of any of the usual criteria. OTOH, Steven did discuss the idea on its Pythonic merits, showing that it arguably violates at least three of the tenets in the Zen of Python. In turn, Steven could have left well-enough alone and avoided explicitly pointing out that, therefore, Tennessee doesn't seem to understand what "Pythonic" is. FWIW, early in my participation in Python-Dev I was told, forcefully, that I wasn't qualified to judge Pythonicity. That statement was (and is) true, and being told off was a very helpful experience for me. How and when to say it is another matter, but avoiding it entirely doesn't serve the individual or the community IMO. Note that this whole argument does not apply to terms like "natural," "intuitive," "readable," etc., only to the pivotal term "Pythonic". The others are matters of individual opinion. AIUI, "Pythonic" is a matter of *the BDFL's* opinion (if it comes down to that, and sometimes it does). Footnotes: [1] If it's a fundamental conceptual problem. "This proposal is incompatible" can be used if it's a matter of violating the language definition according to the language reference and any relevant PEPs. From stephen at xemacs.org Mon May 11 11:29:40 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 11 May 2009 18:29:40 +0900 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905091906u5961df21nb24e7297a5f57b9d@mail.gmail.com> <91ad5bf80905092332ga29182fwc7ccfc0312bca92a@mail.gmail.com> Message-ID: <87d4agxa0r.fsf@uwakimon.sk.tsukuba.ac.jp> Terry Reedy writes: > Given how often this issue is resuscitated, I am will consider raising > my personal vote from -1 to -0. > Do focus on practical issues, properly qualified statements, and how a > proposal would improve Python. One thing I would like to see addressed is use-cases where the *calling* syntax *should* use default arguments.[1] In the case of the original example, the empty list, I think that requiring the argument, and simply writing "foo([])" instead of "foo()" is better, on two counts: EIBTI, and TOOWTDI. And it's certainly not an expensive adjustment. In a more complicated case, it seems to me that defining (and naming) a separate function would often be preferable to defining a complicated default, or explicitly calling a function in the actual argument. Ie, rather than def consume_integers(ints=fibonacci_generator()): for i in ints: # suite and termination condition why not def consume_integers(ints): for i in ints: # suite and termination condition def consume_fibonacci(): consume_integers(fibonacci_generator()) or def consume_integers_internal(ints): for i in ints: # suite and termination condition def consume_integers(): consume_integers_internal(fibonacci_generator()) depending on how frequent or intuitive the "default" Fibonacci sequence is? IMO, for both above use-cases EIBTI applies as an argument that those are preferable to a complex, dynamically- evaluated default, and for the second TOOWTDI also applies. Footnotes: [1] In the particular cases being advocated as support for dynamic evaluation of default arguments, not in general. It is clear to me that having defaults for rarely used option arguments is a good thing, and I think that is a sufficient justification for Python to support default arguments. From aahz at pythoncraft.com Mon May 11 14:29:54 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 11 May 2009 05:29:54 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905101745s50370a21j51a1a325ae887eb1@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905091719j18d0cdf1r6b70a3ffeea01ad4@mail.gmail.com> <200905101123.37344.steve@pearwood.info> <43c8685c0905101745s50370a21j51a1a325ae887eb1@mail.gmail.com> Message-ID: <20090511122954.GA7678@panix.com> On Mon, May 11, 2009, Tennessee Leeuwenburg wrote: > On Sun, May 10, 2009 at 11:23 AM, Steven D'Aprano wrote: >> >> (1) Assignments outside of the body of a function happen once, at >> compile time. Default values are outside the body of the function. You >> want a special case for default values so that they too happen at >> runtime. That's not special enough to warrant breaking the rules. > > Your logic is impeccable :) ... yet, if I may continue to push my > wheelbarrow uphill for a moment longer, I would argue that is an > implementation detail, not a piece of design philosophy. I'm not going to look it up, but in the past, Guido has essentially claimed that this behavior is by design (not so much by itself but in conjunction with other deliberate decisions). I remind you of this: "Programming language design is not a rational science. Most reasoning about it is at best rationalization of gut feelings, and at worst plain wrong." --GvR, python-ideas, 2009-3-1 -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From chambon.pascal at wanadoo.fr Mon May 11 21:49:14 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Mon, 11 May 2009 21:49:14 +0200 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <200905092132.36438.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> Message-ID: <4A08813A.4000000@wanadoo.fr> Well well well, lots of interesting points have flowed there, I won't have any chance of reacting to each one ^^ > >And no one seemed to enjoy the possibilities of getting "potentially > static variables" this way. >You did not search hard enough. Would anyone mind pointing me to people that have made a sweet use of mutable default arguments ? At the moment I've only run into "memoization" thingies, which looked rather awkward to me : either the "memo" has to be used from elsewhere in the program, and having to access it by browsing the "func_defaults" looks not at all "KISS", or it's really some private data from the function, and having it exposed in its interface is rather error-prone. > [...] > >> Does it exist ? Do we have any way, from inside a call block, to >> browse the default arguments that this code block might receive ? >> > [...] > > dir(func_object) is your friend :) > > Whoups, I meant "from inside a *code* block" :p But that's the "self function" thread you noted, I missed that one... thanks for pointing it ^^ >> On the other hand, would anyone support my alternative wish, of >> having a builtin "NotGiven", similar to "NotImplemented", and >> dedicated to this somehow usual taks of "placeholder" ? >> > > There already is such a beast: None is designed to be used as a > placeholder for Not Given, Nothing, No Result, etc. > > If None is not suitable, NotImplemented is also a perfectly good > built-in singleton object which can be used as a sentinel. It's already > used as a sentinel for a number of built-in functions and operators. > There's no reason you can't use it as well. > > Well, to me there was like a ternary semantic for arguments : *None -> make without this argument / don't use this feature *"NotGiven" -> create the parameter yourself *"Some value" -> use this value as a parameter But after reflexion, it's quite rare that the three meanings have to be used in the same place, so I guess it's ok like it is... Even though, still, I'd not be against new, more explicit builtins. "None" has too many meanings to be "self documenting", and I feel "NotImplemented" doesn't really fit where we mean "notGiven" That's the same thing for exceptions : I've seen people forced to make ugly workarounds because they got "ValueError" where they would have loved to get "EmptyIterableError" or other precise exceptions. But maybe I'm worrying on details there :p > Heh heh heh, he thinks beginners read manuals :-) > > ^^ I'm maybe the only one, but I've always found the quickest way to learn a language/library was to read the doc. Wanna learn python ? Read the language reference, then the library reference. Wanna know what's the object model of PHP5 versus PHP4 ? Read the 50 pages chapter on that matter. Wanna play with Qt ? Read the class libraries first. :p Good docs get read like novels, and it's fun to cross most of the gotchas and implementation limitations without having to be biten by them first. People might have the feeling that they gain time by jumping to the practice, I've rather the feeling that they lose hell a lot of it, that way. Back to the "mutable default arguments" thingy : I think I perfectly see the points in favor of having them as attributes of the function method, as it's the case currently. It does make sense in many ways, even if less sense than "class attributes", for sure (the object model of python is rock solid to me, whereas default arguments are on a thin border that few languages had the courage to explore - most of them forbidding non-litteral constants). But imo, it'd be worth having a simple and "consensual way" of obtaining dynamic evaluation of default arguments, without the "if arg==none:" pattern. The decorators that people have submitted have sweet uses, but they either deepcopy arguments, use dynamic evaluation of code strings, or force to lamba-wrap all arguments ; so they're not equivalent to what newbies would most of the time expect - a reevaluation of the python code they entered after '='. So the best, imo, would really be a keyword or some other form that reproduces with an easy syntax the "lambda-wrapping" we had. If adding keywords is too violent, what would you people think of some notation similar to what we already have in the "function arguments world ", i.e stars ? def func(a, c = *[]): pass Having 1, 2 or 3 stars in the "default argument" expression, wouldn't it be OK ? I guess they have no meaning there at the moment, so we could give them one : "keep that code as a lamda functio nand evaluate it at each function call". Couldn't we ? The risk would be confusion with the other "*" and "**", but in this case we might put 3 stars (yeah, that's much but...). Any comment on this ? Regards, Pascal PS : Has anyone read Dale Carneghie's books here ? That guy is a genius of social interactions, and I guess that if 1/3 of posters/mailers had read him, there would never be any flame anymore on forums and mailing lists. Contrarily to what his titles might look like, he doesn't promote hypocrisy or cowardness ; he simply points lots of attitudes (often unconscious) that ruin discussions without helping in any way the matter go forward. I'm myself used to letting scornful or aggressives sentences pass by ; but others don't like them, and I fully understand why. So could't we smoothen edges, in order to keep the discusion as it's supposed to be - a harmless sharing of pros and cons arguments, which endangers no one's life - instead of having it randomly turned into a confrontation of egos, pushing each other down as in an attempt not to drown ? http://en.wikipedia.org/wiki/How_to_Win_Friends_and_Influence_People -------------- next part -------------- An HTML attachment was scrubbed... URL: From pyideas at rebertia.com Mon May 11 22:32:21 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Mon, 11 May 2009 13:32:21 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A08813A.4000000@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> Message-ID: <50697b2c0905111332r3dbabe3ehda7dc332de4c2ac5@mail.gmail.com> On Mon, May 11, 2009 at 12:49 PM, Pascal Chambon wrote: > So the best, imo, would really be a keyword or some other form that > reproduces with an easy syntax the "lambda-wrapping" we had. > > If adding keywords is too violent, what would you people think of some > notation similar to what we already have in the "function arguments world ", > i.e stars ? > > def func(a, c = *[]): > ??? pass > > Having 1, 2 or 3 stars in the "default argument" expression, wouldn't it be > OK ? I guess they have no meaning there at the moment, so we could give them > one : "keep that code as a lamda functio nand evaluate it at each function > call". > Couldn't we ? > The risk would be confusion with the other "*" and "**", but in this case we > might put 3 stars (yeah, that's much but...). > > Any comment on this ? Seems unnecessarily confusing and sufficiently unrelated to the current use of stars in Python. -1 on this syntax. I'd look for a different punctuation/keyword. Cheers, Chris -- http://blog.rebertia.com From benjamin at python.org Mon May 11 23:51:15 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 11 May 2009 21:51:15 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Add_=5F=5Fnonzero=5F=5F_to_threading=2EE?= =?utf-8?q?vent?= References: <20090510170322.GA17770@laurie.devork> Message-ID: Floris Bruynooghe writes: > > Hi > > I was wondering why threading.Event hasn't got the __nonzero__ special > attribute? I think it would allow for code to be more pythonic if you > could just do "if e: ..." instead of "if e.isSet(): ..." > (or "if e.is_set(): ..." in 2.6+). -1 As with Antoine, I find conflating an Event's flag with its boolean value confusing. I also don't see other precedent for this in the standard library. (locks for example) From stephen at xemacs.org Tue May 12 05:36:09 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 12 May 2009 12:36:09 +0900 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <4A08813A.4000000@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> Message-ID: <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> Pascal Chambon writes: > So could't we smoothen edges, in order to keep the discusion as it's > supposed to be - a harmless sharing of pros and cons arguments, which > endangers no one's life - In discussions about Python development, misuse of the term "Pythonic" to support one's personal preference is not harmless. It leads to confusion of newbies, and ambiguity in a term that is already rather precise, and becoming more so with every PEP (though it is hard to express in a few words as a definition). The result is that the BDFL may use that term at his pleasure, but the rest of us risk being brought up short by somebody who knows better. > instead of having it randomly turned into a confrontation of egos, This was not a random event. It was triggered by, *and responded only to*, the misuse of the word "Pythonic". > pushing each other down as in an attempt not to drown ? Hey, I haven't seen one claim that "this feature would look good in Perl" yet. The gloves are still on. From tleeuwenburg at gmail.com Tue May 12 06:22:31 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Tue, 12 May 2009 14:22:31 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> On Tue, May 12, 2009 at 1:36 PM, Stephen J. Turnbull wrote: > Pascal Chambon writes: > > > So could't we smoothen edges, in order to keep the discusion as it's > > supposed to be - a harmless sharing of pros and cons arguments, which > > endangers no one's life - > > In discussions about Python development, misuse of the term "Pythonic" > to support one's personal preference is not harmless. It leads to > confusion of newbies, and ambiguity in a term that is already rather > precise, and becoming more so with every PEP (though it is hard to > express in a few words as a definition). The result is that the BDFL > may use that term at his pleasure, but the rest of us risk being > brought up short by somebody who knows better. > > > instead of having it randomly turned into a confrontation of egos, > > This was not a random event. It was triggered by, *and responded only > to*, the misuse of the word "Pythonic". I guess it's never occurred to me, and I wouldn't have thought it would be immediately clear to everyone, that Pythonic simply means "Whatever BDFL thinks". I've always thought it meant "elegant and in keeping with the design philosophy of Python", and up for discussion and interpretation by everyone. I never thought that it would be used as a means of *preventing* discussion about what was or was not 'Pythonic'. *Obviously*, BDFL's opinions on the language are authoritative, but that doesn't make them beyond discussion. This is the Python Ideas list, not the dev-list, and I was discussing my own interpretation, not trying to force anyone on anything. To recall a quote I heard once, "You are entitled to your own opinion, but not your own facts". I would have thought that expressing ones' opinion about what is or is not Pythonic is a wonderful thing to encourage. It's like encouraging people to discuss what elegant code looks like, or the merits of a piece of writing. I thought was very clear that I was talking about my interpretation of what was Pythonic, and clear that I was in no way talking about trying to claim authority. I feel a bit like I've been targetted by the thought police, truth be told, although that overstates matters. I didn't think I was in any way saying "My way is absolutely more Pythonic, you should all think like me", but much more along the lines of, "Hey, I think my solution captures something elegant and Pythonic, surely that's worth talking about even if there are some practical considerations involved". I just thought I'd be clear in saying "seems to me to be more Pythonic" rather than "is more Pythonic". Where are people going to talk freely about their interpretation of what is and isn't Pythonic, if not the ideas list? I'm also subscribed to the python-dev list, and I've never attempted to force an opinion there. Isn't *this* list the right place to have conversations about these concepts? I don't think people should be pulled short for talking about Pythonicity, just for trying to impose their world-view. That's what rubs wrongly -- being told you're not even supposed to *talk* about something, or not be entitled to an opinion on something. I would have thought that getting involved in discussing the Zen of Python is something that should be a part of everyone's learning and growth, rather than something which is delivered like a dogma. That's not to say there isn't a right answer on many issues, but it has to be acceptable to discuss the issues and to hold personal opinions. Cheers, -T -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Tue May 12 06:42:56 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 11 May 2009 21:42:56 -0700 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> Message-ID: <20090512044255.GA2822@panix.com> On Tue, May 12, 2009, Tennessee Leeuwenburg wrote: > > I thought was very clear that I was talking about my interpretation > of what was Pythonic, and clear that I was in no way talking about > trying to claim authority. I feel a bit like I've been targetted by > the thought police, truth be told, although that overstates matters. I > didn't think I was in any way saying "My way is absolutely more > Pythonic, you should all think like me", but much more along the lines > of, "Hey, I think my solution captures something elegant and Pythonic, > surely that's worth talking about even if there are some practical > considerations involved". I just thought I'd be clear in saying > "seems to me to be more Pythonic" rather than "is more Pythonic". That may have been your intent, but it sure isn't what I read in your original post. I suggest you re-read it looking for what might get interpreted as obstreperous banging on the table: http://mail.python.org/pipermail/python-ideas/2009-May/004601.html If you still don't see it, I'll discuss it with you (briefly!) off-list; that kind of tone discussion is really off-topic for this list. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It is easier to optimize correct code than to correct optimized code." --Bill Harlan From tleeuwenburg at gmail.com Tue May 12 07:06:52 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Tue, 12 May 2009 15:06:52 +1000 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <20090512044255.GA2822@panix.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <20090512044255.GA2822@panix.com> Message-ID: <43c8685c0905112206s48f2e627w297b0717e8e0e325@mail.gmail.com> On Tue, May 12, 2009 at 2:42 PM, Aahz wrote: > On Tue, May 12, 2009, Tennessee Leeuwenburg wrote: > > > > I thought was very clear that I was talking about my interpretation > > of what was Pythonic, and clear that I was in no way talking about > > trying to claim authority. I feel a bit like I've been targetted by > > the thought police, truth be told, although that overstates matters. I > > didn't think I was in any way saying "My way is absolutely more > > Pythonic, you should all think like me", but much more along the lines > > of, "Hey, I think my solution captures something elegant and Pythonic, > > surely that's worth talking about even if there are some practical > > considerations involved". I just thought I'd be clear in saying > > "seems to me to be more Pythonic" rather than "is more Pythonic". > > That may have been your intent, but it sure isn't what I read in your > original post. I suggest you re-read it looking for what might get > interpreted as obstreperous banging on the table: > > http://mail.python.org/pipermail/python-ideas/2009-May/004601.html > > If you still don't see it, I'll discuss it with you (briefly!) off-list; > that kind of tone discussion is really off-topic for this list. Agreed. Anyone else who wants to chime in, feel free to email me off-list. Regardless of the rights and wrongs, I'll of course be extra-careful in future to be crystal clear about my meaning. As far as I can tell, my original email is littered with terms like 'seems to me', 'in my opinion', 'personally' etc which I would think would convey to anyone that I'm talking about a personal opinion and not trying to discredit any one or any thing. Not that it's about counting, but I count no fewer than seven occasions where I point out that I am advancing a personal opinion rather than making a universal statement. I'm not sure what else I should have done. Cheers, -T -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Tue May 12 11:42:15 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 12 May 2009 18:42:15 +0900 Subject: [Python-ideas] Default arguments in Python - the return In-Reply-To: <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> Message-ID: <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> Tennessee Leeuwenburg writes: > Where are people going to talk freely about their interpretation of > what is and isn't Pythonic, if not the ideas list? comp.lang.python seems more appropriate for free discussion. From floris.bruynooghe at gmail.com Tue May 12 17:02:06 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Tue, 12 May 2009 16:02:06 +0100 Subject: [Python-ideas] Add __nonzero__ to threading.Event In-Reply-To: <20090510172016.GA26119@panix.com> References: <20090510170322.GA17770@laurie.devork> <20090510172016.GA26119@panix.com> Message-ID: <20090512150206.GA14820@laurie.devork> On Sun, May 10, 2009 at 10:20:16AM -0700, Aahz wrote: > On Sun, May 10, 2009, Floris Bruynooghe wrote: > > > > I was wondering why threading.Event hasn't got the __nonzero__ special > > attribute? I think it would allow for code to be more pythonic if you > > could just do "if e: ..." instead of "if e.isSet(): ..." > > (or "if e.is_set(): ..." in 2.6+). > > > > [...] > > > > Is it worth submitting this to the tracker or is there some reason why > > this is a bad idea that I've missed? > > Please submit to the tracker so that even if it *is* a bad idea it will > get recorded. http://bugs.python.org/issue5998 Regarsds Floris From grosser.meister.morti at gmx.net Tue May 12 17:56:37 2009 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Tue, 12 May 2009 17:56:37 +0200 Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A01C76A.7030809@activestate.com> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> Message-ID: <4A099C35.9060608@gmx.net> Sridhar Ratnakumar wrote: > On 09-05-06 06:55 AM, Konrad Delong wrote: >> The documentation underhttp://docs.python.org/ could have different >> colour schemes for Python 2.5, 2.6, 2.7, 3.0, 3.1 and so on. This way >> one would know on sight which docs one's reading. > > Good idea, although I'd rather have a vertical bar instead of color > schemes. For example, the vertical bar here, > > http://www.w3.org/TR/2009/PER-xpath-functions-20090421/ > > that reads "W3V Proposed Edited Recommendation" could read "2.6" in > slightly bigger font for the Python docs. +1 for sidebar, -1 for colour codes 8% to 10% of men have dyschromatopsia. encoding something in colour might be good as some sort of enhancement, but all informations have always to be clearly visibly for colourblinds, too. -panzi From debatem1 at gmail.com Tue May 12 20:53:45 2009 From: debatem1 at gmail.com (CTO) Date: Tue, 12 May 2009 11:53:45 -0700 (PDT) Subject: [Python-ideas] Coloured documentation In-Reply-To: <4A099C35.9060608@gmx.net> References: <74401640905060655j250938fga583b65799bdb1b3@mail.gmail.com> <4A01C76A.7030809@activestate.com> <4A099C35.9060608@gmx.net> Message-ID: <7b8d7bf5-fa1d-4fc9-8826-da01f236de42@s31g2000vbp.googlegroups.com> > +1 for sidebar, -1 for colour codes > > 8% to 10% of men have dyschromatopsia. encoding something in colour might be > good as some sort of enhancement, but all informations have always to be clearly > visibly for colourblinds, too. > > ? ? ? ? -panzi No harm in doing both, right? Geremy Condra From chambon.pascal at wanadoo.fr Tue May 12 21:56:28 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Tue, 12 May 2009 21:56:28 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4A09D46C.4060006@wanadoo.fr> Well, since adding new keywords or operators is very sensitive, and the existing ones are rather exhausted, it won't be handy to propose a new syntax... One last idea I might have : what about something like * def myfunc(a, b, c = yield []): pass* I'm not expert in english, but I'd say the following "equivalents" of yield (dixit WordWeb) are in a rather good semantic area : *Be the cause or source of *Give or supply *Cause to happen or be responsible for *Bring in Of course the behaviour of this yield is not so close from the one we know, but there is no interpretation conflict for the parser, and we might quickly get used to it : * yield in default argument => reevaluate the expression each time * yield in function body => return value and prepare to receive one How do you people feel about this ? Regards, Pascal PS : I've heard some month ago someone who compared the new high level languages as new religions - with the appearance of notions like "py-evangelistes" and stuffs - whereas it had (in his opinion) never been so for older languages :p That's imo somehow true (and that's rather a good sign for those languages) ; I feel phrases like "pythonic" or perl's "TIMTOWTDI" have gained some kind of sacred aura ^^ -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Wed May 13 09:34:02 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 13 May 2009 16:34:02 +0900 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A09D46C.4060006@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> Message-ID: <87k54lbgnp.fsf@uwakimon.sk.tsukuba.ac.jp> Pascal Chambon writes: > One last idea I might have : what about something like > > * def myfunc(a, b, c = yield []): > pass* As syntax, I'd be -0.5. But my real problem is that the whole concept seems to violate several of the "Zen" tenets. Besides those that Steven D'Aprano mentioned, I would add two more. First is "explicit is better than implicit" at the function call. True, for the leading cases of "[]" and "{}", "def foo(bar=[])" is an easy mistake for novice Python programmers to make with current semantics. And I agree that it is very natural to initialize an unspecified list or dictionary with the empty object. But those also have a convenient literal syntax that makes it clear that the object is constructed at function call time: "myfunc([])" and "myfunc({})". Since that syntax is always available even with the proposed new semantics, I feel this proposal also violates "there's one -- and preferably only one -- obvious way to do it". I understand the convenience argument, but that is frequently rejected on Python-Dev with "your editor can do that for you; if it doesn't, that's not a problem with Python, it's a problem with your editor." I also see the elegance and coherence of Tennessee's proposal to *always* dynamically evaluate, but I don't like it. Given that always evaluating dynamically is likely to have performance impact that is as surprising as the behavior of "def foo(bar=[])", I find it easy to reject that proposal on the grounds of "although practicality beats purity". I may be missing something, but it seems to me that the proponents of this change have yet to propose any concrete argument that it is more Pythonic than the current behavior of evaluating default expressions at compile time, and expressing dynamic evaluation by explicitly invoking the expression conditionally in the function's suite, or as an argument. Regarding the syntax, the recommended format for argument defaults is def myfunc(a, b, c=None): pass Using a keyword (instead of an operator) to denote dynamically evaluated defaults gives: def myfunc(a, b, c=yield []): pass which looks like a typo to me. I feel there should be a comma after "yield", or an index in the brackets. (Obviously, there can't be a comma because "[]" can't be a formal parameter, and yield is a keyword so it can't be the name of a sequence or mapping. So the syntax probably 'works' in terms of the parser. I'm describing my esthetic feeling, not a technical objection.) As for "yield" itself, I would likely be confused as to when the evaluation takes place. "Yield" strikes me as an imperative, "give me the value *now*", ie, at compile time. From jh at improva.dk Wed May 13 11:00:41 2009 From: jh at improva.dk (Jacob Holm) Date: Wed, 13 May 2009 11:00:41 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A09D46C.4060006@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> Message-ID: <4A0A8C39.9070003@improva.dk> Pascal Chambon wrote: > One last idea I might have : what about something like > > * def myfunc(a, b, c = yield []): > pass* > > [...], but there is no interpretation conflict for the parser, and we > might quickly get used to it I am surprised that there is no conflict, but it looks like you are technically right. The parentheses around the yield expression are required in the following (valid) code: >>> def gen(): ... def func(arg=(yield 'starting')): ... return arg ... yield func ... >>> g = gen() >>> g.next() 'starting' >>> f = g.send(42) >>> f() 42 I would hate to see the meaning of the above change depending on whether the parentheses around the yield expression were there or not, so -1 on using "yield" for this. I'm +0 on the general idea of adding a keyword for delayed evaluation of default argument expressions. - Jacob From denis.spir at free.fr Wed May 13 12:25:23 2009 From: denis.spir at free.fr (spir) Date: Wed, 13 May 2009 12:25:23 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <87k54lbgnp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <87k54lbgnp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20090513122523.41c8af65@o> Le Wed, 13 May 2009 16:34:02 +0900, "Stephen J. Turnbull" s'exprima ainsi: > But those also have a convenient > literal syntax that makes it clear that the object is constructed at > function call time: "myfunc([])" and "myfunc({})". I totaly agree with you, here. And as you say "explicit, etc..." Then, the argument must not be 'defaulted' at all in the function def. This is a semantic change and a pity when the argument has an "obvious & natural" default value. While we're at removing default args and requiring them to be explicit instead, why have default args at all in python? Especially, why should immutable defaults be OK, and mutable ones be wrong and replaced with explicit value at call time? I mean that this distinction makes no sense conceptually: the user just wants a default; this is rather python internal soup detail raising an issue. I would rather require the contrary, namely that static vars should not be messed up with default args. This is not only confusion in code, but also design flaw. here's how I see it various possibilities (with some amount of exageration ;-): def f(arg, lst=[]): # !!! lst is no default arg, !!! # !!! it's a static var instead !!! # !!! that will be updated in code !!! def f(arg): f.lst = [] # init static var or def f(arg): # init static var try: assert f.lst except AttributeError: f.lst = [] Denis ------ la vita e estrany From denis.spir at free.fr Wed May 13 12:38:09 2009 From: denis.spir at free.fr (spir) Date: Wed, 13 May 2009 12:38:09 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <87k54lbgnp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <87k54lbgnp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20090513123809.07467afa@o> Le Wed, 13 May 2009 16:34:02 +0900, "Stephen J. Turnbull" s'exprima ainsi: > I > also see the elegance and coherence of Tennessee's proposal to > *always* dynamically evaluate, but I don't like it. Given that always > evaluating dynamically is likely to have performance impact that is as > surprising as the behavior of "def foo(bar=[])", I find it easy to > reject that proposal on the grounds of "although practicality beats > purity". I do not understand why defaults should be evaluated dynamically (at runtime, I mean) in the proposal. This can happen, I guess, only for default that depend on the non-local scope: def f(arg, x=a): If we want this this a changing at runtime, then we'd better be very clear and explicit: def f(arg, x=UNDEF): # case x is not provided, use current value of a if x is UNDEF: x = a (Also, if x is not intended as a user defined argument, then there is no need for a default at all. We simply have a non-referentially transparent func.) Denis ------ la vita e estrany From stephen at xemacs.org Wed May 13 14:01:43 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 13 May 2009 21:01:43 +0900 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <20090513122523.41c8af65@o> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <87k54lbgnp.fsf@uwakimon.sk.tsukuba.ac.jp> <20090513122523.41c8af65@o> Message-ID: <87eiutb49k.fsf@uwakimon.sk.tsukuba.ac.jp> spir writes: > Especially, why should immutable defaults be OK, and mutable ones > be wrong and replaced with explicit value at call time? "Although practicality beats purity." Consider a wrapper for something like Berkeley DB which has a plethora of options for creating external objects, but which most users are not going to care about. It makes sense for those options to be optional. Otherwise people will be writing def create_db(path): """Create a DB with all default options at `path`.""" create_db_with_a_plethora_of_options(path,a,b,c,d,e,f,g,h,i,j,k,l,m) def create_db_with_nondefault_indexing(path,indexer): """Create a DB with `indexer`, otherwise default options, at `path`.""" create_db_with_a_plethora_of_options(path,indexer,b,c,d,e,f,g,h,i,j,k,l,m) etc, etc, ad nauseum. No, thank you! > I would rather require the contrary, namely that static vars should > not be messed up with default args. This is not only confusion in > code, but also design flaw. here's how I see it various > possibilities (with some amount of exageration ;-): > > def f(arg, lst=[]): > # !!! lst is no default arg, !!! > # !!! it's a static var instead !!! > # !!! that will be updated in code !!! > > > def f(arg): > > f.lst = [] # init static var Don't generators do the right thing here? def f(arg): lst = [] while True: yield None From google at mrabarnett.plus.com Wed May 13 16:28:48 2009 From: google at mrabarnett.plus.com (MRAB) Date: Wed, 13 May 2009 15:28:48 +0100 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A0A8C39.9070003@improva.dk> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> Message-ID: <4A0AD920.8060409@mrabarnett.plus.com> Jacob Holm wrote: > Pascal Chambon wrote: >> One last idea I might have : what about something like >> >> * def myfunc(a, b, c = yield []): >> pass* >> > >> [...], but there is no interpretation conflict for the parser, and we >> might quickly get used to it > > > I am surprised that there is no conflict, but it looks like you are > technically right. The parentheses around the yield expression are > required in the following (valid) code: > > > >>> def gen(): > ... def func(arg=(yield 'starting')): > ... return arg > ... yield func > ... > >>> g = gen() > >>> g.next() > 'starting' > >>> f = g.send(42) > >>> f() > 42 > > > I would hate to see the meaning of the above change depending on whether > the parentheses around the yield expression were there or not, so -1 on > using "yield" for this. > > I'm +0 on the general idea of adding a keyword for delayed evaluation of > default argument expressions. > There's the suggestion that Carl Johnson gave: def myfunc(a, b, c else []): pass or there's: def myfunc(a, b, c def []): pass where 'def' stands for 'default' (or "defaults to"). From jeremy at jeremybanks.ca Wed May 13 16:52:57 2009 From: jeremy at jeremybanks.ca (Jeremy Banks) Date: Wed, 13 May 2009 11:52:57 -0300 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A0AD920.8060409@mrabarnett.plus.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> Message-ID: <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> To someone who's a novice to this, could someone explain to me why it has to be an existing keyword at all? Since not identifiers are valid in that context anyway, why couldn't it be a new keyword that can still be used as an identifier in valid contexts? For example (not that I advocate this choice of keyword at all): def foo(bar reinitialize_default []): # <-- it's a keyword here reinitialize_default = "It's an identifier here!" That would be a syntax error now and if it were defined as a keyword only in that context it wouldn't introduce backwards compatibility problems and wouldn't force us to reuse an existing keyword in a context that may be a bit of a stretch. Is there a reason that this wouldn't be a viable approach? On 2009-05-13, MRAB wrote: > Jacob Holm wrote: >> Pascal Chambon wrote: >>> One last idea I might have : what about something like >>> >>> * def myfunc(a, b, c = yield []): >>> pass* >>> >> >>> [...], but there is no interpretation conflict for the parser, and we >>> might quickly get used to it >> >> >> I am surprised that there is no conflict, but it looks like you are >> technically right. The parentheses around the yield expression are >> required in the following (valid) code: >> >> >> >>> def gen(): >> ... def func(arg=(yield 'starting')): >> ... return arg >> ... yield func >> ... >> >>> g = gen() >> >>> g.next() >> 'starting' >> >>> f = g.send(42) >> >>> f() >> 42 >> >> >> I would hate to see the meaning of the above change depending on whether >> the parentheses around the yield expression were there or not, so -1 on >> using "yield" for this. >> >> I'm +0 on the general idea of adding a keyword for delayed evaluation of >> default argument expressions. >> > There's the suggestion that Carl Johnson gave: > > def myfunc(a, b, c else []): > pass > > or there's: > > def myfunc(a, b, c def []): > pass > > where 'def' stands for 'default' (or "defaults to"). > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From george.sakkis at gmail.com Wed May 13 17:29:44 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 13 May 2009 11:29:44 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> Message-ID: <91ad5bf80905130829o49320276jfb93d293c4a014d1@mail.gmail.com> On Wed, May 13, 2009 at 10:52 AM, Jeremy Banks wrote: > To someone who's a novice to this, could someone explain to me why it > has to be an existing keyword at all? Since not identifiers are valid > in that context anyway, why couldn't it be a new keyword that can > still be used as an identifier in valid contexts? For example (not > that I advocate this choice of keyword at all): > > ? ?def foo(bar reinitialize_default []): # <-- it's a keyword here > ? ? ? ?reinitialize_default = "It's an identifier here!" > > That would be a syntax error now and if it were defined as a keyword > only in that context it wouldn't introduce backwards compatibility > problems and wouldn't force us to reuse an existing keyword in a > context that may be a bit of a stretch. > > Is there a reason that this wouldn't be a viable approach? Traditionally, keywords are recognized at the lexer level, which then passes tokens to the parser. Lexers are pretty simple (typically constants and regular expressions) and don't take the context into account. In principle what you're saying could work, but given the significant reworking of the lexer/parser it would require, it's quite unlikely to happen, for better or for worse. George From denis.spir at free.fr Wed May 13 17:54:37 2009 From: denis.spir at free.fr (spir) Date: Wed, 13 May 2009 17:54:37 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> Message-ID: <20090513175437.7ca69845@o> Le Wed, 13 May 2009 11:52:57 -0300, Jeremy Banks s'exprima ainsi: > To someone who's a novice to this, could someone explain to me why it > has to be an existing keyword at all? Since not identifiers are valid > in that context anyway, why couldn't it be a new keyword that can > still be used as an identifier in valid contexts? For example (not > that I advocate this choice of keyword at all): > > def foo(bar reinitialize_default []): # <-- it's a keyword here > reinitialize_default = "It's an identifier here!" > > That would be a syntax error now and if it were defined as a keyword > only in that context it wouldn't introduce backwards compatibility > problems and wouldn't force us to reuse an existing keyword in a > context that may be a bit of a stretch. > > Is there a reason that this wouldn't be a viable approach? My opinion on this is you're basically right. Even 'print' (for py<3.0) could be an identifier you could use in an assignment (or in any value expression), I guess, for parse patterns are different: print_statement : "print" expression assignment : name '=' expression So you can safely have "print" as name, or inside an expression. Even "print print" should work ! But traditionnally grammars are not built as a single & total definition of the whole language (like is often done using e.g. PEG, see http://en.wikipedia.org/wiki/Parsing_Expression_Grammar) but as a 2-layer definition: one for tokens (lexicon & morphology) and one for higher-level patterns (syntax & structure). The token layer is performed by a lexer that will not take the context into account to recognize tokens, so that it could not distinguish several, syntactically & semantically different, occurrences of "print" like above. As a consequence, in most languages, key word = reserved word There may be other reasons I'm not aware of. Denis ------ la vita e estrany From Scott.Daniels at Acm.Org Wed May 13 21:30:04 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Wed, 13 May 2009 12:30:04 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <20090513175437.7ca69845@o> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> <20090513175437.7ca69845@o> Message-ID: spir wrote: > My opinion on this is you're basically right. Even 'print' (for py<3.0) could be an identifier you could use in an assignment (or in any value expression), I guess, for parse patterns are different: > print_statement : "print" expression > assignment : name '=' expression > So you can safely have "print" as name, or inside an expression. Even "print print" should work ! But you would not want print print and print(print) to have two different meanings. In Python, extra parens are fair around expressions, and print(print) is clearly a function call. --Scott David Daniels Scott.Daniels at Acm.Org From debatem1 at gmail.com Wed May 13 21:18:37 2009 From: debatem1 at gmail.com (CTO) Date: Wed, 13 May 2009 12:18:37 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A09D46C.4060006@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> Message-ID: <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> On May 12, 3:56?pm, Pascal Chambon wrote: > Well, since adding new keywords or operators is very sensitive, and the > existing ones are rather exhausted, it won't be handy to propose a new > syntax... > > One last idea I might have : what about something like > > * ? ?def myfunc(a, b, c = yield []): > ? ? ? ? ? ? ? pass* > > I'm not expert in english, but I'd say the following "equivalents" of > yield (dixit WordWeb) are in a rather good semantic area : > *Be the cause or source of > *Give or supply > *Cause to happen or be responsible for > *Bring in > > Of course the behaviour of this yield is not so close from the one we > know, but there is no interpretation conflict for the parser, and we > might quickly get used to it : > * yield in default argument => reevaluate the expression each time > * yield in function body => return value and prepare to receive one > > How do you people feel about this ? > Regards, > Pascal I'm not a fan. If you thought not reevaluating function expressions was confusing for newbies, wait until you see what making up a new kind of yield will do for them. Why not just push for some decorators that do this to be included in stdlib? I see the utility, but not the point of adding extra syntax. >>> @Runtime ... def f(x=a**2+2b+c): ... return x ... >>> a = 1 >>> b = 2 >>> c = 3 >>> f() 8 This seems much more intuitive and useful to me than adding new meanings to yield. Geremy Condra From arnodel at googlemail.com Wed May 13 21:44:14 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 13 May 2009 20:44:14 +0100 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> Message-ID: <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> On 13 May 2009, at 20:18, CTO wrote: > Why not just push for some decorators that do this to be included in > stdlib? I see the utility, but not the point of adding extra syntax. > >>>> @Runtime > ... def f(x=a**2+2b+c): > ... return x > ... >>>> a = 1 >>>> b = 2 >>>> c = 3 >>>> f() > 8 > > This seems much more intuitive and useful to me than adding new > meanings to yield. This is not possible. def f(x=a**2+2*b+c): return x is compiled to something very much like: _tmp = x**2+2*b+c def f(x=_tmp): return x So it is impossible to find out what expression yields the default value of x by just looking at f. You have to use lambda or use George Sakkis' idea of using strings for defaults and evaluating them at call- time (but I'm not sure this will work reliably with nested functions). -- Arnaud From chambon.pascal at wanadoo.fr Wed May 13 23:09:27 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Wed, 13 May 2009 23:09:27 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> Message-ID: <4A0B3707.1010104@wanadoo.fr> Greg Falcon a ?crit : > Notice that > > def foo(): > def myfunc(a, b, c = (yield [])): > pass > > already has a meaning in Python. Under your proposal, > > Aowh... I didn't even think this was legal, but indeed this works... so it's obviously dead for "yield". > Anyhow, since you've decided you want add a new kind of default > argument spelled differently than the standard one, I have to ask what > this buys you beyond > > def foo(): > def myfunc(a, b, c = None): > if c is not None: # or a setinel > c = [] > > Sure, it takes two more lines of code to write this (three if you want > to use an object() instance as your sentinel to allow None being > passed in), but does not require a new bit of language syntax for > every Python programmer to learn... > > Greg F > > Well, not only do we have to write more code, but we lose in self-documentation I guess (I'd rather have the default appearing in the signature than in the function code - pydoc and stuffs won't notice it), and I find slightly disappointing the principle of a "sentinel", i.e "I would have wanted to do something there but I can't so I'll do it farther, take that in the meantime". > There's the suggestion that Carl Johnson gave: > > def myfunc(a, b, c else []): > pass > > or there's: > > def myfunc(a, b, c def []): > pass > > where 'def' stands for 'default' (or "defaults to"). Those phrases could do it, I'm just worryied about the fact that semantically, they make (to me) no difference with "c = []". None of those ways looks more "dynamic" than teh other, so it might be hard to explain why "=" means compiel time, and "def" means runtime. I must admit I'm totally out of ideas then. Regards, Pascal From tjreedy at udel.edu Wed May 13 23:39:37 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 13 May 2009 17:39:37 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> Message-ID: Jeremy Banks wrote: > To someone who's a novice to this, could someone explain to me why it > has to be an existing keyword at all? Since not identifiers are valid > in that context anyway, why couldn't it be a new keyword that can > still be used as an identifier in valid contexts? For example (not > that I advocate this choice of keyword at all): > > def foo(bar reinitialize_default []): # <-- it's a keyword here > reinitialize_default = "It's an identifier here!" > > That would be a syntax error now and if it were defined as a keyword > only in that context it wouldn't introduce backwards compatibility > problems and wouldn't force us to reuse an existing keyword in a > context that may be a bit of a stretch. > > Is there a reason that this wouldn't be a viable approach? At one time, 'as' was only a keyword in the context of import. So it is 'viable'. But it was a bit confusing for programmers and messy implementation-wise and I think the developers were glad to promote 'as' to a full keyword and would be reluctant to go down that road again. From tjreedy at udel.edu Wed May 13 23:58:46 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 13 May 2009 17:58:46 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A0AD920.8060409@mrabarnett.plus.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> Message-ID: MRAB wrote: > There's the suggestion that Carl Johnson gave: > > def myfunc(a, b, c else []): > pass > > or there's: > > def myfunc(a, b, c def []): > pass > > where 'def' stands for 'default' (or "defaults to"). I had the idea of def f(c=:[]): where ':' is intended to invoke the idea of lambda, since the purpose is to turn the expression into a function that is automatically called (which is why lambda alone is not enough). So I would prefer c = def [] where def reads 'auto function defined by...'. or c = lambda::[] where the extra ':' indicates that that the function is auto-called or c = lambda():[], (now illegal), where () is intended to show that the default arg is the result of calling the function defined by the expression. lambda:[]() (now legal) would mean to (uselessly) call the function immediately. Thinking about it, I think those who want a syntax to indicate that the expression should be compiled into a function and called at runtime should build on the existing syntax (lambda...) for indicating that an expression should be compiled into a function, rather than inventing a replacement for that. Terry Jan Reedy From bruce at leapyear.org Thu May 14 00:04:05 2009 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 13 May 2009 15:04:05 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A0B3707.1010104@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> <4A0B3707.1010104@wanadoo.fr> Message-ID: Here's what I'd like: def myfunc(a, b, c = *lambda: expression): stuff The use of the lambda keyword here makes the scope of any variables in the expression clear. The use of the prefix * makes the syntax invalid today, suggests dereferencing and doesn't hide the overhead. This is equivalent to: __unset = object() __default = lambda: expression def mfunc(a, b, c = __unset): if c == __unset: c = __default() stuff --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Thu May 14 00:01:32 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 14 May 2009 08:01:32 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> References: <4A0496BD.3030709@wanadoo.fr> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> Message-ID: <200905140801.33320.steve@pearwood.info> On Thu, 14 May 2009 05:18:37 am CTO wrote: > If you thought not reevaluating function expressions > was confusing for newbies, wait until you see what making up a new > kind of yield will do for them. > > Why not just push for some decorators that do this to be included in > stdlib? I see the utility, but not the point of adding extra syntax. Even if a decorator solution can be made to work, it seems to me that the difficulty with a decorator solution is that it is all-or-nothing -- you can decorate the entire parameter list, or none of the parameters, but not some of the parameters. You can bet that people will say they want delayed evaluation of some default arguments and compile-time evaluation of others, in the same function definition. There are work-arounds, of course, but there are perfectly adequate work-arounds for the lack of delayed evaluation defaults now, and it hasn't stopped the complaints. I'm going to suggest that any syntax should be applied to the formal parameter name, not the default value. This feels right to me -- we're saying that it's the formal parameter that is "special" for using delayed semantics, not that the default object assigned to it is special. Hence it should be the formal parameter that is tagged, not the default value. By analogy with the use of the unary-* operator, I suggest we use a new unary-operator to indicate the new semantics. Inside the parameter list, &x means to delay evaluation of the default argument to x to runtime: def parrot(a, b, x=[], &y=[], *args, **kwargs): As a bonus, this will allow for a whole new series of bike-shedding arguments about which specific operator should be used. *grin* Tagging a parameter with unary-& but failing to specify a default value should be a syntax error: def parrot(&x, &y=[]): Likewise for unary-& outside of a parameter list. Bike-shedding away... *wink* -- Steven D'Aprano From google at mrabarnett.plus.com Thu May 14 00:20:49 2009 From: google at mrabarnett.plus.com (MRAB) Date: Wed, 13 May 2009 23:20:49 +0100 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905140801.33320.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> <200905140801.33320.steve@pearwood.info> Message-ID: <4A0B47C1.9050705@mrabarnett.plus.com> Steven D'Aprano wrote: > On Thu, 14 May 2009 05:18:37 am CTO wrote: > >> If you thought not reevaluating function expressions >> was confusing for newbies, wait until you see what making up a new >> kind of yield will do for them. >> >> Why not just push for some decorators that do this to be included in >> stdlib? I see the utility, but not the point of adding extra syntax. > > Even if a decorator solution can be made to work, it seems to me that > the difficulty with a decorator solution is that it is > all-or-nothing -- you can decorate the entire parameter list, or none > of the parameters, but not some of the parameters. You can bet that > people will say they want delayed evaluation of some default arguments > and compile-time evaluation of others, in the same function definition. > > There are work-arounds, of course, but there are perfectly adequate > work-arounds for the lack of delayed evaluation defaults now, and it > hasn't stopped the complaints. > > I'm going to suggest that any syntax should be applied to the formal > parameter name, not the default value. This feels right to me -- we're > saying that it's the formal parameter that is "special" for using > delayed semantics, not that the default object assigned to it is > special. Hence it should be the formal parameter that is tagged, not > the default value. > > By analogy with the use of the unary-* operator, I suggest we use a new > unary-operator to indicate the new semantics. Inside the parameter > list, &x means to delay evaluation of the default argument to x to > runtime: > > def parrot(a, b, x=[], &y=[], *args, **kwargs): > > As a bonus, this will allow for a whole new series of bike-shedding > arguments about which specific operator should be used. *grin* > > Tagging a parameter with unary-& but failing to specify a default value > should be a syntax error: > > def parrot(&x, &y=[]): > > Likewise for unary-& outside of a parameter list. > > Bike-shedding away... *wink* > Well, going back to 'def', it could mean 'deferred until call-time': def parrot(a, b, x=[], y=def [], *args, **kwargs): From g.brandl at gmx.net Thu May 14 00:20:46 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 14 May 2009 00:20:46 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> <4A0B3707.1010104@wanadoo.fr> Message-ID: Bruce Leban schrieb: > Here's what I'd like: > > def myfunc(a, b, c = *lambda: expression): > stuff > > The use of the lambda keyword here makes the scope of any variables in > the expression clear. The use of the prefix * makes the syntax invalid > today, suggests dereferencing and doesn't hide the overhead. Why not @calldefaults def myfunc(a, b, c = lambda: expression): pass which should be possible without introducing new syntax. Georg From bruce at leapyear.org Thu May 14 00:33:48 2009 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 13 May 2009 15:33:48 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... Message-ID: Sure you can do it with a decorator. Although you might want one default to have this behavior and one not to. And putting the * right next to the lambda makes this a bit more explicit to my eye. --- Bruce Georg Brandl wrote: >Bruce Leban schrieb: >> Here's what I'd like: >> >> def myfunc(a, b, c = *lambda: expression): >> stuff >> >> The use of the lambda keyword here makes the scope of any variables in >> the expression clear. The use of the prefix * makes the syntax invalid >> today, suggests dereferencing and doesn't hide the overhead. > >Why not > >@calldefaults >def myfunc(a, b, c = lambda: expression): > pass > >which should be possible without introducing new syntax. > >Georg > >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >http://mail.python.org/mailman/listinfo/python-ideas From tjreedy at udel.edu Thu May 14 01:02:08 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 13 May 2009 19:02:08 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905140801.33320.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> <200905140801.33320.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > On Thu, 14 May 2009 05:18:37 am CTO wrote: > >> If you thought not reevaluating function expressions >> was confusing for newbies, wait until you see what making up a new >> kind of yield will do for them. >> >> Why not just push for some decorators that do this to be included in >> stdlib? I see the utility, but not the point of adding extra syntax. > > Even if a decorator solution can be made to work, it seems to me that > the difficulty with a decorator solution is that it is > all-or-nothing -- you can decorate the entire parameter list, or none > of the parameters, but not some of the parameters. You can bet that > people will say they want delayed evaluation of some default arguments > and compile-time evaluation of others, in the same function definition. Not all or nothing, and selection is easy. A decorator could only call callable objects, and could/should be limited to calling function objects or even function objects named ''. And if one wanted the resulting value to such a function, escape the default lambda expression with lambda. x=[1,2] @call_lambdas def f(a=len(x), lst = lambda:[], func = lambda: lambda x: 2*x): # a is int 2, lst is a fresh list, func is a one-parameter function Terry Jan Reedy From debatem1 at gmail.com Thu May 14 01:03:34 2009 From: debatem1 at gmail.com (CTO) Date: Wed, 13 May 2009 16:03:34 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> Message-ID: On May 13, 3:44?pm, Arnaud Delobelle wrote: > On 13 May 2009, at 20:18, CTO wrote: > > > Why not just push for some decorators that do this to be included in > > stdlib? I see the utility, but not the point of adding extra syntax. > > >>>> @Runtime > > ... def f(x=a**2+2b+c): > > ... ? ?return x > > ... > >>>> a = 1 > >>>> b = 2 > >>>> c = 3 > >>>> f() > > 8 > > > This seems much more intuitive and useful to me than adding new > > meanings to yield. > > This is not possible. > > ? ? ?def f(x=a**2+2*b+c): > ? ? ? ? return x > > is compiled to something very much like: > > ? ? ?_tmp = x**2+2*b+c > ? ? ?def f(x=_tmp): > ? ? ? ? return x > > So it is impossible to find out what expression yields the default ? > value of x by just looking at f. ?You have to use lambda or use George ? > Sakkis' idea of using strings for defaults and evaluating them at call- > time (but I'm not sure this will work reliably with nested functions). > > -- > Arnaud Thanks for the input, but I've already written the code to do this. It is available at . For those with hyperlink allergies, the snippet posted above reevaluates the function whenever it is called, and can be used like so: >>> from runtime import runtime >>> @runtime ... def example1(x, y=[]): ... y.append(x) ... return y ... >>> example1(1) [1] >>> example1(2) [2] or, as posted above, >>> a, b, c = 0, 1, 2 >>> @runtime ... def example2(x=a**2+2*b+c): ... return x ... >>> example2() 4 >>> a = 5 >>> example2() 29 The gode given is slow and ugly, but it does appear- at least to me- to do what is being asked here. Geremy Condra From debatem1 at gmail.com Thu May 14 01:08:02 2009 From: debatem1 at gmail.com (CTO) Date: Wed, 13 May 2009 16:08:02 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> Message-ID: A caveat on my previously posted code: as mentioned in another thread earlier today, it will not work on functions entered into the interpreter. Geremy Condra From tjreedy at udel.edu Thu May 14 01:08:50 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 13 May 2009 19:08:50 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> <4A0B3707.1010104@wanadoo.fr> Message-ID: Bruce Leban wrote: > Here's what I'd like: > > def myfunc(a, b, c = *lambda: expression): > stuff > > The use of the lambda keyword here makes the scope of any variables in > the expression clear. The use of the prefix * makes the syntax invalid > today, suggests dereferencing and doesn't hide the overhead. This is > equivalent to: There is a proposal, which I thought was accepted in principle, to make '* seq' valid generally, not just in arg-lists. to mean 'unpack the sequence'. * (lambda:1,2)() would then be valid, and without the call, would be a runtime, not syntax error. Other than that ;0(, it would be an interesting idea. > __unset = object() > __default = lambda: expression > def mfunc(a, b, c = __unset): > if c == __unset: > c = __default() > stuff tjr From george.sakkis at gmail.com Thu May 14 02:10:42 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 13 May 2009 20:10:42 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> <4A0B3707.1010104@wanadoo.fr> Message-ID: <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> On Wed, May 13, 2009 at 7:08 PM, Terry Reedy wrote: > Bruce Leban wrote: >> >> Here's what I'd like: >> >> def myfunc(a, b, c = *lambda: expression): >> ?stuff >> >> The use of the lambda keyword here makes the scope of any variables in the >> expression clear. The use of the prefix * makes the syntax invalid today, >> suggests dereferencing and doesn't hide the overhead. This is equivalent to: > > There is a proposal, which I thought was accepted in principle, to make '* > seq' valid generally, not just in arg-lists. to mean 'unpack the sequence'. > * (lambda:1,2)() would then be valid, and without the call, would be a > runtime, not syntax error. > > Other than that ;0(, it would be an interesting idea. Then how about putting the * before the parameter ? def myfunc(a, b, *c = lambda: expression): It's currently a syntax error, although the fact that "*arg" and "*arg=default" would mean something completely different is problematic. Still the same idea can be applied for some other operator (either valid already or not). Regardless of the actual operator, I came up with the following additional subproposals. Subproposal (1): Get rid of the explicit lambda for dynamic arguments. That is, def myfunc(a, b, *x=[]): would be equivalent to what previous proposals would write as def myfunc(a, b, *x=lambda: []): Subproposal (2): If subproposal (1) is accepted, we could get for free (in terms of syntax at least) dynamic args depending on previous ones. That is, def myfunc(a, b, *m=(a+b)/2): would mean def myfunc(a, b, *m = lambda a,b: (a+b)/2): with the lambda being passed the values of a and b at runtime. Thoughts ? George From bruce at leapyear.org Thu May 14 04:41:17 2009 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 13 May 2009 19:41:17 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> <4A0B3707.1010104@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> Message-ID: On Wed, May 13, 2009 at 5:10 PM, George Sakkis wrote: > On Wed, May 13, 2009 at 7:08 PM, Terry Reedy wrote: > > > Bruce Leban wrote: > >> def myfunc(a, b, c = *lambda: expression): > >> stuff > > > There is a proposal, which I thought was accepted in principle, to make > '* > > seq' valid generally, not just in arg-lists. to mean 'unpack the > sequence'. > > * (lambda:1,2)() would then be valid, and without the call, would be a > > runtime, not syntax error. > > Then how about putting the * before the parameter ? > > def myfunc(a, b, *c = lambda: expression): > > It's currently a syntax error, although the fact that "*arg" and > "*arg=default" would mean something completely different is > problematic. Still the same idea can be applied for some other > operator (either valid already or not). > I think similar syntax should do similar things. If *arg means one thing and &arg means something else, that's confusing. There are lots of non confusing alternatives: def foo(a, b := lambda: bar): def foo(a, b = & lambda: bar): def foo(a, @dynamic b = lambda: bar): # adding decorators on parameters and more Subproposal (1): Get rid of the explicit lambda for dynamic arguments. That > is, > > def myfunc(a, b, *x=[]): > > would be equivalent to what previous proposals would write as > > def myfunc(a, b, *x=lambda: []): > Explicit is better than implicit. There's a thunk getting created here, right? Don't you want that to be obvious? I do. Subproposal (2): If subproposal (1) is accepted, we could get for free > (in terms of syntax at least) dynamic args depending on previous ones. > That is, > > def myfunc(a, b, *m=(a+b)/2): > > would mean > > def myfunc(a, b, *m = lambda a,b: (a+b)/2): > > with the lambda being passed the values of a and b at runtime. It's not free and that adds quite a bit of complexity. Note that default parameters are evaluated in the context of where the function is defined, NOT in the middle of setting the other function parameters. (That's true for this new case too.) Sure Lisp has let and let* but the proposal here is NOT to provide arbitrary computational ability in the parameter list but to provide a way to have defaults that are not static. We shouldn't over-engineer things just because we can. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From george.sakkis at gmail.com Thu May 14 06:51:33 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Thu, 14 May 2009 00:51:33 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> <4A0B3707.1010104@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> Message-ID: <91ad5bf80905132151g4da52746le8e004b10ece922f@mail.gmail.com> On Wed, May 13, 2009 at 10:41 PM, Bruce Leban wrote: > > On Wed, May 13, 2009 at 5:10 PM, George Sakkis > wrote: >> >> On Wed, May 13, 2009 at 7:08 PM, Terry Reedy wrote: >> >> > Bruce Leban wrote: >> >> def myfunc(a, b, c = *lambda: expression): >> >> ?stuff >> >> > There is a proposal, which I thought was accepted in principle, to make >> > '* >> > seq' valid generally, not just in arg-lists. to mean 'unpack the >> > sequence'. >> > * (lambda:1,2)() would then be valid, and without the call, would be a >> > runtime, not syntax error. >> >> Then how about putting the * before the parameter ? >> >> ? ?def myfunc(a, b, *c = lambda: expression): >> >> It's currently a syntax error, although the fact that "*arg" and >> "*arg=default" would mean something completely different is >> problematic. Still the same idea can be applied for some other >> operator (either valid already or not). > > > I think similar syntax should do similar things. If *arg means one thing and > &arg means something else, that's confusing. Confusing?? It's certainly no more confusing than using *args for one thing and **args for something else. >> Subproposal (1): Get rid of the explicit lambda for dynamic arguments. >> That is, >> >> ? ?def myfunc(a, b, *x=[]): >> >> would be equivalent to what previous proposals would write as >> >> ? ?def myfunc(a, b, *x=lambda: []): > > Explicit is better than implicit. There's a thunk getting created here, > right? Don't you want that to be obvious? I do. Explicitness is in the eye of the beholder, and also an acquired taste. @decorator is less explicit than f = decorator(f) and yet it's generally considered a successful addition these days, despite the strong opposition it met initially. >> Subproposal (2): If subproposal (1) is accepted, we could get for free >> (in terms of syntax at least) dynamic args depending on previous ones. >> That is, >> >> ? ?def myfunc(a, b, *m=(a+b)/2): >> >> would mean >> >> ? ?def myfunc(a, b, *m = lambda a,b: (a+b)/2): >> >> with the lambda being passed the values of a and b at runtime. > > It's not free and that adds quite a bit of complexity. Note that default > parameters are evaluated in the context of where the function is defined, > NOT in the middle of setting the other function parameters. (That's true for > this new case too.) Sure, but I'm not sure what's your point here. The compiler can generate bytecode to the effect of: def myfunc(a, b, *m = (a+b)/2): if m is not passed: m = default_m(a,b) # actual body follows Ideally an optimizer would further check whether it's safe to inline the expression (which should be the typical case) to avoid the function call overhead. > Sure Lisp has let and let* but the proposal here is NOT to provide arbitrary > computational ability in the parameter list but to provide a way to have > defaults that are not static. We shouldn't over-engineer things just because > we can. Agreed, the primary target is to fix the common gotcha of mutable defaults, and I'd rather see this handled than nothing at all. A secondary goal that can be achieved here though is to reduce the overusage of None (or other sentinels for that matter). There are several articles to the effect of "Null considered harmful"; also SQL as well as some statically typed languages provide both nullable and non-nullable types and assume the latter by default. The only useful operation to a sentinel is identity check, so you know that every `x = sentinel` assignment should be followed by one or more `if x is sentinel` checks. Fewer nulls means potentially less conditional logic mental overhead. I'm not claiming we should get rid of None of course; there are legitimate reasons for different behavior under different conditions. Here however we're talking about a very specific pattern: def f(a, b=sentinel): if b is sentinel: b = It may not seem such a big deal, but then again I don't think it's less trivial than the case for the (eventually) accepted ternary operator vs an if/else statement. George From denis.spir at free.fr Thu May 14 08:50:42 2009 From: denis.spir at free.fr (spir) Date: Thu, 14 May 2009 08:50:42 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <4A056154.2090103@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> <20090513175437.7ca69845@o> Message-ID: <20090514085042.119efbac@o> Le Wed, 13 May 2009 12:30:04 -0700, Scott David Daniels s'exprima ainsi: > spir wrote: > > My opinion on this is you're basically right. Even 'print' (for py<3.0) > > could be an identifier you could use in an assignment (or in any value > > expression), I guess, for parse patterns are different: print_statement : > > "print" expression assignment : name '=' expression So you can > > safely have "print" as name, or inside an expression. Even "print print" > > should work ! > > But you would not want > print print > and > print(print) > to have two different meanings. > In Python, extra parens are fair around expressions, > and print(print) is clearly a function call. > You're right ;-) Denis ------ la vita e estrany From arnodel at googlemail.com Thu May 14 10:08:13 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Thu, 14 May 2009 09:08:13 +0100 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> Message-ID: <9bfc700a0905140108ma728df8gdab89c91845e2dfe@mail.gmail.com> 2009/5/14 CTO : > Thanks for the input, but I've already written the code to do this. > It > is available at . I should have said "it's impossible short of looking at the source code or doing some very sophisticated introspection of the bytecode of the module the function is defined in". Even so, your recipe doesn't quite work in several cases, aside from when the source code is not accessible. Two examples: def nesting(): default = 3 @runtime def example3(x=default): return x example3() nesting() @runtime def name(x=a): return x name() * The first one fails because default is not a global variable, thus not accessible from within the runtime decorator. I don't know how if this can be fixed. Note that for the function to exec() at all, you need to e.g. modify remove_decorators so that it also removes initial whitespace, something like: def remove_decorators(source): """Removes the decorators from the given function""" lines = source.splitlines() lines = [line for line in lines if not line.startswith('@')] indent = 0 while lines[0][indent] == ' ': indent += 1 new_source = '\n'.join(line[indent:] for line in lines) return new_source * The second one fails because of a clash of names. I guess that can be fixed by specifying what the locals and globals are explicitely in the calls to exec and eval. -- Arnaud From debatem1 at gmail.com Thu May 14 11:32:14 2009 From: debatem1 at gmail.com (CTO) Date: Thu, 14 May 2009 02:32:14 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <9bfc700a0905140108ma728df8gdab89c91845e2dfe@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905092132.36438.steve@pearwood.info> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <24bd2ca9-25f8-4aba-9427-d806b73df088@z7g2000vbh.googlegroups.com> <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> <9bfc700a0905140108ma728df8gdab89c91845e2dfe@mail.gmail.com> Message-ID: <360cc000-f305-4a02-b2dc-5c035abe770c@n4g2000vba.googlegroups.com> > I should have said "it's impossible short of looking at the source > code or doing some very sophisticated introspection of the bytecode of > the module the function is defined in". Any which way you slice this it will require that literal code *not* be interpreted until execution time. There are other ways to do that- storing it in strings, as George Sakkis does, modifying the language itself, as is the proposal here, or reading and parsing the original source. But you're right- more info is needed than what the bytecode contains. > Even so, your recipe doesn't quite work in several cases, aside from > when the source code is not accessible. Obviously, you are quite correct. Scoping in particular is difficult both to understand and to properly handle- had me chasing my tail for about twenty minutes earlier, actually- and I'm sure this is a security nightmare, but it does (generally) what is being asked for here. And it does so without recourse to changing the syntax. Here's another possible mechanism: def runtime(f): """Evaluates a function's annotations at runtime.""" annotations = getfullargspec(f)[-1] @wraps(f) def wrapped(*args, **kwargs): defaults = {k: eval(v) for k, v in annotations.items()} defaults.update(kwargs) return f(*args, **defaults) return wrapped @runtime def example1(x, y:'[]'): y.append(x) return y @runtime def example2(x:'a**2+2*b+c'): return x Pretty simple, although it messes with the call syntax pretty badly, effectively treating a non-keyword argument as a keyword-only argument. There's probably a way around that but I doubt I'm going to see it tonight. The point is, I don't really see the point in adding a new syntax. There are *lots* of incomplete solutions floating around to this issue, and it will probably take a lot less work to make one of those into a complete solution than it will to add a new syntax, if that makes any sense at all. Also, do you mind posting any problems you find in that to the activestate message board so there is a record there? Geremy Condra From ziade.tarek at gmail.com Thu May 14 12:10:41 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 14 May 2009 12:10:41 +0200 Subject: [Python-ideas] Add a "sitedirs" list in the site module Message-ID: <94bdd2610905140310s6fadd95bv58bfbae6ec5f4f8d@mail.gmail.com> Hello Right now there's no way to differenciate in sys.path the directories that where added through site.addsitedirs calls, (like when python starts up and initializes site-packages directories) Anyone can change sys.path, and add elements in them. So there's no way to answer to this simple question for sure : "what are the site-packages directories ?" What about keeping track of all directories added through site.addsitedirs calls, with a simple list called "sitedirs" This list would also override some container methods, so it doesn't keep paths that are not present anymore in sys.path, somehow Tarek -- Tarek Ziad? | http://ziade.org From denis.spir at free.fr Thu May 14 13:23:39 2009 From: denis.spir at free.fr (spir) Date: Thu, 14 May 2009 13:23:39 +0200 Subject: [Python-ideas] func attribute & default arg Message-ID: <20090514132339.32010d65@o> Hello, I think there is some confusion on the topic of static variables in relation to default arguments and function attributes. Or, maybe, my point of view is different from the one of the majority on python-ideas. (a rather long post) To sum up, I think the core of the issue is not about static/dymamic evaluation, or compile time/run time; rather it lies in the fact that python internally does not separate a default *value* from the corresponding local *variable*. What may be, for the programmer, the meaning and purpose of a default argument? I think there is nothing misleading in the fact that it is evaluated at definition time. Instead, this is precisely the way we see them, otherwise there would be no gotcha and we would not discuss. This semantics is properly denoted, I guess, by a default beeing part of the function definition's headline: def f(arg, default=value): ....... Once set, a default becomes a kind of constant attribute of the function. So that the gotcha happens precisely when this expectation is deceived because the local variable called 'default' may change back the previously defined value. To address this issue, having a default value recomputed on every call is not only overkill, but also semantically aside the point. What we need is only *caching* a default value while the function def is first evaluated. A possible implementation (probably requiring few changes) may be to add a kind of __defaults__ dict attribute to functions. This would be filled at definition time and used at call time. Even if probably not needed, for python provides better ways, having it as an available attribute also lets programmers tweak a func's guts if they like it -- e.g. for runtime updatable defaults: func.__defaults__["default_arg"] = newDefaultValue() With this semantics, we are free to use and update local variables safely, without worrying of changing the default value. Here is a short and probably non-exhaustive review of common uses of defaults. -1- real default arg which value is immutable No issue. -2- real default arg which value is mutable but not updated in func body No issue. With the above proposal, we do not need to worry anyway; and we can change the code later safely to case -3-. I find this important. -3- real default arg which value is mutable and updated in func body Present implementation: gotcha. Proposal: default argument and local var point to distinct values. No more issue. Possibly use a __defaults__ dict attribute. -4- default used as static var A common use of mutable defaults is a workaround for python seemingly not having static variables.?Actually, python programmers have them in the form of function attributes. Since this feature seems +/- unknown (undocumented?), some programmers write for instance: def f(arg, l=[]): while the following better expresses the semantics, imho, because l is not a real argument: def f(arg): f.l = [] Generators can also be used in cases where the purpose of the func is (mainly) to return successive values. def f(arg): l = [] while True: yield something -5- func factory In the case of a factory, default arguments are used as function definition *parameters*. Similarly to the previous case, they are rather function attributes, with the difference that this time they are constant ones. Anyway, the proposal does not change the semantics here because each generated function has its own parameter. Still, a common idiom using a dummy default, such as: def paramPower(exponent): def power(number, exponent=exponent): return number ** exponent return power would be better expressed, without any trick: def paramPower(exponent): def power(number): return number ** power.exponent power.exponent = exponent return power -6- variable defaults If we want a runtime evaluation of a default argument, we usually express it in the function body and use a sentinel. This would not change, and anyway it seems to make the intent clearer than in the case where the default would be silently recomputed (read: "explicit better than implicit"): from random import random # provides runtime default number_count = 3 # may also be variable UNDEF = object() def f(i, numbers=UNDEF): # case numbers not provided: must be evaluated at runtime if numbers is UNDEF: numbers = [int(random()*10) for i in range(number_count)] # change pointed number numbers[i] = int(random()*10) print numbers numbers = [1,2,3] f(0, numbers) ; f(2, numbers) ; f(0) ; f(2) ==> [8, 2, 3] [8, 2, 2] [5, 3, 5] [8, 4, 1] Using the proposal of a default attribute, an alternative would be to re-compute the value in calling code -- only when needed: def f(i, numbers=[random() for i in range(number_count)]): # change pointed number numbers[i] = random() print numbers ....... if cond: f.__defaults__['numbers'] = [random() for i in range(number_count)] f(i) A transition phase could be planned toward cached defaults, possibly with proper warnings: "Warning: use of a mutable default value updated in the function body. From python 3.x, default values will be cached so as not to change between calls. If your intention is instead to have a kind of static variable, rather set an attribute on the function." Now, why are func attributes rarely used; while, as shown above, they provide a clear and handy way to express several semantic patterns -- and defaults are (mis)used instead? I wonder whether we could make them more obvious using a dedicated syntax. In the thread about default arguments, a proposed syntax, to have defaults dynamically reevaluated at each call (which I find *wrong*), was to reuse the keyword "yield". The meaning of this idiom is rather for me the creation of some data at definition time. As a consequence, I found it a rather good possibility to denote the creation of a func attribute. : def f(arglist) yield attr=val: for more attributes, reuse import-like syntax: def f(arglist) yield attr1=val1, attr2=val2...: Having them defined outside the parens properly differenciate arguments from attributes (and the content of parens is already complicated enough with defaults and '*' and '**' thingies). Also, the "yield keyword not only suggests the creation of data, but also is used in contexts (generators) where state is kept between function calls. For instance, the following code using a func with a custom attribute: startup_numbers = [4, 5] def f(number, sorting=False): f.numbers.append(number) if sorting: f.numbers.sort() print f.numbers f.numbers = startup_numbers could be expressed with a func def header as follows: def f(number, sorting=False) yield numbers=startup_numbers: without the need to separately set the attribute on the func. Sure, we do not need that at all. Moreover, when the func happens to be a method instance, then the attribute better fits on the instance object. I'm just looking for a way to let programmers be (more) aware of func attributes. To have cached default solving the common gotcha, we need python programmers to use defaults only as defaults, and use other solutions for semantics such as static vars. So that the transition be easier. Denis ------ la vita e estrany From denis.spir at free.fr Thu May 14 13:44:07 2009 From: denis.spir at free.fr (spir) Date: Thu, 14 May 2009 13:44:07 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A08813A.4000000@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <3cdcefb80905121527p61ec73f7tbae4b21406a0acb2@mail.gmail.com> <4A0B3707.1010104@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> Message-ID: <20090514134407.540e9c44@o> Le Wed, 13 May 2009 20:10:42 -0400, George Sakkis s'exprima ainsi: > Subproposal (2): If subproposal (1) is accepted, we could get for free > (in terms of syntax at least) dynamic args depending on previous ones. > That is, > > def myfunc(a, b, *m=(a+b)/2): > > would mean > > def myfunc(a, b, *m = lambda a,b: (a+b)/2): > > with the lambda being passed the values of a and b at runtime. While I understand the intent, this seems complicated to me. I find clearer to express m in the func body; using a sentinel if m is a real default arg (meaning it could possibly be passed by the user). UNDEF = object() def myfunc(a, b, m=UNDEF): if m is UNDEF: m = (a+b)/2) Generally speaking, I find ok the need of sentinels for clarifying rare and non-obvious cases such as runtime-changing default values: def somefunc(arg, m=UNDEF): if m is UNDEF: m = runtimeDefaultVal() While I do not find ok the need of a sentinel to avoid the common gotcha of a default value beeing "back-updated" when the corresponding local var is changed in the func body: def otherfunc(arg, l=UNDEF): if l is UNDEF: l = [] Denis ------ la vita e estrany From denis.spir at free.fr Thu May 14 14:03:20 2009 From: denis.spir at free.fr (spir) Date: Thu, 14 May 2009 14:03:20 +0200 Subject: [Python-ideas] Add a "sitedirs" list in the site module In-Reply-To: <94bdd2610905140310s6fadd95bv58bfbae6ec5f4f8d@mail.gmail.com> References: <94bdd2610905140310s6fadd95bv58bfbae6ec5f4f8d@mail.gmail.com> Message-ID: <20090514140320.1c27fdc2@o> Le Thu, 14 May 2009 12:10:41 +0200, Tarek Ziad? s'exprima ainsi: > Right now there's no way to differenciate in sys.path the directories > that where added through site.addsitedirs calls, > (like when python starts up and initializes site-packages directories) > > Anyone can change sys.path, and add elements in them. > > So there's no way to answer to this simple question for sure : "what > are the site-packages directories ?" Do you mean in fact be able to know e.g. what PYTHONPATH contained before python startup? If yes, I'm +1 for this. I would even differenciate - sys.pythondir (where the executable is, if anyone cares) - sys.defaultlibdir (where we should install a package) Denis ------ la vita e estrany From gerald.britton at gmail.com Thu May 14 16:29:04 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 14 May 2009 10:29:04 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <87fxfb0z86.fsf@uwakimon.sk.tsukuba.ac.jp> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> <20090513175437.7ca69845@o> Message-ID: <5d1a32000905140729g9c2c58bj68a4050a249f5e05@mail.gmail.com> print(print) is not a function call in 2.x: >>> import types >>> def f(): pass ... >>> isinstance(f, types.FunctionType) True >>> isinstance(print, types.FunctionType) File "", line 1 isinstance(print, types.FunctionType) ^ SyntaxError: invalid syntax >>> p = "hi there" >>> print p hi there >>> print(p) hi there (print_) is interpreted as an expression, which is then passed to the print statement On Wed, May 13, 2009 at 3:30 PM, Scott David Daniels wrote: > spir wrote: >> >> My opinion on this is you're basically right. Even 'print' (for py<3.0) >> could be an identifier you could use in an assignment (or in any value >> expression), I guess, for parse patterns are different: >> ? print_statement : "print" expression >> ? assignment ? ? ?: name '=' expression >> So you can safely have "print" as name, or inside an expression. Even >> "print print" should work ! > > But you would not want > ? ?print print > and > ? ?print(print) > to have two different meanings. > In Python, extra parens are fair around expressions, > and print(print) is clearly a function call. > > --Scott David Daniels > Scott.Daniels at Acm.Org > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From gerald.britton at gmail.com Thu May 14 16:30:58 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 14 May 2009 10:30:58 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <5d1a32000905140729g9c2c58bj68a4050a249f5e05@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905112122md28c4d5m33426d520c7bdce0@mail.gmail.com> <87ab5i1wug.fsf@uwakimon.sk.tsukuba.ac.jp> <4A09D46C.4060006@wanadoo.fr> <4A0A8C39.9070003@improva.dk> <4A0AD920.8060409@mrabarnett.plus.com> <9e754ef50905130752q1d815495u3f8e0c99880dad5f@mail.gmail.com> <20090513175437.7ca69845@o> <5d1a32000905140729g9c2c58bj68a4050a249f5e05@mail.gmail.com> Message-ID: <5d1a32000905140730n260e3173m9dafd82c979b9f6b@mail.gmail.com> Typo: > (print_) is interpreted as an expression, which is then passed to the > print statement should be: (p) is interpreted as an expression, which is then passed to the print statement On Thu, May 14, 2009 at 10:29 AM, Gerald Britton wrote: > print(print) is not a function call in 2.x: > >>>> import types >>>> def f(): pass > ... >>>> isinstance(f, types.FunctionType) > True >>>> isinstance(print, types.FunctionType) > ?File "", line 1 > ? ?isinstance(print, types.FunctionType) > ? ? ? ? ? ? ? ? ? ^ > SyntaxError: invalid syntax >>>> p = "hi there" >>>> print p > hi there >>>> print(p) > hi there > > > (print_) is interpreted as an expression, which is then passed to the > print statement > > On Wed, May 13, 2009 at 3:30 PM, Scott David Daniels > wrote: >> spir wrote: >>> >>> My opinion on this is you're basically right. Even 'print' (for py<3.0) >>> could be an identifier you could use in an assignment (or in any value >>> expression), I guess, for parse patterns are different: >>> ? print_statement : "print" expression >>> ? assignment ? ? ?: name '=' expression >>> So you can safely have "print" as name, or inside an expression. Even >>> "print print" should work ! >> >> But you would not want >> ? ?print print >> and >> ? ?print(print) >> to have two different meanings. >> In Python, extra parens are fair around expressions, >> and print(print) is clearly a function call. >> >> --Scott David Daniels >> Scott.Daniels at Acm.Org >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > > > -- > Gerald Britton > -- Gerald Britton From g.brandl at gmx.net Thu May 14 23:04:24 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 14 May 2009 23:04:24 +0200 Subject: [Python-ideas] func attribute & default arg In-Reply-To: <20090514132339.32010d65@o> References: <20090514132339.32010d65@o> Message-ID: spir schrieb: > Hello, > > I think there is some confusion on the topic of static variables in relation > to default arguments and function attributes. Or, maybe, my point of view is > different from the one of the majority on python-ideas. (a rather long post) > > To sum up, I think the core of the issue is not about static/dymamic > evaluation, or compile time/run time; rather it lies in the fact that python > internally does not separate a default *value* from the corresponding local > *variable*. Please try to think in terms of *names* and *objects*, that makes much more sense with Python's namespaces. Georg From steve at pearwood.info Thu May 14 23:25:28 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 07:25:28 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <20090514134407.540e9c44@o> References: <4A0496BD.3030709@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> <20090514134407.540e9c44@o> Message-ID: <200905150725.28954.steve@pearwood.info> On Thu, 14 May 2009 09:44:07 pm spir wrote: > Generally speaking, I find ok the need of sentinels for clarifying > rare and non-obvious cases such as runtime-changing default values: > > def somefunc(arg, m=UNDEF): > if m is UNDEF: > m = runtimeDefaultVal() > > While I do not find ok the need of a sentinel to avoid the common > gotcha of a default value beeing "back-updated" when the > corresponding local var is changed in the func body: > > def otherfunc(arg, l=UNDEF): > if l is UNDEF: > l = [] > But those two idioms are the same thing! In the first case, if m is not provided by the caller, your function has to produce a fresh object at runtime. It does this by calling runtimeDefaultVal() which returns some unspecified object. In the second case, if l is not provided by the caller, your function has to produce a fresh object at runtime. It does this by calling []. This is merely a special case of the first case, where runtimeDefaultVal() simply returns [] every time. -- Steven D'Aprano From tjreedy at udel.edu Thu May 14 23:33:36 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 14 May 2009 17:33:36 -0400 Subject: [Python-ideas] func attribute & default arg In-Reply-To: <20090514132339.32010d65@o> References: <20090514132339.32010d65@o> Message-ID: spir wrote: > To sum up, I think the core of the issue is not about static/dymamic > evaluation, or compile time/run time; rather it lies in the fact that > python internally does not separate a default *value* from the > corresponding local *variable*. Good try, but you seem to have missed the basic point that underlies at several ways that newbies get tripped up*. Python is an *object* based language, not a *value* based language. By 'value', I mean the information carried by the object. Objects have an identity, values do not. This is a defining characteristic of Python. If one wants an immutable value language, Python is not the right choice. For immutable objects, the difference is nearly invisible. For mutable objects, which generally do not exist in mathematics (which is generally timeless) the difference is crucial to understand. When people are tripped up by their 'intuition', it is sometimes intuition based on math that does not apply to mutable objects. Names are bound to objects, not to values. For functions, parameters are local names, arguments are objects (and not values). Functions may have default objects, not 'default values' (as you say above and elsewhere). If a default object is mutable, it has an initial value, but not a constant value. Function objects already have an attribute that is a tuple of default objects. (Parameter names, without and with defaults, are part of the code object.) Python 3.0.1 ,,, on win32 >>> def f(a, b=1, c='1'): pass >>> f.__defaults__ (1, '1') Terry Jan Reedy * Mutable objects with multiple names, mutable objects passed as argument, mutables used as defaults, constructing initialized lists of lists. From steve at pearwood.info Fri May 15 00:05:20 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 08:05:20 +1000 Subject: [Python-ideas] func attribute & default arg In-Reply-To: <20090514132339.32010d65@o> References: <20090514132339.32010d65@o> Message-ID: <200905150805.20534.steve@pearwood.info> On Thu, 14 May 2009 09:23:39 pm spir wrote: > What we need is only *caching* a default value while the function def > is first evaluated. A possible implementation (probably requiring few > changes) may be to add a kind of __defaults__ dict attribute to > functions. You mean like this? >>> def parrot(x=45): ... return x+1 ... >>> parrot() 46 >>> parrot.func_defaults (45,) >>> parrot.func_defaults = (17,) >>> parrot() 18 This has existed since at least Python 1.5. [...] > A transition phase could be planned toward cached defaults, possibly > with proper warnings: > > "Warning: use of a mutable default value updated in the function > body. From python 3.x, default values will be cached so as not to > change between calls. This makes no sense. Python already caches defaults. If you modify a cached object, you will see the modifications in the next call. You have your understanding completely backwards: what some people want is for Python to *stop* caching default values. Denis, with respect, I think your idea is too raw for this list. I don't wish to discourage you, but should try posting to the general python mailing list first, for feedback and suggestions. I think this confusion about cached objects is the sort of fundamental misunderstanding that is best discussed on the general python mailing list rather than here. -- Steven D'Aprano From steve at pearwood.info Fri May 15 00:20:45 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 08:20:45 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> Message-ID: <200905150820.45400.steve@pearwood.info> On Thu, 14 May 2009 12:41:17 pm Bruce Leban wrote: > > ? ?def myfunc(a, b, *x=[]): > > > > would be equivalent to what previous proposals would write as > > > > ? ?def myfunc(a, b, *x=lambda: []): > > Explicit is better than implicit. There's a thunk getting created > here, right? Don't you want that to be obvious? I do. No, I don't want it to be obvious. I don't care about thunks, I care that x gets bound at runtime. I don't care what the implementation is: whether it is a thunk, eval(), voodoo or something else, just so long as it works. As for your argument that it is better to be explicit, when you want to add two numbers and compare them with a third, do you write: (1 .__add__(1)).__eq__(2) instead of 1+1 == 2? "Explicit is better than implicit", right? No. 1+1=2 *is* explicit, because that's the Python syntax for addition. All those double-underscore method calls are implementation details that do not belong in "standard" Python code. If Python develops new syntax for late-binding of default arguments, that too will be explicit, and any reference to thunks (or any other mechanism) will be an implementation detail. The syntax shouldn't depend on the implementation. lambda is already disliked by many people, including Guido. I don't think any suggestion that we make lambda more confusing by giving it two very different meanings ("create a thunk" inside function parameter lists, and "create a function" everywhere else) will be very popular on python-dev. -- Steven D'Aprano From steve at pearwood.info Fri May 15 00:27:07 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 08:27:07 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> Message-ID: <200905150827.07565.steve@pearwood.info> On Thu, 14 May 2009 09:03:34 am CTO wrote: > Thanks for the input, but I've already written the code to do this. > It > is available at . [...] > The gode given is slow and ugly, but it does appear- > at least to me- to do what is being asked here. Your code seems to work only if the source to the function is available. That will mean it can't be used by people who want to distribute .pyc files only. -- Steven D'Aprano From debatem1 at gmail.com Fri May 15 01:28:02 2009 From: debatem1 at gmail.com (CTO) Date: Thu, 14 May 2009 16:28:02 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905150827.07565.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <316DBF99-F56A-4AA8-AB3C-38F0532ACEAF@googlemail.com> <200905150827.07565.steve@pearwood.info> Message-ID: <8701e71e-918f-4a20-a25a-2bd1b572be4c@n4g2000vba.googlegroups.com> On May 14, 6:27?pm, Steven D'Aprano wrote: > On Thu, 14 May 2009 09:03:34 am CTO wrote: > > > Thanks for the input, but I've already written the code to do this. > > It > > is available at . > > [...] > > > The gode given is slow and ugly, but it does appear- > > at least to me- to do what is being asked here. > > Your code seems to work only if the source to the function is available. > That will mean it can't be used by people who want to distribute .pyc > files only. > > -- > Steven D'Aprano I think the list is eating my replies, but suffice to say that there's a new version of the recipe at that doesn't have that limitation and looks pretty close to the syntax proposed above. Example: >>> @runtime ... def myfunc(x, y, z: lambda:[]): ... z.extend((x,y)) ... return z ... >>> myfunc(1, 2) [1, 2] >>> myfunc(3, 4) [3, 4] >>> myfunc(1, 2, z=[3, 4]) [3, 4, 1, 2] Geremy Condra From steve at pearwood.info Fri May 15 02:15:33 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 10:15:33 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <200905140801.33320.steve@pearwood.info> Message-ID: <200905151015.34186.steve@pearwood.info> On Thu, 14 May 2009 09:02:08 am Terry Reedy wrote: > Steven D'Aprano wrote: > > On Thu, 14 May 2009 05:18:37 am CTO wrote: > >> If you thought not reevaluating function expressions > >> was confusing for newbies, wait until you see what making up a new > >> kind of yield will do for them. > >> > >> Why not just push for some decorators that do this to be included > >> in stdlib? I see the utility, but not the point of adding extra > >> syntax. > > > > Even if a decorator solution can be made to work, it seems to me > > that the difficulty with a decorator solution is that it is > > all-or-nothing -- you can decorate the entire parameter list, or > > none of the parameters, but not some of the parameters. You can bet > > that people will say they want delayed evaluation of some default > > arguments and compile-time evaluation of others, in the same > > function definition. > > Not all or nothing, and selection is easy. A decorator could only > call callable objects, and could/should be limited to calling > function objects or even function objects named ''. Some people don't like writing: def f(x=SENTINEL): if x is SENTINEL: x = [] and wish to have syntax so they can write something approaching: def f(x=[]): ... but have a fresh [] bound to x. You're supporting the syntax: @call_lambdas # Geremy Condra uses the name 'runtime' def f(x=lambda:[]): ... (For the record, I've suggested creating a unary-& operator so that we can write "def f(&x=[])" to get late-binding of x.) If I were to use the proposed late-binding feature, I would want it to be easy to use and obvious. I don't mind having to learn special syntax -- I'm not asking for it to be intuitive or guessable. But having to define the default value as a function (with or without lambda!) *and* call a decorator doesn't seem either easy or obvious. It feels like a kludge designed to get around a limitation of the language. (If you don't like the negative connotations of 'kludge', read it as 'hack' instead.) In other words, it looks like your suggestion is "let's find another idiom for late-binding default arguments" rather than "let's give Python built-in support for optional late-binding of default arguments". If the first one is your intention, then I'll just walk away from this discussion. I already have a perfectly obvious and explicit idiom for late-binding of default arguments. I don't need a second one, especially one which I find exceedingly inelegant and ugly. If you want to use that in your own code, go right ahead, but I hope it never makes it into any code I ever need to read. -1 from me on any solution which requires both a decorator and special treatment of defaults in the parameter list. In my opinion, only a solution with built-in support from the compiler is worth supporting. Anything else is a heavyweight, complicated solution for a problem that already has a lightweight, simple solution: use a sentinel. We already have a concise, fast, straightforward idiom which is easily learned and easily written, and while it's not intuitively obvious to newbies, neither is the suggested decorator+lambda solution. We don't need a complicated, verbose, hard-to-explain, hard-to-implement solution as well. -- Steven D'Aprano From steve at pearwood.info Fri May 15 02:19:26 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 10:19:26 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <8701e71e-918f-4a20-a25a-2bd1b572be4c@n4g2000vba.googlegroups.com> References: <4A0496BD.3030709@wanadoo.fr> <200905150827.07565.steve@pearwood.info> <8701e71e-918f-4a20-a25a-2bd1b572be4c@n4g2000vba.googlegroups.com> Message-ID: <200905151019.27019.steve@pearwood.info> On Fri, 15 May 2009 09:28:02 am CTO wrote: > On May 14, 6:27?pm, Steven D'Aprano wrote: > > On Thu, 14 May 2009 09:03:34 am CTO wrote: > > > Thanks for the input, but I've already written the code to do > > > this. It > > > is available at > > > . > > > > [...] > > > > > The gode given is slow and ugly, but it does appear- > > > at least to me- to do what is being asked here. > > > > Your code seems to work only if the source to the function is > > available. That will mean it can't be used by people who want to > > distribute .pyc files only. > > > > -- > > Steven D'Aprano > > I think the list is eating my replies, but suffice to say that > there's a new version of the recipe at http://code.activestate.com/recipes/576754/> that doesn't have that > limitation and looks pretty close to the syntax proposed above. And instead has another limitation, namely that it only works if you pass the non-default argument by keyword. f(123, y=456) # works f(123, 456) # fails if y has been given a default value. -- Steven D'Aprano From tleeuwenburg at gmail.com Fri May 15 03:42:29 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Fri, 15 May 2009 11:42:29 +1000 Subject: [Python-ideas] Off topic but easy question Message-ID: <43c8685c0905141842g5399ce07tcd069451e4e68929@mail.gmail.com> Hi all, I'm trying to find a web link describing the voting system of +1, +0, -0, -1 to show some people at work, but my Google-fu is weak today. Can someone point me in the right direction? Thanks, -T -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From curt at hagenlocher.org Fri May 15 03:44:27 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Thu, 14 May 2009 18:44:27 -0700 Subject: [Python-ideas] Off topic but easy question In-Reply-To: <43c8685c0905141842g5399ce07tcd069451e4e68929@mail.gmail.com> References: <43c8685c0905141842g5399ce07tcd069451e4e68929@mail.gmail.com> Message-ID: http://www.apache.org/foundation/voting.html Scroll halfway down. On Thu, May 14, 2009 at 6:42 PM, Tennessee Leeuwenburg < tleeuwenburg at gmail.com> wrote: > Hi all, > > I'm trying to find a web link describing the voting system of +1, +0, -0, > -1 to show some people at work, but my Google-fu is weak today. Can someone > point me in the right direction? > > Thanks, > -T > > -- > -------------------------------------------------- > Tennessee Leeuwenburg > http://myownhat.blogspot.com/ > "Don't believe everything you think" > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tleeuwenburg at gmail.com Fri May 15 03:46:59 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Fri, 15 May 2009 11:46:59 +1000 Subject: [Python-ideas] Off topic but easy question In-Reply-To: References: <43c8685c0905141842g5399ce07tcd069451e4e68929@mail.gmail.com> Message-ID: <43c8685c0905141846t439994c5raa86c07dcf1b6743@mail.gmail.com> Thanks! On Fri, May 15, 2009 at 11:44 AM, Curt Hagenlocher wrote: > http://www.apache.org/foundation/voting.html > > Scroll halfway down. > > On Thu, May 14, 2009 at 6:42 PM, Tennessee Leeuwenburg < > tleeuwenburg at gmail.com> wrote: > >> Hi all, >> >> I'm trying to find a web link describing the voting system of +1, +0, -0, >> -1 to show some people at work, but my Google-fu is weak today. Can someone >> point me in the right direction? >> >> Thanks, >> -T >> >> -- >> -------------------------------------------------- >> Tennessee Leeuwenburg >> http://myownhat.blogspot.com/ >> "Don't believe everything you think" >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> >> > -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Fri May 15 04:37:19 2009 From: debatem1 at gmail.com (CTO) Date: Thu, 14 May 2009 19:37:19 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151015.34186.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905140801.33320.steve@pearwood.info> <200905151015.34186.steve@pearwood.info> Message-ID: <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> > Some people don't like writing: > > def f(x=SENTINEL): > ? ? if x is SENTINEL: x = [] > > and wish to have syntax so they can write something approaching: > > def f(x=[]): > ? ? ... > And I understand that. However, I don't think it's important enough to make it worth changing the language, adding to Python's already significant function call overhead, or making the job of parsing function signatures more difficult. If there is a mechanism to do this inside of Python- and there are several- it is my personal opinion that those should be used in preference to modifying the language. As I am neither the smartest nor most competent programmer here, feel free to disregard my opinion- but the code I have produced matches one of the proposed syntaxes very closely, even if it is not the one you prefer. > but have a fresh [] bound to x. You're supporting the syntax: > > @call_lambdas ?# Geremy Condra uses the name 'runtime' > def f(x=lambda:[]): > ? ? ... > For the record, I'm not supporting a syntax. I'm simply stating that this can be done in Python as it currently stands, and that I am most emphatically not in favor of making function signatures any more complex than they already are. > (For the record, I've suggested creating a unary-& operator so that we > can write "def f(&x=[])" to get late-binding of x.) > It's simple, short, and concise. If I were to get behind a proposal to change the language to support this feature, I would probably either get behind this one or perhaps a more general system for adding a metaclass equivalent to functions. However, as things stand I remain unconvinced that any of these things are necessary, or even particularly desirable, given the aforementioned complexity of function signatures. > If I were to use the proposed late-binding feature, I would want it to > be easy to use and obvious. I don't mind having to learn special > syntax -- I'm not asking for it to be intuitive or guessable. But > having to define the default value as a function (with or without > lambda!) *and* call a decorator doesn't seem either easy or obvious. It > feels like a kludge designed to get around a limitation of the > language. (If you don't like the negative connotations of 'kludge', > read it as 'hack' instead.) In other words, it looks like your > suggestion is "let's find another idiom for late-binding default > arguments" rather than "let's give Python built-in support for optional > late-binding of default arguments". My suggestion is neither to find another idiom or to build in late-binding support. Some people- yourself included- want a new syntax. I demonstrated that close approximations of some of the mentioned syntaxes were possible in the language already, and while I appreciate that your preferred syntax is not on that list, I remain unconvinced that its purported benefits outweigh what I perceive to be its drawbacks. > If the first one is your intention, then I'll just walk away from this > discussion. I already have a perfectly obvious and explicit idiom for > late-binding of default arguments. I don't need a second one, > especially one which I find exceedingly inelegant and ugly. If you want > to use that in your own code, go right ahead, but I hope it never makes > it into any code I ever need to read. -1 from me on any solution which > requires both a decorator and special treatment of defaults in the > parameter list. If you are satisfied with the existing idiom, then use it. If you're not, my code is out there. If you don't like that, then write your own. > In my opinion, only a solution with built-in support from the compiler > is worth supporting. I'm afraid I'm unconvinced on that point. > Anything else is a heavyweight, complicated > solution for a problem that already has a lightweight, simple solution: > use a sentinel. We already have a concise, fast, straightforward idiom > which is easily learned and easily written, and while it's not > intuitively obvious to newbies, neither is the suggested > decorator+lambda solution. We don't need a complicated, verbose, > hard-to-explain, hard-to-implement solution as well. > > -- > Steven D'Aprano I think I've already addressed this point, but once more for the record, I'm just not convinced that any of this- my code or your proposed changes- are needed. Until then you can have my -1. Geremy Condra From debatem1 at gmail.com Fri May 15 04:41:19 2009 From: debatem1 at gmail.com (CTO) Date: Thu, 14 May 2009 19:41:19 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151019.27019.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905150827.07565.steve@pearwood.info> <8701e71e-918f-4a20-a25a-2bd1b572be4c@n4g2000vba.googlegroups.com> <200905151019.27019.steve@pearwood.info> Message-ID: On May 14, 8:19?pm, Steven D'Aprano wrote: > On Fri, 15 May 2009 09:28:02 am CTO wrote: > > > > > On May 14, 6:27?pm, Steven D'Aprano wrote: > > > On Thu, 14 May 2009 09:03:34 am CTO wrote: > > > > Thanks for the input, but I've already written the code to do > > > > this. It > > > > is available at > > > > . > > > > [...] > > > > > The gode given is slow and ugly, but it does appear- > > > > at least to me- to do what is being asked here. > > > > Your code seems to work only if the source to the function is > > > available. That will mean it can't be used by people who want to > > > distribute .pyc files only. > > > > -- > > > Steven D'Aprano > > > I think the list is eating my replies, but suffice to say that > > there's a new version of the recipe at >http://code.activestate.com/recipes/576754/> that doesn't have that > > limitation and looks pretty close to the syntax proposed above. > > And instead has another limitation, namely that it only works if you > pass the non-default argument by keyword. > > f(123, y=456) ?# works > f(123, 456) ?# fails if y has been given a default value. > > -- > Steven D'Aprano Correct. However, I remain confident that someone with ever so slightly more skill than myself can correct that problem- since you already seem to have taken a look at it, maybe that's something you could do? Thanks in advance, Geremy Condra From bruce at leapyear.org Fri May 15 05:32:39 2009 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 14 May 2009 20:32:39 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905150820.45400.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> <200905150820.45400.steve@pearwood.info> Message-ID: On Thu, May 14, 2009 at 3:20 PM, Steven D'Aprano wrote: > On Thu, 14 May 2009 12:41:17 pm Bruce Leban wrote: > > > Explicit is better than implicit. There's a thunk getting created > > here, right? Don't you want that to be obvious? I do. > > No, I don't want it to be obvious. I don't care about thunks, I care > that x gets bound at runtime. I don't care what the implementation is: > whether it is a thunk, eval(), voodoo or something else, just so long > as it works. > > As for your argument that it is better to be explicit, when you want to > add two numbers and compare them with a third, do you write: > > (1 .__add__(1)).__eq__(2) > > instead of > > 1+1 == 2? Absolutely not. This is a false analogy. The anology would be having an implicit multiplication operator and writing (a b c) instead of (a * b * c). The syntax shouldn't depend on the > implementation. > > lambda is already disliked by many people, including Guido. I don't > think any suggestion that we make lambda more confusing by giving it > two very different meanings ("create a thunk" inside function parameter > lists, and "create a function" everywhere else) will be very popular on > python-dev. > > I'm *not* suggesting a new meaning for lambda! This is the same meaning that it has right now. The new meaning is the decorator attached to the default assignment that says evaluate that lambda. I'll use an @dynamic decorator-like syntax to illustrate. These would be valid: def foo(a, b = @dynamic lambda: []): def foo(a, b = @dynamic lambda: list()): def foo(a, b = @dynamic list): def foo(a, b = @dynamic random.random): and this would not: def foo(a, b = @dynamic []) def foo(a, b = @dynamic 5) because @dynamic says that the thing that follows is called to generate a dynamic default parameter value and you can't call [] or 5. My point about creating a thunk is *not* an implementation detail. The point here is that if you use one of the forms above with a lambda, it's the lambda creating a thunk/closure/voodoo thing at this point in the program, *not* the @dynamic decorator. The scope of that lambda is exactly what it looks like it is with or without the @dynamic decorator. Likewise, in the random.random, example, it's the value of random.random at the time the function is defined, not some later value that might be assigned to that name. If you use some other syntax that doesn't look like a lambda, I have to learn the scoping rules for that syntax. I already know the rules for lambda. --- Bruce -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri May 15 05:38:43 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 13:38:43 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> Message-ID: <200905151338.44654.steve@pearwood.info> On Fri, 15 May 2009 12:37:19 pm CTO wrote: > > Some people don't like writing: > > > > def f(x=SENTINEL): > > ? ? if x is SENTINEL: x = [] > > > > and wish to have syntax so they can write something approaching: > > > > def f(x=[]): > > ? ? ... > > And I understand that. However, I don't think it's important > enough to make it worth changing the language, adding to > Python's already significant function call overhead, or > making the job of parsing function signatures more difficult. > If there is a mechanism to do this inside of Python- and > there are several- it is my personal opinion that those > should be used in preference to modifying the language. As > I am neither the smartest nor most competent programmer > here, feel free to disregard my opinion- but the code I > have produced matches one of the proposed syntaxes very > closely, even if it is not the one you prefer. Your code also "add[s] to Python's already significant function call overhead" as well as "making the job of parsing function signatures more difficult". I don't mean to dump on your code. What you are trying to do is obviously very difficult from pure Python code, and the solutions you have come up with are neat kludges. But a kludge is still a kludge, no matter how neat it is :) [...] > > (For the record, I've suggested creating a unary-& operator so that > > we can write "def f(&x=[])" to get late-binding of x.) > > It's simple, short, and concise. If I were to get behind > a proposal to change the language to support this feature, > I would probably either get behind this one or perhaps > a more general system for adding a metaclass equivalent > to functions. However, as things stand I remain unconvinced > that any of these things are necessary, or even > particularly desirable, given the aforementioned > complexity of function signatures. I think we two at least agree. I don't think there's anything wrong with the current sentinel idiom. It's not entirely intuitive to newbies, or those who don't fully understand Python's object-binding model, but I don't consider that a flaw. So I don't see the compile-time binding of default args to be a problem that needs solving. But other people do, and they are loud and consistent in their complaints. Given that the squeaky wheel (sometimes) gets the grease, I'd just like to see a nice solution to a (non-)problem rather than an ugly solution. So I'm +0 on my proposal -- I don't think it solves a problem that needs solving, but other people do. I'm -1 on decorator+lambda solutions, because not only do they not solve a problem that needs solving, but they don't solve it in a particularly ugly and inefficient way *wink* > My suggestion is neither to find another idiom or to build in > late-binding support. Some people- yourself included- I think you've misunderstood my position. I'm one of the people defending the current semantics of default arg binding. But since others want optional late binding, I'm just trying to find a syntax that doesn't bite :) > want a > new syntax. I demonstrated that close approximations of some > of the mentioned syntaxes were possible in the language already, > and while I appreciate that your preferred syntax is not on that > list, I remain unconvinced that its purported benefits outweigh > what I perceive to be its drawbacks. Just out of curiosity, what do you see as the drawbacks? The ones that come to my mind are: * people who want late binding to be standard will be disappointed (but that will be true of any solution) * requires changes to Python's parser, to allow unary-& (but that will probably be very simple) * requires charges to Python's compiler, to allow for some sort of late-binding semantics (thunks?) (but that will probably be very hard) * requires people to learn one more feature (so newbies will still be confused that def f(x=[]) doesn't behave as they expect). -- Steven D'Aprano From tleeuwenburg at gmail.com Fri May 15 06:16:04 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Fri, 15 May 2009 14:16:04 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151338.44654.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> Message-ID: <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> A thought from another direction... Any chance we could have the interpreter raise a warning for the case def foo(a = []): #stuff ? The empty list and empty dict args would, I imagine, be the two most common mistakes. Showing a warning might, at least, solve the problem of people tripping over the syntax. Cheers, -T -------------- next part -------------- An HTML attachment was scrubbed... URL: From curt at hagenlocher.org Fri May 15 06:31:34 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Thu, 14 May 2009 21:31:34 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> Message-ID: I think this takes the discussion in a more practical direction. Imagine that there were a special method name __immutable__ to be implemented appropriately by all builtin types. Any object passed as a default argument would be checked to see that its type implements __immutable__ and that __immutable__() is True. Failure would mean a warning or even an error in subsequent versions. User-defined types could implement __immutable__ as they saw fit, in the traditional Pythonic consenting-adults-ly way. On Thu, May 14, 2009 at 9:16 PM, Tennessee Leeuwenburg < tleeuwenburg at gmail.com> wrote: > A thought from another direction... > > Any chance we could have the interpreter raise a warning for the case > > def foo(a = []): > #stuff > > ? > > The empty list and empty dict args would, I imagine, be the two most common > mistakes. Showing a warning might, at least, solve the problem of people > tripping over the syntax. > > Cheers, > -T > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Fri May 15 06:48:53 2009 From: debatem1 at gmail.com (CTO) Date: Thu, 14 May 2009 21:48:53 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151338.44654.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> Message-ID: <0a821a3c-dcab-4f50-b228-204c7b1177b5@q2g2000vbr.googlegroups.com> [super-snip] > Just out of curiosity, what do you see as the drawbacks? [snip] 1) It adds to the complexity (and therefore overhead) of calling functions- not just the functions which use it, but even functions which operate as normal. Python already has a hefty penalty for calling functions, and I really don't want it to get any heavier. My 'solutions', as incomplete as they are, at least don't slow down anything else. 2) It adds to the complexity of introspecting functions. Take a good look at inspect.getfullargspec- its a nightmare, and either it gets worse under this (bad) or it doesn't include information that is available to the compiler (even worse). In addition to those minuses, it doesn't actually add to the capabilities of the language. If this were a proposal to add early action to Python (the equivalent of metaclasses or, to some extent, macro replacement) I would be much more likely to support it, despite the heavier syntax. So, the existing idiom works pretty well, there doesn't seem to be a very good substitute, it slows the whole language down to implement, and it doesn't add any power if you do. Like I say, I'm unconvinced. Geremy Condra From stephen at xemacs.org Fri May 15 07:08:18 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 15 May 2009 14:08:18 +0900 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <200905151015.34186.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905140801.33320.steve@pearwood.info> <200905151015.34186.steve@pearwood.info> Message-ID: <87iqk3eywt.fsf@uwakimon.sk.tsukuba.ac.jp> Steven D'Aprano writes: > (For the record, I've suggested creating a unary-& operator so that > we can write "def f(&x=[])" to get late-binding of x.) Could you summarize that discussion briefly? From pyideas at rebertia.com Fri May 15 08:13:23 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Thu, 14 May 2009 23:13:23 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> Message-ID: <50697b2c0905142313o75fb7324sa0ceadbfe36e1aa7@mail.gmail.com> On Thu, May 14, 2009 at 9:16 PM, Tennessee Leeuwenburg wrote: > A thought from another direction... > > Any chance we could have the interpreter raise a warning for the case > > def foo(a = []): > ? #stuff > > ? > > The empty list and empty dict args would, I imagine, be the two most common > mistakes. Showing a warning might, at least, solve the problem of people > tripping over the syntax. +1 on throwing a ValueError for non-hash()-able (and thus probably mutable) default argument values. It's by no means perfect since objects are hash()-able by default using their ID, but it would at least help in the frequent "well-behaved mutable container object" cases. The barrier to this idea would be the code breakage involved; IMHO, code exploiting mutable defaults as static variables is in poor style anyway, but backward compatibility is a significant concern of the BDFL and Python devs; though I would hope the breakage might be seen as justifiable in this case. Cheers, Chris -- http://blog.rebertia.com From pyideas at rebertia.com Fri May 15 08:20:45 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Thu, 14 May 2009 23:20:45 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> Message-ID: <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> > On Thu, May 14, 2009 at 9:16 PM, Tennessee Leeuwenburg > wrote: >> >> A thought from another direction... >> >> Any chance we could have the interpreter raise a warning for the case >> >> def foo(a = []): >> ? #stuff >> >> ? >> >> The empty list and empty dict args would, I imagine, be the two most >> common mistakes. Showing a warning might, at least, solve the problem of >> people tripping over the syntax. On Thu, May 14, 2009 at 9:31 PM, Curt Hagenlocher wrote: > I think this takes the discussion in a more practical direction. Imagine > that there were a special method name __immutable__ to be implemented > appropriately by all builtin types. Any object passed as a default argument > would be checked to see that its type implements __immutable__ and that > __immutable__() is True. Failure would mean a warning or even an error in > subsequent versions. > > User-defined types could implement __immutable__ as they saw fit, in the > traditional Pythonic consenting-adults-ly way. (A) Python's new Abstract Base Classes would probably be a better way of doing such checking rather than introducing a new special method (B) What about having an __immutable__() that returned an immutable version of the object if possible? Then all default arguments could be converted to immutables at definition-time, with errors if a default cannot be made immutable? It would eliminate the performance concerns since the overhead would only be incurred once (when the function gets defined), rather than with each function call. Cheers, Chris -- http://blog.rebertia.com From cmjohnson.mailinglist at gmail.com Fri May 15 09:30:18 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Thu, 14 May 2009 21:30:18 -1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> <200905150820.45400.steve@pearwood.info> Message-ID: <3bdda690905150030u2de15c0frf6de7de27f0466dd@mail.gmail.com> Bruce Leban wrote: > I'll use an @dynamic decorator-like syntax to illustrate. These would be > valid: > > def foo(a, b = @dynamic lambda: []): > def foo(a, b = @dynamic lambda: list()): > def foo(a, b = @dynamic list): > def foo(a, b = @dynamic random.random): > > and this would not: > > def foo(a, b = @dynamic []) > def foo(a, b = @dynamic 5) > > because @dynamic says that the thing that follows is called to generate a > dynamic default parameter value and you can't call [] or 5. Hmm, very interesting, but in your example what is "dynamic" doing? Are you proposing it as a keyword to signal "here comes a dynamic default"? Do we really need it? Why not something like this: def five_appender(x=@list): x.append(5) return x >>> five_appender() [5] >>> five_appender() [5] The idea is that @ is a magic sigil meaning, "call this if no argument is passed in." So, as per your prior example @[] or @5 would be result in a runtime error, since they're not callable. If for some reason you want a fresh 5 (I can't think of why, since it's immutable, but whatever), you would need to use a lambda: def n_appender(n=@lambda: 5, x=@list): x.append(n) return x Do y'all think this is enough inline with how @ is already used to make sense? Or is it too different from the existing use of @? -- Carl Johnson From debatem1 at gmail.com Fri May 15 10:13:36 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 15 May 2009 01:13:36 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <3bdda690905150030u2de15c0frf6de7de27f0466dd@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <91ad5bf80905131710p6462d626x6e0660012914462@mail.gmail.com> <200905150820.45400.steve@pearwood.info> <3bdda690905150030u2de15c0frf6de7de27f0466dd@mail.gmail.com> Message-ID: On May 15, 3:30?am, Carl Johnson wrote: > Bruce Leban wrote: > > I'll use an @dynamic decorator-like syntax to illustrate. These would be > > valid: > > > def foo(a, b = @dynamic lambda: []): > > def foo(a, b = @dynamic lambda: list()): > > def foo(a, b = @dynamic list): > > def foo(a, b = @dynamic random.random): > > > and this would not: > > > def foo(a, b = @dynamic []) > > def foo(a, b = @dynamic 5) > > > because @dynamic says that the thing that follows is called to generate a > > dynamic default parameter value and you can't call [] or 5. > > Hmm, very interesting, but in your example what is "dynamic" doing? > Are you proposing it as a keyword to signal "here comes a dynamic > default"? Do we really need it? Why not something like this: > > def five_appender(x=@list): > ? ? x.append(5) > ? ? return x > > >>> five_appender() > [5] > >>> five_appender() > > [5] > > The idea is that @ is a magic sigil meaning, "call this if no argument > is passed in." So, as per your prior example @[] or @5 would be result > in a runtime error, since they're not callable. If for some reason you > want a fresh 5 (I can't think of why, since it's immutable, but > whatever), you would need to use a lambda: > > def n_appender(n=@lambda: 5, x=@list): > ? ? x.append(n) > ? ? return x > > Do y'all think this is enough inline with how @ is already used to > make sense? Or is it too different from the existing use of @? > > -- Carl Johnson If we're making magical objects, why not just make a magical object that gives you the ability to defer the execution of a block of code until an operation is performed on it? That way at least it makes sense if you've learned the rest of the language. Geremy Condra From steve at pearwood.info Fri May 15 10:14:50 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 18:14:50 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <0a821a3c-dcab-4f50-b228-204c7b1177b5@q2g2000vbr.googlegroups.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151338.44654.steve@pearwood.info> <0a821a3c-dcab-4f50-b228-204c7b1177b5@q2g2000vbr.googlegroups.com> Message-ID: <200905151814.50603.steve@pearwood.info> On Fri, 15 May 2009 02:48:53 pm CTO wrote: > [super-snip] > > > Just out of curiosity, what do you see as the drawbacks? > > [snip] > > 1) It adds to the complexity (and therefore overhead) of > calling functions- not just the functions which use it, > but even functions which operate as normal. Without an implementation, how can you possibly predict the cost of it? > Python > already has a hefty penalty for calling functions, I think you're badly mistaken. Python has a hefty cost for looking up names, but the overhead to *call* a function once you have looked up the name is minimal. >>> from timeit import Timer >>> def f(): ... pass ... >>> min(Timer('f', 'from __main__ import f').repeat()) 0.32181000709533691 >>> min(Timer('f()', 'from __main__ import f').repeat()) 0.35797882080078125 No significant difference between looking up f and looking up f and calling it. Even if you give the function a complex signature, it's still relatively lightweight: >>> def g(a=1, b=2, c=3, d=4, e=5, f=6, g=7, h=8, *args, **kwargs): ... pass ... >>> min(Timer('g()', 'from __main__ import g').repeat()) 0.55176901817321777 > and > I really don't want it to get any heavier. My 'solutions', > as incomplete as they are, at least don't slow down > anything else. Oh the irony. Decorators are very heavyweight. Here's a decorator that essentially does nothing at all, and it triples the cost of calling the function: >>> from functools import wraps >>> def decorator(f): ... @wraps(f) ... def inner(*args, **kwargs): ... return f(*args, **kwargs) ... return inner ... >>> @decorator ... def h(): ... pass ... >>> min(Timer('h()', 'from __main__ import h').repeat()) 1.1645870208740234 I think, before making claims as to what's costly and what isn't, you should actually do some timing measurements. > 2) It adds to the complexity of introspecting functions. > Take a good look at inspect.getfullargspec- its a > nightmare, and either it gets worse under this (bad) > or it doesn't include information that is available > to the compiler (even worse). Well obviously this is going to make getfullargspec more complicated. But tell me, what do you think your solution using decorators does to getfullargspec? > In addition to those minuses, it doesn't actually add > to the capabilities of the language. It's an incremental improvement. Currently, late-binding of defaults requires boilerplate code. This will eliminate that boilerplate code. > If this were a > proposal to add early action to Python (the equivalent > of metaclasses or, to some extent, macro replacement) > I would be much more likely to support it, despite the > heavier syntax. > > So, the existing idiom works pretty well, 100% agreed! > there doesn't seem to be a very good substitute, Not without support in the compiler. > it slows the whole language down to implement, You can't know that. > and it doesn't add any power if you do. It reduces boilerplate, which is a good thing. Probably the *only* good thing, but still a good thing. -- Steven D'Aprano From debatem1 at gmail.com Fri May 15 10:24:36 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 15 May 2009 01:24:36 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151814.50603.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151338.44654.steve@pearwood.info> <0a821a3c-dcab-4f50-b228-204c7b1177b5@q2g2000vbr.googlegroups.com> <200905151814.50603.steve@pearwood.info> Message-ID: <431bdbe0-a555-4f63-a34e-18c2b7f770c3@l28g2000vba.googlegroups.com> On May 15, 4:14?am, Steven D'Aprano wrote: > On Fri, 15 May 2009 02:48:53 pm CTO wrote: > > > [super-snip] > > > > Just out of curiosity, what do you see as the drawbacks? > > > [snip] > > > 1) It adds to the complexity (and therefore overhead) of > > ? ?calling functions- not just the functions which use it, > > ? ?but even functions which operate as normal. > > Without an implementation, how can you possibly predict the cost of it? > > > ? ?Python > > ? ?already has a hefty penalty for calling functions, > > I think you're badly mistaken. Python has a hefty cost for looking up > names, but the overhead to *call* a function once you have looked up > the name is minimal. > > >>> from timeit import Timer > >>> def f(): > > ... ? ? pass > ...>>> min(Timer('f', 'from __main__ import f').repeat()) > 0.32181000709533691 > >>> min(Timer('f()', 'from __main__ import f').repeat()) > > 0.35797882080078125 > > No significant difference between looking up f and looking up f and > calling it. > > Even if you give the function a complex signature, it's still relatively > lightweight: > > >>> def g(a=1, b=2, c=3, d=4, e=5, f=6, g=7, h=8, *args, **kwargs): > > ... ? ? pass > ...>>> min(Timer('g()', 'from __main__ import g').repeat()) > > 0.55176901817321777 > > > ? ?and > > ? ?I really don't want it to get any heavier. My 'solutions', > > ? ?as incomplete as they are, at least don't slow down > > ? ?anything else. > > Oh the irony. Decorators are very heavyweight. Here's a decorator that > essentially does nothing at all, and it triples the cost of calling the > function: > > >>> from functools import wraps > >>> def decorator(f): > > ... ? ? @wraps(f) > ... ? ? def inner(*args, **kwargs): > ... ? ? ? ? ? ? return f(*args, **kwargs) > ... ? ? return inner > ...>>> @decorator > > ... def h(): > ... ? ? pass > ...>>> min(Timer('h()', 'from __main__ import h').repeat()) > > 1.1645870208740234 > > I think, before making claims as to what's costly and what isn't, you > should actually do some timing measurements. > > > 2) It adds to the complexity of introspecting functions. > > ? ?Take a good look at inspect.getfullargspec- its a > > ? ?nightmare, and either it gets worse under this (bad) > > ? ?or it doesn't include information that is available > > ? ?to the compiler (even worse). > > Well obviously this is going to make getfullargspec more complicated. > But tell me, what do you think your solution using decorators does to > getfullargspec? > > > In addition to those minuses, it doesn't actually add > > to the capabilities of the language. > > It's an incremental improvement. Currently, late-binding of defaults > requires boilerplate code. This will eliminate that boilerplate code. > > > If this were a > > proposal to add early action to Python (the equivalent > > of metaclasses or, to some extent, macro replacement) > > I would be much more likely to support it, despite the > > heavier syntax. > > > So, the existing idiom works pretty well, > > 100% agreed! > > > there doesn't seem to be a very good substitute, > > Not without support in the compiler. > > > it slows the whole language down to implement, > > You can't know that. > > > and it doesn't add any power if you do. > > It reduces boilerplate, which is a good thing. Probably the *only* good > thing, but still a good thing. > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-id... at python.orghttp://mail.python.org/mailman/listinfo/python-ideas From steve at pearwood.info Fri May 15 10:44:54 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 18:44:54 +1000 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <87iqk3eywt.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <87iqk3eywt.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <200905151844.55099.steve@pearwood.info> On Fri, 15 May 2009 03:08:18 pm Stephen J. Turnbull wrote: > Could you summarize that discussion briefly? Many newbies, and some more experienced programmers, are confused by the behaviour of functions when parameters are given default mutable arguments: >>> def f(x=[]): ... x.append(1) ... return x ... >>> f() [1] >>> f() [1, 1] Some people are surprised by this behaviour, and would prefer that the default value for x be freshly created each time it is needed. This is one of the most common, and most acrimonious, topics of discussion on comp.lang.python. The standard idiom for the expected behaviour is to insert boilerplate code that checks for a sentinel: def f(x=None): if x is None: x = [] x.append(1) return x The chances of having the standard behaviour changed are slim, at best, for various reasons including backward compatibility and runtime efficiency. Also, I believe Guido has ruled that the standard behaviour will not be changed. However, some have suggested that if the standard compile-time creation of defaults won't be changed, perhaps it could be made optional, with special syntax, or perhaps a decorator, controlling the behaviour. See these two proof-of-concept decorators, by Geremy Condra, for example: http://code.activestate.com/recipes/576751/ http://code.activestate.com/recipes/576754/ I'm not convinced by decorator-based solutions, so I'll pass over them. I assume that any first-class solution will require cooperation from the compiler, and thus move the boilerplate out of the function body into the byte code. (Or whatever implementation is used -- others have suggested using thunks.) Assuming such compiler support is possible, it only remains to decide on syntax for it. Most suggested syntax I've seen has marked the default value itself, e.g.: def f(x = new []). Some have suggested overloading lambda, perhaps with some variation like def f(x = *lambda:[]). I suggest that the markup should go on the formal parameter name, not the default value: we're marking the formal parameter as "special" for using delayed semantics, not that the default object (usually [] or {}) will be special. Some years ago, Python overloaded the binary operators * and ** for use as special markers in parameter lists. I suggest we could do the same, by overloading the & operator in a similar fashion: inside the parameter list, &x would mean to delay evaluation of the default argument: def f(x=[], &y=[]) x would use the current compile-time semantics, y would get the new runtime semantics. I don't have any particular reason for choosing & over any other binary operator. I think ^ would also be a good choice. Tagging a parameter with unary-& but failing to specify a default value should be a syntax error. Likewise for unary-& outside of a parameter list. (At least until such time as somebody suggests a good use for such a thing.) -- Steven D'Aprano From debatem1 at gmail.com Fri May 15 10:45:16 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 15 May 2009 01:45:16 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151814.50603.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151338.44654.steve@pearwood.info> <0a821a3c-dcab-4f50-b228-204c7b1177b5@q2g2000vbr.googlegroups.com> <200905151814.50603.steve@pearwood.info> Message-ID: On May 15, 4:14?am, Steven D'Aprano wrote: [snip] > > Without an implementation, how can you possibly predict the cost of it? > [snip] You're right. Please provide code. Geremy Condra From steve at pearwood.info Fri May 15 10:57:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 18:57:03 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <200905151814.50603.steve@pearwood.info> Message-ID: <200905151857.03826.steve@pearwood.info> On Fri, 15 May 2009 06:45:16 pm CTO wrote: > On May 15, 4:14?am, Steven D'Aprano wrote: > [snip] > > > Without an implementation, how can you possibly predict the cost of > > it? > > [snip] > > You're right. Please provide code. I think that should be up to some person who actually wants delayed evaluation of default arguments. As I've said repeatedly in the past, I'm a very strong -1 on removing the current behaviour, and +0 on allowing delayed evaluation of defaults as an optional feature. But since so many people want it, if the Python-Dev team decide to add it to the language I will need to live with whatever syntax is chosen. -- Steven D'Aprano From steve at pearwood.info Fri May 15 11:05:02 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 19:05:02 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> Message-ID: <200905151905.02445.steve@pearwood.info> On Fri, 15 May 2009 02:16:04 pm Tennessee Leeuwenburg wrote: > A thought from another direction... > > Any chance we could have the interpreter raise a warning for the case > > def foo(a = []): > #stuff > > ? > > The empty list and empty dict args would, I imagine, be the two most > common mistakes. Showing a warning might, at least, solve the problem > of people tripping over the syntax. I made that same suggestion nine months ago: http://mail.python.org/pipermail/python-list/2008-August/504701.html Responses were mixed, some people supported it, others did not, but it went nowhere. -- Steven D'Aprano From tjreedy at udel.edu Fri May 15 11:08:00 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 15 May 2009 05:08:00 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> Message-ID: Tennessee Leeuwenburg wrote: > A thought from another direction... > > Any chance we could have the interpreter raise a warning for the case > > def foo(a = []): > #stuff This would be appropriate for any of the code check programs; PyChecker, PyLint, whatever. they already warn about things that are legal, might be wanted, but have a good chance of being an error. From steve at pearwood.info Fri May 15 11:08:57 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 19:08:57 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <50697b2c0905142313o75fb7324sa0ceadbfe36e1aa7@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142313o75fb7324sa0ceadbfe36e1aa7@mail.gmail.com> Message-ID: <200905151908.57437.steve@pearwood.info> On Fri, 15 May 2009 04:13:23 pm Chris Rebert wrote: > +1 on throwing a ValueError for non-hash()-able (and thus probably > mutable) default argument values. You're not serious are you? What could possibly be either surprising or objectionable about a function like this? def f(data,substitutions = {}): ... name = data['name'] obj.name = substitutions.get(name,name) ... Example taken from: http://mail.python.org/pipermail/python-list/2008-August/504811.html -- Steven D'Aprano From stephen at xemacs.org Fri May 15 11:32:32 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 15 May 2009 18:32:32 +0900 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <50697b2c0905142313o75fb7324sa0ceadbfe36e1aa7@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142313o75fb7324sa0ceadbfe36e1aa7@mail.gmail.com> Message-ID: <87d4aag18v.fsf@uwakimon.sk.tsukuba.ac.jp> Chris Rebert writes: > +1 on throwing a ValueError for non-hash()-able (and thus probably > mutable) default argument values. It's by no means perfect since > objects are hash()-able by default using their ID, but it would at > least help in the frequent "well-behaved mutable container object" > cases. -1 nonet_default_options = { 'dryrun': False, 'verbose': False } net_default_options = { 'dryrun': True, 'verbose': True } def command1(options=nonet_default_options): pass def command2(options=net_default_options): pass def command3(options=net_default_options): pass >>> from mystuff import command1, command2, net_default_options >>> command1() >>> command2() >>> command3() >>> net_default_options['dryrun'] = False >>> command2() >>> command3() is a common use-case for me, both during development and in scripts for occasional personal use. AFAICS this would break under your suggestion. Really, the only commonly-encountered problematic cases I can think of are the anonymous empty objects, because they're typically used not for their contents (d'oh), but rather as containers. pylint and friends can easily detect [] and {} as default values for arguments. > The barrier to this idea would be the code breakage involved; IMHO, > code exploiting mutable defaults as static variables is in poor style > anyway, Sure, but it also breaks "globals" as above. From debatem1 at gmail.com Fri May 15 11:54:22 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 15 May 2009 02:54:22 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151857.03826.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151814.50603.steve@pearwood.info> <200905151857.03826.steve@pearwood.info> Message-ID: On May 15, 4:57?am, Steven D'Aprano wrote: > On Fri, 15 May 2009 06:45:16 pm CTO wrote: > > > On May 15, 4:14?am, Steven D'Aprano wrote: > > [snip] > > > > Without an implementation, how can you possibly predict the cost of > > > it? > > > [snip] > > > You're right. Please provide code. > > I think that should be up to some person who actually wants delayed > evaluation of default arguments. As I've said repeatedly in the past, > I'm a very strong -1 on removing the current behaviour, and +0 on > allowing delayed evaluation of defaults as an optional feature. But > since so many people want it, if the Python-Dev team decide to add it > to the language I will need to live with whatever syntax is chosen. > > -- > Steven D'Aprano Hmm. Well, that doesn't sound like a productive way forward to me, but as I say I'm neither the most intelligent nor the most experienced programmer here, so maybe it's the right way to go. I guess in that case, my current stance is -1 on this, both in added syntax and decorator form, with the caveat that I'd be happy to change my vote if anybody can produce code that does this without mangling performance or introspection. Geremy Condra From larry at hastings.org Fri May 15 11:49:11 2009 From: larry at hastings.org (Larry Hastings) Date: Fri, 15 May 2009 02:49:11 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> Message-ID: <4A0D3A97.5010606@hastings.org> On Thu, May 14, 2009 at 9:31 PM, Curt Hagenlocher wrote: > I think this takes the discussion in a more practical direction. > Imagine that there were a special method name __immutable__ to be > implemented appropriately by all builtin types. Python already has something *vaguely* like this; __hash__ is only supposed to be implemented on immutable objects. So if the object supports __hash__ one could declare that it is supposed to be immutable. However, it's occasionally useful to define __hash__ on mutable objects, and indeed it's defined by default on user-defined classes. So __hash__ isn't a viable substitute for __immutable__ (&c). Chris Rebert wrote: > (A) Python's new Abstract Base Classes would probably be a better way > of doing such checking rather than introducing a new special method > FWIW, I'm +0.3 for either __immutable__ or an ABC to express the concept. I don't know which would be the "right" way to do it--ultimately I defer to my betters. > (B) What about having an __immutable__() that returned an immutable > version of the object if possible? The "freeze" protocol was proposed in PEP 351. http://www.python.org/dev/peps/pep-0351/ It was rejected in 2006. It's a "can of worms". Reading PEPs is fun, /larry/ From ncoghlan at gmail.com Fri May 15 12:26:30 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 15 May 2009 20:26:30 +1000 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <200905151844.55099.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <87iqk3eywt.fsf@uwakimon.sk.tsukuba.ac.jp> <200905151844.55099.steve@pearwood.info> Message-ID: <4A0D4356.3070400@gmail.com> Steven D'Aprano wrote: > Some years ago, Python overloaded the binary operators * and ** for use > as special markers in parameter lists. I suggest we could do the same, > by overloading the & operator in a similar fashion: inside the > parameter list, &x would mean to delay evaluation of the default > argument: Yikes, that syntax would seriously confuse any C++ programmer (me included). I wouldn't be able to avoid thinking of those parameters as pass by reference (i.e. actually referring to the memory location of the passed in argument so you can fiddle with immutable values in the caller, not just pass-a-reference-by-value the way Python does for normal arguments). Marking the parameter name strikes me as wrong anyway - it's only the evaluation of the default argument which is special, not the parameter itself. Cheers, Nick. P.S. (The subject line change caught my attention. The recap of the discussion was very handy, and it does appear to have evolved in a more useful direction, but I'm going back to largely ignoring the thread now...) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From arnodel at googlemail.com Fri May 15 12:52:54 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Fri, 15 May 2009 11:52:54 +0100 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A0D3A97.5010606@hastings.org> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> Message-ID: <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> 2009/5/15 Larry Hastings : > > On Thu, May 14, 2009 at 9:31 PM, Curt Hagenlocher > wrote: > >> I think this takes the discussion in a more practical direction. Imagine >> that there were a special method name __immutable__ to be implemented >> appropriately by all builtin types. > > Python already has something *vaguely* like this; __hash__ is only supposed > to be implemented on immutable objects. ?So if the object supports __hash__ > one could declare that it is supposed to be immutable. > > However, it's occasionally useful to define __hash__ on mutable objects, and > indeed it's defined by default on user-defined classes. ?So __hash__ isn't a > viable substitute for __immutable__ (&c). However immutability is a shallow thing: tuple are immutable but ([]) still can be changed! -- Arnaud From steve at pearwood.info Fri May 15 14:11:33 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 15 May 2009 22:11:33 +1000 Subject: [Python-ideas] =?iso-8859-1?q?Syntax_for_late-binding_of_argument?= =?iso-8859-1?q?s_=5Bwas=3A=09Default_arguments_=2E=2E=2E=5D?= In-Reply-To: <4A0D4356.3070400@gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> Message-ID: <200905152211.34793.steve@pearwood.info> On Fri, 15 May 2009 08:26:30 pm Nick Coghlan wrote: > Steven D'Aprano wrote: > > Some years ago, Python overloaded the binary operators * and ** for > > use as special markers in parameter lists. I suggest we could do > > the same, by overloading the & operator in a similar fashion: > > inside the parameter list, &x would mean to delay evaluation of the > > default argument: [...] > Marking the parameter name strikes me as wrong anyway - it's only the > evaluation of the default argument which is special, not the > parameter itself. But it is the parameter that is special. The default object itself is not. Consider the function definition: def f(x=[], &y=[]): (or any other syntax you prefer). The empty list you get as the default value for x is exactly the same as the empty list you get in y, in every way except for identity. -- Steven D'Aprano From george.sakkis at gmail.com Fri May 15 14:59:35 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 15 May 2009 08:59:35 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> Message-ID: <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> On Fri, May 15, 2009 at 6:52 AM, Arnaud Delobelle wrote: > 2009/5/15 Larry Hastings : >> >> On Thu, May 14, 2009 at 9:31 PM, Curt Hagenlocher >> wrote: >> >>> I think this takes the discussion in a more practical direction. Imagine >>> that there were a special method name __immutable__ to be implemented >>> appropriately by all builtin types. >> >> Python already has something *vaguely* like this; __hash__ is only supposed >> to be implemented on immutable objects. ?So if the object supports __hash__ >> one could declare that it is supposed to be immutable. >> >> However, it's occasionally useful to define __hash__ on mutable objects, and >> indeed it's defined by default on user-defined classes. ?So __hash__ isn't a >> viable substitute for __immutable__ (&c). > > However immutability is a shallow thing: tuple are immutable but ([]) > still can be changed! More to the point, immutability is *not* the issue as Steven D'Aprano showed. There are perfectly legitimate reasons for using a default value that just happens to be mutable, without mutating it in the function body though. Dict is the most common example (especially since there is no frozendict type that could be used in its place). George From jimjjewett at gmail.com Fri May 15 16:14:24 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 15 May 2009 10:14:24 -0400 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <200905152211.34793.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> Message-ID: On 5/15/09, Steven D'Aprano wrote: > On Fri, 15 May 2009 08:26:30 pm Nick Coghlan wrote: >> Steven D'Aprano wrote: >> > ...inside the parameter list, &x would mean to delay evaluation ... >> Marking the parameter name strikes me as wrong anyway - it's only the >> evaluation of the default argument which is special, not the >> parameter itself. > But it is the parameter that is special. The default object itself is > not. Consider the function definition: > def f(x=[], &y=[]): > (or any other syntax you prefer). The empty list you get as the default > value for x is exactly the same as the empty list you get in y, in > every way except for identity. Logically, you're correct. But I think the ('&' ==> addressof) meme may have already grown too strong. What it suggests to me is that normally you *would* create a new list, but the ampersand says not to in just this rare case. -jJ From ncoghlan at gmail.com Fri May 15 16:33:20 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 16 May 2009 00:33:20 +1000 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <200905152211.34793.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> Message-ID: <4A0D7D30.7070706@gmail.com> Steven D'Aprano wrote: > On Fri, 15 May 2009 08:26:30 pm Nick Coghlan wrote: >> Steven D'Aprano wrote: >>> Some years ago, Python overloaded the binary operators * and ** for >>> use as special markers in parameter lists. I suggest we could do >>> the same, by overloading the & operator in a similar fashion: >>> inside the parameter list, &x would mean to delay evaluation of the >>> default argument: > [...] >> Marking the parameter name strikes me as wrong anyway - it's only the >> evaluation of the default argument which is special, not the >> parameter itself. > > But it is the parameter that is special. The default object itself is > not. It's not the object that is being marked as special: it's the expression to create the object. The new syntax is about delaying evaluation of that expression - the parameter itself is perfectly normal, as is the object that is ultimately bound to it. But moving the default argument evaluation to call time instead of definition time - that's special. It may be worth using something like "make_default()" in examples instead of "[]" and see if that makes my point any clearer. I doubt it's possible to come up with a concise syntax for this (particularly one that plays well with function annotations), but best of luck in the search :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From jimjjewett at gmail.com Fri May 15 16:49:42 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 15 May 2009 10:49:42 -0400 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <4A0D7D30.7070706@gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> <4A0D7D30.7070706@gmail.com> Message-ID: On 5/15/09, Nick Coghlan wrote: > It's not the object that is being marked as special: it's the expression > to create the object. The new syntax is about delaying evaluation of > that expression - the parameter itself is perfectly normal, as is the > object that is ultimately bound to it. But moving the default argument > evaluation to call time instead of definition time - that's special. > It may be worth using something like "make_default()" in examples > instead of "[]" and see if that makes my point any clearer. > I doubt it's possible to come up with a concise syntax for this > (particularly one that plays well with function annotations), but best > of luck in the search :) I think this is inching towards the variable-defining keywords, like once, final, static ... This one might be "late" or "lazy", though "new" would work if you kept it only to function definitions. def f( a=make_default_once(), lazy b=make_default_each_time()): ... VAR_IN_MODULE_API = lazy whoops_wanted_a_property_after_all() -jJ From denis.spir at free.fr Fri May 15 17:23:50 2009 From: denis.spir at free.fr (spir) Date: Fri, 15 May 2009 17:23:50 +0200 Subject: [Python-ideas] func attribute & default arg In-Reply-To: References: <20090514132339.32010d65@o> Message-ID: <20090515172350.53ec2691@o> Le Thu, 14 May 2009 17:33:36 -0400, Terry Reedy s'exprima ainsi: > spir wrote: > > > To sum up, I think the core of the issue is not about static/dymamic > > evaluation, or compile time/run time; rather it lies in the fact that > > python internally does not separate a default *value* from the > > corresponding local *variable*. > > Good try, but you seem to have missed the basic point that underlies at > several ways that newbies get tripped up*. Python is an *object* based > language, not a *value* based language. By 'value', I mean the > information carried by the object. Objects have an identity, values do > not. This is a defining characteristic of Python. If one wants an > immutable value language, Python is not the right choice. > > For immutable objects, the difference is nearly invisible. For mutable > objects, which generally do not exist in mathematics (which is generally > timeless) the difference is crucial to understand. When people are > tripped up by their 'intuition', it is sometimes intuition based on math > that does not apply to mutable objects. Names are bound to objects, not > to values. Thank you for this clear reply. Actually, I 101% agree with the object/value discrimination (and even think there should be both in a prog. language, based on the fact that objects have an id and consequently are referenced.) > For functions, parameters are local names, arguments are objects (and > not values). Functions may have default objects, not 'default values' > (as you say above and elsewhere). If a default object is mutable, it > has an initial value, but not a constant value. Right. But this does not imply that a local variable must point to the same object that deals as default. This is actually the point I tried to make; but obviously not clear enough. It simply requires that while evaluating a function definition python caches defaults in a separate "storage place" that will not be later affected by the use of local variables. E.g. updating a 'number_list' local name binding should not update a default value/object for the corresponding parameter, if any. > Function objects > already have an attribute that is a tuple of default objects. > (Parameter names, without and with defaults, are part of the code object.) > > Python 3.0.1 ,,, on win32 > >>> def f(a, b=1, c='1'): pass > > >>> f.__defaults__ > (1, '1') Didn't know about that, but it makes a change even easier. (Unfortunately, I haven't py3 installed to play with this.) So, let's change the example to: def storeNums(num, numList=[1,2,3]): numList.append(num) print "list of nums: %s" % numList The issue is obviously that, at call time, python will bind the func local name representing the *actual* parameter numList to the default object (*) defined together with and for the *formal* parameter of the same name numList. So that not only it may be changed, but this changed object will stay as default object and propagate to later calls. The evident solution is to let python get the "data" (to avoid "value") instead of refering to the object -- which implies copying. End of issue? Denis (*) I use your term and agree it's better when talking of python, but actually it seems that most say "default value", no? Also, value is widely used in the python community, first in official docs, i guess. > Terry Jan Reedy From george.sakkis at gmail.com Fri May 15 17:34:54 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 15 May 2009 11:34:54 -0400 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> Message-ID: <91ad5bf80905150834q350bdafeq98337d7f1b964a42@mail.gmail.com> On Fri, May 15, 2009 at 10:14 AM, Jim Jewett wrote: > Logically, you're correct. ?But I think the ('&' ==> addressof) meme > may have already grown too strong. ?What it suggests to me is that > normally you *would* create a new list, but the ampersand says not to > in just this rare case. Well the connotations are not much stronger than with '*' and '**'. I've been literally asked by an experienced C/C++/Perl guy "what's this pointer to a pointer parameter used for in this function ?". How about '@' instead ? A mnemonic here could be "just like '@decorator\ndef f():' is a shortcut for 'f = decorator(f)', the '@arg=expr' parameter is a shortcut for 'arg = (lambda: expr)()' if 'arg' is not passed". Admittedly, far from a perfect analogy but probably less controversial than '&'. George From denis.spir at free.fr Fri May 15 17:51:31 2009 From: denis.spir at free.fr (spir) Date: Fri, 15 May 2009 17:51:31 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905151338.44654.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> Message-ID: <20090515175131.15e25923@o> Le Fri, 15 May 2009 13:38:43 +1000, Steven D'Aprano s'exprima ainsi: > Just out of curiosity, what do you see as the drawbacks? The ones that > come to my mind are: > [...] > * requires people to learn one more feature > (so newbies will still be confused that def f(x=[]) doesn't behave as > they expect). > That's the relevant drawback for me. A solution that does not solve the issue. A new syntactic pattern to allow call time evaluation of defaults is a (costly) solution for people who don't need it. Denis ------ la vita e estrany From curt at hagenlocher.org Fri May 15 17:52:44 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Fri, 15 May 2009 08:52:44 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> Message-ID: On Fri, May 15, 2009 at 5:59 AM, George Sakkis wrote: > > More to the point, immutability is *not* the issue as Steven D'Aprano > showed. There are perfectly legitimate reasons for using a default > value that just happens to be mutable, without mutating it in the > function body though. Dict is the most common example (especially > since there is no frozendict type that could be used in its place). There seem to be two separate "wants" that relate to this topic: 1. Preventing the "noob" mistake of saying "def f(x = {})" and expecting that a new empty dictionary will be produced for each call, and 2. Creating a more concise syntax for saying def f(x = UNDEF): if x is UNDEF: x = {} So far, the discussion seems to have revolved entirely around the second request -- which I find by far less compelling than the first; it's simply not a painful-enough pattern to warrant a special bit of syntax. Furthermore, it doesn't do anything to address the first desire. -- Curt Hagenlocher curt at hagenlocher.org From denis.spir at free.fr Fri May 15 18:05:39 2009 From: denis.spir at free.fr (spir) Date: Fri, 15 May 2009 18:05:39 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <87d4aag18v.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142313o75fb7324sa0ceadbfe36e1aa7@mail.gmail.com> <87d4aag18v.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20090515180539.1c67af36@o> Le Fri, 15 May 2009 18:32:32 +0900, "Stephen J. Turnbull" s'exprima ainsi: > Chris Rebert writes: > > > +1 on throwing a ValueError for non-hash()-able (and thus probably > > mutable) default argument values. It's by no means perfect since > > objects are hash()-able by default using their ID, but it would at > > least help in the frequent "well-behaved mutable container object" > > cases. > > -1 > > nonet_default_options = { 'dryrun': False, 'verbose': False } > net_default_options = { 'dryrun': True, 'verbose': True } > > def command1(options=nonet_default_options): > pass > > def command2(options=net_default_options): > pass > > def command3(options=net_default_options): > pass Yop. The main issue is not that a default be mutable -- and the example above is perfect to show this -- but that it is changed in the func body, this change affects the default back and propagates to later calls. /This/ could be used as criteria for a warning (or error), but probably it's much more costly to detect (parameter name on left side of "="). Denis ------ la vita e estrany From denis.spir at free.fr Fri May 15 19:26:34 2009 From: denis.spir at free.fr (spir) Date: Fri, 15 May 2009 19:26:34 +0200 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <4A0D7D30.7070706@gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> <4A0D7D30.7070706@gmail.com> Message-ID: <20090515192634.1dc0963f@o> Le Sat, 16 May 2009 00:33:20 +1000, Nick Coghlan s'exprima ainsi: > It's not the object that is being marked as special: it's the expression > to create the object. The new syntax is about delaying evaluation of > that expression - the parameter itself is perfectly normal, as is the > object that is ultimately bound to it. But moving the default argument > evaluation to call time instead of definition time - that's special. I rather agree. Then we should mark the binding sign '=' as special (not the parameter / the object)! E.g. def f(arg &= []) or def f(arg @= []) I looks a bit strange, but we have augmented assignment already. Denis ------ la vita e estrany From debatem1 at gmail.com Fri May 15 20:16:48 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 15 May 2009 11:16:48 -0700 (PDT) Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <20090515192634.1dc0963f@o> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> <4A0D7D30.7070706@gmail.com> <20090515192634.1dc0963f@o> Message-ID: On May 15, 1:26?pm, spir wrote: > Le Sat, 16 May 2009 00:33:20 +1000, > Nick Coghlan s'exprima ainsi: > > > It's not the object that is being marked as special: it's the expression > > to create the object. The new syntax is about delaying evaluation of > > that expression - the parameter itself is perfectly normal, as is the > > object that is ultimately bound to it. But moving the default argument > > evaluation to call time instead of definition time - that's special. > > I rather agree. Then we should mark the binding sign '=' as special (not the parameter / the object)! E.g. > > ? ? def f(arg &= []) > > or > > ? ? def f(arg @= []) > > I looks a bit strange, but we have augmented assignment already. > > Denis > ------ > la vita e estrany Maybe ::? >>> def f(x::a**2+2*b+c): ... return x Also, is there any reason why this has to be specific to function signatures at this point? Geremy Condra From debatem1 at gmail.com Fri May 15 20:20:08 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 15 May 2009 11:20:08 -0700 (PDT) Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <20090515192634.1dc0963f@o> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> <4A0D7D30.7070706@gmail.com> <20090515192634.1dc0963f@o> Message-ID: <16e6b6c3-dacf-4093-a979-80182867094b@o30g2000vbc.googlegroups.com> On May 15, 1:26?pm, spir wrote: > Le Sat, 16 May 2009 00:33:20 +1000, > Nick Coghlan s'exprima ainsi: > > > It's not the object that is being marked as special: it's the expression > > to create the object. The new syntax is about delaying evaluation of > > that expression - the parameter itself is perfectly normal, as is the > > object that is ultimately bound to it. But moving the default argument > > evaluation to call time instead of definition time - that's special. > > I rather agree. Then we should mark the binding sign '=' as special (not the parameter / the object)! E.g. > > ? ? def f(arg &= []) > > or > > ? ? def f(arg @= []) > > I looks a bit strange, but we have augmented assignment already. > > Denis Maybe :: ? >>> def f(x::a**2+2*b+c): ... return x ... >>> a, b, c = 0, 1, 2 >>> f() 4 Geremy Condra From tjreedy at udel.edu Fri May 15 21:33:13 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 15 May 2009 15:33:13 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> Message-ID: Curt Hagenlocher wrote: > There seem to be two separate "wants" that relate to this topic: Good observation. > 1. Preventing the "noob" mistake of saying "def f(x = {})" and > expecting that a new empty dictionary will be produced for each call, > and As a couple of us have suggested, this, like similar jobs, should be handled by program checkers. I leave it to someone else to see if existing programs already check and warn and, if not, suggest this to their authors. > 2. Creating a more concise syntax for saying > def f(x = UNDEF): > if x is UNDEF: > x = {} > > So far, the discussion seems to have revolved entirely around the > second request -- which I find by far less compelling than the first; > it's simply not a painful-enough pattern to warrant a special bit of > syntax. Furthermore, it doesn't do anything to address the first > desire. The existing pattern explicitly says what one wants done. I suspect editors with a macro facility could be given a macro to do most of the boilerplate writing. tjr From phd at phd.pp.ru Fri May 15 21:50:15 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 15 May 2009 23:50:15 +0400 Subject: [Python-ideas] Default arguments in Python In-Reply-To: References: <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> Message-ID: <20090515195015.GB1755@phd.pp.ru> On Fri, May 15, 2009 at 03:33:13PM -0400, Terry Reedy wrote: > Curt Hagenlocher wrote: >> 1. Preventing the "noob" mistake of saying "def f(x = {})" and >> expecting that a new empty dictionary will be produced for each call, >> and > > As a couple of us have suggested, this, like similar jobs, should be > handled by program checkers. I leave it to someone else to see if > existing programs already check and warn PyLint certainly does. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tleeuwenburg at gmail.com Sat May 16 00:03:34 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Sat, 16 May 2009 08:03:34 +1000 Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <16e6b6c3-dacf-4093-a979-80182867094b@o30g2000vbc.googlegroups.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> <4A0D7D30.7070706@gmail.com> <20090515192634.1dc0963f@o> <16e6b6c3-dacf-4093-a979-80182867094b@o30g2000vbc.googlegroups.com> Message-ID: <43c8685c0905151503q40a04f74pc094890c13176e03@mail.gmail.com> How about changing the '=' sign, i.e. def (foo <= []): #stuff i.e. instead of foo 'equals' [], foo 'gets' an [] -T -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Sat May 16 01:36:13 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 15 May 2009 16:36:13 -0700 (PDT) Subject: [Python-ideas] Syntax for late-binding of arguments [was: Default arguments ...] In-Reply-To: <43c8685c0905151503q40a04f74pc094890c13176e03@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <200905151844.55099.steve@pearwood.info> <4A0D4356.3070400@gmail.com> <200905152211.34793.steve@pearwood.info> <4A0D7D30.7070706@gmail.com> <20090515192634.1dc0963f@o> <16e6b6c3-dacf-4093-a979-80182867094b@o30g2000vbc.googlegroups.com> <43c8685c0905151503q40a04f74pc094890c13176e03@mail.gmail.com> Message-ID: <028dd0cd-e569-4e4b-b4b5-133dc6d9a540@n4g2000vba.googlegroups.com> On May 15, 6:03?pm, Tennessee Leeuwenburg wrote: > How about changing the '=' sign, i.e. > def (foo <= []): > ? #stuff > > i.e. instead of foo 'equals' [], foo 'gets' an [] > > -T I see :: as having the advantage of not being used for anything currently, as well as being vaguely reminiscent of :=, which wouldn't be too bad either. It would also leave the door open to adding thunks to the language as a whole rather than just here, and assuming (warning! possibly invalid conclusion ahead!) that this would require thunks to work, that seems like a good idea to me. Geremy Condra From steve at pearwood.info Sat May 16 04:21:13 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 16 May 2009 12:21:13 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <20090515175131.15e25923@o> References: <4A0496BD.3030709@wanadoo.fr> <200905151338.44654.steve@pearwood.info> <20090515175131.15e25923@o> Message-ID: <200905161221.14213.steve@pearwood.info> On Sat, 16 May 2009 01:51:31 am spir wrote: > > * requires people to learn one more feature > > (so newbies will still be confused that def f(x=[]) doesn't behave > > as they expect). > > That's the relevant drawback for me. > A solution that does not solve the issue. A new syntactic pattern to > allow call time evaluation of defaults is a (costly) solution for > people who don't need it. There is no solution to the problem of newbies' confusion. The standard behaviour will remain in Python 2.x and almost certainly Python 3.x. The earliest it could change is Python 3.3: it could be introduced with a "from __future__ import defaults" in 3.2 and become standard in 3.3. (It almost certainly will never be the standard behaviour, but if it did, that would be the earliest it could happen.) And even if it did change, then newbies will be surprised and upset that def f(x=y) doesn't behave as they expect. Here's the current behaviour: >>> y = result_of_some_complex_calculation() # => 11 >>> def f(x=y): ... return x+1 ... >>> f() 12 >>> y = 45 >>> f() 12 Given the proposed behaviour, that second call to f() would surprisingly return 46, or worse, raise a NameError if y is no longer in scope. The real problem is that people don't have a consistent expectation for default arguments. No matter what behaviour Python uses, people will be caught out by it sometimes. -- Steven D'Aprano From steve at pearwood.info Sat May 16 04:34:11 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 16 May 2009 12:34:11 +1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> Message-ID: <200905161234.11580.steve@pearwood.info> On Sat, 16 May 2009 01:52:44 am Curt Hagenlocher wrote: > There seem to be two separate "wants" that relate to this topic: > > 1. Preventing the "noob" mistake of saying "def f(x = {})" and > expecting that a new empty dictionary will be produced for each call, > and > 2. Creating a more concise syntax for saying > def f(x = UNDEF): > if x is UNDEF: > x = {} > > So far, the discussion seems to have revolved entirely around the > second request -- which I find by far less compelling than the first; > it's simply not a painful-enough pattern to warrant a special bit of > syntax. Furthermore, it doesn't do anything to address the first > desire. Agreed. Sort of. I don't believe that we can do anything about #1. Changing the current behaviour of default arguments will almost certainly not happen -- I think Guido has ruled No on that one, although I might be mistaken. But even if it was changed, it would just lead to a different set of newbie mistakes. I personally don't find the boilerplate code in #2 particularly onerous, but it *is* boilerplate, and as a general rule, boilerplate is a bad thing. If we could agree on a syntax, then we could move that boilerplate out of our code (where it marginally complicates the structure of the function), into the bytecode, which would be a small but valuable win for readability. There's also a #3: one possible solution to this is to use thunks. Do thunks have uses outside of default arguments? I imagine they do -- Algol used them extensively. What else can thunks be used for? -- Steven D'Aprano From ncoghlan at gmail.com Sat May 16 09:53:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 16 May 2009 17:53:09 +1000 Subject: [Python-ideas] func attribute & default arg In-Reply-To: <20090515172350.53ec2691@o> References: <20090514132339.32010d65@o> <20090515172350.53ec2691@o> Message-ID: <4A0E70E5.6030409@gmail.com> spir wrote: > Right. But this does not imply that a local variable must point to > the same object that deals as default. This is actually the point I > tried to make; but obviously not clear enough. It simply requires > that while evaluating a function definition python caches defaults in > a separate "storage place" that will not be later affected by the use > of local variables. E.g. updating a 'number_list' local name binding > should not update a default value/object for the corresponding > parameter, if any. There are only two ways to make that happen though: 1. Evaluate the default argument expression at function call time instead of definition time 2. Actually *copy* the default argument to create the object referenced by the parameter on a given invocation Since option 1 will work for any arbitrary object, while option 2 is limited to objects which support copy.copy (or copy.deepcopy, depending on the copying semantics chosen), the "late evaluation of default arguments" (option 1) approach is the more promising angle of attack by far. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From denis.spir at free.fr Sat May 16 14:05:25 2009 From: denis.spir at free.fr (spir) Date: Sat, 16 May 2009 14:05:25 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <200905161221.14213.steve@pearwood.info> References: <4A0496BD.3030709@wanadoo.fr> <200905151338.44654.steve@pearwood.info> <20090515175131.15e25923@o> <200905161221.14213.steve@pearwood.info> Message-ID: <20090516140525.246c85cb@o> Le Sat, 16 May 2009 12:21:13 +1000, Steven D'Aprano s'exprima ainsi: > On Sat, 16 May 2009 01:51:31 am spir wrote: > > > * requires people to learn one more feature > > > (so newbies will still be confused that def f(x=[]) doesn't behave > > > as they expect). > > > > That's the relevant drawback for me. > > A solution that does not solve the issue. A new syntactic pattern to > > allow call time evaluation of defaults is a (costly) solution for > > people who don't need it. Let me expand on 'costly'. You (nicely) totally rejected a previous proposal of mine, but it adressed the issues pointed here (propably not clearly enough, though). I stated that defaults are part of a func def, and should be cached when the definition is evaluated, in a way that they are separate from local vars. This means that a local var should not point to the same object as the one cached. I did not enter implementation stuff, but this obviously requires, I guess, that defaults are (deep?)copied into locals at call time, when the object is mutable. Pseudo code for def f(arg=whatever) # at definition time f.__defaults__["arg"] = whatever # at call time if if arg = f.__defaults__["arg"] else: arg = copy(f.__defaults__["arg"]) The advantage is that if ever "whatever" is a complex expression, it will not be re-evaluated on each call. Unlike with the late-binding proposal. As I see it, re-evaluating 'whatever' at call time does not serve any purpose -- except possibly that the result may change at runtime intentionally, which is another topic. Actually, a default may be changed from inside (a mutable object updated through local var in the func's own body) or from outside (when the expression holds variable items). In the latter case, this may be intentional to get a kind of runtime-changing default value. See also below. > There is no solution to the problem of newbies' confusion. The standard > behaviour will remain in Python 2.x and almost certainly Python 3.x. > The earliest it could change is Python 3.3: it could be introduced with > a "from __future__ import defaults" in 3.2 and become standard in 3.3. > > (It almost certainly will never be the standard behaviour, but if it > did, that would be the earliest it could happen.) I agree with that, it well certainly never change; I would never have brought this topic back again myself. (It's such an obvious issue that I was sure it had been hundred times discussed since the late eighties ;-) Bit it seems to come back regularly anyway. > And even if it did change, then newbies will be surprised and upset > that def f(x=y) doesn't behave as they expect. Here's the current > behaviour: > > >>> y = result_of_some_complex_calculation() # => 11 > >>> def f(x=y): > ... return x+1 > ... > >>> f() > 12 > >>> y = 45 > >>> f() > 12 > > Given the proposed behaviour, that second call to f() would surprisingly > return 46, or worse, raise a NameError if y is no longer in scope. Yes, "surprisingly". That's the reason why I stated earlier that when a default in *intended* to change at runtime, it is worth making it clear with explicit code and even comments. My previous proposal was precisely to make the default fixed, so that this (silent) behaviour disappears. In the case of an intentional runtime-changing default, it is thus a Good Thing to use a sentinel -- and with my proposal we would have to do it because of defaults beeing cached at definition time. so to have the above behaviour, one would need to write: def f(x=SENTINEL): # y will change at runtime if x is SENTINEL: x = y return x+1 For an example: def writeIndent(indent_level, indent_token=SENTINEL): # 'indent_token' may be provided by the caller # to conform to the source beeing edited. # Else read it from current user config. if indent_token is SENTINEL: indent_token = config.indent_token ....... > The real problem is that people don't have a consistent expectation for > default arguments. No matter what behaviour Python uses, people will be > caught out by it sometimes. I rather think that an important fraction of experienced python programmers have expectations dictated by the current semantics they're used to. (Which is indeed correct. This a case of "intuitive = familiar".) But expectations from so to say "ordinary" people unaware of the said semantics are clearly different. The fact that people are bitten (when default is changed from inside the func), or merely surprised (your case above with default changed from outside) rather shows what they expect the func def to mean (both when writing and reading it). Denis ------ la vita e estrany From mwm-keyword-python.b4bdba at mired.org Sat May 16 20:32:41 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Sat, 16 May 2009 14:32:41 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <20090516140525.246c85cb@o> References: <4A0496BD.3030709@wanadoo.fr> <200905151338.44654.steve@pearwood.info> <20090515175131.15e25923@o> <200905161221.14213.steve@pearwood.info> <20090516140525.246c85cb@o> Message-ID: <20090516143241.4003028a@bhuda.mired.org> On Sat, 16 May 2009 14:05:25 +0200 spir wrote: > Le Sat, 16 May 2009 12:21:13 +1000, > Steven D'Aprano s'exprima ainsi: > > > On Sat, 16 May 2009 01:51:31 am spir wrote: > > > > * requires people to learn one more feature > > > > (so newbies will still be confused that def f(x=[]) doesn't behave > > > > as they expect). > > > > > > That's the relevant drawback for me. > > > A solution that does not solve the issue. A new syntactic pattern to > > > allow call time evaluation of defaults is a (costly) solution for > > > people who don't need it. > > Let me expand on 'costly'. By "expand", you mean make things even more costly? > I stated that defaults are part of a func def, and should be cached when the definition is evaluated, in a way that they are separate from local vars. This means that a local var should not point to the same object as the one cached. > I did not enter implementation stuff, but this obviously requires, I guess, that defaults are (deep?)copied into locals at call time, when the object is mutable. Pseudo code for > > def f(arg=whatever) > > # at definition time > f.__defaults__["arg"] = whatever > # at call time > if > if > arg = f.__defaults__["arg"] > else: > arg = copy(f.__defaults__["arg"]) > > The advantage is that if ever "whatever" is a complex expression, it will not be re-evaluated on each call. Unlike with the late-binding proposal. Right. It'll be *copied*. So consider: x = 1000000 * [[]] def f(y=x): y[18][0] = None y[23] = None y[0][1] = None So instead of simply creating a new reference to the object (which you get with either the current semantics or the re-evaluate at call time semantics), you now copy a list with a million elements on every call. For this case, the current semantics are the only one that works well: you can put the expression into the call list, and don't need either an extra variable to avoid rebuilding your long list on every call (if you re-evaluate the argument) or using a sentinel and assignment from that extra variable to avoid copying it if you copy the values. > As I see it, re-evaluating 'whatever' at call time does not serve any purpose -- except possibly that the result may change at runtime intentionally, which is another topic. The problem is *not* with the behavior of default values to arguments. The problem is with the behavior of multiple references to mutable objects. People who aren't used to object/reference semantics don't understand them (of course, they don't understand multiple references to immutable objects either, but that's a different problem). They will be confused when they body of f above throws an exception when you don't pass it y - no matter *what* the calling semantics! Admittedly, default values for arguments are nastier than other because people don't see it as multiple references to one object until it's pointed out. Both copying and reevaluation change that. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From ncoghlan at gmail.com Sun May 17 02:32:32 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 17 May 2009 10:32:32 +1000 Subject: [Python-ideas] func attribute & default arg In-Reply-To: <20090516141522.2894eec9@o> References: <20090514132339.32010d65@o> <20090515172350.53ec2691@o> <4A0E70E5.6030409@gmail.com> <20090516141522.2894eec9@o> Message-ID: <4A0F5B20.3000901@gmail.com> spir wrote: > Le Sat, 16 May 2009 17:53:09 +1000, Nick Coghlan > s'exprima ainsi: > >> Since option 1 will work for any arbitrary object, while option 2 >> is limited to objects which support copy.copy (or copy.deepcopy, >> depending on the copying semantics chosen), the "late evaluation of >> default arguments" (option 1) approach is the more promising angle >> of attack by far. > > Which objects do not support copy.deepcopy? Really surprised. I used > and still use it in personal projects for rather complicated custom > objects* and it works as expected. Bound methods for one: >>> class C(object): ... def m(): pass ... >>> copy.copy(C().m) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.5/copy.py", line 95, in copy return _reconstruct(x, rv, 0) File "/usr/lib/python2.5/copy.py", line 322, in _reconstruct y = callable(*args) File "/usr/lib/python2.5/copy_reg.py", line 92, in __newobj__ return cls.__new__(cls, *args) TypeError: instancemethod expected at least 2 arguments, got 0 >>> copy.deepcopy(C().m) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.5/copy.py", line 189, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.5/copy.py", line 322, in _reconstruct y = callable(*args) File "/usr/lib/python2.5/copy_reg.py", line 92, in __newobj__ return cls.__new__(cls, *args) TypeError: instancemethod expected at least 2 arguments, got 0 Copyability really can't be assumed for Python objects - the copy module does an excellent job of making it "just work" for most objects, but it really isn't any more fundamental a property than being hashable. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ziade.tarek at gmail.com Sun May 17 16:52:43 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 17 May 2009 16:52:43 +0200 Subject: [Python-ideas] Add a "archive_tree" in the tarfile module Message-ID: <94bdd2610905170752x5b984f67kbfcd0b580e30925d@mail.gmail.com> Hello, It happened to me several time : re-write a small function to archive a directory, to mimic "tar -xzvf archive.tgz directory" What about adding in tarfile this small function: def archive_tree(archive_name, src_dir?[[, add_empty_dir], callable]) -> creates an archive that contains the files in directory, by recursively walking into it. if add_empty_dir is set to False (default: True), empty directory founded will not be added if callable is provided, it will be triggered everytime a file is added in the archive and will return True or False If False, the file will not be added regards Tarek -- Tarek Ziad? | http://ziade.org From aahz at pythoncraft.com Sun May 17 16:57:32 2009 From: aahz at pythoncraft.com (Aahz) Date: Sun, 17 May 2009 07:57:32 -0700 Subject: [Python-ideas] Add a "archive_tree" in the tarfile module In-Reply-To: <94bdd2610905170752x5b984f67kbfcd0b580e30925d@mail.gmail.com> References: <94bdd2610905170752x5b984f67kbfcd0b580e30925d@mail.gmail.com> Message-ID: <20090517145732.GA14730@panix.com> On Sun, May 17, 2009, Tarek Ziad? wrote: > > It happened to me several time : re-write a small function to archive > a directory, to mimic "tar -xzvf archive.tgz directory" Given that you already have this code, please upload to bugs.python.org (possibly also PyPI and/or Cookbook) so it doesn't get lost. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "In 1968 it took the computing power of 2 C-64's to fly a rocket to the moon. Now, in 1998 it takes the Power of a Pentium 200 to run Microsoft Windows 98. Something must have gone wrong." --/bin/fortune From grosser.meister.morti at gmx.net Sun May 17 21:27:06 2009 From: grosser.meister.morti at gmx.net (=?ISO-8859-1?Q?Mathias_Panzenb=F6ck?=) Date: Sun, 17 May 2009 21:27:06 +0200 Subject: [Python-ideas] Add a "archive_tree" in the tarfile module In-Reply-To: <94bdd2610905170752x5b984f67kbfcd0b580e30925d@mail.gmail.com> References: <94bdd2610905170752x5b984f67kbfcd0b580e30925d@mail.gmail.com> Message-ID: <4A10650A.5020602@gmx.net> Tarek Ziad? wrote: > Hello, > > It happened to me several time : re-write a small function to archive > a directory, to mimic "tar -xzvf archive.tgz directory" > > What about adding in tarfile this small function: > > def archive_tree(archive_name, src_dir [[, add_empty_dir], > callable]) -> creates an archive that contains the files in directory, > by recursively walking into it. > > > if add_empty_dir is set to False (default: True), > empty directory founded will not be added > > if callable is provided, it will be triggered > everytime a file is added in the archive and will return True or False > > If False, the file will not be added > > > > regards > Tarek > I would call the callback-function "filter" or similar. -panzi From stephen at xemacs.org Mon May 18 05:58:16 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 18 May 2009 12:58:16 +0900 Subject: [Python-ideas] Add a "archive_tree" in the tarfile module In-Reply-To: <94bdd2610905170752x5b984f67kbfcd0b580e30925d@mail.gmail.com> References: <94bdd2610905170752x5b984f67kbfcd0b580e30925d@mail.gmail.com> Message-ID: <87d4a7dpuv.fsf@uwakimon.sk.tsukuba.ac.jp> Tarek Ziad? writes: > if add_empty_dir is set to False (default: True), > empty directory founded will not be added I strongly prefer Booleans to default to "False". How about naming the formal argument "omit_empty_dir" or "ignore_empty_dir", and defaulting to false? From chambon.pascal at wanadoo.fr Mon May 18 22:53:11 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Mon, 18 May 2009 22:53:11 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: References: <4A0496BD.3030709@wanadoo.fr> <200905151015.34186.steve@pearwood.info> <0ee5a9ef-79d1-458e-9012-3752bc420e27@f16g2000vbf.googlegroups.com> <200905151338.44654.steve@pearwood.info> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> Message-ID: <4A11CAB7.60600@wanadoo.fr> Terry Reedy a ?crit : > > Curt Hagenlocher wrote: > > >> 1. Preventing the "noob" mistake of saying "def f(x = {})" and >> expecting that a new empty dictionary will be produced for each call, >> and > > As a couple of us have suggested, this, like similar jobs, should be > handled by program checkers. I leave it to someone else to see if > existing programs already check and warn and, if not, suggest this to > their authors. > Indeed, Pylint does handle this in some way : "W0102: *Dangerous default value %s as argument* Used when a mutable value as list or dictionary is detected in a default value for an argument." Some have talked about thunks or other concepts here ; I too feel that this issue could be the occasion of introducing new features, that exceed the scope of default arguments. Could you peopel develop a little what you think about with "thunk" or "early action" ? Is the former different from an argument-less lambda function ? Else, concerning the syntax for "dynamic" default arguments, I guesse somethng like : def func(a, b @= []): pass would be OK, wouldn't it ? Regards, Pascal From pyideas at rebertia.com Mon May 18 23:17:55 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Mon, 18 May 2009 14:17:55 -0700 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A11CAB7.60600@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> Message-ID: <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> On Mon, May 18, 2009 at 1:53 PM, Pascal Chambon wrote: > Else, concerning the syntax for "dynamic" default arguments, I guesse > somethng like : > > def func(a, b @= []): > ? ? ? ?pass > > would be OK, wouldn't it ? The BDFL has condemned introducing new assignment operators. See http://www.python.org/dev/peps/pep-3099/ : "There will be no alternative binding operators such as :=." Cheers, Chris -- http://blog.rebertia.com From debatem1 at gmail.com Tue May 19 00:00:58 2009 From: debatem1 at gmail.com (CTO) Date: Mon, 18 May 2009 15:00:58 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> Message-ID: <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> On May 18, 5:17?pm, Chris Rebert wrote: > On Mon, May 18, 2009 at 1:53 PM, Pascal Chambon wrote: > > > > > Else, concerning the syntax for "dynamic" default arguments, I guesse > > somethng like : > > > def func(a, b @= []): > > ? ? ? ?pass > > > would be OK, wouldn't it ? > > The BDFL has condemned introducing new assignment operators. Seehttp://www.python.org/dev/peps/pep-3099/: > > "There will be no alternative binding operators such as :=." > > Cheers, > Chris To be fair, the last discussion on thunks turned into a discussion of metalanguages and macros, which were also pretty thoroughly canned on that list. Geremy Condra From aaron.rubin at 4dtechnology.com Tue May 19 18:43:40 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Tue, 19 May 2009 09:43:40 -0700 Subject: [Python-ideas] 80 character line width vs. something wider Message-ID: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> I realize that this is a religious debate which has gone on for many centuries. I appeal to the scientific aspects, with a distinct avoidance of preference and emotion. Preference might be easily explained by "right brain" vs "left brain" preference, but either way, it is merely a preference and I want to stick to facts. Here is a list I have compiled of facts which support a wider than 80 character line width standard (in Python, specifically). Please add to them, subtract from them, or add to the other side of the debate, but please avoid the usage of the word "readable" (which implies preference), unless you are referring to a scientific study of readability, which would be great. 1) Python is three things which the standard was not designed for: One: Object Oriented. Two: Not Hungarian notation Three: Mandatorily uses *whitespace* as its defintion for code blocks. Let me explain each one in a bit more detail: Object Oriented: Because it is not functional-style programming, but instead OO, you have to give defintion as to what object type you are using before using it. This makes definitions and usage longer than in functional programming (when 80 character widths were invented). PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an example (and not an extreme one) of a class (55 characters already) in a rather large code base. Not Hungarian: Not only is Python not Hungarian (in general), but the PEP-8 specifically tells us to use longer, more descriptive variable names. hasInstrumentControllerPhaseDither is an example. Many variables are 15-20 characters and oftentimes longer. How many of these variables can you fit into a line if we are limited to 80? Whitespace: Python is very unique in that it *uses* whitespace for code blocking. It turns out to be very useful, since it visually cues the reader where code blocks begin and end by mandate. This creates many situations where code *starts* at the 10th indentation (40 characters in our standard, 80 characters in some Python standards). Even in normal "great design" mode (which takes more time again), you can't help it....your code starts at the 6th indentation level often. (28 characters, more than 30% of 80 characters already gone. Now how many variables or class names can you fit?) Whitespace (2): Because Python uses whitespace as its sole method of code blocking and because this is the visual cue for code blocks, wrapping lines obfuscates this and makes the reader think about whether this whitespace was there for a code block, or a line-wrap. Thinking about intention of code slows us down. 2) Many of the libraries that are widely used do not adhere to the 80 character width line standard. wxPython, NumPy and Boa Constructor are a few, but I'm sure there are many, many more. Many libraries do adhere to 80 character line width as well. However, a library which is written in 80 characters still fits the paradigm of those which are wider and therefore backward compliant. In other words, if your tools are geared toward 80 character line widths and you are now looking at a wider width, things become quite difficult. The other way around is fine. 3) Writing new code in 80 character line widths takes more time. If I have to worry about hitting this width, I have to de-concentrate my efforts of writing logical code and concentrate instead on how exactly to format this line of code (where to break it, etc....there are a number of rules attached to wrapping lines of code). Then I have to re-concentrate on the actual task at hand. Alternatively, I can code it up without worrying, then when convenient, take some time to reformat to 80 character width. Either way, more time. 4) Code searching. IDEs have powerful searching features. They list all the lines of a file (or files) which match the string you are searching for. If things are in one line, this search is meaningful and you can read it like you can code. If a line of code actually spans two (or more) lines of code, the search is no longer contextually useful and you have to click on each item to see what's actually going on. This feature is used heavily in many environments (especially large code bases) to save time, but time is either lost finding the actual context of a found string, or the search tool is avoided altogether because it does not provide meaningful results (i.e. a predictive waste of time) 5) Monitors are getting bigger, wider, cheaper. This allows us to have two files, side-by-side on a screen that are *both* 120 character width (or even wider), without changing font sizes. 6) Tools are cheap. Time isn't. Get a second monitor, get a more powerful editor, do whatever it takes to save time. If viewing more information at one time is important, then we should try to make that possible with technology, not time. 7) Python is designed to be written more like English than other programming languages. English is written horizontally, not vertically. In furtherance to an attempt to make "readability" an objective argument, here is a scientific study which finds that greater character width lines improve readability: http://psychology.wichita.edu/surl/usabilitynews/72/LineLength.asp. To summarize, the study found that of the choices of 35, 55, 75 and 95 character lengths, 95 was able to be read the fastest. Please note that they did not try 115, 135, etc. and that they found their maximum data point at the farthest edge of their study. One can conclude that they probably should have gone even further to determine where (if ever) it tapers off. This study focuses on reading English, not on reading code. But given the first sentence of this point, it should at least loosely correlate. At any rate, it's an attempt at something scientific on the issue. Thanks for the time spent reading this long-ish post. Thanks for your feedback if you provide it. - Aaron Aaron Rubin Software Engineering Manager 4D Technology -------------- next part -------------- An HTML attachment was scrubbed... URL: From dstanek at dstanek.com Tue May 19 19:14:15 2009 From: dstanek at dstanek.com (David Stanek) Date: Tue, 19 May 2009 13:14:15 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: I'll bite. On Tue, May 19, 2009 at 12:43 PM, Aaron Rubin wrote: > 1)?Python is three things which the standard was not designed for: ?One: > Object Oriented. ?Two: Not Hungarian notation ?Three: Mandatorily uses > *whitespace* as its defintion for code blocks. ?Let me explain each one in a > bit more detail: > ??Object Oriented: ?Because it is not functional-style programming, but > instead OO, you have to give defintion as to what object type you are using > before using it. ?This makes definitions and usage longer than in functional > programming (when?80?character?widths were invented). > ?PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an example (and > not an extreme one) of a class (55 characters already) in a rather large > code base. If you are using more than 5 or 6 levels of indentation you may be doing something wrong. I would guess that your methods are too complex or maybe you are violating the SRP. > ??Not Hungarian: ?Not only is Python not Hungarian (in general), but the > PEP-8 specifically?tells us to use longer, more descriptive variable names. > ?hasInstrumentControllerPhaseDither is an example. ?Many variables are 15-20 > characters and oftentimes longer. ?How many of these variables can you fit > into a?line?if we are limited to?80? I'd like to see an example of your variable names. I don't use hungarian notation and my name are usually under 10 characters. > ??Whitespace: ?Python is very unique in that it *uses* whitespace for code > blocking. ?It turns out to be very useful, since it visually cues the reader > where code blocks begin and end by mandate. ?This creates many situations > where code *starts* at the 10th indentation (40 characters in our > standard,?80?characters in some Python standards). ?Even in normal "great > design" mode (which takes more time again), you can't help it....your code > starts at the 6th indentation level often. ?(28 characters, more than 30% > of?80?characters already gone. ?Now how many variables or class names can > you fit?) I'd like to see an example of where using 10 levels of indentation is good. I'll bet that it's not easy to test. > ??Whitespace (2): ?Because Python uses whitespace as its sole method of code > blocking and because this is the visual cue for code blocks, wrapping lines > obfuscates this and makes the reader think about whether this whitespace was > there for a code block, or a line-wrap. ?Thinking about intention of code > slows us down. I partially think you're right. Although I have the same problem with long lines that have multiple levels of parens. > 2)?Many of the libraries that are widely used do not adhere to > the?80?character?width?line?standard. ?wxPython, NumPy and?Boa Constructor > are a few, but I'm sure there are many, many more. ?Many libraries do adhere > to 80 character line width as well. ?However, a library which is written in > 80 characters still fits the paradigm of those which are wider and therefore > backward compliant. ?In other words, if your tools are geared toward 80 > character line widths and you are now looking at a wider width, things > become quite difficult. ?The other way around is fine. > 3) ?Writing new code in?80?character?line?widths takes more time. ?If I have > to worry about hitting this width, I have to de-concentrate my efforts of > writing logical code and concentrate instead on how exactly to format > this?line?of code (where to break it, etc....there are a number of rules > attached to wrapping lines of code). ?Then I have to re-concentrate on the > actual task at hand. ?Alternatively, I can code it up without worrying, then > when convenient, take some time to reformat to?80?character?width. ?Either > way, more time. I don't really have this issue. The few seconds a day that I waste formatting code are nothing near the time I waste on YouTube :-) > > 4) Code searching. ?IDEs have powerful searching features. ?They list all > the lines of a file (or files) which match the string you are searching for. > ?If things are in one?line, this search is meaningful and you can read it > like you can code. ?If a?line?of code actually spans two (or more) lines of > code, the search is no longer contextually useful and you have to click on > each item to see what's actually going on. ?This feature is used heavily in > many environments (especially large code bases) to save time, but time is > either lost finding the actual context of a found string, or the search tool > is avoided altogether?because it does not provide meaningful results (i.e. a > predictive waste of time) I would actually like to see tools changed to make this better. Maybe similar to the way unified diff shows a few lines of context. > > 5) Monitors are getting bigger, wider, cheaper. ?This allows us to have two > files, side-by-side on a screen that are *both* 120?character?width (or even > wider), without changing font sizes. Sure I guess. I am typing this on my EEE 900 which won't like much more than 90 chars. But even at work I put multiple 80 char wide windows side by side. > 6) Tools are cheap. ?Time isn't. ?Get a second monitor, get a more powerful > editor, do whatever it takes to save time. ?If viewing more information at > one time is important, then we should try to make that possible with > technology, not time. Agreed. Use Vim with 80 characters and you will rock out code like never before. I hear Emacs is good too. What are you currently using. > 7) Python is designed to be written more like English than other programming > languages. That's news to me. On another note when we hire new developers I often hear this argument. Once they start coding in Python and using good OO/TDD techniques they realize that it really doesn't matter. Out of curiosity how much Python coding do you do? -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek From google at mrabarnett.plus.com Tue May 19 19:45:21 2009 From: google at mrabarnett.plus.com (MRAB) Date: Tue, 19 May 2009 18:45:21 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <4A12F031.6060303@mrabarnett.plus.com> David Stanek wrote: > I'll bite. > > On Tue, May 19, 2009 at 12:43 PM, Aaron Rubin > wrote: >> 1) Python is three things which the standard was not designed for: One: >> Object Oriented. Two: Not Hungarian notation Three: Mandatorily uses >> *whitespace* as its defintion for code blocks. Let me explain each one in a >> bit more detail: >> Object Oriented: Because it is not functional-style programming, but >> instead OO, you have to give defintion as to what object type you are using >> before using it. This makes definitions and usage longer than in functional >> programming (when 80 character widths were invented). >> PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an example (and >> not an extreme one) of a class (55 characters already) in a rather large >> code base. > > If you are using more than 5 or 6 levels of indentation you may be > doing something wrong. I would guess that your methods are too complex > or maybe you are violating the SRP. > If there are too many lines or too much indentation then it's time to move some of the code into a function. >> Not Hungarian: Not only is Python not Hungarian (in general), but the >> PEP-8 specifically tells us to use longer, more descriptive variable names. >> hasInstrumentControllerPhaseDither is an example. Many variables are 15-20 >> characters and oftentimes longer. How many of these variables can you fit >> into a line if we are limited to 80? > > I'd like to see an example of your variable names. I don't use > hungarian notation and my name are usually under 10 characters. > [snip] IMHO, the names should be long enough to be meaningful but not too long. From aaron.rubin at 4dtechnology.com Tue May 19 19:53:33 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Tue, 19 May 2009 10:53:33 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> On Tue, May 19, 2009 at 10:14 AM, David Stanek wrote: > I'll bite. > > On Tue, May 19, 2009 at 12:43 PM, Aaron Rubin > wrote: > > 1) Python is three things which the standard was not designed for: One: > > Object Oriented. Two: Not Hungarian notation Three: Mandatorily uses > > *whitespace* as its defintion for code blocks. Let me explain each one > in a > > bit more detail: > > Object Oriented: Because it is not functional-style programming, but > > instead OO, you have to give defintion as to what object type you are > using > > before using it. This makes definitions and usage longer than in > functional > > programming (when 80 character widths were invented). > > PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an example > (and > > not an extreme one) of a class (55 characters already) in a rather large > > code base. > > If you are using more than 5 or 6 levels of indentation you may be > doing something wrong. I would guess that your methods are too complex > or maybe you are violating the SRP. See below for code example > > > > Not Hungarian: Not only is Python not Hungarian (in general), but the > > PEP-8 specifically tells us to use longer, more descriptive variable > names. > > hasInstrumentControllerPhaseDither is an example. Many variables are > 15-20 > > characters and oftentimes longer. How many of these variables can you > fit > > into a line if we are limited to 80? > > I'd like to see an example of your variable names. I don't use > hungarian notation and my name are usually under 10 characters. I gave an example already in the snippet you quoted. > > > > Whitespace: Python is very unique in that it *uses* whitespace for > code > > blocking. It turns out to be very useful, since it visually cues the > reader > > where code blocks begin and end by mandate. This creates many situations > > where code *starts* at the 10th indentation (40 characters in our > > standard, 80 characters in some Python standards). Even in normal "great > > design" mode (which takes more time again), you can't help it....your > code > > starts at the 6th indentation level often. (28 characters, more than 30% > > of 80 characters already gone. Now how many variables or class names can > > you fit?) > > I'd like to see an example of where using 10 levels of indentation is > good. I'll bet that it's not easy to test. Regarding this and the other notion of 5 or 6 being the proper level of indentation: class a(object): def method1(simulation=False): try: if simulation: for x in range(10): if x>5: try: # here might begin some actual math, with two or three more levels of logic, interfacing with other libraries such as NumPy, etc. where you might need specific error handling except CustomError: # customer error handling i.e. the logic code *started* at the 7th indentation level. But I'm sure you can find plenty of examples where it might be more. > > > Whitespace (2): Because Python uses whitespace as its sole method of > code > > blocking and because this is the visual cue for code blocks, wrapping > lines > > obfuscates this and makes the reader think about whether this whitespace > was > > there for a code block, or a line-wrap. Thinking about intention of code > > slows us down. > > I partially think you're right. Although I have the same problem with > long lines that have multiple levels of parens. > > > 2) Many of the libraries that are widely used do not adhere to > > the 80 character width line standard. wxPython, NumPy and Boa > Constructor > > are a few, but I'm sure there are many, many more. Many libraries do > adhere > > to 80 character line width as well. However, a library which is written > in > > 80 characters still fits the paradigm of those which are wider and > therefore > > backward compliant. In other words, if your tools are geared toward 80 > > character line widths and you are now looking at a wider width, things > > become quite difficult. The other way around is fine. > > 3) Writing new code in 80 character line widths takes more time. If I > have > > to worry about hitting this width, I have to de-concentrate my efforts of > > writing logical code and concentrate instead on how exactly to format > > this line of code (where to break it, etc....there are a number of rules > > attached to wrapping lines of code). Then I have to re-concentrate on > the > > actual task at hand. Alternatively, I can code it up without worrying, > then > > when convenient, take some time to reformat to 80 character width. > Either > > way, more time. > > I don't really have this issue. The few seconds a day that I waste > formatting code are nothing near the time I waste on YouTube :-) or responding to people's posts about character width issues ;) > > > > > > 4) Code searching. IDEs have powerful searching features. They list all > > the lines of a file (or files) which match the string you are searching > for. > > If things are in one line, this search is meaningful and you can read it > > like you can code. If a line of code actually spans two (or more) lines > of > > code, the search is no longer contextually useful and you have to click > on > > each item to see what's actually going on. This feature is used heavily > in > > many environments (especially large code bases) to save time, but time is > > either lost finding the actual context of a found string, or the search > tool > > is avoided altogether because it does not provide meaningful results > (i.e. a > > predictive waste of time) > > I would actually like to see tools changed to make this better. Maybe > similar to the way unified diff shows a few lines of context. > > > > > 5) Monitors are getting bigger, wider, cheaper. This allows us to have > two > > files, side-by-side on a screen that are *both* 120 character width (or > even > > wider), without changing font sizes. > > Sure I guess. I am typing this on my EEE 900 which won't like much > more than 90 chars. But even at work I put multiple 80 char wide > windows side by side. > > > 6) Tools are cheap. Time isn't. Get a second monitor, get a more > powerful > > editor, do whatever it takes to save time. If viewing more information > at > > one time is important, then we should try to make that possible with > > technology, not time. > > Agreed. Use Vim with 80 characters and you will rock out code like > never before. I hear Emacs is good too. What are you currently using. Wing IDE. > > > > 7) Python is designed to be written more like English than other > programming > > languages. > > That's news to me. > > On another note when we hire new developers I often hear this > argument. Once they start coding in Python and using good OO/TDD > techniques they realize that it really doesn't matter. Out of > curiosity how much Python coding do you do? Lots and lots and lots :) Started using Python over 10 years ago. Use it all day, every day. Love it :) Using good OO to me seems to accentuate the need for greater line widths (point #1) in a large project (i.e. classes need to be hierarchical in a big project, therefore in order to disambiguate, more characters are needed to refer to this class) > > > -- > David > blog: http://www.traceback.org > twitter: http://twitter.com/dstanek > -------------- next part -------------- An HTML attachment was scrubbed... URL: From santagada at gmail.com Tue May 19 20:03:07 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Tue, 19 May 2009 15:03:07 -0300 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <241642F2-D347-4365-A6E1-AF75310C2E6A@gmail.com> On May 19, 2009, at 1:43 PM, Aaron Rubin wrote: > > 7) Python is designed to be written more like English than other > programming languages. English is written horizontally, not > vertically. In furtherance to an attempt to make "readability" an > objective argument, here is a scientific study which finds that > greater character width lines improve readability: http://psychology.wichita.edu/surl/usabilitynews/72/LineLength.asp > . > To summarize, the study found that of the choices of 35, 55, 75 and > 95 character lengths, 95 was able to be read the fastest. Please > note that they did not try 115, 135, etc. and that they found their > maximum data point at the farthest edge of their study. One can > conclude that they probably should have gone even further to > determine where (if ever) it tapers off. This study focuses on > reading English, not on reading code. But given the first sentence > of this point, it should at least loosely correlate. At any rate, > it's an attempt at something scientific on the issue. What they found is that text set on one single column of 95 cpl was faster to read than if it was less cpl. That was pretty obvious for the get go in the way they organized the text (divided in many screens with next/previous button). They should have either focused on texts that fit on one screen or provided two or even tree columns for the smaller text. I theorize that python has much more semantic information density than common english news (and in a completely different dimension) so you can't possibly extrapolate the results of that paper. -- Leonardo Santagada santagada at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Tue May 19 20:15:31 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 19 May 2009 11:15:31 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> > Whitespace: Python is very unique in that it *uses* whitespace for code blocking. > It turns out to be very useful, since it visually cues the reader where code blocks > begin and end by mandate. This creates many situations where code *starts* at the > 10th indentation (40 characters in our standard, 80 characters in some Python standards). FWIW, I think Google solves this problem by using two space idents. Though I suspect your note was trolling, I agree with you and find the 80 character width to be an anachronism. I've seen too many god-awful mid-phrase line wraps and trailing backslashes that could have been avoided with something a bit longer (like 100 chars). In addition to code wrapping, I'm a little bugged by indented comment blocks with 80 character line wrapping so that fewer than 40 chars are available per line of text. It sucks to update those comment blocks and then rewrap them so that a diff cannot easilly show what had changed. The days of 25x80 black and white CRTs are gone. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at free.fr Tue May 19 19:51:34 2009 From: denis.spir at free.fr (spir) Date: Tue, 19 May 2009 19:51:34 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <20090519195134.7f63598a@o> Le Tue, 19 May 2009 13:14:15 -0400, David Stanek s'exprima ainsi: > I'll bite. Nice to warn. Unfortunately, it won't not help making your reply helpful/pertinent/convincing. Too bad for the OP who precautiously & explicitely tried to avoid yet another religious war on a well-known hot topic, ain't it? Maybe next century... Denis ------ la vita e estrany From denis.spir at free.fr Tue May 19 20:41:50 2009 From: denis.spir at free.fr (spir) Date: Tue, 19 May 2009 20:41:50 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> Message-ID: <20090519204150.402573fd@o> Le Tue, 19 May 2009 11:15:31 -0700, "Raymond Hettinger" s'exprima ainsi: > FWIW, I think Google solves this problem by using two space idents. > > Though I suspect your note was trolling, I agree with you and find the 80 > character width to be an anachronism. I've seen too many god-awful > mid-phrase line wraps and trailing backslashes that could have been avoided > with something a bit longer (like 100 chars). > > In addition to code wrapping, I'm a little bugged by indented comment > blocks with 80 character line wrapping so that fewer than 40 chars are > available per line of text. It sucks to update those comment blocks and > then rewrap them so that a diff cannot easilly show what had changed. I think about the same way on this topic. +1 on relaxing guidelines constraints > The days of 25x80 black and white CRTs are gone. (Except I remember well green on black, and orange on black as well, CRTs ;-) Denis ------ la vita e estrany From stephen at xemacs.org Tue May 19 20:55:53 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 20 May 2009 03:55:53 +0900 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <871vqkhqh2.fsf@uwakimon.sk.tsukuba.ac.jp> Aaron Rubin writes: > 7) Python is designed to be written more like English than other programming > languages. English is written horizontally, not vertically. In furtherance > to an attempt to make "readability" an objective argument, here is a > scientific study which finds that greater character width lines improve > readability: The reporting of the statistical analysis borders on deceptive, and the sample is very small. The procedure is described in insufficient detail, as well. I would not rely on this study (and in fact the study itself reports that other studies have come up with results preferring "medium" lengths of 55-60). In general, I think your arguments mostly come down to "the line length limitation makes it noticably harder to write code quickly" plus a little bit of "longer line lengths are maybe a little faster to read". But I think that's the wrong place to put the emphasis. It's much more important to make the code easy to read and understand. The study you cite reports that 55% of the subjects considered the 95 cpl format the most uncomfortable. I think that's a good reason to be wary of increasing line length. From stephen at xemacs.org Tue May 19 21:08:25 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 20 May 2009 04:08:25 +0900 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> Message-ID: <87zld8gbbq.fsf@uwakimon.sk.tsukuba.ac.jp> Aaron Rubin writes: > class a(object): > def method1(simulation=False): > try: > if simulation: > for x in range(10): > if x>5: > try: > # here might begin some actual math, with two or > three more levels of logic, interfacing with other libraries such as NumPy, > etc. where you might need specific error handling Not in my code; it would almost surely be a function or method call. > except CustomError: > # customer error handling > > i.e. the logic code *started* at the 7th indentation level. But I'm sure > you can find plenty of examples where it might be more. From curt at hagenlocher.org Tue May 19 21:29:47 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 19 May 2009 12:29:47 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A12F031.6060303@mrabarnett.plus.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> Message-ID: On Tue, May 19, 2009 at 10:45 AM, MRAB wrote: >> >> I'd like to see an example of your variable names. I don't use >> hungarian notation and my name are usually under 10 characters. > > IMHO, the names should be long enough to be meaningful but not too long. You can't always control the names you're working with. In our case, we're writing .NET-based code that uses a lot types originally defined in C#. The standard in C#-land tends to be much more verbose than in Python, and definitely presents some challenges when trying to keep lines under 80 chars. I imagine that Jython code has a similar issue when interoperating with Java. -- Curt Hagenlocher curt at hagenlocher.org From chambon.pascal at wanadoo.fr Tue May 19 22:31:00 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Tue, 19 May 2009 22:31:00 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> References: <4A0496BD.3030709@wanadoo.fr> <43c8685c0905142116j902d273vc0b3c16cda49ab5c@mail.gmail.com> <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> Message-ID: <4A131704.6070003@wanadoo.fr> CTO a ?crit : > On May 18, 5:17 pm, Chris Rebert wrote: > >> >> The BDFL has condemned introducing new assignment operators. Seehttp://www.python.org/dev/peps/pep-3099/: >> >> "There will be no alternative binding operators such as :=." >> >> Cheers, >> Chris >> That's weird, in the archives quoted, I've found no exchange around the pros and cons of alternative binding operators, except the BDFL's "Brrh". ---> http://mail.python.org/pipermail/python-dev/2006-July/066995.html I guess that the operators rejected there mostly concerned the differentiation between binding and rebinding, although I couldnt be sure. Without new keyword or operator, a good looking solution for dynamic defaults is unlikely to appear, imo. I could content myself of the proposed solution : @dynamic def func (a, b = lambda : []): pass But I just dislike the fact that the "dynamic" applies to all the defaults, even those which weren't supposed to be dynamic (and writing "lambda : lambda : []") doesn't look good). Would there be any way of separating "to-be-called" lambdas from normal ones ? Except with a syntax like "b = dyn(lambda: [])" ? Regards, Pascal From phd at phd.pp.ru Tue May 19 22:34:09 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 20 May 2009 00:34:09 +0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A131704.6070003@wanadoo.fr> References: <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> <4A131704.6070003@wanadoo.fr> Message-ID: <20090519203409.GA17197@phd.pp.ru> On Tue, May 19, 2009 at 10:31:00PM +0200, Pascal Chambon wrote: > I could content myself of the proposed solution : > @dynamic > def func (a, b = lambda : []): > pass > But I just dislike the fact that the "dynamic" applies to all the > defaults, even those which weren't supposed to be dynamic (and writing > "lambda : lambda : []") doesn't look good). > Would there be any way of separating "to-be-called" lambdas from normal > ones ? Except with a syntax like "b = dyn(lambda: [])" ? @dynamic('b') def func (a, b = lambda : []): pass Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From chambon.pascal at wanadoo.fr Tue May 19 22:56:42 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Tue, 19 May 2009 22:56:42 +0200 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <20090519203409.GA17197@phd.pp.ru> References: <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> <4A131704.6070003@wanadoo.fr> <20090519203409.GA17197@phd.pp.ru> Message-ID: <4A131D0A.5080300@wanadoo.fr> Oleg Broytmann a ?crit : > On Tue, May 19, 2009 at 10:31:00PM +0200, Pascal Chambon wrote: > >> I could content myself of the proposed solution : >> @dynamic >> def func (a, b = lambda : []): >> pass >> But I just dislike the fact that the "dynamic" applies to all the >> defaults, even those which weren't supposed to be dynamic (and writing >> "lambda : lambda : []") doesn't look good). >> Would there be any way of separating "to-be-called" lambdas from normal >> ones ? Except with a syntax like "b = dyn(lambda: [])" ? >> > > @dynamic('b') > def func (a, b = lambda : []): > pass > > Oleg. > *shame on me - /me must be tired this evening* Well, with that solution we don't avoid some amount of boiler plate code, but imo the pros are - we don't have to care about copy/deepcopy constraints, or giving the default argument expression as a string (with all the problems eval() might raise) - most important for me : we have a pattern that could be broadcasted as a "standard practice", and thus warn newbies about the normal behaviour of default arguments. If the "dynamic" (or any other proper name) decorator became part of the builtin ones, like staticmethod or classmethod, in my opinion newbies would quickly run into it, get used to it, and we'd have a common practice instead of the numerous ones currently possible to handle dynamic defaults (sentinels, other decorators...). What do you think ? Is that worth the change compared to the sentinel stuff ? In my opinion, it's clearly more explicit, and the newbie will never think "that programmer is dumb, he has put None as a default whereas we can directly put lists or any other expressions in the signature, we're not restricted to constants like in other languages" as it might currently be the case. I'm eventually +1 on such a decorator/lambda mix (in the absence of more straightforward but language-changing syntax) ++ Pascal -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrts.pydev at gmail.com Tue May 19 23:51:03 2009 From: mrts.pydev at gmail.com (=?ISO-8859-1?Q?Mart_S=F5mermaa?=) Date: Wed, 20 May 2009 00:51:03 +0300 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> Message-ID: While aesthetics is a highly subjective matter, good code tends to have a "paucity begets expressiveness" feeling. I.e. experienced programmers seem to have developed an intuition for finding succinct names for entities -- names that convey the underlying meaning to the onlooker at first glance. That style is accompanied by good structure with groups of usually shortish lines with meaningful empty lines in between. The result is like good minimalist prose or a poem: a consistent piece of thinking, a pause, another piece that follows naturally from the previous. Aesthetically, this style contradicts with verboseness, long lines and generally cramming things together. The latter usually convey the image of uncertainty -- i.e. the writer is only yet trying to find a way of succinct expression, feeling a bit disoriented in the domain or language and wanting to hide the lack of confidence behind verbosity and pompous style. After all, Python has zen built in, so lets walk the path of 'import this' and revere the beauty of PEP-8 :). So, personally, I'm -0 (not that my opinion matters of course, and, after all, this is classic bikeshedding) even though quite a few of the previous arguments supporting longer lines are sensible. From jason.orendorff at gmail.com Wed May 20 00:04:24 2009 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Tue, 19 May 2009 17:04:24 -0500 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> Message-ID: On Tue, May 19, 2009 at 1:15 PM, Raymond Hettinger wrote: > Though I suspect your note was trolling, I agree with you and find the 80 > character width to be an anachronism. I find the long lines in my code are not as contrived as the thread so far suggests: t = t[:m.start()] + t[m.end():] # ok because we're iterating in reverse sys.stderr.write("js-build: WARNING: Due to COMMAND_MODE madness, this can fail on Leopard!\n" "js-build: Workaround is to upgrade to Python 2.5.2 or later.\n") allVariants = [_m + _r + _x for _x in ('', 'x') for _m in ('', 'm') for _r in ('r', 'd')] yield Message(parseTime(attr(u'received')), who, text(msg), isAction) while self.node is not None and self.node.nodeType == self.node.TEXT_NODE: And some examples in Mercurial: if err.errno not in (errno.ENOENT, errno.ENOTDIR, errno.EINVAL): ('U', 'noupdate', None, _('do not update the new working directories')), self.parent[id] = self.lastbranch.get(branch, 'bad') -j From aaron.rubin at 4dtechnology.com Wed May 20 00:36:35 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Tue, 19 May 2009 15:36:35 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> On Tue, May 19, 2009 at 10:51 AM, Mike Meyer wrote: > On May 19, 2009, at 12:43, Aaron Rubin > wrote: > > I realize that this is a religious debate which has gone on for many > centuries. I appeal to the scientific aspects, with a distinct avoidance of > preference and emotion. Preference might be easily explained by "right > brain" vs "left brain" preference, but either way, it is merely a preference > and I want to stick to facts. Here is a list I have compiled of facts which > support a wider than 80 character line width standard (in Python, > specifically). Please add to them, subtract from them, or add to the other > side of the debate, but please avoid the usage of the word "readable" (which > implies preference), unless you are referring to a scientific study of > readability, which would be great. > > > Most of you points don't stand up under inestigation. Of course, negating > them doesn't support am 80 character limit by itself. > > 1) Python is three things which the standard was not designed for: One: > Object Oriented. Two: Not Hungarian notation Three: Mandatorily uses > *whitespace* as its defintion for code blocks. Let me explain each one in a > bit more detail: Object Oriented: Because it is not functional-style > programming, but instead OO, you have to give defintion as to what object > type you are using before using it. This makes definitions and usage longer > than in functional programming (when 80 character widths were invented). > PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an example (and > not an extreme one) of a class (55 characters already) in a rather large > code base. > > > This type of reference is considered by some to be bad style. See the "Law > of Demeter" for more information. > The Law of Demeter applies to objects referenced second-hand. The class name given is an example of a hierarchy of modules, not one class reaching through a second class to get at the class members it uses. > Not Hungarian: Not only is Python not Hungarian (in general), but the > PEP-8 specifically tells us to use longer, more descriptive variable names. > hasInstrumentControllerPhaseDither is an example. Many variables are 15-20 > characters and oftentimes longer. > > > This appears to be false. A quick check of the standard library finds > between 1 and 2 percent of variable references to have fewer than 15 > characters, rising to 8 percent of unique names. This hardly qualified as > many. > do you mean greater than 15 characters? If not, then I don't see your point. At any rate, 8 percent of unique names seems statistically relevant. 8 percent of how many? If the number is 100,000, then I would say that 8,000 variable names qualifies as "many". > Whitespace: Python is very unique in that it *uses* whitespace for code > blocking. It turns out to be very useful, since it visually cues the reader > where code blocks begin and end by mandate. This creates many situations > where code *starts* at the 10th indentation (40 characters in our standard, > 80 characters in some Python standards). > > > This also appears to be false - the standard library has fewer than 200 > lines out of over 80,000 that start that deep. "Rarely" would seem to be > more accurate than "many". > 200 lines might qualify as many. Regardless, there are quite a number of lines which do. Let's not argue over the meaning of "many". As long as "some" exist, the point remains. > > Even in normal "great design" mode (which takes more time again), you can't > help it....your code starts at the 6th indentation level often. (28 > characters, more than 30% of 80 characters already gone. Now how many > variables or class names can you fit?) > Whitespace (2): Because Python uses whitespace as its sole method of > code blocking and because this is the visual cue for code blocks, wrapping > lines obfuscates this and makes the reader think about whether this > whitespace was there for a code block, or a line-wrap. Thinking about > intention of code slows us down. > > 2) Many of the libraries that are widely used do not adhere to the 80 > character width line standard. wxPython, NumPy and Boa Constructor are a > few, but I'm sure there are many, many more. > > > Which just goes to show that you don't hve to be constrained by the PEP if > you don't want to. Notvthat doing so is a good idea. > > 3) Writing new code in 80 character line widths takes more time. If I > have to worry about hitting this width, I have to de-concentrate my efforts > of writing logical code and concentrate instead on how exactly to format > this line of code (where to break it, etc....there are a number of rules > attached to wrapping lines of code). Then I have to re-concentrate on the > actual task at hand. Alternatively, I can code it up without worrying, then > when convenient, take some time to reformat to 80 character width. Either > way, more time. > > > On the other side, the Oulipo school of writing believes that writing with > apparently arbitrary constraints improves the results. I find that if I'm > running into the 80 character limit with any frequency, it's because my code > is poorly structured and in need of a reworking. > That can definitely be a symptom of bad code. Doesn't mean it's the only reason for it, however. > > 4) Code searching. IDEs have powerful searching features. They list all > the lines of a file (or files) which match the string you are searching for. > If things are in one line, this search is meaningful and you can read it > like you can code. If a line of code actually spans two (or more) lines > of code, the search is no longer contextually useful and you have to click > on each item to see what's actually going on. This feature is used heavily > in many environments (especially large code bases) to save time, but time is > either lost finding the actual context of a found string, or the search tool > is avoided altogether because it does not provide meaningful results (i.e. a > predictive waste of time) > > > In that case, you need a better IDE; it should either show more context, > allow multiline searches, or both. Both of these things will help even if > you never wrap your lines. > > 5) Monitors are getting bigger, wider, cheaper. This allows us to have two > files, side-by-side on a screen that are *both* 120 character width (or > even wider), without changing font sizes. > > > Printers aren't. > > 6) Tools are cheap. Time isn't. Get a second monitor, get a more powerful > editor, do whatever it takes to save time. If viewing more information at > one time is important, then we should try to make that possible with > technology, not time. > > > This point seems to be at best neutral to where or if you wrap lines. A > good IDE will wrap the display and indicate it did so. > > 7) Python is designed to be written more like English than other > programming languages. English is written horizontally, not vertically. In > furtherance to an attempt to make "readability" an objective argument, here > is a scientific study which finds that greater character width lines > improve readability: > http://psychology.wichita.edu/surl/usabilitynews/72/LineLength.asp. > To summarize, the study found that of the choices of 35, 55, 75 and 95 > character lengths, 95 was able to be read the fastest. > > > Please note that the study *started* by pointing out that other studies > existed which found the best line length to be anywhere from 35 to 85 > characters, and gave no reason for trusting their results rather than the > earlier studies. I would claim that all studies that looked at written > languages - as opposed to programs - were inapplicable. They almost > certainly used variable width fonts and a left justification (if not both), > either of which has more effect on readability than line length. > > Thanks for the time spent reading this long-ish post. Thanks for your > feedback if you provide it. > > > You really need better justification - studies of program comprehension, > not English reading speed, code bases that support your claims about written > code, etc. > I would love to see these studies done, but at this time I cannot find them. The closest I could come (and I disclaimed that it was only a loose connection) was the study I referenced. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed May 20 01:01:25 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 19 May 2009 19:01:25 -0400 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <20090519203409.GA17197@phd.pp.ru> References: <50697b2c0905142320n6c7ae8a3l430c17a6c5d9a8e6@mail.gmail.com> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> <4A131704.6070003@wanadoo.fr> <20090519203409.GA17197@phd.pp.ru> Message-ID: Oleg Broytmann wrote: > On Tue, May 19, 2009 at 10:31:00PM +0200, Pascal Chambon wrote: >> I could content myself of the proposed solution : >> @dynamic >> def func (a, b = lambda : []): >> pass >> But I just dislike the fact that the "dynamic" applies to all the >> defaults, even those which weren't supposed to be dynamic (and writing To repeat: I think one-usage default function objects defined by lambda are rather rare. The most obvious is def ident(ob): return ob which, if used once, would likely be used more than once and defined as above, and which always could be so defined. I suspect even more rare is such defaults used in the same function as a mutable default such as [] or {} that needs protecting. And in such rare cases, one could either pull the function definition into a def statement or prefix it by a second lambda. >> "lambda : lambda : []") doesn't look good). and I would not suggest it unless what one wanted was for the default arg for each call, after the lambda call, to be 'lambda: []' and not '[]'. >> Would there be any way of separating "to-be-called" lambdas from normal >> ones ? Except with a syntax like "b = dyn(lambda: [])" ? > > @dynamic('b') > def func (a, b = lambda : []): > pass Or dynamic could have an optional explicit list. But that option would be rarely needed, I think. tjr From mwm-keyword-python.b4bdba at mired.org Wed May 20 01:20:22 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Tue, 19 May 2009 19:20:22 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> Message-ID: <20090519192022.4e47f89d@bhuda.mired.org> On Tue, 19 May 2009 15:36:35 -0700 Aaron Rubin wrote: > On Tue, May 19, 2009 at 10:51 AM, Mike Meyer wrote: > > > On May 19, 2009, at 12:43, Aaron Rubin > > wrote: > > > > I realize that this is a religious debate which has gone on for many > > centuries. I appeal to the scientific aspects, with a distinct avoidance of > > preference and emotion. Preference might be easily explained by "right > > brain" vs "left brain" preference, but either way, it is merely a preference > > and I want to stick to facts. Here is a list I have compiled of facts which > > support a wider than 80 character line width standard (in Python, > > specifically). Please add to them, subtract from them, or add to the other > > side of the debate, but please avoid the usage of the word "readable" (which > > implies preference), unless you are referring to a scientific study of > > readability, which would be great. > > > > > > Most of you points don't stand up under inestigation. Of course, negating > > them doesn't support am 80 character limit by itself. > > > > 1) Python is three things which the standard was not designed for: One: > > Object Oriented. Two: Not Hungarian notation Three: Mandatorily uses > > *whitespace* as its defintion for code blocks. Let me explain each one in a > > bit more detail: Object Oriented: Because it is not functional-style > > programming, but instead OO, you have to give defintion as to what object > > type you are using before using it. This makes definitions and usage longer > > than in functional programming (when 80 character widths were invented). > > PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an example (and > > not an extreme one) of a class (55 characters already) in a rather large > > code base. > > > > > > This type of reference is considered by some to be bad style. See the "Law > > of Demeter" for more information. > > > > The Law of Demeter applies to objects referenced second-hand. The class > name given is an example of a hierarchy of modules, not one class reaching > through a second class to get at the class members it uses. In python, modules *are* objects, so this is still a case of one object reaching through a second object to get at members of that object. More to the point, Python allows you to use "from import ", "import as ..." and even "from import as ...", all of which provide much saner ways of dealing with deep trees of modules. > > Not Hungarian: Not only is Python not Hungarian (in general), but the > > PEP-8 specifically tells us to use longer, more descriptive variable names. > > hasInstrumentControllerPhaseDither is an example. Many variables are 15-20 > > characters and oftentimes longer. > > This appears to be false. A quick check of the standard library finds > > between 1 and 2 percent of variable references to have fewer than 15 > > characters, rising to 8 percent of unique names. This hardly qualified as > > many. > do you mean greater than 15 characters? If not, then I don't see your > point. At any rate, 8 percent of unique names seems statistically relevant. > 8 percent of how many? If the number is 100,000, then I would say that > 8,000 variable names qualifies as "many". Yes, I meant greater. And if you make your program small enough, 8,000 will be many. Then again, nearly any number qualifies as "many" if you're innumerate. The point is that if you want to use existing code to provide a reason for changing things, you should be making measurements of an existing code base rather than making vague claims about how common such things are. Since PEP 8 is really only enforced for the standard library, that's a good place to start. > > Whitespace: Python is very unique in that it *uses* whitespace for code > > blocking. It turns out to be very useful, since it visually cues the reader > > where code blocks begin and end by mandate. This creates many situations > > where code *starts* at the 10th indentation (40 characters in our standard, > > 80 characters in some Python standards). > > This also appears to be false - the standard library has fewer than 200 > > lines out of over 80,000 that start that deep. "Rarely" would seem to be > > more accurate than "many". > 200 lines might qualify as many. Regardless, there are quite a number of > lines which do. Let's not argue over the meaning of "many". As long as > "some" exist, the point remains. Yes, but the quantity matters. Changing things will result in pain for some users - that's part of why they don't change. If you want to use a problem with the way things are that causes pain to justify making a change, you need to show that it occurs frequently enough that the pain it's causing outweighs the pain that would be caused by the alternative and adopting the change. > > 3) Writing new code in 80 character line widths takes more time. If I > > have to worry about hitting this width, I have to de-concentrate my efforts > > of writing logical code and concentrate instead on how exactly to format > > this line of code (where to break it, etc....there are a number of rules > > attached to wrapping lines of code). Then I have to re-concentrate on the > > actual task at hand. Alternatively, I can code it up without worrying, then > > when convenient, take some time to reformat to 80 character width. Either > > way, more time. > > > > On the other side, the Oulipo school of writing believes that writing with > > apparently arbitrary constraints improves the results. I find that if I'm > > running into the 80 character limit with any frequency, it's because my code > > is poorly structured and in need of a reworking. > That can definitely be a symptom of bad code. Doesn't mean it's the only > reason for it, however. The number of reasons is irrelevant: you claim that writing to an 80-character limit slows you down, others claim that it causes them to write better code, so this matter also calls for more investigation. Are there studies that deal with either otherwise apocryphal claim? In particular, how many people are affected by each. > > You really need better justification - studies of program comprehension, > > not English reading speed, code bases that support your claims about written > > code, etc. > I would love to see these studies done, but at this time I cannot find them. That's sort of the point. You're making claims with few or no quantitative values to attach to them. I.e. "many variables use long names, and you can't get a lot of those on a line." A few minutes with grep & wc on the standard library suggest to me that this isn't the case. My "study" was admittedly unscientific and inaccurate - back of the envelope type stuff, but it's still more rigorous than what you started with. If you want to put your arguments on a scientific basis, you'll do some analysis of a real code base to provide quantities to back things up. One final addition - while it's true that desktop monitors are getting wider and cheaper, it's also true that we're moving to a world where people regularly work on things that aren't desktops. Laptops are getting cheaper and *smaller* as well as larger, and there are good reasons for not wanting a 17" - or even a 15" - laptop, which means you're back to screens the size of the old glass TTYs (which I fondly remember as light blue on dark blue, at least in the last iteration). Netbooks take us even smaller - and people have been putting development systems on PDAs for over a decade. So while developers now may have access to systems with multiple windows that are 120 or more characters wide, they also now use devices that have to stretch to their limits to display 80 columns across. Thanks, http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From aahz at pythoncraft.com Wed May 20 01:50:03 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 19 May 2009 16:50:03 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> Message-ID: <20090519235002.GA3479@panix.com> On Tue, May 19, 2009, Raymond Hettinger wrote: > > The days of 25x80 black and white CRTs are gone. That's half-true; the problem is that windows more than eighty columns wide cover up too much of other windows. I have generally compromised on thirty lines as being the right balance of information per window versus not covering up too much of other windows. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson From steve at pearwood.info Wed May 20 01:54:46 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 20 May 2009 09:54:46 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> Message-ID: <200905200954.46473.steve@pearwood.info> On Wed, 20 May 2009 03:53:33 am Aaron Rubin wrote: > On Tue, May 19, 2009 at 10:14 AM, David Stanek wrote: > > > Object Oriented: Because it is not functional-style > > > programming, but instead OO, you have to give defintion as to > > > what object type you are using before using it. This makes > > > definitions and usage longer than in functional > > > programming (when 80 character widths were invented). That's a red-herring. Functional programming is quite capable of 80+ character lines too: FetchFrameGrabber(ExtractAbstract(GetFrameSource(GetHardware(MakePhazeMonkey())))) > > > PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an > > > example Such a deeply hierarchical call is poor technique regardless of what each level represents: classes, modules, function calls, or something else. It is reasonable for parts of your code that understand PhazeMonkeys to know that a PhazeMonkey has Hardware, but it is completely unreasonable for that same section of code to rely on the existence of PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber. That sort of tight coupling between the top of the hierarchy and the bottom defeats the purpose of encapsulation. I see that in a later post, Aaron writes: "The Law of Demeter applies to objects referenced second-hand. ?The class name given is an example of a hierarchy of modules, not one class reaching through a second class to get at the class members it uses." That's an irrelevant objection. Demeter is a design principle: it applies to classes, namespaces, military units, corporate business units or any hierarchical structure. Of all the military blunders in World War I, the one they never made was for General Haig to call up the front line and say "Hello, let me speak to Private Baldrick. Baldrick, I want you to take your rifle and point it due east and use your index finger to pull the trigger at 1700 hours precisely." Besides, modules *are* objects: >>> import math >>> isinstance(math, type(math)) True > > If you are using more than 5 or 6 levels of indentation you may be > > doing something wrong. I would guess that your methods are too > > complex or maybe you are violating the SRP. > > See below for code example Namely: > class a(object): > def method1(simulation=False): > try: > if simulation: > for x in range(10): > if x>5: > try: > # here might begin some actual math, with > two or three more levels of logic, interfacing with other libraries > such as NumPy, etc. where you might need specific error handling > except CustomError: > # customer error handling > > i.e. the logic code *started* at the 7th indentation level. But I'm > sure you can find plenty of examples where it might be more. Far too complex in my opinion. That's best refactored into separate methods or functions. For example: class A(object): def simulated_method1(self): self._common() def method1(self): try: for x in range(5): self._metha(x) for x in range(5, 10): self._methb(x) self._common() except CustomError: self._fallback() def _metha(self, x): """Return function(x), for 0 <= x < 5.""" assert 0 <= x < 5 def _methb(self, x): """Return function(x), for 5 <= x < 10.""" assert 5 <= x < 10 That's (possibly?) longer code, but it comes in single-thought-sized chunks. Each method does one thing, rather than wrapping a whole bunch of loosely-related functionality into a single method. Another advantage is that it reduces the amount of test code you need. Instead of what could be an exponentially large number of possible paths through method1(), each individual method only has one or possibly two paths that need testing. > > I'd like to see an example of your variable names. I don't use > > hungarian notation and my name are usually under 10 characters. > > I gave an example already in the snippet you quoted. Namely "hasInstrumentControllerPhaseDither". Well, your naming conventions are yours, naturally, and you can use any convention you like, but I can't imagine ever willingly using a name like that. I would suggest that if you need to distinguish hasInstrumentControllerPhaseDither from (say) hasInstrumentWidgetPhaseDither, hasScannerControllerPhaseDither and hasInstrumentControllerAmplitudeDither in the one function, your function is far too complex, doing too many things, and should be split up into smaller single-purpose functions. > > > 5) Monitors are getting bigger, wider, cheaper. ?This allows us > > > to have two > > > files, side-by-side on a screen that are *both* 120 character > > > width (or even wider), without changing font sizes. True enough for those editing on large monitors, AND who like viewing multiple files side-by-side. (Personally, I very rarely do, and then only when comparing two files line-by-line.) But have pity on those using a EEE or similar machine. Besides, if you can fit two 120-char wide files on screen, you can fit three 80-char wide files on screen. For what it's worth, although I've never needed to interoperate with code using .NET style guidelines or written a 100 KLOC program, I generally have very little difficulty in keeping 90% of my code below *seventy* characters per line, including four-space indents, and virtually all of the rest below eighty. The short line-length encourages me to refactor my code so that every function does one thing and one thing only. -- Steven D'Aprano From google at mrabarnett.plus.com Wed May 20 02:01:43 2009 From: google at mrabarnett.plus.com (MRAB) Date: Wed, 20 May 2009 01:01:43 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090519192022.4e47f89d@bhuda.mired.org> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> <20090519192022.4e47f89d@bhuda.mired.org> Message-ID: <4A134867.5020601@mrabarnett.plus.com> Mike Meyer wrote: > On Tue, 19 May 2009 15:36:35 -0700 > Aaron Rubin wrote: > >> On Tue, May 19, 2009 at 10:51 AM, Mike Meyer wrote: >> >>> On May 19, 2009, at 12:43, Aaron Rubin >>> wrote: >>> >>> I realize that this is a religious debate which has gone on for many >>> centuries. I appeal to the scientific aspects, with a distinct avoidance of >>> preference and emotion. Preference might be easily explained by "right >>> brain" vs "left brain" preference, but either way, it is merely a preference >>> and I want to stick to facts. Here is a list I have compiled of facts which >>> support a wider than 80 character line width standard (in Python, >>> specifically). Please add to them, subtract from them, or add to the other >>> side of the debate, but please avoid the usage of the word "readable" (which >>> implies preference), unless you are referring to a scientific study of >>> readability, which would be great. >>> >>> >>> Most of you points don't stand up under inestigation. Of course, negating >>> them doesn't support am 80 character limit by itself. >>> >>> 1) Python is three things which the standard was not designed for: One: >>> Object Oriented. Two: Not Hungarian notation Three: Mandatorily uses >>> *whitespace* as its defintion for code blocks. Let me explain each one in a >>> bit more detail: Object Oriented: Because it is not functional-style >>> programming, but instead OO, you have to give defintion as to what object >>> type you are using before using it. This makes definitions and usage longer >>> than in functional programming (when 80 character widths were invented). >>> PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber is an example (and >>> not an extreme one) of a class (55 characters already) in a rather large >>> code base. >>> >>> >>> This type of reference is considered by some to be bad style. See the "Law >>> of Demeter" for more information. >>> >> The Law of Demeter applies to objects referenced second-hand. The class >> name given is an example of a hierarchy of modules, not one class reaching >> through a second class to get at the class members it uses. > > In python, modules *are* objects, so this is still a case of one > object reaching through a second object to get at members of that > object. > > More to the point, Python allows you to use "from import ", > "import as ..." and even "from import as ...", > all of which provide much saner ways of dealing with deep trees of > modules. > >>> Not Hungarian: Not only is Python not Hungarian (in general), but the >>> PEP-8 specifically tells us to use longer, more descriptive variable names. >>> hasInstrumentControllerPhaseDither is an example. Many variables are 15-20 >>> characters and oftentimes longer. >>> This appears to be false. A quick check of the standard library finds >>> between 1 and 2 percent of variable references to have fewer than 15 >>> characters, rising to 8 percent of unique names. This hardly qualified as >>> many. >> do you mean greater than 15 characters? If not, then I don't see your >> point. At any rate, 8 percent of unique names seems statistically relevant. >> 8 percent of how many? If the number is 100,000, then I would say that >> 8,000 variable names qualifies as "many". > > Yes, I meant greater. And if you make your program small enough, 8,000 > will be many. Then again, nearly any number qualifies as "many" if > you're innumerate. > > The point is that if you want to use existing code to provide a reason > for changing things, you should be making measurements of an existing > code base rather than making vague claims about how common such things > are. Since PEP 8 is really only enforced for the standard library, > that's a good place to start. > >>> Whitespace: Python is very unique in that it *uses* whitespace for code >>> blocking. It turns out to be very useful, since it visually cues the reader >>> where code blocks begin and end by mandate. This creates many situations >>> where code *starts* at the 10th indentation (40 characters in our standard, >>> 80 characters in some Python standards). >>> This also appears to be false - the standard library has fewer than 200 >>> lines out of over 80,000 that start that deep. "Rarely" would seem to be >>> more accurate than "many". >> 200 lines might qualify as many. Regardless, there are quite a number of >> lines which do. Let's not argue over the meaning of "many". As long as >> "some" exist, the point remains. > > Yes, but the quantity matters. Changing things will result in pain for > some users - that's part of why they don't change. If you want to use > a problem with the way things are that causes pain to justify making a > change, you need to show that it occurs frequently enough that the > pain it's causing outweighs the pain that would be caused by the > alternative and adopting the change. > >>> 3) Writing new code in 80 character line widths takes more time. If I >>> have to worry about hitting this width, I have to de-concentrate my efforts >>> of writing logical code and concentrate instead on how exactly to format >>> this line of code (where to break it, etc....there are a number of rules >>> attached to wrapping lines of code). Then I have to re-concentrate on the >>> actual task at hand. Alternatively, I can code it up without worrying, then >>> when convenient, take some time to reformat to 80 character width. Either >>> way, more time. >>> >>> On the other side, the Oulipo school of writing believes that writing with >>> apparently arbitrary constraints improves the results. I find that if I'm >>> running into the 80 character limit with any frequency, it's because my code >>> is poorly structured and in need of a reworking. >> That can definitely be a symptom of bad code. Doesn't mean it's the only >> reason for it, however. > > The number of reasons is irrelevant: you claim that writing to an > 80-character limit slows you down, others claim that it causes them to > write better code, so this matter also calls for more investigation. > Are there studies that deal with either otherwise apocryphal claim? > In particular, how many people are affected by each. > >>> You really need better justification - studies of program comprehension, >>> not English reading speed, code bases that support your claims about written >>> code, etc. >> I would love to see these studies done, but at this time I cannot find them. > > That's sort of the point. You're making claims with few or no > quantitative values to attach to them. I.e. "many variables use long > names, and you can't get a lot of those on a line." A few minutes with > grep & wc on the standard library suggest to me that this isn't the > case. My "study" was admittedly unscientific and inaccurate - back of > the envelope type stuff, but it's still more rigorous than what you > started with. If you want to put your arguments on a scientific basis, > you'll do some analysis of a real code base to provide quantities to > back things up. > > One final addition - while it's true that desktop monitors are getting > wider and cheaper, it's also true that we're moving to a world where > people regularly work on things that aren't desktops. Laptops are > getting cheaper and *smaller* as well as larger, and there are good > reasons for not wanting a 17" - or even a 15" - laptop, which means > you're back to screens the size of the old glass TTYs (which I fondly > remember as light blue on dark blue, at least in the last > iteration). Netbooks take us even smaller - and people have been > putting development systems on PDAs for over a decade. So while > developers now may have access to systems with multiple windows that > are 120 or more characters wide, they also now use devices that have > to stretch to their limits to display 80 columns across. > With screens in glasses and built-in mini-projectors you'll be able to have wide displays again even on PDAs. :-) From ben+python at benfinney.id.au Wed May 20 02:04:57 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 20 May 2009 10:04:57 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> Message-ID: <87tz3g63me.fsf@benfinney.id.au> Aaron Rubin writes: > Regarding this and the other notion of 5 or 6 being the proper level of > indentation: > > class a(object): > def method1(simulation=False): > try: > if simulation: > for x in range(10): > if x>5: > try: > # here might begin some actual math, with two or > three more levels of logic, interfacing with other libraries such as NumPy, > etc. where you might need specific error handling > except CustomError: > # customer error handling This is fairly clearly a contrived example. Can you give an example of actual in-use code which suffers from such deep indentation? I've no doubt you can produce such code, but it's far easier to discuss ways to improve (and *that* it will improve) an example of actual code than a contrived example. > i.e. the logic code *started* at the 7th indentation level. This would almost certainly be improved by taking some of those deeply-indented parts and re-factoring them to separate functions. But again, it's hard to show that without a real example. -- \ ?Holy uncanny photographic mental processes, Batman!? ?Robin | `\ | _o__) | Ben Finney From ben+python at benfinney.id.au Wed May 20 02:07:40 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 20 May 2009 10:07:40 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> Message-ID: <87pre463hv.fsf@benfinney.id.au> Jason Orendorff writes: > I find the long lines in my code are not as contrived as the thread so > far suggests: > > t = t[:m.start()] + t[m.end():] # ok because > we're iterating in reverse > > sys.stderr.write("js-build: WARNING: Due to COMMAND_MODE > madness, this can fail on Leopard!\n" > "js-build: Workaround is to > upgrade to Python 2.5.2 or later.\n") > > allVariants = [_m + _r + _x for _x in ('', 'x') for _m in ('', 'm') > for _r in ('r', 'd')] [and so on] Jason, if you want to show code with long lines, please use an email tool chain that doesn't wrap those long lines. Somewhere along the way you're getting extra line breaks inserted. -- \ ?Facts do not cease to exist because they are ignored.? ?Aldous | `\ Huxley | _o__) | Ben Finney From cmjohnson.mailinglist at gmail.com Wed May 20 02:53:44 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Tue, 19 May 2009 14:53:44 -1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> Message-ID: <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> Curt Hagenlocher wrote: > You can't always control the names you're working with. In our case, > we're writing .NET-based code that uses a lot types originally defined > in C#. The standard in C#-land tends to be much more verbose than in > Python, and definitely presents some challenges when trying to keep > lines under 80 chars. I imagine that Jython code has a similar issue > when interoperating with Java. That's true. It's also a problem for people doing Cocoa with PyObjC. But who cares? If you want to break PEP8 for your own project, there's nothing to stop you. The only thing that PEP8 is binding on is what gets checked into the cpython source, and, for obvious reasons, that doesn't interact with C#-stuff or JVM-stuff or Cocoa-stuff to any significant degree. I'm not sure what the point of this debate is. If you want to use 500 chars per line, the interpreter won't complain. If you want to say that using more than 80 chars per line would make the cpython source read better, I would have to respectfully disagree. So, what's the problem? From cmjohnson.mailinglist at gmail.com Wed May 20 03:04:27 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Tue, 19 May 2009 15:04:27 -1000 Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <4A131704.6070003@wanadoo.fr> References: <4A0496BD.3030709@wanadoo.fr> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> <4A131704.6070003@wanadoo.fr> Message-ID: <3bdda690905191804v1af36a9dod7363c79d07d158@mail.gmail.com> Pascal Chambon wrote: > Without new keyword or operator, a good looking solution for dynamic > defaults is unlikely to appear, imo. > > I could content myself of the proposed solution : > @dynamic > def func (a, b = lambda : []): > ? pass > But I just dislike the fact that the "dynamic" applies to all the defaults, > even those which weren't supposed to be dynamic (and writing "lambda : > lambda : []") doesn't look good). > Would there be any way of separating "to-be-called" lambdas from normal ones > ? Except with a syntax like "b = dyn(lambda: [])" ? Use function annotations. >>> def f(a: dynamic.override=list, b=1): pass ... >>> f.__annotations__ {'a': dynamic.override} >>> f.__defaults__ (, 1) Just make a decorator that looks in the annotations to figure out what to replace and what to leave be. Also "lambda: []" is clearly inferior to "list". -- Carl From curt at hagenlocher.org Wed May 20 03:53:35 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 19 May 2009 18:53:35 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> Message-ID: On Tue, May 19, 2009 at 5:53 PM, Carl Johnson wrote: > > But who cares? If you want to break PEP8 for your own project, there's > nothing to stop you. True. Unless -- hypothetically -- the architect decides that all the source in the project will follow PEP-8 and you want to do an end-run around the decision because you're tired of losing five minutes every few hours as a result of working out how to reformat a block of code to best respect the 80-char limit. *wink* -- Curt Hagenlocher curt at hagenlocher.org From stephen at xemacs.org Wed May 20 04:38:55 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 20 May 2009 11:38:55 +0900 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> Message-ID: <87ws8cfqgw.fsf@uwakimon.sk.tsukuba.ac.jp> Jason Orendorff writes: > On Tue, May 19, 2009 at 1:15 PM, Raymond Hettinger wrote: > > Though I suspect your note was trolling, I agree with you and find the 80 > > character width to be an anachronism. > > I find the long lines in my code are not as contrived as the thread so > far suggests: I do find several of them unreadable. Brief classification: > t = t[:m.start()] + t[m.end():] # ok because we're iterating in reverse Plausible usage; gets the comment out of the control flow, which is good. > sys.stderr.write("js-build: WARNING: Due to COMMAND_MODE madness, this can fail on Leopard!\n" > "js-build: Workaround is to upgrade to Python 2.5.2 or later.\n") Ditto, for arbitrary I/O content. > allVariants = [_m + _r + _x for _x in ('', 'x') for _m in ('', 'm') for _r in ('r', 'd')] I find that hard to read. With more than two clauses, I prefer to put them on separate lines anyway: allVariants = [_m + _r + _x for _x in ('', 'x') for _m in ('', 'm') for _r in ('r', 'd')] > yield Message(parseTime(attr(u'received')), who, text(msg), isAction) Mildly hard to read; I tend to skip the details of a complex call on one line, and have to go back if I need them. Eg, in review I would very like miss a typo where the 2nd and 3rd arguments are (incorrectly) swapped unless I read that line token by token. I'm much less likely to make that mistake for yield Message(parseTime(attr(u'received')), who, text(msg), isAction) although I grant that at 81 characters for the long line, the two-line format is distinctly uglier. > while self.node is not None and self.node.nodeType == self.node.TEXT_NODE: Unecessary for me. I would use a sentinel whose nodeType is SENTINEL_NODE for this anyway, rather than None. Note that there are two kinds of arguments presented. One is that the long line isn't necessary. That obviously has to depend on higher- level issues of style (eg, using None vs. a special nodeType as a sentinel), so definitely YMMV, and if a lot of people in your project prefer the None-style, that's an argument *for* long lines. The other is that I find some hard to read, and if there are more than a very few in your project who feel as I do, long lines can have a *negative impact* on review. From ben+python at benfinney.id.au Wed May 20 05:51:37 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 20 May 2009 13:51:37 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> Message-ID: <878wks5t4m.fsf@benfinney.id.au> Curt Hagenlocher writes: > Unless -- hypothetically -- the architect decides that all the source > in the project will follow PEP-8 and you want to do an end-run around > the decision because you're tired of losing five minutes every few > hours as a result of working out how to reformat a block of code to > best respect the 80-char limit. As pointed out elsewhere in this thread, very often the superior solution is not to re-*format* the same statement to fit, but to re-*factor* the code so it's less deeply indented or does less in each statement. And that re-factoring isn't lost time; it's saved time when later readers try to understand the code. -- \ ?There was a point to this story, but it has temporarily | `\ escaped the chronicler's mind.? ?Douglas Adams | _o__) | Ben Finney From debatem1 at gmail.com Wed May 20 05:52:37 2009 From: debatem1 at gmail.com (CTO) Date: Tue, 19 May 2009 20:52:37 -0700 (PDT) Subject: [Python-ideas] Default arguments in Python - the return - running out of ideas but... In-Reply-To: <3bdda690905191804v1af36a9dod7363c79d07d158@mail.gmail.com> References: <4A0496BD.3030709@wanadoo.fr> <4A0D3A97.5010606@hastings.org> <9bfc700a0905150352m6e7ae240y464b70374eedeb93@mail.gmail.com> <91ad5bf80905150559r4c1c6c5bv30d03ab5212e0c6d@mail.gmail.com> <4A11CAB7.60600@wanadoo.fr> <50697b2c0905181417o261f0b40gb99701800115b7c@mail.gmail.com> <9fd21491-b402-45c6-ae30-4fd001db5f6a@o20g2000vbh.googlegroups.com> <4A131704.6070003@wanadoo.fr> <3bdda690905191804v1af36a9dod7363c79d07d158@mail.gmail.com> Message-ID: On May 19, 9:04?pm, Carl Johnson wrote: > Pascal Chambon wrote: > > Without new keyword or operator, a good looking solution for dynamic > > defaults is unlikely to appear, imo. > > > I could content myself of the proposed solution : > > @dynamic > > def func (a, b = lambda : []): > > ? pass > > But I just dislike the fact that the "dynamic" applies to all the defaults, > > even those which weren't supposed to be dynamic (and writing "lambda : > > lambda : []") doesn't look good). > > Would there be any way of separating "to-be-called" lambdas from normal ones > > ? Except with a syntax like "b = dyn(lambda: [])" ? > > Use function annotations. > > >>> def f(a: dynamic.override=list, b=1): pass > ... > >>> f.__annotations__ > > {'a': dynamic.override}>>> f.__defaults__ > > (, 1) > > Just make a decorator that looks in the annotations to figure out what > to replace and what to leave be. > > Also "lambda: []" is clearly inferior to "list". > > -- Carl Already done, as mentioned further up in this list: Geremy Condra From curt at hagenlocher.org Wed May 20 06:08:02 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 19 May 2009 21:08:02 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <878wks5t4m.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> <878wks5t4m.fsf@benfinney.id.au> Message-ID: On Tue, May 19, 2009 at 8:51 PM, Ben Finney wrote: > Curt Hagenlocher writes: > >> Unless -- hypothetically -- the architect decides that all the source >> in the project will follow PEP-8 and you want to do an end-run around >> the decision because you're tired of losing five minutes every few >> hours as a result of working out how to reformat a block of code to >> best respect the 80-char limit. > > As pointed out elsewhere in this thread, very often the superior > solution is not to re-*format* the same statement to fit, but to > re-*factor* the code so it's less deeply indented or does less in each > statement. Yes, "refactor" is a much better word than "reformat" for what I was thinking. -- Curt Hagenlocher curt at hagenlocher.org From ben+python at benfinney.id.au Wed May 20 07:20:01 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 20 May 2009 15:20:01 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> <878wks5t4m.fsf@benfinney.id.au> Message-ID: <87zld84agu.fsf@benfinney.id.au> Curt Hagenlocher writes: > Yes, "refactor" is a much better word than "reformat" for what I was > thinking. Well, before deciding that, be aware that it already has a fairly specific meaning in programming terminology . -- \ ?I believe in making the world safe for our children, but not | `\ our children's children, because I don't think children should | _o__) be having sex.? ?Jack Handey | Ben Finney From cesare.dimauro at a-tono.com Wed May 20 10:08:15 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Wed, 20 May 2009 10:08:15 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87tz3g63me.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> <87tz3g63me.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Aaron Rubin > writes: > >> Regarding this and the other notion of 5 or 6 being the proper level of >> indentation: >> >> class a(object): >> def method1(simulation=False): >> try: >> if simulation: >> for x in range(10): >> if x>5: >> try: >> # here might begin some actual math, with two or >> three more levels of logic, interfacing with other libraries such as NumPy, >> etc. where you might need specific error handling >> except CustomError: >> # customer error handling > > This is fairly clearly a contrived example. Can you give an example of > actual in-use code which suffers from such deep indentation? > >From threading.py: class Thread(_Verbose): def __bootstrap_inner(self): try: self.__ident = _get_ident() self.__started.set() _active_limbo_lock.acquire() _active[self.__ident] = self del _limbo[self] _active_limbo_lock.release() if __debug__: self._note("%s.__bootstrap(): thread started", self) if _trace_hook: self._note("%s.__bootstrap(): registering trace hook", self) _sys.settrace(_trace_hook) if _profile_hook: self._note("%s.__bootstrap(): registering profile hook", self) _sys.setprofile(_profile_hook) try: self.run() except SystemExit: if __debug__: self._note("%s.__bootstrap(): raised SystemExit", self) except: if __debug__: self._note("%s.__bootstrap(): unhandled exception", self) # If sys.stderr is no more (most likely from interpreter # shutdown) use self.__stderr. Otherwise still use sys (as in # _sys) in case sys.stderr was redefined since the creation of # self. if _sys: _sys.stderr.write("Exception in thread %s:\n%s\n" % (self.name, _format_exc())) else: # Do the best job possible w/o a huge amt. of code to # approximate a traceback (code ideas from # Lib/traceback.py) exc_type, exc_value, exc_tb = self.__exc_info() try: print>>self.__stderr, ( "Exception in thread " + self.name + " (most likely raised during interpreter shutdown):") print>>self.__stderr, ( "Traceback (most recent call last):") while exc_tb: print>>self.__stderr, ( ' File "%s", line %s, in %s' % (exc_tb.tb_frame.f_code.co_filename, exc_tb.tb_lineno, exc_tb.tb_frame.f_code.co_name)) exc_tb = exc_tb.tb_next print>>self.__stderr, ("%s: %s" % (exc_type, exc_value)) # Make sure that exc_tb gets deleted since it is a memory # hog; deleting everything else is just for thoroughness finally: del exc_type, exc_value, exc_tb else: if __debug__: self._note("%s.__bootstrap(): normal return", self) finally: # Prevent a race in # test_threading.test_no_refcycle_through_target when # the exception keeps the target alive past when we # assert that it's dead. self.__exc_clear() finally: with _active_limbo_lock: self.__stop() try: # We don't call self.__delete() because it also # grabs _active_limbo_lock. del _active[_get_ident()] except: pass Cheers, Cesare From denis.spir at free.fr Wed May 20 10:20:52 2009 From: denis.spir at free.fr (spir) Date: Wed, 20 May 2009 10:20:52 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87ws8cfqgw.fsf@uwakimon.sk.tsukuba.ac.jp> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> <87ws8cfqgw.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20090520102052.1da6ca97@o> Le Wed, 20 May 2009 11:38:55 +0900, "Stephen J. Turnbull" s'exprima ainsi: > > yield Message(parseTime(attr(u'received')), who, text(msg), > > isAction) > > Mildly hard to read; I tend to skip the details of a complex call on > one line, and have to go back if I need them. Eg, in review I would > very like miss a typo where the 2nd and 3rd arguments are > (incorrectly) swapped unless I read that line token by token. I'm much > less likely to make that mistake for > > yield Message(parseTime(attr(u'received')), who, > text(msg), isAction) > > although I grant that at 81 characters for the long line, the two-line > format is distinctly uglier. One wall I run into with lines < 80 characters is, paradoxally, that I miss width for lines I want to break (!). The issue is that, ~ in order to have logical, comprehensible, layout, ~ and to avoid messing up with python's own indent the continuation lines have to align with items on the same logical level in the first line, for instance: try: my_final_result = finalResultProducer(arg1_from_abunch_of_args, args_should_align_properly, [so, what, if, they, re, compound]) except AttributeError, error: raise ResultError( "foo ......... bar" "<--All message text lines should start here." %(String, interpolation, argument, list, as, well) ) Note that I started the try...except at a rather low level (3). Denis ------ la vita e estrany From ben+python at benfinney.id.au Wed May 20 11:16:02 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 20 May 2009 19:16:02 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191053j1759409cnfc7b4991a3d119ab@mail.gmail.com> <87tz3g63me.fsf@benfinney.id.au> Message-ID: <87octo3zjh.fsf@benfinney.id.au> "Cesare Di Mauro" writes: > >From threading.py: > That's a perfect example of doing too much in one function, thank you. Here's a first draft refactor for both making it easier to keep track of, and reducing the indentation as an added benefit:: class Thread(_Verbose): def _prepare_bootstrap(self): self.__ident = _get_ident() self.__started.set() _active_limbo_lock.acquire() _active[self.__ident] = self del _limbo[self] _active_limbo_lock.release() if __debug__: self._note("%s.__bootstrap(): thread started", self) if _trace_hook: self._note("%s.__bootstrap(): registering trace hook", self) _sys.settrace(_trace_hook) if _profile_hook: self._note("%s.__bootstrap(): registering profile hook", self) _sys.setprofile(_profile_hook) def _bootstrap_run(self): try: self.run() except SystemExit: if __debug__: self._note("%s.__bootstrap(): raised SystemExit", self) except: self._unhandled_exception() else: if __debug__: self._note("%s.__bootstrap(): normal return", self) finally: # Prevent a race in # test_threading.test_no_refcycle_through_target when # the exception keeps the target alive past when we # assert that it's dead. self.__exc_clear() def _unhandled_exception(self): if __debug__: self._note("%s.__bootstrap(): unhandled exception", self) # If sys.stderr is no more (most likely from interpreter # shutdown) use self.__stderr. Otherwise still use sys (as in # _sys) in case sys.stderr was redefined since the creation of # self. if _sys: _sys.stderr.write("Exception in thread %s:\n%s\n" % (self.name, _format_exc())) else: # Do the best job possible w/o a huge amt. of code to # approximate a traceback (code ideas from # Lib/traceback.py) exc_type, exc_value, exc_tb = self.__exc_info() try: print>>self.__stderr, ( "Exception in thread " + self.name + " (most likely raised during interpreter shutdown):") print>>self.__stderr, ( "Traceback (most recent call last):") while exc_tb: print>>self.__stderr, ( ' File "%s", line %s, in %s' % (exc_tb.tb_frame.f_code.co_filename, exc_tb.tb_lineno, exc_tb.tb_frame.f_code.co_name)) exc_tb = exc_tb.tb_next print>>self.__stderr, ("%s: %s" % (exc_type, exc_value)) # Make sure that exc_tb gets deleted since it is a memory # hog; deleting everything else is just for thoroughness finally: del exc_type, exc_value, exc_tb def _limbo_lock_stop(self): with _active_limbo_lock: self.__stop() try: # We don't call self.__delete() because it also # grabs _active_limbo_lock. del _active[_get_ident()] except: pass def __bootstrap_inner(self): try: self._prepare_bootstrap() self._bootstrap_run() finally: self._limbo_lock_stop() -- \ ?Apologize, v. To lay the foundation for a future offense.? | `\ ?Ambrose Bierce, _The Devil's Dictionary_, 1906 | _o__) | Ben Finney From denis.spir at free.fr Wed May 20 11:30:13 2009 From: denis.spir at free.fr (spir) Date: Wed, 20 May 2009 11:30:13 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: <20090520113013.23200e09@o> I think that, aside dogmatisms, we can try and clarify when, how, and why line breaks and/or longer lines can help us writing comprehensible code. This is, or should be, the purpose of defining style guidelines. My position reads as follows -- expansion, critics, comments welcome. -1- Use line breaks. Line breaks often help showing the logics of code deeper than, but in a similar way as, python's forced indentation. This is done by splitting complexity into smaller bits visually presented in parallel. They can also help organising literals in a sensible manner. Both often play together to offer "easy to read" (whatever this means) code. This is common for function definitions and function calls, nested container literals, multiline output strings,... -2- Use long lines. One the other hand, sometimes we wish not to break a line because this would break the logics alltogether. The breaking place would just be arbitrary if there is no logical structure in what we have to split (just to comply with guidelines); so that instead of helping readibility, line breaks introduce "semantic distortion" in such cases. This is common for longer flat containers, single-line string literals and probably some other cases. -3- How to use line breaks, then? While I love using line breaks when they make sense, I often to have no other choice than disregarding the 80-chars rule *precisely when breaking lines*. The issue is that, in adition to indentation, the continuation line(s) must _visually_ align with items at the same _logical_ level. This level can well be 20 or 30 characters farther on right from the indentation level: try: my_final_result = finalResultComputer(arg1_from_abunch_of_args, args_should_align_properly, [so, what, if, they, re, compound]) except AttributeError, error: raise computeError( "foo ......... bar" "<--All message text lines should start here." %(String, interpolation, argument, list, as, well) ) [If we do not respect alignment, then the code is semantically wrong. It's like if we would (have to) write for instance: if condition: task1.................................... |80 char limit task2........................................ task3....................................... just to save some indent characters.] -4- Narrow text even in wide lines! This raises the question of available free width. With both indentation and alignment coming into play, we often have less than 40 characters left. All arguments pro or contra broader lines for legibility thus are irrelevant, because we have rather narrow content anyway ;- -5- Style Then comes the question of style. Numerous authors have argued that "readibility" or "legibility" simply are masked aliases for "familiarity". On the other hand, a programmer should be able to express things the way s/he really thinks them; to express his or her real meaning and intent. A programmer must be able to feel easy with his or her own code! So that there is a balance to be found between uniformity and personal ease. As a summary, I would like a recommandation rather than a rule. A note that shows how (0) short line _content_ usually is a sign of good code (*), (1) line breaking can help expressing code structure, (2) but this can require rather big overall line widths, whith leading spacing (3) and in some cases single-lines simply better mirror semantics. (*) How much, typically? Denis ------ la vita e estrany From ben+python at benfinney.id.au Wed May 20 11:40:31 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 20 May 2009 19:40:31 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <20090520113013.23200e09@o> Message-ID: <87k54c3yeo.fsf@benfinney.id.au> spir writes: > try: > my_final_result = finalResultComputer(arg1_from_abunch_of_args, > args_should_align_properly, > [so, what, if, they, re, compound]) > except AttributeError, error: > raise computeError( "foo ......... bar" > "<--All message text lines should start here." > %(String, interpolation, argument, list, as, well) ) It's for this reason that I advocate indenting continued lines *one* level, and not this hideously large increase in indentation for a single step. Simply break at the opening container character, and indent a single level to make all the contents line up:: try: my_final_result = finalResultComputer( arg1_from_abunch_of_args, args_should_align_properly, [so, what, if, they, re, compound]) except AttributeError, error: raise computeError( "foo ......... bar" "<--All message text lines should start here." % (String, interpolation, argument, list, as, well)) -- \ ?I put instant coffee in a microwave oven and almost went back | `\ in time.? ?Steven Wright | _o__) | Ben Finney From denis.spir at free.fr Wed May 20 11:41:11 2009 From: denis.spir at free.fr (spir) Date: Wed, 20 May 2009 11:41:11 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090519192022.4e47f89d@bhuda.mired.org> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> <20090519192022.4e47f89d@bhuda.mired.org> Message-ID: <20090520114111.6e670e49@o> On the other side, the Oulipo school of writing believes that writing with apparently arbitrary constraints improves the results. ??? How can you assert such a non-pertinent (and wrong) statement (in regard to the point beeing discussed)? Oulipo games are about helping *creativity*. A set of strict constraints helps the poet be creative -- or rather they let whatever unconsciounsly creates "popping" good mattery into conscious poetic minds. Denis From denis.spir at free.fr Wed May 20 12:19:41 2009 From: denis.spir at free.fr (spir) Date: Wed, 20 May 2009 12:19:41 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87k54c3yeo.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <20090520113013.23200e09@o> <87k54c3yeo.fsf@benfinney.id.au> Message-ID: <20090520121941.66037786@o> Le Wed, 20 May 2009 19:40:31 +1000, Ben Finney s'exprima ainsi: > spir writes: > > > try: > > my_final_result = > > finalResultComputer(arg1_from_abunch_of_args, args_should_align_properly, > > [so, what, if, > > they, re, compound]) except AttributeError, error: > > raise computeError( "foo ......... bar" > > "<--All message text lines should > > start here." %(String, interpolation, argument, list, as, well) ) > > It's for this reason that I advocate indenting continued lines *one* > level, and not this hideously large increase in indentation for a single > step. > > Simply break at the opening container character, and indent a single > level to make all the contents line up:: > > try: > my_final_result = finalResultComputer( > arg1_from_abunch_of_args, > args_should_align_properly, > [so, what, if, they, re, compound]) > except AttributeError, error: > raise computeError( > "foo ......... bar" > "<--All message text lines should start here." > % (String, interpolation, argument, list, as, well)) Yes, I partially agree with you indentation mode. But to my eyes (maybe it's only me) the result (1) seems to suggest ordinary python indentation, which it is not and (2) does not make obvious what continuation lines belong to. This can be a bit improved (again, to my eyes) using ~ double indentation for continuation lines ~ closing sings (parens, etc) on their own line Which gives: try: my_final_result = finalResultComputer( arg1_from_abunch_of_args, args_should_align_properly, [so, what, if, they, re, compound] ) except AttributeError, error: raise computeError( "foo ......... bar" "<--All message text lines should start here." % (String, interpolation, argument, list, as, well) ) This looks acceptable -- even if 2 more lines are spent vertically (another relevant point). For any reason, while in general I really do not support C-like style, in these precise cases I find the final closing mark on its own really helpful (esp. where I placed it here). Denis ------ la vita e estrany From rdmurray at bitdance.com Wed May 20 14:04:56 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 20 May 2009 12:04:56 +0000 (UTC) Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> <20090519192022.4e47f89d@bhuda.mired.org> <20090520114111.6e670e49@o> Message-ID: spir wrote: > On the other side, the Oulipo school of writing believes that writing with > apparently arbitrary constraints improves the results. > > ??? > > How can you assert such a non-pertinent (and wrong) statement (in regard to > the point beeing discussed)? Oulipo games are about helping *creativity*. A > set of strict constraints helps the poet be creative -- or rather they let > whatever unconsciounsly creates "popping" good mattery into conscious poetic > minds. Do you think that writing code is an act that does not involve creativity? Writing with constraints to improve creativity applies to any kind of writing, not just poetry. I heard about the idea in my creative writing class; I've never heard of Oulipo before. It also has already been invoked in this thread with respect to writing programs: constrained to 80 characters, you are encouraged to refactor the code, thereby producing superior code. --David From aahz at pythoncraft.com Wed May 20 14:10:08 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 20 May 2009 05:10:08 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87k54c3yeo.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <20090520113013.23200e09@o> <87k54c3yeo.fsf@benfinney.id.au> Message-ID: <20090520121006.GA6926@panix.com> On Wed, May 20, 2009, Ben Finney wrote: > > Simply break at the opening container character, and indent a single > level to make all the contents line up:: > > try: > my_final_result = finalResultComputer( > arg1_from_abunch_of_args, > args_should_align_properly, > [so, what, if, they, re, compound]) > except AttributeError, error: > raise computeError( > "foo ......... bar" > "<--All message text lines should start here." > % (String, interpolation, argument, list, as, well)) That's almost what I do, except that I put the terminating container character on its own line to simplify adding and deleting arguments:: try: my_final_result = finalResultComputer( arg1_from_abunch_of_args, args_should_align_properly, [so, what, if, they, re, compound], ) except AttributeError, error: raise computeError( "foo ......... bar" "<--All message text lines should start here." % (String, interpolation, argument, list, as, well) ) This practice comes more from lists/dicts; I simply generalize it to arguments. You'll also notice that similarly I put a trailing comma on the last argument. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson From ben+python at benfinney.id.au Wed May 20 14:59:57 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Wed, 20 May 2009 22:59:57 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <20090520113013.23200e09@o> <87k54c3yeo.fsf@benfinney.id.au> <20090520121941.66037786@o> Message-ID: <877i0c3p6a.fsf@benfinney.id.au> spir writes: > Ben Finney s'exprima ainsi: > > Simply break at the opening container character, and indent a single > > level to make all the contents line up:: > > > > try: > > my_final_result = finalResultComputer( > > arg1_from_abunch_of_args, > > args_should_align_properly, > > [so, what, if, they, re, compound]) > > except AttributeError, error: > > raise computeError( > > "foo ......... bar" > > "<--All message text lines should start here." > > % (String, interpolation, argument, list, as, well)) > > Yes, I partially agree with you indentation mode. But to my eyes > (maybe it's only me) the result (1) seems to suggest ordinary python > indentation, which it is not I beg to differ. Python indentation is inspired by the indentation that programmers do *anyway* to make the structure of their programs clearer. Some of it is syntactically significant, but that doesn't make the practice of indentation any less meaningful where it's not syntax. The indentation, in this case, is indicating that the indented lines are subordinate to the lines at an outer level of indentation. > and (2) does not make obvious what continuation lines belong to. I think it's obvious that the continuation lines ?belong to? the statement that begins on an outer level of indentation. This is, indeed, the entire point of indenting those continuation lines beneath the line which begins the statement, no? > This can be a bit improved (again, to my eyes) using > ~ double indentation for continuation lines > ~ closing sings (parens, etc) on their own line I have no objection to either of those, except for a slight waste of space for no benefit that I can appreciate. Certainly not egregious like the original example. -- \ ?Yesterday I told a chicken to cross the road. It said, ?What | `\ for??? ?Steven Wright | _o__) | Ben Finney From denis.spir at free.fr Wed May 20 15:29:36 2009 From: denis.spir at free.fr (spir) Date: Wed, 20 May 2009 15:29:36 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090520121006.GA6926@panix.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <20090520113013.23200e09@o> <87k54c3yeo.fsf@benfinney.id.au> <20090520121006.GA6926@panix.com> Message-ID: <20090520152936.36d883e2@o> Le Wed, 20 May 2009 05:10:08 -0700, Aahz s'exprima ainsi: > You'll also notice that similarly I put a trailing comma on > the last argument. Clever ;-) (I always get bitten) Denis ------ la vita e estrany From george.sakkis at gmail.com Wed May 20 15:44:46 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 20 May 2009 09:44:46 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87k54c3yeo.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <20090520113013.23200e09@o> <87k54c3yeo.fsf@benfinney.id.au> Message-ID: <91ad5bf80905200644x13d10a6ev90a83e6832ae1be0@mail.gmail.com> On Wed, May 20, 2009 at 5:40 AM, Ben Finney wrote: > spir writes: > >> ? ? ? ? ? ? try: >> ? ? ? ? ? ? ? ? my_final_result = finalResultComputer(arg1_from_abunch_of_args, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? args_should_align_properly, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [so, what, if, they, re, compound]) >> ? ? ? ? ? ? except AttributeError, error: >> ? ? ? ? ? ? ? ? raise computeError( "foo ......... ? ? ? ? bar" >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "<--All message text lines should start here." >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? %(String, interpolation, argument, list, as, well) ) > > It's for this reason that I advocate indenting continued lines *one* > level, and not this hideously large increase in indentation for a single > step. > > Simply break at the opening container character, and indent a single > level to make all the contents line up:: > > ? ?try: > ? ? ? ?my_final_result = finalResultComputer( > ? ? ? ? ? ?arg1_from_abunch_of_args, > ? ? ? ? ? ?args_should_align_properly, > ? ? ? ? ? ?[so, what, if, they, re, compound]) > ? ?except AttributeError, error: > ? ? ? ?raise computeError( > ? ? ? ? ? ?"foo ......... ? ? ? ? bar" > ? ? ? ? ? ?"<--All message text lines should start here." > ? ? ? ? ? ?% (String, interpolation, argument, list, as, well)) I do this sometimes, but only if not every argument fits between the opening parenthesis and the 80-char limit. Otherwise, I typically indent them at the "(" level, i.e.: try: my_final_result = finalResultComputer(somelongname, otherlongname, [some, subexpression]) except AttributeError, error: raise computeError("foo ......... bar", "<--some message" % (interpolation, args)) George From gerald.britton at gmail.com Wed May 20 17:17:14 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Wed, 20 May 2009 11:17:14 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <91ad5bf80905200644x13d10a6ev90a83e6832ae1be0@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <20090520113013.23200e09@o> <87k54c3yeo.fsf@benfinney.id.au> <91ad5bf80905200644x13d10a6ev90a83e6832ae1be0@mail.gmail.com> Message-ID: <5d1a32000905200817l203b0151x1871948caea511cd@mail.gmail.com> Also, it's convenient to remember that you can wrap any expression in parentheses which can help in situations like this: raise ( ValueError ), ( "what were you thinking?" ) Using this approach, you can indent (or not) however you like. It's a convenient way to break up long lines and you can get improved readability as a side-effect. On Wed, May 20, 2009 at 9:44 AM, George Sakkis wrote: > On Wed, May 20, 2009 at 5:40 AM, Ben Finney wrote: > >> spir writes: >> >>> ? ? ? ? ? ? try: >>> ? ? ? ? ? ? ? ? my_final_result = finalResultComputer(arg1_from_abunch_of_args, >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? args_should_align_properly, >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? [so, what, if, they, re, compound]) >>> ? ? ? ? ? ? except AttributeError, error: >>> ? ? ? ? ? ? ? ? raise computeError( "foo ......... ? ? ? ? bar" >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "<--All message text lines should start here." >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? %(String, interpolation, argument, list, as, well) ) >> >> It's for this reason that I advocate indenting continued lines *one* >> level, and not this hideously large increase in indentation for a single >> step. >> >> Simply break at the opening container character, and indent a single >> level to make all the contents line up:: >> >> ? ?try: >> ? ? ? ?my_final_result = finalResultComputer( >> ? ? ? ? ? ?arg1_from_abunch_of_args, >> ? ? ? ? ? ?args_should_align_properly, >> ? ? ? ? ? ?[so, what, if, they, re, compound]) >> ? ?except AttributeError, error: >> ? ? ? ?raise computeError( >> ? ? ? ? ? ?"foo ......... ? ? ? ? bar" >> ? ? ? ? ? ?"<--All message text lines should start here." >> ? ? ? ? ? ?% (String, interpolation, argument, list, as, well)) > > I do this sometimes, but only if not every argument fits between the > opening parenthesis and the 80-char limit. Otherwise, I typically > indent them at the "(" level, i.e.: > > ? ?try: > ? ? ? ?my_final_result = finalResultComputer(somelongname, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?otherlongname, > > [some, subexpression]) > ? ?except AttributeError, error: > ? ? ? ?raise computeError("foo ......... ? ? ? ? bar", > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "<--some message" % (interpolation, args)) > > George > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From python at rcn.com Wed May 20 19:44:15 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 20 May 2009 10:44:15 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><4A12F031.6060303@mrabarnett.plus.com><3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com><878wks5t4m.fsf@benfinney.id.au> Message-ID: <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> >> As pointed out elsewhere in this thread, very often the superior >> solution is not to re-*format* the same statement to fit, but to >> re-*factor* the code so it's less deeply indented or does less in each >> statement. > > Yes, "refactor" is a much better word than "reformat" for what I was thinking. So, it seems that support for an 80 character limit is rooted in a desire to have other people program differently than they do, somehow making their programs better just because they are adhering to an arbitrary limit on horizontal text width. That seems somewhat magical. Maybe the limit should be 40 chars, then everyone will have to refactor, and line wrap, and use smaller idents (like Google does), and use more abbreviated variable names. Maybe the origin of the 80 char limit isn't an anachronism. Maybe it had nothing to do with teletypes and low resolution CRTs. Perhaps, the programming gods of old were reaching into the future with certain knowledge that they were forcing everyone to do the right thing. Who knew? Raymond From drobinow at gmail.com Wed May 20 20:54:56 2009 From: drobinow at gmail.com (David Robinow) Date: Wed, 20 May 2009 14:54:56 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> <878wks5t4m.fsf@benfinney.id.au> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> Message-ID: <4eb0089f0905201154v461ed47at9f55434087af0dfd@mail.gmail.com> On Wed, May 20, 2009 at 1:44 PM, Raymond Hettinger wrote: > Maybe the origin of the 80 char limit isn't an anachronism. ?Maybe it had > nothing to do with teletypes and low resolution CRTs. ?Perhaps, the > programming gods of old were reaching into the future with certain > knowledge that they were forcing everyone to do the right thing. Actually, 80 is the number of columns in a Hollerith card. It would be totally stupid to write code that couldn't be punched! From george.sakkis at gmail.com Wed May 20 20:36:24 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 20 May 2009 14:36:24 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> <878wks5t4m.fsf@benfinney.id.au> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> Message-ID: <91ad5bf80905201136m2302da77o3e8ae1c4acf2787f@mail.gmail.com> On Wed, May 20, 2009 at 1:44 PM, Raymond Hettinger wrote: > >>> As pointed out elsewhere in this thread, very often the superior >>> solution is not to re-*format* the same statement to fit, but to >>> re-*factor* the code so it's less deeply indented or does less in each >>> statement. >> >> Yes, "refactor" is a much better word than "reformat" for what I was >> thinking. > > So, it seems that support for an 80 character limit is rooted in a desire > to have other people program differently than they do, somehow making > their programs better just because they are adhering to an arbitrary limit > on horizontal text width. ?That seems somewhat magical. ?Maybe the limit > should be 40 chars, then everyone will have to refactor, and line wrap, and > use smaller idents (like Google does), and use more abbreviated variable > names. > > Maybe the origin of the 80 char limit isn't an anachronism. ?Maybe it had > nothing to do with teletypes and low resolution CRTs. ?Perhaps, the > programming gods of old were reaching into the future with certain > knowledge that they were forcing everyone to do the right thing. > > Who knew? Or we can simply interpret the N-chars bound as a soft limit and feel free to exceed it by one character or three if it makes more sense. I am certainly not breaking a line for a single extra character. George From tjreedy at udel.edu Wed May 20 22:12:47 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 20 May 2009 16:12:47 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4eb0089f0905201154v461ed47at9f55434087af0dfd@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> <878wks5t4m.fsf@benfinney.id.au> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <4eb0089f0905201154v461ed47at9f55434087af0dfd@mail.gmail.com> Message-ID: David Robinow wrote: > On Wed, May 20, 2009 at 1:44 PM, Raymond Hettinger wrote: >> Maybe the origin of the 80 char limit isn't an anachronism. Maybe it had >> nothing to do with teletypes and low resolution CRTs. Perhaps, the >> programming gods of old were reaching into the future with certain >> knowledge that they were forcing everyone to do the right thing. > Actually, 80 is the number of columns in a Hollerith card. It would > be totally stupid to write code that couldn't be punched! It is also the maximum number of chars that one can sensibly type on US standard office paper. 8 " X 10 char/" + 1/4 inch margin on each side. (or nearly 1" margin with 12 pitch). I thought the stdlib guideline was so that people could print stdlib files. From steve at pearwood.info Thu May 21 01:02:47 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 21 May 2009 09:02:47 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> Message-ID: <200905210902.48106.steve@pearwood.info> On Thu, 21 May 2009 03:44:15 am Raymond Hettinger wrote: > >> As pointed out elsewhere in this thread, very often the superior > >> solution is not to re-*format* the same statement to fit, but to > >> re-*factor* the code so it's less deeply indented or does less in > >> each statement. > > > > Yes, "refactor" is a much better word than "reformat" for what I > > was thinking. > > So, it seems that support for an 80 character limit is rooted in a > desire to have other people program differently than they do, somehow > making their programs better just because they are adhering to an > arbitrary limit on horizontal text width. That seems somewhat > magical. Can I remind you that the 80 character limit is for the standard library, not your own personal code? Use whatever limit you like in your own personal libraries, or no limit at all. If your productivity is enhanced by writing 100-character lines, or 300 for that matter, then go right ahead. Some people find their productivity is enhanced with an 80 character limit. There's nothing "magical" about this -- we've given some reasons. 80 character lines fit comfortably in email and printed pages without wrapping, they can be viewed even on small screens without horizontal scrolling, and, yes, some people find that this constraint enhances our productivity and so we use the same limit in our own code, not just the stdlib. We're not forcing you to do the same. The 80 character limit is a lowest-common denominator. Having a 36" high-resolution monitor, or the visual acuity to read 7pt text, should not be a prerequisite for reading the stdlib. For this common code base, forcing everyone to limit line length is less of an imposition than forcing everyone to deal with long lines. > Maybe the limit should be 40 chars, then everyone will have > to refactor, and line wrap, and use smaller idents (like Google > does), and use more abbreviated variable names. No, 40 characters would be just foolish. The cost-benefit of constraints is not linear: there is a sweet-spot, where the cost is the minimum and the benefit is the maximum, and I believe that is around 80 characters, or possibly even a little shorter: I use a soft-limit of 70 characters in my own code, but I don't force that on anyone else. > Maybe the origin of the 80 char limit isn't an anachronism. Maybe it > had nothing to do with teletypes and low resolution CRTs. Perhaps, > the programming gods of old were reaching into the future with > certain knowledge that they were forcing everyone to do the right > thing. What possible relevance is the origin? Standards can change their justification over time. Commons used to exist so that the peasants would have somewhere to graze their goats. Not a lot of people have goats any more, but we still have commons, only now they're called "parks". If the character limit didn't remain relevant, we'd remove it, but it is relevant: people do print out code onto paper, people do still have poor eyesight requiring larger font sizes, or small monitors, or large monitors with four side-by-side editor windows. -- Steven D'Aprano From aaron.rubin at 4dtechnology.com Thu May 21 02:44:54 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Wed, 20 May 2009 17:44:54 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <200905210902.48106.steve@pearwood.info> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> Message-ID: <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> On Wed, May 20, 2009 at 4:02 PM, Steven D'Aprano wrote: > On Thu, 21 May 2009 03:44:15 am Raymond Hettinger wrote: > > >> As pointed out elsewhere in this thread, very often the superior > > >> solution is not to re-*format* the same statement to fit, but to > > >> re-*factor* the code so it's less deeply indented or does less in > > >> each statement. > > > > > > Yes, "refactor" is a much better word than "reformat" for what I > > > was thinking. > > > > So, it seems that support for an 80 character limit is rooted in a > > desire to have other people program differently than they do, somehow > > making their programs better just because they are adhering to an > > arbitrary limit on horizontal text width. That seems somewhat > > magical. > > Can I remind you that the 80 character limit is for the standard > library, not your own personal code? Use whatever limit you like in > your own personal libraries, or no limit at all. If your productivity > is enhanced by writing 100-character lines, or 300 for that matter, > then go right ahead. I see part of the problem now. People perceive the PEP-8 as the way they should write all code, not just the standard library. It seems to be passed around as the be-all-end-all, but in fact it might only represent what is good for the standard library and only the standard library. Perhaps commercial code bases, or other types of code bases should not blindly subscribe to this ideal. The 80 character width limit seems like this might be a candidate for consideration of flexibility depending on some critical factors. However, having a line width *limit* seems like a uniformly good one...to say which specific character to break up the lines might depend greatly (or solely) on the tools you (or your company) use(s). As time marches forward and email editors don't wrap (mine doesn't), printers are used less (which is already happening), etc. then standards for core libraries will probably change as well. Forward thinking is important and backwards compatibility is also important. Writing code at 80 characters takes more time, but it definitely ensures that any future standard will be compatible with it (i.e. a future 100 character width standard won't be offended by 80 character wrapped lines). So, at some point, I would predict 100 will become more prudent for standard libraries in Python and then (assuming the language is still thriving after many many years) it might become 120, etc. But it will just take time. Either way, the style for coding within a non-standard library might want to be revisited much sooner. In other words, programming to a standard which is common to all people who use it means you probably must accommodate the lowest common denominator. This would not be true for anything but the standard library. > > Some people find their productivity is enhanced with an 80 character > limit. There's nothing "magical" about this -- we've given some > reasons. 80 character lines fit comfortably in email and printed pages > without wrapping, they can be viewed even on small screens without > horizontal scrolling, and, yes, some people find that this constraint > enhances our productivity and so we use the same limit in our own code, > not just the stdlib. We're not forcing you to do the same. > > The 80 character limit is a lowest-common denominator. Having a 36" > high-resolution monitor, or the visual acuity to read 7pt text, should > not be a prerequisite for reading the stdlib. For this common code > base, forcing everyone to limit line length is less of an imposition > than forcing everyone to deal with long lines. > > > > Maybe the limit should be 40 chars, then everyone will have > > to refactor, and line wrap, and use smaller idents (like Google > > does), and use more abbreviated variable names. > > No, 40 characters would be just foolish. The cost-benefit of constraints > is not linear: there is a sweet-spot, where the cost is the minimum and > the benefit is the maximum, and I believe that is around 80 characters, > or possibly even a little shorter: I use a soft-limit of 70 characters > in my own code, but I don't force that on anyone else. > > > > Maybe the origin of the 80 char limit isn't an anachronism. Maybe it > > had nothing to do with teletypes and low resolution CRTs. Perhaps, > > the programming gods of old were reaching into the future with > > certain knowledge that they were forcing everyone to do the right > > thing. > > What possible relevance is the origin? Standards can change their > justification over time. Commons used to exist so that the peasants > would have somewhere to graze their goats. Not a lot of people have > goats any more, but we still have commons, only now they're > called "parks". > > If the character limit didn't remain relevant, we'd remove it, but it is > relevant: people do print out code onto paper, people do still have > poor eyesight requiring larger font sizes, or small monitors, or large > monitors with four side-by-side editor windows. This is a minor point, but I would argue that the origin is relevant. It helps to realize what assumptions are being made about the code base, which may or may not be true anymore. In the case of a "park", it is obvious that the origin was questioned at some point, since "parks" needn't have grass anymore. (where grass=80 character limit width) > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Thu May 21 03:02:54 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 20 May 2009 18:02:54 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> Message-ID: <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> > If the character limit didn't remain relevant, we'd remove it, My suggestion is to just bump it up a bit (perhaps to 100 chars). Docstrings and code for method definitions in Python typically start at two tabs deep. That means that most of the body of a program is in a somewhat narrow band. This most troublesome with string literals embedded in the code and with trailing comments. IMO, if a string literal is less than 80 chars, then cutting it midstring is an anti-pattern. Breaking the literals is awkward and uncomfortable. This seems to come-up often with error messages which seem to usually be defined several levels deep so that there isn't much space left for the message. class X: def Y(self): 'Comment' for something in Z: if somecondition: raise Exception(some_message % somevalue) Another problematic area is with unittests: class TestX(unittest.testcase): def test_z(self): self.assertRaises(TypeError, real_code_starts_here) Both of those situations seem to lead to awkward line wrapping right in the middle of a literal or expression. It both cases, the literal or expression of interest starts without a whole lot of room left. Sometimes we're able to fit it in gracefully, sometimes not. > people do print out code onto paper My experience with printing code is that vertical space is the main challenge. The unittest module for example takes at least 25 pages depending on how much you're willing to give-up margins and large type. > Some people find their productivity is enhanced with an 80 character > limit. Perhaps this is true, though I've not heard one jot of evidence to support it and certainly no evidence focusing on Python coding practices which are affected by the use of whitespace for control flow. Nor have I seen evidence of comparative productivity of 80 char limits versus 100 char limits. All we have is anecdotal evidence and personal tastes. What I have seen is a company where all of the Python programmers routinely moved into the 100 character range when necessary and it worked out fine. Also, I've seen plenty of code be made "pep-8 compliant" and not look any better for the effort. Raymond From greg.ewing at canterbury.ac.nz Thu May 21 03:05:03 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 21 May 2009 13:05:03 +1200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> Message-ID: <4A14A8BF.3080403@canterbury.ac.nz> Aaron Rubin wrote: > I see part of the problem now. People perceive the PEP-8 as the way > they should write all code, not just the standard library. I think they're reasonable guidelines to follow for code that you intend to publicise, for the same reasons that they're good for the stdlib. But nobody is going to beat you up with a stick if you don't follow them to the letter. > In the case of a "park", it is > obvious that the origin was questioned at some point, since "parks" > needn't have grass anymore. (where grass=80 character limit width) Yes, but the relevant question remains "Do we still need it, and if so, why?", not "It's old, so it must be out of date by now, surely?" -- Greg From greg.ewing at canterbury.ac.nz Thu May 21 01:19:04 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 21 May 2009 11:19:04 +1200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090520114111.6e670e49@o> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> <20090519192022.4e47f89d@bhuda.mired.org> <20090520114111.6e670e49@o> Message-ID: <4A148FE8.70407@canterbury.ac.nz> spir wrote: > On the other side, the Oulipo school of writing believes that writing with > apparently arbitrary constraints improves the results. > > Oulipo games are about helping *creativity*. The best explanation I've seen for this phenomenon is that it works by forcing you to avoid cliches. For example, if you have to fit your words into a fixed meter, you can't just use the first phrasing that comes into your head. You have to hunt around for alternative words that fit the pattern, and in the process you most likely come up with something original and surprising. I don't think this applies in the same way when you're writing a program. The goal there is *not* to be original and surprising -- if anything it's the opposite! You want to convey the meaning of the code to the reader as clearly as possible, and if it uses an idiom that the reader has seen before and can instantly recognise, then so much the better. -- Greg From greg.ewing at canterbury.ac.nz Thu May 21 03:09:21 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 21 May 2009 13:09:21 +1200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> Message-ID: <4A14A9C1.5070306@canterbury.ac.nz> Raymond Hettinger wrote: > My suggestion is to just bump it up a bit (perhaps to 100 chars). If there's anything I'd suggest changing, it would be to reduce the recommended indentation from 4 chars to 2. That's what I use internally for my own code, and it seems to be a good compromise between readability and eating up too much width with indentation. -- Greg From stephen at xemacs.org Thu May 21 03:23:34 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 21 May 2009 10:23:34 +0900 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4eb0089f0905201154v461ed47at9f55434087af0dfd@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <4A12F031.6060303@mrabarnett.plus.com> <3bdda690905191753n683f1603i388ac1539e135e95@mail.gmail.com> <878wks5t4m.fsf@benfinney.id.au> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <4eb0089f0905201154v461ed47at9f55434087af0dfd@mail.gmail.com> Message-ID: <87hbzffdux.fsf@uwakimon.sk.tsukuba.ac.jp> David Robinow writes: > Actually, 80 is the number of columns in a Hollerith card. It would > be totally stupid to write code that couldn't be punched! That's right. After all, punching the author of the code will get you locked up, no matter how richly he deserves it! From google at mrabarnett.plus.com Thu May 21 03:22:19 2009 From: google at mrabarnett.plus.com (MRAB) Date: Thu, 21 May 2009 02:22:19 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A14A9C1.5070306@canterbury.ac.nz> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <4A14A9C1.5070306@canterbury.ac.nz> Message-ID: <4A14ACCB.6010400@mrabarnett.plus.com> Greg Ewing wrote: > Raymond Hettinger wrote: > >> My suggestion is to just bump it up a bit (perhaps to 100 chars). > > If there's anything I'd suggest changing, it would be to > reduce the recommended indentation from 4 chars to 2. > That's what I use internally for my own code, and it > seems to be a good compromise between readability and > eating up too much width with indentation. > I find an indentation of 4 characters to be a good compromise in a language like Python where indentation is important. From mwm-keyword-python.b4bdba at mired.org Thu May 21 03:30:13 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Wed, 20 May 2009 21:30:13 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> Message-ID: <20090520213013.2dfc48f2@bhuda.mired.org> On Wed, 20 May 2009 17:44:54 -0700 Aaron Rubin wrote: > On Wed, May 20, 2009 at 4:02 PM, Steven D'Aprano wrote: > As time marches forward and email editors don't wrap (mine doesn't), That makes your mail editor a throwback. Used to be, none of them wrapped. Then an 800 lb gorilla of a company decided to ignore what the RFCs said, and made their mailer act like there WP products and wrap text. It's hard to find a modern mailer that doesn't wrap. > printers are used less (which is already happening), etc. then standards for > core libraries will probably change as well. But PDAs and Netbooks are being used more and more. > Either way, the style for coding within a non-standard library might want to > be revisited much sooner. In other words, programming to a standard which > is common to all people who use it means you probably must accommodate the > lowest common denominator. This would not be true for anything but the > standard library. Um, I'd say it holds for pretty much every other open source project done in Python, excepting those that are targeted for some specific platform. I.e. - if you're writing scripts for a high-end CAD program, then you get one set of assumptions about the working environment. But if you're developing on and for N60, you get a completely different set. The problem is that on it's good days, this is a bikeshed issue. On it's bad day out and out religious. Choosing to follow PEP-8 so as to not waste time discussing it is a rational choice for any group working in Python. It's certainly better than no standards at all! http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From aaron.rubin at 4dtechnology.com Thu May 21 03:55:11 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Wed, 20 May 2009 18:55:11 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090520213013.2dfc48f2@bhuda.mired.org> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <20090520213013.2dfc48f2@bhuda.mired.org> Message-ID: <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> On Wed, May 20, 2009 at 6:30 PM, Mike Meyer wrote: > On Wed, 20 May 2009 17:44:54 -0700 > Aaron Rubin wrote: > > On Wed, May 20, 2009 at 4:02 PM, Steven D'Aprano >wrote: > > As time marches forward and email editors don't wrap (mine doesn't), > > That makes your mail editor a throwback. Used to be, none of them > wrapped. Then an 800 lb gorilla of a company decided to ignore what > the RFCs said, and made their mailer act like there WP products and > wrap text. It's hard to find a modern mailer that doesn't wrap. I use gmail. > > > > printers are used less (which is already happening), etc. then standards > for > > core libraries will probably change as well. > > But PDAs and Netbooks are being used more and more. You can't possibly make a case that programming on either of these devices should be what we should cater to. I want to program on my Palm phone as well, but I have to accept that the world won't try to accommodate me, but the other way around. > > > > Either way, the style for coding within a non-standard library might want > to > > be revisited much sooner. In other words, programming to a standard > which > > is common to all people who use it means you probably must accommodate > the > > lowest common denominator. This would not be true for anything but the > > standard library. > > Um, I'd say it holds for pretty much every other open source project > done in Python, excepting those that are targeted for some specific > platform. I.e. - if you're writing scripts for a high-end CAD program, > then you get one set of assumptions about the working environment. But > if you're developing on and for N60, you get a completely different > set. I am trying to discern between a code base which must cater to all, and a code base which mustn't. An open source project is akin to a widely used closed source project. What they share is they don't need to cater to *the* least common denominator, but instead *a* least common denominator, which gives them more flexibility. > > > The problem is that on it's good days, this is a bikeshed issue. On > it's bad day out and out religious. Choosing to follow PEP-8 so as to > not waste time discussing it is a rational choice for any group > working in Python. It's certainly better than no standards at all! But choosing to pinpoint particular aspects which do not appear to make much sense and attempt to further understand the rationale behind it, so as to assess whether it is worth it to follow those aspects is very worthwhile. Whether the entire community accepts it is another thing. The entire community appears to be quite divided on this issue and rightfully so...we appear to be in the middle of change :) Otherwise, there wouldn't be so much debate on the topic. > > -- > Mike Meyer > http://www.mired.org/consulting.html > Independent Network/Unix/Perforce consultant, email for more information. > > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dstanek at dstanek.com Thu May 21 04:25:27 2009 From: dstanek at dstanek.com (David Stanek) Date: Wed, 20 May 2009 22:25:27 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <20090520213013.2dfc48f2@bhuda.mired.org> <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> Message-ID: On Wed, May 20, 2009 at 9:55 PM, Aaron Rubin wrote: > On Wed, May 20, 2009 at 6:30 PM, Mike Meyer wrote: >> >> On Wed, 20 May 2009 17:44:54 -0700 >> Aaron Rubin wrote: >> >> > printers are used less (which is already happening), etc. then standards >> > for >> > core libraries will probably change as well. >> >> But PDAs and Netbooks are being used more and more. > > You can't possibly make a case that programming on either of these devices > should be what we should cater to. ?I want to program on my Palm phone as > well, but I have to accept that the world won't try to accommodate me, but > the other way around. > I like programming on my EEE. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek From aaron.rubin at 4dtechnology.com Thu May 21 04:32:47 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Wed, 20 May 2009 19:32:47 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <20090520213013.2dfc48f2@bhuda.mired.org> <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> Message-ID: <985db6080905201932g3a9f9288t1cc50952abad255f@mail.gmail.com> On Wed, May 20, 2009 at 7:25 PM, David Stanek wrote: > On Wed, May 20, 2009 at 9:55 PM, Aaron Rubin > wrote: > > On Wed, May 20, 2009 at 6:30 PM, Mike Meyer wrote: > >> > >> On Wed, 20 May 2009 17:44:54 -0700 > >> Aaron Rubin wrote: > >> > >> > printers are used less (which is already happening), etc. then > standards > >> > for > >> > core libraries will probably change as well. > >> > >> But PDAs and Netbooks are being used more and more. > > > > You can't possibly make a case that programming on either of these > devices > > should be what we should cater to. I want to program on my Palm phone as > > well, but I have to accept that the world won't try to accommodate me, > but > > the other way around. > > > > I like programming on my EEE. I have an EEE as well. But I don't expect the community to cater to me. > > > -- > David > blog: http://www.traceback.org > twitter: http://twitter.com/dstanek > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dstanek at dstanek.com Thu May 21 04:37:43 2009 From: dstanek at dstanek.com (David Stanek) Date: Wed, 20 May 2009 22:37:43 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905201932g3a9f9288t1cc50952abad255f@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <20090520213013.2dfc48f2@bhuda.mired.org> <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> <985db6080905201932g3a9f9288t1cc50952abad255f@mail.gmail.com> Message-ID: On Wed, May 20, 2009 at 10:32 PM, Aaron Rubin wrote: > On Wed, May 20, 2009 at 7:25 PM, David Stanek wrote: >> >> On Wed, May 20, 2009 at 9:55 PM, Aaron Rubin >> wrote: >> > On Wed, May 20, 2009 at 6:30 PM, Mike Meyer wrote: >> >> >> >> On Wed, 20 May 2009 17:44:54 -0700 >> >> Aaron Rubin wrote: >> >> >> >> > printers are used less (which is already happening), etc. then >> >> > standards >> >> > for >> >> > core libraries will probably change as well. >> >> >> >> But PDAs and Netbooks are being used more and more. >> > >> > You can't possibly make a case that programming on either of these >> > devices >> > should be what we should cater to. ?I want to program on my Palm phone >> > as >> > well, but I have to accept that the world won't try to accommodate me, >> > but >> > the other way around. >> > >> >> I like programming on my EEE. > > I have an EEE as well. ?But I don't expect the community to cater to me. > I don't think I'm expecting to be catered to because of the netbook. To me 80 just works well and is in the current PEP. So I'm happy. I just think that the people that say using 80 characters takes more time is using a very crappy editor. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek From mwm-keyword-python.b4bdba at mired.org Thu May 21 04:37:57 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Wed, 20 May 2009 22:37:57 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <20090520213013.2dfc48f2@bhuda.mired.org> <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> Message-ID: <20090520223757.0399a0a1@bhuda.mired.org> On Wed, 20 May 2009 18:55:11 -0700 Aaron Rubin wrote: > > > printers are used less (which is already happening), etc. then standards > > for > > > core libraries will probably change as well. > > But PDAs and Netbooks are being used more and more. > You can't possibly make a case that programming on either of these devices > should be what we should cater to. I want to program on my Palm phone as > well, but I have to accept that the world won't try to accommodate me, but > the other way around. Nope, you don't make things more difficult for everybody else just to make things easier for them. On the other hand, you can't possibly disenfranchise them, either - which means that a change that makes things harder on people using them needs to benefit everyone else enough to make up for it. > > > Either way, the style for coding within a non-standard library might want > > to > > > be revisited much sooner. In other words, programming to a standard > > which > > > is common to all people who use it means you probably must accommodate > > the > > > lowest common denominator. This would not be true for anything but the > > > standard library. > > Um, I'd say it holds for pretty much every other open source project > > done in Python, excepting those that are targeted for some specific > > platform. I.e. - if you're writing scripts for a high-end CAD program, > > then you get one set of assumptions about the working environment. But > > if you're developing on and for N60, you get a completely different > > set. > I am trying to discern between a code base which must cater to all, and a > code base which mustn't. An open source project is akin to a widely used > closed source project. What they share is they don't need to cater to *the* > least common denominator, but instead *a* least common denominator, which > gives them more flexibility. But the PSL is just another open source project. It has no requirement to cater to all, or to *the* lcd, any more than any other such project. In fact, it clearly doesn't, as it certainly never catered to people working in the Python implementation for the old Palm series III devices, or the N60 implementation, etc. > > The problem is that on it's good days, this is a bikeshed issue. On > > it's bad day out and out religious. Choosing to follow PEP-8 so as to > > not waste time discussing it is a rational choice for any group > > working in Python. It's certainly better than no standards at all! > But choosing to pinpoint particular aspects which do not appear to make much > sense and attempt to further understand the rationale behind it, so as to > assess whether it is worth it to follow those aspects is very worthwhile. > Whether the entire community accepts it is another thing. The entire > community appears to be quite divided on this issue and rightfully so...we > appear to be in the middle of change :) Otherwise, there wouldn't be so > much debate on the topic. Oh? Doesn't seem to be much different than last time (I saw it, anyway - it may have happened again during a hiatus). Lots of people with strong personal preferences, but the best real reason around is "we've always done it that way". Of course, everyone also seems to be forgetting one of the most important points in the PEP: But most importantly: know when to be inconsistent -- sometimes the style guide just doesn't apply. When in doubt, use your best judgment. Look at other examples and decide what looks best. And don't hesitate to ask! But why not help things along even further, by cutting the length of what's probably the most common variable name in python by 50%, and replace "self" with "my". We can't shorten cls, but we can bring it into the same part of speach by making it "our". http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From ben+python at benfinney.id.au Thu May 21 04:41:20 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 May 2009 12:41:20 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> Message-ID: <87vdnv2n5b.fsf@benfinney.id.au> Aaron Rubin writes: > I see part of the problem now. People perceive the PEP-8 as the way > they should write all code, not just the standard library. It seems to > be passed around as the be-all-end-all, but in fact it might only > represent what is good for the standard library and only the standard > library. Perhaps. Or perhaps those style constraints were chosen for the standard library *because* they're widely applicable and make sense for Python code in general. > As time marches forward and email editors don't wrap (mine doesn't), > printers are used less (which is already happening), etc. then > standards for core libraries will probably change as well. Forward > thinking is important and backwards compatibility is also important. Indeed. Such forward thinking must take into account that devices with smaller and smaller displays are being used for viewing documents, including code. > Writing code at 80 characters takes more time, but it definitely > ensures that any future standard will be compatible with it (i.e. a > future 100 character width standard won't be offended by 80 character > wrapped lines). So, at some point, I would predict 100 will become > more prudent for standard libraries in Python and then (assuming the > language is still thriving after many many years) it might become 120, > etc. But it will just take time. You assume that human eyesight and capacity to comprehend complex syntax will also scale with the display resolution. That doesn't seem a well-supported assumption. I think average human ability to perceive lines of text, having remained pretty much constant over the history of computing so far, will tend to remain fairly constant for the foreseeable future, and hence the 80-column limit will continue to be a good standard for Python code. -- \ ?Are you thinking what I'm thinking, Pinky?? ?Uh... yeah, | `\ Brain, but where are we going to find rubber pants our size?? | _o__) ?_Pinky and The Brain_ | Ben Finney From tleeuwenburg at gmail.com Thu May 21 04:57:14 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Thu, 21 May 2009 12:57:14 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87vdnv2n5b.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <87vdnv2n5b.fsf@benfinney.id.au> Message-ID: <43c8685c0905201957h35c217e3kf67345a0418bac2c@mail.gmail.com> For what it's worth, I'm in favour of sticking with the current 80 character limit. It's really not hard to work with or accommodate. In my workplace, I go much wider, but we all have standard environments and it's no big deal. 80 characters looks narrow to me, but I can adjust, and it makes sense to use this as a common denominator for a widely-shared piece of code like the Python core libraries. As everyone has said, it works well in emails, funny terminals, vi, emacs, IDEs, can be easily used side-by-side with other open editors, etc. These 'special cases' seem very remote until you are suddenly confronted by them. For me, the most common is when telnetting / sshing into a foreign machine which assumes an 80-character width and the terminal emulation setting is a bit mucked up, so all of a sudden you can't resize to make use of your screen real estate. Trying to read code over such a link, or through a funny email client which wraps long lines, is awful and I'm happy to accommodate those people. Working with the 80-character limit is really easy in my editor, and it's certainly much easier to fit in with the existing style of a shared codebase than to have idiosyncrasies within sections that have been edited here and there. I don't think the 80-character limit in PEP-8 is doing many people a great deal of harm. Cheers, -T -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Thu May 21 05:16:09 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 20 May 2009 20:16:09 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><2690E090A497426AA35558462DB8FC2A@RaymondLaptop1><200905210902.48106.steve@pearwood.info><985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com><87vdnv2n5b.fsf@benfinney.id.au> <43c8685c0905201957h35c217e3kf67345a0418bac2c@mail.gmail.com> Message-ID: <3F08A7AC476F4E8A868F4731328FC564@RaymondLaptop1> > I don't think the 80-character limit in PEP-8 is doing many people a great deal of harm. FWIW, I posted two recurring examples of where it is awkward in normal python programming. One, raising exceptions with messages -- the message doesn't typically start until 30th column or so. Two, in unittests using self.assertRaises(SomeException, expr), the expr part doesn't typically start until the 35th column or so. Also, there is a problem with indented multiline string literals where the indentation is taking up space that will never be shown to the end-user. Indented comments and in-line comments are also space challenged. I think the Google example is instructive. Their two space tabs are not a natural preference for anyone I know. Instead, it is an accommodation born from trying to fit nested, indented code into 80 columns. I'm pretty much in agreement with the OP that going to 90 or 100 columns is a better solution. Of course, if Hollerith punch cards come back into vogue, then the OP's proposal won't work out so well ;-) Raymond From ben+python at benfinney.id.au Thu May 21 05:45:32 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 May 2009 13:45:32 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <87vdnv2n5b.fsf@benfinney.id.au> <43c8685c0905201957h35c217e3kf67345a0418bac2c@mail.gmail.com> <3F08A7AC476F4E8A868F4731328FC564@RaymondLaptop1> Message-ID: <87my972k6b.fsf@benfinney.id.au> "Raymond Hettinger" writes: > FWIW, I posted two recurring examples of where it is awkward in normal > python programming. One, raising exceptions with messages -- the > message doesn't typically start until 30th column or so. As was covered earlier in the thread, cases like that are why the following style makes sense for call parameters:: def error_emitter(): raise some.name.space.LongDescriptiveError( "Message for the error" " which can cross multiple lines if necessary", other, args) > Two, in unittests using self.assertRaises(SomeException, expr), the > expr part doesn't typically start until the 35th column or so. class FrobnicationTest(unittest.TestCase): def test_frobnicates_the_spangle(self): self.assertRaises( SomeException, expr) So neither of these use cases is materially harmed by an 80-column limit. > I think the Google example is instructive. Their two space tabs are > not a natural preference for anyone I know. Instead, it is an > accommodation born from trying to fit nested, indented code into 80 > columns. I'm pretty much in agreement with the OP that going to 90 or > 100 columns is a better solution. Given the available options for making code fit nicely within 80-column lines and 4-column indentation levels, and the benefits to be had from doing so, I don't agree with your position. -- \ ?Never express yourself more clearly than you are able to | `\ think.? ?Niels Bohr | _o__) | Ben Finney From cs at zip.com.au Thu May 21 07:01:55 2009 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 21 May 2009 15:01:55 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <3F08A7AC476F4E8A868F4731328FC564@RaymondLaptop1> Message-ID: <20090521050155.GA14128@cskk.homeip.net> On 20May2009 20:16, Raymond Hettinger wrote: > I think the Google example is instructive. Their two space tabs are not a natural > preference for anyone I know. I regret to inform you that two space tabs are my preferred indentation. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ To be positive: To be mistaken at the top of one's voice. Ambrose Bierce (1842-1914), U.S. author. The Devil's Dictionary (1881-1906). From denis.spir at free.fr Thu May 21 09:06:55 2009 From: denis.spir at free.fr (spir) Date: Thu, 21 May 2009 09:06:55 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> Message-ID: <20090521090655.7695a0c5@o> Le Wed, 20 May 2009 18:02:54 -0700, "Raymond Hettinger" s'exprima ainsi: > > If the character limit didn't remain relevant, we'd remove it, (to Steven) We definitely don't live in the same world ;-) In mine, any purely legacy trait change raises religious wars. People fighting against change _always_ find "good" reasons. That seems to be part of human nature (in the world I live in). > My suggestion is to just bump it up a bit (perhaps to 100 chars). > Docstrings and code for method definitions in Python typically > start at two tabs deep. That means that most of the body of > a program is in a somewhat narrow band. This most troublesome > with string literals embedded in the code and with trailing comments. > IMO, if a string literal is less than 80 chars, then cutting it midstring > is an anti-pattern. Breaking the literals is awkward and uncomfortable. > This seems to come-up often with error messages which seem to > usually be defined several levels deep so that there isn't much space > left for the message. We could differenciate between overall line width and "busy" line width (from fist non-white). The latter may have a (recommanded) limit less than 80 characters. It should not be too wide for human overall comprehension (but optimum may vary greatly). Except for literals and probably some other issue (e.g. imported names*). About the former, we probably could define algorithms that nicely reformat code in a sensible manner, for printing or whatever physical constraint. Denis (*) Someone suggested to "import ... as ..." but in many cases it's wrong because other programmers cannot rely anymore on their knowledge of libraries. ------ la vita e estrany From denis.spir at free.fr Thu May 21 09:20:54 2009 From: denis.spir at free.fr (spir) Date: Thu, 21 May 2009 09:20:54 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87vdnv2n5b.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <87vdnv2n5b.fsf@benfinney.id.au> Message-ID: <20090521092054.0411b56c@o> Le Thu, 21 May 2009 12:41:20 +1000, Ben Finney s'exprima ainsi: > You assume that human eyesight and capacity to comprehend complex syntax > will also scale with the display resolution. That doesn't seem a > well-supported assumption. > > I think average human ability to perceive lines of text, having remained > pretty much constant over the history of computing so far, will tend to > remain fairly constant for the foreseeable future, and hence the > 80-column limit will continue to be a good standard for Python code. You are certainly right here, but this applies to "busy" line-width, which is most often < 50 chars (left-side spacing excluded), even when overall line width is > 80. Breaking lines that sensibly mirror the semantics and hold less than 50 content characters, only because of an arbitrary right-side limit, prevents confortable reading. Also, most wider lines go over the limit because they contain long literals (strings, lists) which are not complex conceptually -- they just eat horizontal space. Breaking the line in such cases hinders human parsing, it does not help it. To sum up, the human parsing factor rather requires intelligently relaxing limits, not the contrary. Denis ------ la vita e estrany ------ la vita e estrany From denis.spir at free.fr Thu May 21 09:26:57 2009 From: denis.spir at free.fr (spir) Date: Thu, 21 May 2009 09:26:57 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A14ACCB.6010400@mrabarnett.plus.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <4A14A9C1.5070306@canterbury.ac.nz> <4A14ACCB.6010400@mrabarnett.plus.com> Message-ID: <20090521092657.650f0421@o> Le Thu, 21 May 2009 02:22:19 +0100, MRAB s'exprima ainsi: > > reduce the recommended indentation from 4 chars to 2. > > That's what I use internally for my own code, and it > > seems to be a good compromise between readability and > > eating up too much width with indentation. > > > I find an indentation of 4 characters to be a good compromise in a > language like Python where indentation is important. I do, too. 3 is the very minimum for my eyes to vertically parse code structure. 2 is definitely too narrow for me, even with rather short blocks. Denis ------ la vita e estrany From scott+python-ideas at scottdial.com Thu May 21 09:39:43 2009 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Thu, 21 May 2009 03:39:43 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A148FE8.70407@canterbury.ac.nz> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> <20090519192022.4e47f89d@bhuda.mired.org> <20090520114111.6e670e49@o> <4A148FE8.70407@canterbury.ac.nz> Message-ID: <4A15053F.9050807@scottdial.com> Greg Ewing wrote: > spir wrote: >> On the other side, the Oulipo school of writing believes that writing >> with >> apparently arbitrary constraints improves the results. >> >> Oulipo games are about helping *creativity*. > > I don't think this applies in the same way when you're > writing a program. The goal there is *not* to be original > and surprising -- if anything it's the opposite! You > want to convey the meaning of the code to the reader as > clearly as possible, and if it uses an idiom that the > reader has seen before and can instantly recognise, then > so much the better. And therein, you have defeated your own argument. Given the large body of code that already follows PEP8 (and other style guides for other languages that commonly use an 80-character boundary), it is a common constraint which yields many common idioms (such as placing list items on separate lines with similar indention). The readers wished hard For the thread to die quickly Sadly it goes on -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From lie.1296 at gmail.com Thu May 21 10:04:55 2009 From: lie.1296 at gmail.com (Lie Ryan) Date: Thu, 21 May 2009 18:04:55 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > >> Some people find their productivity is enhanced with an 80 character >> limit. > > Perhaps this is true, though I've not heard one jot of evidence to > support it > and certainly no evidence focusing on Python coding practices which are > affected by the use of whitespace for control flow. Nor have I seen > evidence > of comparative productivity of 80 char limits versus 100 char limits. > All we > have is anecdotal evidence and personal tastes. This is how I will write program if we have 160 character limit: def foo(f, a, b, c): return [[((f(x, y) * i, i) if i % 2 else 0) for i, x in enumerate(a) if f(y, x) == a + x] for y in [c(z) for z in range(a, a * b + c, c)]] except there will be no line breaks... I love list comprehension so much that I often _unconsciously_ write a very complex list comprehensions. 80-character convention acts as a reminder to consider refactoring. From jeremiah.dodds at gmail.com Thu May 21 10:39:15 2009 From: jeremiah.dodds at gmail.com (Jeremiah Dodds) Date: Thu, 21 May 2009 09:39:15 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> Message-ID: <12cbbbfc0905210139r7d386b6eg70c9616379c7b9bc@mail.gmail.com> On Thu, May 21, 2009 at 9:04 AM, Lie Ryan wrote: > Raymond Hettinger wrote: > > >> Some people find their productivity is enhanced with an 80 character >>> limit. >>> >> >> Perhaps this is true, though I've not heard one jot of evidence to support >> it >> and certainly no evidence focusing on Python coding practices which are >> affected by the use of whitespace for control flow. Nor have I seen >> evidence >> of comparative productivity of 80 char limits versus 100 char limits. All >> we >> have is anecdotal evidence and personal tastes. >> > > This is how I will write program if we have 160 character limit: > > def foo(f, a, b, c): > return [[((f(x, y) * i, i) if i % 2 else 0) for i, x in enumerate(a) if > f(y, x) == a + x] for y in [c(z) for z in range(a, a * b + c, c)]] > > except there will be no line breaks... > > I love list comprehension so much that I often _unconsciously_ write a very > complex list comprehensions. 80-character convention acts as a reminder to > consider refactoring. > > hmm, the 80-character convention does not stop me from unconsciously writing really complex list comprehensions, I just write them like so: def foo(f, a, b, c): return [[((f(x,y) * i, i) if i % 2 else 0) for i, x in enumerate(a) if f(y, x) == a + x] for y in [c(z) for z in range(a, a*b+c, c)]] not that that's really any better. I really don't mind the 80-character convention. I occasionally have problems with going over it, but most of the time it serves (as you point out) as a flag for something that probably needs refactoring. When I do go over it, it's normally only by a few chars. The other benefits of it, for me, is easily reading code in a terminal in the few cases I end up without X , and for opening multiple files in emacs with vertical-split. The latter is a huge benefit, and could still be done if the convention was upped to, say, 100 chars - but would start to get annoying with wrapping shortly after that. But yeah, this is all anectodal evidence and personal taste, as Raymond points out. The 80 char limit _had_ a practical reason for existing, and I don't doubt that there are people out there that are still using the (hard|soft)ware that caused the limit to be practical, but they're probably few and far between. I also haven't seen anything other than anectodal evidence and personal taste in favor of increasing the limit, which suggests to me that it should just stay how it is. It is just a convention after all. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Thu May 21 11:09:04 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 21 May 2009 18:09:04 +0900 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <985db6080905201744s1ffa4f69l57fb27c6f5a4b984@mail.gmail.com> <20090520213013.2dfc48f2@bhuda.mired.org> <985db6080905201855r5b3d0ca8rea47633a9cf58ff3@mail.gmail.com> Message-ID: <87eiuig6vj.fsf@uwakimon.sk.tsukuba.ac.jp> Aaron Rubin writes: > Whether the entire community accepts it is another thing. The entire > community appears to be quite divided on this issue and rightfully so...we > appear to be in the middle of change :) Nah. Plus ?a change plus c'est la m?me chose. Excuse my French. From ben+python at benfinney.id.au Thu May 21 12:01:13 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 21 May 2009 20:01:13 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <12cbbbfc0905210139r7d386b6eg70c9616379c7b9bc@mail.gmail.com> Message-ID: <874ove3hcm.fsf@benfinney.id.au> Jeremiah Dodds writes: > hmm, the 80-character convention does not stop me from unconsciously > writing really complex list comprehensions, I just write them like so: > > def foo(f, a, b, c): > return [[((f(x,y) * i, i) if i % 2 else 0) > for i, x in enumerate(a) > if f(y, x) == a + x] > for y in [c(z) for z in range(a, a*b+c, c)]] > > not that that's really any better. On the contrary, I find that *much* easier to grasp than the same statement on a single line. You have been required, by choosing to follow the 80-column limit, to choose points at which to break the line; and have responded by breaking it up into conceptually discrete chunks and indented to suggest the structure. This example is, for me, a very convincing (anecdotal) demonstration of why an 80-column limit is a good constraint to follow. > But yeah, this is all anectodal evidence and personal taste, as > Raymond points out. Yet it's also more than that; to call it ?personal taste? is to imply that it's nothing more than aesthetics. If that's all it were, I'd care far less for changes in convention. I consider it rather more importantly a matter of software ergonomics, which should therefore not be changed unless there's good supporting evidence that the proposed change results in improvement. -- \ ?A cynic is a man who knows the price of everything and the | `\ value of nothing.? ?Oscar Wilde | _o__) | Ben Finney From google at mrabarnett.plus.com Thu May 21 12:49:44 2009 From: google at mrabarnett.plus.com (MRAB) Date: Thu, 21 May 2009 11:49:44 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090521090655.7695a0c5@o> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> Message-ID: <4A1531C8.7020900@mrabarnett.plus.com> spir wrote: > Le Wed, 20 May 2009 18:02:54 -0700, > "Raymond Hettinger" s'exprima ainsi: > >>> If the character limit didn't remain relevant, we'd remove it, > > (to Steven) We definitely don't live in the same world ;-) > In mine, any purely legacy trait change raises religious wars. People fighting against change _always_ find "good" reasons. That seems to be part of human nature (in the world I live in). > [snip] "If it's not necessary to change then it's necessary not to change." From rrr at ronadam.com Thu May 21 17:07:17 2009 From: rrr at ronadam.com (Ron Adam) Date: Thu, 21 May 2009 10:07:17 -0500 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <12cbbbfc0905210139r7d386b6eg70c9616379c7b9bc@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <12cbbbfc0905210139r7d386b6eg70c9616379c7b9bc@mail.gmail.com> Message-ID: <4A156E25.4070907@ronadam.com> Jeremiah Dodds wrote: > > > On Thu, May 21, 2009 at 9:04 AM, Lie Ryan > wrote: > > Raymond Hettinger wrote: > > > Some people find their productivity is enhanced with an 80 > character limit. > > > Perhaps this is true, though I've not heard one jot of evidence > to support it > and certainly no evidence focusing on Python coding practices > which are > affected by the use of whitespace for control flow. Nor have I > seen evidence > of comparative productivity of 80 char limits versus 100 char > limits. All we > have is anecdotal evidence and personal tastes. [clipped] > But yeah, this is all anectodal evidence and personal taste, as Raymond > points out. The 80 char limit _had_ a practical reason for existing, and > I don't doubt that there are people out there that are still using the > (hard|soft)ware that caused the limit to be practical, but they're > probably few and far between. I also haven't seen anything other than > anectodal evidence and personal taste in favor of increasing the limit, > which suggests to me that it should just stay how it is. It is just a > convention after all. With the popularity of smaller netbooks, laptops, and other hand held computing devices going up, I think the 80 char limit guideline may still serve a good and practical purpose. So I'm for keeping it as it is on the principle that it is only a guideline and anyone can disregard it if it is advantageous for them to do so. Ron From jeremiah.dodds at gmail.com Thu May 21 17:22:15 2009 From: jeremiah.dodds at gmail.com (Jeremiah Dodds) Date: Thu, 21 May 2009 16:22:15 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <874ove3hcm.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <12cbbbfc0905210139r7d386b6eg70c9616379c7b9bc@mail.gmail.com> <874ove3hcm.fsf@benfinney.id.au> Message-ID: <12cbbbfc0905210822v3a68e398s61692df821c7715@mail.gmail.com> On Thu, May 21, 2009 at 11:01 AM, Ben Finney > wrote: > Jeremiah Dodds > writes: > > > hmm, the 80-character convention does not stop me from unconsciously > > writing really complex list comprehensions, I just write them like so: > > > > def foo(f, a, b, c): > > return [[((f(x,y) * i, i) if i % 2 else 0) > > for i, x in enumerate(a) > > if f(y, x) == a + x] > > for y in [c(z) for z in range(a, a*b+c, c)]] > > > > not that that's really any better. > > On the contrary, I find that *much* easier to grasp than the same > statement on a single line. You have been required, by choosing to > follow the 80-column limit, to choose points at which to break the line; > and have responded by breaking it up into conceptually discrete chunks > and indented to suggest the structure. > > This example is, for me, a very convincing (anecdotal) demonstration of > why an 80-column limit is a good constraint to follow. > Oh, yes - I consider it much easier to read than the single-line equivalent. What I meant by "not that that's really any better" was more along the lines of "that statement should probably be refactored". It's very rare that I run into a case where a convoluted list comprehension like that isn't better written some other way. Sometimes I end up with stuff like that when I've made incorrect assumptions while designing a program, etc. > > > But yeah, this is all anectodal evidence and personal taste, as > > Raymond points out. > > Yet it's also more than that; to call it ?personal taste? is to imply > that it's nothing more than aesthetics. If that's all it were, I'd care > far less for changes in convention. > > I consider it rather more importantly a matter of software ergonomics, > which should therefore not be changed unless there's good supporting > evidence that the proposed change results in improvement. > I tend to agree. That's very well-said. -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Thu May 21 18:59:13 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 21 May 2009 09:59:13 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info><64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> Message-ID: <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> [Lie Ryan] > This is how I will write program if we have 160 character limit: > > def foo(f, a, b, c): > return [[((f(x, y) * i, i) if i % 2 else 0) for i, x in > enumerate(a) if f(y, x) == a + x] for y in [c(z) for z in range(a, a * b > + c, c)]] > > except there will be no line breaks... > > I love list comprehension so much that I often _unconsciously_ write a > very complex list comprehensions. 80-character convention acts as a > reminder to consider refactoring. that makes sense to because your example starts close to the left margin. What I'm more concerned about is lines that start many tabs deep. Those become awkward, causing you to wrap them differently than if they were not tabbed. So, I think the 80 char limit should be relaxed only when there is a bunch of whitespace to the left. Your cues for refactoring and coding style should not depend on the initial level of indentation. Raymond From lie.1296 at gmail.com Thu May 21 19:33:56 2009 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 22 May 2009 03:33:56 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info><64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > > [Lie Ryan] >> This is how I will write program if we have 160 character limit: >> >> def foo(f, a, b, c): >> return [[((f(x, y) * i, i) if i % 2 else 0) for i, x in >> enumerate(a) if f(y, x) == a + x] for y in [c(z) for z in range(a, a * >> b + c, c)]] >> >> except there will be no line breaks... >> >> I love list comprehension so much that I often _unconsciously_ write a >> very complex list comprehensions. 80-character convention acts as a >> reminder to consider refactoring. > > that makes sense to because your example starts close to the left margin. > What I'm more concerned about is lines that start many tabs deep. > Those become awkward, causing you to wrap them differently than > if they were not tabbed. So, I think the 80 char limit should be relaxed > only when there is a bunch of whitespace to the left. Your cues for > refactoring and coding style should not depend on the initial level of > indentation. On contrary, if the left margin causes me to struggle to keep lines below 80-char, it is an indication that the whole class/function could benefit from refactoring. It is rarely necessary to need more than 2 levels of initial (function level) indentation or 3 if you used closure: e.g.: class def def (closure) and any function that have more than another 2-3 level is an indication that it needs refactoring. class def with for if So in total, 3-5 levels are the maximum in any sane code. To me, 3-5 level is not that deep for 80-char to become a hindrance. Anyway, it seems people keep forgetting that PEP 8 is a guideline, not a syntax. I usually keep lines below 80, but I don't mind breaking them now and then if I think it is justified. From python at rcn.com Thu May 21 19:44:12 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 21 May 2009 10:44:12 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info><64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: > I consider that a critical clue that I should consider refaceoring. > Not just the one long line, but the entire structure that resulted in > that deep nesting. After you get the class and method def in, each > level should indicate a possible control flow branch. So deeply > nested implies lots of branches implies a complicated structure. > Which means it's time to consider refactoring to simplify it. Okay, I'm convinced. 80 is EXACTLY the right number. 79 is too few and 81 is WAY over the top. It doesn't matter what language your coding in, which century you're living in, or your preference for tab sizes. Looking back, I'm amazed that there was every any commercial success for line printers that had more than 80 characters. Those guys who made green bar paper were seriously misguided. How could they ignore the one true universal programming constant. Raymond From gerald.britton at gmail.com Thu May 21 19:48:13 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Thu, 21 May 2009 13:48:13 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: <5d1a32000905211048t8c3d764ke8c11baa8d4cd95e@mail.gmail.com> Why isn't the ideal line length the same as the answer to life, the universe, and everything? (if it were, I could read a line on my Blackberry without resorting to 2-point type) On Thu, May 21, 2009 at 1:44 PM, Raymond Hettinger wrote: > >> I consider that a critical clue that I should consider refaceoring. ? Not >> just the one long line, but the entire structure that resulted in ?that deep >> nesting. ?After you get the class and method def in, each ?level should >> indicate a possible ?control flow branch. So deeply ?nested implies lots of >> branches implies a complicated structure. ? Which means it's time to >> consider refactoring to simplify it. > > Okay, I'm convinced. ?80 is EXACTLY the right number. ?79 is > too few and 81 is WAY over the top. ?It doesn't matter what language your > coding in, which century you're living in, or your > preference for tab sizes. > > Looking back, I'm amazed that there was every any commercial > success for line printers that had more than 80 characters. ?Those > guys who made green bar paper were seriously misguided. ?How > could they ignore the one true universal programming constant. > > Raymond > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton From aahz at pythoncraft.com Thu May 21 19:53:06 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 21 May 2009 10:53:06 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: <20090521175306.GA7799@panix.com> On Thu, May 21, 2009, Raymond Hettinger wrote: > > Okay, I'm convinced. 80 is EXACTLY the right number. 79 is too few > and 81 is WAY over the top. It doesn't matter what language your > coding in, which century you're living in, or your preference for tab > sizes. > > Looking back, I'm amazed that there was every any commercial success > for line printers that had more than 80 characters. Those guys who > made green bar paper were seriously misguided. How could they ignore > the one true universal programming constant. Could you explain why you have so much emotion invested in this issue? >From my POV, it looks like you're one step short of outright flaming anyone who disagrees with you. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson From python at rcn.com Thu May 21 20:00:02 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 21 May 2009 11:00:02 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> <20090521175306.GA7799@panix.com> Message-ID: > Could you explain why you have so much emotion invested in this issue? I'm mostly having light-hearted fun with it. The conversation wasn't going anywhere from the start. I do find it funny that so many think 80 is exactly the right number regardless of other whitespace conventions or regardless of the language being used. I do think the OP was right on at least one point. I think all of us at one time or another has seen code get mangled in attempts to comply with an arbitrary line length limit. But this is the exception, not the norm. Raymond From aaron.rubin at 4dtechnology.com Thu May 21 20:01:55 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Thu, 21 May 2009 11:01:55 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: <985db6080905211101k1a2f49f3qd37d899431982f1d@mail.gmail.com> This thread is going where the OP didn't want it to go. Personal preference has already been decided to vary greatly. Seems to be about 50/50. Obviously THAT won't change. I was looking for objective arguments, not subjective ones. Arguing over *how* to break lines is actually a pretty strong argument that time is wasted spent on such issues. A longer line width would reduce these arguments, since less would need to be wrapped. So far, let me do a tally of objective arguments, just to keep this thing rational: New facts supporting 80 character line width: 1) Most conventional printers in portrait format can print 80 characters nicely in a line. 2) New netbooks have less width on their screens, making 80 characters nicer. (This actually probably calls for an even narrower width if this is a big concern) 3) If the code is standard library code, given that we have some folks out there who still write with 80 character line widths, then that code should still be 80 characters to fit the Least Common Denominator. 4) 2 space indentations reduce the preceding whitespace on each line by 50%, making 80 characters more reasonable. (and obviously, if people are using 2 space indents, horizontal space was at a premium). Also it appears from context that Google uses 2 space indents. 5) Older editors might have trouble handling greater than 80 character widths, including older terminal emulators. New facts supporting greater than 80 width: 1) Comment blocks are even more limited than originally thought, especially if following the PEP 8. 2) The standard library generally needn't leverage OO as much as code using it (in fact, much of the stdlib is procedural in nature), thereby making the stdlib less wide by nature. This is true of any core library...it is generally simpler code reused most often. Complex business logic, scientific programming, etc. also tends to need longer variable names since these notions can't be called simply "tool", but what type of tool and in an environment with many "tools" this name gets longer to be more descriptive. Other new facts: 1) Around 200 lines of code even within the standard library start at the 10th indentation level (40 characters gone already) 2) Around 10 percent of unique variable names even within the standard library have 15 characters or more 3) The origin of 80 character limit widths came from punch cards Of course, the original facts still remain (with original fact #7 held out as a weaker point, but still remains the only scientific study mentioned thusfar). What did I leave out? - Aaron On Thu, May 21, 2009 at 9:59 AM, Raymond Hettinger wrote: > > [Lie Ryan] > >> This is how I will write program if we have 160 character limit: >> >> def foo(f, a, b, c): >> return [[((f(x, y) * i, i) if i % 2 else 0) for i, x in enumerate(a) if >> f(y, x) == a + x] for y in [c(z) for z in range(a, a * b + c, c)]] >> >> except there will be no line breaks... >> >> I love list comprehension so much that I often _unconsciously_ write a >> very complex list comprehensions. 80-character convention acts as a reminder >> to consider refactoring. >> > > that makes sense to because your example starts close to the left margin. > What I'm more concerned about is lines that start many tabs deep. > Those become awkward, causing you to wrap them differently than > if they were not tabbed. So, I think the 80 char limit should be relaxed > only when there is a bunch of whitespace to the left. Your cues for > refactoring and coding style should not depend on the initial level of > indentation. > > > Raymond > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dstanek at dstanek.com Thu May 21 20:11:49 2009 From: dstanek at dstanek.com (David Stanek) Date: Thu, 21 May 2009 14:11:49 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: On Thu, May 21, 2009 at 12:59 PM, Raymond Hettinger wrote: > > What I'm more concerned about is lines that start many tabs deep. > Those become awkward, causing you to wrap them differently than > if they were not tabbed. ?So, I think the 80 char limit should be relaxed > only when there is a bunch of whitespace to the left. ?Your cues for > refactoring ?and coding style should not depend on the initial level of > indentation. > I disagree. I find that highly nested code probably needs refactoring. The code in question is likely too complex and hard to test. Of course this is not always the case, just a general rule in my experience. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek From python at rcn.com Thu May 21 20:42:24 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 21 May 2009 11:42:24 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> <985db6080905211101k1a2f49f3qd37d899431982f1d@mail.gmail.com> Message-ID: <64B72348C68642E193F680DCB7D9E582@RaymondLaptop1> [Aaron Rubin] > Around 200 lines of code even within the standard library start at the 10th indentation level (40 characters gone already) Can you modify your script to show the full distribution (percentage of lines at the nth indentation level)? Also, I'm curious how much distribution differs between Lib/test vs Lib vs Lib/email. Raymond From jess.austin at gmail.com Thu May 21 21:28:53 2009 From: jess.austin at gmail.com (Jess Austin) Date: Thu, 21 May 2009 14:28:53 -0500 Subject: [Python-ideas] 80 character line width vs. something wider Message-ID: On Thu, May 21, 2009 at 1:01 PM, Aaron Rubin wrote: > subjective ones. ?Arguing over *how* to break lines is actually a pretty > strong argument that time is wasted spent on such issues. ?A longer line > width would reduce these arguments, since less would need to be wrapped. This is an unsupported, and IMHO largely incorrect, assumption. Several correspondents have noted that they most often overrun their intended line length by one or two characters. Just as there's nothing magical about the number "80", there's nothing magical about "81" or "82" either. In a regime of 90-character lines, the limit will most often be exceeded by one or two characters. The same will happen in a regime of 100-character lines, etc. We'll still need to break lines, and wrapping them in parentheses will still be the best way to do that. This sentiment reminds me of those road-construction enthusiasts who are always certain that the _next_ newly-built highway will end traffic jams forever. What happens instead is that the new road fills with cars too, and it still takes forever to get to work. cheers, Jess ps. ride a bike! From aahz at pythoncraft.com Thu May 21 21:59:30 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 21 May 2009 12:59:30 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <20090521175306.GA7799@panix.com> Message-ID: <20090521195930.GA19799@panix.com> On Thu, May 21, 2009, Raymond Hettinger wrote: > Aahz: >> >> Could you explain why you have so much emotion invested in this issue? > > I'm mostly having light-hearted fun with it. The conversation wasn't > going anywhere from the start. I do find it funny that so many > think 80 is exactly the right number regardless of other whitespace > conventions or regardless of the language being used. If that's what you think people are saying, I don't think you are using sufficient care in your reading. It seems to me that people are mostly saying that eighty columns is enough for many purposes, there are problems that would come from increasing the number of columns, and there aren't any solid arguments that clearly demonstrate that the gains outweigh the detriments. My take is that the people who claim that we would be having this exact conversation if the standard were 100 characters are precisely correct. There will always be people pushing longer lines, and there needs to be better argumentation before we abandon the current standard. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson From aaron.rubin at 4dtechnology.com Thu May 21 22:40:39 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Thu, 21 May 2009 13:40:39 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090521195930.GA19799@panix.com> References: <20090521175306.GA7799@panix.com> <20090521195930.GA19799@panix.com> Message-ID: <985db6080905211340y443255c2x5b488bffb8807743@mail.gmail.com> Wow, Aahz. I can't believe how applicable your own signature quote is! :P On Thu, May 21, 2009 at 12:59 PM, Aahz wrote: > On Thu, May 21, 2009, Raymond Hettinger wrote: > > Aahz: > >> > >> Could you explain why you have so much emotion invested in this issue? > > > > I'm mostly having light-hearted fun with it. The conversation wasn't > > going anywhere from the start. I do find it funny that so many > > think 80 is exactly the right number regardless of other whitespace > > conventions or regardless of the language being used. > > If that's what you think people are saying, I don't think you are using > sufficient care in your reading. It seems to me that people are mostly > saying that eighty columns is enough for many purposes, there are > problems that would come from increasing the number of columns, and there > aren't any solid arguments that clearly demonstrate that the gains > outweigh the detriments. > > My take is that the people who claim that we would be having this exact > conversation if the standard were 100 characters are precisely correct. > There will always be people pushing longer lines, and there needs to be > better argumentation before we abandon the current standard. > -- > Aahz (aahz at pythoncraft.com) <*> > http://www.pythoncraft.com/ > > "A foolish consistency is the hobgoblin of little minds, adored by little > statesmen and philosophers and divines." --Ralph Waldo Emerson > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.rubin at 4dtechnology.com Thu May 21 22:45:49 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Thu, 21 May 2009 13:45:49 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: Message-ID: <985db6080905211345t11494800hdff89fb808d75fe1@mail.gmail.com> On Thu, May 21, 2009 at 12:28 PM, Jess Austin wrote: > On Thu, May 21, 2009 at 1:01 PM, Aaron Rubin > wrote: > > subjective ones. Arguing over *how* to break lines is actually a pretty > > strong argument that time is wasted spent on such issues. A longer line > > width would reduce these arguments, since less would need to be wrapped. > > This is an unsupported, and IMHO largely incorrect, assumption. > Several correspondents have noted that they most often overrun their > intended line length by one or two characters. Just as there's > nothing magical about the number "80", there's nothing magical about > "81" or "82" either. In a regime of 90-character lines, the limit > will most often be exceeded by one or two characters. The same will > happen in a regime of 100-character lines, etc. We'll still need to > break lines, and wrapping them in parentheses will still be the best > way to do that. How can you argue that it wouldn't create *less* line wrapping? According to your argument having 10,000 character width lines wouldn't create less line wrapping either. Nobody ever said it would eliminate it, just reduce it. > > This sentiment reminds me of those road-construction enthusiasts who > are always certain that the _next_ newly-built highway will end > traffic jams forever. What happens instead is that the new road fills > with cars too, and it still takes forever to get to work. > > cheers, > Jess > > ps. ride a bike! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri May 22 00:18:40 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 22 May 2009 08:18:40 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <64B72348C68642E193F680DCB7D9E582@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905211101k1a2f49f3qd37d899431982f1d@mail.gmail.com> <64B72348C68642E193F680DCB7D9E582@RaymondLaptop1> Message-ID: <200905220818.41038.steve@pearwood.info> On Fri, 22 May 2009 04:42:24 am Raymond Hettinger wrote: > [Aaron Rubin] > > > Around 200 lines of code even within the standard library start at > > the 10th indentation level (40 characters gone already) > > Can you modify your script to show the full distribution (percentage > of lines at the nth indentation level)? I'd also like to see some information about the structure of said lines, e.g. are those 200 lines in a single giant block, or 200 blocks of one line each, or something in-between? > Also, I'm curious how much distribution differs between Lib/test vs > Lib vs Lib/email. I wonder whether some sort of source code analyser should be included in the std lib (or perhaps in Tools?). -- Steven D'Aprano From ben+python at benfinney.id.au Fri May 22 01:04:01 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 22 May 2009 09:04:01 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: <87vdnu12ji.fsf@benfinney.id.au> "Raymond Hettinger" writes: > Okay, I'm convinced. 80 is EXACTLY the right number. 79 is too few and > 81 is WAY over the top. It doesn't matter what language your coding > in, which century you're living in, or your preference for tab sizes. That's a strange thing to be convinced of, since it's nowhere near a position anyone has been advocating in this discussion. Perhaps you're intent on seeing only the extremes? -- \ ?As the most participatory form of mass speech yet developed, | `\ the Internet deserves the highest protection from governmental | _o__) intrusion.? ?U.S. District Court Judge Dalzell | Ben Finney From greg.ewing at canterbury.ac.nz Fri May 22 02:02:20 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 22 May 2009 12:02:20 +1200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A15053F.9050807@scottdial.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> <20090519192022.4e47f89d@bhuda.mired.org> <20090520114111.6e670e49@o> <4A148FE8.70407@canterbury.ac.nz> <4A15053F.9050807@scottdial.com> Message-ID: <4A15EB8C.5030608@canterbury.ac.nz> Scott Dial wrote: > Given the large body > of code that already follows PEP8 (and other style guides for other > languages that commonly use an 80-character boundary), it is a common > constraint which yields many common idioms (such as placing list items > on separate lines with similar indention). I still don't think it's the same thing. Limiting lines to 80 chars or thereabouts is not an *arbitrary* constraint. There are intrinsic merits to it, e.g. people find very long lines hard to read, you can fit multiple windows side by side, etc. On the other hand, there's nothing inherently virtuous about writing in iambic pentameter or avoiding the use of the letter "e". The only merit of such constraints is that they push you *away* from a very small area of badness (i.e. cliches) and out into a much bigger area of non-badness (any non-cliched way of saying the same thing). I don't think you can do the same thing with programming. You can't get a good program just by avoiding bad things, you have to actively aim for the good things. -- Greg From python at rcn.com Fri May 22 02:48:25 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 21 May 2009 17:48:25 -0700 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com><985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com><20090519192022.4e47f89d@bhuda.mired.org> <20090520114111.6e670e49@o><4A148FE8.70407@canterbury.ac.nz> <4A15053F.9050807@scottdial.com> <4A15EB8C.5030608@canterbury.ac.nz> Message-ID: <2CB45D31FEA44C559155453622C0A35A@RaymondLaptop1> > You can't get a good program just by > avoiding bad things, you have to actively aim for > the good things. +1 QOTW Raymond From aaron.rubin at 4dtechnology.com Fri May 22 03:09:43 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Thu, 21 May 2009 18:09:43 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> <985db6080905211101k1a2f49f3qd37d899431982f1d@mail.gmail.com> Message-ID: <985db6080905211809x502a912h507758716be48bad@mail.gmail.com> Yes, I used your information. Feel free to add more information gleaned from it to the list of facts. If your numbers weren't accurate, then it would be nice to get an accurate number. Perhaps Greg Ewing might already have something useful for us to use. I started to write a script to analyze the standard library and quickly realized I should be using a lexical analyzer to weed out comments and (funny enough) wrapped lines themselves from the analysis. I took your numbers as a rough fact, using words like "Around", i.e. taking what you said at face value. I am about to leave on vacation and don't have the time to do the lexical analysis, otherwise I would! Any excuse to write a useful script ;) - Aaron On Thu, May 21, 2009 at 12:27 PM, Mike Meyer wrote: > On May 21, 2009, at 14:01, Aaron Rubin > >> Other new facts: >> > > I think I need to cry foul here! > > 1) Around 200 lines of code even within the standard library start at the >> 10th indentation level (40 characters gone already) >> 2) Around 10 percent of unique variable names even within the standard >> library have 15 characters or more >> > > These look like the numbers I provided, which I said were crude estimates > at best. If someone did a good count, I missed it, and I apologize. If not, > I wouldn't count them as "facts". Worse, you left off related numbers that > bias these: that's 200 lines out of 80000, and about 2 percent of all > variable references are to variables longer than 15 characters. But all of > those numbers should be considered tenative estimated, at best. > > 3) The origin of 80 character limit widths came from punch cards >> >> What did I leave out? >> > > That punch cards were available in sizes between 51 and 96 columns before > and after the 80-column card became standard, so this debate is older than > most people realize. > > Many uses of those cards reserved the last 8 columns for card numbers, so > the practical limit was 72 columns, not 80. > > Finally, the 80 column card became standard when most printers used > fan-fold 132 column paper, so someone felt there was a good reason to limit > the input line length even then. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jimjjewett at gmail.com Fri May 22 03:44:33 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 21 May 2009 21:44:33 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <08CC815EFEFE4CAC869C9F6843E3DF2D@RaymondLaptop1> Message-ID: On 5/19/09, Jason Orendorff wrote: > On Tue, May 19, 2009 at 1:15 PM, Raymond Hettinger wrote: > I find the long lines in my code are not as contrived as the thread so > far suggests: FWIW, I would find it easier to read all but one of these if they were broken into multiple lines -- and that is regardless of what monitor I happened to be using at the time. > sys.stderr.write("js-build: WARNING: Due to COMMAND_MODE > madness, this can fail on Leopard!\n" > "js-build: Workaround is to > upgrade to Python 2.5.2 or later.\n") My idiom here is to take out the literal, at least to its own line. For example: msg="""\ js-build: WARNING: Due to COMMAND_MODE madness, this can fail on Leopard! js-build: Workaround is to upgrade to Python 2.5.2 or later. """ sys.stderr.write(msg) > self.parent[id] = self.lastbranch.get(branch, > 'bad') This was the only one where a line break might hurt readability for me, and I still suspect a rewording would make it easier to read. -jJ From rrr at ronadam.com Fri May 22 04:33:27 2009 From: rrr at ronadam.com (Ron Adam) Date: Thu, 21 May 2009 21:33:27 -0500 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090521195930.GA19799@panix.com> References: <20090521175306.GA7799@panix.com> <20090521195930.GA19799@panix.com> Message-ID: <4A160EF7.1040704@ronadam.com> Aahz wrote: > On Thu, May 21, 2009, Raymond Hettinger wrote: >> Aahz: >>> Could you explain why you have so much emotion invested in this issue? >> I'm mostly having light-hearted fun with it. The conversation wasn't >> going anywhere from the start. I do find it funny that so many >> think 80 is exactly the right number regardless of other whitespace >> conventions or regardless of the language being used. > > If that's what you think people are saying, I don't think you are using > sufficient care in your reading. It seems to me that people are mostly > saying that eighty columns is enough for many purposes, there are > problems that would come from increasing the number of columns, and there > aren't any solid arguments that clearly demonstrate that the gains > outweigh the detriments. Well, on Ubuntu I use the gnome terminal which has line width settings of 80 and 132. It doesn't have any settings in-between. So I use 80X24 because it gives me plenty of room to see other windows. Possibly if there was a 100 character width, I might use it. The 132 character width setting takes up too much horizontal room on my desktop. I found that if I use small font sizes, I tend to lean closer to the monitor than I should and that causes my neck and back to ache if I'm at my computer for longer than a few hours. So although I can reduce the font size and get a lot more characters across the screen, it is better for my health if I don't do that. Cheers, Ron From aahz at pythoncraft.com Fri May 22 04:51:08 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 21 May 2009 19:51:08 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A160EF7.1040704@ronadam.com> References: <20090521175306.GA7799@panix.com> <20090521195930.GA19799@panix.com> <4A160EF7.1040704@ronadam.com> Message-ID: <20090522025108.GA24459@panix.com> On Thu, May 21, 2009, Ron Adam wrote: > Aahz wrote: >> On Thu, May 21, 2009, Raymond Hettinger wrote: >>> >>> I'm mostly having light-hearted fun with it. The conversation wasn't >>> going anywhere from the start. I do find it funny that so many >>> think 80 is exactly the right number regardless of other whitespace >>> conventions or regardless of the language being used. >> >> If that's what you think people are saying, I don't think you are using >> sufficient care in your reading. It seems to me that people are mostly >> saying that eighty columns is enough for many purposes, there are >> problems that would come from increasing the number of columns, and there >> aren't any solid arguments that clearly demonstrate that the gains >> outweigh the detriments. > > Well, on Ubuntu I use the gnome terminal which has line width settings of > 80 and 132. It doesn't have any settings in-between. So I use 80X24 > because it gives me plenty of room to see other windows. Actually, you can have different window sizes, either by dragging with a mouse or using the command-line option --geometry. > I found that if I use small font sizes, I tend to lean closer to the > monitor than I should and that causes my neck and back to ache if I'm at > my computer for longer than a few hours. So although I can reduce the > font size and get a lot more characters across the screen, it is better > for my health if I don't do that. Very this. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines." --Ralph Waldo Emerson From ben+python at benfinney.id.au Fri May 22 05:29:47 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 22 May 2009 13:29:47 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <20090521175306.GA7799@panix.com> <20090521195930.GA19799@panix.com> <4A160EF7.1040704@ronadam.com> Message-ID: <87iqjt24t0.fsf@benfinney.id.au> Ron Adam writes: > So although I can reduce the font size and get a lot more characters > across the screen, it is better for my health if I don't do that. +1 QOTW -- \ ?I used to be an airline pilot. I got fired because I kept | `\ locking the keys in the plane. They caught me on an 80 foot | _o__) stepladder with a coathanger.? ?Steven Wright | Ben Finney From jimjjewett at gmail.com Fri May 22 07:37:05 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 22 May 2009 01:37:05 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: On 5/19/09, Aaron Rubin wrote: > PhazeMonkey.Hardware.FrameSource.Abstract.Framegrabber During the PEP for with, this was considered (as a competing meaning for "with"). There was some concern that it might be needed for integration with other frameworks, such as .net, java GUI code, etc. In the end, Guido rejected it because you can just do it with a temporary variable. frm=PhazeMonkey.Hardware.FrameSource frm.Abstract.Framegrabber > 6) Tools are cheap. Time isn't. Get a second monitor, get a more powerful > editor, do whatever it takes to save time. If viewing more information at > one time is important, then we should try to make that possible with > technology, not time. I just looked at a sampling of books from my shelf. The *longest* line length I found was 71 characters. I have seen (but can't find at the moment) studies showing that lines wider than a certain percentage of the visual field are problematic. The exact portion (and how many characters fit in it at a readable font) will vary from person to person; what is comfortable for you may already be very disruptive for a teammate -- and your savings from a longer line are small enough that this is one case where lowest common denominator should be given great weight. (Leading whitespace is much "cheaper" than actual text, but there is still some effort in sliding that focal window back and forth with the indent/dedent dance.) And of course, this concern with visual focal area already assumes that everyone will have windows as wide as yours and fonts as small; I can assure you that neither is true. The first person to need horizontal scrolling will lose more time than you saved. The first person using a window/terminal where the line wrapped on its own because it was too long will squander far more. -jJ From jimjjewett at gmail.com Fri May 22 07:52:46 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 22 May 2009 01:52:46 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <97B7C0D4B4974401BEA335D3D58ED7E1@RaymondLaptop1> Message-ID: On 5/21/09, Raymond Hettinger wrote: > I think the 80 char limit should be relaxed > only when there is a bunch of whitespace to the left. Your cues for > refactoring and coding style should not depend on the initial level > of indentation. Level of indentation is itself one of the strongest cues. I will agree that (ideally) it should not affect the strength of *other* cues. -jJ From rrr at ronadam.com Fri May 22 08:10:19 2009 From: rrr at ronadam.com (Ron Adam) Date: Fri, 22 May 2009 01:10:19 -0500 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090522025108.GA24459@panix.com> References: <20090521175306.GA7799@panix.com> <20090521195930.GA19799@panix.com> <4A160EF7.1040704@ronadam.com> <20090522025108.GA24459@panix.com> Message-ID: <4A1641CB.9030104@ronadam.com> Aahz wrote: > On Thu, May 21, 2009, Ron Adam wrote: >> Aahz wrote: >>> On Thu, May 21, 2009, Raymond Hettinger wrote: >>>> I'm mostly having light-hearted fun with it. The conversation wasn't >>>> going anywhere from the start. I do find it funny that so many >>>> think 80 is exactly the right number regardless of other whitespace >>>> conventions or regardless of the language being used. >>> If that's what you think people are saying, I don't think you are using >>> sufficient care in your reading. It seems to me that people are mostly >>> saying that eighty columns is enough for many purposes, there are >>> problems that would come from increasing the number of columns, and there >>> aren't any solid arguments that clearly demonstrate that the gains >>> outweigh the detriments. >> Well, on Ubuntu I use the gnome terminal which has line width settings of >> 80 and 132. It doesn't have any settings in-between. So I use 80X24 >> because it gives me plenty of room to see other windows. > > Actually, you can have different window sizes, either by dragging with a > mouse or using the command-line option --geometry. Heh, I'm sure I've done that quite a few times and then forgotten about it. >> I found that if I use small font sizes, I tend to lean closer to the >> monitor than I should and that causes my neck and back to ache if I'm at >> my computer for longer than a few hours. So although I can reduce the >> font size and get a lot more characters across the screen, it is better >> for my health if I don't do that. > > Very this. From stephen at xemacs.org Fri May 22 08:46:07 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 22 May 2009 15:46:07 +0900 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A15EB8C.5030608@canterbury.ac.nz> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <985db6080905191536i7f2f3e51v44409d7c6efcca8f@mail.gmail.com> <20090519192022.4e47f89d@bhuda.mired.org> <20090520114111.6e670e49@o> <4A148FE8.70407@canterbury.ac.nz> <4A15053F.9050807@scottdial.com> <4A15EB8C.5030608@canterbury.ac.nz> Message-ID: <87y6speits.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > I don't think you can do the same thing with > programming. You can't get a good program just by > avoiding bad things, you have to actively aim for > the good things. You think you can get a good poem simply by avoiding bad things? Surely not! There are still an infinite number of ways to write a bad haiku, despite the extreme style constraint, and great haiku writers remain rare. I think the analogy programming to poetry is quite strong. And in a discussion of Python, there should be no question that an arbitrary line-length limitation aims at the good things: "flat is better than nested". The question here is, how accurate is that aim? Does it too often hit our feet instead? >From the examples presented here (both actual and contrived!) by the *opponents* of line-length limitation, it seems to me that being strict about an 80-character limit is not "a foolish consistency", but rather a quite cheap and accurate way to identify flat-is-better-than- nested violations. The time spent *in implementing Python and the stdlib* on avoiding and fixing those is time very well spent, IMO. Other projects *should* make their own judgments. There are two specific usages that I think may deserve exceptions, because they don't involve flat-is-better-than-nested violations. The first is line-terminating comments. These are *good* style IMHO in cases where a particular statement requires glossing. I don't see how to reformat those to meet line-length limitations except by moving them above the glossed statement, which is ambiguous (how many statements are in the "scope" of the comment?) and, worse, interrupts the flow of control when reading the code. The second is strings for presentation in a line-oriented terminal context, such as error messages and docstrings. I like these to be about 65-70 characters wide, but there are some cases (eg, first lines of docstrings) where that is regularly desirable to violate. Here in practice I use Ben Finney's "indent one extra level" and Jim Jewett's "escape newline and start string at left margin" techniques, but I'm not 100% satisfied with them. From g.brandl at gmx.net Fri May 22 10:14:19 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 22 May 2009 10:14:19 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090521090655.7695a0c5@o> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> Message-ID: spir schrieb: > Le Wed, 20 May 2009 18:02:54 -0700, "Raymond Hettinger" > s'exprima ainsi: > >>> If the character limit didn't remain relevant, we'd remove it, > > (to Steven) We definitely don't live in the same world ;-) In mine, any > purely legacy trait change raises religious wars. People fighting against > change _always_ find "good" reasons. That seems to be part of human nature > (in the world I live in). And who determines if it *is* a purely legacy trait? Don't you think that these reasons may be valid when it isn't? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From jeremiah.dodds at gmail.com Fri May 22 10:16:01 2009 From: jeremiah.dodds at gmail.com (Jeremiah Dodds) Date: Fri, 22 May 2009 09:16:01 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905211345t11494800hdff89fb808d75fe1@mail.gmail.com> References: <985db6080905211345t11494800hdff89fb808d75fe1@mail.gmail.com> Message-ID: <12cbbbfc0905220116nac2f007m1f8429536a26ed33@mail.gmail.com> On Thu, May 21, 2009 at 9:45 PM, Aaron Rubin wrote: > > How can you argue that it wouldn't create *less* line wrapping? According > to your argument having 10,000 character width lines wouldn't create less > line wrapping either. Nobody ever said it would eliminate it, just reduce > it. > > He's not arguing that it wouldn't create less line wrapping, he's arguing that it wouldn't create less arguments about line-wrapping, and thus wouldn't reduce the wasted time spent arguing about line-wrapping. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jan.kanis at phil.uu.nl Fri May 22 15:32:42 2009 From: jan.kanis at phil.uu.nl (Jan Kanis) Date: Fri, 22 May 2009 15:32:42 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> Message-ID: <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> There is, of course, a correct solution to this line length problem. The ideal maximum line length is dependent on how much horizontal space you've got, so the code should be automatically wrapped to your window size. That's how all other text displaying systems work, from html to word processors. It's just code that's the exception*. This would require good automatic wrapping, which is a bit more difficult for code than for English, but automatic reformatting utilities already exist. It would also require tool support, so everyone would need to use editors like emacs or something else that's programmable. Just your favorite notepad-on-steroids variant won't suffice anymore. Of course, it's only 2009 and this is all very much py-in-the-sky. (Though who knows what Python 6000 will bring...) * another exception are config files, but more and more people are writing gui config editors, which adapt to the window size, so these too are following the trend. The only backward compatible way I can think of to implement this is to tell your emacs or other programmable editor to automatically display files according to your window width, and wrap them to 80/whatever cpl in the on disk representation when you save. And add some magic so it only changes those parts of the file in the on disk representation that you actually edited, deal smartly with line numbering, etc, etc. Jan From jess.austin at gmail.com Fri May 22 17:00:56 2009 From: jess.austin at gmail.com (Jess Austin) Date: Fri, 22 May 2009 10:00:56 -0500 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905211345t11494800hdff89fb808d75fe1@mail.gmail.com> References: <985db6080905211345t11494800hdff89fb808d75fe1@mail.gmail.com> Message-ID: On Thu, May 21, 2009 at 3:45 PM, Aaron Rubin wrote: > On Thu, May 21, 2009 at 12:28 PM, Jess Austin wrote: >> On Thu, May 21, 2009 at 1:01 PM, ?Aaron Rubin >> wrote: >> > subjective ones. ?Arguing over *how* to break lines is actually a pretty >> > strong argument that time is wasted spent on such issues. ?A longer line >> > width would reduce these arguments, since less would need to be wrapped. >> >> This is an unsupported, and IMHO largely incorrect, assumption. >> Several correspondents have noted that they most often overrun their >> intended line length by one or two characters. ?Just as there's >> nothing magical about the number "80", there's nothing magical about >> "81" or "82" either. ?In a regime of 90-character lines, the limit >> will most often be exceeded by one or two characters. ?The same will >> happen in a regime of 100-character lines, etc. ?We'll still need to >> break lines, and wrapping them in parentheses will still be the best >> way to do that. > > How can you argue that it wouldn't create *less* line wrapping? ? According > to your argument having 10,000 character width lines wouldn't create less > line wrapping either. ?Nobody ever said it would eliminate it, just reduce > it. My point is that line length is a habit of programming. Like any habit, it is largely determined by the context in which it arises. Different contexts will yield different habits. I'll accept your implication that there are some idioms that will never take more than 100 (is that the number you like?) characters regardless of the verbosity of the programmer, and I'll further stipulate an inverse linear relation between line length and line-breakage rate. However, I'm pretty confident that this relation has a _very_ shallow slope, so that if we assume 10% line breakage at 80 characters, I'd expect 9% line breakage at 100 characters. Is a 1% absolute benefit really worth any argument at all? I doubt you can disprove my slope estimate, but if you try please understand how few projects in the wild are completely accepting of excessive line length, and base your statistics only on programmers who primarily code in those environments. If you succeed in overthrowing heaven and earth here, you'll change programming environments and habits for everyone. Jeremiah's point concerning argumentation rates is also valid. cheers, Jess From lie.1296 at gmail.com Fri May 22 17:05:17 2009 From: lie.1296 at gmail.com (Lie Ryan) Date: Sat, 23 May 2009 01:05:17 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> Message-ID: Jan Kanis wrote: > There is, of course, a correct solution to this line length problem. > The ideal maximum line length is dependent on how much horizontal > space you've got, so the code should be automatically wrapped to your > window size. That's how all other text displaying systems work, from > html to word processors. It's just code that's the exception*. > > This would require good automatic wrapping, which is a bit more > difficult for code than for English, but automatic reformatting > utilities already exist. It would also require tool support, so > everyone would need to use editors like emacs or something else that's > programmable. Just your favorite notepad-on-steroids variant won't > suffice anymore. > > Of course, it's only 2009 and this is all very much py-in-the-sky. > (Though who knows what Python 6000 will bring...) > > * another exception are config files, but more and more people are > writing gui config editors, which adapt to the window size, so these > too are following the trend. > > > The only backward compatible way I can think of to implement this is > to tell your emacs or other programmable editor to automatically > display files according to your window width, and wrap them to > 80/whatever cpl in the on disk representation when you save. And add > some magic so it only changes those parts of the file in the on disk > representation that you actually edited, deal smartly with line > numbering, etc, etc. > > > Jan There are reasons why carpenters used simple tools like hammer and nails instead of sophisticated machine that can do everything from chopping the trees to polishing. From aaron.rubin at 4dtechnology.com Fri May 22 20:44:45 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Fri, 22 May 2009 11:44:45 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905211345t11494800hdff89fb808d75fe1@mail.gmail.com> Message-ID: <985db6080905221144i15afd206l8f323b2355606e1d@mail.gmail.com> Good stuff, Jess. It should be easily observed...what percentage of wrapped lines in the stdlib would end at <=100 characters vs. <=120 vs. <=140, etc? That would be great facts to know... (don't currently have the time to analyze the stdlib) In a quick analysis of my code base (easier to analyze since it is not wrapped already at 80 or so), I find that roughly 70% of the long lines end at <=100 and 90% end at <=120. That's a much steeper slope than you predicted. The question I'm faced with is whether the extra 20% of 120 vs. 100 is worth it. On Fri, May 22, 2009 at 8:00 AM, Jess Austin wrote: > On Thu, May 21, 2009 at 3:45 PM, Aaron Rubin > wrote: > > On Thu, May 21, 2009 at 12:28 PM, Jess Austin > wrote: > >> On Thu, May 21, 2009 at 1:01 PM, Aaron Rubin > >> wrote: > >> > subjective ones. Arguing over *how* to break lines is actually a > pretty > >> > strong argument that time is wasted spent on such issues. A longer > line > >> > width would reduce these arguments, since less would need to be > wrapped. > >> > >> This is an unsupported, and IMHO largely incorrect, assumption. > >> Several correspondents have noted that they most often overrun their > >> intended line length by one or two characters. Just as there's > >> nothing magical about the number "80", there's nothing magical about > >> "81" or "82" either. In a regime of 90-character lines, the limit > >> will most often be exceeded by one or two characters. The same will > >> happen in a regime of 100-character lines, etc. We'll still need to > >> break lines, and wrapping them in parentheses will still be the best > >> way to do that. > > > > How can you argue that it wouldn't create *less* line wrapping? > According > > to your argument having 10,000 character width lines wouldn't create less > > line wrapping either. Nobody ever said it would eliminate it, just > reduce > > it. > > My point is that line length is a habit of programming. Like any > habit, it is largely determined by the context in which it arises. > Different contexts will yield different habits. I'll accept your > implication that there are some idioms that will never take more than > 100 (is that the number you like?) characters regardless of the > verbosity of the programmer, and I'll further stipulate an inverse > linear relation between line length and line-breakage rate. However, > I'm pretty confident that this relation has a _very_ shallow slope, so > that if we assume 10% line breakage at 80 characters, I'd expect 9% > line breakage at 100 characters. Is a 1% absolute benefit really > worth any argument at all? > > I doubt you can disprove my slope estimate, but if you try please > understand how few projects in the wild are completely accepting of > excessive line length, and base your statistics only on programmers > who primarily code in those environments. If you succeed in > overthrowing heaven and earth here, you'll change programming > environments and habits for everyone. > > Jeremiah's point concerning argumentation rates is also valid. > > cheers, > Jess > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Fri May 22 21:20:53 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 22 May 2009 21:20:53 +0200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> Message-ID: Jim Jewett schrieb: > I have seen (but can't find at the moment) studies showing that lines > wider than a certain percentage of the visual field are problematic. > The exact portion (and how many characters fit in it at a readable > font) will vary from person to person; what is comfortable for you may > already be very disruptive for a teammate -- and your savings from a > longer line are small enough that this is one case where lowest common > denominator should be given great weight. (Leading whitespace is much > "cheaper" than actual text, but there is still some effort in sliding > that focal window back and forth with the indent/dedent dance.) > > And of course, this concern with visual focal area already assumes > that everyone will have windows as wide as yours and fonts as small; I > can assure you that neither is true. The first person to need > horizontal scrolling will lose more time than you saved. The first > person using a window/terminal where the line wrapped on its own > because it was too long will squander far more. I agree completely. I also find, and that is of course a subjective finding, that my "on-sight" estimation of code quality partly depends on line length. Code that looks "frayed" (forgive me if that's not the right word) immediately looks less "tidy" to me. Of course, other factors come into play, like the amount of whitespace applied, the consistency of indentation, the consistency and spelling in comments etc. These are all stylistic points, but in summa, the code itself more often than not turns out to confirm the first look assessment. This means that most decent Python programmers know PEP 8 and follow it. The reasons may be as simple as mindlessly following the established standard, but I suspect otherwise: if PEP 8 was completely unreasonable, a different standard would have emerged. 2?-ly, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ben+python at benfinney.id.au Sat May 23 00:43:04 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 23 May 2009 08:43:04 +1000 Subject: [Python-ideas] 80 character line width vs. something wider References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> Message-ID: <87pre0zrlz.fsf@benfinney.id.au> Jan Kanis writes: > There is, of course, a correct solution to this line length problem. > The ideal maximum line length is dependent on how much horizontal > space you've got It's also dependent on how different line lengths affect ease of rapid comprehension. As mentioned many times in this thread. > so the code should be automatically wrapped to your window size. No, since that fails to account for the dependency I mention above. -- \ ?Science is a way of trying not to fool yourself. The first | `\ principle is that you must not fool yourself, and you are the | _o__) easiest person to fool.? ?Richard P. Feynman, 1964 | Ben Finney From ziade.tarek at gmail.com Sat May 23 00:51:34 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 23 May 2009 00:51:34 +0200 Subject: [Python-ideas] os.listdir with current working directory as default Message-ID: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> Hello, A very small change: what about making the path argument optional for os.listdir ? and use the current working directory if not provided. listdir(...) listdir(path=None) -> list_of_strings Return a list containing the names of the entries in the directory. path: path of directory to list. If path is None, os.getcwd() is used. The list is in arbitrary order. It does not include the special entries '.' and '..' even if they are present in the directory. Cheers Tarek -- Tarek Ziad? | http://ziade.org From solipsis at pitrou.net Sat May 23 01:14:20 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 22 May 2009 23:14:20 +0000 (UTC) Subject: [Python-ideas] os.listdir with current working directory as default References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> Message-ID: Tarek Ziad? writes: > > A very small change: what about making the path argument optional for > os.listdir ? and use the current working directory if > not provided. Disagreed. If you want the current directory, you just have to use ".". Regards Antoine. From g.brandl at gmx.net Sat May 23 01:16:18 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 23 May 2009 01:16:18 +0200 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> Message-ID: Antoine Pitrou schrieb: > Tarek Ziad? writes: >> >> A very small change: what about making the path argument optional for >> os.listdir ? and use the current working directory if >> not provided. > > Disagreed. If you want the current directory, you just have to use ".". or os.curdir to be precise :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sat May 23 01:19:59 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 23 May 2009 01:19:59 +0200 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> Message-ID: <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> On Sat, May 23, 2009 at 1:16 AM, Georg Brandl wrote: > Antoine Pitrou schrieb: >> Tarek Ziad? writes: >>> >>> A very small change: what about making the path argument optional for >>> os.listdir ? and use the current working directory if >>> not provided. >> >> Disagreed. If you want the current directory, you just have to use ".". > > or os.curdir to be precise :) yes, same question then, make these equivalent: os.listdir(os.curdir) <-> os.listdir() From g.brandl at gmx.net Sat May 23 01:46:08 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 23 May 2009 01:46:08 +0200 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> Message-ID: Tarek Ziad? schrieb: > On Sat, May 23, 2009 at 1:16 AM, Georg Brandl wrote: >> Antoine Pitrou schrieb: >>> Tarek Ziad? writes: >>>> >>>> A very small change: what about making the path argument optional for >>>> os.listdir ? and use the current working directory if >>>> not provided. >>> >>> Disagreed. If you want the current directory, you just have to use ".". >> >> or os.curdir to be precise :) > > yes, same question then, make these equivalent: > > os.listdir(os.curdir) <-> os.listdir() This seems to be a nice case of EIBTI to me. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sat May 23 02:01:10 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sat, 23 May 2009 02:01:10 +0200 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> Message-ID: <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> On Sat, May 23, 2009 at 1:46 AM, Georg Brandl wrote: >> >> os.listdir(os.curdir) ?<-> os.listdir() > > This seems to be a nice case of EIBTI to me. I would rather call that a default behavior, otherwise every function with arguments that have default values would be nice cases of EIBTI. From python at rcn.com Sat May 23 02:08:15 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 22 May 2009 17:08:15 -0700 Subject: [Python-ideas] os.listdir with current working directory asdefault References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> Message-ID: >> os.listdir(os.curdir) <-> os.listdir() +1 Lots of functions have useful defaults and this one would be familiar to everyone who programs (what does "dir" do without arguments on Windows or an "ls" do without arguments for Linux). Raymond From george.sakkis at gmail.com Sat May 23 02:09:06 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Fri, 22 May 2009 20:09:06 -0400 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> Message-ID: <91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com> On Fri, May 22, 2009 at 8:01 PM, Tarek Ziad? wrote: > On Sat, May 23, 2009 at 1:46 AM, Georg Brandl wrote: >>> >>> os.listdir(os.curdir) ?<-> os.listdir() >> >> This seems to be a nice case of EIBTI to me. > > I would rather call that a default behavior, otherwise every function > with arguments that have default > values would be nice cases of EIBTI. Agreed, nobody writes "dir ." or "ls .", the implicit argument is obvious. George From rdmurray at bitdance.com Sat May 23 03:28:32 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 23 May 2009 01:28:32 +0000 (UTC) Subject: [Python-ideas] os.listdir with current working directory asdefault References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> Message-ID: "Raymond Hettinger" wrote: > >> os.listdir(os.curdir) <-> os.listdir() > > +1 > > Lots of functions have useful defaults and this one would be familiar to everyone who programs (what does "dir" do without arguments > on Windows or an "ls" do without arguments for Linux). +1. I remember being surprised that os.listdir() did not work (I was expecting the requested behavior). If this were not parallel to the default behavior of ls and dir I'd be -1, but as it is I think it is less surprising for it to work. --David From tleeuwenburg at gmail.com Sat May 23 03:46:45 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Sat, 23 May 2009 11:46:45 +1000 Subject: [Python-ideas] os.listdir with current working directory asdefault In-Reply-To: References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> Message-ID: <43c8685c0905221846o69670991ye001d0c63571d238@mail.gmail.com> +1 On Sat, May 23, 2009 at 11:28 AM, R. David Murray wrote: > "Raymond Hettinger" wrote: > > >> os.listdir(os.curdir) <-> os.listdir() > > > > +1 > > > > Lots of functions have useful defaults and this one would be familiar to > everyone who programs (what does "dir" do without arguments > > on Windows or an "ls" do without arguments for Linux). > > +1. I remember being surprised that os.listdir() did not work > (I was expecting the requested behavior). If this were not > parallel to the default behavior of ls and dir I'd be -1, but > as it is I think it is less surprising for it to work. > > --David > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Sat May 23 18:54:20 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 23 May 2009 18:54:20 +0200 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: <91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com> References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> <91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com> Message-ID: George Sakkis schrieb: > On Fri, May 22, 2009 at 8:01 PM, Tarek Ziad? wrote: >> On Sat, May 23, 2009 at 1:46 AM, Georg Brandl wrote: >>>> >>>> os.listdir(os.curdir) <-> os.listdir() >>> >>> This seems to be a nice case of EIBTI to me. >> >> I would rather call that a default behavior, otherwise every function >> with arguments that have default >> values would be nice cases of EIBTI. If a default value is there, it should be obvious. In this case, my feeling was that it wasn't, but thinking about it I don't really see another default, so +0. > Agreed, nobody writes "dir ." or "ls .", the implicit argument is obvious. Well, the interactive shell shouldn't be considered a programming language, and even if you do, it's hardly Pythonic :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sun May 24 02:08:44 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 24 May 2009 02:08:44 +0200 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> <91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com> Message-ID: <94bdd2610905231708o1222551axf7ef449fc1329621@mail.gmail.com> On Sat, May 23, 2009 at 6:54 PM, Georg Brandl wrote: > > If a default value is there, it should be obvious. ?In this case, my feeling > was that it wasn't, but thinking about it I don't really see another default, > so +0. Since the feedbacks are positive, I have created an issue, and I'll work on a patch (#6095) > >> Agreed, nobody writes "dir ." or "ls .", the implicit argument is obvious. > > Well, the interactive shell shouldn't be considered a programming language, > and even if you do, it's hardly Pythonic :) I don't think it's a matter of being in the interactive shell or not. As soon as your program is working with the filesystem, and is navigating into it by calling APIs like os.chdir, os.getcwd, etc, you have good chances to end up calling os.listdir on the current directory, and specifying os.curdir becomes superfluous imho because you know you are in os.curdir. Tarek From rdmurray at bitdance.com Sun May 24 19:54:46 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 24 May 2009 17:54:46 +0000 (UTC) Subject: [Python-ideas] os.listdir with current working directory as default References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> <91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com> Message-ID: Georg Brandl wrote: > George Sakkis schrieb: > > On Fri, May 22, 2009 at 8:01 PM, Tarek Ziad? wrote: > >> On Sat, May 23, 2009 at 1:46 AM, Georg Brandl wrote: > >>>> > >>>> os.listdir(os.curdir) <-> os.listdir() > >>> > >>> This seems to be a nice case of EIBTI to me. > >> > >> I would rather call that a default behavior, otherwise every function > >> with arguments that have default > >> values would be nice cases of EIBTI. > > If a default value is there, it should be obvious. In this case, my feeling > was that it wasn't, but thinking about it I don't really see another default, > so +0. > > > Agreed, nobody writes "dir ." or "ls .", the implicit argument is obvious. > > Well, the interactive shell shouldn't be considered a programming language, > and even if you do, it's hardly Pythonic :) Ah, but in unix the interactive shell _is_ a programming language. :) But no, it isn't very pythonic, in many areas. --David From guido at python.org Sun May 24 23:13:53 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 24 May 2009 14:13:53 -0700 Subject: [Python-ideas] os.listdir with current working directory asdefault In-Reply-To: References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> Message-ID: 2009/5/22 Raymond Hettinger : >>> os.listdir(os.curdir) <-> os.listdir() > > +1 > > Lots of functions have useful defaults and this one would be familiar to > everyone who programs (what does "dir" do without arguments on Windows or an > "ls" do without arguments for Linux). +1 Can't see how it would really cause any confusion. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Sun May 24 23:40:31 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 24 May 2009 21:40:31 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?os=2Elistdir_with_current_working_direct?= =?utf-8?q?ory_as=09default?= References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> <91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com> <94bdd2610905231708o1222551axf7ef449fc1329621@mail.gmail.com> Message-ID: Tarek Ziad? writes: > > As soon as your program is working with the filesystem, and is > navigating into it by calling APIs like os.chdir, os.getcwd, etc, > you have good chances to end up calling os.listdir on the current > directory, and specifying os.curdir becomes superfluous imho > because you know you are in os.curdir. You'd better be careful with it. The "current directory" is process-wide and so modifying it in a thread will affect all other threads in your program. I'm not saying all programs have this characteristic, but relying on the current directory rather than explicit paths isn't always a good idea... Regards Antoine. From cs at zip.com.au Mon May 25 01:05:57 2009 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 25 May 2009 09:05:57 +1000 Subject: [Python-ideas] Python-ideas os.listdir with current working directory as?default In-Reply-To: Message-ID: <20090524230557.GA22907@cskk.homeip.net> On 24May2009 21:40, Antoine Pitrou wrote: | Tarek Ziad? writes: | > As soon as your program is working with the filesystem, and is | > navigating into it by calling APIs like os.chdir, os.getcwd, etc, | > you have good chances to end up calling os.listdir on the current | > directory, and specifying os.curdir becomes superfluous imho | > because you know you are in os.curdir. | | You'd better be careful with it. The "current directory" is process-wide | and so modifying it in a thread will affect all other threads in your | program. I'm not saying all programs have this characteristic, but relying | on the current directory rather than explicit paths isn't always a good idea... True, but if he's doing that the issue is already there. Changing listdir(curdir()) into listdir() isn't going the change any semantics. On the other hand, I've several times been bitten by shell scripts that change directory before doing something and in so doing quietly break some relative filenames... Same problem. Still, the problem's not with giving listdir a default, specificly. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Trust the computer industry to shorten Year 2000 to Y2K. It was this thinking that caused the problem in the first place. - Mark Ovens From Ronny.Pfannschmidt at gmx.de Mon May 25 10:23:24 2009 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Mon, 25 May 2009 10:23:24 +0200 Subject: [Python-ideas] proxy objects and call-stack depend globals Message-ID: <1243239804.23239.8.camel@localhost> Hi, for testing and dealing with thread-depend stdout/stderr redirection (__so__ helpfull for embedded debug consoles in gui apps), i wrote a simple lib to create global proxy objects which may point to different objects depending on the context of the call stack. The whole thing is based on a simple proxy implementation taken from werkzeug and a context managers to handle the current context of that global. I extracted the proxying bits to a base class I'ts availiable at http://pypi.python.org/pypi/pyscope and http://bitbucket.org/RonnyPfannschmidt/pyscope/overview/ Im hoping for some input on how to do it better and if something along that lines would be a usefull extension to the stdlib. Regards Ronny Pfannschmidt From aaron.rubin at 4dtechnology.com Mon May 25 21:35:18 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Mon, 25 May 2009 12:35:18 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <87pre0zrlz.fsf@benfinney.id.au> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> Message-ID: <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> OK. Again, this thread appears dead to the OP, since it has been delving into subjectiveness for quite a while now. If anybody has more objective arguments to propose, I would love to hear them. If not, then hopefully someone will read this thread and pick out the objective points for future consideration of this highly debated point. Remember that if Python is to remain a choice for future generations, its ability to be dynamic needs to remain. Times change, tools change. Languages must change as well. Making Python a hospitable environment for future generations to contribute will be important. This is what made my choice of Python 10 years ago an easy choice. Let's not forget to be dynamic so that others will make the same choice down the road. - Aaron On Fri, May 22, 2009 at 3:43 PM, Ben Finney > wrote: > Jan Kanis writes: > > > There is, of course, a correct solution to this line length problem. > > The ideal maximum line length is dependent on how much horizontal > > space you've got > > It's also dependent on how different line lengths affect ease of rapid > comprehension. As mentioned many times in this thread. > > > so the code should be automatically wrapped to your window size. > > No, since that fails to account for the dependency I mention above. > > -- > \ ?Science is a way of trying not to fool yourself. The first | > `\ principle is that you must not fool yourself, and you are the | > _o__) easiest person to fool.? ?Richard P. Feynman, 1964 | > Ben Finney > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mwm-keyword-python.b4bdba at mired.org Mon May 25 22:48:24 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Mon, 25 May 2009 16:48:24 -0400 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> Message-ID: <20090525164824.1ee30ecb@bhuda.mired.org> On Mon, 25 May 2009 12:35:18 -0700 Aaron Rubin wrote: > OK. Again, this thread appears dead to the OP, since it has been delving > into subjectiveness for quite a while now. Always a problem on those pesky interwebs :-). > Remember that if Python is to remain a choice for future generations, its > ability to be dynamic needs to remain. Times change, tools change. > Languages must change as well. Making Python a hospitable environment for > future generations to contribute will be important. And this is another example of topic drift. The OP wasn't suggesting a change to Python, so a claim that it must change is pretty clearly off topic. One thing that those espousing such a change keep forgetting is that the 79 character limit (it's not 80 - go read the PEP) isn't part of the language. It's part of a style guideline that describes what the community feels is a good style. Nothing actually forces anyone to follow it. The standard library includes code that violates the PEP, even in the public APIs (though the PEP-conforming variants have been made available recently). If you disagree with PEP 8, you're free to ignore all or part of it in your code. That won't keep the code out of the standard library if it's otherwise worthy and still readable. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From paddy3118 at gmail.com Tue May 26 12:32:58 2009 From: paddy3118 at gmail.com (Donald 'Paddy' McCarthy) Date: Tue, 26 May 2009 11:32:58 +0100 Subject: [Python-ideas] pipe function for itertools? Message-ID: <4A1BC55A.9090708@gmail.com> Hi, I have a blog entry, http://paddy3118.blogspot.com/2009/05/pipe-fitting-with-python-generators.html, in which I define a helper function to get around the problem of reversion in the nesting of generators and the proliferation of nested brackets. See the blog entry for a fuller treatment, but in essence if you have a series of generators, and want the data to conceptually flow like this: gen1 -> gen2 -> gen3 -> gen4 ... You have to write: ...(gen4(gen3(gen2(gen1()))))... With pipe, you would write: pipe(gen1, gen2, gen3, gen4, ...) Which you could use like this: for data in pipe(...): do_something_with_data() If I use dots for indentation, then maybe the definition will come through to the group: def pipe(*cmds): ....gen = cmds[0] ....for cmd in cmds[1:]: ....... gen = cmd(gen) ....for x in gen: ....... yield x A couple of readers thought that it might be a good tool for itertools to have. There are other solutions out there that use classes to overload '|', or, but it seems more natural to me to use pipe(), even though I am a heavy Unix user. Please discuss. From zooko at zooko.com Tue May 26 17:58:05 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Tue, 26 May 2009 09:58:05 -0600 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> References: <985db6080905190943r662faa9fm26a87b4504d61551@mail.gmail.com> <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> Message-ID: On May 25, 2009, at 13:35 PM, Aaron Rubin wrote: > OK. Again, this thread appears dead to the OP, since it has been > delving into subjectiveness for quite a while now. > > If anybody has more objective arguments to propose, I would love to > hear them. Personally, I would consider "objective arguments" to be controlled, repeatasble, studies with quantitative results. I've seen such studies about light-background-dark-foreground vs. dark-foreground- light-background, which is why I use the former now. I haven't seen such studies about line width, especially not with Python text as opposed to English text. I will therefore spare you my personal, subjective opinion about this topic. Regards, Zooko From steve at pearwood.info Tue May 26 18:46:16 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 27 May 2009 02:46:16 +1000 Subject: [Python-ideas] pipe function for itertools? In-Reply-To: <4A1BC55A.9090708@gmail.com> References: <4A1BC55A.9090708@gmail.com> Message-ID: <200905270246.17920.steve@pearwood.info> On Tue, 26 May 2009 08:32:58 pm Donald 'Paddy' McCarthy wrote: > Hi, I have a blog entry, > http://paddy3118.blogspot.com/2009/05/pipe-fitting-with-python-genera >tors.html, in which I define a helper function to get around the > problem of reversion in the nesting of generators and the > proliferation of nested brackets. > > See the blog entry for a fuller treatment, but in essence if you have > a series of generators, and want the data to conceptually flow like > this: > > gen1 -> gen2 -> gen3 -> gen4 ... > > You have to write: > > ...(gen4(gen3(gen2(gen1()))))... > > With pipe, you would write: > > pipe(gen1, gen2, gen3, gen4, ...) > > Which you could use like this: > > for data in pipe(...): do_something_with_data() > > > If I use dots for indentation, then maybe the definition will come > through to the group: > > def pipe(*cmds): > ....gen = cmds[0] > ....for cmd in cmds[1:]: > ....... gen = cmd(gen) > ....for x in gen: > ....... yield x The function signature is misleading. It doesn't take a series of generator functions ("cmds"), it takes an initial iterable followed by a series of generator functions. It seems to me that a cleaner definition would be: def pipe(iterable, *generators): for gen in generators: iterable = gen(iterable) for x in iterable: yield x This does seem to be a special case of function composition. If functools grew a compose() function, you could write: from functools import compose def pipe(it, *gens): for x in compose(*gens)(it): yield x Here's an untested definition for compose: def compose(f, *funcs, **kwargs): if not funcs: raise TypeError('compose() requires at least two functions') if kwargs.keys() not in ([], ['doc']): # I wish Python 2.5 had keyword only args... raise TypeError('bad keyword argument(s)') def function_composition(*args, **kwargs): value = f(*args, **kwargs) for g in funcs: value = g(value) return value function_composition.__doc__ = kwargs.get('doc') return function_composition which is more complicated than your version, of course, but also more general. > A couple of readers thought that it might be a good tool for > itertools to have. Do you have any other use-cases? -- Steven D'Aprano From aahz at pythoncraft.com Tue May 26 18:47:05 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 26 May 2009 09:47:05 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> Message-ID: <20090526164704.GA21219@panix.com> On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: > > Personally, I would consider "objective arguments" to be controlled, > repeatasble, studies with quantitative results. I've seen such studies > about light-background-dark-foreground vs. dark-foreground- > light-background, which is why I use the former now. I haven't seen > such studies about line width, especially not with Python text as > opposed to English text. Personally, I would be amazed to see any significant difference between the two foreground/background combinations you list. ;-) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "In many ways, it's a dull language, borrowing solid old concepts from many other languages & styles: boring syntax, unsurprising semantics, few automatic coercions, etc etc. But that's one of the things I like about it." --Tim Peters on Python, 16 Sep 1993 From steve at pearwood.info Tue May 26 18:53:20 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 27 May 2009 02:53:20 +1000 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090526164704.GA21219@panix.com> References: <20090526164704.GA21219@panix.com> Message-ID: <200905270253.20468.steve@pearwood.info> On Wed, 27 May 2009 02:47:05 am Aahz wrote: > On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: > > Personally, I would consider "objective arguments" to be > > controlled, repeatasble, studies with quantitative results. I've > > seen such studies about light-background-dark-foreground vs. > > dark-foreground- light-background, which is why I use the former > > now. I haven't seen such studies about line width, especially not > > with Python text as opposed to English text. > > Personally, I would be amazed to see any significant difference > between the two foreground/background combinations you list. ;-) Well, if you put the foreground *behind* the background, you can't see it at all unless you turn the monitor around, and then all the letters are reversed. -- Steven D'Aprano From dangyogi at gmail.com Tue May 26 19:07:43 2009 From: dangyogi at gmail.com (Bruce Frederiksen) Date: Tue, 26 May 2009 13:07:43 -0400 Subject: [Python-ideas] pipe function for itertools? In-Reply-To: <4A1BC55A.9090708@gmail.com> References: <4A1BC55A.9090708@gmail.com> Message-ID: <4A1C21DF.4030901@gmail.com> Donald 'Paddy' McCarthy wrote: > def pipe(*cmds): > ....gen = cmds[0] > ....for cmd in cmds[1:]: > ....... gen = cmd(gen) > ....for x in gen: > ....... yield x def pipe(*cmds): ....gen = cmds[0] ....for cmd in cmds[1:]: ....... gen = cmd(gen) ....return gen Would be more efficient. And doesn't change how it's used... -bruce frederiksen From floris.bruynooghe at gmail.com Tue May 26 19:35:49 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Tue, 26 May 2009 18:35:49 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090526164704.GA21219@panix.com> References: <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> <20090526164704.GA21219@panix.com> Message-ID: <20090526173549.GB30647@laurie.devork> On Tue, May 26, 2009 at 09:47:05AM -0700, Aahz wrote: > On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: > > > > Personally, I would consider "objective arguments" to be controlled, > > repeatasble, studies with quantitative results. I've seen such studies > > about light-background-dark-foreground vs. dark-foreground- > > light-background, which is why I use the former now. I haven't seen > > such studies about line width, especially not with Python text as > > opposed to English text. > > Personally, I would be amazed to see any significant difference between > the two foreground/background combinations you list. ;-) Entirely off topic by now, but these differences are more significant for e.g. dyslectic people. And these studies exist and are the reason exam papers in the UK are printed black on yellow etc. To be more no-topic, for as long as I've knowingly used computers my text editor window (and thus programming environment) has been 40x80 (emacs for me ;-)). Also having code no longer then 80 chars is nice when debugging on (serial) consoles, which does happen unfortunately. You're not always in your comfy development environment. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From python at rcn.com Tue May 26 21:30:01 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 26 May 2009 12:30:01 -0700 Subject: [Python-ideas] pipe function for itertools? References: <4A1BC55A.9090708@gmail.com> Message-ID: [Donald 'Paddy' McCarthy] > Hi, I have a blog entry, http://paddy3118.blogspot.com/2009/05/pipe-fitting-with-python-generators.html, > in which I define a helper function to get around the problem of > reversion in the nesting of generators and the proliferation of nested > brackets. > > See the blog entry for a fuller treatment, but in essence if you have > a series of generators, and want the data to conceptually flow like > this: > > gen1 -> gen2 -> gen3 -> gen4 ... > > You have to write: > > ...(gen4(gen3(gen2(gen1()))))... > > With pipe, you would write: > > pipe(gen1, gen2, gen3, gen4, ...) I'm not too excited about this for several reasons. * Many itertools do not stack linearly. The groupby() tool emits a stream of (key, generator) pairs that do not readily feed into other itertools (the generator part is meant to be consumed right away). The tee() tool emits multiple streams meant to be consumed contemporaneously. The zip tools take in multiple input steams. Some like count() and repeat() are meant to be feed into zip alongside other streams. IOW, piping is only a natural model for a limited subset of use cases. * The existing approaches let you handle multiply nested tools by simply assigning intermedate results to variables: it1, it2 = tee(iterable) decorated = izip(imap(func, it1), count(), it2) processed = sometool(decorated, somearg) undecorated = imap(itemgetter(2), processed) return undecorated * I don't like conflating the mental models for pipes with those for generators. While there are similarities, there are also differences that become less evident when a piping notation is used. Operating system pipes are buffered and producer processes can get suspended while consumers catch up. With generators, the consumer functions are in-charge, not the producers. * There doesn't seem to be much of an advantage to a pipe notation for generators. No new capabilites are added. It is essentially just a new syntax for a subset of things we already do. And, the notation becomes awkward and forced for use cases with multiple input streams and multiple output streams. Why add yet another way to do it? Raymond From aahz at pythoncraft.com Wed May 27 01:12:09 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 26 May 2009 16:12:09 -0700 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090526173549.GB30647@laurie.devork> References: <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> <20090526164704.GA21219@panix.com> <20090526173549.GB30647@laurie.devork> Message-ID: <20090526231209.GB18886@panix.com> On Tue, May 26, 2009, Floris Bruynooghe wrote: > On Tue, May 26, 2009 at 09:47:05AM -0700, Aahz wrote: >> On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: >>> >>> Personally, I would consider "objective arguments" to be controlled, >>> repeatasble, studies with quantitative results. I've seen such studies >>> about light-background-dark-foreground vs. dark-foreground- >>> light-background, which is why I use the former now. I haven't seen >>> such studies about line width, especially not with Python text as >>> opposed to English text. >> >> Personally, I would be amazed to see any significant difference between >> the two foreground/background combinations you list. ;-) > > Entirely off topic by now, but these differences are more significant > for e.g. dyslectic people. And these studies exist and are the reason > exam papers in the UK are printed black on yellow etc. In case it wasn't clear, Zooko gave exactly the same color combo for foreground/background, only reversed in ordering. It was a jest at his expense for the typo. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "In many ways, it's a dull language, borrowing solid old concepts from many other languages & styles: boring syntax, unsurprising semantics, few automatic coercions, etc etc. But that's one of the things I like about it." --Tim Peters on Python, 16 Sep 1993 From robert.kern at gmail.com Wed May 27 01:56:30 2009 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 26 May 2009 18:56:30 -0500 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090526231209.GB18886@panix.com> References: <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> <20090526164704.GA21219@panix.com> <20090526173549.GB30647@laurie.devork> <20090526231209.GB18886@panix.com> Message-ID: On 2009-05-26 18:12, Aahz wrote: > On Tue, May 26, 2009, Floris Bruynooghe wrote: >> On Tue, May 26, 2009 at 09:47:05AM -0700, Aahz wrote: >>> On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: >>>> Personally, I would consider "objective arguments" to be controlled, >>>> repeatasble, studies with quantitative results. I've seen such studies >>>> about light-background-dark-foreground vs. dark-foreground- >>>> light-background, which is why I use the former now. I haven't seen >>>> such studies about line width, especially not with Python text as >>>> opposed to English text. >>> Personally, I would be amazed to see any significant difference between >>> the two foreground/background combinations you list. ;-) >> Entirely off topic by now, but these differences are more significant >> for e.g. dyslectic people. And these studies exist and are the reason >> exam papers in the UK are printed black on yellow etc. > > In case it wasn't clear, Zooko gave exactly the same color combo for > foreground/background, only reversed in ordering. It was a jest at his > expense for the typo. Floris is obviously using a dark-foreground-light-background color scheme, or he wouldn't have misread that. :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From floris.bruynooghe at gmail.com Wed May 27 08:21:27 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 27 May 2009 07:21:27 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: References: <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> <20090526164704.GA21219@panix.com> <20090526173549.GB30647@laurie.devork> <20090526231209.GB18886@panix.com> Message-ID: <20090527062127.GA7159@laurie.devork> On Tue, May 26, 2009 at 06:56:30PM -0500, Robert Kern wrote: > On 2009-05-26 18:12, Aahz wrote: >> On Tue, May 26, 2009, Floris Bruynooghe wrote: >>> On Tue, May 26, 2009 at 09:47:05AM -0700, Aahz wrote: >>>> On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: >>>>> Personally, I would consider "objective arguments" to be controlled, >>>>> repeatasble, studies with quantitative results. I've seen such studies >>>>> about light-background-dark-foreground vs. dark-foreground- >>>>> light-background, which is why I use the former now. I haven't seen >>>>> such studies about line width, especially not with Python text as >>>>> opposed to English text. >>>> Personally, I would be amazed to see any significant difference between >>>> the two foreground/background combinations you list. ;-) >>> Entirely off topic by now, but these differences are more significant >>> for e.g. dyslectic people. And these studies exist and are the reason >>> exam papers in the UK are printed black on yellow etc. >> >> In case it wasn't clear, Zooko gave exactly the same color combo for >> foreground/background, only reversed in ordering. It was a jest at his >> expense for the typo. > > Floris is obviously using a dark-foreground-light-background color > scheme, or he wouldn't have misread that. :-) Oh, re-read that a few times before I finally saw it. Doh! I actually use both, light-before-dark on my terminals and dark-before-light on my editor. They're both so pretty... Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From cmjohnson.mailinglist at gmail.com Wed May 27 09:15:13 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Tue, 26 May 2009 21:15:13 -1000 Subject: [Python-ideas] pipe function for itertools? In-Reply-To: References: <4A1BC55A.9090708@gmail.com> Message-ID: <3bdda690905270015s17df7f67o57ff3cfbf17f5bef@mail.gmail.com> I was one of the commenters who said you should put the idea here. I end up using generators in a pipe-like fashion quite a bit, so a pipe [iter, func]tool would be useful. I also liked the head function listed on the page, because when using the interactive interpreter I often want to see just a part of a larger output without having to go to all the trouble of whipping up something using islice or izip + irange. I think we could also use a tail function, and although obviously it wouldn't work with itertools.count it is useful when you're filtering a large file, and you just want to see the last part of the output to tell if your filters are basically doing the right thing. Of course, all of these functions are somewhat trivial, but it could save a little work for a lot of people if they were in the stdlib. Also, they could be made to work like the proposed "yield from" and pass through .send and .raise commands, etc., which actually is a fair amount of work to implement. -- Carl Johnson From arnodel at googlemail.com Wed May 27 14:52:27 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Wed, 27 May 2009 13:52:27 +0100 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <20090526173549.GB30647@laurie.devork> References: <2690E090A497426AA35558462DB8FC2A@RaymondLaptop1> <200905210902.48106.steve@pearwood.info> <64C29AE7A74B41A78998758031ADBCFF@RaymondLaptop1> <20090521090655.7695a0c5@o> <59a221a0905220632n4f977145q81d77ceaf2235a56@mail.gmail.com> <87pre0zrlz.fsf@benfinney.id.au> <985db6080905251235m18e3da5ds3583168cf0ff0f0a@mail.gmail.com> <20090526164704.GA21219@panix.com> <20090526173549.GB30647@laurie.devork> Message-ID: On 26 May 2009, at 18:35, Floris Bruynooghe wrote: > On Tue, May 26, 2009 at 09:47:05AM -0700, Aahz wrote: >> On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: >>> >>> Personally, I would consider "objective arguments" to be controlled, >>> repeatasble, studies with quantitative results. I've seen such >>> studies >>> about light-background-dark-foreground vs. dark-foreground- >>> light-background, which is why I use the former now. I haven't seen >>> such studies about line width, especially not with Python text as >>> opposed to English text. >> >> Personally, I would be amazed to see any significant difference >> between >> the two foreground/background combinations you list. ;-) > > Entirely off topic by now, but these differences are more significant > for e.g. dyslectic people. And these studies exist and are the reason > exam papers in the UK are printed black on yellow etc. Which papers? Not SATs, GCSEs or A-Levels. -- Arnaud From dreamingforward at gmail.com Wed May 27 22:29:57 2009 From: dreamingforward at gmail.com (average) Date: Wed, 27 May 2009 13:29:57 -0700 Subject: [Python-ideas] 80 character line width vs. something wider Message-ID: <913f9f570905271329k433eca1am759f6cb3a054e2c1@mail.gmail.com> On Tue, May 26, 2009 at 09:47:05AM -0700, Aahz wrote: > On Tue, May 26, 2009, Zooko Wilcox-O'Hearn wrote: >> >> Personally, I would consider "objective arguments" to be controlled, >> repeatasble, studies with quantitative results. I've seen such studies >> about light-background-dark-foreground vs. dark-foreground- >> light-background, which is why I use the former now. I haven't seen >> such studies about line width, especially not with Python text as >> opposed to English text. > > Personally, I would be amazed to see any significant difference between > the two foreground/background combinations you list. ;-) BTW, I know the topic's pretty dead by now, but there is a pretty conclusive argument on this topic (which I bring back up only because I find myself continually annoyed at most IDE's). If the medium *emits* light, it's significantly better to have a dark background (the reverse being true if the medium is reflective--like the surface of a book or Kindle). Several arguments *support* this and none *contradict* it: 1) Examine the edge cases. Which would you fatigue in front of faster: a monitor emitting and filled with bright white light or one that is dark? (duh) 2) Adjust screen refresh to 60Hz. Text your source code on the screen with the different foregrounds. Stand back and move your head around. With which do you see the flicker? (answer: white background) 3) The retina has to resolve the actual raw and otherwise meaningless dark and light spots into useful information. Partly due to the resolving power of the retina being greater than the display and also because of interpolation effects that happen within the retina itself, there's going to be a slight advantage if the actual data on the page is emitting the light (i.e. a light foreground) rather than having to compete with trying to stand out against the white wash bleeding over. The significance of this problem is inversely proportional to the fatness of the fonts (finer fonts-->more problem). 4) It's easier on your monitor and your ears (if your using a CRT). The photon gun only has to emit streams of particles on the "on" bits and can be off less, saving your shadow mask and those 15kHz screams emitting from the back. 5) Less power consumption. I'd estimate the power savings roughly 60% and higher on CRTs. If anyone has one of those $15 Kill-o-Watt power measuring device they can check. And for those who would otherwise take a populist approach to the problem ("gee if it was better, it would have been adopted by now...."), consider that the main reason that white backgrounds are common is because the guys at XEROX Park took either a naive approach to wanting a display "more like the real world" or because they knew that it would be that much more striking and novel at a time when green monochrome was the norm. It was then subsequently and blindly copied by Apple who wanted to be like XEROX, and then again by MS Windows which wanted to compete with the Mac. I seriously doubt ($0.02 blah blah) that any of these companies propagated the idea based on research into white-foreground effects on neural fatigue and such. And now it's a serious chore to re-customize all the colors to work well with a dark background and that totally *blows*.... Hope that puts THAT issue to rest.... ;^) marcos From zooko at zooko.com Thu May 28 20:25:35 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Thu, 28 May 2009 12:25:35 -0600 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <913f9f570905271329k433eca1am759f6cb3a054e2c1@mail.gmail.com> References: <913f9f570905271329k433eca1am759f6cb3a054e2c1@mail.gmail.com> Message-ID: <908366B0-79BE-48C6-86B9-F9C26AEE5469@zooko.com> On May 27, 2009, at 14:29 PM, average wrote: > BTW, I know the topic's pretty dead by now, but there is a pretty > conclusive argument on this topic (which I bring back up only > because I find myself continually annoyed at most IDE's). > > If the medium *emits* light, it's significantly better to have a > dark background (the reverse being true if the medium is > reflective--like the surface of a book or Kindle). Several > arguments *support* this and none *contradict* it: Ah yes, there are arguments and there are arguments. The ones you posted sound very persuasive. But in several controlled studies, the test subjects detected errors at a significantly better rate when the background was light than when it was dark. They did not report greater fatigue than the subjects who had a dark background. I try to rely on arguments only when I can't find controlled studies to rely on. Oh, thanks to denis.spir I now see that the explanation is probably due to overall luminance rather than to polarity per se: http://www.math-nat-fak.uni-duesseldorf.de/WE/Psychologie/abteilungen/ aap/Dokumente/Buchner-et-al.-in-press-Ergonomics.pdf Although this probably doesn't change the fact that you'll find errors better if you set your background to be light and your text to be dark. Regards, Zooko P.S. The tenuous link to the topic of this list is that if there *were* any controlled studies about Python programmers using different line lengths, then they could give us something to go on. Maybe somebody out there knows how to get research funding for experimenting on programmers? That would be wonderful. From greg.ewing at canterbury.ac.nz Fri May 29 03:20:41 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 29 May 2009 13:20:41 +1200 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <908366B0-79BE-48C6-86B9-F9C26AEE5469@zooko.com> References: <913f9f570905271329k433eca1am759f6cb3a054e2c1@mail.gmail.com> <908366B0-79BE-48C6-86B9-F9C26AEE5469@zooko.com> Message-ID: <4A1F3869.1000608@canterbury.ac.nz> > On May 27, 2009, at 14:29 PM, average wrote: >> If the medium *emits* light, it's significantly better to have a dark >> background (the reverse being true if the medium is reflective--like >> the surface of a book or Kindle). This assertion seems to contradict common sense. All the eye detects is patterns of light and dark -- how can it know whether the light it receives was emitted by the screen itself or reflected by it? If there is any such effect, there must be other factors involved, such as sharpness or resolution differences between the two display surfaces being compared. In my experience, dark-on-light tends to look sharper than light-on-dark for the same resolution on the same medium -- both emissive and reflective -- and it is therefore easier to read small-sized text that way. I expect that's why Xerox and followers chose black on white. It's also probably why we have a long tradition of printing black ink on white paper and not vice versa. So if Xerox were imitating paper, they weren't just doing it blindly, but for a reason. -- Greg From rrr at ronadam.com Fri May 29 04:16:58 2009 From: rrr at ronadam.com (Ron Adam) Date: Thu, 28 May 2009 21:16:58 -0500 Subject: [Python-ideas] 80 character line width vs. something wider In-Reply-To: <4A1F3869.1000608@canterbury.ac.nz> References: <913f9f570905271329k433eca1am759f6cb3a054e2c1@mail.gmail.com> <908366B0-79BE-48C6-86B9-F9C26AEE5469@zooko.com> <4A1F3869.1000608@canterbury.ac.nz> Message-ID: <4A1F459A.2040507@ronadam.com> Greg Ewing wrote: >> On May 27, 2009, at 14:29 PM, average wrote: >>> If the medium *emits* light, it's significantly better to have a >>> dark background (the reverse being true if the medium is >>> reflective--like the surface of a book or Kindle). > > This assertion seems to contradict common sense. All the > eye detects is patterns of light and dark -- how can it > know whether the light it receives was emitted by the > screen itself or reflected by it? > > If there is any such effect, there must be other factors > involved, such as sharpness or resolution differences > between the two display surfaces being compared. > > In my experience, dark-on-light tends to look sharper > than light-on-dark for the same resolution on the same > medium -- both emissive and reflective -- and it is > therefore easier to read small-sized text that way. > > I expect that's why Xerox and followers chose black > on white. It's also probably why we have a long > tradition of printing black ink on white paper and not > vice versa. So if Xerox were imitating paper, they > weren't just doing it blindly, but for a reason. It seems to me (just speculation) that it was probably easier and cheaper to bleach paper white or leave it a light color after making it than it would be to dye or color it black. And also probably easier and cheaper to make black ink than to make white ink. So I think the factors as to why books are black letters on white pages is historically economic in nature for practical reasons. I would also speculate that because we are so used to reading black letters on a white background, that it would be a bit more natural and easier to most people to also do that on computer screens. The reason may be precisely to imitate paper because that is what most people are used to reading ... and it was a selling point. ;-) Ron From gerald.britton at gmail.com Fri May 29 20:18:10 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 29 May 2009 14:18:10 -0400 Subject: [Python-ideas] with statement vs. try...except...finally Message-ID: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> I'm wondering if the "with" statement should have exception clauses like the "try" statement, even though this seems to defeat part of the reason for the "with" statement. Currently I have a program segment that opens a file and reads a line, something like this (distilled to its elements for illustration): try: f = open('foo') line = f.readline() f.close() except IOError: line = 'default' So that I get a default value if anything goes awry whilst reading the file. If I write it using a "with" statement, I might have: line = 'default' with open('foo') as f: line = f.readline() Fine so far, but what if I want to be more granular? e.g. with "try...except": try: f = open('foo') except IOError: line = "can't open" try: line = f.readline() except IOError: line = "can't read" try: f.close() except IOError: line = "can't close" I can't see how to replace the above try-triplet with a "with" encapsulation. Or, do I have to wrap the "with" statement in try like this: try: with open('foo') as f: line = f.readline() except IOError: line = 'problem with read or close" -- Gerald Britton From chambon.pascal at wanadoo.fr Fri May 29 21:49:31 2009 From: chambon.pascal at wanadoo.fr (Pascal Chambon) Date: Fri, 29 May 2009 21:49:31 +0200 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> Message-ID: <4A203C4B.4040806@wanadoo.fr> Gerald Britton a ?crit : > I'm wondering if the "with" statement should have exception clauses > like the "try" statement, even though this seems to defeat part of the > reason for the "with" statement. > > Currently I have a program segment that opens a file and reads a line, > something like this (distilled to its elements for illustration): > > try: > f = open('foo') > line = f.readline() > f.close() > except IOError: > line = 'default' > > So that I get a default value if anything goes awry whilst reading the file. > > If I write it using a "with" statement, I might have: > > line = 'default' > with open('foo') as f: > line = f.readline() > > Fine so far, but what if I want to be more granular? e.g. with "try...except": > > try: > f = open('foo') > except IOError: > line = "can't open" > > try: > line = f.readline() > except IOError: > line = "can't read" > > try: > f.close() > except IOError: > line = "can't close" > > I can't see how to replace the above try-triplet with a "with" > encapsulation. Or, do I have to wrap the "with" statement in try like > this: > > try: > > with open('foo') as f: > line = f.readline() > > except IOError: > line = 'problem with read or close" > > > I'd say the triplet of "try...except" clauses above isn't OK, because if the file opening fails, teh code will try anyway to read it and to close it, leading to nameError and other uncaught exception. But the last clause, wrapped in try..except, seems fine to me. ++ Pascal From gerald.britton at gmail.com Fri May 29 22:58:55 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 29 May 2009 16:58:55 -0400 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <4A203C4B.4040806@wanadoo.fr> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> <4A203C4B.4040806@wanadoo.fr> Message-ID: <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> Good point, so can this be translated to a "with" statement: try: f = open('foo') try: line = f.readline() try: f.close() except IOError: line = "can't close" except IOError: line = "can't read" except IOError: line = "can't open" ?? I guess I'm wondering if we need a with...except construct so that we can get exceptions that happen after the with context is entered without wrapping the with statement in try...except. On Fri, May 29, 2009 at 3:49 PM, Pascal Chambon wrote: > Gerald Britton a ?crit : >> >> I'm wondering if the "with" statement should have exception clauses >> like the "try" statement, even though this seems to defeat part of the >> reason for the "with" statement. >> >> Currently I have a program segment that opens a file and reads a line, >> something like this (distilled to its elements for illustration): >> >> try: >> ? ?f = open('foo') >> ? ?line = f.readline() >> ? ?f.close() >> except IOError: >> ? ?line = 'default' >> >> So that I get a default value if anything goes awry whilst reading the >> file. >> >> If I write it using a "with" statement, I might have: >> >> line = 'default' >> with open('foo') as f: >> ? line = f.readline() >> >> Fine so far, but what if I want to be more granular? ?e.g. with >> "try...except": >> >> try: >> ? ?f = open('foo') >> except IOError: >> ? ?line = "can't open" >> >> try: >> ? ?line = f.readline() >> except IOError: >> ? ?line = "can't read" >> >> try: >> ? ?f.close() >> except IOError: >> ? ?line = "can't close" >> >> I can't see how to replace the above try-triplet with a "with" >> encapsulation. Or, do I have to wrap the "with" statement in try like >> this: >> >> try: >> >> ?with open('foo') as f: >> ? ? line = f.readline() >> >> except IOError: >> ?line = 'problem with read or close" >> >> >> > > I'd say the triplet of "try...except" clauses above isn't OK, because if the > file opening fails, teh code will try anyway to read it and to close it, > leading to nameError and other uncaught exception. > > But the last clause, wrapped in try..except, seems fine to me. > > ++ > Pascal > > > > -- Gerald Britton From cmjohnson.mailinglist at gmail.com Sat May 30 00:02:31 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Fri, 29 May 2009 12:02:31 -1000 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> <4A203C4B.4040806@wanadoo.fr> <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> Message-ID: <3bdda690905291502r72ed5e11u2dad19ba3715f12d@mail.gmail.com> This example is too simplified to understand if it's worth adding an except clause or not. If all you're doing is pulling the text out of the file, why can't you just make a function for this? lines = fancyread("myfile.txt") #Returns "couldn't open", "couldn't close", etc. in case of errors The point of the with statement is that your want to do more sophisticated stuff with the file contents, but still want the file closed at the end. But if you're doing more sophisticated stuff, it's not clear that you'd suddenly want to have "can't close" substituted in for whatever it was you whipped up by processing the file contents. So, I think we need a better example before we can judge the merits of an except clause. There's also the question of what the except clause would apply to. There are three places where a "with" could throw an exception: with open("myfile") as f: #Could barf on opening f.read() #Could barf in processing #Could barf during clean up Which of the three spots will the except clause deal with? All of them? As it is, barfing in the middle is already given to the context manager to clean up (though the context manager could always throw its own error during its own clean up), so it would be weird to have any other custom clean up that went in addition to what the context manager is already doing. -- Carl From Scott.Daniels at Acm.Org Sat May 30 00:25:44 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri, 29 May 2009 15:25:44 -0700 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> Message-ID: Gerald Britton wrote: > I'm wondering if the "with" statement should have exception clauses > like the "try" statement, even though this seems to defeat part of the > reason for the "with" statement.... > something like: > try: > f = open('foo') > line = f.readline() > f.close() > except IOError: > line = 'default' > ... Fine so far, but what if I want to be more granular? ... > I can't see how to replace the above try-triplet with a "with" > encapsulation. Or, do I have to wrap the "with" statement in try like > this: How about using the error information?: try: with open('pov_export/.hg/requires') as src: line = src.read(30) except IOError, why: line = 'Problem: %s' % why --Scott David Daniels Scott.Daniels at Acm.Org From daniel at stutzbachenterprises.com Sat May 30 00:30:38 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 29 May 2009 17:30:38 -0500 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> Message-ID: On Fri, May 29, 2009 at 1:18 PM, Gerald Britton wrote: > I'm wondering if the "with" statement should have exception clauses > like the "try" statement, even though this seems to defeat part of the > reason for the "with" statement. FWIW, the same suggestion previously came up in the following thread: http://mail.python.org/pipermail/python-ideas/2009-February/003169.html -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat May 30 02:30:40 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 30 May 2009 10:30:40 +1000 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> <4A203C4B.4040806@wanadoo.fr> <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> Message-ID: <200905301030.41813.steve@pearwood.info> On Sat, 30 May 2009 06:58:55 am Gerald Britton wrote: > I guess I'm wondering if we need a with...except construct so that we > can get exceptions that happen after the with context is entered > without wrapping the with statement in try...except. Would this hypothetical construct: with some_context_manager() as obj: do_something_with(obj) except Exception: error_handler() catch exceptions in the context manager, the do_something_with() block, or both? Give reasons for why you make that choice. Personally, I don't see the need for this. We can already do: # Catch errors in both the context manager and the with block try: with open(filename) as f: process(f) except Exception: pass or this: # Catch errors in just the with block with open(filename) as f: try: process(f) except Exception: pass or this: # Catch errors in the context manager and the with block separately: try: with open(filename) as f: try: process(f) except Exception: print "Error in the with block" except Exception: print "Error in the context manager" and if for some bizarre reason you want to catch errors in the context manager but not the body of with, you can manually emulate with: # untested try: try: f = open(filename).__enter__() except Exception: f = None process(f) finally: if f is not None: f.__exit__() That fourth case is ugly, but it's also rare. I can't imagine a case where you'd need it, but if you do, you can do it. The point is, you can make your error handlers as fine grained as you want, by wrapping just the bits you need. You can nest try blocks, or execute them sequentially. All the flexibility you want is there, at the cost of nothing but an extra line and an indent level. What does with...except gain you that you can't already do easily? -- Steven D'Aprano From steve at pearwood.info Sat May 30 02:52:23 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 30 May 2009 10:52:23 +1000 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <3bdda690905291502r72ed5e11u2dad19ba3715f12d@mail.gmail.com> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> <3bdda690905291502r72ed5e11u2dad19ba3715f12d@mail.gmail.com> Message-ID: <200905301052.23712.steve@pearwood.info> On Sat, 30 May 2009 08:02:31 am Carl Johnson wrote: > This example is too simplified to understand if it's worth adding an > except clause or not. If all you're doing is pulling the text out of > the file, why can't you just make a function for this? > > lines = fancyread("myfile.txt") #Returns "couldn't open", "couldn't > close", etc. in case of errors Python has exceptions for exception handling. Unless you have a *really good reason*, you shouldn't go backwards to error codes, especially not such course-grained error codes that throw away useful information, and even more especially not error codes which can easily be mistaken for legitimate output: lines = fancyread(filename) if lines == "couldn't open": pass else: # oops, forgot to check for "couldn't read" for line in lines: process(line) will accidentally process each of 'c' 'o' 'u' ... 'r' 'e' 'a' 'd' if the file could be opened but there was an error reading from it. On the very rare case that I don't want to raise an exception on error, I return a tuple of (success_flag, result). If the function is successful, it returns (True, whatever) and if it fails, it returns (False, error-description). Sometimes I'll use the same idiom as string.find() and re.match(): return a sentinel (usually None) to stand in for "not found" or similar. I only do this if it represents something which is not an error condition, but an expected result. -- Steven D'Aprano From pyideas at rebertia.com Sat May 30 02:53:26 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Fri, 29 May 2009 17:53:26 -0700 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> Message-ID: <50697b2c0905291753tcfaed94mb299ac55da99ef7a@mail.gmail.com> On Fri, May 29, 2009 at 3:30 PM, Daniel Stutzbach wrote: > On Fri, May 29, 2009 at 1:18 PM, Gerald Britton > wrote: >> >> I'm wondering if the "with" statement should have exception clauses >> like the "try" statement, even though this seems to defeat part of the >> reason for the "with" statement. > > FWIW, the same suggestion previously came up in the following thread: > http://mail.python.org/pipermail/python-ideas/2009-February/003169.html Sounds like it also got a thumbs-down from the BDFL: http://mail.python.org/pipermail/python-ideas/2009-February/003184.html Cheers, Chris -- http://blog.rebertia.com From mwm-keyword-python.b4bdba at mired.org Sat May 30 03:14:01 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Fri, 29 May 2009 21:14:01 -0400 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <200905301030.41813.steve@pearwood.info> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> <4A203C4B.4040806@wanadoo.fr> <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> <200905301030.41813.steve@pearwood.info> Message-ID: <20090529211401.75c0ba35@bhuda.mired.org> On Sat, 30 May 2009 10:30:40 +1000 Steven D'Aprano wrote: > and if for some bizarre reason you want to catch errors in the context > manager but not the body of with, you can manually emulate with: > > # untested > try: > try: > f = open(filename).__enter__() > except Exception: > f = None > process(f) > finally: > if f is not None: > f.__exit__() > > > That fourth case is ugly, but it's also rare. I can't imagine a case > where you'd need it, but if you do, you can do it. That was the case that first drove me to think about extending with. The starting point was a loop: while retries < maxretries: try: with my_magic_lock: process() except ProcessException: handle() except: pass retries += 1 If entering my_magic_lock throws an exception, I don't care - I'm going to retry it anyway. We encountered a condition that caused my_magic_lock to throw ProcessException, so handle() would run without the lock and potentially corrupt our data. Extending with and wanting a way to throw an except *over* the first surrounding try:/except: both occurred to me. What I eventually did was push the loop into a wrapper context manager. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From guido at python.org Sat May 30 04:22:00 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 29 May 2009 19:22:00 -0700 Subject: [Python-ideas] with statement vs. try...except...finally In-Reply-To: <200905301030.41813.steve@pearwood.info> References: <5d1a32000905291118t4d1fa5b2saa899d89db53122d@mail.gmail.com> <4A203C4B.4040806@wanadoo.fr> <5d1a32000905291358v2c5416ffye676718574bd13e1@mail.gmail.com> <200905301030.41813.steve@pearwood.info> Message-ID: > On Sat, 30 May 2009 06:58:55 am Gerald Britton wrote: > >> I guess I'm wondering if we need a with...except construct so that we >> can get exceptions that happen after the with context is entered >> without wrapping the with statement in try...except. On Fri, May 29, 2009 at 5:30 PM, Steven D'Aprano wrote: > Would this hypothetical construct: > > with some_context_manager() as obj: > ? ?do_something_with(obj) > except Exception: > ? ?error_handler() > > catch exceptions in the context manager, the do_something_with() block, > or both? Give reasons for why you make that choice. -1. As Steven hints, this will just cause more bugs because people won't guess the right semantics for corner cases, and corner cases are all what except clauses are about. -- --Guido van Rossum (home page: http://www.python.org/~guido/)