From mwh at python.net Wed Jun 1 10:46:36 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 01 Jun 2005 09:46:36 +0100 Subject: [Python-Dev] Closing old bugs In-Reply-To: <000001c565d7$65bebc20$5807a044@oemcomputer> (Raymond Hettinger's message of "Tue, 31 May 2005 07:53:44 -0400") References: <000001c565d7$65bebc20$5807a044@oemcomputer> Message-ID: <2md5r6wjkj.fsf@starship.python.net> "Raymond Hettinger" writes: > There should be some greater care exercised in closing old bugs. Possibly. OTOH, we have something like 900 open bugs to work on, and it's not like bug reporters can't re-open a bug report if they think it's been closed in error (this has happened a few times already, and it's a good thing, IMHO). > Marking them "deprecated" and then erasing them is only a good strategy > if we have no means of reproducing the error or ascertaining what the OP > was talking about. > > For instance, in www.python.org/sf/640553 , it was possible for a > reviewer to directly verify whether usr/local local was still being used > in setup.py. Well, that one in particular was always in the "is it really a bug?" category. > Likewise, www.python.org/sf/728515 should not have been > closed (Martin's post could have been taken as a clue that the bug was > valid and simply waiting for some volunteer to submit a patch). > > Old age and a missing OP is not sufficient reason to close a bug. But if closing a bug is an effective way of kicking things into life again... Cheers, mwh -- ARTHUR: Don't ask me how it works or I'll start to whimper. -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From gvanrossum at gmail.com Wed Jun 1 17:16:44 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed, 1 Jun 2005 08:16:44 -0700 Subject: [Python-Dev] PEP 343 rewrite complete Message-ID: I hope that I've got the rewrite of PEP 343 to include generator extensions right now. I've chosen the 'with' keyword. Please review here; I think this is ready for review by the unwashed masses. :-) http://www.python.org/peps/pep-0343.html -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Wed Jun 1 17:23:49 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed, 1 Jun 2005 08:23:49 -0700 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: Message-ID: > http://www.python.org/peps/pep-0343.html I should add that IMO this obsoletes PEP 288 and PEP 325; I plan to reject those when PEP 343 is accepted. I've already withdrawn PEP 340. PEP 342 is separate (but I'll probably present it together with PEP 343). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Jun 1 18:16:56 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Jun 2005 12:16:56 -0400 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: Message-ID: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> At 08:16 AM 6/1/2005 -0700, Guido van Rossum wrote: >I hope that I've got the rewrite of PEP 343 to include generator >extensions right now. I've chosen the 'with' keyword. Please review >here; I think this is ready for review by the unwashed masses. :-) > > http://www.python.org/peps/pep-0343.html Looks great. A few questions/comments: * What's the rationale for raising TypeError from close()? Wasn't RuntimeError discussed previously for that role? (and it's also used by the with_template example) OTOH, maybe that means we want a ControlFlowError or some such that can be used for both. * The "opening" example under "Generator Decorator" seems to be missing a try/finally block. * The transaction handler could also be written as: @with_template def transactional(db): db.begin() try: yield db except: db.rollback() else: db.commit() at least, if I understand it correctly. From gvanrossum at gmail.com Wed Jun 1 18:50:39 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed, 1 Jun 2005 09:50:39 -0700 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> References: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> Message-ID: [Guido van Rossum] > > http://www.python.org/peps/pep-0343.html [Phillip J. Eby] > Looks great. A few questions/comments: > > * What's the rationale for raising TypeError from close()? Wasn't > RuntimeError discussed previously for that role? (and it's also used by > the with_template example) OTOH, maybe that means we want a > ControlFlowError or some such that can be used for both. I really don't want a new exception for this, since it's just a bug in the generator's code. I could go with RuntimeError here too, but I figured TypeError's meaning in a wider sense applies: the generator doesn't respond appropriately to the throw() call, which is similar to not handling a particular argument (list) correctly. > * The "opening" example under "Generator Decorator" seems to be missing a > try/finally block. Good catch. I've fixed this in the PEP. > * The transaction handler could also be written as: > > @with_template > def transactional(db): > db.begin() > try: > yield db > except: > db.rollback() > else: > db.commit() > > at least, if I understand it correctly. Ah, of course. I've updated the PEP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric.nieuwland at xs4all.nl Wed Jun 1 20:46:20 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Wed, 1 Jun 2005 20:46:20 +0200 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: Message-ID: <1cb1006397f1deed5a6670201ef95147@xs4all.nl> Nice going! But ... Could we extend the 'try' syntax for this instead of introducing 'with'? If I look at the translation it an augmented 'try'. with EXPR as VAR: BLOCK1 except EXCEPTION: BLOCK2 could then be translated to abc = EXPR exc = (None, None, None) VAR = abc.__enter__() try: try: BLOCK1 except EXCEPTION: BLOCK2 except: exc = sys.exc_info() raise finally: abc.__exit__(*exc) Can the 'throw()' method be renamed 'raise()'? IMHO that makes much clearer what happens. Same thing with 'GeneratorExit', 'StopGeneration' more closely matches 'StopIteration'. --eric From eric.nieuwland at xs4all.nl Wed Jun 1 20:57:47 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Wed, 1 Jun 2005 20:57:47 +0200 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <1cb1006397f1deed5a6670201ef95147@xs4all.nl> References: <1cb1006397f1deed5a6670201ef95147@xs4all.nl> Message-ID: Eric Nieuwland wrote: > If I look at the translation it an augmented 'try'. > with EXPR as VAR: > BLOCK1 > except EXCEPTION: > BLOCK2 Oops, that should read: try EXPR as VAR: BLOCK1 except EXCEPTION: BLOCK2 --eric From pje at telecommunity.com Wed Jun 1 21:18:28 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Jun 2005 15:18:28 -0400 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <1cb1006397f1deed5a6670201ef95147@xs4all.nl> References: Message-ID: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> At 08:46 PM 6/1/2005 +0200, Eric Nieuwland wrote: >Nice going! But ... > >Could we extend the 'try' syntax for this instead of introducing >'with'? If I look at the translation it an augmented 'try'. > with EXPR as VAR: > BLOCK1 > except EXCEPTION: > BLOCK2 >could then be translated to -1, too confusing. >Can the 'throw()' method be renamed 'raise()'? IMHO that makes much >clearer what happens. No, 'raise' is a reserved word. It would have to be 'raise_()'. -0. >Same thing with 'GeneratorExit', 'StopGeneration' more closely matches >'StopIteration'. StopIteration is raised the *other* way, so closely matching isn't really a benefit. -1. From MFC at uk.ibm.com Wed Jun 1 21:44:56 2005 From: MFC at uk.ibm.com (Mike Cowlishaw) Date: Wed, 1 Jun 2005 20:44:56 +0100 Subject: [Python-Dev] Adventures with Decimal In-Reply-To: <429140D5.7010908@canterbury.ac.nz> Message-ID: > Raymond Hettinger wrote: > > IMO, user input (or > > the full numeric strings in a text data file) is sacred and presumably > > done for a reason -- the explicitly requested digits should not be > > throw-away without good reason. > > I still don't understand what's so special about the > input phase that it should be treated sacredly, while > happily desecrating the result of any *other* operation. The 'difference' here is, with unlimited precision decimal representations, there is no "input phase". The decimal number can represent the value, sign, and exponent in the character string the user provided _exactly_, and indeed it could be implemented using strings as the internal representation -- in which case the 'construction' of a new number is simply a string copy operation. There is no operation taking place as there is no narrowing necessary. This is quite unlike (for example) converting an ASCII string "1.01" to a binary floating-point double which has a fixed precision and no base-5 component. mfc From eric.nieuwland at xs4all.nl Wed Jun 1 22:00:20 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Wed, 1 Jun 2005 22:00:20 +0200 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> References: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 08:46 PM 6/1/2005 +0200, Eric Nieuwland wrote: >> If I look at the translation it an augmented 'try'. >> with EXPR as VAR: >> BLOCK1 >> except EXCEPTION: >> BLOCK2 >> could then be translated to > > -1, too confusing. A matter of taste, I guess. IMHO 'with' secretly handling exceptions is confusing. >> Can the 'throw()' method be renamed 'raise()'? IMHO that makes much >> clearer what happens. > > No, 'raise' is a reserved word. It would have to be 'raise_()'. -0. My bad. Should have thought about that. >> Same thing with 'GeneratorExit', 'StopGeneration' more closely matches >> 'StopIteration'. > > StopIteration is raised the *other* way, so closely matching isn't > really a benefit. -1. Yep! Misread that one. --eric From pje at telecommunity.com Wed Jun 1 22:26:44 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 01 Jun 2005 16:26:44 -0400 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> At 10:00 PM 6/1/2005 +0200, Eric Nieuwland wrote: >Phillip J. Eby wrote: > > At 08:46 PM 6/1/2005 +0200, Eric Nieuwland wrote: > >> If I look at the translation it an augmented 'try'. > >> with EXPR as VAR: > >> BLOCK1 > >> except EXCEPTION: > >> BLOCK2 > >> could then be translated to > > > > -1, too confusing. > >A matter of taste, I guess. IMHO 'with' secretly handling exceptions is >confusing. It doesn't secretly handle them; it simply gets access to them, which is an entirely different thing. By confusing, I mean that it is not clear from your construct what exceptions are caught by the 'except' clause, due to its structural layout. It's also not clear whether the __enter__/__exit__ of EXPR wrap BLOCK1 only, or both BLOCK1 and BLOCK2. These aspects are "confusing" because whatever decision you make about the semantics, someone will have to *remember* them, as opposed to being unambiguously represented by the block structure. By contrast, if you remove the except: clause from your construct, it is clear that BLOCK1 is what is wrapped, and there is no possible confusion about who sees what exceptions. Exceptions inside the block are communicated to __exit__, exceptions outside (including those in the 'with' statement itself) are not. From kbk at shore.net Thu Jun 2 06:40:11 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu, 2 Jun 2005 00:40:11 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200506020440.j524eB3m025074@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 344 open ( +2) / 2845 closed ( +6) / 3189 total ( +8) Bugs : 916 open (-20) / 5014 closed (+40) / 5930 total (+20) RFE : 191 open ( +2) / 163 closed ( +4) / 354 total ( +6) New / Reopened Patches ______________________ Optimization for textwrap (2005-05-26) http://python.org/sf/1209527 opened by Connelly Build Python2.4.1 on AIX5 using xlc v6 (2005-05-27) http://python.org/sf/1209781 opened by Gangadhar NPK Split email headers near a space (2005-05-29) http://python.org/sf/1210680 opened by Noam Raphael Add st_flags support to (l)stat function (2005-05-31) http://python.org/sf/1212117 opened by Diego Petten? mode argument for fileinput class (2005-05-31) http://python.org/sf/1212287 opened by Reinhold Birkenfeld Improved profiler (2005-06-01) http://python.org/sf/1212837 opened by Brett Rosen option to allow reload to affect existing instances (2005-06-01) http://python.org/sf/1212921 opened by Brett Rosen new patch for fixing skipitem() in getargs.c (2005-06-01) http://python.org/sf/1212928 opened by Reinhold Birkenfeld note that os.chown can have -1 as an argument (2005-06-01) http://python.org/sf/1213031 opened by Reinhold Birkenfeld Patches Closed ______________ updates for the compiler package (2005-05-21) http://python.org/sf/1206077 closed by sxanth make float packing copy bytes when they can (2005-04-12) http://python.org/sf/1181301 closed by mwh webbrowser.Netscape.open bug fix (2005-02-20) http://python.org/sf/1144816 closed by birkenfeld cgitb: make more usable for 'binary-only' software (2003-06-10) http://python.org/sf/751943 closed by birkenfeld bug skipping optional keyword arguments of type "w#" (2004-07-06) http://python.org/sf/985713 closed by birkenfeld Optional keyword unicode args not handled correctly (2003-12-04) http://python.org/sf/853890 closed by birkenfeld New / Reopened Bugs ___________________ urllib2's urlopen() method causes a memory leak (2005-05-25) http://python.org/sf/1208304 opened by Petr Toman no CoreGraphics library under Python 2.4 (2005-05-25) CLOSED http://python.org/sf/1208468 opened by Jurjen N.E. Bos longs should be pickled in hexadecimal (2005-05-26) CLOSED http://python.org/sf/1209324 opened by Fredrik Johansson divmod documentation shd reference // not / (2005-05-26) CLOSED http://python.org/sf/1209411 opened by Alan os.path.join() fails if 2nd arg is a UNC path (2005-05-26) http://python.org/sf/1209447 opened by John Ehresman spurious blank page in dist.pdf (2005-05-27) http://python.org/sf/1209560 opened by paul rubin dict.popitem documentation should mention empty dict case (2005-05-27) CLOSED http://python.org/sf/1209671 opened by William Chang doc bug in Lock.acquire (2005-05-27) http://python.org/sf/1209880 opened by Chris Perkins Typo in "Differences from mimelib" (2005-05-27) http://python.org/sf/1210001 opened by Zumi comma separated cookie values (2005-05-28) http://python.org/sf/1210326 opened by tvogt Cursors not correctly closed after exception. (2005-05-28) http://python.org/sf/1210377 opened by Ragnar Ouchterlony An error in Python Tutorial (2005-05-29) CLOSED http://python.org/sf/1210832 opened by Gene mmap's resize method resizes the file in win32 but not unix (2003-04-27) CLOSED http://python.org/sf/728515 reopened by facundobatista parser tells invalid syntax with correct code (2005-05-31) CLOSED http://python.org/sf/1211639 opened by ntrunk itertools.groupby ungraceful, un-Pythonic (2005-05-31) CLOSED http://python.org/sf/1212077 opened by Mike Coleman str.lower() to have an IMPORTANT NOTE or it's for magicians (2005-05-31) http://python.org/sf/1212195 opened by Nikos Kouremenos anydbm and 'n' flag (2005-05-31) CLOSED http://python.org/sf/1212223 opened by Jack Moffitt Incorrect result for regular expression - "|(hello)|(world)" (2005-06-01) http://python.org/sf/1212411 opened by Vijay Kumar Python 2.5 CVS broken for HP-UX platform? (2005-06-01) http://python.org/sf/1212703 opened by Vincent Jamart Python segfaults on OpenBSD (tested 3.4 and 3.5) (2005-06-01) http://python.org/sf/1212900 opened by Fabien Devaux Bugs Closed ___________ bug in unichr() documentation (2005-02-11) http://python.org/sf/1120777 closed by fdrake Line ending documentation is misleading (2005-03-21) http://python.org/sf/1167922 closed by fdrake no CoreGraphics library under Python 2.4 (2005-05-25) http://python.org/sf/1208468 closed by mwh Issue in grammar (2005-05-24) http://python.org/sf/1207501 closed by mwh Problem with abs function (2005-05-16) http://python.org/sf/1202946 closed by rhettinger longs should be pickled in hexadecimal (2005-05-26) http://python.org/sf/1209324 closed by rhettinger divmod documentation shd reference // not / (2005-05-26) http://python.org/sf/1209411 closed by rhettinger dict.popitem documentation should mention empty dict case (2005-05-27) http://python.org/sf/1209671 closed by rhettinger An error in Python Tutorial (2005-05-29) http://python.org/sf/1210832 closed by birkenfeld Windows os.path.isdir bad if drive only (2002-02-11) http://python.org/sf/516232 closed by facundobatista test_signal hangs -- signal broken on OpenBSD? (2002-04-26) http://python.org/sf/549081 closed by facundobatista popen4 doesn't close filedescriptors when in Threads (2003-07-09) http://python.org/sf/768649 closed by facundobatista Sudden death with SIGSEGV in RtlEnterCriticalSection (2003-06-30) http://python.org/sf/763190 closed by facundobatista can't CNTRL-C when running os.system in a thread (2003-06-18) http://python.org/sf/756940 closed by facundobatista socketmodule.c: inet_pton() expects 4-byte packed_addr (2003-04-30) http://python.org/sf/730222 closed by facundobatista mmap's resize method resizes the file in win32 but not unix (2003-04-27) http://python.org/sf/728515 closed by facundobatista Core Dumps : Python2.2.2 (2003-04-24) http://python.org/sf/727241 closed by facundobatista "build_ext" "libraries" subcommand not s (2003-04-07) http://python.org/sf/716634 closed by facundobatista Thread running (os.system or popen#) (2003-03-11) http://python.org/sf/701836 closed by facundobatista Canvas origin is off-canvas in create_<item>(). Worka (2003-03-10) http://python.org/sf/700650 closed by facundobatista Canvas Widget origin is off-screen (2003-03-08) http://python.org/sf/699816 closed by facundobatista Profilier hooked into SystemExit (2003-02-15) http://python.org/sf/687297 closed by facundobatista String formatting operation Unicode problem. (2003-01-28) http://python.org/sf/676346 closed by facundobatista os.popen+() can take string list and bypass shell. (2003-01-12) http://python.org/sf/666700 closed by facundobatista doctest and exception messages (2002-12-16) http://python.org/sf/654783 closed by facundobatista Misuse of /usr/local/in setup.py (2002-11-19) http://python.org/sf/640553 closed by facundobatista parser tells invalid syntax with correct code (2005-05-31) http://python.org/sf/1211639 closed by doerwalter urllib has spurious print statement (2005-05-20) http://python.org/sf/1205544 closed by birkenfeld Description of string.lstrip() needs improvement (2005-05-15) http://python.org/sf/1202395 closed by rhettinger FutureWarning: %u/%o/%x/%X of negative int will return a sig (2003-06-19) http://python.org/sf/757365 closed by birkenfeld digraphs on komment lines / xlib (2003-07-16) http://python.org/sf/772176 closed by birkenfeld itertools.groupby ungraceful, un-Pythonic (2005-05-31) http://python.org/sf/1212077 closed by rhettinger anydbm and 'n' flag (2005-05-31) http://python.org/sf/1212223 closed by jafo memory leak in pickle/cPickle [2.2.3] (2003-07-14) http://python.org/sf/770997 closed by birkenfeld file.read returns reaches EOF when it shouldn't (2004-01-11) http://python.org/sf/875157 closed by birkenfeld Minor bug in urllib docs (2005-05-03) http://python.org/sf/1194249 closed by akuchling asyncore.loop() documentation (2005-04-12) http://python.org/sf/1181939 closed by akuchling Typo in Curses-Function doc (2005-02-15) http://python.org/sf/1123268 closed by akuchling SMTPHandler argument misdescribed (2005-02-13) http://python.org/sf/1121875 closed by akuchling incorrect constant names in curses window objects page (2005-01-19) http://python.org/sf/1105706 closed by akuchling curses.textpad raises error (2005-02-27) http://python.org/sf/1152762 closed by akuchling New / Reopened RFE __________________ expat binding for XML_ParserReset (2005-05-25) http://python.org/sf/1208730 opened by John Eikenberry add single html files (2005-05-27) http://python.org/sf/1209562 opened by paul rubin calling it revert is fine with me (2005-05-27) CLOSED http://python.org/sf/1209664 opened by paul rubin "break" and "continue"-ing out of nested 'for' loops (2005-05-29) CLOSED http://python.org/sf/1210975 opened by Geraint Luff sets needs an 'arbitrary element' method (2005-05-31) http://python.org/sf/1212091 opened by Mike Coleman Prepending Text (2005-05-31) http://python.org/sf/1212169 opened by Aaron Elbaz RFE Closed __________ Clipboard Cleared on Close (2005-05-24) http://python.org/sf/1207592 closed by kbk Bottom Scroll Bar (2005-05-24) http://python.org/sf/1207613 closed by kbk calling it revert is fine with me (2005-05-27) http://python.org/sf/1209664 closed by rhettinger "break" and "continue"-ing out of nested 'for' loops (2005-05-29) http://python.org/sf/1210975 closed by rhettinger From facundobatista at gmail.com Thu Jun 2 06:41:51 2005 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 2 Jun 2005 01:41:51 -0300 Subject: [Python-Dev] Closing old bugs In-Reply-To: <2md5r6wjkj.fsf@starship.python.net> References: <000001c565d7$65bebc20$5807a044@oemcomputer> <2md5r6wjkj.fsf@starship.python.net> Message-ID: On 6/1/05, Michael Hudson wrote: > > Old age and a missing OP is not sufficient reason to close a bug. > > But if closing a bug is an effective way of kicking things into life > again... I'm seeing this effect in a lot of bugs I closed as old ones. I think that using the mail-OP-and-commenters property of the SF bug tracking is a good thing here, and that that mail is enough alert to the interested people for them to reopen the bug if not closed correctly. Take note that for closing it, first there's a warning, and if in a *month* (which really happens to delay into several months, my fault) the interested people don't take care again of that bug... However, I'm not opposing myself to a change in our behaviour about this old bugs. Let's just define a new procedure (if we want to) and then I'll follow it. Thanks. . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From raymond.hettinger at verizon.net Thu Jun 2 08:05:13 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 02 Jun 2005 02:05:13 -0400 Subject: [Python-Dev] Closing old bugs Message-ID: <000001c56739$0afee180$28b69d8d@oemcomputer> > > Old age and a missing OP is not sufficient reason to close a bug. > > > > But if closing a bug is an effective way of kicking things into life > > again... > > I'm seeing this effect in a lot of bugs I closed as old ones. That means they shouldn't have been closed and that we almost lost a valid report. Also, for the most part, "kicking to life" means getting a qualified reviewer to take time to decide an appropriate course of action. Typically, the OP is not that person. Usually, the only "kick to life" we need from an OP is clarification if their post was not sufficiently specific; otherwise, they usually shouldn't have to do anything. > Take note that for closing it, first there's a warning, and if in a > *month* (which really happens to delay into several months, my fault) > the interested people don't take care again of that bug... A better use of time is to BE one of the interested people and take care of the bug. Just closing it doesn't make the problem go away. Also, inactivity does not imply that a bug is not a recurring irritant. We encourage posters to scan existing bug reports before filing a new one. Likewise, we immediately close duplicates. If the original report then disappears without having been cleared, then we've broken our promises to the othesr who did or would have posted a more current report. The existence of an old report means the problem has been registered and is awaiting a thoughtful response. Because of the way SF is setup, the other interested people are not likely to receive your "one month warnings". Closing reports without analyzing their contents is not progress. AFAICT, that has never been our policy. Is there anyone else on python-dev who thinks it's a bad idea to automatically close reports without checking whether the issue is real? Raymond From gmccaughan at synaptics-uk.com Thu Jun 2 13:51:54 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu, 2 Jun 2005 12:51:54 +0100 Subject: [Python-Dev] AST manipulation and source code generation In-Reply-To: References: Message-ID: <200506021251.54661.gmccaughan@synaptics-uk.com> > ?!ng: Well, i should refine that a bit to say that the Lisp macro system > is a little more specific. Whereas AST transformations in Python > are open-ended (you could generate any result you want), the key > interesting property of Lisp macros is that they are constrained > to be "safe", in the sense that the bindings of variable names are > always preserved. I'm not quite sure what ?!ng means by "the bindings of variable names are always preserved", but I conjecture that he is thinking of the "hygienic macros" in Scheme rather than the macros in Common Lisp which permit arbitrary code transformations. -- g From mcherm at mcherm.com Thu Jun 2 14:28:36 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Thu, 2 Jun 2005 05:28:36 -0700 Subject: [Python-Dev] Closing old bugs Message-ID: <20050602052836.f2cx4lj85ps04wkw@login.werra.lunarpages.com> Raymond writes: > Is there anyone else on python-dev who thinks it's a bad idea to automatically > close reports without checking whether the issue is real? Raymond: I'm speaking up with some trepidation here, since I am NOT one of those who frequently closes bugs. But I think that at least sometimes there IS an advantage to closing old bugs. I can envision a world in which there would be only 5-10 open bug reports at any given time, with enough developer volunteers available to respond to new reports as they come in. And I realize that we are nowhere near such a world and we're not going to be getting there anytime soon. But it is still the case that MOST people are intimidated by the enormous stack of open bugs. Perhaps "intimidated" is the wrong word... what I want to convey is that most people have no idea what's important or where to start -- it's just too big a pile. Perhaps all of the people who would actually work to close Python bugs are perfectly happy with the existing system. Or perhaps we scare off some potential volunteers because they have no idea where to start. I've seen some systems that solve this problem by allowing users to "vote" for favorite bugs... then you can tell the "important" bugs because they are more likely to have lots of votes. As I see it, Facundo is using a variant of that system. He is asking whether there is *ONE PERSON* out there who cares enough about a bug to subscribe to it and then to respond to his inquiry. If there's not even one such person, then he's closing the bug (but if one such person comes along later, they can re-report it). Sure, we may be throwing away some useful information by closing a bug report without proper investigation. But we're not in a situation where we are short of information... we've got plenty of information (bug reports), what we lack is developer time. If throwing away information helps to better focus the developer time we have, then it may be a useful process! And someday when nirvana arrives and there are only 5-10 open bugs, we can send intrepid volunteers digging through the archives to examine bugs that got closed without proper investigation. I'm not holding my breath. -- Michael Chermside From amk at amk.ca Thu Jun 2 14:52:05 2005 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 2 Jun 2005 08:52:05 -0400 Subject: [Python-Dev] Vestigial code in threadmodule? Message-ID: <20050602125205.GA3501@rogue.amk.ca> Looking at bug #1209880, the following function from threadmodule.c is referenced. I think the args==NULL case, which can return None instead of a Boolean value, can never be reached because PyArg_ParseTuple() will fail if args==NULL. Before ripping the args==NULL code out, I wanted to be sure my analysis is correct; is there some subtlety here I'm missing that makes args==NULL possible? --amk static PyObject * lock_PyThread_acquire_lock(lockobject *self, PyObject *args) { int i = 1; if (!PyArg_ParseTuple(args, "|i:acquire", &i)) return NULL; Py_BEGIN_ALLOW_THREADS i = PyThread_acquire_lock(self->lock_lock, i); Py_END_ALLOW_THREADS if (args == NULL) { Py_INCREF(Py_None); return Py_None; } else return PyBool_FromLong((long)i); } From python at rcn.com Thu Jun 2 15:01:09 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu, 2 Jun 2005 09:01:09 -0400 Subject: [Python-Dev] Closing old bugs In-Reply-To: <20050602052836.f2cx4lj85ps04wkw@login.werra.lunarpages.com> Message-ID: <000101c56773$2663c5a0$3bb02c81@oemcomputer> > I've seen some systems that solve this problem by allowing users to "vote" > for favorite bugs... then you can tell the "important" bugs because they > are more likely to have lots of votes. As I see it, Facundo is using a > variant of that system. He is asking whether there is *ONE PERSON* out > there who cares enough about a bug to subscribe to it and then to respond > to his inquiry. If there's not even one such person, then he's closing > the bug (but if one such person comes along later, they can re-report it). -1 This is both silly and harmful. It in no way resembles a professional approach to bug resolution. It throws away valuable information based on some vague theory of developer marketing (i.e. threatening to close a bug will cause a qualified, interested developer to suddenly have both the time and inclination to address it properly). If the real goal is to "kick some life" into bug resolution, then do something that directly fulfills that goal. Host a bug day. Make a newsgroup posting requesting thoughts on your favorite ten bugs. Write email to people who you think are capable of addressing the particular issue in question. Go recruit some qualified developers. Or just find a bug that interests you and fix it. Closing bugs without reading them is an anti-contribution. If it were a best practice, then there would already be a cron job in place to do it automatically. Raymond From reinhold-birkenfeld-nospam at wolke7.net Thu Jun 2 15:14:36 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Thu, 02 Jun 2005 15:14:36 +0200 Subject: [Python-Dev] Closing old bugs In-Reply-To: <000101c56773$2663c5a0$3bb02c81@oemcomputer> References: <20050602052836.f2cx4lj85ps04wkw@login.werra.lunarpages.com> <000101c56773$2663c5a0$3bb02c81@oemcomputer> Message-ID: Raymond Hettinger wrote: >> I've seen some systems that solve this problem by allowing users to > "vote" >> for favorite bugs... then you can tell the "important" bugs because > they >> are more likely to have lots of votes. As I see it, Facundo is using a >> variant of that system. He is asking whether there is *ONE PERSON* out >> there who cares enough about a bug to subscribe to it and then to > respond >> to his inquiry. If there's not even one such person, then he's > closing >> the bug (but if one such person comes along later, they can re-report > it). > > -1 This is both silly and harmful. It in no way resembles a > professional approach to bug resolution. It throws away valuable > information based on some vague theory of developer marketing (i.e. > threatening to close a bug will cause a qualified, interested developer > to suddenly have both the time and inclination to address it properly). ACK so far. > If the real goal is to "kick some life" into bug resolution, then do > something that directly fulfills that goal. Host a bug day. Make a > newsgroup posting requesting thoughts on your favorite ten bugs. Write > email to people who you think are capable of addressing the particular > issue in question. Go recruit some qualified developers. Or just find > a bug that interests you and fix it. Well, to "fix it" is not so easy for most people. You have to post a patch or attach it to the bug and then wait for someone to check it in. It's not sure that this is done anytime soon (no complaint; time is short, I know). Even fixes that are agreed upon by several people are not always checked in for a long time. Reinhold -- Mail address is perfectly valid! From nnorwitz at gmail.com Thu Jun 2 15:19:36 2005 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 2 Jun 2005 09:19:36 -0400 Subject: [Python-Dev] Vestigial code in threadmodule? In-Reply-To: <20050602125205.GA3501@rogue.amk.ca> References: <20050602125205.GA3501@rogue.amk.ca> Message-ID: On 6/2/05, A.M. Kuchling wrote: > Looking at bug #1209880, the following function from threadmodule.c is > referenced. I think the args==NULL case, which can return None > instead of a Boolean value, can never be reached because > PyArg_ParseTuple() will fail if args==NULL. > > Before ripping the args==NULL code out, I wanted to be sure my > analysis is correct; is there some subtlety here I'm missing that > makes args==NULL possible? I think the args == NULL code should be ripped out, but there seems to be a different problem. If args is NULL to PyArg_ParseTuple() an assertion will be triggered (or it'll just crash). See vgetargs1() in Python/getargs.c::138. args can be NULL if load_args() in Python/ceval.c fails (line 3724). The trace starts at line 3551, PyCFunction_Call() will be called with the NULL args on line 3553. PyCFunction_Call() will call a PyMethod that will likely call PyArg_ParseTuple() or something like it. I think the following patch should fix this (it just adds an if condition). Does this make sense or am I missing something? n -- Index: Python/ceval.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v retrieving revision 2.422 diff -u -r2.422 ceval.c --- Python/ceval.c 4 Apr 2005 15:49:02 -0000 2.422 +++ Python/ceval.c 2 Jun 2005 13:16:14 -0000 @@ -3549,9 +3549,13 @@ else { PyObject *callargs; callargs = load_args(pp_stack, na); - READ_TIMESTAMP(*pintr0); - C_TRACE(x=PyCFunction_Call(func,callargs,NULL)); - READ_TIMESTAMP(*pintr1); + if (callargs) { + READ_TIMESTAMP(*pintr0); + C_TRACE(x=PyCFunction_Call(func,callargs,NULL));+ READ_TIMESTAMP(*pintr1); + } + else + x = NULL; Py_XDECREF(callargs); } } else { From adv at langdale.com.au Thu Jun 2 15:47:57 2005 From: adv at langdale.com.au (Arnold deVos) Date: Thu, 02 Jun 2005 23:47:57 +1000 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> Message-ID: <429F0E0D.2020709@langdale.com.au> Guido van Rossum wrote: > [Phillip J. Eby] >>* The transaction handler could also be written as: >> >> @with_template >> def transactional(db): >> db.begin() >> try: >> yield db >> except: >> db.rollback() >> else: >> db.commit() >> >>at least, if I understand it correctly. > > > Ah, of course. I've updated the PEP. > This template eats eats the exception, which will cause a RuntimeError in the proposed Wrapper, I think. A raise after rollback is needed. - Arnold From adv at langdale.com.au Thu Jun 2 15:47:57 2005 From: adv at langdale.com.au (Arnold deVos) Date: Thu, 02 Jun 2005 23:47:57 +1000 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> Message-ID: <429F0E0D.2020709@langdale.com.au> Guido van Rossum wrote: > [Phillip J. Eby] >>* The transaction handler could also be written as: >> >> @with_template >> def transactional(db): >> db.begin() >> try: >> yield db >> except: >> db.rollback() >> else: >> db.commit() >> >>at least, if I understand it correctly. > > > Ah, of course. I've updated the PEP. > This template eats eats the exception, which will cause a RuntimeError in the proposed Wrapper, I think. A raise after rollback is needed. - Arnold From anothermax at gmail.com Thu Jun 2 15:57:14 2005 From: anothermax at gmail.com (Jeremy Maxfield) Date: Thu, 2 Jun 2005 15:57:14 +0200 Subject: [Python-Dev] Vestigial code in threadmodule? In-Reply-To: <20050602125205.GA3501@rogue.amk.ca> References: <20050602125205.GA3501@rogue.amk.ca> Message-ID: <93dc9c320506020657580ca956@mail.gmail.com> If you're digging into the threadmodule.c could you take a look at bug #1163563 (http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&group_id=5470&atid=105470) I've posted a patch (http://sourceforge.net/tracker/index.php? func=detail&aid=1203393&group_id=5470&atid=305470) Regards, Max On 6/2/05, A.M. Kuchling wrote: > Looking at bug #1209880, the following function from threadmodule.c is > referenced. I think the args==NULL case, which can return None > instead of a Boolean value, can never be reached because > PyArg_ParseTuple() will fail if args==NULL. > > Before ripping the args==NULL code out, I wanted to be sure my > analysis is correct; is there some subtlety here I'm missing that > makes args==NULL possible? > > --amk > > static PyObject * > lock_PyThread_acquire_lock(lockobject *self, PyObject *args) > { > int i = 1; > > if (!PyArg_ParseTuple(args, "|i:acquire", &i)) > return NULL; > > Py_BEGIN_ALLOW_THREADS > i = PyThread_acquire_lock(self->lock_lock, i); > Py_END_ALLOW_THREADS > > if (args == NULL) { > Py_INCREF(Py_None); > return Py_None; > } > else > return PyBool_FromLong((long)i); > } > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com > -- flickr: http://www.flickr.com/photos/anothermax/sets From tim.peters at gmail.com Thu Jun 2 16:12:16 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 2 Jun 2005 10:12:16 -0400 Subject: [Python-Dev] Vestigial code in threadmodule? In-Reply-To: <20050602125205.GA3501@rogue.amk.ca> References: <20050602125205.GA3501@rogue.amk.ca> Message-ID: <1f7befae0506020712132e4067@mail.gmail.com> [A.M. Kuchling] > Looking at bug #1209880, the following function from threadmodule.c is > referenced. I think the args==NULL case, which can return None > instead of a Boolean value, can never be reached because > PyArg_ParseTuple() will fail if args==NULL. It would assert-fail in a debug build. In a release build the most likely outcome would be a segfault (NULL-pointer dereference in the expansion of vgetargs1's "PyTuple_Check(args)"). > Before ripping the args==NULL code out, I wanted to be sure my > analysis is correct; is there some subtlety here I'm missing that > makes args==NULL possible? Rip it out; blame me . > --amk > > static PyObject * > lock_PyThread_acquire_lock(lockobject *self, PyObject *args) Noe that this is a file-local function. The only references are here: static PyMethodDef lock_methods[] = { {"acquire_lock", (PyCFunction)lock_PyThread_acquire_lock, METH_VARARGS, acquire_doc}, {"acquire", (PyCFunction)lock_PyThread_acquire_lock, METH_VARARGS, acquire_doc}, METH_VARARGS always passes a tuple (possibly empty). These are very old functions, so I bet they used to use METH_OLDARGS (implied by absence at the time) ... yup, METH_VARARGS was introduced here in rev 2.48, and unintentionally changed the return contract of this function. So that was a backward incompatibility introduced in Python 2.3a1. Since nobody complained then or since, I vote to keep "the new" return contract and fiddle the docs to match it. From adv at langdale.com.au Thu Jun 2 16:14:49 2005 From: adv at langdale.com.au (Arnold deVos) Date: Fri, 03 Jun 2005 00:14:49 +1000 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <429F0E0D.2020709@langdale.com.au> References: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> <429F0E0D.2020709@langdale.com.au> Message-ID: Arnold deVos wrote: > > This template eats eats the exception, which will cause a RuntimeError > in the proposed Wrapper, I think. A raise after rollback is needed. Actually, the Wrapper as written in the PEP does not raise RuntimeError if the generator catches a block's exception. Shouldn't the relevant clause in the Wrapper go like this: try: self.gen.throw(type, value, traceback) except type: return except StopIteration: raise RuntimeError("generator caught exception") else: raise RuntimeError("generator didn't stop") And the transaction template would go like this (re-raising the exception): @with_template def transactional(db): db.begin() try: yield None except: db.rollback() raise else: db.commit() At least this is what I gleaned from the earlier threads. It means that the template does not appear to supress an exception that it cannot actually supress. - Arnold From gvanrossum at gmail.com Thu Jun 2 16:38:48 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 2 Jun 2005 07:38:48 -0700 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> <429F0E0D.2020709@langdale.com.au> Message-ID: [Arnold deVos, responding to himself] > > This template eats eats the exception, which will cause a RuntimeError > > in the proposed Wrapper, I think. A raise after rollback is needed. No, the generator returns after rolling back, which causes throw() to raise StopIteration, which is good enough for the wrapper as written. > Actually, the Wrapper as written in the PEP does not raise RuntimeError > if the generator catches a block's exception. > > Shouldn't the relevant clause in the Wrapper go like this: > > try: > self.gen.throw(type, value, traceback) > except type: > return > except StopIteration: > raise RuntimeError("generator caught exception") > else: > raise RuntimeError("generator didn't stop") I considered that, but decided that it should be okay for the generator to respond to a throw() by returning (thus replacing the exception thrown by StopIteration) since the call to __exit__() is contained inside a finally-clause, so that when __exit__() returns normally, the finally-clause will re-raise the original exception anyway. Note that there are currently no other use cases for throw() except the with_template decorator and the close() method. Both allow the generator to respond either by letting the exception pass through it unchanged (after executing finally-clauses if present) or by simply returning (which will raise StopIteration in the caller of throw()). Erroneous behaviors are, in both cases, raising some other exception or *yielding* another value. There may be *other* use cases for yielding a value, for example, a future (looping) block-statement a la PEP 343. Raising another exception is always an indication of a bug in the generator. > And the transaction template would go like this (re-raising the exception): > > @with_template > def transactional(db): > db.begin() > try: > yield None > except: > db.rollback() > raise > else: > db.commit() > > At least this is what I gleaned from the earlier threads. It means that > the template does not appear to supress an exception that it cannot > actually supress. I disagree (at the -0 to -0.5 level); given that the with_template decorator allows StopIteration, I think that "appearing to suppress an exception it doesn't actually suppress" is a minor sin -- the key point is that the cleanup gets executed and doesn't raise a new exception or reaches a yield-statement. I'll summarize this discussion in the PEP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Thu Jun 2 16:58:46 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 2 Jun 2005 07:58:46 -0700 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> <429F0E0D.2020709@langdale.com.au> Message-ID: [me] > I'll summarize this discussion in the PEP. I've added this section to the PEP. Is anyone dead set against the tentative resolutions here? Open Issues Discussion on python-dev has revealed some open issues. I list them here, with my preferred resolution and its motivation. The PEP as currently written reflects this preferred resolution. 1. What exception should be raised by close() when the generator yields another value as a response to the GeneratorExit exception? I originally chose TypeError because it represents gross misbehavior of the generator function, which should be fixed by changing the code. But the with_template decorator class uses RuntimeError for similar offenses. Arguably they should all use the same exception. I'd rather not introduce a new exception class just for this purpose, since it's not an exception that I want people to catch: I want it to turn into a traceback which is seen by the programmer who then fixes the code. So now I believe they should both raise RuntimeError. There are some precedents for that: it's raised by the core Python code in situations where endless recursion is detected, and for uninitialized objects (and for a variety of miscellaneous conditions). 2. Both the generator close() method and the __exit__() method of the with_template decorator class catch StopIteration and consider it equivalent to re-raising the exception passed to throw(). Is allowing StopIteration right here? This is so that a generator doing cleanup depending on the exception thrown (like the transactional() example below) can *catch* the exception thrown if it wants to and doesn't have to worry about re-raising it. I find this more convenient for the generator writer. Against this was brought in that the generator *appears* to suppress an exception that it cannot suppress: the transactional() example would be more clear according to this view if it re-raised the original exception after the call to db.rollback(). I personally would find the requirement to re-raise the exception an annoyance in a generator used as a with-template, since all the code after yield is used for is cleanup, and it is invoked from a finally-clause (the one implicit in the with-statement) which re-raises the original exception anyway. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Jun 2 17:08:27 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 03 Jun 2005 01:08:27 +1000 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: Message-ID: <429F20EB.8070800@gmail.com> Guido van Rossum wrote: > I hope that I've got the rewrite of PEP 343 to include generator > extensions right now. I've chosen the 'with' keyword. Please review > here; I think this is ready for review by the unwashed masses. :-) > > http://www.python.org/peps/pep-0343.html > Looks pretty good to me (looking at version 1.19). One comment is that, in the 'optional extensions' section, where you say 'such mistakes are easily diagnosed', you could point to your generator wrapper as an example where attempting to reuse the block handler raises a RuntimeError on the second attempt. Also, I'm wondering if it would be useful to have a 'closing' template that looked like: @with_template def closing(obj): try: yield obj finally: obj.close() That can be used to deterministically close anything with a close method, be it file, generator, or something else: with closing(open("argument.txt")) as contradiction: for line in contradiction: print line with closing(some_gen()) as data: for datum in data: process(datum) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From pje at telecommunity.com Thu Jun 2 17:51:26 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 02 Jun 2005 11:51:26 -0400 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <429F20EB.8070800@gmail.com> References: Message-ID: <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com> At 01:08 AM 6/3/2005 +1000, Nick Coghlan wrote: >Also, I'm wondering if it would be useful to have a 'closing' template >that looked like: > > @with_template > def closing(obj): > try: > yield obj > finally: > obj.close() +1 if you make it 'if hasattr(obj,"close"): obj.close()' in the finally clause, so that it will still work if the type of the object changes. From gvanrossum at gmail.com Thu Jun 2 18:20:14 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 2 Jun 2005 09:20:14 -0700 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com> References: <429F20EB.8070800@gmail.com> <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com> Message-ID: On 6/2/05, Phillip J. Eby wrote: > At 01:08 AM 6/3/2005 +1000, Nick Coghlan wrote: > >Also, I'm wondering if it would be useful to have a 'closing' template > >that looked like: > > > > @with_template > > def closing(obj): > > try: > > yield obj > > finally: > > obj.close() > > +1 if you make it 'if hasattr(obj,"close"): obj.close()' in the finally > clause, so that it will still work if the type of the object changes. But then you'd get back the bug where it would silently do nothing when you pass it the wrong thing; wasn't that argument used against an earlier proposal to skip calling __exit__() when it doesn't exist? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu Jun 2 18:58:43 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 02 Jun 2005 12:58:43 -0400 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com> <429F20EB.8070800@gmail.com> <5.1.1.6.0.20050602114932.021f9ea8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050602122814.02202cc0@mail.telecommunity.com> At 09:20 AM 6/2/2005 -0700, Guido van Rossum wrote: >On 6/2/05, Phillip J. Eby wrote: > > At 01:08 AM 6/3/2005 +1000, Nick Coghlan wrote: > > >Also, I'm wondering if it would be useful to have a 'closing' template > > >that looked like: > > > > > > @with_template > > > def closing(obj): > > > try: > > > yield obj > > > finally: > > > obj.close() > > > > +1 if you make it 'if hasattr(obj,"close"): obj.close()' in the finally > > clause, so that it will still work if the type of the object changes. > >But then you'd get back the bug where it would silently do nothing >when you pass it the wrong thing; That's not a bug, it's a feature. If the object doesn't have a 'close()' method, clearly it doesn't need to be closed. If it's the "wrong" object for what you're using it for in the body of the 'with' block, it'll show up there, so this doesn't hide any errors. The idea is that "with closing(foo):" is just an assertion that you only intend to use 'foo' in the body of that block, and not afterwards, so if 'foo' is something that needs closing, go ahead and close it. The specific use case I have in mind is situations where a piece of code expects an iterable, but there are callers (or callees) that need to do cleanup. On the other hand some callers (or callees) would like to just pass in or return a list, or are passing or returning an object from somewhere else that may or may not have a close() method. Thus, a whole bunch of code would suddenly be strewed with assumptions about whether things can be closed or not, which seems like pure administrivia. The important functionality is what the object does *before* you close it; that's what the programmer should be able to remain focused on. >wasn't that argument used against an >earlier proposal to skip calling __exit__() when it doesn't exist? I thought the resolution of that discussion was that you should use an explicit wrapper if you want this behavior -- which is what 'closing()' is. Anyway, if you don't like it, don't put it in. I just thought it would be a good example to include for a wrapper whose behavior is more optional in nature. From python at rcn.com Thu Jun 2 21:34:17 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu, 2 Jun 2005 15:34:17 -0400 Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib sre_compile.py, 1.57, 1.58 In-Reply-To: Message-ID: <002501c567aa$11428300$3bb02c81@oemcomputer> > Index: sre_compile.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/sre_compile.py,v > retrieving revision 1.57 > retrieving revision 1.58 > diff -u -d -r1.57 -r1.58 > --- sre_compile.py 28 Feb 2005 19:27:52 -0000 1.57 > +++ sre_compile.py 2 Jun 2005 13:35:52 -0000 1.58 > @@ -167,7 +167,7 @@ > emit(av-1) > elif op is GROUPREF_EXISTS: > emit(OPCODES[op]) > - emit((av[0]-1)*2) > + emit(av[0]-1) > skipyes = _len(code); emit(0) > _compile(code, av[1], flags) > if av[2]: The times two operation also occurs twice in nearby code. Are those also incorrect? Raymond Hettinger From amk at amk.ca Thu Jun 2 21:51:40 2005 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 2 Jun 2005 15:51:40 -0400 Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib sre_compile.py, 1.57, 1.58 In-Reply-To: <002501c567aa$11428300$3bb02c81@oemcomputer> References: <002501c567aa$11428300$3bb02c81@oemcomputer> Message-ID: <20050602195140.GA5899@rogue.amk.ca> On Thu, Jun 02, 2005 at 03:34:17PM -0400, Raymond Hettinger wrote: > The times two operation also occurs twice in nearby code. Are those > also incorrect? I believe they're correct. EXPN: The regex engine refers to both 'groups', where group #N means the corresponding group in the pattern, and 'marks', which is a table of indexes into the string; group #0 -> mark 0 and 1, group #1 -> mark 2 and 3, etc. The compiler was storing the mark value, but then the code for the GROUPREF_EXISTS opcode was doubling it, converting group to mark: case SRE_OP_GROUPREF_EXISTS: TRACE(("|%p|%p|GROUPREF_EXISTS %d\n", ctx->pattern, ctx->ptr, ctx->pattern[0])); /* codeyes codeno ... */ i = ctx->pattern[0]; { int groupref = i+i; if (groupref >= state->lastmark) { ctx->pattern += ctx->pattern[1]; break; } else { ... } The two other uses of *2 are for the MARK opcode, which stores the current position in the string in the corresponding entry in the table; it really does want the mark number, so the doubling is correct. The 'groupref = i+i' in the above code is a bit odd -- why not write 2*i? I thought it might be a typo for i+1, but that wouldn't make sense either; eventually I decided this was just a stylistic thing. --am"Only the effbot knows for sure"k From eric.nieuwland at xs4all.nl Thu Jun 2 22:04:20 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Thu, 2 Jun 2005 22:04:20 +0200 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> References: <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> Message-ID: <9330e57c131b13f7d0327412fbf4926f@xs4all.nl> Phillip J. Eby wrote: > At 10:00 PM 6/1/2005 +0200, Eric Nieuwland wrote: >> Phillip J. Eby wrote: >> > -1, too confusing. >> >> A matter of taste, I guess. IMHO 'with' secretly handling exceptions >> is >> confusing. > > It doesn't secretly handle them; it simply gets access to them, which > is an entirely different thing. > > By confusing, I mean that it is not clear from your construct what > exceptions are caught by the 'except' clause, due to its structural > layout. It's also not clear whether the __enter__/__exit__ of EXPR > wrap BLOCK1 only, or both BLOCK1 and BLOCK2. These aspects are > "confusing" because whatever decision you make about the semantics, > someone will have to *remember* them, as opposed to being > unambiguously represented by the block structure. > > By contrast, if you remove the except: clause from your construct, it > is clear that BLOCK1 is what is wrapped, and there is no possible > confusion about who sees what exceptions. Exceptions inside the block > are communicated to __exit__, exceptions outside (including those in > the 'with' statement itself) are not. OK. This forwarding (is that the proper expression here?) of an exception to __exit__ is what I meant by 'secretly handling'. If everybody agrees I'll write with EXPR as VAR: try: BLOCK1 except EXCEPTION: BLOCK2 instead. Seems a waiste to me, though. I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles uncaught exceptions by forwarding it to EXPR's __exit__ method. No confusion with me. --eric From pje at telecommunity.com Thu Jun 2 22:12:03 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 02 Jun 2005 16:12:03 -0400 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <9330e57c131b13f7d0327412fbf4926f@xs4all.nl> References: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com> At 10:04 PM 6/2/2005 +0200, Eric Nieuwland wrote: >I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles uncaught >exceptions by forwarding it to EXPR's __exit__ method. No confusion with me. No doubt. However, it's not obvious what happens to an exception in EXPR; surely it can't be passed to EXPR's __exit__ method. So, is it handled by the try, or does it pass out of the block? Whichever answer you give, there is somebody who will think the opposite. And this is precisely the ambiguity I've been talking about. In contrast, a 'with' unmixed with 'try' is absolutely unambiguous as to which except: clauses handle what exceptions where. From eric.nieuwland at xs4all.nl Thu Jun 2 22:16:42 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Thu, 2 Jun 2005 22:16:42 +0200 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com> References: <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com> Message-ID: <989928a3906aa6b0258875486e16f4cf@xs4all.nl> On 2 jun 2005, at 22:12, Phillip J. Eby wrote: > At 10:04 PM 6/2/2005 +0200, Eric Nieuwland wrote: >> I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles >> uncaught exceptions by forwarding it to EXPR's __exit__ method. No >> confusion with me. > > No doubt. However, it's not obvious what happens to an exception in > EXPR; surely it can't be passed to EXPR's __exit__ method. So, is it > handled by the try, or does it pass out of the block? Whichever > answer you give, there is somebody who will think the opposite. And > this is precisely the ambiguity I've been talking about. > > In contrast, a 'with' unmixed with 'try' is absolutely unambiguous as > to which except: clauses handle what exceptions where. I never thought of that! Now I see what you mean. I could only save my idea by stating "the scope of 'try' only starts after the ':'", but that seems too artificial. --eric From pje at telecommunity.com Thu Jun 2 22:24:46 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 02 Jun 2005 16:24:46 -0400 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <989928a3906aa6b0258875486e16f4cf@xs4all.nl> References: <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com> <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601151709.02200378@mail.telecommunity.com> <5.1.1.6.0.20050601162046.02200e30@mail.telecommunity.com> <5.1.1.6.0.20050602160928.021fbf98@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050602162313.01fd0558@mail.telecommunity.com> At 10:16 PM 6/2/2005 +0200, Eric Nieuwland wrote: >On 2 jun 2005, at 22:12, Phillip J. Eby wrote: > > > At 10:04 PM 6/2/2005 +0200, Eric Nieuwland wrote: > >> I was thinking about 'try EXPR [as VAR]:' as a 'try' that handles > >> uncaught exceptions by forwarding it to EXPR's __exit__ method. No > >> confusion with me. > > > > No doubt. However, it's not obvious what happens to an exception in > > EXPR; surely it can't be passed to EXPR's __exit__ method. So, is it > > handled by the try, or does it pass out of the block? Whichever > > answer you give, there is somebody who will think the opposite. And > > this is precisely the ambiguity I've been talking about. > > > > In contrast, a 'with' unmixed with 'try' is absolutely unambiguous as > > to which except: clauses handle what exceptions where. > > I never thought of that! Now I see what you mean. >I could only save my idea by stating "the scope of 'try' only starts >after the ':'", but that seems too artificial. That's what I mean: if you have to solve it by decree, it becomes a rule that people have to *remember* (or at least read carefully and think about it), which goes against the "executable pseudocode" nature. From tdelaney at avaya.com Thu Jun 2 22:45:19 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Fri, 3 Jun 2005 06:45:19 +1000 Subject: [Python-Dev] PEP 343 rewrite complete Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE0252057E@au3010avexu1.global.avaya.com> Phillip J. Eby wrote: > That's not a bug, it's a feature. If the object doesn't have a > 'close()' > method, clearly it doesn't need to be closed. If it's the "wrong" > object > for what you're using it for in the body of the 'with' block, it'll > show up > there, so this doesn't hide any errors. For those semantics, I think one of the following would be better: with local(obj): with scoped(obj): but those semantics apply better to __enter__ and __exit__ anyway. I think a "closing adaptor" should only work with objects that have a close() method. Perhaps: with scoped_closable(obj): Tim Delaney From gvanrossum at gmail.com Thu Jun 2 22:58:05 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 2 Jun 2005 13:58:05 -0700 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <429F20EB.8070800@gmail.com> References: <429F20EB.8070800@gmail.com> Message-ID: [Nick Coghlan] > Also, I'm wondering if it would be useful to have a 'closing' template > that looked like: > > @with_template > def closing(obj): > try: > yield obj > finally: > obj.close() > > That can be used to deterministically close anything with a close > method, be it file, generator, or something else: > > with closing(open("argument.txt")) as contradiction: > for line in contradiction: > print line > > with closing(some_gen()) as data: > for datum in data: > process(datum) I just realized this has a race condition. The bytecode for the expression closing(open("...")) must necessarily contain a bytecode that calls open() followed by another bytecode that calls closing(). If a ^C happens between these two byte codes, the stack contains an open file object that won't be closed explicitly. With the original opening() template, this race can be avoided (and I intend to do so) by implementing opening() in C (as a class with __enter__ and __exit__ methods), and by making sure that the interpreter does *not* check for interrupts between the call to __enter__ and the start of the try-finally-statement in the translation of the with-statement. The only way to avoid the race that I can see with the closing() template would be to disable signals for the duration of the evaluation of the expression in the with-statement, but I really don't like that solution at all -- system calls like that can be excruciatingly expensive compared to bytecode execution. Most applications don't catch ^C so they don't need this extra protection -- but the compiler doesn't know that so everybody pays for it. The solution for avoiding interrupts between the __enter__() call and the try start doesn't require disabling signals; there can be a single opcode that calls __enter__ and sets up the try-finally context. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From reinhold-birkenfeld-nospam at wolke7.net Thu Jun 2 23:14:44 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Thu, 02 Jun 2005 23:14:44 +0200 Subject: [Python-Dev] sys.path in interactive session Message-ID: While looking at bug #779191, I saw that sys.path's first element is '' in interactive sessions, but the current dir otherwise. Is this intentional? Reinhold -- Mail address is perfectly valid! From gvanrossum at gmail.com Fri Jun 3 01:50:07 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 2 Jun 2005 16:50:07 -0700 Subject: [Python-Dev] sys.path in interactive session In-Reply-To: References: Message-ID: I've always liked it this way; using "" instead of "." means that if you os.path.join() it with a script name you don't get a spurious "./" prepended. I think that the "absolutizing" of sys.path entries is relatively new (seems to have started in 2.3). Also note that it's not really the current directory but the directory containing the script; that is definitely intentional. On 6/2/05, Reinhold Birkenfeld wrote: > While looking at bug #779191, I saw that sys.path's first element > is '' in interactive sessions, but the current dir otherwise. Is this > intentional? > > Reinhold > > -- > Mail address is perfectly valid! > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bob at redivi.com Fri Jun 3 02:08:54 2005 From: bob at redivi.com (Bob Ippolito) Date: Thu, 2 Jun 2005 17:08:54 -0700 Subject: [Python-Dev] sys.path in interactive session In-Reply-To: References: Message-ID: <5A9AFDC6-6CFD-46A6-9B2D-2C91A59FD00D@redivi.com> On Jun 2, 2005, at 4:50 PM, Guido van Rossum wrote: > On 6/2/05, Reinhold Birkenfeld nospam at wolke7.net> wrote: > >> While looking at bug #779191, I saw that sys.path's first element >> is '' in interactive sessions, but the current dir otherwise. Is this >> intentional? >> > > I've always liked it this way; using "" instead of "." means that if > you os.path.join() it with a script name you don't get a spurious "./" > prepended. > > I think that the "absolutizing" of sys.path entries is relatively new > (seems to have started in 2.3). > > Also note that it's not really the current directory but the directory > containing the script; that is definitely intentional. The absolutizing of sys.path in site.py is misbehavior anyway.. it spits in the face of path hooks, for example . I still haven't committed that patch, I haven't had a whole lot of Python-time lately with co-founding a new company, preparing to move, WWDC, etc... but nobody bothered to complain, so if someone wants to commit before I get a chance, feel free! -bob From pje at telecommunity.com Fri Jun 3 02:48:27 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 02 Jun 2005 20:48:27 -0400 Subject: [Python-Dev] sys.path in interactive session In-Reply-To: <5A9AFDC6-6CFD-46A6-9B2D-2C91A59FD00D@redivi.com> References: Message-ID: <5.1.1.6.0.20050602203346.021fe9e0@mail.telecommunity.com> At 05:08 PM 6/2/2005 -0700, Bob Ippolito wrote: >On Jun 2, 2005, at 4:50 PM, Guido van Rossum wrote: > > > > I think that the "absolutizing" of sys.path entries is relatively new > > (seems to have started in 2.3). > > > > Also note that it's not really the current directory but the directory > > containing the script; that is definitely intentional. > >The absolutizing of sys.path in site.py is misbehavior anyway.. Your patch doesn't fix it doing this to __file__; probably this should not touch __file__ if a module has a __loader__ set, or if the path it's absolutizing doesn't exist. Actually, your patch doesn't fix it doing this to all the sys.path entries, either. It appears that site.py may be quite broken with respect to PEP 302. Fixing it is going to be interesting, since it's quite possible that you can have stuff on PYTHONPATH for which the sys.path_hooks entries don't exist yet. Thus, you can't guarantee when you absolutize a path entry that it is in fact a filename, unless you check for its existence - which in principle could be a coincidence. It may be that we need to define more clearly what is allowed on sys.path, such that site.py can tell the difference between filesystem paths, and non-filesystem PEP 302 specifiers. Right now, the only major PEP 302 implementation that I know of is zipimport, but zipimport uses strings that are effectively filesystem paths, so site.py doesn't break them. From adv at langdale.com.au Fri Jun 3 02:49:49 2005 From: adv at langdale.com.au (Arnold deVos) Date: Fri, 03 Jun 2005 10:49:49 +1000 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <5.1.1.6.0.20050601120734.021f6bc8@mail.telecommunity.com> <429F0E0D.2020709@langdale.com.au> Message-ID: Guido van Rossum wrote: > > [...] a generator doing cleanup depending on the > exception thrown (like the transactional() example below) can > *catch* the exception thrown if it wants to and doesn't have to > worry about re-raising it. I find this more convenient for the > generator writer. Against this was brought in that the > generator *appears* to suppress an exception that it cannot > suppress: the transactional() example would be more clear > according to this view if it re-raised the original exception > after the call to db.rollback(). [...] Of course, the explicit re-raise is only needed in a minority of use cases where the exception is caught. Two additional points in favour of this: - refactoring a naked try as a with + template is more direct and uniform across all use cases. - it is upwards compatible if the prohibition on templates suppressing exceptions is ever reconsidered. (Flow control macro discussion not withstanding.) - Arnold From gvanrossum at gmail.com Fri Jun 3 05:09:02 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 2 Jun 2005 20:09:02 -0700 Subject: [Python-Dev] For review: PEP 343: Anonymous Block Redux and Generator Enhancements Message-ID: After many rounds of discussion on python-dev, I'm inviting public comments for PEP 343. Rather than posting the entire PEP text here, I'm inviting everyone to read it on line (http://www.python.org/peps/pep-0343.html) and then post comments on a Wiki page I've created for this purpose (http://wiki.python.org/moin/WithStatement). I think this is a good one; I hope people agree. Its acceptance will obsolete about 4 other PEPs! (A sign that it fulfills a need and that the proposed solution is powerful.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fcano at ono.com Sat Jun 4 19:49:47 2005 From: fcano at ono.com (Florencio Cano Gabarda) Date: Sat, 04 Jun 2005 19:49:47 +0200 Subject: [Python-Dev] Summer of Code: Developing complete SSL support for Python Message-ID: <20050604193203.03D9.FCANO@ono.com> Hello, This is my first post to the list but I wish to be an active developer of Python in the future. This project of Summer of Code from Google has opened my eyes to the open source community. The money has been a great motivation but thinking about coding something that will be remembered in the future and specially USEFUL to the community motivates me very much too. After this introduction I want to talk about SSL support in Python. I have read in the Summer of Code section of Python web that a possible project is coding a new complete module that implements SSL to Python. And possibly add this module to the python standard library. Appart from this the project would include adding functionality to the modules (like urlib) to support SSL as trasparent as possible. ?What do you think about it? **I'm actually looking for information in the archives of this list and web** but I would like to ask the list something: - Is necessary a complete new SSL module for Python? - What do you need about SSL ? - What do you think about incorporing SSL to act trasparently in the modules that use it in the actuality ? - Any kind of information, references, would be appreciate. I would like to do the new SSL module as good as possible. A piece of art and efficiency if possible and obviusly having in mind all programming standards. Thanks very much for any help. -- Florencio Cano Gabarda From aahz at pythoncraft.com Sat Jun 4 20:10:16 2005 From: aahz at pythoncraft.com (Aahz) Date: Sat, 4 Jun 2005 11:10:16 -0700 Subject: [Python-Dev] Summer of Code: Developing complete SSL support for Python In-Reply-To: <20050604193203.03D9.FCANO@ono.com> References: <20050604193203.03D9.FCANO@ono.com> Message-ID: <20050604181016.GA13484@panix.com> On Sat, Jun 04, 2005, Florencio Cano Gabarda wrote: > > This is my first post to the list but I wish to be an active developer > of Python in the future. This project of Summer of Code from Google has > opened my eyes to the open source community. The money has been a great > motivation but thinking about coding something that will be remembered > in the future and specially USEFUL to the community motivates me very > much too. > > After this introduction I want to talk about SSL support in Python. Great! Given the nature of your project, python-dev probably is the best place for you to get information, but I also encourage you to join the Summer of Code mailing list: http://mail.python.org/mailman/listinfo/summerofcode -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "The only problem with Microsoft is they just have no taste." --Steve Jobs From shane at hathawaymix.org Sat Jun 4 20:26:47 2005 From: shane at hathawaymix.org (Shane Hathaway) Date: Sat, 04 Jun 2005 12:26:47 -0600 Subject: [Python-Dev] Summer of Code: Developing complete SSL support for Python In-Reply-To: <20050604193203.03D9.FCANO@ono.com> References: <20050604193203.03D9.FCANO@ono.com> Message-ID: <42A1F267.6000504@hathawaymix.org> Florencio Cano Gabarda wrote: > I would like to do the new SSL module as good as possible. A piece of > art and efficiency if possible and obviusly having in mind all > programming standards. Guido and much of the community would certainly be appreciative of a new SSL module, especially if you can overcome the problems that plague M2Crypto. http://www.artima.com/weblogs/viewpost.jsp?thread=95863 I would say that the criteria for success would be: 1) A module, expected to be included in the standard library, that makes it easy to create both client and server SSL sockets. 2) No leaks or segfaults. 3) An API that any programmer can use without knowing much about cryptography. I want to be able to write code that's as simple as this: import socket import ssl def open_ssl_socket(address): base = socket.socket() base.connect(address) sock = ssl.client(base) return sock def run_server(port, handler, pki_files): keys = ssl.load_keys(pki_files) s = socket.socket() s.bind(('', port)) s.listen(5) while True: base, address = s.accept() sock = ssl.server(base, keys) handler(sock) sock.close() "pki_filenames" in the example is a list of key files, certificate files, certificiate signing requests, and perhaps other PKI files. I want the ssl module to figure out for itself what each file means, so that I as a mere human can forget about those details. :-) However, if there's any ambiguity in the set of files provided, the SSL module should throw an exception rather than try to guess the intent. If you're ambitious, you could also figure out how to make this work with non-blocking sockets. I believe Twisted has made progress there. Shane From gjc at inescporto.pt Sun Jun 5 14:05:25 2005 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Sun, 05 Jun 2005 13:05:25 +0100 Subject: [Python-Dev] Summer of Code: Developing complete SSL support for Python In-Reply-To: <42A1F267.6000504@hathawaymix.org> References: <20050604193203.03D9.FCANO@ono.com> <42A1F267.6000504@hathawaymix.org> Message-ID: <1117973125.8137.6.camel@emperor> On Sat, 2005-06-04 at 12:26 -0600, Shane Hathaway wrote: > Florencio Cano Gabarda wrote: > > I would like to do the new SSL module as good as possible. A piece of > > art and efficiency if possible and obviusly having in mind all > > programming standards. > > Guido and much of the community would certainly be appreciative of a new > SSL module, especially if you can overcome the problems that plague > M2Crypto. > > http://www.artima.com/weblogs/viewpost.jsp?thread=95863 > > I would say that the criteria for success would be: > > 1) A module, expected to be included in the standard library, that makes > it easy to create both client and server SSL sockets. > > 2) No leaks or segfaults. > > 3) An API that any programmer can use without knowing much about > cryptography. > > I want to be able to write code that's as simple as this: > > import socket > import ssl > > def open_ssl_socket(address): > base = socket.socket() > base.connect(address) > sock = ssl.client(base) > return sock > > def run_server(port, handler, pki_files): > keys = ssl.load_keys(pki_files) > s = socket.socket() > s.bind(('', port)) > s.listen(5) > while True: > base, address = s.accept() > sock = ssl.server(base, keys) > handler(sock) > sock.close() > > "pki_filenames" in the example is a list of key files, certificate > files, certificiate signing requests, and perhaps other PKI files. I > want the ssl module to figure out for itself what each file means, so > that I as a mere human can forget about those details. :-) However, if > there's any ambiguity in the set of files provided, the SSL module > should throw an exception rather than try to guess the intent. > > If you're ambitious, you could also figure out how to make this work > with non-blocking sockets. I believe Twisted has made progress there. 4. In the socket module documentation: ssl( sock[, keyfile, certfile]) Initiate a SSL connection over the socket sock. keyfile is the name of a PEM formatted file that contains your private key. certfile is a PEM formatted certificate chain file. On success, a new SSLObject is returned. Warning: This does not do any certificate verification! I would make it a top priority to enable certificate verification in ssl sockets. I don't see the point in doing SSL without certificate verification. It's just false security. Maybe adding a callback asking the application what to do if certificate validation fails, so that application writers can show a GUI dialogue or something like that... Best regards. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic From p.f.moore at gmail.com Sun Jun 5 18:10:17 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 5 Jun 2005 17:10:17 +0100 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: Message-ID: <79990c6b050605091051094e4@mail.gmail.com> On 6/1/05, Guido van Rossum wrote: > I hope that I've got the rewrite of PEP 343 to include generator > extensions right now. I've chosen the 'with' keyword. Please review > here; I think this is ready for review by the unwashed masses. :-) > > http://www.python.org/peps/pep-0343.html A fairly minor point - with_template read wrongly to me for quite a while: I read it as something along the lines of "with a template" (which then doesn't make much sense...) rather than "template for the with statement". Unfortunately, I don't have a better naming suggestion (other than simply "template", which is probably too general to work), so I just raise this as something you should expect to need to clarify a few times... Paul. From martin at v.loewis.de Sun Jun 5 18:24:43 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 05 Jun 2005 18:24:43 +0200 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: References: <429F20EB.8070800@gmail.com> Message-ID: <42A3274B.6090407@v.loewis.de> Guido van Rossum wrote: >> @with_template >> def closing(obj): >> try: >> yield obj >> finally: >> obj.close() >> > I just realized this has a race condition. The bytecode for the > expression closing(open("...")) must necessarily contain a bytecode > that calls open() followed by another bytecode that calls closing(). This is not a convincing point. That race condition always existed, e.g. in the traditional f = open(filename) try: process(f) finally: f.close() as you could always get an async exception between open returns and f is assigned. This isn't much of an issue, since CPython would always release the file immediately as the stack frame is cleared due to the exception. I think would should explicitly weaken our guarantees for "asynchronous" exceptions (of which we currently only have KeyboardInterrupt). The PEP should point out that an async exception between the beginning of the with statement and the assignment to the variable may or may not cause __exit__ to be called (depending on how far you are into __enter__) Regards, Martin From gvanrossum at gmail.com Sun Jun 5 18:33:28 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun, 5 Jun 2005 09:33:28 -0700 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <42A3274B.6090407@v.loewis.de> References: <429F20EB.8070800@gmail.com> <42A3274B.6090407@v.loewis.de> Message-ID: On 6/5/05, "Martin v. L?wis" wrote: > Guido van Rossum wrote: > >> @with_template > >> def closing(obj): > >> try: > >> yield obj > >> finally: > >> obj.close() > >> > > I just realized this has a race condition. The bytecode for the > > expression closing(open("...")) must necessarily contain a bytecode > > that calls open() followed by another bytecode that calls closing(). > > This is not a convincing point. That race condition always existed, > e.g. in the traditional > > f = open(filename) > try: > process(f) > finally: > f.close() > > as you could always get an async exception between open returns and > f is assigned. This isn't much of an issue, since CPython would always > release the file immediately as the stack frame is cleared due to > the exception. > > I think would should explicitly weaken our guarantees for > "asynchronous" exceptions (of which we currently only have > KeyboardInterrupt). The PEP should point out that an async > exception between the beginning of the with statement and > the assignment to the variable may or may not cause __exit__ > to be called (depending on how far you are into __enter__) That is pretty clear from the translation given in the PEP. What is not so clear is that the cace can be completely avoided *if* the call to __enter__ and the try-finally setup are combined in one opcode, *and* __enter__ is implemented in C, *and* the reversible actions are all done by __enter__. This is also why I don't like giving file objects __enter__ and __exit__ methods. I know all this doesn't matter in most situations, but sometimes it does, and it would be good if there was a solution -- today, there really isn't one except to rely on CPython's reference counting. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From noamraph at gmail.com Mon Jun 6 00:30:06 2005 From: noamraph at gmail.com (Noam Raphael) Date: Mon, 6 Jun 2005 00:30:06 +0200 Subject: [Python-Dev] Split MIME headers into multiple lines near a space In-Reply-To: <429A3BFD.2040805@gmail.com> References: <429A3BFD.2040805@gmail.com> Message-ID: On 5/30/05, Nick Coghlan wrote: > Noam's suggestion seems reasonable to me, but I'm not > sure what the performance implications are. I think that they are not critical. The number of lines can grow by at most twice, because shorter words would not have a line of their own. The added rfind call seems not very significant to me, since it is preceded by about log2n string encodings, to test if an encoded prefix fits the required line length. Have a good day, Noam From ncoghlan at gmail.com Mon Jun 6 11:31:56 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 06 Jun 2005 19:31:56 +1000 Subject: [Python-Dev] PEP 343 rewrite complete In-Reply-To: <79990c6b050605091051094e4@mail.gmail.com> References: <79990c6b050605091051094e4@mail.gmail.com> Message-ID: <42A4180C.1030009@gmail.com> Paul Moore wrote: > Unfortunately, I don't have a better naming suggestion (other than > simply "template", which is probably too general to work), so I just > raise this as something you should expect to need to clarify a few > times... PEP 346 used "statement_template" for the generator decorator. That could be shortened to "stmt_template" without losing much in readability ("stmt" being a fairly common abbreviation of "statement"). Avoiding the problem with the English meaning of the phrases "do template" and "with template" was part of the reason for PEP 346's choice of name (the other part being that the name tied in nicely with that PEP's "user defined statement" terminology). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From reinhold-birkenfeld-nospam at wolke7.net Mon Jun 6 20:20:26 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Mon, 06 Jun 2005 20:20:26 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement Message-ID: Hello, I am currently having some thoughts about the standard library, with regard to Python 2.5 and 3.0. Since I don't want to withhold them from you, here are they ;) - Flat namespace: Should we tend to a more hierarchic library (e.g. inet.url, inet.http, inet.nntp)? This would increase clarity when searching for a module. - 3rd party modules: There are many superb modules out there, some of which really do have a "standard" character. Examples include PIL, numarray, ElementTree, [wxPython - I know this is a hairy issue], - User interface: Tkinter may be "the" standard, but other Toolkits deserve notice, notably wxPython and PyGtk, the latter of which might be even easier to package. Reinhold, please don't hit me! -- Mail address is perfectly valid! From python at rcn.com Mon Jun 6 20:36:06 2005 From: python at rcn.com (Raymond Hettinger) Date: Mon, 6 Jun 2005 14:36:06 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: Message-ID: <004901c56ac6$be0e3c40$7105a044@oemcomputer> > - 3rd party modules: There are many superb modules out there, some of > which > really do have a "standard" character. Examples include PIL, numarray, > ElementTree, [wxPython - I know this is a hairy issue], If Fred were up for it, I think ElementTree would be a wonderful, must-have addition. Raymond Hettinger From skip at pobox.com Mon Jun 6 20:38:36 2005 From: skip at pobox.com (Skip Montanaro) Date: Mon, 6 Jun 2005 13:38:36 -0500 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: Message-ID: <17060.38956.60212.120855@montanaro.dyndns.org> Reinhold> - Flat namespace: Should we tend to a more hierarchic library Reinhold> (e.g. inet.url, inet.http, inet.nntp)? This would increase Reinhold> clarity when searching for a module. We've talked about this before. The main technical challenge seems to be backward compatibility. You need to support both flat ("import urllib") and packaged namespaces ("from www import urllib"), possibly within the same application. That is, postulating a www package, if I execute import urllib from www.urllib import urlopen the module-level code should only be executed once, and urlopen == urllib.urlopen should evaluate to True. Reinhold> - 3rd party modules: There are many superb modules out there, Reinhold> some of which really do have a "standard" Reinhold> character. Examples include PIL, numarray, ElementTree, Reinhold> [wxPython - I know this is a hairy issue], I think a good first step would be deciding on a package hierarchy. Third party packages could plug in wherever they fit best. Skip From barry at python.org Mon Jun 6 21:43:22 2005 From: barry at python.org (Barry Warsaw) Date: Mon, 06 Jun 2005 15:43:22 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <17060.38956.60212.120855@montanaro.dyndns.org> References: <17060.38956.60212.120855@montanaro.dyndns.org> Message-ID: <1118087002.3503.18.camel@geddy.wooz.org> On Mon, 2005-06-06 at 14:38, Skip Montanaro wrote: > import urllib > from www.urllib import urlopen > > the module-level code should only be executed once, and > > urlopen == urllib.urlopen > > should evaluate to True. Not to mention "urlopen is urllib.urlopen" -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050606/8b324452/attachment.pgp From skip at pobox.com Mon Jun 6 22:06:42 2005 From: skip at pobox.com (Skip Montanaro) Date: Mon, 6 Jun 2005 15:06:42 -0500 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <1118087002.3503.18.camel@geddy.wooz.org> References: <17060.38956.60212.120855@montanaro.dyndns.org> <1118087002.3503.18.camel@geddy.wooz.org> Message-ID: <17060.44242.802034.654426@montanaro.dyndns.org> >>>>> "Barry" == Barry Warsaw writes: Barry> On Mon, 2005-06-06 at 14:38, Skip Montanaro wrote: >> import urllib >> from www.urllib import urlopen >> >> the module-level code should only be executed once, and >> >> urlopen == urllib.urlopen >> >> should evaluate to True. Barry> Not to mention "urlopen is urllib.urlopen" Whoops, yeah. I was thinking in terms of module-level functions and classes, where I think == is sufficient: >>> import foo2 >>> foo2.f >>> g = foo2.f >>> reload(foo2) >>> foo2.f == g False Obviously, for data objects "is" is what you want. Skip From eric.nieuwland at xs4all.nl Mon Jun 6 22:35:07 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Mon, 6 Jun 2005 22:35:07 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <17060.38956.60212.120855@montanaro.dyndns.org> References: <17060.38956.60212.120855@montanaro.dyndns.org> Message-ID: Skip Montanaro wrote: > The main technical challenge seems to be > backward compatibility. You need to support both flat ("import > urllib") and > packaged namespaces ("from www import urllib"), possibly within the > same > application. That is, postulating a www package, if I execute > > import urllib > from www.urllib import urlopen > > the module-level code should only be executed once, and > > urlopen == urllib.urlopen > > should evaluate to True. Unless from __future__ import new_std_library and we don't allow mixed use within the same module. --eric From martin at v.loewis.de Mon Jun 6 23:55:43 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 06 Jun 2005 23:55:43 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <004901c56ac6$be0e3c40$7105a044@oemcomputer> References: <004901c56ac6$be0e3c40$7105a044@oemcomputer> Message-ID: <42A4C65F.6070301@v.loewis.de> Raymond Hettinger wrote: >> ElementTree, [wxPython - I know this is a hairy issue], > > > If Fred were up for it, I think ElementTree would be a wonderful, > must-have addition. I might be missing fine details of the English language here (what does "to be up for something" mean?), however, I believe ElementTree is an unlikely addition to the standard library. I also might be confusing people: AFAIK, ElementTree is Fredrik Lundh's work, not Fred Drake's, so it would be Fredrik who would need to contribute. My understanding is that Fredrik has indicated a dislike to contribute certain pieces of work, primarily out of fear to lose control. My experience with integrating his work without his explicit cooperation (specifically: sgmlop in PyXML) show that he won't object to somebody else doing such integration, but also won't try to cooperate afterwards. Regards, Martin From python at rcn.com Tue Jun 7 00:10:26 2005 From: python at rcn.com (Raymond Hettinger) Date: Mon, 6 Jun 2005 18:10:26 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <42A4C65F.6070301@v.loewis.de> Message-ID: <000b01c56ae4$8afdbe20$7105a044@oemcomputer> > > If Fred were up for it, I think ElementTree would be a wonderful, > > must-have addition. > I might be missing fine details of the English language here > (what does "to be up for something" mean?), however, I believe > ElementTree is an unlikely addition to the standard library. Rewritten: If Fredrik Lundh were supportive of the idea, I think the Standard Library would benefit greatly from incorporating ElementTree. Raymond From bob at redivi.com Tue Jun 7 00:17:54 2005 From: bob at redivi.com (Bob Ippolito) Date: Mon, 6 Jun 2005 15:17:54 -0700 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <000b01c56ae4$8afdbe20$7105a044@oemcomputer> References: <000b01c56ae4$8afdbe20$7105a044@oemcomputer> Message-ID: <0A1F09D1-415A-4E02-9622-FBFBE63E35E0@redivi.com> On Jun 6, 2005, at 3:10 PM, Raymond Hettinger wrote: >>> If Fred were up for it, I think ElementTree would be a wonderful, >>> must-have addition. >>> > > > >> I might be missing fine details of the English language here >> (what does "to be up for something" mean?), however, I believe >> ElementTree is an unlikely addition to the standard library. >> > > Rewritten: > > If Fredrik Lundh were supportive of the idea, I think the Standard > Library would benefit greatly from incorporating ElementTree. > Personally I'd like to see the standard library get smaller rather than larger. There's a whole lot of bit rot in there, and since sys.path prefers the standard library over anything else it's a really huge pain to integrate patches on a faster release schedule than Python's while remaining sane at the same time. -bob From tlesher at gmail.com Tue Jun 7 00:36:48 2005 From: tlesher at gmail.com (Tim Lesher) Date: Mon, 6 Jun 2005 18:36:48 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: Message-ID: <9613db600506061536520abece@mail.gmail.com> On 6/6/05, Reinhold Birkenfeld wrote: > - Flat namespace: Should we tend to a more hierarchic library (e.g. > inet.url, inet.http, inet.nntp)? This would increase clarity when > searching for a module. -1. I feel the opposite way: when trying to figure out where something "lives", I prefer Python's flat namespace to (for example) Java's com.company.baz.bar.foo hierarchy. > - 3rd party modules: There are many superb modules out there, some of which > really do have a "standard" character. Examples include PIL, numarray, > ElementTree, [wxPython - I know this is a hairy issue], I think the most important question for each of these is "is the module's release schedule at least as stable as Python's?". For many of these, I suspect the answer is "no". -- Tim Lesher From pje at telecommunity.com Tue Jun 7 00:46:59 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 06 Jun 2005 18:46:59 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <0A1F09D1-415A-4E02-9622-FBFBE63E35E0@redivi.com> References: <000b01c56ae4$8afdbe20$7105a044@oemcomputer> <000b01c56ae4$8afdbe20$7105a044@oemcomputer> Message-ID: <5.1.1.6.0.20050606182742.01f7d118@mail.telecommunity.com> At 03:17 PM 6/6/2005 -0700, Bob Ippolito wrote: >Personally I'd like to see the standard library get smaller rather >than larger. There's a whole lot of bit rot in there, and since >sys.path prefers the standard library over anything else it's a >really huge pain to integrate patches on a faster release schedule >than Python's while remaining sane at the same time. You know, before you said that, it hadn't occurred to me that the Python standard library is subject to the same economic forces that cause mega-packages like Twisted, SciPy, Zope, etc. to develop. Specifically, the cost incurred by relying on an externally-distributed dependency causes anyone with non-trivial needs to create "batteries included" libraries. One of my goals for Python Eggs and EasyInstall was to lower this dependency-cost barrier by reducing the "dependency cost" to zero at the point of installation, by making it as easy to install ten packages as one. (Another was to reduce the dependency cost for the developer, who need only add package metadata for the dependency to be fulfilled at installation time.) Now that you've pointed out the parallel between the stdlib and the other modules, I wonder if Python 3.0 might be able to take a more minimalist approach to the standard library, if it included the equivalents of easy_install and pkg_resources. From skip at pobox.com Tue Jun 7 01:21:11 2005 From: skip at pobox.com (Skip Montanaro) Date: Mon, 6 Jun 2005 18:21:11 -0500 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <9613db600506061536520abece@mail.gmail.com> References: <9613db600506061536520abece@mail.gmail.com> Message-ID: <17060.55911.922987.863770@montanaro.dyndns.org> Tim> On 6/6/05, Reinhold Birkenfeld wrote: >> - Flat namespace: Should we tend to a more hierarchic library (e.g. >> inet.url, inet.http, inet.nntp)? This would increase clarity when >> searching for a module. Tim> -1. I feel the opposite way: when trying to figure out where Tim> something "lives", I prefer Python's flat namespace to (for Tim> example) Java's com.company.baz.bar.foo hierarchy. I think Reinhold's suggestion (e.g., inet.*) was for a flatter namespace than Java's scheme, but for a slightly more structured namespace than the current standard library provides. Some amount of structure helps to collect modules together whose names don't obviously suggest who their neighbors are in the functionality space. For example, to the naive user it might not be obvious that these groups of modules and packages are related: * getopt and optparse * bsddb, gdbm and anydbm * email, mhlib, quopri * heapq, collections * user, site, sitecustomize * profile, hotshot, pstats * pipes, subprocess, os I wouldn't mind a stdlib that defined a set of top-level packages (some of which might be wholly unpopulated by modules in the standard distribution) It might, for example, define a gui package and gui.Tkinter and gui._tkinter modules, leaving the remainder of gui namespace available for 3rd party modules. Such a scheme would probably work best if there was some fairly lightweight registration/approval process in the community to avoid needless collisions. For example, it might be a bit confusing if two organizations began installing their packages in a subpackage named gui.widgets. An unsuspecting person downloading an application that used one version of gui.widgets into environment with the conflicting gui.widgets would run into trouble. Is the CPAN namespace wide open? If I wanted to create a Perl module to fit into the HTML namespace is there some procedure involved or is it an example of squatters' rights? >> - 3rd party modules: There are many superb modules out there, some of >> which really do have a "standard" character. Examples include PIL, >> numarray, ElementTree, [wxPython - I know this is a hairy issue], Tim> I think the most important question for each of these is "is the Tim> module's release schedule at least as stable as Python's?". For Tim> many of these, I suspect the answer is "no". If you provide the necessary namespace structure for them to nestle into, I suspect most of them could be maintained outside the stdlib just fine. Skip From fperez.net at gmail.com Tue Jun 7 02:12:26 2005 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 06 Jun 2005 18:12:26 -0600 Subject: [Python-Dev] Thoughts on stdlib evolvement References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> Message-ID: Skip Montanaro wrote: > I wouldn't mind a stdlib that defined a set of top-level packages (some of > which might be wholly unpopulated by modules in the standard distribution) > It might, for example, define a gui package and gui.Tkinter and gui._tkinter > modules, leaving the remainder of gui namespace available for 3rd party > modules. Such a scheme would probably work best if there was some fairly > lightweight registration/approval process in the community to avoid needless > collisions. For example, it might be a bit confusing if two organizations > began installing their packages in a subpackage named gui.widgets. An > unsuspecting person downloading an application that used one version of > gui.widgets into environment with the conflicting gui.widgets would run into > trouble. I've wondered if it wouldn't be better if the std lib were all stuffed into its own namespace: from std import urllib If a more structured approach is desired, it could be from std.www import urllib for example. But having std. as the top-level namespace for the stdlib, could simplify (I think) life a lot in the long run. If a decision for a more structured namespace is made, then it might be nice to have the same top-level structure in site-packages, albeit empty by default: from std.www import foo -> standard library www packages from www import bar -> third-party www packages Third-party packages can still be put into base site-packages, of course, but developers could be encouraged to transition into putting things into these categories. This would also ease the process of 'staging' a module as external for a while before deciding whether it meets the requirement for being put into the stdlib. Just an idea (sorry if it's been discussed and shot down before). best, f From jcarlson at uci.edu Tue Jun 7 06:24:52 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 06 Jun 2005 21:24:52 -0700 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> Message-ID: <42A52194.9020008@uci.edu> Fernando Perez wrote: > Skip Montanaro wrote: > > >>I wouldn't mind a stdlib that defined a set of top-level packages (some of >>which might be wholly unpopulated by modules in the standard distribution) >>It might, for example, define a gui package and gui.Tkinter and gui._tkinter >>modules, leaving the remainder of gui namespace available for 3rd party >>modules. Such a scheme would probably work best if there was some fairly >>lightweight registration/approval process in the community to avoid needless >>collisions. For example, it might be a bit confusing if two organizations >>began installing their packages in a subpackage named gui.widgets. An >>unsuspecting person downloading an application that used one version of >>gui.widgets into environment with the conflicting gui.widgets would run into >>trouble. > > > I've wondered if it wouldn't be better if the std lib were all stuffed into its > own namespace: > > from std import urllib > > If a more structured approach is desired, it could be > > from std.www import urllib This generally brings up the intersection of stdlib and nonstdlib naming hierarchy. More specifically, what does "import email" mean? Presumably it means to import the email module or package, but from the current module directory, or from the standard library? I posted about this months ago, and got to writing an import hook that used a method suggested by Guido by naming things (sys.modules names) based on whether they were imported relative to the module with a __name__ of '__main__', and using the prefix dot semantics for package-relative imports to mean any relative imports. I eventually got hung up on some nested import bits, and got busy with work. I may have time to hack on it next week to finish the 'proof of concept' (currently in the Bay area for work), but if someone wants or needs more explanation, I would be happy to offer it. - Josiah From stephen at xemacs.org Tue Jun 7 06:26:41 2005 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 07 Jun 2005 13:26:41 +0900 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <17060.55911.922987.863770@montanaro.dyndns.org> (Skip Montanaro's message of "Mon, 6 Jun 2005 18:21:11 -0500") References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> Message-ID: <87r7fe7pxa.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Skip" == Skip Montanaro writes: Skip> If you provide the necessary namespace structure for them to Skip> nestle into, I suspect most of them could be maintained Skip> outside the stdlib just fine. FWIW, this has worked well for XEmacs; it's one of our most popular features vis-a-vis GNU Emacs. Users like it because we do provide a centralized directory of modules (with many mirrors) that they can download updates from, as well as a "batteries included" library distribution (which we call "SUMO tarballs" for historical reasons). Package developers like it because they can maintain closer control, yet have up-to-date versions distributed from our repository. Finally the core maintainers like it because there's a clear delegation of responsibility. We provide a centralized repository, which is mirrored by a couple score sites; I'm not sure how much harder it would be to make it distributed. The essential feature is the centralized directory. We do have problems. One is the megapackage issue that Phillip Eby referred in his reply to Bob Ippolito <5.1.1.6.0.20050606182742.01f7d118 at mail.telecommunity.com>. Our packaging system doesn't work well for them. Some of the problems are our fault, but I think it's actually a hard problem; we could fix up some details, but not really address the core issues of megapackages. The other one is that our initial division into "core" and "package" was pretty radical, and several core developers still want to pull certain packages back into core. But this creates backward compatibility problems. I think this would be less of a problem for Python. Creating the package repository could be more conservative: Python's standard library has plenty of modules that would not present such problems. In any case, Python has better facilities for dealing with backward compatibility issues (such as deprecation warnings and the like) than we do, and the language itself is simply more suited to preserving modularity than Emacs Lisp is (with its single dynamically scoped namespace). Our package namespace structure is trivial (it's basically flat); what is more important than the namespace structure, I think, is provision of package organization and directory utilities as Phillip Eby mentions. If they're reasonably flexible and Python invests some effort in helping package maintainers with different current distribution methods to adapt, I think this would work well for Python too. One thing we have found is that it takes only a small amount of incremental work to maintain the package directory and repository as such, even as it has grown dramatically. By contrast, I find the Zope wiki-based organization of Products quite annoying to work with. Of course the packaged code itself can and does bitrot if the external maintainer goes AWOL, or simply doesn't like hanging out with XEmacs people. In such cases we've found that users will often step up to help with the nitty-gritty management of "their" package, such as tracking bug reports, liaison with an "unsociable" upstream, regression testing, and integrating changes (anybody can run make and patch), even if they have to ask for help with code-related work. Ie, the fact that there is a well-defined package requiring specific management tasks encourages them to take on the work they can handle. For XEmacs the effort of separating out the packages was moderately large; it took about a year, with three or four developers spending a substantial fraction of their time on it, either organizing the repository or creating infrastructure. It was a very worthwhile investment. Python's investment probably would be proportionately smaller, as much of the necessary infrastructure seems to be already available. For XEmacs, I think the main returns come from creating well-defined chunks that people can "own" and take responsibility for. This helps to mitigate the "dormant bug" problem, and generally keeps the packaged code fresher than it was in the old days, even though we have a much larger volume of code and probably less manpower now. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From fperez.net at gmail.com Tue Jun 7 06:27:41 2005 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 06 Jun 2005 22:27:41 -0600 Subject: [Python-Dev] Thoughts on stdlib evolvement References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> <42A52194.9020008@uci.edu> Message-ID: Josiah Carlson wrote: > Fernando Perez wrote: >> I've wondered if it wouldn't be better if the std lib were all stuffed into >> its own namespace: >> >> from std import urllib >> >> If a more structured approach is desired, it could be >> >> from std.www import urllib > > This generally brings up the intersection of stdlib and nonstdlib naming > hierarchy. More specifically, what does "import email" mean? > Presumably it means to import the email module or package, but from the > current module directory, or from the standard library? Well, I've thought of this (ligthly) mostly as a py3k thing, since it would require that 'import email' in the naked fails, as it would become 'import std.email', or 'import std.www.email' or whatever. A plain 'import email' would then refer to some third-party 'email' module, not part of the standard library. Since this would mean a massive break of exisiting code, it would necessarily be a py3k issue. But nonetheless the idea of confinign the stdlib to the 'std' namespace does have some appeal, at least to me. Best, f From jcarlson at uci.edu Tue Jun 7 07:24:14 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 06 Jun 2005 22:24:14 -0700 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> <42A52194.9020008@uci.edu> Message-ID: <42A52F7E.3050206@uci.edu> Fernando Perez wrote: > Josiah Carlson wrote: >>Fernando Perez wrote: >>>I've wondered if it wouldn't be better if the std lib were all stuffed into >>>its own namespace: >>> >>>from std import urllib >>> >>>If a more structured approach is desired, it could be >>> >>>from std.www import urllib >> >>This generally brings up the intersection of stdlib and nonstdlib naming >>hierarchy. More specifically, what does "import email" mean? >>Presumably it means to import the email module or package, but from the >>current module directory, or from the standard library? > > > Well, I've thought of this (ligthly) mostly as a py3k thing, since it would > require that 'import email' in the naked fails, as it would become 'import > std.email', or 'import std.www.email' or whatever. A plain 'import email' > would then refer to some third-party 'email' module, not part of the standard > library. > > Since this would mean a massive break of exisiting code, it would necessarily be > a py3k issue. But nonetheless the idea of confinign the stdlib to the 'std' > namespace does have some appeal, at least to me. Actually no. Re: std.* If there were a single path (or set of paths) know at install time where standard library modules are, then that can be placed in a PYSDTLIB environment variable (or some other specified place). Inside Python, whenever an import is done relative to std, one merely examines the paths offered via the PYSTDLIB environment variable for the looked-for module. For modules/packages in site-packages, one could use another name, perhaps "packages" or somesuch, to refer to it. sys.path is not usable as-is because it can contain the path of the module named __main__, and is (not uncommonly) modified by user code. This can be used with a proper import hook, which is turned on per-module via a __future__ import, hence no backwards compatibility issues. What I had been working on is the use of special import semantics for non-stdlib/site-packages modules. Specifically if you were to write 'mymodule.py' and import your other module 'myothermodule.py' in the same path, you would use "from . import myothermodule". It doesn't solve any problems with standard library naming, but it does allow one to do cousin imports... "from ..uncle import cousin", as well as all sorts of other things, which allows (in writing software) for modules and packages to be shared by other packages. - Josiah From dstanek at dstanek.com Tue Jun 7 13:09:26 2005 From: dstanek at dstanek.com (David Stanek) Date: Tue, 7 Jun 2005 07:09:26 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <17060.38956.60212.120855@montanaro.dyndns.org> References: <17060.38956.60212.120855@montanaro.dyndns.org> Message-ID: <20050607110926.GA7863@goliath.hsd1.oh.comcast.net> On Mon, Jun 06, 2005 at 01:38:36PM -0500, Skip Montanaro wrote: > > > Reinhold> - Flat namespace: Should we tend to a more hierarchic library > Reinhold> (e.g. inet.url, inet.http, inet.nntp)? This would increase > Reinhold> clarity when searching for a module. > > We've talked about this before. The main technical challenge seems to be > backward compatibility. You need to support both flat ("import urllib") and > packaged namespaces ("from www import urllib"), possibly within the same > application. That is, postulating a www package, if I execute > > import urllib > from www.urllib import urlopen > > the module-level code should only be executed once, and > > urlopen == urllib.urlopen > > should evaluate to True. Can't this be handled with an __init__ module? Create an inet package and having the __init__ module pull whatever it wanted into it's own namespace. It would be nice to then use warnings to show deprecation messages when the old import is used (import url instead of import inet.url), but I don't think that is quite so easy. -- David Stanek www.roninds.net GPG keyID #6272EDAF on http://pgp.mit.edu Key fingerprint = 8BAA 7E11 8856 E148 6833 655A 92E2 3E00 6272 EDAF -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050607/0d897c63/attachment.pgp From maliger at gmail.com Tue Jun 7 16:15:44 2005 From: maliger at gmail.com (Martin Aliger) Date: Tue, 7 Jun 2005 16:15:44 +0200 Subject: [Python-Dev] python running in several threads Message-ID: <63fd41d40506070715129c904a@mail.gmail.com> Hello Python developers, I know this is developers list and I hope I'm right here. I'm using python as scripting engine embedded in application but issue should be more general. When several scripts are executed simultaneously (under several threads) one have to acquire/release locks. This degrades performance IMHO, mainly on server installations (many web servers use python engine). Moreover those scripts are commonly independent chunks of code which do not create other threats. Under those limitations, do you think, an easy rewrite of execution engine is possible to allow simultaneous run of all threads? I checked the sources esp. PyThreadState object and it seems all those variables could be held locally. Another way is using thread local storage, but it could be a little problematic with porting issues. How do you see it? Do you have any plans here? Thanks and regards, Martin From aahz at pythoncraft.com Tue Jun 7 17:32:53 2005 From: aahz at pythoncraft.com (Aahz) Date: Tue, 7 Jun 2005 08:32:53 -0700 Subject: [Python-Dev] python running in several threads In-Reply-To: <63fd41d40506070715129c904a@mail.gmail.com> References: <63fd41d40506070715129c904a@mail.gmail.com> Message-ID: <20050607153253.GB2787@panix.com> On Tue, Jun 07, 2005, Martin Aliger wrote: > > Under those limitations, do you think, an easy rewrite of execution > engine is possible to allow simultaneous run of all threads? Short answer: no Longer answer: see previous threads about removing the GIL; if you still have questions, please post to comp.lang.python. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ f u cn rd ths, u cn gt a gd jb n nx prgrmmng. From reinhold-birkenfeld-nospam at wolke7.net Tue Jun 7 19:40:49 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Tue, 07 Jun 2005 19:40:49 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <17060.55911.922987.863770@montanaro.dyndns.org> References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> Message-ID: Skip Montanaro wrote: > Tim> On 6/6/05, Reinhold Birkenfeld wrote: > > >> - Flat namespace: Should we tend to a more hierarchic library (e.g. > >> inet.url, inet.http, inet.nntp)? This would increase clarity when > >> searching for a module. > > Tim> -1. I feel the opposite way: when trying to figure out where > Tim> something "lives", I prefer Python's flat namespace to (for > Tim> example) Java's com.company.baz.bar.foo hierarchy. > > I think Reinhold's suggestion (e.g., inet.*) was for a flatter namespace > than Java's scheme, but for a slightly more structured namespace than the > current standard library provides. Some amount of structure helps to > collect modules together whose names don't obviously suggest who their > neighbors are in the functionality space. For example, to the naive user it > might not be obvious that these groups of modules and packages are related: > > * getopt and optparse > * bsddb, gdbm and anydbm > * email, mhlib, quopri > * heapq, collections > * user, site, sitecustomize > * profile, hotshot, pstats > * pipes, subprocess, os Yep, exactly. Java's namespaces are ugly, that's right, but a flatter version certainly improves readability. """Namespaces are one honking great idea -- let's do more of those!""" > I wouldn't mind a stdlib that defined a set of top-level packages (some of > which might be wholly unpopulated by modules in the standard distribution) > It might, for example, define a gui package and gui.Tkinter and gui._tkinter > modules, leaving the remainder of gui namespace available for 3rd party > modules. Such a scheme would probably work best if there was some fairly > lightweight registration/approval process in the community to avoid needless > collisions. For example, it might be a bit confusing if two organizations > began installing their packages in a subpackage named gui.widgets. An > unsuspecting person downloading an application that used one version of > gui.widgets into environment with the conflicting gui.widgets would run into > trouble. Is the CPAN namespace wide open? If I wanted to create a Perl > module to fit into the HTML namespace is there some procedure involved or is > it an example of squatters' rights? Personally, I think that CPAN is one of the greatest strengths of Perl. The language is a mess, and the quality of many modules is questionable, but it's incredibly easy to find a module for handling a special problem, and the namespaces are IMHO well thought out. Additionally, the docs > >> - 3rd party modules: There are many superb modules out there, some of > >> which really do have a "standard" character. Examples include PIL, > >> numarray, ElementTree, [wxPython - I know this is a hairy issue], > > Tim> I think the most important question for each of these is "is the > Tim> module's release schedule at least as stable as Python's?". For > Tim> many of these, I suspect the answer is "no". > > If you provide the necessary namespace structure for them to nestle into, I > suspect most of them could be maintained outside the stdlib just fine. Exactly! There needn't be such a big separation between stdlib and 3rd party. Access to the net is standard nowadays, though it wouldn't be of any harm making a big distribution with all modules available, for downloading and burning on CD, for example. PJE's great EasyInstall and Python Eggs will be a perfect starting point for this, I think. Reinhold -- Mail address is perfectly valid! From reinhold-birkenfeld-nospam at wolke7.net Tue Jun 7 19:40:40 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Tue, 07 Jun 2005 19:40:40 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> Message-ID: Fernando Perez wrote: > Skip Montanaro wrote: > >> I wouldn't mind a stdlib that defined a set of top-level packages (some of >> which might be wholly unpopulated by modules in the standard distribution) >> It might, for example, define a gui package and gui.Tkinter and gui._tkinter >> modules, leaving the remainder of gui namespace available for 3rd party >> modules. Such a scheme would probably work best if there was some fairly >> lightweight registration/approval process in the community to avoid needless >> collisions. For example, it might be a bit confusing if two organizations >> began installing their packages in a subpackage named gui.widgets. An >> unsuspecting person downloading an application that used one version of >> gui.widgets into environment with the conflicting gui.widgets would run into >> trouble. > > I've wondered if it wouldn't be better if the std lib were all stuffed into its > own namespace: > > from std import urllib > > If a more structured approach is desired, it could be > > from std.www import urllib One may want to look at the "py.std" approach of "the py lib", found at http://codespeak.net/py/current/doc/misc.html#the-py-std-hook Reinhold -- Mail address is perfectly valid! From rrr at ronadam.com Tue Jun 7 20:31:34 2005 From: rrr at ronadam.com (Ron Adam) Date: Tue, 07 Jun 2005 14:31:34 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: Message-ID: <42A5E806.1060802@ronadam.com> Reinhold Birkenfeld wrote: > Hello, > > I am currently having some thoughts about the standard library, with regard > to Python 2.5 and 3.0. Since I don't want to withhold them from you, here > are they ;) > > - Flat namespace: Should we tend to a more hierarchic library (e.g. > inet.url, inet.http, inet.nntp)? This would increase clarity when > searching for a module. I think putting the standard library in it's own package would be the first step. "std" has been mentioned a couple of times and I think its a good name for it. For backwards compatibility, maybe a "from __compatible__ import ver_2.x", or something similar, could be used as a way to allow future versions a bit more flexibility to change? Trying to move the modules in python/lib to python/packages/stdlib to see what would happen revealed the following files are required to be in python/lib to start the python shell without errors. copy_reg.py ntpath.py os.py site.py stat.py types.py These files are modules site.py imports. Maybe these should be builtins so that the shell could start without importing any modules? > - 3rd party modules: There are many superb modules out there, some of which > really do have a "standard" character. Examples include PIL, numarray, > ElementTree, [wxPython - I know this is a hairy issue], I'm on the side of less in builtins and less in the standard library, and for having a central repository for developers to distribute their packages. A python updater that gets a list of installable packages to install or uninstall would be fantastic. ;-) Having less in the core distribution means smaller complete applications to install when py2exe is used. There also needs to be some assurance that the standard library has as few bugs in it as possible. The more that's added to it, the more work and testing to do that is needed. > - User interface: Tkinter may be "the" standard, but other Toolkits deserve > notice, notably wxPython and PyGtk, the latter of which might be even > easier to package. I think these fall into the category of optional "official" extension packages that could be included in the core installer, but are not part of the standard library package itself. They are packages that would be maintained separately from the core and have their own developers dedicated to maintaining them. The 'official', or other designator if appropriate, would mean that they are important packages that fulfill a need and efforts to make and keep them compatible with the current python release is made. An "official" package designation might be something that developers could strive for and proudly display on their website or resume. Some minimum standards would be needed... ie current with current version of python, fully functional, fulfilling a need, etc. A request and voting process of some sort where python-dev can award "officialdom" to package developers might be usefull for this. Just a few thoughts, Cheers, Ron From jcarlson at uci.edu Tue Jun 7 20:47:18 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 07 Jun 2005 11:47:18 -0700 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <42A5E806.1060802@ronadam.com> References: <42A5E806.1060802@ronadam.com> Message-ID: <42A5EBB6.1020505@uci.edu> Ron Adam wrote: [snip] > Having less in the core distribution means smaller complete applications > to install when py2exe is used. There also needs to be some assurance > that the standard library has as few bugs in it as possible. The more > that's added to it, the more work and testing to do that is needed. Almost by definition, applications using py2exe are minimal Python installations without external dependencies. - Josiah From irmen at xs4all.nl Tue Jun 7 20:49:11 2005 From: irmen at xs4all.nl (Irmen de Jong) Date: Tue, 07 Jun 2005 20:49:11 +0200 Subject: [Python-Dev] problem installing current cvs Message-ID: <42A5EC27.8010409@xs4all.nl> Hi, I'm having 2 problems with the current cvs : During compilation this warning occurs: *** WARNING: renaming "dbm" since importing it failed: build/lib.linux-i686-2.5/ dbm.so: undefined symbol: dbm_firstkey and the 'dbm' module is unavailable. I'm running MandrakeLinux 2005 (10.2) gcc 3.4.3 (I'm also having this problem when compiling python 2.3.5 or 2.4.1) furthermore the 'make install' of current cvs fails halfway trough with the following errors: ..... ..... Compiling /opt/python25/lib/python2.5/bsddb/test/test_associate.py ... Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', ('/opt/python25/lib/python2.5/bsddb/test/test_associate.py', 97, 23, '\t os.mkdir(homeDir)\n')) Compiling /opt/python25/lib/python2.5/bsddb/test/test_basics.py ... Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', ('/opt/python25/lib/python2.5/bsddb/test/test_basics.py', 400, 26, '\t if get_raises_error:\n')) Compiling /opt/python25/lib/python2.5/bsddb/test/test_compare.py ... Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', ('/opt/python25/lib/python2.5/bsddb/test/test_compare.py', 167, 5, '\t"""\n')) ..... ..... Compiling /opt/python25/lib/python2.5/bsddb/test/test_recno.py ... Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', ('/opt/python25/lib/python2.5/bsddb/test/test_recno.py', 38, 46, '\tget_returns_none = d.set_get_returns_none(2)\n')) ..... ..... make: *** [libinstall] Error 1 $ And then it quits. Fixing the tab indentation errors locally makes the problem go away. Regards, Irmen de Jong From bob at redivi.com Tue Jun 7 20:54:58 2005 From: bob at redivi.com (Bob Ippolito) Date: Tue, 7 Jun 2005 11:54:58 -0700 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <42A5EBB6.1020505@uci.edu> References: <42A5E806.1060802@ronadam.com> <42A5EBB6.1020505@uci.edu> Message-ID: <603C535E-BADD-4A96-9D2F-664DD6017ACA@redivi.com> On Jun 7, 2005, at 11:47 AM, Josiah Carlson wrote: > Ron Adam wrote: > [snip] > >> Having less in the core distribution means smaller complete >> applications >> to install when py2exe is used. There also needs to be some >> assurance >> that the standard library has as few bugs in it as possible. The >> more >> that's added to it, the more work and testing to do that is needed. >> > > Almost by definition, applications using py2exe are minimal Python > installations without external dependencies. Yeah, py2exe only includes the portion of the standard library that you use, anyway... so the size of it is not an issue. -bob From rrr at ronadam.com Tue Jun 7 21:13:06 2005 From: rrr at ronadam.com (Ron Adam) Date: Tue, 07 Jun 2005 15:13:06 -0400 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <603C535E-BADD-4A96-9D2F-664DD6017ACA@redivi.com> References: <42A5E806.1060802@ronadam.com> <42A5EBB6.1020505@uci.edu> <603C535E-BADD-4A96-9D2F-664DD6017ACA@redivi.com> Message-ID: <42A5F1C2.4010203@ronadam.com> Bob Ippolito wrote: > > On Jun 7, 2005, at 11:47 AM, Josiah Carlson wrote: > >> Ron Adam wrote: >> [snip] >> >>> Having less in the core distribution means smaller complete >>> applications >>> to install when py2exe is used. There also needs to be some assurance >>> that the standard library has as few bugs in it as possible. The more >>> that's added to it, the more work and testing to do that is needed. >>> >> >> Almost by definition, applications using py2exe are minimal Python >> installations without external dependencies. > > > Yeah, py2exe only includes the portion of the standard library that you > use, anyway... so the size of it is not an issue. > > -bob This wasn't the main point of my post in any case. Yes, as high bandwidth connections and disk space become more common, the organizational aspects are really more important than the size aspects. Cheers, Ron From dstanek at dstanek.com Tue Jun 7 21:28:32 2005 From: dstanek at dstanek.com (dstanek@dstanek.com) Date: Tue, 7 Jun 2005 15:28:32 -0400 (EDT) Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: <9613db600506061536520abece@mail.gmail.com> <17060.55911.922987.863770@montanaro.dyndns.org> Message-ID: <26887.207.58.192.150.1118172512.squirrel@207.58.192.150> > Fernando Perez wrote: >> Skip Montanaro wrote: >> >> I've wondered if it wouldn't be better if the std lib were all stuffed >> into its >> own namespace: >> >> from std import urllib >> >> If a more structured approach is desired, it could be >> >> from std.www import urllib > > One may want to look at the "py.std" approach of "the py lib", found at > http://codespeak.net/py/current/doc/misc.html#the-py-std-hook > > Reinhold Thats where I hijacked the idea. Doing that would allow both to work. Legacy code could still use: import logging and new code could use: import std.logging At some point the legacy way to do things should be deprecated. David From greg at electricrain.com Tue Jun 7 23:08:16 2005 From: greg at electricrain.com (Gregory P. Smith) Date: Tue, 7 Jun 2005 14:08:16 -0700 Subject: [Python-Dev] problem installing current cvs - TabError In-Reply-To: <42A5EC27.8010409@xs4all.nl> References: <42A5EC27.8010409@xs4all.nl> Message-ID: <20050607210816.GB19337@zot.electricrain.com> On Tue, Jun 07, 2005 at 08:49:11PM +0200, Irmen de Jong wrote: > furthermore the 'make install' of current cvs fails halfway trough > with the following errors: > > ..... > ..... > Compiling /opt/python25/lib/python2.5/bsddb/test/test_associate.py ... > Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', > ('/opt/python25/lib/python2.5/bsddb/test/test_associate.py', 97, 23, '\t > os.mkdir(homeDir)\n')) > Compiling /opt/python25/lib/python2.5/bsddb/test/test_basics.py ... > Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', > ('/opt/python25/lib/python2.5/bsddb/test/test_basics.py', 400, 26, '\t if > get_raises_error:\n')) > Compiling /opt/python25/lib/python2.5/bsddb/test/test_compare.py ... > Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', > ('/opt/python25/lib/python2.5/bsddb/test/test_compare.py', 167, 5, '\t"""\n')) > ..... > ..... > Compiling /opt/python25/lib/python2.5/bsddb/test/test_recno.py ... > Sorry: TabError: ('inconsistent use of tabs and spaces in indentation', > ('/opt/python25/lib/python2.5/bsddb/test/test_recno.py', 38, 46, '\tget_returns_none = > d.set_get_returns_none(2)\n')) > ..... > ..... > make: *** [libinstall] Error 1 > $ > > And then it quits. > Fixing the tab indentation errors locally makes the problem go away. ugh. this may be the result of me working on those files recently without my usual .vimrc in place. i'll take a look. regardless, where is the TabError coming from? those files are all valid python even if they do have an annoying mix of spaces and tabs. major gripe to -dev: a 'make install' of the full cvs tree should not be required to test that some changes to existing .py files that pass their tests (in this case they -are- the tests) are ok to check in. Greg From martin at v.loewis.de Wed Jun 8 00:28:29 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 08 Jun 2005 00:28:29 +0200 Subject: [Python-Dev] python running in several threads In-Reply-To: <20050607153253.GB2787@panix.com> References: <63fd41d40506070715129c904a@mail.gmail.com> <20050607153253.GB2787@panix.com> Message-ID: <42A61F8D.2090704@v.loewis.de> Aahz wrote: >>Under those limitations, do you think, an easy rewrite of execution >>engine is possible to allow simultaneous run of all threads? > > > Short answer: no > > Longer answer: see previous threads about removing the GIL; if you still > have questions, please post to comp.lang.python. Or, as a medium-sized answer: google for "python free threading". Removing the GIL is easy, but then the interpreter crashes in cases of simultaneous accesses to dictionaries, reference counters, etc. I also disagree with Martin Aliger's premises: # This degrades performance IMHO, mainly on server installations # (many web servers use python engine). This is just not true. On a single-processor machine, the GIL does *not* degrade performance. Instead, it increases throughput (and thus performance). On a multi-processor machine, you often use multiple operating system processes to serve request (e.g. in CGI or Apache mpm-prefork, or even the typical mpm-worker configuration). If you then have different processes running Python, they don't interfere with each other at all. Regards, Martin From bac at OCF.Berkeley.EDU Wed Jun 8 06:28:48 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue, 07 Jun 2005 21:28:48 -0700 Subject: [Python-Dev] problem installing current cvs - TabError In-Reply-To: <20050607210816.GB19337@zot.electricrain.com> References: <42A5EC27.8010409@xs4all.nl> <20050607210816.GB19337@zot.electricrain.com> Message-ID: <42A67400.6070102@ocf.berkeley.edu> Gregory P. Smith wrote: [SNIP] > major gripe to -dev: a 'make install' of the full cvs tree should not > be required to test that some changes to existing .py files that pass > their tests (in this case they -are- the tests) are ok to check in. > You actually don't have to. If you execute the interpreter that is built in your cvs checkout it will use the stdlib files as found in your cvs tree. So if you are in your cvs checkout, ``./python.exe `` will actually execute using the code in the cvs checkout itself. -Brett From anthony at interlink.com.au Wed Jun 8 06:49:05 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed, 8 Jun 2005 14:49:05 +1000 Subject: [Python-Dev] problem installing current cvs - TabError In-Reply-To: <20050607210816.GB19337@zot.electricrain.com> References: <42A5EC27.8010409@xs4all.nl> <20050607210816.GB19337@zot.electricrain.com> Message-ID: <200506081449.09254.anthony@interlink.com.au> On Wednesday 08 June 2005 07:08, Gregory P. Smith wrote: > ugh. this may be the result of me working on those files recently > without my usual .vimrc in place. i'll take a look. regardless, > where is the TabError coming from? those files are all valid python > even if they do have an annoying mix of spaces and tabs. There's a scripts Tools/scripts/reindent.py - put it somewhere on your PATH and run it before checkin, like "reindent.py -r Lib". It means Tim or I don't have to run it for you Anthony -- Anthony Baxter It's never too late to have a happy childhood. From fredrik at pythonware.com Wed Jun 8 14:30:39 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 8 Jun 2005 14:30:39 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement References: <42A4C65F.6070301@v.loewis.de> <000b01c56ae4$8afdbe20$7105a044@oemcomputer> Message-ID: Raymond Hettinger wrote: >> > If Fred were up for it, I think ElementTree would be a wonderful, >> > must-have addition. > >> I might be missing fine details of the English language here >> (what does "to be up for something" mean?), however, I believe >> ElementTree is an unlikely addition to the standard library. > > Rewritten: > > If Fredrik Lundh were supportive of the idea, I think the Standard > Library would benefit greatly from incorporating ElementTree. shipping stable versions of ElementTree/cElementTree (or PIL, or python- doc, or exemaker, or what else you might find useful) with official Python releases is perfectly okay. moving the main trunk and main development over to the Python CVS is another thing, entirely. From fredrik at pythonware.com Wed Jun 8 14:40:52 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 8 Jun 2005 14:40:52 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement References: <42A4C65F.6070301@v.loewis.de><000b01c56ae4$8afdbe20$7105a044@oemcomputer> Message-ID: > moving the main trunk and main development over to the Python CVS is > another thing, entirely. (as I've said many times before, both the user community and the developer community would benefit if the core standard library were made smaller, and more externally maintained packages were included in the standard releases) (the development platform could need an overhaul as well; cvs/sourceforge/tex is a serious bottleneck. but the former doesn't require the latter). From amk at amk.ca Wed Jun 8 22:57:34 2005 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 8 Jun 2005 16:57:34 -0400 Subject: [Python-Dev] Bug day on the 25th? Message-ID: <20050608205734.GA13329@rogue.amk.ca> It seems like a good idea to have another Python bug day. Saturday June 25th seems the most practical date (this coming weekend is too soon, and the weekend after is a minor holiday -- Father's Day). We'd convene in the usual place: the #pydotorg IRC channel, on irc.freenode.net. Assuming no one can think of a reason why the 25th would be unsuitable, I'll send an announcement to c.l.py.a in a few days. --amk From askpete at gmail.com Thu Jun 9 05:21:20 2005 From: askpete at gmail.com (Peter Moore) Date: Thu, 9 Jun 2005 13:21:20 +1000 Subject: [Python-Dev] FishEye on Python CVS Repository Message-ID: <52431c5005060820217cb1f1fb@mail.gmail.com> Greetings Python Developers, I'm responsible for setting up free FishEye hosting for community projects. As a long time python user I of course added Python up front. You can see it here: http://fisheye.cenqua.com/viewrep/python/ If you aren't familiar with FishEye, it is a repository browsing, searching, analysis, monitoring..., tool for CVS (SVN coming soon). Cool features include: RSS feeds http://fisheye.cenqua.com/changelog/%7Erss/python/rss.xml Synthetic changesets http://fisheye.cenqua.com/changelog/python/ and Pretty ediffs http://fisheye.cenqua.com/viewrep/python/python/nondist/peps/pep-0343.txt?r1=1.22&r2=1.23 SQL like search http://fisheye.cenqua.com/search/python/?ql= Note that normally FishEye is pretty close to real time (i.e. when run on a local repository), but we can only get SF updates once a day, and they are potentially a bit old when we grab them. Feel free to mail me direct pete _at_ cenqua _dot_ com if you want to tweak the python instance or have any other questions/comments not appropriate for this list. Cheers, Pete. --- +61-414-587637 www.cenqua.com/fisheye From kbk at shore.net Fri Jun 10 05:03:56 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu, 9 Jun 2005 23:03:56 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200506100303.j5A33uIC002882@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 339 open ( -5) / 2857 closed (+12) / 3196 total ( +7) Bugs : 908 open ( -8) / 5036 closed (+22) / 5944 total (+14) RFE : 189 open ( -2) / 168 closed ( +5) / 357 total ( +3) New / Reopened Patches ______________________ Trivial typo in unicodedata._getcode (2005-06-02) http://python.org/sf/1213831 opened by Darek Suchojad Support non-file source/dest in marshal (2005-06-04) http://python.org/sf/1214879 opened by Skip Montanaro file.encoding support for file.write and file.writelines (2005-06-04) http://python.org/sf/1214889 opened by Reinhold Birkenfeld two fileinput enhancements (fileno, openhook) (2005-06-05) http://python.org/sf/1215184 opened by Reinhold Birkenfeld Suggested Additional Material for urllib2 docs (2005-06-08) http://python.org/sf/1216942 opened by Mike Foord proposed patch for tls wrapped ssl support added to smtplib (2005-06-08) http://python.org/sf/1217246 opened by Pete G Patches Closed ______________ [ast] fix for 1183468: return/yield in class (2005-04-16) http://python.org/sf/1184418 closed by nascheme [AST] throw SyntaxError in "from x import y," (2005-05-04) http://python.org/sf/1194895 closed by nascheme fix for trivial flatten bug in astgen (2005-01-04) http://python.org/sf/1095541 closed by nascheme Add st_flags support to (l)stat function (2005-06-01) http://python.org/sf/1212117 closed by perky buffer overflow in _cursesmodule.c (2005-05-11) http://python.org/sf/1200134 closed by akuchling Typo in Curses-Function doc (2005-04-20) http://python.org/sf/1186781 closed by akuchling binary formats for marshalling floats (2005-04-11) http://python.org/sf/1180995 closed by mwh test_locale fix on modern linux (2005-05-07) http://python.org/sf/1197218 closed by anthonybaxter An URL for UnicodeData File Format 3.2 has changed. (2005-05-25) http://python.org/sf/1207985 closed by perky Patch for [ 1170331 ] Error in base64.b32decode (2005-03-27) http://python.org/sf/1171487 closed by akuchling #1074261 gzip dies on gz files with many appended headers (2004-11-27) http://python.org/sf/1074381 closed by akuchling asynchat does not accept 'long' terminator (2004-08-03) http://python.org/sf/1002763 closed by akuchling New / Reopened Bugs ___________________ Queue.qsize() better info about unreliability (2005-06-02) http://python.org/sf/1213475 opened by Nikos Kouremenos os.path.realpath() cannot handle symlinks (2005-06-02) CLOSED http://python.org/sf/1213894 opened by Ilya Sandler test_marshal.py discards marshal.load val (2005-06-04) CLOSED http://python.org/sf/1214662 opened by Skip Montanaro subprocess auto-reaps children (2005-06-04) http://python.org/sf/1214859 opened by Mattias Engdeg?rd bsddb dbobj.DB.associate doesn't accept dbobj.DB param (2005-06-04) http://python.org/sf/1215023 opened by Gregory P. Smith int('x',radix) puzzle (2005-06-05) CLOSED http://python.org/sf/1215146 opened by elgordo String and list methods deeply hidden (2005-06-06) http://python.org/sf/1215887 opened by Reinhold Birkenfeld Large tarfiles cause overflow (2005-06-06) http://python.org/sf/1215928 opened by Tom Emerson Replace MSVC memory allocator with ptmalloc2 (2005-06-07) http://python.org/sf/1216562 opened by Niall Douglas csv module sometimes raises _csv.Error (2005-06-08) CLOSED http://python.org/sf/1216831 opened by Mary Gardiner LINKCC->CXX, -ltermcap->-lncurses (2005-06-08) http://python.org/sf/1216923 opened by Niki W. Waibel Info from man python not in docs (2005-06-08) http://python.org/sf/1217152 opened by Kent Johnson Omission in docs for smtplib.SMTP.sendmail() (2005-06-09) http://python.org/sf/1217513 opened by Kent Johnson make frameworkinstall fails for non-default location (2005-06-09) http://python.org/sf/1217591 opened by Mitch Chapman pydoc includes unnecessary files for a package. (2005-06-09) http://python.org/sf/1217881 opened by Igor Belyi Bugs Closed ___________ check for return/yield outside function is wrong (2005-04-15) http://python.org/sf/1183468 closed by nascheme Incorrect result for regular expression - "|(hello)|(world)" (2005-06-01) http://python.org/sf/1212411 closed by tim_one doc bug in Lock.acquire (2005-05-27) http://python.org/sf/1209880 closed by akuchling (?(id)yes|no) only works when referencing the first group (2005-04-06) http://python.org/sf/1177831 closed by akuchling Notation (2005-04-30) http://python.org/sf/1193001 closed by akuchling os.path.realpath() cannot handle symlinks (2005-06-03) http://python.org/sf/1213894 closed by birkenfeld bz2.BZ2File doesn't handle modes correctly (2005-05-03) http://python.org/sf/1194181 closed by birkenfeld test_marshal.py discards marshal.load val (2005-06-04) http://python.org/sf/1214662 closed by montanaro WeakValueDictionary.__init__ is backwards (2005-05-06) http://python.org/sf/1196315 closed by birkenfeld Typo in "Differences from mimelib" (2005-05-27) http://python.org/sf/1210001 closed by birkenfeld IDLE 1.0.5 (Python 2.3.5) crashes under Windows (2005-05-21) http://python.org/sf/1206232 closed by kbk example broken in section 1.12 of Extending & Embedding (2005-04-16) http://python.org/sf/1184380 closed by birkenfeld try to open /dev/null as directory (2005-04-15) http://python.org/sf/1183585 closed by birkenfeld int('x',radix) puzzle (2005-06-05) http://python.org/sf/1215146 closed by birkenfeld compileall doesn't notice syntax errors (2001-03-30) http://python.org/sf/412436 closed by birkenfeld doc speaks of extensions option while it's *called* ext_modu (2005-03-25) http://python.org/sf/1170422 closed by akuchling dumbdbm hoses index on load failure (2005-03-29) http://python.org/sf/1172763 closed by akuchling csv module sometimes raises _csv.Error (2005-06-07) http://python.org/sf/1216831 closed by montanaro error in documentation for splitting empty string (2005-03-28) http://python.org/sf/1171994 closed by akuchling Error in base64.b32decode (2005-03-24) http://python.org/sf/1170331 closed by akuchling gzip dies on gz files with many appended headers (2004-11-27) http://python.org/sf/1074261 closed by akuchling datetime changes missing from "Porting from 2.3 to 2.4" (2004-12-04) http://python.org/sf/1079134 closed by akuchling New / Reopened RFE __________________ Move test_method_checksum from test_unicodedata. (2005-06-02) CLOSED http://python.org/sf/1213703 opened by Darek Suchojad module warnings lacks a remove filter function (2005-06-04) http://python.org/sf/1214675 opened by Torsten Bronger Add Error Code Dictionary to urllib2 (2005-06-08) http://python.org/sf/1216944 opened by Mike Foord RFE Closed __________ Move test_method_checksum from test_unicodedata. (2005-06-02) http://python.org/sf/1213703 closed by rhettinger Prepending Text (2005-05-31) http://python.org/sf/1212169 closed by rhettinger The array module and the buffer interface (2005-04-26) http://python.org/sf/1190033 closed by rhettinger expm1 and log1p (2005-03-14) http://python.org/sf/1163139 closed by rhettinger Support file path concat with "/" (2002-11-27) http://python.org/sf/644940 closed by birkenfeld From kentsin at gmail.com Thu Jun 9 10:13:31 2005 From: kentsin at gmail.com (kent sin) Date: Thu, 9 Jun 2005 16:13:31 +0800 Subject: [Python-Dev] Example workaround classes for using Unicode with csv module... Message-ID: The suggestion Skip is indeed very useful, however it does not work when some of the item is not string, here is another try: class UnicodeReader: def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds): self.reader = csv.reader(f, dialect=dialect, **kwds) self.encoding = encoding def next(self): row = self.reader.next() t = [] for s in row: try: t.append(unicode(s,self.encoding)) except: t.append(s) return t # [unicode(s, self.encoding) for s in row] This will not work with non string type def __iter__(self): return self class UnicodeWriter: def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds): self.writer = csv.writer(f, dialect=dialect, **kwds) self.encoding = encoding def writerow(self, row): t = [] for s in row: try: t.append(unicode(s,"utf-8")) except: t.append(s) self.writer.writerow(t) #self.writer.writerow([s.encode("utf-8") for s in row]) #! This is not working with non-string objects. def writerows(self, rows): for row in rows: self.writerow(row) -- Sin Hang Kin. From pp64 at codelock.com Wed Jun 8 05:23:18 2005 From: pp64 at codelock.com (Pavel Pergamenshchik) Date: Tue, 7 Jun 2005 20:23:18 -0700 Subject: [Python-Dev] b32encode and NUL bytes Message-ID: <20050607202318.45a459c2@dunce> Hi. Is this a feature? I do see b32encode padding the string with NULs first. >>> b32decode(b32encode('\x00')) Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/base64.py", line 228, in b32decode last = binascii.unhexlify(hex(acc)[2:-1]) TypeError: Odd-length string From skip at pobox.com Fri Jun 10 14:23:40 2005 From: skip at pobox.com (Skip Montanaro) Date: Fri, 10 Jun 2005 07:23:40 -0500 Subject: [Python-Dev] [Python-checkins] python/dist/src/Doc/lib libtokenize.tex, 1.5, 1.6 In-Reply-To: References: Message-ID: <17065.34380.208609.448951@montanaro.dyndns.org> Raymond> Add untokenize() function to allow full round-trip tokenization. Raymond> Should significantly enhance the utility of the module by Raymond> supporting the creation of tools that modify the token stream Raymond> and writeback the modified result. Raymond, Very interesting. Skip From gustavo at niemeyer.net Fri Jun 10 16:35:47 2005 From: gustavo at niemeyer.net (Gustavo Niemeyer) Date: Fri, 10 Jun 2005 11:35:47 -0300 Subject: [Python-Dev] Bug day on the 25th? In-Reply-To: <20050608205734.GA13329@rogue.amk.ca> References: <20050608205734.GA13329@rogue.amk.ca> Message-ID: <20050610143546.GA5929@burma.localdomain> Greetings, > It seems like a good idea to have another Python bug day. Saturday > June 25th seems the most practical date (this coming weekend is too > soon, and the weekend after is a minor holiday -- Father's Day). > We'd convene in the usual place: the #pydotorg IRC channel, on > irc.freenode.net. > > Assuming no one can think of a reason why the 25th would be > unsuitable, I'll send an announcement to c.l.py.a in a few days. Not sure if that's a reason to prevent the event, since fixing bugs is a good thing no matter when, but that's two days before EuroPython, and many people might be moving to the conference at that time. -- Gustavo Niemeyer http://niemeyer.net From gustavo at niemeyer.net Fri Jun 10 16:48:35 2005 From: gustavo at niemeyer.net (Gustavo Niemeyer) Date: Fri, 10 Jun 2005 11:48:35 -0300 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: Message-ID: <20050610144835.GB5929@burma.localdomain> > > moving the main trunk and main development over to the Python CVS is > > another thing, entirely. > > (as I've said many times before, both the user community and the developer > community would benefit if the core standard library were made smaller, and > more externally maintained packages were included in the standard releases) An issue to consider about this is that maintainers (not talking about you or anyone else specifically) have different concepts of stability, and while it may seem perfectly ok to refactor external modules between two stable releases, doing so in the standard library would spread fear and "python is so untrustful" feelings. That's something a good police, reflecting practices that we learn "by osmose" while living in that environment (python-dev) for a while, could try to handle. Enforcing it is another issue, of course. -- Gustavo Niemeyer http://niemeyer.net From amk at amk.ca Fri Jun 10 17:23:00 2005 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 10 Jun 2005 11:23:00 -0400 Subject: [Python-Dev] Bug day on the 25th? In-Reply-To: <20050610143546.GA5929@burma.localdomain> References: <20050608205734.GA13329@rogue.amk.ca> <20050610143546.GA5929@burma.localdomain> Message-ID: <20050610152300.GA28316@rogue.amk.ca> On Fri, Jun 10, 2005 at 11:35:47AM -0300, Gustavo Niemeyer wrote: > Not sure if that's a reason to prevent the event, since > fixing bugs is a good thing no matter when, but that's two > days before EuroPython, and many people might be moving to > the conference at that time. Thanks for pointing that out. It'll probably mean that most of the participants will be North Americans. The next available weekend would be July 9th, though, so I think we should go ahead with the bug day on the 25th. I'll try to have another one in July, too. Maybe we should adopt a standard day for bug days, e.g. second Saturday of the month, or something like that. --amk From mdehoon at c2b2.columbia.edu Fri Jun 10 18:59:03 2005 From: mdehoon at c2b2.columbia.edu (Michiel De Hoon) Date: Fri, 10 Jun 2005 12:59:03 -0400 Subject: [Python-Dev] Five patch reviews & patch request Message-ID: <6CA15ADD82E5724F88CB53D50E61C9AE7AC151@cgcmail.cgc.cpmc.columbia.edu> Recently, I reviewed the following five patches: [ 827386 ] absolute paths cause problems for MSVC [ 1161914 ] python-config [ 1187396 ] Add const specifier to PySpam_System prototype [ 1196895 ] in IDLE, assume new text files are python source by default [ 1196917 ] change recall in IDLE shell to not overwrite current command Below are the summaries of the patch reviews. I'd like to ask your attention to this patch: [ 1049855 ] PyOS_InputHook inconsistency on Windows Kurt Kaiser has already looked at this patch, but was hesitant to check it in because he wasn't able to test it on Windows. I'd be happy to send a compiled Python for Windows to anybody willing to have a look at this patch. Thanks! --Michiel. Patch review summaries: ----------------------- [ 827386 ] absolute paths cause problems for MSVC Solves an inconsistency between distutils' msvccompiler.py and ccompiler.py. I am surprised by how absolute paths are handled in ccompiler.py, so I'm not wholeheartedly in favor of copying this behavior to msvccompiler.py. [ 1161914 ] python-config Currently, the patch seems to be broken. Better integration with distutils is needed. Can't recommend this patch in its current form. [ 1187396 ] Add const specifier to PySpam_System prototype This error is already fixed in Revision 1.33 of python/dist/src/Doc/ext/extending.tex. Patch can be closed. [ 1196895 ] in IDLE, assume new text files are python source by default When going to File > New in IDLE, currently IDLE assumes that the new file is a generic text file. With the patch, IDLE assumes that the new file contains python source code by default, and uses syntax highlighting. Whereas currently syntax highlighting is switched on as soon as the file is saved as *.py, after the patch syntax highlighting is not switched off if the file is saved as *.txt. So the patch does not seem to be perfectly OK. [ 1196917 ] change recall in IDLE shell to not overwrite current command I agree that this patch makes IDLE easier to use. Recommend applying this patch. Michiel de Hoon Center for Computational Biology and Bioinformatics Columbia University 1150 St Nicholas Avenue New York, NY 10032 From theller at python.net Fri Jun 10 19:46:00 2005 From: theller at python.net (Thomas Heller) Date: Fri, 10 Jun 2005 19:46:00 +0200 Subject: [Python-Dev] Five patch reviews & patch request In-Reply-To: <6CA15ADD82E5724F88CB53D50E61C9AE7AC151@cgcmail.cgc.cpmc.columbia.edu> (Michiel De Hoon's message of "Fri, 10 Jun 2005 12:59:03 -0400") References: <6CA15ADD82E5724F88CB53D50E61C9AE7AC151@cgcmail.cgc.cpmc.columbia.edu> Message-ID: "Michiel De Hoon" writes: > I'd like to ask your attention to this patch: > > [ 1049855 ] PyOS_InputHook inconsistency on Windows > > Kurt Kaiser has already looked at this patch, but was hesitant to check it in > because he wasn't able to test it on Windows. I'd be happy to send a compiled > Python for Windows to anybody willing to have a look at this patch. Comment about the patch (only from reading it): The PyOS_StdioReadline function calls my_fgets with a file pointer argument. The my_fgets function in the patch assumes that STD_INPUT_HANDLE is the handle to use - is this assumption always correct? Thomas From jepler at unpythonic.net Fri Jun 10 20:13:35 2005 From: jepler at unpythonic.net (Jeff Epler) Date: Fri, 10 Jun 2005 13:13:35 -0500 Subject: [Python-Dev] b32encode and NUL bytes In-Reply-To: <20050607202318.45a459c2@dunce> References: <20050607202318.45a459c2@dunce> Message-ID: <20050610181335.GB6998@unpythonic.net> On Tue, Jun 07, 2005 at 08:23:18PM -0700, Pavel Pergamenshchik wrote: > Hi. > Is this a feature? I do see b32encode padding the string with NULs first. > > >>> b32decode(b32encode('\x00')) > Traceback (most recent call last): > File "", line 1, in ? > File "/usr/lib/python2.4/base64.py", line 228, in b32decode > last = binascii.unhexlify(hex(acc)[2:-1]) > TypeError: Odd-length string This also seems suspect: >>> base64.b32encode("\0a") 'ABQQ====' >>> base64.b32decode(_) 'a' Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050610/36128810/attachment.pgp From gvanrossum at gmail.com Fri Jun 10 22:23:04 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 10 Jun 2005 13:23:04 -0700 Subject: [Python-Dev] PEP 343 - next steps Message-ID: While there's still some activity in the Wiki, nothing (to me) sounds like signs of serious disagreement or truly better alternatives. So I think I'd like to move forward towards acceptance soon (before EuroPython). Two issues brought up in the Wiki are worth considering: - throw() is a term taken from Java & C++. We can't call the method raise() -- but perhaps we can call it next_raising() or next_raise(), which emphasizes the similarity with next(). Thoughts? I'm not strong on this; I think throw() is fine too, especially since I expect that it will be used explicitly extremely rarely -- the only customer is the with_template decorator. - Whether and how to keep a door open for a future extension to the syntax that allows multiple resources to be acquired in a single with-statement. Possible syntax could be (a) with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...: or (b) with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...: Variant (a) seems better and is more analogous to the use of 'as' in import statements, and (b) has the disadvantage that if you want to acquire several resources and not all of them have an associated variable, you'll have to sprinkle dummy variables on the right of 'as'. So (a) would have my preference. But I would still like to start off without this extension. The issue is: if we allow VAR to be a comma-separated list of variables now, that cuts off the extension to (a) in the future; so the PEP would have to be amended to state that VAR must be a single variable or a list of variables IN PARENTHESES. Thoughts? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Jun 10 22:49:13 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 10 Jun 2005 16:49:13 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: Message-ID: <5.1.1.6.0.20050610164649.0218fea0@mail.telecommunity.com> At 01:23 PM 6/10/2005 -0700, Guido van Rossum wrote: >- throw() is a term taken from Java & C++. We can't call the method >raise() -- but perhaps we can call it next_raising() or next_raise(), >which emphasizes the similarity with next(). Thoughts? I'm not strong >on this; I think throw() is fine too, especially since I expect that >it will be used explicitly extremely rarely -- the only customer is >the with_template decorator. I'm fine with throw, but if you really want to get 'raise' in there, how about 'raise_exc' or 'raise_exc_info'? >The issue is: if we allow VAR to be a >comma-separated list of variables now, that cuts off the extension to >(a) in the future; so the PEP would have to be amended to state that >VAR must be a single variable or a list of variables IN PARENTHESES. >Thoughts? Parentheses would make it clearer what's going on, so I'd be fine with that. From python at rcn.com Sat Jun 11 00:49:50 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri, 10 Jun 2005 18:49:50 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: Message-ID: <002d01c56e0e$d9699520$fa2ec797@oemcomputer> > - throw() is a term taken from Java & C++. This was intended. There was much discussion about this for PEP 288 and this was more or less a concensus choice. The term is already associated with exceptions in other languages and it captures the concept of the raise occurring somewhere else (what is thrown at or into). It is simple, clear, and tends to suggest all the right things. I have no doubt that someone can point out differing semantics in other languages but I don't care. The verbal cue is what we are after. Raymond From rowen at cesmail.net Sat Jun 11 01:11:34 2005 From: rowen at cesmail.net (Russell E. Owen) Date: Fri, 10 Jun 2005 16:11:34 -0700 Subject: [Python-Dev] PEP 343 - next steps References: Message-ID: In article , Guido van Rossum wrote: >... > - Whether and how to keep a door open for a future extension to the > syntax that allows multiple resources to be acquired in a single > with-statement. Possible syntax could be > > (a) with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...: > > or > > (b) with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...: > > Variant (a) seems better and is more analogous to the use of 'as' in > import statements, and (b) has the disadvantage that if you want to > acquire several resources and not all of them have an associated > variable, you'll have to sprinkle dummy variables on the right of > 'as'. So (a) would have my preference. But I would still like to start > off without this extension. The issue is: if we allow VAR to be a > comma-separated list of variables now, that cuts off the extension to > (a) in the future; so the PEP would have to be amended to state that > VAR must be a single variable or a list of variables IN PARENTHESES. > Thoughts? I agree that (a) sounds better. Also, I think it is very reasonable to require that multiple variables be in parenthesis. You can always lift the restriction later (i.e. if enough folks complain and you decide never to implement (a)), but you can't add such a restriction later. One nit-pick: I suggest not using the term "list of in parenthesis" in the PEP or other docs. It's clear here but may add to some people's list vs tuple confusion. -- Russell From gvanrossum at gmail.com Sat Jun 11 02:05:14 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 10 Jun 2005 17:05:14 -0700 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <002d01c56e0e$d9699520$fa2ec797@oemcomputer> References: <002d01c56e0e$d9699520$fa2ec797@oemcomputer> Message-ID: On 6/10/05, Raymond Hettinger wrote: > > - throw() is a term taken from Java & C++. > > This was intended. There was much discussion about this for PEP 288 and > this was more or less a concensus choice. The term is already > associated with exceptions in other languages and it captures the > concept of the raise occurring somewhere else (what is thrown at or > into). It is simple, clear, and tends to suggest all the right things. > > I have no doubt that someone can point out differing semantics in other > languages but I don't care. The verbal cue is what we are after. Cool. throw() it is. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Sat Jun 11 07:02:20 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri, 10 Jun 2005 22:02:20 -0700 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: Message-ID: <42AA705C.4010408@ocf.berkeley.edu> Guido van Rossum wrote: [SNIP - Guido already said throw() is the name to be used] > - Whether and how to keep a door open for a future extension to the > syntax that allows multiple resources to be acquired in a single > with-statement. Possible syntax could be > > (a) with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...: > > or > > (b) with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...: > > Variant (a) seems better and is more analogous to the use of 'as' in > import statements, and (b) has the disadvantage that if you want to > acquire several resources and not all of them have an associated > variable, you'll have to sprinkle dummy variables on the right of > 'as'. So (a) would have my preference. But I would still like to start > off without this extension. The issue is: if we allow VAR to be a > comma-separated list of variables now, that cuts off the extension to > (a) in the future; so the PEP would have to be amended to state that > VAR must be a single variable or a list of variables IN PARENTHESES. > Thoughts? > I like a), so I say restrict the grammar as needed to allow it become a possibility. Forcing parens is a minor thing and since this is not syntactically the same as assignment the difference is negligible. -Brett From amk at amk.ca Sat Jun 11 13:08:35 2005 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 11 Jun 2005 07:08:35 -0400 Subject: [Python-Dev] b32encode and NUL bytes In-Reply-To: <20050610181335.GB6998@unpythonic.net> References: <20050607202318.45a459c2@dunce> <20050610181335.GB6998@unpythonic.net> Message-ID: <20050611110835.GA2217@rogue.amk.ca> On Fri, Jun 10, 2005 at 01:13:35PM -0500, Jeff Epler wrote: > > Is this a feature? I do see b32encode padding the string with NULs first. This is bug #1170331, which was fixed when I applied patch #1171487 earlier this week. > This also seems suspect: > >>> base64.b32encode("\0a") > 'ABQQ====' > >>> base64.b32decode(_) > 'a' This case is also fixed in CVS. --amk From ark-mlist at att.net Sat Jun 11 16:09:26 2005 From: ark-mlist at att.net (Andrew Koenig) Date: Sat, 11 Jun 2005 10:09:26 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: Message-ID: <002c01c56e8f$2d6ca240$6402a8c0@arkdesktop> > The issue is: if we allow VAR to be a > comma-separated list of variables now, that cuts off the extension to > (a) in the future; so the PEP would have to be amended to state that > VAR must be a single variable or a list of variables IN PARENTHESES. > Thoughts? I am not sure how relevant this is, but the syntax "x as (a, b)" sure looks to me like (a, b) is a tuple. Of course, I'm biased by ML, which has a feature that I wish Python had: fun f(x as (a, b)) = ... This says that f's argument must be a 2-element tuple. It binds x to the argument and also binds a and b to the argument's components. Of course the program could be written this way: fun f(x) = let val (a, b) = x in ... end but the "as" syntax is more succinct, direct, and convenient. If such a feature were implemented in Python, I would imagine it to allow usages such as x as (a, b) = (3, 4) which would be equivalent to x = (a, b) = (3, 4) Of course, this usage shows that the syntax is unnecessary in this context, but what I care about is def f(x as (a, b)): # ... which has the advantage over the alternative def f(x): (a, b) = x # ... that if you call f with the wrong arguments, you get a clearer diagnostic message. It's kind of an edge case, and I am not seriously pushing for its adoption, but I do think it worth pointing out that when I see "x as (a, b)", that's what it immediately brings to mind. From tanzer at swing.co.at Sat Jun 11 16:15:16 2005 From: tanzer at swing.co.at (tanzer@swing.co.at) Date: Sat, 11 Jun 2005 16:15:16 +0200 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: Your message of "Sat, 11 Jun 2005 10:09:26 EDT." <002c01c56e8f$2d6ca240$6402a8c0@arkdesktop> Message-ID: "Andrew Koenig" wrote: > Of course, this usage shows that the syntax is unnecessary in this context, > but what I care about is > > def f(x as (a, b)): > # ... > > which has the advantage over the alternative > > def f(x): > (a, b) = x > # ... > > that if you call f with the wrong arguments, you get a clearer diagnostic > message. Thanks to the time machine, you can already do this: def f((a, b)): # ... -- Christian Tanzer http://www.c-tanzer.at/ From ncoghlan at gmail.com Sat Jun 11 16:24:18 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Jun 2005 00:24:18 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <002c01c56e8f$2d6ca240$6402a8c0@arkdesktop> References: <002c01c56e8f$2d6ca240$6402a8c0@arkdesktop> Message-ID: <42AAF412.8060006@gmail.com> Andrew Koenig wrote: > Of course, this usage shows that the syntax is unnecessary in this context, > but what I care about is > > def f(x as (a, b)): > # ... > > which has the advantage over the alternative > > def f(x): > (a, b) = x > # ... > > that if you call f with the wrong arguments, you get a clearer diagnostic > message. Doesn't tuple unpacking give you what you want here already? Py> def f((a, b)): ... print a, b ... Py> f(1, 2) Traceback (most recent call last): File "", line 1, in ? TypeError: f() takes exactly 1 argument (2 given) Py> f((1, 2)) 1 2 A couple of more complex examples: Py> def f((a, b), (c, d)): ... print a, b, c, d ... Py> f((1, 2), (3, 4)) 1 2 3 4 Py> def f((a, b, (c, d))): ... print a, b, c, d ... Py> f((1, 2, (3, 4))) 1 2 3 4 About the only downside is the need to rebuild the tuple if you actually need it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Sat Jun 11 16:56:54 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Jun 2005 00:56:54 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: Message-ID: <42AAFBB6.2070802@gmail.com> Guido van Rossum wrote: > While there's still some activity in the Wiki, nothing (to me) sounds > like signs of serious disagreement or truly better alternatives. So I > think I'd like to move forward towards acceptance soon (before > EuroPython). I agree with keeping throw() as the exception injection method name, and the addition of required parentheses when using tuple unpacking when creating VAR. I also added a new question to the Wiki regarding what, if any, guarantees will be made regarding when (or if) asynchronous interrupts will be blocked. Given that the call to __enter__() can invoke arbitrary Python code, is pushing the with statement setup inside a single opcode actually sufficient? It also considers the possibility of using with statements in an RAII style by acquiring the resource in __init__ or __new__ rather than __enter__. This is a topic that got a brief mention during early discussion here, but doesn't seem to be covered in the current version of PEP 343. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From stephen at xemacs.org Sat Jun 11 18:01:09 2005 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 12 Jun 2005 01:01:09 +0900 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: <20050610144835.GB5929@burma.localdomain> (Gustavo Niemeyer's message of "Fri, 10 Jun 2005 11:48:35 -0300") References: <20050610144835.GB5929@burma.localdomain> Message-ID: <877jh0x4qi.fsf@tleepslib.sk.tsukuba.ac.jp> >>>>> "Gustavo" == Gustavo Niemeyer writes: Gustavo> An issue to consider about this is that maintainers (not Gustavo> talking about you or anyone else specifically) have Gustavo> different concepts of stability, and while it may seem Gustavo> perfectly ok to refactor external modules between two Gustavo> stable releases, doing so in the standard library would Gustavo> spread fear and "python is so untrustful" feelings. This simply hasn't been a problem in XEmacs's equivalent to the standard library. In fact, the library modules tend to be more stable than the core, for several reasons. One important one is that the modules we distribute are not necessarily the bleeding edge. In particular, we try to keep up (within a couple weeks) with bugfixes to stable lines, but often lag several months after a major refactoring. This multitrack approach does involve some extra administrative work, but the burden on the core team is minimal. Most modules are managed by volunteers who do not feel able to contribute to the core (and often not to the coding of "their" module), but are very pleased to be able to contribute in this way. They are usually far more competent than the core developers to judge when new features are sufficiently attractive and stable to warrant updating to a refactored version, too. They tend to be more conservative than the module's lead maintainer, too. I will grant that XEmacs deserves (and probably has) a lower trust rating than Python, but that is due to flaws in the _core_, not in the package management. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. From gvanrossum at gmail.com Sat Jun 11 18:02:39 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat, 11 Jun 2005 09:02:39 -0700 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42AAFBB6.2070802@gmail.com> References: <42AAFBB6.2070802@gmail.com> Message-ID: [Nick] > I also added a new question to the Wiki regarding what, if any, > guarantees will be made regarding when (or if) asynchronous interrupts > will be blocked. Given that the call to __enter__() can invoke > arbitrary Python code, is pushing the with statement setup inside a > single opcode actually sufficient? Not sufficient if __enter__ is written in Python; in that case, *nothing* can be done to make it sufficient. The single opcode makes it possible to implement __enter__ in C to solve the issue in specific cases. > It also considers the possibility of using with statements in an RAII > style by acquiring the resource in __init__ or __new__ rather than > __enter__. Python isn't C++; you shouldn't do that. > This is a topic that got a brief mention during early discussion here, > but doesn't seem to be covered in the current version of PEP 343. I haven't been infected with the RAII meme, so I can't help you here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sat Jun 11 21:09:59 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 11 Jun 2005 15:09:59 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: <42AAFBB6.2070802@gmail.com> <42AAFBB6.2070802@gmail.com> Message-ID: <5.1.1.6.0.20050611143158.01f73808@mail.telecommunity.com> At 09:02 AM 6/11/2005 -0700, Guido van Rossum wrote: >[Nick] > > I also added a new question to the Wiki regarding what, if any, > > guarantees will be made regarding when (or if) asynchronous interrupts > > will be blocked. Given that the call to __enter__() can invoke > > arbitrary Python code, is pushing the with statement setup inside a > > single opcode actually sufficient? > >Not sufficient if __enter__ is written in Python; in that case, >*nothing* can be done to make it sufficient. I think PEP 343 would greatly benefit from some discussion of the signalling issue. I've been wracking my brain a bit on it, and I'm having trouble seeing where the problem is. >The single opcode makes it possible to implement __enter__ in C to >solve the issue in specific cases. Or if __enter__ is a no-op, that would also solve the problem. Suitability for 'with' could be keyed off of the __exit__ method instead. If there's no __enter__, or the __enter__ is a default no-op __enter__ written in C, then __exit__ is sufficient for many resources. > > It also considers the possibility of using with statements in an RAII > > style by acquiring the resource in __init__ or __new__ rather than > > __enter__. > >Python isn't C++; you shouldn't do that. Actually, it seems to me that it's standard procedure in today's Python: objects allocate resources in __init__, and then have a 'close()' or some similar method to release them. That's why so many people see this: with open('foo') as f: ... As a natural use of the 'with' statement, whether they have any familiarity with C++ or not. (My own experiences with C++ are perhaps a decade out of date, long before RAII was a meme in any event.) It seems to me that there are two sensible patterns for using 'with': 1. __enter__ allocates or acquires a resource, __exit__ releases it 2. EXPR allocates or acquires a resource, __exit__ releases it (and __enter__ doesn't execute any Python, because it's a no-op) Pattern 1 is useful for transactions, locks, redirection, decimal contexts, etc. Pattern 2 is useful for resources that are acquired by creation anyway, like files. But maybe my understanding is flawed, because I don't *really* understand the signal issue. I've been assuming that you're talking about the fact that an exception can occur anywhere, due to e.g. KeyboardInterrupt or an exception raised by some other signal handler. It seems to me that using pattern 2, one could write 'closing()' safely by doing this: class closing(object): def __init__(self, resource): try: self.resource = resource except: resource.close() raise def __exit__(self,*exc): self.resource.close() but maybe I'm still fundamentally "not getting" how the signal issue manifests in practice. I suppose it's true that *another* exception could then occur between the LOAD_ATTR and the CALL opcodes in the exception handler here, but it's also true that this could happen with any Python exception-handling code, and it could also happen in __exit__(). So I'm not sure I see why this should influence the design of 'with' very much, other than to point out that no Python code is safe from asynchronous exceptions. In any case, C-implemented types can handle this situation just fine, so a 'closing' written in C could easily be included with the implementation. From barry at python.org Sat Jun 11 23:43:43 2005 From: barry at python.org (Barry Warsaw) Date: Sat, 11 Jun 2005 17:43:43 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: Message-ID: <1118526223.9425.127.camel@geddy.wooz.org> On Fri, 2005-06-10 at 16:23, Guido van Rossum wrote: > While there's still some activity in the Wiki, nothing (to me) sounds > like signs of serious disagreement or truly better alternatives. So I > think I'd like to move forward towards acceptance soon (before > EuroPython). Well, I finally read PEP 343 in its entirety. Having not really had the time to follow this thread, I have to say "well done". I like the PEP a lot and can see many places in my own code where it would make things more readable. I have one comment, and I may be too late or may be repeating other suggestions (no, I haven't poured through the wiki page yet). I actually don't think "with" reads as well and would propose "when". The word I'd really like to use here is "while" but given that we already have while loops and "as" is optional, I don't think that's a possibility. If you made "as" required, it would be though, I think. Look at the given examples: with locking(mylock): when locking(mylock): with opening("/etc/passwd") as f: when opening("/etc/passwd") as f: with opening(filename, "w") as f: with redirecting_stdout(f): print "Hello world" when opening(filename, "w") as f: when redirecting_stdout(f): print "Hello world" with signal_blocking(): when signal_blocking(): > - throw() is a term taken from Java & C++. We can't call the method > raise() -- but perhaps we can call it next_raising() or next_raise(), > which emphasizes the similarity with next(). Thoughts? I'm not strong > on this; I think throw() is fine too, especially since I expect that > it will be used explicitly extremely rarely -- the only customer is > the with_template decorator. As others have mentioned "throw()" is fine. > - Whether and how to keep a door open for a future extension to the > syntax that allows multiple resources to be acquired in a single > with-statement. Possible syntax could be > > (a) with EXPR1 [as VAR1], EXPR2 [as VAR2], EXPR3 [as VAR3], ...: > > or > > (b) with EXPR1, EXPR2, EXPR3, ... as VAR1, VAR2, VAR3, ...: I agree that (a) is better and feel +1 on keeping the door open to it. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050611/235c0e15/attachment-0001.pgp From ncoghlan at gmail.com Sun Jun 12 03:18:02 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Jun 2005 11:18:02 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: <42AAFBB6.2070802@gmail.com> Message-ID: <42AB8D4A.5020407@gmail.com> Guido van Rossum wrote: >>It also considers the possibility of using with statements in an RAII >>style by acquiring the resource in __init__ or __new__ rather than >>__enter__. > > > Python isn't C++; you shouldn't do that. > > >>This is a topic that got a brief mention during early discussion here, >>but doesn't seem to be covered in the current version of PEP 343. > > > I haven't been infected with the RAII meme, so I can't help you here. I expected your reaction to be along these lines, and at one point updated the Wiki to say basically this. Then I went back to PEP 343, and had another look at the examples. The one that sprang out at me was the generic 'closing()' template. Being able to say "ensure method X of object Y is called when the block exits" seems like a *really* useful feature for 'with' statements. However, when using this approach, the resource is always going to be acquired during evaluation of EXPR, since it needs to be passed an an argument to the 'closing()' (or equivalent) template. If the guarantee only applies to __enter__ though, then such templates would have to be written 'correctly' using deferred evaluation of the argument: class closing(object): def __init__(self, resource_factory) self.factory = resource_factory self.resource = None def __enter__(self): if self.resource is not None: raise RuntimeException("Cannot nest use of template") self.resource = self.factory() return self.resource def __exit__(self, *exc_info): self.resource.close() self.resource = None Then usage gets a bit uglier, because a lambda is needed in order to defer evaluation of the argument: with closing(lambda: open("somefile.txt", "r")) as f: # Do stuff If, however, two new opcodes were added that blocked and unblocked asynchronous exceptions in a given thread, then not only would it be possible to implement 'closing()' in the natural way, then resource handlers written in Python would 'do the right thing' to. Specifically, what if we added a new boolean flag 'permit_async_exc' to the thread state that is checked at the top of main loop in ceval.c where the asynchronous events are handled? The simplest trick would be to change the line "if (--_Py_Ticker < 0)" to "if ((--_Py_Ticker < 0) && (_Py_Ticker = 0, tstate->allow_async_exc))" (using the comma operator to keep _Py_Ticker from overflowing in pathological cases). Then all the two new opcodes (e.g. ALLOW_ASYNC, BLOCK_ASYNC) would have to do is set the state of tstate->allow_async_exc appropriately. Notice that _Py_Ticker keeps decrementing while the async_exceptions are blocked, so that any queued events should be handled immediately after execution of the BLOCK_ASYNC call (unless the next opcode is SETUP_FINALLY, in which case that opcode will be executed first). The setting of _Py_Ticker to zero, and the check of the tstate field only occur when _Py_Ticker goes negative, so they only add to the cost of executing the ceval loop when the check interval has expired. Compared to the cost of Py_MakePendingCalls, and the check for thread context switches, these costs should be negligible. An alternative implementation would still allow thread switches to happen, and prevent just the asynchronous events. Then the ALLOW_ASYNC opcode could just force _Py_Ticker to zero in order to ensure prompt evaluation of any pending events. Either way, the code generation for the with statement could simply emit BLOCK_ASYNC as the first opcode, then ALLOW_ASYNC as the opcode immediate preceding SETUP_FINALLY. Correct behaviour in the face of asynchronous events is then guaranteed, even for cases where the resource is acquired in EXPR, or when __enter__ is written in Python. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Sun Jun 12 03:26:07 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Jun 2005 11:26:07 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42AB8D4A.5020407@gmail.com> References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> Message-ID: <42AB8F2F.7060204@gmail.com> Nick Coghlan wrote: > Then all the two new opcodes (e.g. ALLOW_ASYNC, BLOCK_ASYNC) would > have to do is set the state of tstate->allow_async_exc appropriately. Actually, to allow the use of 'with' statements inside functions called via EXPR and the call to __enter__, it would be necessary to support nesting of calls to BLOCK_ASYNC and ALLOW_ASYNC, so a better approach would be to make the field "tstate->block_async_exc", incrementing it in BLOCK_ASYNC, and decrementing it in ALLOW_ASYNC. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Sun Jun 12 05:26:14 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Jun 2005 13:26:14 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42AB8D4A.5020407@gmail.com> References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> Message-ID: <42ABAB56.1050405@gmail.com> Nick Coghlan wrote: > Either way, the code generation for the with statement could simply > emit BLOCK_ASYNC as the first opcode, then ALLOW_ASYNC as the opcode > immediate preceding SETUP_FINALLY. Correct behaviour in the face of > asynchronous events is then guaranteed, even for cases where the > resource is acquired in EXPR, or when __enter__ is written in Python. Coming up out of the implementation weeds, this can be described as follows: """ Atomic resource acquisition The resource acquisition line that starts the with statement is executed atomically. That is, the Python interpreter prevents asynchronous events and automatic context switches while the resource is being acquired. Context switches due to the current thread becoming blocked (e.g. by attempting to acquire a synchronisation lock) will still occur, avoiding deadlock. Automatic context switches or asynchronous events that would have occurred during evaluation of the resource acquisition line are deferred so that they occur immediately prior to the execution of the contained suite. This ensures that the acquired resource is still properly released in the face of an asynchronous exception such as KeyboardInterrupt. To avoid blocking exceptions for an extended period, 'expensive' calculations can be carried out on the line preceding the with statement, and stored in a local variable for use in the resource acquisition: tmp = some_expensive_calculation() with tmp as rsrc: # Perform operations with resource Note that, in this case, the actual resource acquisition should occur in tmp.__enter__, in order to enjoy the benefits of the atomic resource acquisition. """ Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From greg at electricrain.com Sun Jun 12 05:47:23 2005 From: greg at electricrain.com (Gregory P. Smith) Date: Sat, 11 Jun 2005 20:47:23 -0700 Subject: [Python-Dev] hashlib - faster md5/sha, adds sha256/512 support Message-ID: <20050612034723.GE19337@zot.electricrain.com> I have finished up the hashlib work I started on in feb/march for patch 1121611 and 935454 after some discussion on this list. The full patch including tests and module documentation has been posted in the sf patch 1121611 as hashlib-008. I believe it is done and ready and would like to commit it after a review. Let the reviewing begin! For easy viewing, here's what the module documentation looks like: http://electricrain.com/greg/hashlib-py25-doc/module-hashlib.html The code is in the hashlib-008.patch file: http://redirx.com/?3e19 hashlib incorporates both the 1121611 openssl hash support and the 935454 sha256+sha512 module support into a single hashlib module that picks the fastest implementation of each algorithm to use. OpenSSLs implementations of md5 and sha1 are nearly 2x as fast as the existing python builtin versions. The (now legacy) md5 and sha modules are turned into simple stubs that use hashlib. The actual sourceforge patch tracker entries: https://sourceforge.net/tracker/?func=detail&aid=1121611&group_id=5470&atid=305470 https://sourceforge.net/tracker/?func=detail&aid=935454&group_id=5470&atid=305470 Greg From ncoghlan at gmail.com Sun Jun 12 06:10:04 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Jun 2005 14:10:04 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42AB8F2F.7060204@gmail.com> References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com> Message-ID: <42ABB59C.5050608@gmail.com> Nick Coghlan wrote: > Nick Coghlan wrote: > >>Then all the two new opcodes (e.g. ALLOW_ASYNC, BLOCK_ASYNC) would >>have to do is set the state of tstate->allow_async_exc appropriately. > > > Actually, to allow the use of 'with' statements inside functions > called via EXPR and the call to __enter__, it would be necessary to > support nesting of calls to BLOCK_ASYNC and ALLOW_ASYNC, so a better > approach would be to make the field "tstate->block_async_exc", > incrementing it in BLOCK_ASYNC, and decrementing it in ALLOW_ASYNC. And, replying to myself yet again. . . Such an approach would obviously break in the face of an exception that occurs between the BLOCK_ASYNC and ALLOW_ASYNC opcodes (async exceptions would remain blocked in the thread). This can be dealt with by storing the question of whether or not async exceptions are permitted as part of the *frame* state, rather than the thread state. That is, make the field access be "f->allow_async_events", and have ALLOW_ASYNC and BLOCK_ASYNC set or clear that flag, respectively. The error handling code after the main opcode switch statement can then simply set "f->allow_async_exc" back to true in order to ensure that handling of async exceptions is resumed correctly in the event of try/finally or try/except blocks causing further execution within the frame. (Given that only expressions are possible on the resource acquisition line, dealing with the WHY_EXPRESSION case seems to be all that would be needed). Regards, Nick. P.S. My suggested semantics and the above implementation outline should be up on the Wiki shortly. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Sun Jun 12 06:52:47 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 12 Jun 2005 14:52:47 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <1118526223.9425.127.camel@geddy.wooz.org> References: <1118526223.9425.127.camel@geddy.wooz.org> Message-ID: <42ABBF9F.1010801@gmail.com> Barry Warsaw wrote: > On Fri, 2005-06-10 at 16:23, Guido van Rossum wrote: > >>While there's still some activity in the Wiki, nothing (to me) sounds >>like signs of serious disagreement or truly better alternatives. So I >>think I'd like to move forward towards acceptance soon (before >>EuroPython). > > > Well, I finally read PEP 343 in its entirety. Having not really had the > time to follow this thread, I have to say "well done". I like the PEP a > lot and can see many places in my own code where it would make things > more readable. > > I have one comment, and I may be too late or may be repeating other > suggestions (no, I haven't poured through the wiki page yet). I > actually don't think "with" reads as well and would propose "when". The idea behind 'with' is that the block is executed while holding (i.e. 'with') the resource. I think the '-ing' form of the example templates is a holdover from the days when the PEP used the 'do' keyword - I find that past tense or noun forms tend to read better than present tense for custom built with templates named after the action that occurs on entry: # Past tense with locked(my_lock): with opened(my_file, mode): with redirected_stdout(my_stream): # Noun forms with my_lock: with open(my_file, mode) as f: with extra_precision(): with decimal.getcontext() as ctx: The place where the '-ing' form still makes some sense is when the template is named after the action that will occur at the *end* of the block: with closing(some_gen()) as g: with releasing(some_rsrc()) as r: Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From gvanrossum at gmail.com Sun Jun 12 18:56:13 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun, 12 Jun 2005 09:56:13 -0700 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42ABB59C.5050608@gmail.com> References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com> <42ABB59C.5050608@gmail.com> Message-ID: I can't make time for this right now, but the more I think about it, the more I believe the worry about asynchronous exceptions is mostly irrational. Despite your "proof" that it *can* be implemented I really don't want to see anything like that implemented -- I doubt that you've covered all the cases and I don't like inserting new opcodes to handle a case that most programs don't care about. Declaring something "atomic" that could span an arbitrary amount of Python code execution (including calls) just doesn't seem right. Disregarding asynchronous exceptions, you can already use with-statements for your beloved RAII pattern, right? I'm still skeptical how important it is in Python -- its seems something invented very specifically for C++'s guarantee of local destructor execution, and I don't expect to see all that many applications for it in Python. The with-statement is about the reduction of boiler-plate code involving try-finally, and that is typically not used for resource release (unless you count locks, where RAII is inappropriate -- at least in Python). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ducasse at iam.unibe.ch Sun Jun 12 18:19:12 2005 From: ducasse at iam.unibe.ch (ducasse) Date: Sun, 12 Jun 2005 18:19:12 +0200 Subject: [Python-Dev] [ANN] Have fun programming with your kids Message-ID: Hi If you are a parent, an educator or a programmer having kids this is for you! After 4 years of work, my new book "Squeak: Learn programming with Robots" will be out soon. http://smallwiki.unibe.ch/botsinc/ http://www.amazon.com/exec/obidos/tg/detail/-/1590594916/ qid=1117218524/sr=1-1/ref=sr_1_1/002-5642974-5143261?v=glance&s=books With Bots Inc you will learn how to program robots in an interactive environment. Bots Inc proposes three teaching approaches: direct command of robots, scripting robots and programming robots. The book contains 24 chapters going step by step over topics with a lot of examples. Bots Inc is fun but it is not a toy, it teaches you 100% real programming. Bots Inc is built on top of the rich open-source multimedia Squeak environment that you can also discover. My goal is to explain key elementary programming concepts (such as loops, abstraction, composition, and conditionals) to novices of all ages. I believe that learning by experimenting and solving problems with fun is central to human knowledge acquisition. Therefore, I have presented programming concepts through simple but not trivial problems such as drawing golden rectangles or simulating animal behavior. The ideal reader I have in mind is an individual who wants to have fun programming. This person may be a teenager or an adult, a schoolteacher, or somebody teaching programming to children in some other organization. Such an individual does not have to be fluent in programming in any language. As a father of two young boys I also wrote this book for all the parents that want to have fun programming with their kids in a powerful interactive environment. Programming in Squeak is an interactive, fun but deep experience. The present book teaches elementary programming aspect, the following book will introduce a new fun environment and teach object-oriented programming. Testimonies "I am using the version of the book on your web site to teach my oldest daughter Becca some programming. She absolutely loves it. We are doing the Bot graphics right. My other kids are showing interest as well. My Fall semester schedule leaves me with almost no time free but in the Spring I hope to bring Squeak and your book to our elementary school's "gifted" program." C. David Shaffer "I'm using the Bot Lab environment for three years and found it really valuable in teaching computer science concepts for a young audience (and even less young !). The bots commanded through balloon (as in comic strips) is a very nice introduction for young children, and when this aspect is well understood, you can use the Bot Workspace to teach the notion of script, a first step in programming languages. The Micro Browser allows children to add new behavior for their bots, and have fun with their creation. This three-layers tool - Balloon, Micro Workspace, Micro Browser - offers to the teacher a fun way to introduce gently the basis of object-oriented programming concepts. With Bots Inc, learning is playing ! ;-)" Samir Saidani - University of Caen - France "I recently started a course with 7th-graders (age about 13 years) with Stephane's book --- they love it. They all know about syntactic issues from maths --- in a way they know that an expression in a formal language must be well formed. So they easily grasp the fact such as "there must be a colon after the message-name if an argument follows". Of cause they don't really read the error-messages, they just see "there must be some error" and they remember the simple rules. Don't underestimate Smalltalk --- it's easy understandable because it has a simple and straight-forward design." Klaus Fuller - Germany Have fun... Stef http://www.iam.unibe.ch/~ducasse/ "if you knew today was your last day on earth, what would you do different? ... especially if, by doing something different, today might not be your last day on earth" Calvin&Hobbes From mwh at python.net Sun Jun 12 22:17:38 2005 From: mwh at python.net (Michael Hudson) Date: Sun, 12 Jun 2005 21:17:38 +0100 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42AAFBB6.2070802@gmail.com> (Nick Coghlan's message of "Sun, 12 Jun 2005 00:56:54 +1000") References: <42AAFBB6.2070802@gmail.com> Message-ID: <2m4qc3fhy5.fsf@starship.python.net> Nick Coghlan writes: > It also considers the possibility of using with statements in an RAII > style by acquiring the resource in __init__ or __new__ rather than > __enter__. While you can probably do this (after all, most of the time __new__/__init__ and __enter__ will be called in rapid succession) I really think the RAII meme doesn't apply to Python; you generally want a resource to be acquired for a period of definite extent, and objects in Python have indefinite extent, unlike C++ where automatic variables have definite extent (Common Lisp terminology here). I've gone on at length about this before: http://starship.python.net/crew/mwh/pep310/ Cheers, mwh -- Have you considered downgrading your arrogance to a reasonable level? -- Erik Naggum, comp.lang.lisp, to yet another C++-using troll From arigo at tunes.org Sun Jun 12 23:10:29 2005 From: arigo at tunes.org (Armin Rigo) Date: Sun, 12 Jun 2005 23:10:29 +0200 Subject: [Python-Dev] Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005 Message-ID: <20050612211029.GA26015@code1.codespeak.net> Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005 ====================================================== The next PyPy sprint is scheduled right after EuroPython 2005 in Gothenborg, Sweden. Its main focus is translation to lower level backends but there are also other possible topics. We'll give newcomer-friendly introductions. To learn more about the new PyPy Python-in-Python implementation look here: http://codespeak.net/pypy On a side note, there are a number of sub projects that may be interesting for participating in google's summer-of-code event (deadline June 14th!). The PyPy group is willing to mentor projects that have some link with PyPy, so if you are accepted in such a project, the sprint could also serve as a good meeting and kick-off point. Further down you'll find some examples, but there are certainly more and bigger ones :-) Goals and topics of the sprint ------------------------------ The main, though not the only, focus of the sprint will be on the "translation" aspect of PyPy. The goal here is to progress towards a completely translated PyPy. How much will already have been done before EuroPython is unknown; as a guess, we will be left with: - completing the "rtyper", the piece of code that assigns low-level C-like types to high-level RPython objects (lists, dicts, instances, etc.) and low-level control flow graphs to high-level ones; - polish off the GenC and GenLLVM back-ends, responsible for turning the low-level C-like flow graphs into real C or LLVM source code. See http://codespeak.net/pipermail/pypy-dev/2005q2/002136.html for more information (10th of June status). Non-translation-related topics are welcome too. Here are some suggestions from the issue tracker (https://codespeak.net/issue/pypy-dev/): - integrate the parser module, possibly making it RPython conformant; - rewrite in Python a C module you are familiar with (partial list of missing/incomplete modules: os, math, array, regular expressions, binascii...) - implement Python 2.3's import hook extensions (zip-imports etc.) - fix Windows-related issues, '%'-formatting rounding errors, add missing docstrings on app-level built-in types and functions, etc. - weakrefs (but this requires discussion and planning on pypy-dev before the sprint! feel free to start such a discussion, though.) Location & Accomodation ------------------------ The sprint will be held in the former Math Center building near the crossing of Gibraltargatan and Eklandagatan. Entrance is on the middle of the side facing Gibraltargatan. The doors to the building are normally locked, so you need the phone number of somebody inside to get in. Instructions on whom to call will be posted on the door. The sprint will be co-located with several other sprints. See the `EuroPython Wiki`_, to find out what other sprints will be running. Nearest, and probably cheapest is to book accomodation at SGS Veckobostäder through the Europython website. This option will be available until about 20 June. .. _`EuroPython special accomodation`: http://www.europython.org/sections/accomodation/special_accomodation .. _`EuroPython Wiki`: http://www.europython.org/sections/sprints_and_wiki Exact times ----------- The public Pypy sprint is held Friday 1st July - Thursday 7 July 2005. Hours will be from 09:00 until people have had enough. It's a good idea to arrive a day before the sprint starts. (There is a sprint for people who are familiar with the Pypy codebase before Europython as well. This will be held at Jacob & Laura's home on Götabergsgatan 22.) Network, Food, currency ------------------------ Sweden is not part of the Euro zone. One SEK (krona in singular, kronor in plural) is roughly 1/10th of a Euro (9.15 SEK to 1 Euro). There are some pizzerias, kebab places and the like close to the venue. Their food is edible and cheap, but not very good. For good food, you need to go downtown. You need a wireless network card to access the network. You will be issued a login to the Chalmers NOMAD network. This will allow you to use access points all over Chalmers. However, we can likely provide a wireless/ethernet bridge. Sweden uses the same kind of plugs as Germany. 230V AC. Registration etc.pp. -------------------- Please subscribe to the `PyPy sprint mailing list`_, introduce yourself and post a note that you want to come. Feel free to ask any questions there! .. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint -- Armin Rigo & the PyPy team From tjreedy at udel.edu Sun Jun 12 23:35:57 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 12 Jun 2005 17:35:57 -0400 Subject: [Python-Dev] [ANN] Have fun programming with your kids References: Message-ID: Dear Mr. Ducasse, The PyDev mailing list, bidirectionally gatewayed to gmane.comp.python.devel, which you apparently used as the insertion point, is a specialized list for discussion and development of the next Python version and occasionally beyond. Even most posts to comp.lang.python (== gmane.comp.python.general) are off-topic here. Your annoucement of a non-Python application book, which you sent numerous places, appears to be out of the ballpark of relevance here. Your book may be wonderful and I might find it useful were I to buy it, but if everyone imitated you, PyDev would have to be closed in order to function. Please desist, both for this and future projects. PS. Ignorance is no excuse. Internet etiquette specifies that before posting, one read a group (mailing list, forum) enough to learn what it is about. From skip at pobox.com Mon Jun 13 03:35:31 2005 From: skip at pobox.com (Skip Montanaro) Date: Sun, 12 Jun 2005 20:35:31 -0500 Subject: [Python-Dev] Multiple expression eval in compound if statement? Message-ID: <17068.58083.342369.872731@montanaro.dyndns.org> I'm horsing around with recognizing switch-like if statements like: if x == 1: print 1 elif x == 2: print 2 else: print "unknown" in the compiler and generating O(1) code. "x" can be any expression, but must be precisely the same in each elif clause, the comparison operator must be "==" and the RHS of the test must evaluate to a simple hashable constant (string, float, integer - still working on None, constant tuples may be allowed later). I can currently recognize such constructs and am working on code generation. This would represent a semantic change to the Python runtime, because x would only be evaluated once, whereas in existing usage it can potentially be evaluated many times. If evaluating x has side-effects, code like the above that currently works would break. In reading the language reference manual, it says: It selects exactly one of the suites by evaluating the expressions one by one until one is found to be true (see section 5.10 for the definition of true and false); then that suite is executed (and no other part of the if statement is executed or evaluated). If all expressions are false, the suite of the else clause, if present, is executed. It says nothing about possibly caching values, which is what I'm doing effectively, but it seems pretty clear that each full test expression will be evaluated until one evaluates to true. I can't imagine anyone relying on a side-effect when evaluating x in this context, and if someone did, their code would be truly fragile. Still, any thought as to whether or not such a semantic change to the language might be acceptable? Skip From greg.ewing at canterbury.ac.nz Mon Jun 13 03:55:22 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 13 Jun 2005 13:55:22 +1200 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: Message-ID: <42ACE78A.1040909@canterbury.ac.nz> Guido van Rossum wrote: > So (a) would have my preference. Mine, too. > the PEP would have to be amended to state that > VAR must be a single variable or a list of variables IN PARENTHESES. +1 -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From ncoghlan at gmail.com Mon Jun 13 03:58:18 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Jun 2005 11:58:18 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com> <42ABB59C.5050608@gmail.com> Message-ID: <42ACE839.1070804@gmail.com> Guido van Rossum wrote: > I can't make time for this right now, but the more I think about it, > the more I believe the worry about asynchronous exceptions is mostly > irrational. Despite your "proof" that it *can* be implemented I really > don't want to see anything like that implemented -- I doubt that > you've covered all the cases and I don't like inserting new opcodes to > handle a case that most programs don't care about. Declaring something > "atomic" that could span an arbitrary amount of Python code execution > (including calls) just doesn't seem right. As I just added to the Wiki, dropping the issue completely works for me. I'm also far from convinced I'd covered all the corner cases (besides, blocking KeyboardInterrupt for an arbitrary amount of time made me uncomfortable). > Disregarding asynchronous exceptions, you can already use > with-statements for your beloved RAII pattern, right? I'm still > skeptical how important it is in Python -- its seems something > invented very specifically for C++'s guarantee of local destructor > execution, and I don't expect to see all that many applications for it > in Python. You'd mostly convinced me that RAII was rare in Python, but two possible use cases remained - files, and "call method X of object Y at the end of the block" (e.g. "closing()" from the PEP). Both of those would require any async guarantee to cover evaluation of EXPR in order to be included in the guarantee. If with statements make no guarantees about async exceptions, then there is no problem (as there is officially no difference between the two styles). If we ever do resurrect the idea, I also realised a far simpler solution would involve moving evaluation of EXPR and the call to __enter__ inside the try/finally block, and having an internal flag to indicate whether or not __exit__ should be called as the block is exited. Far less screwing around with ceval.c, and far easier to get right. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From raymond.hettinger at verizon.net Mon Jun 13 03:57:22 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 12 Jun 2005 21:57:22 -0400 Subject: [Python-Dev] Wishlist: dowhile Message-ID: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> More than case-statement semantics or PEP343, I wish for a dowhile statement. The most straight-forward way is to put the conditional expression at the beginning of the block with the understanding that a dowhile keyword will evaluate the condition only after the block runs: dowhile : being the equivalent of; _firstpass = True while _firstpass or : _firstpass = False Raymond From ncoghlan at gmail.com Mon Jun 13 04:14:37 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Jun 2005 12:14:37 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <2m4qc3fhy5.fsf@starship.python.net> References: <42AAFBB6.2070802@gmail.com> <2m4qc3fhy5.fsf@starship.python.net> Message-ID: <42ACEC0D.20508@gmail.com> Michael Hudson wrote: > While you can probably do this (after all, most of the time > __new__/__init__ and __enter__ will be called in rapid succession) I > really think the RAII meme doesn't apply to Python; you generally want > a resource to be acquired for a period of definite extent, and objects > in Python have indefinite extent, unlike C++ where automatic variables > have definite extent (Common Lisp terminology here). Using the RAII term was probably a bad idea, as it brings in too many extra connotations (like the converse 'RRID' - 'resource release is destruction'). Anyway, Guido's taking the option of not providing any explicit guarantees at this point, which is probably the most sensible idea. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From python at rcn.com Mon Jun 13 04:20:31 2005 From: python at rcn.com (Raymond Hettinger) Date: Sun, 12 Jun 2005 22:20:31 -0400 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <17068.58083.342369.872731@montanaro.dyndns.org> Message-ID: <000501c56fbe$79998220$4ff9a244@oemcomputer> > I'm horsing around with recognizing switch-like if statements like: > > if x == 1: > print 1 > elif x == 2: > print 2 > else: > print "unknown" > > in the compiler and generating O(1) code. "x" can be any expression, but > must be precisely the same in each elif clause, the comparison operator > must > be "==" and the RHS of the test must evaluate to a simple hashable > constant > (string, float, integer - still working on None, constant tuples may be > allowed later). I can currently recognize such constructs and am working > on > code generation. I think it unwise to allow x to be any expression. Besides altering existing semantics, it leads to code redundancy and to a fragile construct (where the slightest alteration of any of the expressions triggers a silent reversion to O(n) behavior). Raymond From ncoghlan at gmail.com Mon Jun 13 04:39:11 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Jun 2005 12:39:11 +1000 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> Message-ID: <42ACF1CF.5000203@gmail.com> Raymond Hettinger wrote: > More than case-statement semantics or PEP343, I wish for a dowhile > statement. > > The most straight-forward way is to put the conditional expression at > the beginning of the block with the understanding that a dowhile keyword > will evaluate the condition only after the block runs Out-of-order code execution rarely counts as 'straightforward' ;) With PEP 315, a do-while loop would look like: do: while : pass But then, I'm reasonably happy with the 'break out of an infinite loop' approach, so *shrug*. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From gvanrossum at gmail.com Mon Jun 13 04:55:13 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun, 12 Jun 2005 19:55:13 -0700 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42ACE839.1070804@gmail.com> References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com> <42ABB59C.5050608@gmail.com> <42ACE839.1070804@gmail.com> Message-ID: [Nick] > As I just added to the Wiki, dropping the issue completely works for > me. I'm also far from convinced I'd covered all the corner cases > (besides, blocking KeyboardInterrupt for an arbitrary amount of time > made me uncomfortable). OK. > You'd mostly convinced me that RAII was rare in Python, but two > possible use cases remained - files, and "call method X of object Y at > the end of the block" (e.g. "closing()" from the PEP). > > Both of those would require any async guarantee to cover evaluation of > EXPR in order to be included in the guarantee. If with statements make > no guarantees about async exceptions, then there is no problem (as > there is officially no difference between the two styles). Any exception, async or otherwise, that happens before the 'try' of the translation is reached, does *not* cause __exit__ to be called. I just realized that the assignment to VAR (if present) also has to be represented by one or more opcodes, so a special guarantee that __enter__() called by the same opcode that sets up the try cannot work anyway. Forget I said anything about that. > If we ever do resurrect the idea, I also realised a far simpler > solution would involve moving evaluation of EXPR and the call to > __enter__ inside the try/finally block, and having an internal flag to > indicate whether or not __exit__ should be called as the block is > exited. Far less screwing around with ceval.c, and far easier to get > right. We considered this at some point during the PEP 340/343/346 discussion (in fact I had it this way in an early version of one of the PEPs), and in the end decided against it. I believe the argument was that any failure *before* __enter__() returns can be handled by try/except clauses inside __init__() or __enter__(). Changing to this style later would be a major backward incompatibility because the guarantees made to __exit__() are different. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Mon Jun 13 05:05:59 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun, 12 Jun 2005 20:05:59 -0700 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <17068.58083.342369.872731@montanaro.dyndns.org> References: <17068.58083.342369.872731@montanaro.dyndns.org> Message-ID: On 6/12/05, Skip Montanaro wrote: > > I'm horsing around with recognizing switch-like if statements like: > > if x == 1: > print 1 > elif x == 2: > print 2 > else: > print "unknown" > > in the compiler and generating O(1) code. "x" can be any expression, but > must be precisely the same in each elif clause, the comparison operator must > be "==" and the RHS of the test must evaluate to a simple hashable constant > (string, float, integer - still working on None, constant tuples may be > allowed later). I can currently recognize such constructs and am working on > code generation. Cool! > This would represent a semantic change to the Python runtime, because x > would only be evaluated once, whereas in existing usage it can potentially > be evaluated many times. If evaluating x has side-effects, code like the > above that currently works would break. > > In reading the language reference manual, it says: > > It selects exactly one of the suites by evaluating the expressions one > by one until one is found to be true (see section 5.10 for the > definition of true and false); then that suite is executed (and no other > part of the if statement is executed or evaluated). If all expressions > are false, the suite of the else clause, if present, is executed. > > It says nothing about possibly caching values, which is what I'm doing > effectively, but it seems pretty clear that each full test expression will > be evaluated until one evaluates to true. > > I can't imagine anyone relying on a side-effect when evaluating x in this > context, and if someone did, their code would be truly fragile. Still, any > thought as to whether or not such a semantic change to the language might be > acceptable? I think it would be wiser if you only did this when x is a simple local variable. For locals, we *know* that no side effect of any expression can change the value. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From foom at fuhm.net Mon Jun 13 05:42:45 2005 From: foom at fuhm.net (James Y Knight) Date: Sun, 12 Jun 2005 23:42:45 -0400 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <000501c56fbe$79998220$4ff9a244@oemcomputer> References: <000501c56fbe$79998220$4ff9a244@oemcomputer> Message-ID: <23D1E8FC-A5DB-4CD4-9EB2-D5B637B1C552@fuhm.net> On Jun 12, 2005, at 10:20 PM, Raymond Hettinger wrote: > I think it unwise to allow x to be any expression. Besides altering > existing semantics, it leads to code redundancy and to a fragile > construct (where the slightest alteration of any of the expressions > triggers a silent reversion to O(n) behavior). Yes, changing the semantics of the if statement would be pretty bad. Unless evaluating x provably has no side effects (e.g. accessing a local variable), I would say that the duplicate evaluations must not be eliminated. >> if x == 1: >> print 1 >> elif x == 2: >> print 2 >> else: >> print "unknown" Unfortunately, I'm afraid this will not be as useful as it at first appears, because python has no concept of a constant variable binding. I hardly ever see an if/else chain being done with hardcoded integers. Rather, more often a variable is compared to global "constants", which could not be optimized away. However, this construction does appear reasonably often with constant strings on the RHS, so there may actually be a real-world gain to be had. With a Sufficiently Smart Compiler, of course, you could make assumptions at runtime about values not changing, and recompile code depending upon that assumption if necessary, but that's more in the realm of a new runtime than modifications to CPython. Maybe PyPy will eventually provide the ability to do that kind of interesting optimization. James From gvanrossum at gmail.com Mon Jun 13 06:01:42 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun, 12 Jun 2005 21:01:42 -0700 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <42ACF1CF.5000203@gmail.com> References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> Message-ID: On 6/12/05, Nick Coghlan wrote: > Raymond Hettinger wrote: > > More than case-statement semantics or PEP343, I wish for a dowhile > > statement. > > > > The most straight-forward way is to put the conditional expression at > > the beginning of the block with the understanding that a dowhile keyword > > will evaluate the condition only after the block runs > > Out-of-order code execution rarely counts as 'straightforward' ;) Right. Millions of years of programming language design have shown us how to write a loop with the condition tested at the end -- the condition is written after the loop body. > With PEP 315, a do-while loop would look like: > > do: > > while : > pass > > But then, I'm reasonably happy with the 'break out of an infinite > loop' approach, so *shrug*. Amen. If we have to do this, PEP 315 has my +0. It is Pythonically minimal and the motivation rings true: I've often written code like this in the past: line = f.readline() while line: line = f.readline() But these days we do that using "for line in f". I wonder if other similar use cases can't be rewritten using better iterators? Maybe Raymond can show us some motivating use cases. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Mon Jun 13 06:37:30 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 12 Jun 2005 21:37:30 -0700 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <42ACF1CF.5000203@gmail.com> References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> Message-ID: <20050612212924.7236.JCARLSON@uci.edu> Nick Coghlan wrote: > > Raymond Hettinger wrote: > > More than case-statement semantics or PEP343, I wish for a dowhile > > statement. > > > > The most straight-forward way is to put the conditional expression at > > the beginning of the block with the understanding that a dowhile keyword > > will evaluate the condition only after the block runs > > Out-of-order code execution rarely counts as 'straightforward' ;) > > With PEP 315, a do-while loop would look like: > > do: > > while : > pass > > But then, I'm reasonably happy with the 'break out of an infinite > loop' approach, so *shrug*. PEP 315 offers the creation/use of an additional keyword. It also happens to push the condition to the end of the loop body. I'm not sure that pushing condition evaluation to the end of the loop body (with a trailing 'pass' as provided) is necessarily more readable than Raymond's offered dowhile. In fact, if I remember the discussion over decorators correctly, putting an operation later actually reduces readability, though at least in this case, the bare "do:" would signal to the user "Hey, I have a condition later!" Though we again run into, how is that any better than having the condition at the front to begin with, especially if a new keyword is necessary regardless of the resulting syntax? - Josiah From pje at telecommunity.com Mon Jun 13 07:25:51 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 13 Jun 2005 01:25:51 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: References: <42ACF1CF.5000203@gmail.com> <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> Message-ID: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> At 09:01 PM 6/12/2005 -0700, Guido van Rossum wrote: >If we have to do this, PEP 315 has my +0. > >It is Pythonically minimal and the motivation rings true: I've often >written code like this in the past: > > line = f.readline() > while line: > > line = f.readline() > >But these days we do that using "for line in f". Block-copying a file with read() has the same pattern (e.g. in shutil), but you can't make it a for loop. By the way, whatever happened to "and while"? i.e.: while True: data = inp.read(blocksize) and while data: out.write(data) It seemed to me this variation of the syntax had some traction at one point. Anyway, most of the times that I'm annoyed by the current while loop's limitations are due to the need to write things like the above as: while True: data = inp.read(blocksize) if not data: break out.write(data) Which makes it a bit harder to see the control flow. Or more precisely, something like the 'do/while' or 'while/and while' would make it *easier* to see the control flow, since really while isn't *bad*. I seem to recall that Knuth had something to say about the ideal loop being one that allowed you to execute code both before and after the exit condition, and that many languages allow you to have code before, or code after, but few allow you to do both in the same loop, forcing you to either duplicate some code or restructure your approach in order to fit the limitation of the language. From python at rcn.com Mon Jun 13 08:43:29 2005 From: python at rcn.com (Raymond Hettinger) Date: Mon, 13 Jun 2005 02:43:29 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> Message-ID: <000201c56fe3$535b9f60$5ab79d8d@oemcomputer> > By the way, whatever happened to "and while"? That is making something hard and weird out of something simple. There were no shortage of odd suggestions to force the condition line to appear lower in the block than the starting line. All of them smelled of rotten eggs -- they just don't fit the Python mold. In contrast, the dowhile solution only takes about 30 seconds to get used to. Maybe that idea should follow the path of the genexp PEP, get rejected now, and then get resurrected as a bright idea two years from now. Or maybe not. Raymond From bjourne at gmail.com Mon Jun 13 11:14:24 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 13 Jun 2005 11:14:24 +0200 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <000201c56fe3$535b9f60$5ab79d8d@oemcomputer> References: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> <000201c56fe3$535b9f60$5ab79d8d@oemcomputer> Message-ID: <740c3aec050613021421c8a76@mail.gmail.com> > In contrast, the dowhile solution only takes about 30 seconds to get > used to. Maybe that idea should follow the path of the genexp PEP, get > rejected now, and then get resurrected as a bright idea two years from > now. Or maybe not. dowhile foo != 42: <10 lines of body> foo = random.randint(1, 50) Now it becomes annoying because the name is visible long before it is bound to something. I would like to have do-while's like this: do: until But I'm sure that has problems too. -- mvh Bj?rn From python at rcn.com Mon Jun 13 11:55:14 2005 From: python at rcn.com (Raymond Hettinger) Date: Mon, 13 Jun 2005 05:55:14 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <740c3aec050613021421c8a76@mail.gmail.com> Message-ID: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> [BJ?rn Lindqvist] > I would like to have do-while's like this: > > do: > > until > > But I'm sure that has problems too. That looks nice to me. Raymond From dmitry at targeted.org Mon Jun 13 12:43:36 2005 From: dmitry at targeted.org (Dmitry Dvoinikov) Date: Mon, 13 Jun 2005 16:43:36 +0600 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> References: <740c3aec050613021421c8a76@mail.gmail.com> <000901c56ffe$369f19e0$b8b3958d@oemcomputer> Message-ID: <2316076.20050613164336@targeted.org> >> do: >> >> until > That looks nice to me. Except this puts loop entry statement (do) and loop condition test (until) on different levels of indentation, which is I don't quite like. Compared to the existing loop statements, ------------------- while Cond: body else: exit ------------------- for Iter: body else: exit ------------------- This would seem more logical: ------------------- do: body while Cond: else: exit ------------------- do: body until Cond: else: exit ------------------- Although the trailing colon is still a bit unpleasant: ------------------- do: body while Cond: # no else here ------------------- Sincerely, Dmitry Dvoinikov http://www.targeted.org/ --- Original message follows --- > [BJ?rn Lindqvist] >> I would like to have do-while's like this: >> >> do: >> >> until >> >> But I'm sure that has problems too. > That looks nice to me. > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/dmitry%40targeted.org From skip at pobox.com Mon Jun 13 13:14:38 2005 From: skip at pobox.com (Skip Montanaro) Date: Mon, 13 Jun 2005 06:14:38 -0500 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <000501c56fbe$79998220$4ff9a244@oemcomputer> References: <17068.58083.342369.872731@montanaro.dyndns.org> <000501c56fbe$79998220$4ff9a244@oemcomputer> Message-ID: <17069.27294.581932.647705@montanaro.dyndns.org> Raymond> I think it unwise to allow x to be any expression. How do you decide what's "too complex"? Even an apparently simple variable can have side effects, so the semantic change bit is important no matter how complex the expression. (Consider the builtin help object. Type it at the prompt with no parens.) Like I said, "I'm horsing around with...". If this never makes it into Python I can accept that. The interesting thing for me at this point is the code generation anyway. I figure if Guido reconsiders PEP 275 (I think it covers all the bases), then the code generation stuff will probably be of some use. Skip From mcherm at mcherm.com Mon Jun 13 13:48:31 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 13 Jun 2005 04:48:31 -0700 Subject: [Python-Dev] Multiple expression eval in compound if statement? Message-ID: <20050613044831.kec9rksum2ec80o0@login.werra.lunarpages.com> Skip writes: > Even an apparently simple variable can have side effects [...] Consider > the builtin help object. Type it at the prompt with no parens. I tried. As far as I can tell, the result (in the interactive interpreter) is the same as with any other object except None: the _ variable gets set, the interpreter prints out the __str__ of the object, sets the _ variable, and prints a new '>>>' prompt. It happens that the __str__ of the help object is: 'Type help() for interactive help, or help(object for help about object.' ... but if there's some side effect going on here, I don't see it. What am I missing? -- Michael Chermside From mcherm at mcherm.com Mon Jun 13 14:07:32 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 13 Jun 2005 05:07:32 -0700 Subject: [Python-Dev] Multiple expression eval in compound if statement? Message-ID: <20050613050732.gw9sc59tj6dc0804@login.werra.lunarpages.com> A few thought's on Skip's idea here: Skip: > I can't imagine anyone relying on a side-effect when evaluating x in this > context, and if someone did, their code would be truly fragile. Yeah... people like that really annoy me. If people just wouldn't write code that depends on the FULL dynamics possible, then it'd be a lot easier for the VM to optimize. Unfortunately, we have to support such people. Raymond: I think it unwise to allow x to be any expression. Besides altering existing semantics, it leads to code redundancy and to a fragile construct (where the slightest alteration of any of the expressions triggers a silent reversion to O(n) behavior). Skip's proposal isn't the problem here. The syntax of the if-elif-else statement requires that when it is used in lieu of C's switch() statement that the switch() argument be repeated in each elif clause. Repeating it DOES violate once-and-only-once, but we've decided that that's an acceptable price to pay to avoid needing an extra kind of flow control statement for which there wasn't a really elegent syntax anyhow. A reasonably tradeoff, I agree, but the cost is that we DO violate once-and-only-once. The REAL danger from the "slightest alteration of any of the expressions" is not that it will alter from O(1) behavior to O(n) behavior, the REAL danger is that you may have MEANT it to be the same everywhere and the alteration is a typo and a bug! Guido: > I think it would be wiser if you only did this when x is a simple > local variable. I agree. I think of Skip's proposal as just an optimization, of the sort that the Python interpreter is allowed to make when there's no possible semantic effect. It would then restrict the use to situations where the 'switch() argument' was a local variable and the 'case values' were string literals, integer literals, float literals (but that shouldn't count because it's a really bad idea anyhow), None, and perhaps (I wouldn't bother, myself) tuples of the above. It sounds pretty restricted, which raises the question of whether it's worth it given the harsh restrictions on when the optimization would kick in. I write such things with local variables and literal strings quite often. On the other hand, I don't usually have more than 4-10 values though, so O(1) and O(n) may not be THAT different. It's one of those cases where the only thing I'd really believe was experiments done on real code. But it's a cool optimization if it actually pays off. -- Michael Chermside From ncoghlan at gmail.com Mon Jun 13 14:07:38 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Jun 2005 22:07:38 +1000 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> Message-ID: <42AD770A.1040001@gmail.com> Raymond Hettinger wrote: > [BJ?rn Lindqvist] > >>I would like to have do-while's like this: >> >>do: >> >> until >> >>But I'm sure that has problems too. > > > That looks nice to me. And this could easily be extended to allow code both before and after the 'until', giving a fully general loop: do: until else: In fact, this would simply be giving "looks like executable pseudocode" syntactic sugar for the current 'do-until' workarounds: while 1: if : break And: _exit = False: while not _exit: _exit = else: The 'until' is less hidden than the 'yield' that turns a function into a generator, and its presence is obviously signalled by the preceding 'do'. Its also less hidden than the 'if'/'break' construct in the infinite loop workaround, and about as hidden as the exit flag in the other style of workaround (although the 'until' can be more easily picked out by a syntax highlighter). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From exarkun at divmod.com Mon Jun 13 14:14:23 2005 From: exarkun at divmod.com (Jp Calderone) Date: Mon, 13 Jun 2005 08:14:23 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> Message-ID: <20050613121423.5047.539927633.divmod.quotient.2056@ohm> On Mon, 13 Jun 2005 01:25:51 -0400, "Phillip J. Eby" wrote: >At 09:01 PM 6/12/2005 -0700, Guido van Rossum wrote: >>If we have to do this, PEP 315 has my +0. >> >>It is Pythonically minimal and the motivation rings true: I've often >>written code like this in the past: >> >> line = f.readline() >> while line: >> >> line = f.readline() >> >>But these days we do that using "for line in f". > >Block-copying a file with read() has the same pattern (e.g. in shutil), but >you can't make it a for loop. Anything can be a for loop. for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): f2.write(chunk) Jp From ncoghlan at gmail.com Mon Jun 13 14:40:05 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 13 Jun 2005 22:40:05 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com> <42ABB59C.5050608@gmail.com> <42ACE839.1070804@gmail.com> Message-ID: <42AD7EA5.2070805@gmail.com> Guido van Rossum wrote: > We considered this at some point during the PEP 340/343/346 discussion > (in fact I had it this way in an early version of one of the PEPs), > and in the end decided against it. I believe the argument was that any > failure *before* __enter__() returns can be handled by try/except > clauses inside __init__() or __enter__(). Hmm, you're right. Also, given implementation of PEP 343, code which genuinely has to care about this can do so manually: with async_exceptions_blocked(): with critical_resource(): with async_exceptions_unblocked(): # Do the real work This ensures acquisition of the critical resource is not interrupted. Even better, the above can be wrapped up in a template and still give the desired guarantees: @stmt_template def safe_acquisition(resource): with async_exceptions_blocked(): with resource(): with async_exceptions_unblocked(): yield None with safe_acquisition(critical_resource): # Do the real work Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From skip at pobox.com Mon Jun 13 16:05:50 2005 From: skip at pobox.com (Skip Montanaro) Date: Mon, 13 Jun 2005 09:05:50 -0500 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <20050613044831.kec9rksum2ec80o0@login.werra.lunarpages.com> References: <20050613044831.kec9rksum2ec80o0@login.werra.lunarpages.com> Message-ID: <17069.37566.869168.203775@montanaro.dyndns.org> Michael> ... but if there's some side effect going on here, I don't see Michael> it. What am I missing? Mea culpa. I was thinking of the print as a side efefct. Obviously mistaken. Skip From anthony at interlink.com.au Mon Jun 13 16:16:00 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue, 14 Jun 2005 00:16:00 +1000 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <17069.27294.581932.647705@montanaro.dyndns.org> References: <17068.58083.342369.872731@montanaro.dyndns.org> <000501c56fbe$79998220$4ff9a244@oemcomputer> <17069.27294.581932.647705@montanaro.dyndns.org> Message-ID: <200506140016.02598.anthony@interlink.com.au> On Monday 13 June 2005 21:14, Skip Montanaro wrote: > Raymond> I think it unwise to allow x to be any expression. > > How do you decide what's "too complex"? Even an apparently simple variable > can have side effects, so the semantic change bit is important no matter > how complex the expression. (Consider the builtin help object. Type it at > the prompt with no parens.) Note also that saying 'just a simple local variable' doesn't cut it, either, because you will break on something like: class stupid: v = 0 def __eq__(self, other): self.v += 1 return self.v == other Really, you'd have to make sure you didn't optimise any LHS that defined a comparision operator (I _think_ that covers all the cases). Anthony -- Anthony Baxter It's never too late to have a happy childhood. From gvanrossum at gmail.com Mon Jun 13 16:41:45 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon, 13 Jun 2005 07:41:45 -0700 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <42AD770A.1040001@gmail.com> References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> Message-ID: > > [BJ?rn Lindqvist] > > > >>I would like to have do-while's like this: > >> > >>do: > >> > >> until > >> > >>But I'm sure that has problems too. > > [Raymond Hettinger] > > That looks nice to me. [Nick Coghlan] > And this could easily be extended to allow code both before and after > the 'until', giving a fully general loop: > > do: > > until > > else: > Which is exactly like PEP 315 except there 'until' must be spelled 'while not' and the while is properly indented. (I'm still not sure whether BJ?rn *meant* the 'until' to be indented or whether he simply made a mistake; his proposal resembles a Pythonic version of Pascal's repeat-until, which would have an unindented until-clause.) > The 'until' is less hidden than the 'yield' that turns a function into > a generator, and its presence is obviously signalled by the preceding > 'do'. Its also less hidden than the 'if'/'break' construct in the > infinite loop workaround, and about as hidden as the exit flag in the > other style of workaround (although the 'until' can be more easily > picked out by a syntax highlighter). Why are you so excited about having until indented? You didn't give any examples with multiple occurrences. A single occurrence works just fine unindented, as PEP 315 has already shown. The indented until sounds like unnecessary syntactic sugar for 'if X: break' -- not very Pythonic. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bjourne at gmail.com Mon Jun 13 17:53:52 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Mon, 13 Jun 2005 17:53:52 +0200 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> Message-ID: <740c3aec05061308536bceeead@mail.gmail.com> > > >>do: > > >> > > >> until > > >> > > >>But I'm sure that has problems too. > > > > [Raymond Hettinger] > > > That looks nice to me. > > [Nick Coghlan] > > And this could easily be extended to allow code both before and after > > the 'until', giving a fully general loop: > > > > do: > > > > until > > > > else: > > > > Which is exactly like PEP 315 except there 'until' must be spelled > 'while not' and the while is properly indented. > > (I'm still not sure whether BJ?rn *meant* the 'until' to be indented > or whether he simply made a mistake; his proposal resembles a Pythonic > version of Pascal's repeat-until, which would have an unindented > until-clause.) Sorry, I should have made that clear. I *meant* for 'until' to be indented. I envisonaged it to be treatened similar to how the raise, pass and return statements work. Those statements are indented in a suite and they always end the current block. Well, technically you can write stuff like this: suite: raise Exception But 'block2' will always be dead code. Emacs, for example, always dedents when you write any of those three statements. The reason I suggested was not so that you could extend it to fully general loops like Nick suggested (which I can't comment on because I never use else: in looping constructs and always keep the condition check and the top or bottom), but because I thought that a dedented 'until' would be very hard to parse and in general look very unpythonic: do: until Written like this it is not very obvious that the 'unil' is part of the do-until suite. I also imagine it to be difficult to parse and it breaks the rule that suites end when there is a dedentation. So, IMHO using an indented 'until' is the least evil of a number of evils. > Why are you so excited about having until indented? You didn't give > any examples with multiple occurrences. A single occurrence works just > fine unindented, as PEP 315 has already shown. > > The indented until sounds like unnecessary syntactic sugar for 'if X: > break' -- not very Pythonic. Yes, but grepping the stdlib produces over 300 hits for "while 1:" and "while True:" combined. Some of those a "if : break" in the middle and some would be better written as generators, but lots of them would be rewritten as do-while's. So I think there is more than enough use cases for syntactic sugar for do-while loops. -- mvh Bj?rn From steve at holdenweb.com Mon Jun 13 18:17:08 2005 From: steve at holdenweb.com (Steve Holden) Date: Mon, 13 Jun 2005 12:17:08 -0400 Subject: [Python-Dev] Summer of Code: Developing complete SSL support for Python In-Reply-To: <1117973125.8137.6.camel@emperor> References: <20050604193203.03D9.FCANO@ono.com> <42A1F267.6000504@hathawaymix.org> <1117973125.8137.6.camel@emperor> Message-ID: <42ADB184.7070700@holdenweb.com> Gustavo J. A. M. Carneiro wrote: [...] > > 4. In the socket module documentation: > > > ssl( > sock[, keyfile, certfile]) > Initiate a SSL connection over the socket sock. keyfile is the > name of a PEM formatted file that contains your private key. > certfile is a PEM formatted certificate chain file. On success, > a new SSLObject is returned. > > Warning: This does not do any certificate verification! > > I would make it a top priority to enable certificate verification in > ssl sockets. I don't see the point in doing SSL without certificate > verification. It's just false security. Maybe adding a callback asking > the application what to do if certificate validation fails, so that > application writers can show a GUI dialogue or something like that... > > Best regards. > I believe that SSL sockets without certificate verification will still retain the advantages of encryption ind integrity checking, though you are right to say that *authentication* is lost without certificate checking: the certificate is essentially the CA's assertion that they have applied the process described in their Certification Practices Statement to identify the subject. Then you must consider which CA's will be acceptable certificate issuers, and build trust for their certificates into the system in some modifiable way - we need to be able to add CA's - by the incorporation of the CAs' self-signed certificates, as in the browsers. So almost certainly if the platform has a certificate repository it might be good to offer an interface to that, as well as offering a private certificate repository. regards Steve -- Steve Holden +1 703 861 4237 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/ From pje at telecommunity.com Mon Jun 13 18:29:17 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 13 Jun 2005 12:29:17 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <20050613121423.5047.539927633.divmod.quotient.2056@ohm> References: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050613122620.0339f598@mail.telecommunity.com> At 08:14 AM 6/13/2005 -0400, Jp Calderone wrote: >On Mon, 13 Jun 2005 01:25:51 -0400, "Phillip J. Eby" > wrote: > >Block-copying a file with read() has the same pattern (e.g. in shutil), but > >you can't make it a for loop. > >Anything can be a for loop. > > for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): > f2.write(chunk) Showoff. ;) More seriously, I think your translation makes an excellent argument in *favor* of having a do/while statement for greater clarity. :) From pje at telecommunity.com Mon Jun 13 18:40:02 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 13 Jun 2005 12:40:02 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42AD7EA5.2070805@gmail.com> References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com> <42ABB59C.5050608@gmail.com> <42ACE839.1070804@gmail.com> Message-ID: <5.1.1.6.0.20050613123758.0339dc00@mail.telecommunity.com> At 10:40 PM 6/13/2005 +1000, Nick Coghlan wrote: >Hmm, you're right. Also, given implementation of PEP 343, code which >genuinely has to care about this can do so manually: > > with async_exceptions_blocked(): > with critical_resource(): > with async_exceptions_unblocked(): > # Do the real work > >This ensures acquisition of the critical resource is not interrupted. >Even better, the above can be wrapped up in a template and still give >the desired guarantees: > > @stmt_template > def safe_acquisition(resource): > with async_exceptions_blocked(): > with resource(): > with async_exceptions_unblocked(): > yield None > > > with safe_acquisition(critical_resource): > # Do the real work Ow. My head hurts. :) Seriously, though, wouldn't it make more sense to put the 'with async_exceptions_blocked()' in the __init__ or __enter__ of 'critical_resource'? Or am I still missing something? From gvanrossum at gmail.com Mon Jun 13 18:36:07 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon, 13 Jun 2005 09:36:07 -0700 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <740c3aec05061308536bceeead@mail.gmail.com> References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> <740c3aec05061308536bceeead@mail.gmail.com> Message-ID: On 6/13/05, BJ?rn Lindqvist wrote: > do: > > until > > Written like this it is not very obvious that the 'unil' is part of > the do-until suite. I also imagine it to be difficult to parse and it > breaks the rule that suites end when there is a dedentation. So, IMHO > using an indented 'until' is the least evil of a number of evils. Not difficult to parse at all, nor un-Pythonic. Multi-part blocks abound in Python: if / elif / else, try / finally, etc. > > Why are you so excited about having until indented? You didn't give > > any examples with multiple occurrences. A single occurrence works just > > fine unindented, as PEP 315 has already shown. > > > > The indented until sounds like unnecessary syntactic sugar for 'if X: > > break' -- not very Pythonic. > > Yes, but grepping the stdlib produces over 300 hits for "while 1:" and > "while True:" combined. Some of those a "if : break" in the > middle and some would be better written as generators, but lots of > them would be rewritten as do-while's. So I think there is more than > enough use cases for syntactic sugar for do-while loops. The PEP 315 solution looks much better than an "until" that isn't what it looks like. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mclay at python.net Mon Jun 13 16:16:53 2005 From: mclay at python.net (Michael McLay) Date: Mon, 13 Jun 2005 10:16:53 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <42AD770A.1040001@gmail.com> References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> Message-ID: <200506131016.53962.mclay@python.net> On Monday 13 June 2005 08:07, Nick Coghlan wrote: > Raymond Hettinger wrote: > > [BJ?rn Lindqvist] > > > >>I would like to have do-while's like this: > >> > >>do: > >> > >> until > >> > >>But I'm sure that has problems too. > > > > That looks nice to me. > > And this could easily be extended to allow code both before and after > the 'until', giving a fully general loop: > > do: > > until > > else: > > > In fact, this would simply be giving "looks like executable > pseudocode" syntactic sugar for the current 'do-until' workarounds: > > while 1: > > if : > > break > Yet another way to spell this would be make the default for the while statement be true so the 1 could be omitted and then add a condition to break. while: break : I think this would be feature creep. It complicates the language for a very small gain. While the added syntax would be intuitive, it only saves a line or two over the existing syntax. From mwh at python.net Mon Jun 13 20:06:21 2005 From: mwh at python.net (Michael Hudson) Date: Mon, 13 Jun 2005 19:06:21 +0100 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <200506131016.53962.mclay@python.net> (Michael McLay's message of "Mon, 13 Jun 2005 10:16:53 -0400") References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> <200506131016.53962.mclay@python.net> Message-ID: <2mwtoydtcy.fsf@starship.python.net> Michael McLay writes: > I think this would be feature creep. It complicates the language > for a very small gain. While the added syntax would be intuitive, it > only saves a line or two over the existing syntax. FWIW, this is my opinion too (and I've written a bunch of while True:...if x: break loops today, even). It would be much nicer if people spent time worrying about the threads in multiple subinterpreters bug, or why def sc(*classes): return type('', classes, {}) sc(sc(str), sc(str)) fails. Cheers, mwh -- Guido (like us!) is a bit schizophrenic here: he wants to be a benevolent dictator, but also wants to treat people like grownups. This probably worked better before Python got a large American audience <0.9 wink>. -- Tim Peters, 10 Feb 2000 From mkent at webmd.net Mon Jun 13 20:21:36 2005 From: mkent at webmd.net (Michael Kent) Date: Mon, 13 Jun 2005 18:21:36 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?AIX_4=2E3=2C_Python_2=2E4=2E1_fails_in_tes?= =?utf-8?q?t=5Fexceptions_with_a_core_dump?= Message-ID: I'm attempting to switch from 2.3.2 to 2.4.1 on our AIX 4.3 development system. I have no problems building Python 2.3.2. I build Python 2.4.1 using 'configure --without-threads; gmake; gmake test', and always get a coredump during the tests on 'test_exceptions'. I've searched for any reports of this problem on the Sourceforge bug list and here, with no success. Has anyone seen this problem? From mcherm at mcherm.com Mon Jun 13 21:36:33 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 13 Jun 2005 12:36:33 -0700 Subject: [Python-Dev] Wishlist: dowhile Message-ID: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> Jp Calderone writes: > for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): > f2.write(chunk) Phillip J. Eby responds: > Showoff. ;) > > More seriously, I think your translation makes an excellent argument in > *favor* of having a do/while statement for greater clarity. :) Interesting... I had the opposite reaction. I often see someone do something "far too clever", which looks cute and impresses me, but makes me think that one would be wiser to avoid clever tricks and just write straightforward (if dull) code. But when I read JP's code I immediately thought that it WAS the straightforward way to write that code, and that I was just not smart enough to have realized it until he showed me. -- Michael Chermside From ark at acm.org Mon Jun 13 22:24:37 2005 From: ark at acm.org (Andrew Koenig) Date: Mon, 13 Jun 2005 16:24:37 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: Message-ID: <007301c57055$eb99bf60$6402a8c0@arkdesktop> > > But I cannot do this: > > > > def f(x as (a, b)): > > # ... > > which is what I was suggesting. > Yeah, I knew that. How important is that to you? I would really have liked it for a program I was working on at one time that had lots of small methods that passed around values of forms such as (a,(b,c)). Each method would pick one or more of the components out of its compound value and then would sometimes pass the entire value down to the next level. Of course I could do it this way: def f(x): (a,(b,c)) = x # ... but in this particular application, "as" would have offered significant extra convenience. I understand that this is not a real big deal, which is why I haven't written up a PEP :-) From python at rcn.com Mon Jun 13 22:46:18 2005 From: python at rcn.com (Raymond Hettinger) Date: Mon, 13 Jun 2005 16:46:18 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <20050613121423.5047.539927633.divmod.quotient.2056@ohm> Message-ID: <000001c57058$f30d5f60$3f21a044@oemcomputer> [Jp Calderone] > Anything can be a for loop. > > for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): > f2.write(chunk) It would be nice to have this encapsulated in a file method: for chunk in f1.iterblocks(CHUNK_SIZE): f2.write(chunk) Raymond From tdelaney at avaya.com Tue Jun 14 00:38:03 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Tue, 14 Jun 2005 08:38:03 +1000 Subject: [Python-Dev] PEP 342 - Enhanced Iterators Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE025205A6@au3010avexu1.global.avaya.com> There's a thread on c.l.py at the moment ("Controlling a generator the pythonic way") which is basically coming up with PEP 342. I've pointed them to PEP 342, but it's made me think that the name of the PEP could better indicate what it does. I propose "Coroutines via Enhanced Iterators". Tim Delaney From aahz at pythoncraft.com Tue Jun 14 01:25:12 2005 From: aahz at pythoncraft.com (Aahz) Date: Mon, 13 Jun 2005 16:25:12 -0700 Subject: [Python-Dev] AIX 4.3, Python 2.4.1 fails in test_exceptions with a core dump In-Reply-To: References: Message-ID: <20050613232511.GA590@panix.com> On Mon, Jun 13, 2005, Michael Kent wrote: > > I'm attempting to switch from 2.3.2 to 2.4.1 on our AIX 4.3 > development system. I have no problems building Python 2.3.2. I > build Python 2.4.1 using 'configure --without-threads; gmake; > gmake test', and always get a coredump during the tests on > 'test_exceptions'. I've searched for any reports of this problem on > the Sourceforge bug list and here, with no success. Has anyone seen > this problem? Your question is more on the borderline than most, but please try comp.lang.python before posting here; although you've got more core developers on python-dev, there's a broader spectrum on c.l.py. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ f u cn rd ths, u cn gt a gd jb n nx prgrmmng. From gvanrossum at gmail.com Tue Jun 14 03:30:40 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon, 13 Jun 2005 18:30:40 -0700 Subject: [Python-Dev] PEP 342 - Enhanced Iterators In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE025205A6@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE025205A6@au3010avexu1.global.avaya.com> Message-ID: Good idea, done. On 6/13/05, Delaney, Timothy C (Timothy) wrote: > There's a thread on c.l.py at the moment ("Controlling a generator the > pythonic way") which is basically coming up with PEP 342. I've pointed > them to PEP 342, but it's made me think that the name of the PEP could > better indicate what it does. > > I propose "Coroutines via Enhanced Iterators". > > Tim Delaney > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From iextream at naver.com Tue Jun 14 04:18:32 2005 From: iextream at naver.com (=?euc-kr?B?IsGkw+K/rCI=?=) Date: Tue, 14 Jun 2005 11:18:32 +0900 (KST) Subject: [Python-Dev] PEP 342 - Enhanced Iterators Message-ID: <42AE3E78.000001.05103@nhn429> <-----Original Message-----> > From: "Guido van Rossum" > To: "Delaney, Timothy C (Timothy)" > Cc: > Sent: 2005-06-14 10:31 > Subject: Re: [Python-Dev] PEP 342 - Enhanced Iterators Good idea, done. On 6/13/05, Delaney, Timothy C (Timothy) wrote: > There's a thread on c.l.py at the moment ("Controlling a generator the > pythonic way") which is basically coming up with PEP 342. I've pointed > them to PEP 342, but it's made me think that the name of the PEP could > better indicate what it does. > > I propose "Coroutines via Enhanced Iterators". > > Tim Delaney > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/iextream%40naver.com ------------------------------------------------------------------------ ??? :: ?! ???? ?? ??? ??? ?????. http://mail.naver.com/ From greg.ewing at canterbury.ac.nz Tue Jun 14 04:20:26 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 14 Jun 2005 14:20:26 +1200 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> References: <42ACF1CF.5000203@gmail.com> <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> Message-ID: <42AE3EEA.7090704@canterbury.ac.nz> Phillip J. Eby wrote: > By the way, whatever happened to "and while"? i.e.: > > while True: > data = inp.read(blocksize) > and while data: > out.write(data) My favourite version of this is while: data = inp.read(blocksize) gives data: out.write(data) -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From bob at redivi.com Tue Jun 14 04:27:31 2005 From: bob at redivi.com (Bob Ippolito) Date: Mon, 13 Jun 2005 22:27:31 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <42AE3EEA.7090704@canterbury.ac.nz> References: <42ACF1CF.5000203@gmail.com> <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> <5.1.1.6.0.20050613011832.01dffd48@mail.telecommunity.com> <42AE3EEA.7090704@canterbury.ac.nz> Message-ID: On Jun 13, 2005, at 10:20 PM, Greg Ewing wrote: > Phillip J. Eby wrote: > > >> By the way, whatever happened to "and while"? i.e.: >> >> while True: >> data = inp.read(blocksize) >> and while data: >> out.write(data) >> > > My favourite version of this is > > while: > data = inp.read(blocksize) > gives data: > out.write(data) Well, we could always just make iterator factories for the common cases and stuff them in itertools. I do need to use this pattern from time to time, but primarily for this exact use case, so an "itertools.readby" or the like would probably solve this problem for most people most of the time. -bob From python at rcn.com Tue Jun 14 07:25:38 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue, 14 Jun 2005 01:25:38 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: Message-ID: <001a01c570a1$80625260$3f21a044@oemcomputer> > >> By the way, whatever happened to "and while"? i.e.: > >> > >> while True: > >> data = inp.read(blocksize) > >> and while data: > >> out.write(data) > >> > > > > My favourite version of this is > > > > while: > > data = inp.read(blocksize) > > gives data: > > out.write(data) > > Well, we could always just make iterator factories for the common > cases and stuff them in itertools. I do need to use this pattern > from time to time, but primarily for this exact use case, so an > "itertools.readby" or the like would probably solve this problem for > most people most of the time. Doesn't work. You still need to be able to pass through the blocksize argument and the target bound method. To get what you want, there would need to be a new method in the file API. The reason is roughly similar to why we have iteritems as part of the mapping API and not as a standalone itertool. for data in inp.read(blocksize): . . . Raymond From bob at redivi.com Tue Jun 14 07:44:31 2005 From: bob at redivi.com (Bob Ippolito) Date: Tue, 14 Jun 2005 01:44:31 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <001a01c570a1$80625260$3f21a044@oemcomputer> References: <001a01c570a1$80625260$3f21a044@oemcomputer> Message-ID: <35C64EB9-1FCC-4169-9331-D974DBC8DCC5@redivi.com> On Jun 14, 2005, at 1:25 AM, Raymond Hettinger wrote: >>>> By the way, whatever happened to "and while"? i.e.: >>>> >>>> while True: >>>> data = inp.read(blocksize) >>>> and while data: >>>> out.write(data) >>>> >>>> >>> >>> My favourite version of this is >>> >>> while: >>> data = inp.read(blocksize) >>> gives data: >>> out.write(data) >>> >> >> Well, we could always just make iterator factories for the common >> cases and stuff them in itertools. I do need to use this pattern >> from time to time, but primarily for this exact use case, so an >> "itertools.readby" or the like would probably solve this problem for >> most people most of the time. >> > > Doesn't work. You still need to be able to pass through the blocksize > argument and the target bound method. To get what you want, there > would > need to be a new method in the file API. The reason is roughly > similar > to why we have iteritems as part of the mapping API and not as a > standalone itertool. > > for data in inp.read(blocksize): > . . . Reality check? def readby(inp, blocksize=1024): while True: data = inp.read(blocksize) if not data: break yield data for data in readby(inp, blocksize): . . . I always thought was that iteritems is part of the mapping API and not as a standalone itertool because it is the *most primitive* way to iterate over the key/value pairs. It is simply not implementable without access to the private guts of the dict. That is simply not the case here. If you need an ultra-flexible (but less obvious) implementation you should just use the sentinel version of the iter function as JP demonstrated. -bob From fredrik at pythonware.com Tue Jun 14 08:16:54 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 14 Jun 2005 08:16:54 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement References: <20050610144835.GB5929@burma.localdomain> Message-ID: Gustavo Niemeyer wrote: > > > moving the main trunk and main development over to the Python CVS is > > > another thing, entirely. > > > > (as I've said many times before, both the user community and the developer > > community would benefit if the core standard library were made smaller, and > > more externally maintained packages were included in the standard releases) > > An issue to consider about this is that maintainers (not talking about > you or anyone else specifically) have different concepts of stability, > and while it may seem perfectly ok to refactor external modules between > two stable releases, doing so in the standard library would spread fear > and "python is so untrustful" feelings. I think you're seriously underestimating the competence of other developers, and seriously overestimating the competence of the python core developers. in my experience, any external library that supports more than one Python version on more than one platform is likely to be more robust than code in the core. add the multilevel volunteer approach de- described by Steven (with the right infrastructure, things like that just appear), and you get more competent manpower contributing to the standard distribution than you can get in any other way. From python at rcn.com Tue Jun 14 08:25:40 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue, 14 Jun 2005 02:25:40 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: Message-ID: <000001c570a9$e37c8660$3f21a044@oemcomputer> > def readby(inp, blocksize=1024): > while True: > data = inp.read(blocksize) > if not data: > break > yield data > > for data in readby(inp, blocksize): > . . . readby() relies on the existence of a read() method for inp. itertools work with generic iterators, not ones with a specific API. Law of Demeter. Raymond From mcherm at mcherm.com Tue Jun 14 14:09:33 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Tue, 14 Jun 2005 05:09:33 -0700 Subject: [Python-Dev] Wishlist: dowhile Message-ID: <20050614050933.cl98pa45q6o8k0ws@login.werra.lunarpages.com> Jp Calderone writes: > Anything can be a for loop. > > for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): > f2.write(chunk) Raymond responds: > It would be nice to have this encapsulated in a file method: > > for chunk in f1.iterblocks(CHUNK_SIZE): > f2.write(chunk) What ever happened to "Not every 3 line function needs to be a builtin"? It's a common pattern. It's easy to do. Where's the problem? -- Michael Chermside From bob at redivi.com Tue Jun 14 14:41:14 2005 From: bob at redivi.com (Bob Ippolito) Date: Tue, 14 Jun 2005 08:41:14 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <000001c570a9$e37c8660$3f21a044@oemcomputer> References: <000001c570a9$e37c8660$3f21a044@oemcomputer> Message-ID: <327DBAAC-947F-420B-A207-425D5CC4B5AB@redivi.com> On Jun 14, 2005, at 2:25 AM, Raymond Hettinger wrote: >> def readby(inp, blocksize=1024): >> while True: >> data = inp.read(blocksize) >> if not data: >> break >> yield data >> >> for data in readby(inp, blocksize): >> . . . >> > > readby() relies on the existence of a read() method for inp. > itertools work with generic iterators, not ones with a specific API. > Law of Demeter. islice depends on __getitem__. -bob From ncoghlan at gmail.com Tue Jun 14 15:25:11 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Jun 2005 23:25:11 +1000 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <5.1.1.6.0.20050613123758.0339dc00@mail.telecommunity.com> References: <42AAFBB6.2070802@gmail.com> <42AB8D4A.5020407@gmail.com> <42AB8F2F.7060204@gmail.com> <42ABB59C.5050608@gmail.com> <42ACE839.1070804@gmail.com> <5.1.1.6.0.20050613123758.0339dc00@mail.telecommunity.com> Message-ID: <42AEDAB7.10601@gmail.com> Phillip J. Eby wrote: > Ow. My head hurts. :) Seriously, though, wouldn't it make more sense > to put the 'with async_exceptions_blocked()' in the __init__ or > __enter__ of 'critical_resource'? Or am I still missing something? Heck, I *suggested* the trick, and still had to look at it a half dozen times, including frontways, and backways and upside down in order to satisfy myself that it actually addressed all the issues Guido and I had been talking about. Anyway, the kicker is that (in the rare cases where it matters), you want asynchronous exceptions blocked before EXPR gets evaluated, and then unblocked again after the try/finally has been entered, but before BLOCK starts evaluating. I believe the only real way to do that deterministically is to do the blocking and unblocking of asynchronous exceptions inline - when the asynchronous exception occurs, either the critical resource hasn't been acquired yet, or we are inside a try/finally that will ensure its release. Trying to push the blocking/unblocking inside a template like safe_acquisition actually results in the loss of that determinism, since it may be the case that safe_acquisition itself is not finalised properly (the asynchronous exception goes off prior to entry of the relevant try/finally block). However, the generator object's __del__ method will clean things up eventually. Anyway, the main point is that with statements won't make any more guarantees about this than try/finally statements do - but they do provide the tool to deal with it reasonably cleanly when it matters (most of the time the asynchronous error will indicate program shutdown, and the remote possibility of misbehaviour won't really matter in practice). Cheers, Nick. P.S. I've included this trick as part of the Wiki discussion, meaning my last PEP 343 question has been well and truly dealt with. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From fperez.net at gmail.com Tue Jun 14 16:07:12 2005 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 14 Jun 2005 08:07:12 -0600 Subject: [Python-Dev] A bug in pyconfig.h under Linux? Message-ID: Hi all, sorry for posting to this list, but I'm not even 100% sure this is a bug. If it is, I'll gladly post it to SF if you folks want it there. I use scipy a lot, and the weave.inline component in there allows dynamic inclusion of C/C++ code in Python sources. In particular, it supports Blitz++ array expressions for access to Numeric arrays. However, whenever I use blitz-based code, I get these annoying warnings: ================================================================================ In file included from /usr/include/python2.3/Python.h:8, from sc_weave.cpp:5: /usr/include/python2.3/pyconfig.h:850:1: warning: "_POSIX_C_SOURCE" redefined In file included from /usr/lib/gcc/i386-redhat-linux/3.4.3/../../../../include/c++/3.4.3/i386-redhat-linux/bits/os_defines.h:39, from /usr/lib/gcc/i386-redhat-linux/3.4.3/../../../../include/c++/3.4.3/i386-redhat-linux/bits/c++config.h:35, from /usr/lib/gcc/i386-redhat-linux/3.4.3/../../../../include/c++/3.4.3/string:45, from /usr/lib/python2.3/site-packages/weave/blitz-20001213/blitz/blitz.h:153, from /usr/lib/python2.3/site-packages/weave/blitz-20001213/blitz/array-impl.h:154, from /usr/lib/python2.3/site-packages/weave/blitz-20001213/blitz/array.h:94, from sc_weave.cpp:4: /usr/include/features.h:150:1: warning: this is the location of the previous definition ================================================================================ This is on a Fedora Core 3 box, using glibc-headers.i386 version 2.3.5. The source of the problem seems to be that in file /usr/include/python2.3/pyconfig.h, line 850, I have: /* Define to activate features from IEEE Stds 1003.1-2001 */ #define _POSIX_C_SOURCE 200112L But the system headers, in /usr/include/features.h, line 150 give: # define _POSIX_C_SOURCE 199506L Hence the double-define. Now, I noticed that the system headers all use the following approach to defining these constants: # undef _POSIX_SOURCE # define _POSIX_SOURCE 1 # undef _POSIX_C_SOURCE # define _POSIX_C_SOURCE 199506L etc. That is, they undef everything before defining their value. I applied the same change manually to pyconfig.h: /* Define to activate features from IEEE Stds 1003.1-2001 */ #undef _POSIX_C_SOURCE #define _POSIX_C_SOURCE 200112L and my spurious warnings went away. But I realize that pyconfig.h is auto-generated, so the right solution, if this is indeed a bug, has to be applied somewhere else, at the code generation source. I am unfortunately not familiar enough with Python's build system and the autoconf toolset to do that. Furthermore, I am not even 100% sure this is really a bug, though the spurious warning is very annoying. If this is indeed a bug, do you folks want it reported on SF as such? In that case, is this explanation enough/correct? Any advice would be much appreciated. Regards, Fernando. From martin at v.loewis.de Tue Jun 14 17:39:29 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 14 Jun 2005 17:39:29 +0200 Subject: [Python-Dev] A bug in pyconfig.h under Linux? In-Reply-To: References: Message-ID: <42AEFA31.40004@v.loewis.de> Fernando Perez wrote: > sorry for posting to this list, but I'm not even 100% sure this is a bug. If it > is, I'll gladly post it to SF if you folks want it there. This is not a bug. Most likely, sc_weave.cpp fails to meet the requirement documented in http://docs.python.org/api/includes.html "Warning: Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included. " If you follow this recommendation, this kind of error should not occur. Regards, Martin From fperez.net at gmail.com Tue Jun 14 17:56:47 2005 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 14 Jun 2005 09:56:47 -0600 Subject: [Python-Dev] A bug in pyconfig.h under Linux? References: <42AEFA31.40004@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: > Fernando Perez wrote: >> sorry for posting to this list, but I'm not even 100% sure this is a bug. >> If it is, I'll gladly post it to SF if you folks want it there. > > This is not a bug. Most likely, sc_weave.cpp fails to meet the > requirement documented in > > http://docs.python.org/api/includes.html > > "Warning: Since Python may define some pre-processor definitions which > affect the standard headers on some systems, you must include Python.h > before any standard headers are included. " Many thanks to Martin and Jeff Epler for the feedback, and sorry for the noise, the bug was in weave and not in pyconfig.h. I was able to fix scipy's weave to respect this constraint, which it didn't in the case of blitz-enhanced array handling code. Regards, f From python at rcn.com Tue Jun 14 18:21:08 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue, 14 Jun 2005 12:21:08 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <327DBAAC-947F-420B-A207-425D5CC4B5AB@redivi.com> Message-ID: <003e01c570fd$12a27280$3f21a044@oemcomputer> [Bob] > islice depends on __getitem__. Incorrect. It uses the iterator protocol. From aqua_azul_2000 at yahoo.com.mx Tue Jun 14 18:26:55 2005 From: aqua_azul_2000 at yahoo.com.mx (Vero) Date: Tue, 14 Jun 2005 11:26:55 -0500 (CDT) Subject: [Python-Dev] Dynamic class inheritance && something else Message-ID: <20050614162655.52654.qmail@web61019.mail.yahoo.com> Hi. My name is Veronica, I am a master student at UNAM. I am working on something related to Artificial Inteligence and I have been looking for the most appropriated programming language to implement my algorithms. I found python to be very close to what I need, but there are still a couple of details that I haven't been able to solve. First, (and most important) I would like to be able to dynamically modify the classes from which a class inherits. I haven't found any way to do it with the language as it is. If there is a way, any suggestion will be truly appreciated. If I had to modify the source code to achieve this, I hope that you could give me some hints; I have an idea of how something like this could be achieved but since I don't know deeply the python sourcode I could get lost. Second, since my program will be modifying classes during run time, I would like to have a way to write on a file the python code that would have defined the class with the functions and attributes as they were left, just as if it had been writen like that at the very begining. I need it to be python code because I will be using that latter. Maybe I will have to solve this second problem by myself but I just wrote in case anybody had a good idea. Thank you very much for your help. Vero La sociedad es inherentemente democr?tica: la mayor?a decide si pensar por si misma o si dejar que alguien m?s les diga qu? pensar. http://mx.geocities.com/aqua_azul_2000/ --------------------------------- Do You Yahoo!? La mejor conexi?n a Internet y 2GB extra a tu correo por $100 al mes. http://net.yahoo.com.mx -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050614/9adeafe8/attachment.htm From aahz at pythoncraft.com Tue Jun 14 18:29:44 2005 From: aahz at pythoncraft.com (Aahz) Date: Tue, 14 Jun 2005 09:29:44 -0700 Subject: [Python-Dev] Dynamic class inheritance && something else In-Reply-To: <20050614162655.52654.qmail@web61019.mail.yahoo.com> References: <20050614162655.52654.qmail@web61019.mail.yahoo.com> Message-ID: <20050614162944.GA28529@panix.com> On Tue, Jun 14, 2005, Vero wrote: > > Hi. My name is Veronica, I am a master student at UNAM. I am working > on something related to Artificial Inteligence and I have been looking > for the most appropriated programming language to implement my > algorithms. I found python to be very close to what I need, but there > are still a couple of details that I haven't been able to solve. Hi Veronica, python-dev is for future development of Python; for questions about using Python, please go to comp.lang.python. Thanks. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ f u cn rd ths, u cn gt a gd jb n nx prgrmmng. From gvanrossum at gmail.com Tue Jun 14 18:38:31 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue, 14 Jun 2005 09:38:31 -0700 Subject: [Python-Dev] Dynamic class inheritance && something else In-Reply-To: <20050614162655.52654.qmail@web61019.mail.yahoo.com> References: <20050614162655.52654.qmail@web61019.mail.yahoo.com> Message-ID: On 6/14/05, Vero wrote: > Hi. My name is Veronica, I am a master student at UNAM. I am working on > something related to Artificial Inteligence and I have been looking for the > most appropriated programming language to implement my algorithms. I found > python to be very close to what I need, but there are still a couple of > details that I haven't been able to solve. > > First, (and most important) I would like to be able to dynamically modify > the classes from which a class inherits. I haven't found any way to do it > with the language as it is. If there is a way, any suggestion will be truly > appreciated. If I had to modify the source code to achieve this, I hope > that you could give me some hints; I have an idea of how something like this > could be achieved but since I don't know deeply the python sourcode I could > get lost. I wonder if assignment to __class__ (for instances) or to __bases__ (for classes) is what you are looking for? >>> class C: def foo(self): print "foo" >>> class D: def foo(self): print "D.foo" >>> x = C() >>> x.__class__ = D >>> x.foo() D.foo >>> x.__class__ = C >>> x.foo() foo >>> > Second, since my program will be modifying classes during run time, I would > like to have a way to write on a file the python code that would have > defined the class with the functions and attributes as they were left, just > as if it had been writen like that at the very begining. I need it to be > python code because I will be using that latter. Maybe I will have to solve > this second problem by myself but I just wrote in case anybody had a good > idea. That one's not so easy; there's no standard solution for this problem. I recommend that you follow Aahz' suggestion of asking on c.l.py instead. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bjourne at gmail.com Tue Jun 14 20:24:44 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Tue, 14 Jun 2005 20:24:44 +0200 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> <740c3aec05061308536bceeead@mail.gmail.com> Message-ID: <740c3aec05061411241149a557@mail.gmail.com> > > do: > > > > until > > > > Written like this it is not very obvious that the 'unil' is part of > > the do-until suite. I also imagine it to be difficult to parse and it > > breaks the rule that suites end when there is a dedentation. So, IMHO > > using an indented 'until' is the least evil of a number of evils. > > Not difficult to parse at all, nor un-Pythonic. Multi-part blocks > abound in Python: if / elif / else, try / finally, etc. Oh. I had the impression that the reason do-while's (or do-until's which I like more) wasn't implemented in the language was because it was impossible to find an acceptable looking syntax. > > Yes, but grepping the stdlib produces over 300 hits for "while 1:" and > > "while True:" combined. Some of those a "if : break" in the > > middle and some would be better written as generators, but lots of > > them would be rewritten as do-while's. So I think there is more than > > enough use cases for syntactic sugar for do-while loops. > > The PEP 315 solution looks much better than an "until" that isn't what > it looks like. What do you mean by that? The proposed until would work exactly like it do in languages that support until loops. I'm also reasonably happy with PEP 315, except that it seems to force you to end it with a pass if the condition is last in the loop. But is discussing do-while's beating a dead horse or is there a possibility that do-while's will someday make it into the language? -- mvh Bj?rn From eric.nieuwland at xs4all.nl Tue Jun 14 20:26:37 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Tue, 14 Jun 2005 20:26:37 +0200 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <42ACF1CF.5000203@gmail.com> References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> Message-ID: <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl> Nick Coghlan wrote: > With PEP 315, a do-while loop would look like: > > do: > > while : > pass > > But then, I'm reasonably happy with the 'break out of an infinite > loop' approach, so *shrug*. From Programming Languages 101 I remember this construct in Algol 68. It was then claimed to be *the* universal loop construct. If that is true __and__ it is easy to implement, I'd say +INF for PEP 315. --eric From gvanrossum at gmail.com Tue Jun 14 20:46:39 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue, 14 Jun 2005 11:46:39 -0700 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <740c3aec05061411241149a557@mail.gmail.com> References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> <740c3aec05061308536bceeead@mail.gmail.com> <740c3aec05061411241149a557@mail.gmail.com> Message-ID: On 6/14/05, BJ?rn Lindqvist wrote: > Oh. I had the impression that the reason do-while's (or do-until's > which I like more) wasn't implemented in the language was because it > was impossible to find an acceptable looking syntax. No, just because it's fairly redundant. > > > Yes, but grepping the stdlib produces over 300 hits for "while 1:" and > > > "while True:" combined. Some of those a "if : break" in the > > > middle and some would be better written as generators, but lots of > > > them would be rewritten as do-while's. So I think there is more than > > > enough use cases for syntactic sugar for do-while loops. > > > > The PEP 315 solution looks much better than an "until" that isn't what > > it looks like. > > What do you mean by that? The proposed until would work exactly like > it do in languages that support until loops. I'm also reasonably happy > with PEP 315, except that it seems to force you to end it with a pass > if the condition is last in the loop. But is discussing do-while's > beating a dead horse or is there a possibility that do-while's will > someday make it into the language? You are proposing 'until' that is indented. That is unlike any other language. Translating this to curly-braces style, you're proposing this: do { ...body... until (x <= 0); } I find this bizarre, not " exactly like it do [sic] in languages that support until loops." -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ark-mlist at att.net Tue Jun 14 20:48:03 2005 From: ark-mlist at att.net (Andrew Koenig) Date: Tue, 14 Jun 2005 14:48:03 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl> Message-ID: <001301c57111$985335e0$6402a8c0@arkdesktop> > > With PEP 315, a do-while loop would look like: > > > > do: > > > > while : > > pass > > But then, I'm reasonably happy with the 'break out of an infinite > > loop' approach, so *shrug*. > From Programming Languages 101 I remember this construct in Algol 68. > It was then claimed to be *the* universal loop construct. If that is > true __and__ it is easy to implement, I'd say +INF for PEP 315. I believe that Algol 68 loops looked like the following (in somewhat more Python-like terms): [for [from ] [step ] to ] [while ] do od where either "for" or "while" had to be present. Note that in Algol 68, the value of a "suite" (which I think they called a "serial clause") is the value of the last expression in it. From gvanrossum at gmail.com Tue Jun 14 20:48:53 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue, 14 Jun 2005 11:48:53 -0700 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl> References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl> Message-ID: On 6/14/05, Eric Nieuwland wrote: > From Programming Languages 101 I remember this construct in Algol 68. > It was then claimed to be *the* universal loop construct. If that is > true __and__ it is easy to implement, I'd say +INF for PEP 315. It's true, but this both dates you (A68 has been dead for several decades) and locates you: it was said that A68's popularity was inversely proportional to (the square of?) the distance from Amsterdam. It also dates and locates me. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric.nieuwland at xs4all.nl Tue Jun 14 22:57:35 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Tue, 14 Jun 2005 22:57:35 +0200 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: References: <000001c56fbb$3d7d0c60$4ff9a244@oemcomputer> <42ACF1CF.5000203@gmail.com> <7d15fcc39dd3086e0aadcbb981e9742f@xs4all.nl> Message-ID: <625faffa5cb24ecad85843dce2f090eb@xs4all.nl> Guido van Rossum wrote: > On 6/14/05, Eric Nieuwland wrote: >> From Programming Languages 101 I remember this construct in Algol 68. >> It was then claimed to be *the* universal loop construct. If that is >> true __and__ it is easy to implement, I'd say +INF for PEP 315. > > It's true, but this both dates you (A68 has been dead for several > decades) and locates you: it was said that A68's popularity was > inversely proportional to (the square of?) the distance from > Amsterdam. It also dates and locates me. :-) Lucky me ;-) From gvanrossum at gmail.com Tue Jun 14 23:11:47 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue, 14 Jun 2005 14:11:47 -0700 Subject: [Python-Dev] Compiling Python with Intel compiler? Message-ID: Intel has a free (as in beer) C compiler for Linux. A friend of mine is involved in its production and marketing. He would like to see how "his" compiler does on Python -- does it work at all, and is there a speedup? I promised him I'd look into this, but between PEPs and my day job I don't seem to have the time, so now I'm making up on my promise by trying to delegate... The compiler is here: http://www.intel.com/software/products/compilers/clin/index.htm On the right is a link to a "Free Non-Commercial Download". That's all I know. I think that if someone puts some time in this, they should get a free Intel T-shirt. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jun 14 23:40:54 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 14 Jun 2005 23:40:54 +0200 Subject: [Python-Dev] Compiling Python with Intel compiler? In-Reply-To: References: Message-ID: <42AF4EE6.2040000@v.loewis.de> Guido van Rossum wrote: > Intel has a free (as in beer) C compiler for Linux. A friend of mine > is involved in its production and marketing. He would like to see how > "his" compiler does on Python -- does it work at all, and is there a > speedup? There is a bug report on this compiler: python.org/sf/1162001 There is also a patch, python.org/sf/1162023 which works around the bug. I don't like the work-around, because it assumes that the compiler binary is called "icc", however, it might be called just "cc" on some installations. With that issue resolved, the compiler apparently can build Python. However, the resulting interpreter doesn't work for some people: http://mail.python.org/pipermail/python-list/2005-March/270672.html This issue apparently only exists if you use both -O3 and -xN; if you omit one of them, it builds fine. I suspect a compiler bug. As for performance, Perky compared ICC (unspecified version) with gcc 2.95 on FreeBSD 4.5: http://groups-beta.google.com/group/sol.lists.freebsd.ports/msg/90a736b7097d9b05 At that point, the then-current version of ICC produced faster code than the then-current version of gcc (this was in 2002). In Dec 2004, Mark Asbach did the same thing with ICC 8 vs. gcc 3.3.4, on Linux: http://groups-beta.google.com/group/comp.lang.python/msg/a4af997f8f95592e?hl=en He reported a small speedup. I couldn't find anything on Python and ICC V9. Regards, Martin From kbk at shore.net Wed Jun 15 03:14:02 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue, 14 Jun 2005 21:14:02 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200506150114.j5F1E2Zf015509@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 338 open ( -1) / 2861 closed ( +4) / 3199 total ( +3) Bugs : 909 open ( +1) / 5047 closed (+11) / 5956 total (+12) RFE : 188 open ( -1) / 170 closed ( +2) / 358 total ( +1) New / Reopened Patches ______________________ byteorder issue in PyMac_GetOSType (2005-06-10) http://python.org/sf/1218421 opened by Ronald Oussoren os.kill on windows (2005-06-14) http://python.org/sf/1220212 opened by Miki Tebeka _csv.c leaks when dialect creation fails (2005-06-14) http://python.org/sf/1220242 opened by Michael Hudson Patches Closed ______________ in IDLE, assume new text files are python source by default (2005-05-06) http://python.org/sf/1196895 closed by kbk Add const specifier to PySpam_System prototype (2005-04-21) http://python.org/sf/1187396 closed by birkenfeld Newline in error output of py_compile.compile (2005-03-26) http://python.org/sf/1171150 closed by birkenfeld trivial bug in error message text (2005-05-06) http://python.org/sf/1196980 closed by kbk New / Reopened Bugs ___________________ email.Utils.py: "'" in RFC2231 header (2005-06-10) http://python.org/sf/1218081 opened by Tokio Kikuchi inspect.getsource doesn't update when a module is reloaded (2005-06-10) http://python.org/sf/1218234 opened by Bj?rn Lindqvist Parser chokes on big files (2005-06-11) CLOSED http://python.org/sf/1218930 opened by Alexander Schremmer 'Expression' AST Node not documented (2005-06-12) http://python.org/sf/1219273 opened by Martin Miller copy.py typo (2005-06-13) CLOSED http://python.org/sf/1219342 opened by Ori Avtalion copy.py typo (2005-06-13) CLOSED http://python.org/sf/1219361 opened by Ori Avtalion small output bug (2005-06-12) CLOSED http://python.org/sf/1219448 opened by Alex Levchuk Need locale arg to strftime() (2005-06-13) http://python.org/sf/1219840 opened by Wilfredo Sanchez misdocumented argument range for curses.pair_content (2005-06-13) CLOSED http://python.org/sf/1219862 opened by dcrosta tp_richcompare documentation wrong and incomplete (2005-06-13) http://python.org/sf/1219903 opened by Barry A. Warsaw subprocess call() helper should close stdin if PIPE (2005-06-14) http://python.org/sf/1220113 opened by Stuart Bishop Re-importing embedded thread dumps core (2005-06-14) http://python.org/sf/1220756 opened by Jay T Miller Bugs Closed ___________ Parser chokes on big files (2005-06-11) http://python.org/sf/1218930 closed by doerwalter color highlighting not working on EDIT windows (2003-09-04) http://python.org/sf/800432 closed by kbk No syntax hilite on new file until saved as *.py (2003-07-21) http://python.org/sf/775012 closed by kbk copy.py typo (2005-06-12) http://python.org/sf/1219342 closed by rhettinger copy.py typo (2005-06-12) http://python.org/sf/1219361 closed by rhettinger small output bug (2005-06-12) http://python.org/sf/1219448 closed by rhettinger misdocumented argument range for curses.pair_content (2005-06-13) http://python.org/sf/1219862 closed by akuchling lax error-checking in new-in-2.4 marshal stuff (2005-04-11) http://python.org/sf/1180997 closed by mwh String and list methods documentation deeply hidden (2005-06-06) http://python.org/sf/1215887 closed by rhettinger Queue.qsize() better info about unreliability (2005-06-02) http://python.org/sf/1213475 closed by rhettinger weakref cannot handle bound methods (in contrast to docu) (2005-05-22) http://python.org/sf/1206537 closed by rhettinger New / Reopened RFE __________________ Create a fat build on darwin (2005-06-10) http://python.org/sf/1218333 opened by Ronald Oussoren RFE Closed __________ Make bisect.* functions accept an optional compare function (2005-04-18) http://python.org/sf/1185383 closed by rhettinger sets needs an 'arbitrary element' method (2005-05-31) http://python.org/sf/1212091 closed by rhettinger From raymond.hettinger at verizon.net Wed Jun 15 11:09:21 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 15 Jun 2005 05:09:21 -0400 Subject: [Python-Dev] PEP 343 question Message-ID: <000201c57189$eb273280$1c2ba044@oemcomputer> In gen.throw(), are all three arguments required? Or do the value and traceback Nones need to be listed explicitly? g.throw(MyException) or g.throw(MyException, None, None) FWIW, I prefer the former. That will make throw() as flexible as the raise statement. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050615/bc65f786/attachment.htm From ncoghlan at gmail.com Wed Jun 15 12:57:57 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 15 Jun 2005 20:57:57 +1000 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> Message-ID: <42B009B5.4040904@gmail.com> Guido van Rossum wrote: > Why are you so excited about having until indented? You didn't give > any examples with multiple occurrences. A single occurrence works just > fine unindented, as PEP 315 has already shown. I hadn't actually thought about the temptation to try and have multiple 'until' statements inside the suite. Having two suites of the statement included in the loop, while the third is excluded, feels slightly odd, but other than that PEP 315 is fine by me. The fact that PEP 315 only needs a single new keyword is a good thing, as is the fact that it means the 'else' clause still reads correctly. As you said earlier: "It is Pythonically minimal and the motivation rings true" Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From mwh at python.net Wed Jun 15 14:35:35 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 15 Jun 2005 13:35:35 +0100 Subject: [Python-Dev] refcounting vs PyModule_AddObject Message-ID: <2mzmtrbxwo.fsf@starship.python.net> I've just fixed a bug where Py_INCREF wasn't called when it should have been before a call to PyModule_AddObject (rev. 2.62 of Modules/threadmodule.c). So I went looking for other instances of the same problem. I didn't find any (though I don't understand how _csv.c gets away with line 1579), but what I *did* find were absolutely masses of what seemed like unnecessary increfs. Consider this code from _hotshot.c: Py_INCREF(&LogReaderType); PyModule_AddObject(module, "LogReaderType", (PyObject *)&LogReaderType); Py_INCREF(&ProfilerType); PyModule_AddObject(module, "ProfilerType", (PyObject *)&ProfilerType); if (ProfilerError == NULL) ProfilerError = PyErr_NewException("hotshot.ProfilerError", NULL, NULL); if (ProfilerError != NULL) { Py_INCREF(ProfilerError); PyModule_AddObject(module, "ProfilerError", ProfilerError); } The first two calls are fine; it was an incref like this that was missing in threadmodule.c. The second seems like a reference "leak": PyErr_NewException returns a new reference, then the Py_INCREF/PyModule_AddObject pair is refcount neutral, so control falls off the end of the function owning a reference to ProfilerError. I think the Py_INCREF should just be removed, but I'm wondering if I'm missing something... Cheers, mwh -- MacOSX: Sort of like a pedigree persian cat. Very sleek, very sexy, but a little too prone to going cross-eyed, biting you on your thumb and then throwing up on your trousers. -- Jim's pedigree of operating systems, asr From hoffman at ebi.ac.uk Wed Jun 15 12:13:37 2005 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Wed, 15 Jun 2005 11:13:37 +0100 Subject: [Python-Dev] Compiling Python with Intel compiler? In-Reply-To: <42AF4EE6.2040000@v.loewis.de> References: <42AF4EE6.2040000@v.loewis.de> Message-ID: On Tue, 14 Jun 2005, "Martin v. L?wis" wrote: > Guido van Rossum wrote: >> Intel has a free (as in beer) C compiler for Linux. A friend of mine >> is involved in its production and marketing. He would like to see how >> "his" compiler does on Python -- does it work at all, and is there a >> speedup? > > There is a bug report on this compiler: python.org/sf/1162001 > There is also a patch, python.org/sf/1162023 which works around > the bug. I don't like the work-around, because it assumes that the > compiler binary is called "icc", however, it might be called just > "cc" on some installations. I submitted this bug, patch, and the c.l.p posting and thank Martin for his help with them all. Martin's reservations on the workaround are correct but I can't make more time right now to fix autoconf "properly." To be honest, if icc were more gcc-compatible and returned an error code for -OPT:Olimit=0, that would obviate the need for any patch to Python. I never published these results before but here's what I found in terms of performance (full results available on request): $ python pybench.py -s icc-python -c fedora-python | tail -n 4 Average round time: 3677.00 ms -15.95% *) measured against: fedora-python (rounds=10, warp=20) $ python pybench.py -s icc-xN-python -c fedora-python | tail -n 4 Average round time: 3911.00 ms -10.61% *) measured against: fedora-python (rounds=10, warp=20) fedora-python: Python 2.4/GCC 3.4.2 20041017 from python.org yum repository icc-python: Python 2.4/icc (8.1) -pthread -fno-strict-aliasing -DNDEBUG -O3 icc-python-xN: Python2.4/icc (8.1) -pthread -fno-strict-aliasing -DNDEBUG -O3 -xN Intel Pentium 4 2.66GHz; 512 KB cache; 507644 kB RAM; Linux 2.6.11-1.14_FC3 While a 16% speedup is nothing to sneeze at, I felt I wasted a lot of time dealing with incompatibilities with extension modules and libcprts.so, and so I no longer run it on my desktop. I do use it for heavy computes on our compute farm when I'm not using extension modules. [Guido] > I think that if someone puts some time in this, they should get a > free Intel T-shirt. I'll accept a free Intel T-shirt if one is offered but I'll probably pass it on to our long-suffering sysadmins who have spent even more time dealing with icc, yet remain evangelists for it. Since this was my first message to python-dev, I think the custom is to introduce myself. I'm a PhD student at the University of Cambridge studying computational biology. I've been using Python for the last three years and have loved almost every minute of it. I've written sequence alignment algorithms in Pyrex, glue to genome database APIs in Jython, and a whole lot of other stuff using CPython. We run Python on a 1145-CPU compute farm that consists of Alpha/Tru64 and Intel/Linux boxes. -- Michael Hoffman European Bioinformatics Institute From ark-mlist at att.net Wed Jun 15 14:52:32 2005 From: ark-mlist at att.net (Andrew Koenig) Date: Wed, 15 Jun 2005 08:52:32 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <77991a6494eb379e2fc724e26b4103cc@xs4all.nl> Message-ID: <001101c571a9$1939d780$6402a8c0@arkdesktop> > > I believe that Algol 68 loops looked like the following (in somewhat > > more Python-like terms): > > > > [for [from ] [step ] to ] > > [while ] > > do od > As far as I remember, it was: > do > > while > > od > It might be that this could be preceded by a 'for' clause, but memory > fails me there. To be completely sure, I dug out my copy of the "Informal Introduction to Algol 68" by Lindsey and van der Meulen. This book is the official Algol 68 tutorial book. It describes the "loop clause" as follows: for some int identifier, which is hereby declared from some meek int unit by some other meek int unit to a third meek int unit while a meek bool enquiry-clause do a void serial-clause od where any of these keywords and its sequel can be omitted, except for "do" or "od". Here, the term "meek" refers to how aggressively the compiler should try to convert the result to int; it specifies, for example, that if after "from" you write the name of a procedure without arguments, that procedure will be called instead of using the procedure itself as a starting point (which would be nonsensical). From skip at pobox.com Wed Jun 15 14:57:00 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 15 Jun 2005 07:57:00 -0500 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <2mzmtrbxwo.fsf@starship.python.net> References: <2mzmtrbxwo.fsf@starship.python.net> Message-ID: <17072.9628.999188.106351@montanaro.dyndns.org> Michael> So I went looking for other instances of the same problem. I Michael> didn't find any (though I don't understand how _csv.c gets away Michael> with line 1579)... Same reason the Py_INCREF of ProfileError isn't necessary I think. PyDict_New() returns a new reference which is passed onto PyModule_AddObject(). No Py_DECREF of the dialects dict occurs, so it exits reference-neutral w.r.t. the dialects object. Skip From mwh at python.net Wed Jun 15 15:05:25 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 15 Jun 2005 14:05:25 +0100 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <17072.9628.999188.106351@montanaro.dyndns.org> (Skip Montanaro's message of "Wed, 15 Jun 2005 07:57:00 -0500") References: <2mzmtrbxwo.fsf@starship.python.net> <17072.9628.999188.106351@montanaro.dyndns.org> Message-ID: <2mvf4fbwiy.fsf@starship.python.net> Skip Montanaro writes: > Michael> So I went looking for other instances of the same problem. I > Michael> didn't find any (though I don't understand how _csv.c gets away > Michael> with line 1579)... > > Same reason the Py_INCREF of ProfileError isn't necessary I think. > PyDict_New() returns a new reference which is passed onto > PyModule_AddObject(). No Py_DECREF of the dialects dict occurs, so it exits > reference-neutral w.r.t. the dialects object. Oops; I meant line 1590. Cheers, mwh -- Windows 2000: Smaller cow. Just as much crap. -- Jim's pedigree of operating systems, asr From skip at pobox.com Wed Jun 15 15:40:10 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 15 Jun 2005 08:40:10 -0500 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <2mvf4fbwiy.fsf@starship.python.net> References: <2mzmtrbxwo.fsf@starship.python.net> <17072.9628.999188.106351@montanaro.dyndns.org> <2mvf4fbwiy.fsf@starship.python.net> Message-ID: <17072.12218.103715.151682@montanaro.dyndns.org> Michael> ... (though I don't understand how _csv.c gets away Michael> with line 1579)... Michael> Oops; I meant line 1590. Hmmm... Me either. Is it possible it was just never DECREF'd? I checked in the obvious fix on both head and the 2.4 release branch. Skip From arigo at tunes.org Wed Jun 15 15:51:10 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 15 Jun 2005 15:51:10 +0200 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <2mzmtrbxwo.fsf@starship.python.net> References: <2mzmtrbxwo.fsf@starship.python.net> Message-ID: <20050615135110.GA2560@code1.codespeak.net> Hi Michael, On Wed, Jun 15, 2005 at 01:35:35PM +0100, Michael Hudson wrote: > if (ProfilerError == NULL) > ProfilerError = PyErr_NewException("hotshot.ProfilerError", > NULL, NULL); > if (ProfilerError != NULL) { > Py_INCREF(ProfilerError); > PyModule_AddObject(module, "ProfilerError", ProfilerError); > } I think the Py_INCREF is needed here. The ProfilerError is a global variable that needs the extra reference. Otherwise, a malicious user could do "del _hotshot.ProfilerError" and have it garbage-collected under the feet of _hotshot.c which still uses it. What I don't get is how ProfilerError could fail to be NULL in the first 'if' above, but that's a different matter. While we're at strange refcounting problems, PyModule_AddObject() only decrefs its last argument if no error occurs. This is probably wrong. In general I've found that the C modules' init code is fragile. This might be due to the idea that it runs only once anyway, and global C-module objects are immortal anyway, so sloppiness sneaks in. But for example, the following is common: m = Py_InitModule3("xxx", NULL, module_doc); Py_INCREF(&Xxx_Type); PyModule_AddObject(m, "Xxx", (PyObject *)&Xxx_Type); This generates a segfault if Py_InitModule3() returns NULL (however rare that situation is). A bientot, Armin From arigo at tunes.org Wed Jun 15 16:06:35 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 15 Jun 2005 16:06:35 +0200 Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c, 1.37, 1.38 In-Reply-To: References: Message-ID: <20050615140635.GA3193@code1.codespeak.net> Hi Skip, On Wed, Jun 15, 2005 at 06:35:10AM -0700, montanaro at users.sourceforge.net wrote: > Why this worked is a bit mystical. Perhaps it never gets freed because the > object just happens never to be DECREF'd (but that seems unlikely). > /* Add the Dialect type */ > + Py_INCREF(&Dialect_Type); > if (PyModule_AddObject(module, "Dialect", (PyObject *)&Dialect_Type)) > return; Hum, you probably don't want to know, but it works just fine to forget a Py_INCREF before PyModule_AddObject() for the following reason: 1. the reference is stored in the module's dict, so the object is kept alive from there. 2. after the module initialization code is completed, the import mechanism make a copy of the dict (!) just in case some users wants to reload() the module (!!) in which case the module's dict is simplify overwritten with the copy again (!!!). So there is a reference left to the object from this hidden dict, and no way for the user to kill it -- short of using gc.getreferrers(), which is how I figured this out, but gc.getreferrers() is officially dangerous. So unlike what I thought in my previous e-mail, even if the user deletes the entry in the module's normal dict, nothing bad can happend because of this particular feature of import... So it just works. Hum. Armin From mwh at python.net Wed Jun 15 16:17:02 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 15 Jun 2005 15:17:02 +0100 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <20050615135110.GA2560@code1.codespeak.net> (Armin Rigo's message of "Wed, 15 Jun 2005 15:51:10 +0200") References: <2mzmtrbxwo.fsf@starship.python.net> <20050615135110.GA2560@code1.codespeak.net> Message-ID: <2mr7f3bt7l.fsf@starship.python.net> Armin Rigo writes: > Hi Michael, > > On Wed, Jun 15, 2005 at 01:35:35PM +0100, Michael Hudson wrote: >> if (ProfilerError == NULL) >> ProfilerError = PyErr_NewException("hotshot.ProfilerError", >> NULL, NULL); >> if (ProfilerError != NULL) { >> Py_INCREF(ProfilerError); >> PyModule_AddObject(module, "ProfilerError", ProfilerError); >> } > > I think the Py_INCREF is needed here. The ProfilerError is a global > variable that needs the extra reference. Otherwise, a malicious user > could do "del _hotshot.ProfilerError" and have it garbage-collected > under the feet of _hotshot.c which still uses it. Hmm. Point. But then how doesn't this apply to things like 'del thread._local'? (after my recent fix) > What I don't get is how ProfilerError could fail to be NULL in the > first 'if' above, but that's a different matter. Well, could it fail to be NULL in the multiple interpreters case? Though I'm not at all sure that reusing is wise in that case... > While we're at strange refcounting problems, PyModule_AddObject() only > decrefs its last argument if no error occurs. This is probably wrong. Quite probably :-/ > In general I've found that the C modules' init code is fragile. This > might be due to the idea that it runs only once anyway, and global > C-module objects are immortal anyway, so sloppiness sneaks in. Oh yes. Cheers, mwh -- It's actually a corruption of "starling". They used to be carried. Since they weighed a full pound (hence the name), they had to be carried by two starlings in tandem, with a line between them. -- Alan J Rosenthal explains "Pounds Sterling" on asr From mwh at python.net Wed Jun 15 16:26:00 2005 From: mwh at python.net (Michael Hudson) Date: Wed, 15 Jun 2005 15:26:00 +0100 Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c, 1.37, 1.38 In-Reply-To: <20050615140635.GA3193@code1.codespeak.net> (Armin Rigo's message of "Wed, 15 Jun 2005 16:06:35 +0200") References: <20050615140635.GA3193@code1.codespeak.net> Message-ID: <2mmzprbssn.fsf@starship.python.net> Armin Rigo writes: > Hi Skip, > > On Wed, Jun 15, 2005 at 06:35:10AM -0700, montanaro at users.sourceforge.net wrote: >> Why this worked is a bit mystical. Perhaps it never gets freed because the >> object just happens never to be DECREF'd (but that seems unlikely). >> /* Add the Dialect type */ >> + Py_INCREF(&Dialect_Type); >> if (PyModule_AddObject(module, "Dialect", (PyObject *)&Dialect_Type)) >> return; > > Hum, you probably don't want to know, but it works just fine to forget > a Py_INCREF before PyModule_AddObject() for the following reason: No, it's more complicated than that, at least in an embedded scenario (see bug #1220756). I don't understand how thread._local differed from _cvs.Dialect, though. Cheers, mwh -- faassen: anyway, if nothing else flapping your arms running around will at least give an impression of activity. :) From maliger at gmail.com Wed Jun 15 17:00:19 2005 From: maliger at gmail.com (Martin Aliger) Date: Wed, 15 Jun 2005 17:00:19 +0200 Subject: [Python-Dev] python running in several threads In-Reply-To: <42A61F8D.2090704@v.loewis.de> References: <63fd41d40506070715129c904a@mail.gmail.com> <20050607153253.GB2787@panix.com> <42A61F8D.2090704@v.loewis.de> Message-ID: <63fd41d40506150800ae6e358@mail.gmail.com> Thanks both for answers, I read some articles you link and I'd to say - alot discussion around! I also found some notes that mod_python use some technique other than GIL to allow multi-threading in Apache 2.0. Does someone know anything closer? Any links or so? Sorry I raise this old stuff again. Martin Just short notes: > Removing the GIL is easy, but then the interpreter crashes in cases of > simultaneous accesses to dictionaries, reference counters, etc. I know. But under mine conditions is impossible to share dictionaty or ref.counters so this do not happen. This dont happen neither on web server running simmultaniously several .py scripts (same conditions) > This is just not true. On a single-processor machine, the GIL does > *not* degrade performance. Instead, it increases throughput (and > thus performance). Under bigger load yes. It degrades a little under small load, but its usually not big problem. > On a multi-processor machine, you often use multiple operating > system processes to serve request (e.g. in CGI or Apache mpm-prefork, > or even the typical mpm-worker configuration). If you then have > different processes running Python, they don't interfere with > each other at all. Unfortunatelly many configurations dont do that. I run several servers with hyperthreading processors and none of them runs two IISs or apaches just for load-balance python scripts. Regards, Martin From gvanrossum at gmail.com Wed Jun 15 17:11:25 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed, 15 Jun 2005 08:11:25 -0700 Subject: [Python-Dev] PEP 343 question In-Reply-To: <000201c57189$eb273280$1c2ba044@oemcomputer> References: <000201c57189$eb273280$1c2ba044@oemcomputer> Message-ID: On 6/15/05, Raymond Hettinger wrote: > > In gen.throw(), are all three arguments required? Or do the value and > traceback Nones need to be listed explicitly? > > g.throw(MyException) > > or > > g.throw(MyException, None, None) > > FWIW, I prefer the former. That will make throw() as flexible as the raise > statement. I don't particularly care much; the analogy with raise makes sense, but the PEP carefully only calls it with 3 arguments, and since there are no use cases outside the PEP, the simpler implementation (requiring 3 arguments) makes more sense to me. If you want to implement this part, feel free to make it the way you like! I consider PEP 343 as accepted at this point as it ever will be; additional tweaks will be found during implementation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From arigo at tunes.org Wed Jun 15 18:48:21 2005 From: arigo at tunes.org (Armin Rigo) Date: Wed, 15 Jun 2005 18:48:21 +0200 Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c, 1.37, 1.38 In-Reply-To: <2mmzprbssn.fsf@starship.python.net> References: <20050615140635.GA3193@code1.codespeak.net> <2mmzprbssn.fsf@starship.python.net> Message-ID: <20050615164821.GA4757@code1.codespeak.net> Hi Michael, On Wed, Jun 15, 2005 at 03:26:00PM +0100, Michael Hudson wrote: > > Hum, you probably don't want to know, but it works just fine to forget > > a Py_INCREF before PyModule_AddObject() for the following reason: > > No, it's more complicated than that, at least in an embedded > scenario (see bug #1220756). Uh, yes, indeed. I guess that my explanations don't survive an interpreter shutdown, after which the reference counter of the object effectively drops to zero. I don't see how this doesn't crash Python when you exit. It could be that the gc is not called after the reference from the import mechanism was effectively removed -- type objects after a PyType_Ready() are full of ref cycles. But it's all quite obscure. Armin From skip at pobox.com Wed Jun 15 19:10:29 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 15 Jun 2005 12:10:29 -0500 Subject: [Python-Dev] [Python-checkins] python/dist/src/Modules _csv.c, 1.37, 1.38 In-Reply-To: <20050615164821.GA4757@code1.codespeak.net> References: <20050615140635.GA3193@code1.codespeak.net> <2mmzprbssn.fsf@starship.python.net> <20050615164821.GA4757@code1.codespeak.net> Message-ID: <17072.24837.306667.831660@montanaro.dyndns.org> Armin> But it's all quite obscure. If this was comp.lang.python I'd nominate this for QOTW... Skip From martin at v.loewis.de Wed Jun 15 20:24:29 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 15 Jun 2005 20:24:29 +0200 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <2mzmtrbxwo.fsf@starship.python.net> References: <2mzmtrbxwo.fsf@starship.python.net> Message-ID: <42B0725D.9040200@v.loewis.de> Michael Hudson wrote: > if (ProfilerError == NULL) > ProfilerError = PyErr_NewException("hotshot.ProfilerError", > NULL, NULL); > if (ProfilerError != NULL) { > Py_INCREF(ProfilerError); > PyModule_AddObject(module, "ProfilerError", ProfilerError); > } > > > I think the Py_INCREF should just be removed, but I'm wondering if I'm > missing something... It may be me who is missing something, but... On reference is added to the dictionary, this is the one the explicit INCREF creates. The other reference is held in the C variable ProfilerError; this is the one that creating the exception object creates. It is convention that C variables which are explicitly used also hold their own references, even if they are global, and even if there is no procedure to clear them. The reason is that they would become stale if the module object went away. As there is no way to protect against this case, they just keep a garbage reference. Regards, Martin From nidoizo at yahoo.com Wed Jun 15 20:37:05 2005 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Wed, 15 Jun 2005 14:37:05 -0400 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> Message-ID: Guido van Rossum wrote: > Why are you so excited about having until indented? You didn't give > any examples with multiple occurrences. A single occurrence works just > fine unindented, as PEP 315 has already shown. FWIW, I must say I disagree (about "works just fine"). I find PEP 315 counter-intuitive. There's multi-part blocks in Python, but they all behave like a step-sequence where you never come back to a previous step. What I mean is that after entering a finally/except/elif/else, there's no coming back in the try/if/for/while. My first impression when looking at: do: while condition: is that is executed only once. Yes, it would not make any sense since you would write instead: while condition: But, FWIW, that's still my impression. I remember reading Stroustrup saying he was never using do-while loops because he find them counter-intuitive. I feel the same way. I think this is important, because it means that even if do-while is available in C++, some programmers, even the creator himself, are still always using while loops. Even if different languages, I don't see why that would be different in Python. My mental image of a do-while loop looks more like a while-True-if-break Python loop than a PEP315-like loop. For one thing, since the loop content is executed at least once, I think about the condition breaking the loop, not the one maintaining it alive. The keyword 'until' has been proposed; I guess it would be as in Perl where it is an equivalent of "while not". Also, the "do" keyword means nothing to me; maybe something like "loop" would be better. Still, my feeling is that Python is just getting it right with the status quo. Regards, Nicolas From python at discworld.dyndns.org Wed Jun 15 21:03:52 2005 From: python at discworld.dyndns.org (Charles Cazabon) Date: Wed, 15 Jun 2005 13:03:52 -0600 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: References: <000901c56ffe$369f19e0$b8b3958d@oemcomputer> <42AD770A.1040001@gmail.com> Message-ID: <20050615190352.GA28330@discworld.dyndns.org> Nicolas Fleury wrote: > Guido van Rossum wrote: > > Why are you so excited about having until indented? You didn't give > > any examples with multiple occurrences. A single occurrence works just > > fine unindented, as PEP 315 has already shown. > > FWIW, I must say I disagree (about "works just fine"). I find PEP 315 > counter-intuitive. There's multi-part blocks in Python, but they all > behave like a step-sequence where you never come back to a previous > step. What I mean is that after entering a finally/except/elif/else, > there's no coming back in the try/if/for/while. My first impression > when looking at: > > do: > > while condition: > > > is that is executed only once. Indeed. The original poster seems to want something that would work (not necessarily look) like this: do: while with executed once prior to first being tested. But the above is ugly, and you can get much the same effect with Python today: firsttime = True while firsttime or : firsttime = False Seems fairly Pythonic to me. YMMV. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From jcarlson at uci.edu Wed Jun 15 21:38:16 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 15 Jun 2005 12:38:16 -0700 Subject: [Python-Dev] Wishlist: dowhile In-Reply-To: <20050615190352.GA28330@discworld.dyndns.org> References: <20050615190352.GA28330@discworld.dyndns.org> Message-ID: <20050615122018.726B.JCARLSON@uci.edu> Charles Cazabon wrote: > Indeed. The original poster seems to want something that would work (not > necessarily look) like this: > > do: > > while > > with executed once prior to first being tested. But the > above is ugly, and you can get much the same effect with Python today: > > firsttime = True > while firsttime or : > > firsttime = False > > Seems fairly Pythonic to me. YMMV. I agree. It also handles 'else' clauses very well. Heck, even Michael McLay's offering of ... while True: if : break ... doesn't look too bad, and supports the full range of pre and post condition code. Just like the indented "until", it may not be obvious where the breaking condition lies. However, a syntax highlighting editor and a properly located comment or two would solve the problem with "until" and the breaking "if" above. I'm of the opinion that the "do while" and "loop and a half" variants of the while loop should be offered as a recipe somewhere in the documentation. Perhaps not directly in the tutorial (maybe a link), describing quickly how you can use while loops to emulate the full spectrum of looping constructs. - Josiah From steven.bethard at gmail.com Thu Jun 16 03:17:53 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 15 Jun 2005 19:17:53 -0600 Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was: Wishlist: dowhile) In-Reply-To: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> Message-ID: Jp Calderone: > for chunk in iter(lambda: f1.read(CHUNK_SIZE), ''): > f2.write(chunk) Phillip J. Eby: > More seriously, I think your translation makes an excellent argument in > *favor* of having a do/while statement for greater clarity. :) Michael Chermside > Interesting... I had the opposite reaction. I, too, thought that Jp's solution was quite easy to read, though every few months I forget about iter()'s alternate form again. ;-) Perhaps one of the things that makes Jp's solution slightly hard to read is the fact that f1.read must be wrapped in a function (in Jp's example, a lambda). It reminds me of one of my minor gripes about the standard lib -- a number of functions that take another function as an argument don't take *args and **kwargs to be passed to that function when it's called. The iter() alternate form is a common example of this. I would prefer that the alternate iter() form was broken off into another separate function, say, iterfunc(), that would let me write Jp's solution something like: for chunk in iterfunc('', f1.read, CHUNK_SIZE): f2.write(chunk) Even better would be to add the *args and **kwargs to iter() directly, but of course, that would be backwards incompatible. Anyway, in general, I'd like to see the standard lib in Python 3K convert functions like iter()'s alternate form to functions like iterfunc(), i.e. provide *args and **kwargs whenever possible. Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From benji at benjiyork.com Thu Jun 16 05:07:05 2005 From: benji at benjiyork.com (Benji York) Date: Wed, 15 Jun 2005 23:07:05 -0400 Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was: Wishlist: dowhile) In-Reply-To: References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> Message-ID: <42B0ECD9.3060207@benjiyork.com> Steven Bethard wrote: > It reminds me of one of my minor gripes about the standard lib -- a > number of functions that take another function as an argument don't > take *args and **kwargs to be passed to that function when it's > called. The iter() alternate form is a common example of this. I > would prefer that the alternate iter() form was broken off into > another separate function, say, iterfunc(), that would let me write > Jp's solution something like: > > for chunk in iterfunc('', f1.read, CHUNK_SIZE): > f2.write(chunk) How about 2.5's "partial": for chunk in iter(partial(f1.read, CHUNK_SIZE), ''): f2.write(chunk) -- Benji York From mwh at python.net Thu Jun 16 11:51:13 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 16 Jun 2005 10:51:13 +0100 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <42B0725D.9040200@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Wed, 15 Jun 2005 20:24:29 +0200") References: <2mzmtrbxwo.fsf@starship.python.net> <42B0725D.9040200@v.loewis.de> Message-ID: <2m4qbybpf2.fsf@starship.python.net> "Martin v. L?wis" writes: > Michael Hudson wrote: > >> if (ProfilerError == NULL) >> ProfilerError = PyErr_NewException("hotshot.ProfilerError", >> NULL, NULL); >> if (ProfilerError != NULL) { >> Py_INCREF(ProfilerError); >> PyModule_AddObject(module, "ProfilerError", ProfilerError); >> } >> >> >> I think the Py_INCREF should just be removed, but I'm wondering if I'm >> missing something... > > It may be me who is missing something, but... Well, quite possibly not. > On reference is added to the dictionary, this is the one the explicit > INCREF creates. The other reference is held in the C variable > ProfilerError; this is the one that creating the exception object > creates. > > It is convention that C variables which are explicitly used also > hold their own references, even if they are global, and even if there > is no procedure to clear them. The reason is that they would become > stale if the module object went away. As there is no way to protect > against this case, they just keep a garbage reference. This means two things, as I see it: 1) Py_Initialize()/Py_Finalize() loops are going to leak quite a lot. Maybe we don't care about this. 2) In the case of the init_hotshot code above and such a loop, the ProfilerError object from the first interpreter will be reused by the second, which doesn't seem like a good idea (won't it be inheriting from the wrong PyExc_Exception?). Currently running Demo/embed/loop 'import gc' crashes for a similar kind of reason -- the gc.garbage object is shared between interpreters, but the only reference to it is in the module's __dict__ (well, if the module exists...). I've been looking at this area partly to try and understand this bug: [ 1163563 ] Sub threads execute in restricted mode but I'm not sure the whole idea of multiple interpreters isn't inherently doomed :-/ Cheers, mwh -- Premature optimization is the root of all evil. -- Donald E. Knuth, Structured Programming with goto Statements From faassen at infrae.com Thu Jun 16 12:49:44 2005 From: faassen at infrae.com (Martijn Faassen) Date: Thu, 16 Jun 2005 12:49:44 +0200 Subject: [Python-Dev] Thoughts on stdlib evolvement In-Reply-To: References: <20050610144835.GB5929@burma.localdomain> Message-ID: <42B15948.9060605@infrae.com> Fredrik Lundh wrote: [snip] > in my experience, any external library that supports more than one > Python version on more than one platform is likely to be more robust > than code in the core. add the multilevel volunteer approach de- > described by Steven (with the right infrastructure, things like that > just appear), and you get more competent manpower contributing > to the standard distribution than you can get in any other way. In this context PEP 2 might be useful to look at again: http://www.python.org/peps/pep-0002.html It separates between library integrators (into the Python standard library) and library maintainers, and tries to ensure maintenance happens on a continuing basis. A multi-level setup to develop the Python standard library could take other forms, of course. I sometimes feel the Python-dev community is more focused on the development of the interpreter than of the library, and that this priority tends to be reversed outside the Python-dev community. So, it might be nice if the Python standard library development integrators and maintainers could be more separate from the Python core developers. A python-library-dev, say. Then again, this shouldn't result in large changes in the standard library, as old things should continue to work for the forseeable future. So for larger reorganizations and refactorings, such development should likely take place entirely outside the scope of the core distribution, at least for the time being. Finally, I haven't really seen much in the way of effort by developers to actually do such a large-scale cleanup. Nobody seems to have stepped up, taking the standard library, and made it undergo a radical refactoring (and just releasing it separately). That this hasn't happened seems to indicate the priority is not very high in the mind of people, so the problem might not be high either. :) Regards, Martijn From barry at python.org Thu Jun 16 15:37:35 2005 From: barry at python.org (Barry Warsaw) Date: Thu, 16 Jun 2005 09:37:35 -0400 Subject: [Python-Dev] PEP 343 - next steps In-Reply-To: <42ABBF9F.1010801@gmail.com> References: <1118526223.9425.127.camel@geddy.wooz.org> <42ABBF9F.1010801@gmail.com> Message-ID: <1118929055.22208.19.camel@presto.wooz.org> On Sun, 2005-06-12 at 00:52, Nick Coghlan wrote: > The idea behind 'with' is that the block is executed while holding > (i.e. 'with') the resource. > > I think the '-ing' form of the example templates is a holdover from > the days when the PEP used the 'do' keyword - I find that past tense > or noun forms tend to read better than present tense for custom built > with templates named after the action that occurs on entry: > > # Past tense > with locked(my_lock): > with opened(my_file, mode): > with redirected_stdout(my_stream): Work for me. Thanks. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050616/44debb85/attachment.pgp From steven.bethard at gmail.com Thu Jun 16 16:50:42 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 16 Jun 2005 08:50:42 -0600 Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was: Wishlist: dowhile) In-Reply-To: <42B0ECD9.3060207@benjiyork.com> References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> <42B0ECD9.3060207@benjiyork.com> Message-ID: On 6/15/05, Benji York wrote: > Steven Bethard wrote: > > I would prefer that the alternate iter() form was broken off into > > another separate function, say, iterfunc(), that would let me write > > Jp's solution something like: > > > > for chunk in iterfunc('', f1.read, CHUNK_SIZE): > > f2.write(chunk) > > How about 2.5's "partial": > > for chunk in iter(partial(f1.read, CHUNK_SIZE), ''): > f2.write(chunk) Yeah, there are a number of workarounds. Using partial, def-ing a function, or using a lambda will all work. My point was that, with the right API, these workarounds wouldn't be necessary. Look at unittest.TestCase.assertRaises to see another example of the kind of API I think should be supported (when possible, of course). Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From mwh at python.net Thu Jun 16 17:04:53 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 16 Jun 2005 16:04:53 +0100 Subject: [Python-Dev] iter alternate form and *args and **kwargs In-Reply-To: (Steven Bethard's message of "Thu, 16 Jun 2005 08:50:42 -0600") References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> <42B0ECD9.3060207@benjiyork.com> Message-ID: <2m8y1a9wbu.fsf@starship.python.net> Steven Bethard writes: > On 6/15/05, Benji York wrote: >> Steven Bethard wrote: >> > I would prefer that the alternate iter() form was broken off into >> > another separate function, say, iterfunc(), that would let me write >> > Jp's solution something like: >> > >> > for chunk in iterfunc('', f1.read, CHUNK_SIZE): >> > f2.write(chunk) >> >> How about 2.5's "partial": >> >> for chunk in iter(partial(f1.read, CHUNK_SIZE), ''): >> f2.write(chunk) > > Yeah, there are a number of workarounds. Using partial, def-ing a > function, or using a lambda will all work. My point was that, with > the right API, these workarounds wouldn't be necessary. Well, I dunno. I can see where you're coming from, but I think you could make the argument that the form using partial is clearer to read -- it's not absolutely clear that the CHUNK_SIZE argument is intended to be passed to f1.read. Also, the partial approach works better when there is more than one callable. Cheers, mwh -- Like most people, I don't always agree with the BDFL (especially when he wants to change things I've just written about in very large books), ... -- Mark Lutz, http://python.oreilly.com/news/python_0501.html From foom at fuhm.net Thu Jun 16 17:43:34 2005 From: foom at fuhm.net (James Y Knight) Date: Thu, 16 Jun 2005 11:43:34 -0400 Subject: [Python-Dev] iter alternate form and *args and **kwargs (Was: Wishlist: dowhile) In-Reply-To: References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> <42B0ECD9.3060207@benjiyork.com> Message-ID: <081482E7-D65B-445A-8CC3-59C3A185029C@fuhm.net> On Jun 16, 2005, at 10:50 AM, Steven Bethard wrote: > On 6/15/05, Benji York wrote: > >> Steven Bethard wrote: >> >>> I would prefer that the alternate iter() form was broken off into >>> another separate function, say, iterfunc(), that would let me write >>> Jp's solution something like: >>> >>> for chunk in iterfunc('', f1.read, CHUNK_SIZE): >>> f2.write(chunk) >>> >> >> How about 2.5's "partial": >> >> for chunk in iter(partial(f1.read, CHUNK_SIZE), ''): >> f2.write(chunk) >> > > Yeah, there are a number of workarounds. Using partial, def-ing a > function, or using a lambda will all work. My point was that, with > the right API, these workarounds wouldn't be necessary. Look at > unittest.TestCase.assertRaises to see another example of the kind of > API I think should be supported (when possible, of course). I think it's really the other way around. Forcing every API that takes a callable to also take a *args, **kwargs is a workaround for not having partial. James From mwh at python.net Thu Jun 16 19:00:14 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 16 Jun 2005 18:00:14 +0100 Subject: [Python-Dev] Multiple interpreters not compatible with current thread module In-Reply-To: <93dc9c320505170425392e3d80@mail.gmail.com> (Jeremy Maxfield's message of "Tue, 17 May 2005 13:25:32 +0200") References: <93dc9c320505170425392e3d80@mail.gmail.com> Message-ID: <2m4qby9qzl.fsf@starship.python.net> Jeremy Maxfield writes: > The current threadmodule.c does not seem to correctly support multiple > (sub) interpreters. This would seem to be an accurate statement. A short history: The GILState functions were implemented. The way they work is that when you call PyGILState_Ensure, (essentially) a thread local variable is checked to see if a thread state is known for this thread. If one is found, fine, it is used. If not, one is created (using the interpreter state what was passed to _PyGILState_Init() by Py_Initialize()) and stored in the thread local variable. This had a (pretty obvious in hindsight) problem (bug #1010677): when you create a thread via thread.start_new_thread a thread state is created, but not stored in the thread local variable consulted by PyGILState_Ensure. So if you call PyGILState_Ensure another thread state is created for the thread (generally a no-no) and if you already hold the GIL PyGILState_Ensure will attempt to acquire it again -- a deadlock. This was fixed by essentially using PyGILState_Ensure to create the threadstate. This has a (pretty obvious in hindsight) problem (bug #1163563): PyGILState_Ensure always uses the interpreter state created by Py_Initialize, ignoring the interpreter state carefully supplied to t_bootstrap. Hilarity ensues. So, what's the fix? Well, the original problem was only the lack of association between a C level thread and a thread state. This can be fixed by setting up this association in t_bootstrap (and I've posted a patch that does this to the report of #1163563). This suffices for all known problems, but it's a bit hackish. Another approach is to set up this association PyThreadState_New(), which is possibly a bit more elegant, but carries a risk: is PyThreadState_New always called from the new thread? ISTM that it is, but I'm not sure. I'm not expecting anyone else to think hard about this on recent form, so I'll think about it for a bit and then fix it in the way that seems best after that. Feel free to surprise me. Cheers, mwh -- I would hereby duly point you at the website for the current pedal powered submarine world underwater speed record, except I've lost the URL. -- Callas, cam.misc From gvanrossum at gmail.com Thu Jun 16 20:41:05 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 16 Jun 2005 11:41:05 -0700 Subject: [Python-Dev] Please spread the word about OSCON early reg deadline In-Reply-To: <25fced5c5f9467929cacbd05a06c7438@oreilly.com> References: <49e0655542c64ebcabaf05533f0bfa48@oreilly.com> <200506160944.27095.josh@agliodbs.com> <20050616171341.GA1857@panix.com> <42B1A5F4.1090203@shiflett.org> <25fced5c5f9467929cacbd05a06c7438@oreilly.com> Message-ID: FYI. Check it out, the Python program is really good! ---------- Forwarded message ---------- From: Gina Blaber Date: Jun 16, 2005 11:31 AM Subject: [Oscon] please spread the word about OSCON early reg deadline To: OSCON Committee Mailing List Cc: Gina Blaber Can all of you on the OSCON program committee please take a moment today to either blog OSCON or mention it on some relevant lists? Timing is important because the Early Registration period (with the early reg discount) ends on Monday June 20 (end of day). It would be great if you could mention the early reg deadline. Here are few relevant links to include, if you're so inclined: http://conferences.oreillynet.com/os2005/index_new.csp (OSCON home page) http://conferences.oreillynet.com/pub/w/38/speakers.html (OSCON speakers) http://conferences.oreillynet.com/cs/os2005/create/ord_os05 (OSCON attendee registration page) Thanks, -Gina ------------------------------------------------------------------------ ------- Gina Blaber, Director of Conferences O'Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 gina at oreilly.com (707) 827-7185 http://conferences.oreilly.com _______________________________________________ Oscon mailing list Oscon at labs.oreilly.com http://labs.oreilly.com/mailman/listinfo/oscon -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Thu Jun 16 21:28:46 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 16 Jun 2005 13:28:46 -0600 Subject: [Python-Dev] iter alternate form and *args and **kwargs In-Reply-To: <2m8y1a9wbu.fsf@starship.python.net> References: <20050613123633.6z1g3d5izzco8cgg@login.werra.lunarpages.com> <42B0ECD9.3060207@benjiyork.com> <2m8y1a9wbu.fsf@starship.python.net> Message-ID: Steven Bethard wrote: > I would prefer that the alternate iter() form was broken off into > another separate function, say, iterfunc(), that would let me write > Jp's solution something like: > > for chunk in iterfunc('', f1.read, CHUNK_SIZE): > f2.write(chunk) Benji York wrote: > for chunk in iter(partial(f1.read, CHUNK_SIZE), ''): > f2.write(chunk) Steven Bethard wrote: > Yeah, there are a number of workarounds. Using partial, def-ing a > function, or using a lambda will all work. My point was that, with > the right API, these workarounds wouldn't be necessary. Michael Hudson wrote: > Well, I dunno. I can see where you're coming from, but I think you > could make the argument that the form using partial is clearer to read > -- it's not absolutely clear that the CHUNK_SIZE argument is intended > to be passed to f1.read. True, but the same argument could be made for unittest.TestCase.assertRaises, and I don't find that confusing at all. YMMV, of course. > Also, the partial approach works better when there is more than one > callable. Yeah, I thought of that. But how often are there multiple callables? Just scanning through the builtin functions I see that filter, iter, map, and reduce all take functions to be called, and none of them take multiple functions. (Note that classmethod, property and staticmethod are not good examples because they don't take callables to be *called*, they take callables to be *wrapped*.) OTOH, filter, map and reduce are all basically deprecated at this point thanks to list comprehensions and generator expressions, so I guess YMMV. Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From raymond.hettinger at verizon.net Fri Jun 17 02:24:49 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 16 Jun 2005 20:24:49 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 Message-ID: <003601c572d2$f9933540$bb07a044@oemcomputer> PEP 288 is now withdrawn. The generator exceptions portion is subsumed by PEP 343, and the generator attributes portion never garnered any support. The fate of generator attributes is interesting v?s-a-v?s PEP 342. The motivation was always related to supporting advanced generator uses such as emulating coroutines and writing generator based data consumer functions. At the time, Guido and everyone else found those use cases to be less than persuasive. Also, people countered that that functionality could be easily simulated with class based iterators, global variables, or passing a mutable argument to a generator. Amazingly, none of those objections seem to be directed toward 342 which somehow seems on the verge of acceptance even without use cases, clear motivation, examples, or a draft implementation. Looking back at the history of 288, generator attributes surfaced only in later drafts. In the earlier drafts, the idea for passing arguments to and from running generators used an argument to next() and a return value for yield. If this sounds familiar, it is because it is not much different from the new PEP 342 proposal. However, generator argument passing via next() was shot down early-on. The insurmountable concept flaw was an off-by-one issue. The very first call to next() does not correspond to a yield statement; instead, it corresponds to the first lines of a generator (those run *before* the first yield). All of the proposed use cases needed to have the data passed in earlier. With the death of that idea, generator attributes were born as a way of being able to pass in data before the first yield was encountered and to receive data after the yield. This was workable and satisfied the use cases. Coroutine simulations such as those in Dr Mertz's articles were easily expressed with generator attributes. As a further benefit, using attributes was a natural approach because that same technique has long been used with classes (so no new syntax was needed and the learning curve was zero). In contrast to PEP 288's low impact approach, PEP 342 changes the implementation of the for-loop, alters the semantics of "continue", introduces new and old-style iterators, and creates a new magic method. Meanwhile, it hasn't promised any advantages over the dead PEP 288 proposals. IOW, I don't follow how 342 got this far, how 342 intends to overcome the off-by-one issue, how it addresses all of the other objections leveled at the now dead PEP 288, and why no one appears concerned about introducing yet another new-style/old-style issue that will live in perpetuity. Raymond Sidenote: generator attributes also failed because generators lacked a sufficiently elegant way to refer to running instances of themselves (there is no self argument so we would need an access function or have a dynamic function attribute accessible only from within a running generator). From raymond.hettinger at verizon.net Fri Jun 17 03:03:57 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 16 Jun 2005 21:03:57 -0400 Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries by Value Message-ID: <004301c572d8$711919e0$bb07a044@oemcomputer> May I suggest rejecting PEP 265. As of Py2.4, its use case is easily solved with: >>> sorted(d.iteritems(), key=itemgetter(1), reverse=True) [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)] Further, Py2.5 offers a parallel solution to the more likely use case of wanting the access only the largest counts: >>> nlargest(2, d.iteritems(), itemgetter(1)) [('b', 23), ('d', 17)] Raymond From raymond.hettinger at verizon.net Fri Jun 17 03:13:31 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Thu, 16 Jun 2005 21:13:31 -0400 Subject: [Python-Dev] Propose to reject PEP 281 -- Loop Counter Iteration with range and xrange Message-ID: <004601c572d9$c7578ac0$bb07a044@oemcomputer> The need for the indices() proposal was mostly met by PEP 279's enumerate() builtin. Commenting on 279 before it was accepted for Py2.3, PEP 281's author, Magnus Lie Hetland, wrote, "I'm quite happy to have it make PEP 281 obsolete." Raymond From skip at pobox.com Fri Jun 17 03:38:08 2005 From: skip at pobox.com (Skip Montanaro) Date: Thu, 16 Jun 2005 20:38:08 -0500 Subject: [Python-Dev] PEP 304 "Controlling Generation of Bytecode Files" - patch updated Message-ID: <17074.10624.578497.392072@montanaro.dyndns.org> I updated the patch that supports PEP 304, "Controlling Generation of Bytecode Files" to apply cleanly against current CVS. I've tested it on Mac OS X (straight Unix build only). I'd appreciate it if some Linux, Windows and Mac framework folks could apply the patch, rebuild, then run the tests (there is a "testbcb" target in the Makefile that should give Windows people an idea what to do). The patch is attached to http://python.org/sf/677103 Now that I think about it, there is probably a file in the Windows build tree (equivalent of pyconfig.h?) that still needs to be updated. The text of the PEP has not been updated in awhile. I will try to look at that in the next couple of days. I'd appreciate some critical review by people with Windows filesystem experience. There was a comment ages ago about problems with this scheme due to Windows multi-rooted directory tree that I can no longer find (and failed to record in the PEP at the time). I'd like to see if the problem can be resurrected then addressed. Thanks, Skip From steven.bethard at gmail.com Fri Jun 17 03:39:11 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 16 Jun 2005 19:39:11 -0600 Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries by Value In-Reply-To: <004301c572d8$711919e0$bb07a044@oemcomputer> References: <004301c572d8$711919e0$bb07a044@oemcomputer> Message-ID: Raymond Hettinger wrote: > May I suggest rejecting PEP 265. > > As of Py2.4, its use case is easily solved with: > > >>> sorted(d.iteritems(), key=itemgetter(1), reverse=True) > [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)] +1. I find that usually when I want something like this, I use: sorted(d, key=d.__getitem__, reverse=True) because it doesn't require the operator module and most of the time I just need the keys anyway. py> sorted(d, key=d.__getitem__, reverse=True) ['b', 'd', 'c', 'a', 'e'] Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From pje at telecommunity.com Fri Jun 17 03:53:11 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 16 Jun 2005 21:53:11 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <003601c572d2$f9933540$bb07a044@oemcomputer> Message-ID: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> At 08:24 PM 6/16/2005 -0400, Raymond Hettinger wrote: >Looking back at the history of 288, generator attributes surfaced only >in later drafts. In the earlier drafts, the idea for passing arguments >to and from running generators used an argument to next() and a return >value for yield. If this sounds familiar, it is because it is not much >different from the new PEP 342 proposal. However, generator argument >passing via next() was shot down early-on. The insurmountable concept >flaw was an off-by-one issue. The very first call to next() does not >correspond to a yield statement; instead, it corresponds to the first >lines of a generator (those run *before* the first yield). All of the >proposed use cases needed to have the data passed in earlier. Huh? I don't see why this is a problem. PEP 342 says: """When the *initial* call to __next__() receives an argument that is not None, TypeError is raised; this is likely caused by some logic error.""" >With the death of that idea, generator attributes were born as a way of >being able to pass in data before the first yield was encountered and to >receive data after the yield. This was workable and satisfied the use >cases. Coroutine simulations such as those in Dr Mertz's articles were >easily expressed with generator attributes. As a further benefit, using >attributes was a natural approach because that same technique has long >been used with classes (so no new syntax was needed and the learning >curve was zero). Ugh. Having actually emulated co-routines using generators, I have to tell you that I don't find generator attributes natural for this at all; returning a value or error (via PEP 343's throw()) from a yield expression as in PEP 342 is just what I've been wanting. >In contrast to PEP 288's low impact approach, PEP 342 changes the >implementation of the for-loop, alters the semantics of "continue", >introduces new and old-style iterators, and creates a new magic method. I could definitely go for dropping __next__ and the next() builtin from PEP 342, as they don't do anything extra. I also personally don't care about the new continue feature, so I could do without for-loop alteration too. I'd be perfectly happy passing arguments to next() explicitly; I just want yield expressions. >Meanwhile, it hasn't promised any advantages over the dead PEP 288 >proposals. Reading the comments in PEP 288's revision history, it sounds like the argument was to postpone implementation of next(arg) and yield expressions to a later version of Python, after more community experience with generators. We've had that experience now. >IOW, I don't follow how 342 got this far, how 342 intends to overcome >the off-by-one issue, It explicitly addresses it already. > how it addresses all of the other objections >leveled at the now dead PEP 288 Arguments for waiting aren't the same thing as arguments for never doing. I interpret the comments in 288's history as ranging from -0 to +0 on the yield expr/next(arg) issue, and didn't see any -1's except on the generator attribute concept. >and why no one appears concerned about >introducing yet another new-style/old-style issue that will live in >perpetuity. I believe it has been brought up before, and I also believe I pointed out once or twice that __next__ wasn't needed. I think Guido even mentioned something to that effect himself, but everybody was busy with PEP 340-inspired ideas at the time. 342 was split off in part to avoid losing the ideas that were in it. From python at rcn.com Fri Jun 17 04:26:19 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu, 16 Jun 2005 22:26:19 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> Message-ID: <005301c572e3$f2518aa0$bb07a044@oemcomputer> [Phillip] > I could definitely go for dropping __next__ and the next() builtin from > PEP > 342, as they don't do anything extra. I also personally don't care about > the new continue feature, so I could do without for-loop alteration > too. I'd be perfectly happy passing arguments to next() explicitly; I > just > want yield expressions. That's progress! Please do what you can to get the non-essential changes out of 342. > >Meanwhile, it hasn't promised any advantages over the dead PEP 288 > >proposals. > > Reading the comments in PEP 288's revision history, it sounds like the > argument was to postpone implementation of next(arg) and yield expressions > to a later version of Python, after more community experience with > generators. We've had that experience now. 288 was brought out of retirement a few months ago. Guido hated every variation of argument passing and frequently quipped that data passing was trivially accomplished though mutable arguments to a generator, through class based iterators, or via a global variable. I believe all of those comments were made recently and they all apply equally to 342. Raymond From pje at telecommunity.com Fri Jun 17 05:05:17 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 16 Jun 2005 23:05:17 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <005301c572e3$f2518aa0$bb07a044@oemcomputer> References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050616225814.01e1ba98@mail.telecommunity.com> At 10:26 PM 6/16/2005 -0400, Raymond Hettinger wrote: >288 was brought out of retirement a few months ago. Guido hated every >variation of argument passing and frequently quipped that data passing >was trivially accomplished though mutable arguments to a generator, >through class based iterators, or via a global variable. I believe all >of those comments were made recently and they all apply equally to 342. Clearly, then, he's since learned the error of his ways. :) More seriously, I would say that data passing is not the same thing as coroutine suspension, and that PEP 340 probably gave Guido a much better look at at least one use case for the latter. In the meantime, I applaud your foresight in having invented significant portions of PEP 343 years ahead of time. Now give Guido back his time machine, please. :) If you hadn't borrowed it to write the earlier PEP, he could have seen for himself that all this would happen, and neatly avoided it by just approving PEP 288 to start with. :) From gvanrossum at gmail.com Fri Jun 17 05:03:49 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 16 Jun 2005 20:03:49 -0700 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <005301c572e3$f2518aa0$bb07a044@oemcomputer> References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> <005301c572e3$f2518aa0$bb07a044@oemcomputer> Message-ID: On 6/16/05, Raymond Hettinger wrote: > [Phillip] > > I could definitely go for dropping __next__ and the next() builtin from PEP > > 342, as they don't do anything extra. I also personally don't care about > > the new continue feature, so I could do without for-loop alteration > > too. I'd be perfectly happy passing arguments to next() explicitly; I just > > want yield expressions. > > That's progress! Please do what you can to get the non-essential > changes out of 342. Here's my current position: instead of g.__next__(arg) I'd like to use g.next(arg). The next() builtin then isn't needed. I do like "continue EXPR" but I have to admit I haven't even tried to come up with examples -- it may be unnecessary. As Phillip says, yield expressions and g.next(EXPR) are the core -- and also incidentally look like they will cause the most implementation nightmares. (If someone wants to start implementing these two now, go right ahead!) > > >Meanwhile, it hasn't promised any advantages over the dead PEP 288 > > >proposals. > > > > Reading the comments in PEP 288's revision history, it sounds like the > > argument was to postpone implementation of next(arg) and yield expressions > > to a later version of Python, after more community experience with > > generators. We've had that experience now. > > 288 was brought out of retirement a few months ago. Guido hated every > variation of argument passing and frequently quipped that data passing > was trivially accomplished though mutable arguments to a generator, > through class based iterators, or via a global variable. I believe all > of those comments were made recently and they all apply equally to 342. That was all before I (re-)discovered yield-expressions (in Ruby!), and mostly in response to the most recent version of PEP 288, with its problem of accessing the generator instance. I now strongly feel that g.next(EXPR) and yield-expressions are the way to go. Making g.next(EXPR) an error when this is the *initial* resumption of the frame was also a (minor) breakthrough. Any data needed by the generator at this point can be passed in as an argument to the generator. Someone should really come up with some realistic coroutine examples written using PEP 342 (with or without "continue EXPR"). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Fri Jun 17 06:00:49 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 16 Jun 2005 21:00:49 -0700 Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries by Value In-Reply-To: References: <004301c572d8$711919e0$bb07a044@oemcomputer> Message-ID: Agreed. I don't want to add sorting abilities (with all its infinite variants) to every data structure -- or even one or two common data structures. You want something sorted that's not already a list? Use the sorted() method. On 6/16/05, Steven Bethard wrote: > Raymond Hettinger wrote: > > May I suggest rejecting PEP 265. > > > > As of Py2.4, its use case is easily solved with: > > > > >>> sorted(d.iteritems(), key=itemgetter(1), reverse=True) > > [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)] > > +1. > > I find that usually when I want something like this, I use: > sorted(d, key=d.__getitem__, reverse=True) > because it doesn't require the operator module and most of the time I > just need the keys anyway. > > py> sorted(d, key=d.__getitem__, reverse=True) > ['b', 'd', 'c', 'a', 'e'] > > Steve > -- > You can wordify anything if you just verb it. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Fri Jun 17 06:01:38 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 16 Jun 2005 21:01:38 -0700 Subject: [Python-Dev] Propose to reject PEP 265 -- Sorting Dictionaries by Value In-Reply-To: References: <004301c572d8$711919e0$bb07a044@oemcomputer> Message-ID: On 6/16/05, Guido van Rossum wrote: > Agreed. I don't want to add sorting abilities (with all its infinite > variants) to every data structure -- or even one or two common data > structures. You want something sorted that's not already a list? Use > the sorted() method. I meant the sorted() function, of course. Java on my mind. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Jun 17 06:07:22 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 17 Jun 2005 00:07:22 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: References: <005301c572e3$f2518aa0$bb07a044@oemcomputer> <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> <005301c572e3$f2518aa0$bb07a044@oemcomputer> Message-ID: <5.1.1.6.0.20050616233011.01e26378@mail.telecommunity.com> At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote: >Someone should really come up with some realistic coroutine examples >written using PEP 342 (with or without "continue EXPR"). How's this? def echo(sock): while True: try: data = yield nonblocking_read(sock) yield nonblocking_write(sock, data) except ConnectionLost: pass def run_server(sock, handler): while True: connected_socket = yield nonblocking_accept(sock) schedule_coroutine(handler(connected_socket)) schedule_coroutine( run_server( setup_listening_socket("localhost","echo"), echo ) Of course, I'm handwaving a lot here, but this is a much clearer example than anything I tried to pull out of the coroutines I've written for actual production use. That is, I originally started this email with a real routine from a complex multiprocess application doing lots of IPC, and quickly got bogged down in explaining all the details of things like yielding to semaphores and whatnot. But I can give you that example too, if you like. Anyway, the handwaving above is only in explanation of details, not in their implementability. It would be pretty straightforward to use Twisted's callback facilities to trigger next() or throw() calls to resume the coroutine in progress. In fact, schedule_coroutine is probably implementable as something like this in Twisted: def schedule_coroutine(geniter, *arg): def resume(): value = geniter.next(*arg) if value is not None: schedule_coroutine(value) reactor.callLater(0, resume) This assumes, of course, that you only yield between coroutines. A better implementation would need to be more like the events.Task class in peak.events, which can handle yielding to Twisted's "Deferreds" and various other kinds of things that can provide callbacks. But this snippet is enough to show that yield expressions let you write event-driven code without going crazy writing callback functions. And of course, you can do this without yield expressions today, with a suitably magic function, but it doesn't read as well: yield nonblocking_accept(sock); connected_socket = events.resume() This is how I actually do this stuff today. 'events.resume()' is a magic function that uses sys._getframe() to peek at the argument passed to the equivalent of 'next()' on the Task that wraps the generator. events.resume() can also raise an error if the equivalent of 'throw()' was called instead. With yield expressions, the code in those Task methods would just do next(arg) or throw(*sys.exc_info()) on the generator-iterator, and 'events.resume()' and its stack hackery could go away. From gvanrossum at gmail.com Fri Jun 17 06:03:51 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 16 Jun 2005 21:03:51 -0700 Subject: [Python-Dev] Propose to reject PEP 281 -- Loop Counter Iteration with range and xrange In-Reply-To: <004601c572d9$c7578ac0$bb07a044@oemcomputer> References: <004601c572d9$c7578ac0$bb07a044@oemcomputer> Message-ID: On 6/16/05, Raymond Hettinger wrote: > The need for the indices() proposal was mostly met by PEP 279's > enumerate() builtin. > > Commenting on 279 before it was accepted for Py2.3, PEP 281's author, > Magnus Lie Hetland, wrote, "I'm quite happy to have it make PEP 281 > obsolete." Yes please. These examples are especially jarring: >>> range(range(5), range(10), range(2)) [5, 7, 9] (etc.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Jun 17 06:41:43 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 17 Jun 2005 00:41:43 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <5.1.1.6.0.20050616233011.01e26378@mail.telecommunity.com> References: <005301c572e3$f2518aa0$bb07a044@oemcomputer> <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> <005301c572e3$f2518aa0$bb07a044@oemcomputer> Message-ID: <5.1.1.6.0.20050617000918.01e1b840@mail.telecommunity.com> At 12:07 AM 6/17/2005 -0400, Phillip J. Eby wrote: > def schedule_coroutine(geniter, *arg): > def resume(): > value = geniter.next(*arg) > if value is not None: > schedule_coroutine(value) > reactor.callLater(0, resume) Oops. I just realized that this is missing a way to return a value back to a calling coroutine, and that I also forgot to handle exceptions: def schedule_coroutine(coroutine, stack=(), *args): def resume(): try: if len(args)==3: value = coroutine.throw(*args) else: value = coroutine.next(*args) except: if stack: # send the error back to the "calling" coroutine schedule_coroutine(stack[0], stack[1], *sys.exc_info()) else: # Nothing left in this pseudothread, let the # event loop handle it raise if isinstance(value,types.GeneratorType): # Yielded to a specific coroutine, push the current # one on the stack, and call the new one with no args schedule_coroutine(value, (coroutine,stack)) elif stack: # Yielded a result, pop the stack and send the # value to the caller schedule_coroutine(stack[0], stack[1], value) # else: this pseudothread has ended reactor.callLater(0, resume) There, that's better. Now, if a coroutine yields a coroutine, the yielding coroutine is pushed on a stack. If a coroutine yields a non-coroutine value, the stack is popped and the value returned to the previously-suspended coroutine. If a coroutine raises an exception, the stack is popped and the exception is thrown to the previously-suspended coroutine. This little routine basically replaces a whole bunch of code in peak.events that manages a similar coroutine stack right now, but is complicated by the absence of throw() and next(arg); the generators have to be wrapped by objects that add equivalent functionality, and the whole thing gets a lot more complicated as a result. Note that we could add a version of the above to the standard library without using Twisted. A simple loop class could have a deque of "callbacks to invoke", and the reactor.callLater() could be replaced by appending the 'resume' closure to the deque. A main loop function would then just peel items off the deque and call them, looping until an unhandled exception (such as SystemExit) occurs, or some other way of indicating an exit occurs. From python at rcn.com Fri Jun 17 06:43:32 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri, 17 Jun 2005 00:43:32 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: Message-ID: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer> [Phillip] > > I also personally don't care about the new continue feature, > > so I could do without for-loop alteration too. [Guido] > I do like "continue EXPR" but I have to admit I haven't even tried to > come up with examples -- it may be unnecessary. As Phillip says, yield > expressions and g.next(EXPR) are the core -- and also incidentally > look like they will cause the most implementation nightmares. Let me go on record as a strong -1 for "continue EXPR". The for-loop is our most basic construct and is easily understood in its present form. The same can be said for "continue" and "break" which have the added advantage of a near zero learning curve for people migrating from other languages. Any urge to complicate these basic statements should be seriously scrutinized and held to high standards of clarity, explainability, obviousness, usefulness, and necessity. IMO, it fails most of those tests. I would not look forward to explaining "continue EXPR" in the tutorial and think it would stand out as an anti-feature. Raymond From raymond.hettinger at verizon.net Fri Jun 17 07:19:51 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 17 Jun 2005 01:19:51 -0400 Subject: [Python-Dev] Propose to reject PEP 276 -- Simple iterator for ints Message-ID: <000201c572fc$30d94e80$bb07a044@oemcomputer> The principal use case was largely met by enumerate(). From PEP 276's rationale section: """ A common programming idiom is to take a collection of objects and apply some operation to each item in the collection in some established sequential order. Python provides the "for in" looping control structure for handling this common idiom. Cases arise, however, where it is necessary (or more convenient) to access each item in an "indexed" collection by iterating through each index and accessing each item in the collection using the corresponding index. """ Also, while some nice examples are provided, the proposed syntax allows and encourages some horrid examples as well: >>> for i in 3: print i 0 1 2 The backwards compatability section lists another problematic consequence; the following would stop being a syntax error and would become valid: x, = 1 The proposal adds iterability to all integers but silently does nothing for negative values. A minor additional concern is that floats are not given an equivalent capability (for obvious reasons) but this breaks symmetry with range/xrange which still accept float args. Raymond From gvanrossum at gmail.com Fri Jun 17 08:09:54 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 16 Jun 2005 23:09:54 -0700 Subject: [Python-Dev] Propose to reject PEP 276 -- Simple iterator for ints In-Reply-To: <000201c572fc$30d94e80$bb07a044@oemcomputer> References: <000201c572fc$30d94e80$bb07a044@oemcomputer> Message-ID: I've never liked that idea. Down with it! On 6/16/05, Raymond Hettinger wrote: > The principal use case was largely met by enumerate(). From PEP 276's > rationale section: > > """ > A common programming idiom is to take a collection of objects and apply > some operation to each item in the collection in some established > sequential order. Python provides the "for in" looping control > structure for handling this common idiom. Cases arise, however, where > it is necessary (or more convenient) to access each item in an "indexed" > collection by iterating through each index and accessing each item in > the collection using the corresponding index. > """ > > Also, while some nice examples are provided, the proposed syntax allows > and encourages some horrid examples as well: > > >>> for i in 3: print i > 0 > 1 > 2 > > The backwards compatability section lists another problematic > consequence; the following would stop being a syntax error and would > become valid: > > x, = 1 > > The proposal adds iterability to all integers but silently does nothing > for negative values. > > A minor additional concern is that floats are not given an equivalent > capability (for obvious reasons) but this breaks symmetry with > range/xrange which still accept float args. > > > Raymond > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond.hettinger at verizon.net Fri Jun 17 08:39:22 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 17 Jun 2005 02:39:22 -0400 Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral Literals to Python Message-ID: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> While the majority of Python users deem this to be a nice-to-have feature, the community has been unable to reach a consensus on the proper syntax after more than two years of intensive debate (the PEP was introduced in early April 2003). Most agree that there should be only-one-way-to-do-it; however, the proponents are evenly split into two camps, with the modernists preferring IX for nine and the classicists preferring VIIII which was the most likely spelling in ancient Rome. The classicists not only rely on set-in-stone tradition, they point to pragmatic issues such as avoidance of subtraction, ease of coding, easier mental parsing (much less error prone), and ease of teaching to beginners. They assert that the modernists have introduced unnecessary algorithmic complexity just to save two keystrokes. The modernists point to compatible Java implementations and current grade school textbooks. They believe that users from other languages will expect the IX form. Note however, not all the modernists agree on whether MXM would be a well-formed spelling of 1990; most, but not all prefer MCMXC despite its likelihood of being mis-parsed on a first reading. There is also a small but vocal user group demanding that lowercase forms be allowed. Their use cases fall into four categories: (i) academia, (ii) the legal profession, (iii) research paper writing, and (iv) powerpoint slideshows. Reportedly, this is also a common convention among Perl programmers. Links: http://hrsbstaff.ednet.ns.ca/waymac/History%20A/A%20Term%201/1.%20Rome/R oman_Numerals.htm http://www.sizes.com/numbers/roman_numerals.htm Raymond From gvanrossum at gmail.com Fri Jun 17 08:43:12 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu, 16 Jun 2005 23:43:12 -0700 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer> References: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer> Message-ID: [Raymond] > Let me go on record as a strong -1 for "continue EXPR". The for-loop is > our most basic construct and is easily understood in its present form. > The same can be said for "continue" and "break" which have the added > advantage of a near zero learning curve for people migrating from other > languages. > > Any urge to complicate these basic statements should be seriously > scrutinized and held to high standards of clarity, explainability, > obviousness, usefulness, and necessity. IMO, it fails most of those > tests. > > I would not look forward to explaining "continue EXPR" in the tutorial > and think it would stand out as an anti-feature. You sometimes seem to compound a rational argument with too much rhetoric. The correct argument against "continue EXPR" is that there are no use cases yet; if there were a good use case, the explanation would follow easily. The original use case (though not presented in PEP 340) was to serve as the equivalent to "return EXPR" in a Ruby block. In Ruby you have something like this (I probably get the syntax wrong): a.foreach() { |x| ...some code... } This executes the block for each item in a, with x (a formal parameter to the block) set to each consecutive item. In Python we would write it like this of course: for x in a: ...some code... In Ruby, the block is an anonymous procedure (a thunk) and foreach() a method that receives a margic (anonymous) parameter which is the thunk. Inside foreach(), you write "yield EXPR" which calls the block with x set to EXPR. When the block contains a return statement, the return value is delivered to the foreach() method as the return value of yield, which can be assigned like this: VAR = yield EXPR Note that Ruby's yield is just a magic call syntax that calls the thunk! But this means that the thunks can be used for other purposes as well. One common use is to have the block act as a Boolean function that selects items from a list; this way you could write filter() with an inline selection, for example (making this up): a1 = a.filter() { |x| return x > 0 } might set a1 to the list of a's elements that are > 0. (Not saying that this is a built-in array method in Ruby, but I think you could write one.) This particular example doesn't translate well into Python because a for-loop doesn't have a return value. Maybe that would be a future possibility if yield-expressions become accepted (just kidding:-). However, I can see other uses for looping over a sequence using a generator and telling the generator something interesting about each of the sequence's items, e.g. whether they are green, or should be printed, or which dollar value they represent if any (to make up a non-Boolean example). Anyway, "continue EXPR" was born as I was thinking of a way to do this kind of thing in Python, since I didn't want to give up return as a way of breaking out of a loop (or several!) and returning from a function. But I'm the first to admit that the use case is still very much hypothetical -- unlike that for g.next(EXPR) and VAR = yield. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond.hettinger at verizon.net Fri Jun 17 08:55:26 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 17 Jun 2005 02:55:26 -0400 Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable Message-ID: <000b01c57309$8b347d20$bb07a044@oemcomputer> After nine months, no support has grown beyond the original poster. The PEP did however generate some negative responses when brought-up on comp.lang.python (it made some people's stomach churn). The PEP fails the tests of obviousness and necessity. The PEP's switch example is easily handled by replacing the callable None with a simple, null lambda: def __call__(self, input): return {1 : self.a, 2 : self.b, 3 : self.c }.get(input, lambda *args: 0)(input) The PEP does not address conflicts with other uses of None (such as default arguments or indicating values that are not applicable). Mysteriously, the PEP consumes only *args but not **kwargs. It also fails with respect to clarity and explicitness. Defining a short, explicit default function is a hands-down winner in these two important measures of merit. Raymond From raymond.hettinger at verizon.net Fri Jun 17 10:36:01 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 17 Jun 2005 04:36:01 -0400 Subject: [Python-Dev] Propose rejection of PEPs 239 and 240 -- a builtin rational type and rational literals Message-ID: <000e01c57317$984f40e0$bb07a044@oemcomputer> These PEPs are four years old. Nothing is intrinsically wrong with them, but they have garnered little enthusiasm, discussion, or support, suggesting that the original need was somewhat modest. In addition, the principal (but not only) use cases for a builtin rational type and corresponding rational literal have already been addressed by the decimal module (and the expected future integration of decimal literals). From rationale section of the PEPs: """ Rational numbers are useful for exact and unsurprising arithmetic. They give the correct results people have been taught in various math classes. Making the "obvious" non-integer type one with more predictable semantics will surprise new programmers less than using floating point numbers. As quite a few posts on c.l.py and on tutor at python.org have shown, people often get bit by strange semantics of floating point numbers: for example, round(0.98, 2) still gives 0.97999999999999998. """ The future direction of the decimal module likely entails literals in the form of 123.45d with binary floats continuing to have the form 123.45. This conflicts with the rational literal proposal of having 123.45 interpreted as 123 + 45/100. There may still be a use for a rational module in the standard library, but builtin support is no longer needed or desirable. The PEPs also touch on division ambiguities which were largely resolved by the acceptance of PEP 238 which introduced the floor division operator and from __future__ import division. The rational proposal also has an intrinsic weakness shared with Java's prior versions of BigDecimal which they found to be unworkable in practice. The weakness was that repeated operations caused the internal number of digits to grow beyond reason. For this reason, the PEP proposes some arbitrary level of truncation which conflicts with the goals of having "obvious and exact" arithmetic. The proposed truncation methodology was undeveloped and made no proposal for the same fine level of control as its counterpart in the decimal module where issues of variable precision, multiple contexts, and alternate rounding modes have been fully thought out. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050617/8367e132/attachment.htm From Michaels at rd.bbc.co.uk Fri Jun 17 11:12:25 2005 From: Michaels at rd.bbc.co.uk (Michael Sparks) Date: Fri, 17 Jun 2005 10:12:25 +0100 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> Message-ID: <200506171012.26036.Michaels@rd.bbc.co.uk> At 08:24 PM 6/16/2005 -0400, Raymond Hettinger wrote: > As a further benefit, using >attributes was a natural approach because that same technique has long >been used with classes (so no new syntax was needed and the learning >curve was zero). On Friday 17 Jun 2005 02:53, Phillip J. Eby wrote: > Ugh. Having actually emulated co-routines using generators, I have to tell > you that I don't find generator attributes natural for this at all; > returning a value or error (via PEP 343's throw()) from a yield expression > as in PEP 342 is just what I've been wanting. We've been essentially emulating co-routines using generators embedded into a class to give us the equivalent of generator attributes. We've found this very natural for system composition. (Essentially it's a CSP type system, though with an aim of ease of use) I've written up my talk from ACCU/Python UK this year, and it's available here: http://www.bbc.co.uk/rd/pubs/whp/whp113.shtml I'll also be talking about it at Europython later this month. At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote: >Someone should really come up with some realistic coroutine examples >written using PEP 342 (with or without "continue EXPR"). On Friday 17 Jun 2005 05:07:22, Phillip J. Eby wrote: > How's this? > > def echo(sock): > while True: > try: > data = yield nonblocking_read(sock) > yield nonblocking_write(sock, data) ... snip ... For comparison, our version of this would be: from Axon.Component import component from Kamaelia.SimpleServerComponent import SimpleServer class Echo(component): def mainBody(self): while True: if self.dataReady("inbox"): self.send(data,"outbox") yield1 SimpleServer(protocol=EchoProtocol, port=1501).run() For more interesting pipelines we have: pipeline(TCPClient("127.0.0.1",1500), VorbisDecode(), AOAudioPlaybackAdaptor() ).run() Which works in the same way as a Unix pipeline. I haven't written the "pipegraph" or similar component yet that could allow this: graph(A=SingleServer("0.0.0.0", 1500), B=Echo(), layout = { "A:outbox": "B:inbox", "B:outbox" : "A:inbox" } ) (Still undecided on API for that really, currently the above is a lot more verbose -) By contrast I really can't see how passing attributes in via .next() helps this approach in any way (Not that that's a problem for us :). I CAN see though it helps if you're taking the approach for generator composition if you're using twisted.flow (though I'll defer a good example for that to someone else since although I've been asked for a comparison in the past, I don't think I'm sufficiently twisted to do so!). Michael. -- Michael Sparks, Senior R&D Engineer, Digital Media Group Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/ British Broadcasting Corporation, Research and Development Kingswood Warren, Surrey KT20 6NP This e-mail may contain personal views which are not the views of the BBC. From raymond.hettinger at verizon.net Fri Jun 17 11:57:22 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 17 Jun 2005 05:57:22 -0400 Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod() for Multiple Divisors Message-ID: <001d01c57322$f5af0ee0$bb07a044@oemcomputer> This PEP has been open for two and half years without generating discussion or support. Its primary case (converting cumulative seconds into a tuple days, hours, minutes, and seconds) is a bit wanting because it doesn't generalize to months and years. That need is already met in a more robust and flexible way by date and time specific modules. The use case is also somewhat unique. Recalling 25 years of programming, almost every real case of repeated divmod() calls have been in a loop with a single constant denominator (i.e. dividing by a base in a radix conversion). The PEP does suggest other applications but they are all narrow offerings (gallon/quart/pint/ounce, miles/yards/feet, pound/shilling/pence) that are extremely rare in real apps. More importantly, the gain in succinctness is offset by a loss of clarity. Consider: dist, inches = divmod(dist, 12) yards, feet = divmod(dist, 3) versus a proposed: yards, feet, inches = divmod(dist, 3, 12) The latter form makes it difficult to visually confirm the correct number of target variables. Likewise, it is not at all obvious that the order of the 3 and 12 are correct. Users from other languages will tend to automatically understand the current form and be mystified by the second (especially given how infrequently it will arise). The PEP draws its inspiration from an APL operator. APL imbues many of its operators with scalar/scalar, scalar/array, and array/array capabilities. But in Python (not numeric) we tend to leave the operators in simple form and abstract the vectorization into operator independent primitives (i.e. map and reduce). Mathematica takes a similar approach by offering functions like NestList(). So, instead of a vectorized divmod(), it is wiser for someone to post a single accumulation recipe that would work with a variety of binary operators. Executive summary: cute, but unpersuasive and unnecessary, not worth the time to code, test, document, maintain, and explain. Raymond From gjc at inescporto.pt Fri Jun 17 12:29:49 2005 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Fri, 17 Jun 2005 11:29:49 +0100 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <200506171012.26036.Michaels@rd.bbc.co.uk> References: <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> <200506171012.26036.Michaels@rd.bbc.co.uk> Message-ID: <1119004190.648.13.camel@localhost> Hello, I found your paper very interesting. I have also written a very minimalistic white paper, mostly aimed at the PyGTK community, with a small module for pseudo-threads using python generators: http://www.gnome.org/~gjc/gtasklet/gtasklets.html I don't have time to follow this whole discussion, but I leave it here as another example of python pseudo-threads. I also am very much in favour of having yield receive return values or exceptions, as this would make pseudo-threads much more elegant. And I very much wish python had this builtin or in std library. In conjunction with pseudo-threads, I think a "python main loop" implementation is fundamental. Such main loop with permit the programmer to register callbacks for events, such as timeouts, IO conditions, idle tasks, etc., such as one found glib (gtk+'s underlying library). I already pointed out one such implementation that I use for one of my projects, and it already has unit tests to prove that it works. This is also related to the "deprecate asyncore/asynchat" discussions going on earlier. IMHO, they should really be deprecated, and a pseudo-threads solution could be used instead. Anyway, I'd love to help more in this area, but unfortunately I don't have time for these endless discussions... :P Best regards. On Fri, 2005-06-17 at 10:12 +0100, Michael Sparks wrote: > At 08:24 PM 6/16/2005 -0400, Raymond Hettinger wrote: > > As a further benefit, using > >attributes was a natural approach because that same technique has long > >been used with classes (so no new syntax was needed and the learning > >curve was zero). > > On Friday 17 Jun 2005 02:53, Phillip J. Eby wrote: > > Ugh. Having actually emulated co-routines using generators, I have to tell > > you that I don't find generator attributes natural for this at all; > > returning a value or error (via PEP 343's throw()) from a yield expression > > as in PEP 342 is just what I've been wanting. > > We've been essentially emulating co-routines using generators embedded > into a class to give us the equivalent of generator attributes. We've found > this very natural for system composition. (Essentially it's a CSP type system, > though with an aim of ease of use) > > I've written up my talk from ACCU/Python UK this year, and it's available > here: http://www.bbc.co.uk/rd/pubs/whp/whp113.shtml > > I'll also be talking about it at Europython later this month. > > At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote: > >Someone should really come up with some realistic coroutine examples > >written using PEP 342 (with or without "continue EXPR"). > > On Friday 17 Jun 2005 05:07:22, Phillip J. Eby wrote: > > How's this? > > > > def echo(sock): > > while True: > > try: > > data = yield nonblocking_read(sock) > > yield nonblocking_write(sock, data) > ... snip ... > > For comparison, our version of this would be: > > from Axon.Component import component > from Kamaelia.SimpleServerComponent import SimpleServer > class Echo(component): > def mainBody(self): > while True: > if self.dataReady("inbox"): > self.send(data,"outbox") > yield1 > > SimpleServer(protocol=EchoProtocol, port=1501).run() > > > For more interesting pipelines we have: > > pipeline(TCPClient("127.0.0.1",1500), > VorbisDecode(), > AOAudioPlaybackAdaptor() > ).run() > > Which works in the same way as a Unix pipeline. I haven't written the > "pipegraph" or similar component yet that could allow this: > > graph(A=SingleServer("0.0.0.0", 1500), > B=Echo(), > layout = { "A:outbox": "B:inbox", "B:outbox" : "A:inbox" } ) > > (Still undecided on API for that really, currently the above is a lot more > verbose -) > > By contrast I really can't see how passing attributes in via .next() helps > this approach in any way (Not that that's a problem for us :). > > I CAN see though it helps if you're taking the approach for generator > composition if you're using twisted.flow (though I'll defer a good example > for that to someone else since although I've been asked for a comparison in > the past, I don't think I'm sufficiently twisted to do so!). > > > Michael. -- Gustavo J. A. M. Carneiro The universe is always one step beyond logic. From ncoghlan at gmail.com Fri Jun 17 12:54:22 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 17 Jun 2005 20:54:22 +1000 Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod() for Multiple Divisors In-Reply-To: <001d01c57322$f5af0ee0$bb07a044@oemcomputer> References: <001d01c57322$f5af0ee0$bb07a044@oemcomputer> Message-ID: <42B2ABDE.60608@gmail.com> Raymond Hettinger wrote: > Executive summary: cute, but unpersuasive and unnecessary, not worth > the time to code, test, document, maintain, and explain. Plus, it fails the "not every 3-line function has to be a builtin" guideline: def extended_divmod(numerator, *denominators): remainders = [] for denominator in reversed(denominators): numerator, remainder = divmod(numerator, denominator) remainders.insert(0, remainder) return tuple(remainders) OK, 5 lines. Anyway, not very hard to write for anyone with a genuine use case - and, like you, I've never used divmod for anything other than extracting digits (or groups of digits) from numbers. I also don't buy the 'tedious and easy to get wrong each time you need it' justification in the PEP. Getting the argument order to the extended divmod wrong seems to be even easier. For each of the cited use cases, a well-named function, or a proper class seems like a much cleaner solution. e.g. class Declination(object): def __init__(self, value): try: # Copy a duck-typed declination self.degrees = value.degrees self.minutes = value.minutes self.seconds = value.seconds except AttributeError: try: # Allow any three-value sequence self.degrees, self.minutes, self.seconds = value except TypeError: # Divide a number value, self.seconds = divmod(value, 60) value, self.minutes = divmod(value, 60) value, self.degrees = divmod(value, 360) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From skip at pobox.com Fri Jun 17 13:29:03 2005 From: skip at pobox.com (Skip Montanaro) Date: Fri, 17 Jun 2005 06:29:03 -0500 Subject: [Python-Dev] PEP 304 "Controlling Generation of Bytecode Files" - patch updated In-Reply-To: References: Message-ID: <17074.46079.101393.509174@montanaro.dyndns.org> Skip> http://python.org/sf/677103 Thomas> There's no patch attached. *sigh* Thanks for noticing the problem. Apparently, since I last updated the patch, SF implemented a 250kbyte limit on file uploads. This one is big because it includes a suitably modified configure script that was generated with a slightly different version of autoconf than what's in the current Python distro so people don't need to have autoconf installed to work with the patch. I've attached a gzipped version of the patch (bcb.diffs.gz). Skip From amk at amk.ca Fri Jun 17 14:30:23 2005 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 17 Jun 2005 08:30:23 -0400 Subject: [Python-Dev] Request to rewrite PEP 206 Message-ID: <20050617123023.GA15532@rogue.amk.ca> Just a note, sparked by Raymond's recent work cleaning up old PEPs: I'd like to take over PEP 206, the "Batteries Included" PEP, and rewrite it to describe a "Python Advanced Library", a set of third-party packages to complement the standard library. I've written to Moshe, the original author, to check that this is OK with him, and am waiting for a response. --amk From barry at python.org Fri Jun 17 15:29:00 2005 From: barry at python.org (Barry Warsaw) Date: Fri, 17 Jun 2005 09:29:00 -0400 Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral Literals to Python In-Reply-To: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> Message-ID: <1119014940.21565.7.camel@geddy.wooz.org> On Fri, 2005-06-17 at 02:39, Raymond Hettinger wrote: > While the majority of Python users deem this to be a nice-to-have > feature Really? Where's the supporting poll data? In over 10 years of Python programming, I've never once needed a Roman number literal. Worse, I don't buy the compatibility argument. I'm as anal as anyone about PEP 8 style, but that's still no reason to break code like >>> MIX = True I wouldn't be opposed to a library that provided a function to convert to and from Romans but I don't think Python needs Roman numeral literal support. +1 for rejecting. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/9b09348e/attachment.pgp From barry at python.org Fri Jun 17 15:34:35 2005 From: barry at python.org (Barry Warsaw) Date: Fri, 17 Jun 2005 09:34:35 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer> References: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer> Message-ID: <1119015274.21570.10.camel@geddy.wooz.org> On Fri, 2005-06-17 at 00:43, Raymond Hettinger wrote: > Let me go on record as a strong -1 for "continue EXPR". The for-loop is > our most basic construct and is easily understood in its present form. > The same can be said for "continue" and "break" which have the added > advantage of a near zero learning curve for people migrating from other > languages. > > Any urge to complicate these basic statements should be seriously > scrutinized and held to high standards of clarity, explainability, > obviousness, usefulness, and necessity. IMO, it fails most of those > tests. > > I would not look forward to explaining "continue EXPR" in the tutorial > and think it would stand out as an anti-feature. I'm sympathetic to this argument. I also find yield expressions jarring. I don't have any better suggestions though. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/1d15dc92/attachment.pgp From barry at python.org Fri Jun 17 15:36:25 2005 From: barry at python.org (Barry Warsaw) Date: Fri, 17 Jun 2005 09:36:25 -0400 Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable In-Reply-To: <000b01c57309$8b347d20$bb07a044@oemcomputer> References: <000b01c57309$8b347d20$bb07a044@oemcomputer> Message-ID: <1119015384.21571.12.camel@geddy.wooz.org> On Fri, 2005-06-17 at 02:55, Raymond Hettinger wrote: > After nine months, no support has grown beyond the original poster. The > PEP did however generate some negative responses when brought-up on > comp.lang.python (it made some people's stomach churn). > > The PEP fails the tests of obviousness and necessity. I agree. The fact that None is not callable is a /feature/ IMO. +1 for rejecting this PEP. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/e79027f4/attachment.pgp From barry at python.org Fri Jun 17 15:50:38 2005 From: barry at python.org (Barry Warsaw) Date: Fri, 17 Jun 2005 09:50:38 -0400 Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod() for Multiple Divisors In-Reply-To: <001d01c57322$f5af0ee0$bb07a044@oemcomputer> References: <001d01c57322$f5af0ee0$bb07a044@oemcomputer> Message-ID: <1119016238.21566.18.camel@geddy.wooz.org> On Fri, 2005-06-17 at 05:57, Raymond Hettinger wrote: > This PEP has been open for two and half years without generating > discussion or support. Interesting. Just yesterday I wrote a simple stopwatch-like timer script and I found that I needed three divmod calls to convert from seconds into a datetime.time object. This PEP might have come in handy there, but OTOH, I'm not so sure that's enough justification to accept the PEP. > Its primary case (converting cumulative seconds into a tuple days, > hours, minutes, and seconds) is a bit wanting because it doesn't > generalize to months and years. That need is already met in a more > robust and flexible way by date and time specific modules. Actually, no, because datetime.time(seconds=50227) throws an exception. But in my specific case, I didn't find the need for three divmod calls nearly as frustrating as the lack of a datetime.time.fromseconds() call. > More importantly, the gain in succinctness is offset by a loss of > clarity. Consider: > > dist, inches = divmod(dist, 12) > yards, feet = divmod(dist, 3) > > versus a proposed: > > yards, feet, inches = divmod(dist, 3, 12) > > The latter form makes it difficult to visually confirm the correct > number of target variables. Likewise, it is not at all obvious that the > order of the 3 and 12 are correct. I agree. My three divmod solution is perfectly readable and simple to write. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050617/a6d19c01/attachment.pgp From tim.peters at gmail.com Fri Jun 17 16:41:12 2005 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 17 Jun 2005 10:41:12 -0400 Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral Literals to Python In-Reply-To: <1119014940.21565.7.camel@geddy.wooz.org> References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> <1119014940.21565.7.camel@geddy.wooz.org> Message-ID: <1f7befae05061707413b58bf94@mail.gmail.com> [Raymond Hettinger] >> While the majority of Python users deem this to be a nice-to-have >> feature [Barry Warsaw] > Really? Where's the supporting poll data? We've run IV polls since this PEP was introduced, and the geometric mean of those shows LXVIII% of Python users strongly in favor (+I), and an additional XXI% not opposed (+roman(0), really -- I'm not sure how to spell zero in Roman numerals, but trust that the new `roman()` builtin will handle it correctly). > In over 10 years of Python programming, I've never once needed a Roman > number literal. Who cares what someone still stuck on Python Challenge #XVII thinks? Input from real Python programmers would be appreciated, though. For example, I'm eager to switch ZODB's object ids to Roman numerals. For example, as Raymond acknowledged, that would make it much easier to produce ZODB PowerPoint slides. > Worse, I don't buy the compatibility argument. I'm as anal as anyone about > PEP 8 style, but that's still no reason to break code like > > >>> MIX = True And you care nothing for the possibility that an ancient Roman explorer found frozen in the Arctic may be revived, and find such code incomprehensibly obscure? Python should be for everyone, not just the living. BTW, you might want to reread Raymond's post -- he was having an indecently good time at, well, someone's expense . From gvanrossum at gmail.com Fri Jun 17 16:48:57 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 17 Jun 2005 07:48:57 -0700 Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral Literals to Python In-Reply-To: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> Message-ID: +M to reject. On 6/16/05, Raymond Hettinger wrote: > While the majority of Python users deem this to be a nice-to-have > feature, the community has been unable to reach a consensus on the > proper syntax after more than two years of intensive debate (the PEP was > introduced in early April 2003). > > Most agree that there should be only-one-way-to-do-it; however, the > proponents are evenly split into two camps, with the modernists > preferring IX for nine and the classicists preferring VIIII which was > the most likely spelling in ancient Rome. > > The classicists not only rely on set-in-stone tradition, they point to > pragmatic issues such as avoidance of subtraction, ease of coding, > easier mental parsing (much less error prone), and ease of teaching to > beginners. They assert that the modernists have introduced unnecessary > algorithmic complexity just to save two keystrokes. > > The modernists point to compatible Java implementations and current > grade school textbooks. They believe that users from other languages > will expect the IX form. Note however, not all the modernists agree on > whether MXM would be a well-formed spelling of 1990; most, but not all > prefer MCMXC despite its likelihood of being mis-parsed on a first > reading. > > There is also a small but vocal user group demanding that lowercase > forms be allowed. Their use cases fall into four categories: (i) > academia, (ii) the legal profession, (iii) research paper writing, and > (iv) powerpoint slideshows. Reportedly, this is also a common > convention among Perl programmers. > > Links: > > http://hrsbstaff.ednet.ns.ca/waymac/History%20A/A%20Term%201/1.%20Rome/R > oman_Numerals.htm > http://www.sizes.com/numbers/roman_numerals.htm > > > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Jun 17 17:33:22 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 17 Jun 2005 11:33:22 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <1119004190.648.13.camel@localhost> References: <200506171012.26036.Michaels@rd.bbc.co.uk> <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> <200506171012.26036.Michaels@rd.bbc.co.uk> Message-ID: <5.1.1.6.0.20050617111758.01e021e0@mail.telecommunity.com> At 11:29 AM 6/17/2005 +0100, Gustavo J. A. M. Carneiro wrote: > In conjunction with pseudo-threads, I think a "python main loop" >implementation is fundamental. Such main loop with permit the programmer >to register callbacks for events, such as timeouts, IO conditions, idle >tasks, etc., such as one found glib (gtk+'s underlying library). I >already pointed out one such implementation that I use for one of my >projects, and it already has unit tests to prove that it works. I think it's important to point out that such a "main loop" needs to be defined as an interface, rather than an implementation, because there are many such "main loops" out there as far as GTK, wx, OS X, etc. that have different implementation details as to how timeouts and I/O have to be managed. Since I see from your web page that you've looked at peak.events, I would refer you to the IEventLoop interface as an example of such an interface; any Twisted reactor can be adapted to provide most of the IEventLoop features (and vice versa) which means an interface like it should be usable on a variety of platforms. Of course, I also think that before proposing an event loop facility for the stdlib, we should actually succeed in implementing next(arg), yield expressions, and throw(). I'll probably take a look at next()/throw() this weekend to see if there's anything I can contribute to the ceval.c part; I'm a bit less likely to be able to help on the compilation side of things, though. (Apart from maybe adding a POP_TOP after existing yield statements.) From ncoghlan at gmail.com Fri Jun 17 17:50:02 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Jun 2005 01:50:02 +1000 Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod() for Multiple Divisors In-Reply-To: <000001c57330$28e13740$bb07a044@oemcomputer> References: <000001c57330$28e13740$bb07a044@oemcomputer> Message-ID: <42B2F12A.2020809@gmail.com> Raymond Hettinger wrote: >>Plus, it fails the "not every 3-line function has to be a builtin" >>guideline: > > > Not to pick, but I hope this doesn't become a recurring refrain. That > isn't a real guideline, it's more of a snipe. It also runs counter to > Zen about proposals not being complex and being easy to explain. I guess I really mean "the use case is obscure enough that simply writing the 5-line function is better than cluttering the API of a builtin", but that doesn't have quite the same ring to it ;) > There > are tons of exceptions. Guido's new any() and all() replace only a > single line. The sorted() builtin was very successful and it only > replaced a couple of lines. The key= argument is also successful but > replaces a simple, three-line Schwarzian transform. Reading DictMixin > shows that most dictionary methods are trivially expressed in terms of a > few primitives; however, we've learned that the mapping API is > excellent, expressive, and useful (except for setdefault which I've > grown to hate ;-). IOW, the quip is food for thought but not > necessarily either a positive or negative point about a proposal. That's basically what I meant - and what I take the phrase to mean when someone else uses it. If something is simple to write, but the use case is obscure, then that's an argument *against* making it a builtin, since half the time you'll have the function written before you remember there's a builtin for it. On the other hand, if the use case is common enough, rewriting it every time you need it is just a pain. The 'not every . . .' comment just tries to say all that using only ten words. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From gvanrossum at gmail.com Fri Jun 17 17:55:48 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 17 Jun 2005 08:55:48 -0700 Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable In-Reply-To: <1119015384.21571.12.camel@geddy.wooz.org> References: <000b01c57309$8b347d20$bb07a044@oemcomputer> <1119015384.21571.12.camel@geddy.wooz.org> Message-ID: Yes please. That one will never get past me as long as I'm alive. (At PyCon I met one person who proposed an even more radical approach; I think he wanted getattr(None, ) to return None. But he was certified insane. :-) On 6/17/05, Barry Warsaw wrote: > On Fri, 2005-06-17 at 02:55, Raymond Hettinger wrote: > > After nine months, no support has grown beyond the original poster. The > > PEP did however generate some negative responses when brought-up on > > comp.lang.python (it made some people's stomach churn). > > > > The PEP fails the tests of obviousness and necessity. > > I agree. The fact that None is not callable is a /feature/ IMO. +1 for > rejecting this PEP. > > -Barry > > > > BodyID:129997985.2.n.logpart (stored separately) > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.peters at gmail.com Fri Jun 17 18:16:10 2005 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 17 Jun 2005 12:16:10 -0400 Subject: [Python-Dev] Propose to reject PEP 336 -- Make None Callable In-Reply-To: <000b01c57309$8b347d20$bb07a044@oemcomputer> References: <000b01c57309$8b347d20$bb07a044@oemcomputer> Message-ID: <1f7befae050617091673caabe8@mail.gmail.com> [Raymond Hettinger] > After nine months, no support has grown beyond the original poster. Never will, either -- even Roman numeral literals are more Pythonic than this one. More Pythonic: make integers callable: i(arglist) returns the i'th argument. So, e.g., people who find it inconvenient to index a list like this: x[i] could index it like this instead: i(*x) Punchline: I didn't make this up -- that's how integers work in Icon! Kinda. y := 2 y(x, y, z) := 3 also works to bind `y` to 3. Python is falling _way_ behind . From tim.peters at gmail.com Fri Jun 17 18:33:32 2005 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 17 Jun 2005 12:33:32 -0400 Subject: [Python-Dev] Propose rejection of PEP 303 -- Extend divmod() for Multiple Divisors In-Reply-To: <1119016238.21566.18.camel@geddy.wooz.org> References: <001d01c57322$f5af0ee0$bb07a044@oemcomputer> <1119016238.21566.18.camel@geddy.wooz.org> Message-ID: <1f7befae0506170933636c2dc@mail.gmail.com> About PEP 303, I use divmod for lots (and lots) of things, but I've got no real use for an extended divmod() either. -1: it would be low-use, confusing clutter. [Barry] > Interesting. Just yesterday I wrote a simple stopwatch-like timer > script and I found that I needed three divmod calls to convert from > seconds into a datetime.time object. You don't need any divmods for that ... ... > Actually, no, because datetime.time(seconds=50227) throws an exception. That's right: the time class models a time of day, not "seconds from Barry's implied midnight epoch" (or something like that). What you wanted was a timedelta instead. Converting that to a time is then simplicity itself : >>> print (datetime(1, 1, 1) + timedelta(seconds=50227)).time() 13:57:07 You have to go thru a datetime object first because time objects don't support arithmetic either. That isn't all bad. By going thru a datetime first, it's clear what happens if the number of seconds you feed in exceeds a day's worth. You can check for that or ignore it then, depending on what your app wants (it may or may not be "an error", depending on the app). From gvanrossum at gmail.com Fri Jun 17 20:23:05 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 17 Jun 2005 11:23:05 -0700 Subject: [Python-Dev] Request to rewrite PEP 206 In-Reply-To: <20050617123023.GA15532@rogue.amk.ca> References: <20050617123023.GA15532@rogue.amk.ca> Message-ID: Sounds great! If Moshe doesn't respond within a reasonable time, you can take it over by default IMO. On 6/17/05, A.M. Kuchling wrote: > Just a note, sparked by Raymond's recent work cleaning up old PEPs: > I'd like to take over PEP 206, the "Batteries Included" PEP, and > rewrite it to describe a "Python Advanced Library", a set of > third-party packages to complement the standard library. > > I've written to Moshe, the original author, to check that this is OK > with him, and am waiting for a response. > > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Fri Jun 17 20:28:57 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 17 Jun 2005 11:28:57 -0700 Subject: [Python-Dev] Propose rejection of PEPs 239 and 240 -- a builtin rational type and rational literals In-Reply-To: <000e01c57317$984f40e0$bb07a044@oemcomputer> References: <000e01c57317$984f40e0$bb07a044@oemcomputer> Message-ID: Agreed. Rational arithmetic was the default "exact" arithmetic in ABC and it did not work out as expected. On 6/17/05, Raymond Hettinger wrote: > > These PEPs are four years old. Nothing is intrinsically wrong with them, > but they have garnered little enthusiasm, discussion, or support, suggesting > that the original need was somewhat modest. > > In addition, the principal (but not only) use cases for a builtin rational > type and corresponding rational literal have already been addressed by the > decimal module (and the expected future integration of decimal literals). > From rationale section of the PEPs: > > """ > Rational numbers are useful for exact and unsurprising arithmetic. They > give the correct results people have been taught in various math classes. > Making the "obvious" non-integer type one with more predictable semantics > will surprise new programmers less than using floating point numbers. As > quite a few posts on c.l.py and on tutor at python.org have shown, people often > get bit by strange semantics of floating point numbers: for example, > round(0.98, 2) still gives 0.97999999999999998. > """ > > The future direction of the decimal module likely entails literals in the > form of 123.45d with binary floats continuing to have the form 123.45. This > conflicts with the rational literal proposal of having 123.45 interpreted as > 123 + 45/100. > > There may still be a use for a rational module in the standard library, but > builtin support is no longer needed or desirable. > > The PEPs also touch on division ambiguities which were largely resolved by > the acceptance of PEP 238 which introduced the floor division operator and > from __future__ import division. > > The rational proposal also has an intrinsic weakness shared with Java's > prior versions of BigDecimal which they found to be unworkable in practice. > The weakness was that repeated operations caused the internal number of > digits to grow beyond reason. For this reason, the PEP proposes some > arbitrary level of truncation which conflicts with the goals of having > "obvious and exact" arithmetic. The proposed truncation methodology was > undeveloped and made no proposal for the same fine level of control as its > counterpart in the decimal module where issues of variable precision, > multiple contexts, and alternate rounding modes have been fully thought out. > > > > Raymond > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Fri Jun 17 20:29:52 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Fri, 17 Jun 2005 11:29:52 -0700 Subject: [Python-Dev] Propose rejection of PEPs 239 and 240 -- a builtin rational type and rational literals In-Reply-To: <000e01c57317$984f40e0$bb07a044@oemcomputer> References: <000e01c57317$984f40e0$bb07a044@oemcomputer> Message-ID: <20050617103216.7277.JCARLSON@uci.edu> "Raymond Hettinger" wrote: > There may still be a use for a rational module in the standard library, > but builtin support is no longer needed or desirable. Sounds good. > The rational proposal also has an intrinsic weakness shared with Java's > prior versions of BigDecimal which they found to be unworkable in > practice. The weakness was that repeated operations caused the internal > number of digits to grow beyond reason. For this reason, the PEP > proposes some arbitrary level of truncation which conflicts with the > goals of having "obvious and exact" arithmetic. The proposed truncation > methodology was undeveloped and made no proposal for the same fine level > of control as its counterpart in the decimal module where issues of > variable precision, multiple contexts, and alternate rounding modes have > been fully thought out. Here is an option: exploit all of the thought and effort that has gone into decimals. Offer a "context" argument which handles the precision and rounding of the rational numerator and denominator as necessary, defaulting to the current decimal context, with automatic rounding (using the same kinds of semantics that decimals currently uses). Offer a mechanism by which one can use exact rationals, which use longs rather than decimals. The idea obviously needs fleshing out, but perhaps it is a start for someone who wants rationals in the standard library. - Josiah From martin at v.loewis.de Fri Jun 17 23:34:27 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 17 Jun 2005 23:34:27 +0200 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <2m4qbybpf2.fsf@starship.python.net> References: <2mzmtrbxwo.fsf@starship.python.net> <42B0725D.9040200@v.loewis.de> <2m4qbybpf2.fsf@starship.python.net> Message-ID: <42B341E3.5000609@v.loewis.de> Michael Hudson wrote: > I've been looking at this area partly to try and understand this bug: > > [ 1163563 ] Sub threads execute in restricted mode > > but I'm not sure the whole idea of multiple interpreters isn't > inherently doomed :-/ That's what Tim asserts, saying that people who want to use the feature should fix it themselves. Regards, Martin From raymond.hettinger at verizon.net Fri Jun 17 23:49:37 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 17 Jun 2005 17:49:37 -0400 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda Message-ID: <000001c57386$75c23180$bb07a044@oemcomputer> This PEP is an excellent example of improving readability and usability by omitting a keyword and simplifying syntax. It neither provides nor takes away functionality; instead, it is a bit of a beautification effort. Essentially it creates a lighter-weight, more usable syntax for specifying deferred function arguments. Like decorators, this helps out programmers who understand and use this technique, and it is harmless and invisible for the rest. It is especially handy in the absence of an if-then-else expression. The PEP is backwards compatible. Recommend accepting just the basic PEP which only targets simple, obvious cases. The discussed extensions are unattractive and should be skipped. Raymond From gvanrossum at gmail.com Fri Jun 17 23:59:58 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 17 Jun 2005 14:59:58 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <000001c57386$75c23180$bb07a044@oemcomputer> References: <000001c57386$75c23180$bb07a044@oemcomputer> Message-ID: [Raymond Hettinger] > This PEP is an excellent example of improving readability and usability > by omitting a keyword and simplifying syntax. It neither provides nor > takes away functionality; instead, it is a bit of a beautification > effort. > > Essentially it creates a lighter-weight, more usable syntax for > specifying deferred function arguments. Like decorators, this helps out > programmers who understand and use this technique, and it is harmless > and invisible for the rest. It is especially handy in the absence of an > if-then-else expression. The PEP is backwards compatible. > > Recommend accepting just the basic PEP which only targets simple, > obvious cases. The discussed extensions are unattractive and should be > skipped. -1. The "unary colon" looks unPythonic to me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond.hettinger at verizon.net Sat Jun 18 00:25:07 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri, 17 Jun 2005 18:25:07 -0400 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers Message-ID: <000301c5738b$6afc5fa0$bb07a044@oemcomputer> IIRC, there was a decision to not implement phase C and to keep the trailing L in representations of long integers. If so, I believe the PEP can be marked as final. We've done all we're going to do. Raymond From gvanrossum at gmail.com Sat Jun 18 00:33:47 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri, 17 Jun 2005 15:33:47 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: <000301c5738b$6afc5fa0$bb07a044@oemcomputer> References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer> Message-ID: On 6/17/05, Raymond Hettinger wrote: > IIRC, there was a decision to not implement phase C and to keep the > trailing L in representations of long integers. Actually, the PEP says phase C will be implemented in Python 3.0 and that's still my plan. > If so, I believe the PEP can be marked as final. We've done all we're > going to do. For 2.x, yes. I'm fine with marking it as Final and adding this to PEP 3000 instead. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From Scott.Daniels at Acm.Org Sat Jun 18 01:23:36 2005 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri, 17 Jun 2005 16:23:36 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer> Message-ID: Guido van Rossum wrote: > On 6/17/05, Raymond Hettinger wrote: >>IIRC, there was a decision to not implement phase C and to keep the >>trailing L in representations of long integers. > For 2.x, yes. I'm fine with marking it as Final and adding this to PEP > 3000 instead. Since PEP 313 has been rejected, the trailing L no longer introduces ambiguity in the representation of roman(40) vs. roman(10L). --Scott David Daniels Scott.Daniels at Acm.Org From ncoghlan at gmail.com Sat Jun 18 01:33:28 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Jun 2005 09:33:28 +1000 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <000001c57386$75c23180$bb07a044@oemcomputer> Message-ID: <42B35DC8.6040308@gmail.com> Guido van Rossum wrote: >>Recommend accepting just the basic PEP which only targets simple, >>obvious cases. The discussed extensions are unattractive and should be >>skipped. > > > -1. The "unary colon" looks unPythonic to me. > Step 1 would be to require parentheses around the whole thing (ala generator expressions) to make it easier to see where the deferred expression ends. But all my use cases that I can think off the top of my head involve 'sorted', where it wouldn't help at all because of the need for an argument. So I'd rather see a serious discussion regarding giving lambdas a more Pythonic syntax in general, rather than one that only applied to the 'no-argument' case [1] Cheers, Nick. [1] http://wiki.python.org/moin/AlternateLambdaSyntax The 'expression-before-args' version using just the 'from' keyword is still my favourite. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From pje at telecommunity.com Sat Jun 18 02:54:22 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 17 Jun 2005 20:54:22 -0400 Subject: [Python-Dev] Implementing PEP 342 (was Re: Withdrawn PEP 288 and thoughts on PEP 342) In-Reply-To: References: <005301c572e3$f2518aa0$bb07a044@oemcomputer> <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> <005301c572e3$f2518aa0$bb07a044@oemcomputer> Message-ID: <5.1.1.6.0.20050617155758.035955d8@mail.telecommunity.com> At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote: >I do like "continue EXPR" but I have to admit I haven't even tried to >come up with examples -- it may be unnecessary. As Phillip says, yield >expressions and g.next(EXPR) are the core -- and also incidentally >look like they will cause the most implementation nightmares. (If >someone wants to start implementing these two now, go right ahead!) FYI, I've started work on a patch. I've got argument passing into generators working, and compiling of parenthesized yield expressions, in both the C and Python compilers (although the output of the compiler package isn't tested yet). I haven't implemented no-argument yields yet, either, or unparenthesized yields on the far RHS of an assignment. I do plan to implement throw() as part of the same patch. Much of what remains is expanding the test suite and writing documentation, though. It turns out that making 'next(EXPR)' work is a bit tricky; I was going to use METH_COEXIST and METH_VARARGS, but then it occurred to me that METH_VARARGS adds overhead to normal Python calls to 'next()', so I implemented a separate 'send(EXPR)' method instead, and left 'next()' a no-argument call. Whether this is the way it should really work or not is a PEP discussion, of course, but it does seem to me that making send(ob) and throw(typ,val,tb) separate methods from the iterator protocol is a reasonable thing to do. Anyway, the patch isn't ready yet, but I hope to be able to post something for review before the weekend is out. From joachim.koenig-baltes at emesgarten.de Fri Jun 17 22:53:32 2005 From: joachim.koenig-baltes at emesgarten.de (Joachim Koenig-Baltes) Date: Fri, 17 Jun 2005 22:53:32 +0200 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: References: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer> Message-ID: <42B3384C.7080908@emesgarten.de> Guido van Rossum wrote: > >However, I can see other uses for looping over a sequence using a >generator and telling the generator something interesting about each >of the sequence's items, e.g. whether they are green, or should be >printed, or which dollar value they represent if any (to make up a >non-Boolean example). > >Anyway, "continue EXPR" was born as I was thinking of a way to do this >kind of thing in Python, since I didn't want to give up return as a >way of breaking out of a loop (or several!) and returning from a >function. > >But I'm the first to admit that the use case is still very much >hypothetical -- unlike that for g.next(EXPR) and VAR = yield. > > > My use case for this is a directory tree walking generator that yields all the files including the directories in a depth first manner. If a directory satisfies a condition (determined by the caller) the generator shall not descend into it. Something like: DONOTDESCEND=1 for path in mywalk("/usr/src"): if os.path.isdir(path) and os.path.basename(path) == "CVS": continue DONOTDESCEND # do something with path Of course there are different solutions to this problem with callbacks or filters but i like this one as the most elegant. Joachim From abo at minkirri.apana.org.au Sat Jun 18 03:18:07 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Fri, 17 Jun 2005 18:18:07 -0700 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <42B3384C.7080908@emesgarten.de> References: <006e01c572f7$1ec8eca0$bb07a044@oemcomputer> <42B3384C.7080908@emesgarten.de> Message-ID: <1119057487.15054.196.camel@warna.corp.google.com> On Fri, 2005-06-17 at 13:53, Joachim Koenig-Baltes wrote: [...] > My use case for this is a directory tree walking generator that > yields all the files including the directories in a depth first manner. > If a directory satisfies a condition (determined by the caller) the > generator shall not descend into it. > > Something like: > > DONOTDESCEND=1 > for path in mywalk("/usr/src"): > if os.path.isdir(path) and os.path.basename(path) == "CVS": > continue DONOTDESCEND > # do something with path > > Of course there are different solutions to this problem with callbacks > or filters but i like this one as the most elegant. I have implemented almost exactly this use-case using the standard Python generators, and shudder at the complexity something like this would introduce. For me, the right solution would be to either write your own generator that "wraps" the other generator and filters it, or just make the generator with additional (default value) parameters that support the DONOTDECEND filtering. FWIW, my usecase is a directory comparison generator that walks two directorys producing tuples of corresponding files. It optionally will not decend directories in either tree that do not have a corresponding directory in the other tree. See; http://minkirri.apana.org.au/~abo/projects/utils/ -- Donovan Baarda From python at rcn.com Sat Jun 18 06:58:06 2005 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Jun 2005 00:58:06 -0400 Subject: [Python-Dev] Withdrawn PEP 288 and thoughts on PEP 342 In-Reply-To: <1119057487.15054.196.camel@warna.corp.google.com> Message-ID: <003801c573c2$5145b660$bb07a044@oemcomputer> [Joachim Koenig-Baltes] > > My use case for this is a directory tree walking generator that > > yields all the files including the directories in a depth first manner. > > If a directory satisfies a condition (determined by the caller) the > > generator shall not descend into it. > > > > Something like: > > > > DONOTDESCEND=1 > > for path in mywalk("/usr/src"): > > if os.path.isdir(path) and os.path.basename(path) == "CVS": > > continue DONOTDESCEND > > # do something with path > > > > Of course there are different solutions to this problem with callbacks > > or filters but i like this one as the most elegant. [Donovan Baarda] > I have implemented almost exactly this use-case using the standard > Python generators, and shudder at the complexity something like this > would introduce. > > For me, the right solution would be to either write your own generator > that "wraps" the other generator and filters it, or just make the > generator with additional (default value) parameters that support the > DONOTDECEND filtering. > > FWIW, my usecase is a directory comparison generator that walks two > directorys producing tuples of corresponding files. It optionally will > not decend directories in either tree that do not have a corresponding > directory in the other tree. See; > > http://minkirri.apana.org.au/~abo/projects/utils/ Thank both of you for the excellent posts. This is exactly kind of feedback and analysis that will show whether continue EXPR is worth it. Raymond From simonwittber at gmail.com Sat Jun 18 08:19:34 2005 From: simonwittber at gmail.com (Simon Wittber) Date: Sat, 18 Jun 2005 14:19:34 +0800 Subject: [Python-Dev] PEP for RFE 46738 (first draft) Message-ID: <4e4a11f805061723192888915e@mail.gmail.com> Hello Chaps, The attached PEP (pep.txt) is for RFE 46738, which you can view here: http://sourceforge.net/tracker/index.php?func=detail&aid=467384&group_id=5470&atid=355470 It provides a safe, documented class for serialization of simple python types. A sample implementation is also attached (gherkin.py). Critcism and comments on the PEP and the implementation are appreciated. Simon Wittber. -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep.txt Url: http://mail.python.org/pipermail/python-dev/attachments/20050618/b5fa6bf6/pep.txt -------------- next part -------------- A non-text attachment was scrubbed... Name: gherkin.py Type: text/x-python Size: 7225 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050618/b5fa6bf6/gherkin.py From kay.schluehr at gmx.net Sat Jun 18 09:16:59 2005 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sat, 18 Jun 2005 09:16:59 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B35DC8.6040308@gmail.com> References: <000001c57386$75c23180$bb07a044@oemcomputer> <42B35DC8.6040308@gmail.com> Message-ID: Nick Coghlan wrote: > Guido van Rossum wrote: > >>>Recommend accepting just the basic PEP which only targets simple, >>>obvious cases. The discussed extensions are unattractive and should be >>>skipped. >> >> >>-1. The "unary colon" looks unPythonic to me. >> > > > Step 1 would be to require parentheses around the whole thing (ala > generator expressions) to make it easier to see where the deferred > expression ends. > > But all my use cases that I can think off the top of my head involve > 'sorted', where it wouldn't help at all because of the need for an > argument. > > So I'd rather see a serious discussion regarding giving lambdas a more > Pythonic syntax in general, rather than one that only applied to the > 'no-argument' case [1] > > Cheers, > Nick. > > [1] http://wiki.python.org/moin/AlternateLambdaSyntax > The 'expression-before-args' version using just the 'from' keyword is > still my favourite. > Maybe anonymus function closures should be pushed forward right now not only syntactically? Personally I could live with lambda or several of the alternative syntaxes listed on the wiki page. But asking for a favourite syntax I would skip the "def" keyword from your def-arrow syntax proposal and use: ((a, b, c) -> f(a) + o(b) - o(c)) ((x) -> x * x) (() -> x) ((*a, **k) -> x.bar(*a, **k)) ( ((x=x, a=a, k=k) -> x(*a, **k)) for x, a, k in funcs_and_args_list) The arrow is a straightforward punctuation for function definitions. Reusing existing keywords for different semantics seems to me as a kind of inbreeding. For pushing anymus functions forward I propose to enable explizit partial evaluation as a programming technique: Example 1: >>> ((x,y) -> (x+1)*y**2) ((x,y) -> (x+1)*y**2) >>> ((x,y) -> (x+1)*y**2)(x=5) ((y) -> 6*y**2) Example 2: def f(x): return x**2 >>> ((x,y) -> f(x)+f(y))(x=2) ((y) -> 4 + f(y)) Example 3: >>> ((f,x,y) -> f(x)+f(y))(f=((x)-> x**2), y=3) ((x) -> ((x)-> x**2))(x)+9) The keyword style argument passing can be omitted in case of complete evaluation where pattern matching on the argument tuple is applied. Regards, Kay From jcarlson at uci.edu Sat Jun 18 10:34:30 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 18 Jun 2005 01:34:30 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <42B35DC8.6040308@gmail.com> Message-ID: <20050618011502.728D.JCARLSON@uci.edu> Kay Schluehr wrote: > Maybe anonymus function closures should be pushed forward right now not > only syntactically? Personally I could live with lambda or several > of the alternative syntaxes listed on the wiki page. > But asking for a favourite syntax I would skip the "def" keyword from > your def-arrow syntax proposal and use: > > ((a, b, c) -> f(a) + o(b) - o(c)) ... > The arrow is a straightforward punctuation for function definitions. > Reusing existing keywords for different semantics seems to me as a kind > of inbreeding. That's starting to look like the pseudocode from old algorithms textbooks, which is very similar to bad pseudocode from modern CS theory papers. Punctuation as a replacement for words does not always win (perfect examples being 'and' vs. &&, 'or' vs. ||, 'not' vs. !, ...) -1 on the syntax offering. > For pushing anymus functions forward I propose to enable explizit > partial evaluation as a programming technique: If I remember correctly, we've got rightcurry and leftcurry for that (or rightpartial and leftpartial, or something). > >>> ((x,y) -> (x+1)*y**2) > ((x,y) -> (x+1)*y**2) > > >>> ((x,y) -> (x+1)*y**2)(x=5) > ((y) -> 6*y**2) I'll assume that you don't actually want it to rewrite the source, or actually return the source representation of the anonymous function (those are almost non-starters). As for all anonymous functions allowing partial evaluation via keywords: it would hide errors. Right now, if you forget an argument or add too many arguments, you get a TypeError. Your proposal would make forgetting an argument in certain ways return a partially evaluated function. -1 on partial evaluation. - Josiah From raymond.hettinger at verizon.net Sat Jun 18 10:45:32 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat, 18 Jun 2005 04:45:32 -0400 Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops Message-ID: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> I recommend that the proposed syntax be altered to be more parallel with the existing for-loop syntax to make it more parsable for both humans and for the compiler. Like existing for-statements, the target expression should immediately follow the 'for' keyword. Since this is known to be a range assignment, only an expression is needed, not a full expression list. Immediately following should be a token to distinguish the new and old syntaxes. Putting that distinction early in the statement prepares readers (who scan left-to-right) for what follows. IOW, augment the existing syntax: for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite] with an alternative syntax in the form: for_stmt: 'for' expr 'tok' rangespec ':' suite ['else' ':' suite] Given that form, the PEP authors can choose the best options for 'tok'. Basically, anything will do as long as it is not 'in'. Likewise, they can choose any rangespec format. Within that framework, there are many possibilities: for i between 2 < i <= 10: ... for i over 2 < i <= 10: ... # chained comparison style for i over [2:11]: ... # Slice style for i = 3 to 10: ... # Basic style The rangespecs with comparison operators offer more flexibility in terms of open/closed intervals. In contrast, the slice notation version and the Basic versions can admit a step argument. The key changes are stating the target variable first and then using a specific token to distinguish between the two forms. Also, I recommend tightening the PEP's motivation. There are only two advantages, encoding and readability. The former is only a minor gain because all it saves is a function call, an O(1) savings in an O(n) context. The latter is where the real benefits lay. The PEP authors should also put to rest the issues section: * Decide once and for all on the simplest approach of immediately evaluating the whole rangespec prior to execution. This best parallels the range() approach and it is the least surprising. * Note that new proposal works equally well with list comps and genexps. * Decide to always return an iterator rather than a list as there is never an advantage to doing otherwise. * If you go for a chained comparison style rangespec, then drop the issue of a step argument. If the slice or Basic style rangespecs are chosen, then there is no reason not to allow a step argument. * Competition with PEP 276 is no longer an issue. * Simply disallow floating point bounds. We've already got deprecation warnings in place for Py2.5. There is no need to exacerbate the problem. Alternately, simplify the issue by declaring that the values will be handled as if by xrange(). The above recommendations should get the PEP ready for judgement day. Good luck. Raymond From kdart at kdart.com Sat Jun 18 11:06:34 2005 From: kdart at kdart.com (Keith Dart) Date: Sat, 18 Jun 2005 02:06:34 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer> Message-ID: <42B3E41A.2020608@kdart.com> Guido van Rossum wrote: >On 6/17/05, Raymond Hettinger wrote: > > >>IIRC, there was a decision to not implement phase C and to keep the >>trailing L in representations of long integers. >> >> > >Actually, the PEP says phase C will be implemented in Python 3.0 and >that's still my plan. > > > >>If so, I believe the PEP can be marked as final. We've done all we're >>going to do. >> >> > >For 2.x, yes. I'm fine with marking it as Final and adding this to PEP >3000 instead. > > > I am very concernced about something. The following code breaks with 2.4.1: fcntl.ioctl(self.rtc_fd, RTC_RD_TIME, ...) Where RTC_RD_TIME = 2149871625L In Python 2.3 it is -2145095671. Actually, this is supposed to be an unsigned int, and it was construced with hex values and shifts. Now, with the integer unification, how is ioctl() supposed to work? I cannot figure out how to make it work in this case. I suppose the best thing is to introduce an "unsignedint" type for this purpose. As it is right now, I cannot use 2.4 at all. -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: F3D288E4 ===================================================================== -------------- next part -------------- A non-text attachment was scrubbed... Name: kdart.vcf Type: text/x-vcard Size: 179 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050618/ec966cbe/kdart.vcf From ncoghlan at gmail.com Sat Jun 18 12:37:41 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Jun 2005 20:37:41 +1000 Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops In-Reply-To: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> Message-ID: <42B3F975.5010601@gmail.com> Raymond Hettinger wrote: > Also, I recommend tightening the PEP's motivation. There are only two > advantages, encoding and readability. The former is only a minor gain > because all it saves is a function call, an O(1) savings in an O(n) > context. The latter is where the real benefits lay. The introduction of 'enumerate', and the proliferation of better iterators that reduce the need for pure-integer loops should be adressed in the revised motivation. I know my use of 'range' drops close to zero when I'm working with Python versions that supply 'enumerate'. Even when I do have a pure integer loop, I'll often assign the result of the range/xrange call to a local variable, just so I can give it a name that indicates the *significance* of that particular bunch of numbers. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From kay.schluehr at gmx.net Sat Jun 18 13:43:27 2005 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sat, 18 Jun 2005 13:43:27 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <20050618011502.728D.JCARLSON@uci.edu> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> Message-ID: Josiah Carlson wrote: > Kay Schluehr wrote: > > >> Maybe anonymus function closures should be pushed forward right now not only syntactically? Personally I could live with lambda or several >> of the alternative syntaxes listed on the wiki page. >> > > > > >> But asking for a favourite syntax I would skip the "def" keyword from your def-arrow syntax proposal and use: >> >> ((a, b, c) -> f(a) + o(b) - o(c)) >> > > ... > > > >> The arrow is a straightforward punctuation for function definitions. Reusing existing keywords for different semantics seems to me as a kind of inbreeding. >> > > > That's starting to look like the pseudocode from old algorithms > textbooks, which is very similar to bad pseudocode from modern CS theory > papers. Punctuation as a replacement for words does not always win > (perfect examples being 'and' vs. &&, 'or' vs. ||, 'not' vs. !, ...) > > Writing functions as arrows is very convenient not only in CS but also in mathematics. Looking like pseudo-code was not one of Guidos Python regrets if I remember them correctly. > -1 on the syntax offering. > > > >> For pushing anymus functions forward I propose to enable explizit partial evaluation as a programming technique: >> > > > If I remember correctly, we've got rightcurry and leftcurry for that (or > rightpartial and leftpartial, or something). > > Currying usually does not perform a function evaluation in order to create another more special function. Partial evaluation is a dynamic programming and optimization technique. Psyco uses specialization and caching implicitely. I propose to use it explicitely but in a more restricted context. >> >>> ((x,y) -> (x+1)*y**2) >> ((x,y) -> (x+1)*y**2) >> >> >>> ((x,y) -> (x+1)*y**2)(x=5) >> ((y) -> 6*y**2) >> > > > I'll assume that you don't actually want it to rewrite the source, or > actually return the source representation of the anonymous function > (those are almost non-starters). > > Computer algebra systems store expressions in internal tree form, manipulate them efficiently and pretty-print them in textual or latex output on demand. There would be much more involved than a tree to tree translation starting with Pythons internal parse tree and an evaluator dedicated to it. > As for all anonymous functions allowing partial evaluation via keywords: > it would hide errors. Right now, if you forget an argument or add too > many arguments, you get a TypeError. Your proposal would make > forgetting an argument in certain ways return a partially evaluated > function. That's why I like to dump the function in a transparent mode. Personally I could dispense a little security in favor for cheap metainformation. Regards, Kay From mwh at python.net Sat Jun 18 14:50:57 2005 From: mwh at python.net (Michael Hudson) Date: Sat, 18 Jun 2005 13:50:57 +0100 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <42B341E3.5000609@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Fri, 17 Jun 2005 23:34:27 +0200") References: <2mzmtrbxwo.fsf@starship.python.net> <42B0725D.9040200@v.loewis.de> <2m4qbybpf2.fsf@starship.python.net> <42B341E3.5000609@v.loewis.de> Message-ID: <2macln7rri.fsf@starship.python.net> "Martin v. L?wis" writes: > Michael Hudson wrote: >> I've been looking at this area partly to try and understand this bug: >> >> [ 1163563 ] Sub threads execute in restricted mode >> >> but I'm not sure the whole idea of multiple interpreters isn't >> inherently doomed :-/ > > That's what Tim asserts, saying that people who want to use the > feature should fix it themselves. Well, they've tried, and I think I've worked out a proper fix (life would be easier if people didn't check in borken code :). Cheers, mwh -- Premature optimization is the root of all evil. -- Donald E. Knuth, Structured Programming with goto Statements From mwh at python.net Sat Jun 18 14:56:53 2005 From: mwh at python.net (Michael Hudson) Date: Sat, 18 Jun 2005 13:56:53 +0100 Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops In-Reply-To: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> (Raymond Hettinger's message of "Sat, 18 Jun 2005 04:45:32 -0400") References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> Message-ID: <2m64wb7rhm.fsf@starship.python.net> "Raymond Hettinger" writes: > I recommend that the proposed syntax be altered to be more parallel > with the existing for-loop syntax to make it more parsable for both > humans and for the compiler. Although all your suggestions are improvments, I'm still -1 on the PEP. Cheers, mwh -- Windows installation day one. Getting rid of the old windows was easy - they fell apart quite happily, and certainly wont be re-installable anywhere else. -- http://www.linux.org.uk/diary/ (not *that* sort of windows...) From mwh at python.net Sat Jun 18 15:10:57 2005 From: mwh at python.net (Michael Hudson) Date: Sat, 18 Jun 2005 14:10:57 +0100 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: <42B3E41A.2020608@kdart.com> (Keith Dart's message of "Sat, 18 Jun 2005 02:06:34 -0700") References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer> <42B3E41A.2020608@kdart.com> Message-ID: <2m1x6z7qu6.fsf@starship.python.net> Keith Dart writes: > I am very concernced about something. The following code breaks with 2.4.1: > > fcntl.ioctl(self.rtc_fd, RTC_RD_TIME, ...) > > Where RTC_RD_TIME = 2149871625L > > In Python 2.3 it is -2145095671. Well, you could always use "-2145095671"... > Actually, this is supposed to be an unsigned int, and it was construced > with hex values and shifts. But well, quite. > Now, with the integer unification, how is ioctl() supposed to work? I > cannot figure out how to make it work in this case. The shortest way I know of going from 2149871625L to -2145095671 is the still-fairly-gross: >>> v = 2149871625L >>> ~int(~v&0xFFFFFFFF) -2145095671 > I suppose the best thing is to introduce an "unsignedint" type for this > purpose. Or some kind of bitfield type, maybe. C uses integers both as bitfields and to count things, and at least in my opinion the default assumption in Python should be that this is what an integer is being used for, but when you need a bitfield it can all get a bit horrible. That said, I think in this case we can just make fcntl_ioctl use the (new-ish) 'I' format argument to PyArg_ParseTuple and then you'll just be able to use 2149871625L and be happy (I think, haven't tried this). > As it is right now, I cannot use 2.4 at all. /Slightly/ odd place to mae this report! Hope this mail helped. Cheers, mwh -- I'm okay with intellegent buildings, I'm okay with non-sentient buildings. I have serious reservations about stupid buildings. -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year) From oren.tirosh at gmail.com Sat Jun 18 15:44:02 2005 From: oren.tirosh at gmail.com (Oren Tirosh) Date: Sat, 18 Jun 2005 16:44:02 +0300 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <4e4a11f805061723192888915e@mail.gmail.com> References: <4e4a11f805061723192888915e@mail.gmail.com> Message-ID: <7168d65a0506180644367753b9@mail.gmail.com> Please don't invent new serialization formats. I think we have enough of those already. The RFE suggests that "the protocol is specified in the documentation, precisely enough to write interoperating implementations in other languages". If interoperability with other languages is really the issue, use an existing format like JSON. If you want an efficient binary format you can use a subset of the pickle protocol supporting only basic types. I tried this once. I ripped out all the fancy parts from pickle.py and left only binary pickling (protocol version 2) of basic types. It took less than hour and I was left with something only marginally more complex than your new proposed protocol. Oren From simonwittber at gmail.com Sat Jun 18 16:12:16 2005 From: simonwittber at gmail.com (Simon Wittber) Date: Sat, 18 Jun 2005 22:12:16 +0800 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <7168d65a0506180644367753b9@mail.gmail.com> References: <4e4a11f805061723192888915e@mail.gmail.com> <7168d65a0506180644367753b9@mail.gmail.com> Message-ID: <4e4a11f805061807127f39700b@mail.gmail.com> > The RFE suggests that "the protocol is specified in the documentation, > precisely enough to write interoperating implementations in other > languages". If interoperability with other languages is really the > issue, use an existing format like JSON. JSON is slow (this is true of the python version, at least). Whether it is slow because of the implementation, or because of its textual nature, I do not know. The implementation I tested also failed to encode {1:2}. I am not sure if this is a problem with JSON or the implementation. > If you want an efficient binary format you can use a subset of the > pickle protocol supporting only basic types. I tried this once. I > ripped out all the fancy parts from pickle.py and left only binary > pickling (protocol version 2) of basic types. It took less than hour > and I was left with something only marginally more complex than your > new proposed protocol. I think you are missing the point. Is your pickle hack available for viewing? If it, or JSON is a better choice, then so be it. The point of the PEP is not the protocol, but the need for a documented, efficient, _safe_ serializion module in the standard library. Do you disagree? Simon Wittber. From pje at telecommunity.com Sat Jun 18 16:55:19 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 18 Jun 2005 10:55:19 -0400 Subject: [Python-Dev] Implementing PEP 342 (was Re: Withdrawn PEP 288 and thoughts on PEP 342) In-Reply-To: <7168d65a05061806553bad04b3@mail.gmail.com> References: <5.1.1.6.0.20050617155758.035955d8@mail.telecommunity.com> <5.1.1.6.0.20050616213226.01e14390@mail.telecommunity.com> <005301c572e3$f2518aa0$bb07a044@oemcomputer> <5.1.1.6.0.20050617155758.035955d8@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050618104339.02bbf650@mail.telecommunity.com> At 04:55 PM 6/18/2005 +0300, Oren Tirosh wrote: >On 6/18/05, Phillip J. Eby wrote: > > At 08:03 PM 6/16/2005 -0700, Guido van Rossum wrote: > > It turns out that making 'next(EXPR)' work is a bit tricky; I was going to > > use METH_COEXIST and METH_VARARGS, but then it occurred to me that > > METH_VARARGS adds overhead to normal Python calls to 'next()', so I > > implemented a separate 'send(EXPR)' method instead, and left 'next()' a > > no-argument call. > >Please use the name "feed", not "send". That would make the enhanced >generators already compatible "out of the box" with existing code >expecting the de-facto consumer interface (see >http://effbot.org/zone/consumer.htm). The other part of the consumer >interface (the close() method) is already being added in PEP 343. Hm. Do you want reset() as well? :) More seriously, I'm not sure that PEP 343's close() does something desirable for the consumer interface. Overall, this sounds like something that should be added to the PEPs though. I hadn't really thought of using inbound generator communication for parsing; it's an interesting use case. However, looking more closely at the consumer interface, it seems to me the desired semantics for feed() are different than for send(), because of the "just-started generator can't receive data" problem. Also, the consumer interface doesn't include handling for StopIteration. Maybe feed() should prime the generator if it's just started, and throw away the yield result as long as it's None? Maybe it should ignore StopIteration? Perhaps it should raise an error if the generator yields anything but None in response? These seem like questions worth discussing. From gerrit at nl.linux.org Sat Jun 18 17:59:07 2005 From: gerrit at nl.linux.org (Gerrit Holl) Date: Sat, 18 Jun 2005 17:59:07 +0200 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <000501c56fbe$79998220$4ff9a244@oemcomputer> References: <17068.58083.342369.872731@montanaro.dyndns.org> <000501c56fbe$79998220$4ff9a244@oemcomputer> Message-ID: <20050618155907.GA8136@topjaklont.student.utwente.nl> Hi, Raymond Hettinger wrote: > I think it unwise to allow x to be any expression. Besides altering > existing semantics, it leads to code redundancy and to a fragile > construct (where the slightest alteration of any of the expressions > triggers a silent reversion to O(n) behavior). What would happen if 'x' were to be an object whose class has a __eq__ that is defined in an odd way, e.g. having side effects? Might this mean behaviour change even if 'x' is a local variable? yours, Gerrit Holl. -- Weather in Twenthe, Netherlands 18/06 17:25: 24.0?C wind 3.1 m/s NE (57 m above NAP) -- In the councils of government, we must guard against the acquisition of unwarranted influence, whether sought or unsought, by the military-industrial complex. The potential for the disastrous rise of misplaced power exists and will persist. -Dwight David Eisenhower, January 17, 1961 From python at rcn.com Sat Jun 18 18:09:27 2005 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Jun 2005 12:09:27 -0400 Subject: [Python-Dev] Multiple expression eval in compound if statement? In-Reply-To: <20050618155907.GA8136@topjaklont.student.utwente.nl> Message-ID: <000a01c57420$1b234940$0636c797@oemcomputer> [Raymond Hettinger] > > I think it unwise to allow x to be any expression. Besides altering > > existing semantics, it leads to code redundancy and to a fragile > > construct (where the slightest alteration of any of the expressions > > triggers a silent reversion to O(n) behavior). [Gerrit Holl] > What would happen if 'x' were to be an object whose class has a __eq__ > that is defined in an odd way, e.g. having side effects? Every molecule in your body would simultaneously implode at the speed of light. Raymond From skip at pobox.com Sat Jun 18 19:02:45 2005 From: skip at pobox.com (Skip Montanaro) Date: Sat, 18 Jun 2005 12:02:45 -0500 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <4e4a11f805061807127f39700b@mail.gmail.com> References: <4e4a11f805061723192888915e@mail.gmail.com> <7168d65a0506180644367753b9@mail.gmail.com> <4e4a11f805061807127f39700b@mail.gmail.com> Message-ID: <17076.21429.501499.914085@montanaro.dyndns.org> Why this discussion of yet another serialization format? The wire-encoding for XML-RPC is quite stable, handles all the basic Python types proposed in the proto-PEP, and is highly interoperable. If performance is an issue, make sure you have a C-based accelerator module like sgmlop installed. If size is an issue, gzip it before sending it over the wire or to a file. Skip From gvanrossum at gmail.com Sat Jun 18 21:54:00 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat, 18 Jun 2005 12:54:00 -0700 Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops In-Reply-To: <2m64wb7rhm.fsf@starship.python.net> References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> <2m64wb7rhm.fsf@starship.python.net> Message-ID: On 6/18/05, Michael Hudson wrote: > "Raymond Hettinger" writes: > > > I recommend that the proposed syntax be altered to be more parallel > > with the existing for-loop syntax to make it more parsable for both > > humans and for the compiler. > > Although all your suggestions are improvments, I'm still -1 on the PEP. Same here. The whole point (15 years ago) of range() was to *avoid* needing syntax to specify a loop over numbers. I think it's worked out well and there's nothing that needs to be fixed (except range() needs to become an interator, which it will in Python 3.0). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sat Jun 18 22:34:20 2005 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Jun 2005 16:34:20 -0400 Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops In-Reply-To: Message-ID: <001901c57445$1b4b1cc0$0636c797@oemcomputer> [Raymond Hettinger] > > > I recommend that the proposed syntax be altered to be more parallel > > > with the existing for-loop syntax to make it more parsable for both > > > humans and for the compiler. [Michael Hudson] > > Although all your suggestions are improvments, I'm still -1 on the PEP. [Guido] > Same here. The whole point (15 years ago) of range() was to *avoid* > needing syntax to specify a loop over numbers. I think it's worked out > well and there's nothing that needs to be fixed (except range() needs > to become an interator, which it will in Python 3.0). I concur. Saying that no form of the idea is viable will save the PEP authors from another round or two of improvements. Marking as rejected and noting why. Raymond From gustavo at niemeyer.net Sat Jun 18 22:39:42 2005 From: gustavo at niemeyer.net (Gustavo Niemeyer) Date: Sat, 18 Jun 2005 17:39:42 -0300 Subject: [Python-Dev] PyPI: no space left on device Message-ID: <20050618203942.GA15154@burma.localdomain> PyPI seems to be out of space: % ./setup.py register --show-response running register Using PyPI login from /home/niemeyer/.pypirc --------------------------------------------------------------------------- Error... There's been a problem with your request psycopg.ProgrammingError: ERROR: could not extend relation "releases": No space left on device HINT: Check free disk space. -- Gustavo Niemeyer http://niemeyer.net From aahz at pythoncraft.com Sat Jun 18 23:09:01 2005 From: aahz at pythoncraft.com (Aahz) Date: Sat, 18 Jun 2005 14:09:01 -0700 Subject: [Python-Dev] PyPI: no space left on device In-Reply-To: <20050618203942.GA15154@burma.localdomain> References: <20050618203942.GA15154@burma.localdomain> Message-ID: <20050618210901.GB848@panix.com> On Sat, Jun 18, 2005, Gustavo Niemeyer wrote: > > PyPI seems to be out of space: FYI, python-dev is not a good place to send messages like this. Please use webmaster at python.org. (I've already notified the appropriate parties.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ f u cn rd ths, u cn gt a gd jb n nx prgrmmng. From gustavo at niemeyer.net Sat Jun 18 23:20:21 2005 From: gustavo at niemeyer.net (Gustavo Niemeyer) Date: Sat, 18 Jun 2005 18:20:21 -0300 Subject: [Python-Dev] PyPI: no space left on device In-Reply-To: <20050618210901.GB848@panix.com> References: <20050618203942.GA15154@burma.localdomain> <20050618210901.GB848@panix.com> Message-ID: <20050618212021.GA23867@burma.localdomain> > > PyPI seems to be out of space: > > FYI, python-dev is not a good place to send messages like this. Please > use webmaster at python.org. (I've already notified the appropriate > parties.) Before sending the message I thought, "let's see how much time it takes until someone mentions the right place to deliver the message". Adding that address to the PyPI page itself would be valueable, and will probably save python-dev from further misinformed reporters. Thanks for forwarding it this time, -- Gustavo Niemeyer http://niemeyer.net From pje at telecommunity.com Sat Jun 18 23:50:36 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 18 Jun 2005 17:50:36 -0400 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects Message-ID: <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> Working on the PEP 342/343 generator enhancements, I've got working send/throw/close() methods, but am not sure how to deal with getting __del__ to invoke close(). Naturally, I can add a "__del__" entry to its methods list easily enough, but the 'has_finalizer()' function in gcmodule.c only checks for a __del__ attribute on instance objects, and for tp_del on heap types. It looks to me like the correct fix would be to check for tp_del always, not just on heap types. However, when I tried this, I started getting warnings from the tests, saying that 22 uncollectable objects were being created (all generators, in test_generators). It seems that the tests create cycles via globals(), since they define a bunch of generator functions and then call them, saving the generator iterators (or objects that reference them) in global variables after investigating this a bit, it seems to me that either has_finalizer() needs to From pje at telecommunity.com Sat Jun 18 23:56:54 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 18 Jun 2005 17:56:54 -0400 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects In-Reply-To: <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com> At 05:50 PM 6/18/2005 -0400, Phillip J. Eby wrote: >Working on the PEP 342/343 generator enhancements, I've got working >send/throw/close() methods, but am not sure how to deal with getting >__del__ to invoke close(). Naturally, I can add a "__del__" entry to its >methods list easily enough, but the 'has_finalizer()' function in >gcmodule.c only checks for a __del__ attribute on instance objects, and for >tp_del on heap types. > >It looks to me like the correct fix would be to check for tp_del always, >not just on heap types. However, when I tried this, I started getting >warnings from the tests, saying that 22 uncollectable objects were being >created (all generators, in test_generators). > >It seems that the tests create cycles via globals(), since they define a >bunch of generator functions and then call them, saving the generator >iterators (or objects that reference them) in global variables > >after investigating this a bit, it seems to me that either has_finalizer() >needs to Whoops. I hit send by accident. Anyway, the issue seems to mostly be that the tests create generator-iterators in global variables. With a bit of effort, I've been able to stomp most of the cycles. >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com From pje at telecommunity.com Sun Jun 19 00:00:46 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 18 Jun 2005 18:00:46 -0400 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects In-Reply-To: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com> References: <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050618175716.01e413a8@mail.telecommunity.com> Argh! My email client's shortcut for Send is Ctrl-E, which is the same as end-of-line in the editor I've been using all day. Anyway, the problem is that it seems to me as though actually checking for tp_del is too aggressive (conservative?) for generators, because sometimes a generator object is finished or un-started, and therefore can't resurrect objects during close(). However, I don't really know how to implement another strategy; gcmodule isn't exactly my forte. :) Any input from the GC gurus would be appreciated. Thanks! At 05:56 PM 6/18/2005 -0400, Phillip J. Eby wrote: >At 05:50 PM 6/18/2005 -0400, Phillip J. Eby wrote: > >Working on the PEP 342/343 generator enhancements, I've got working > >send/throw/close() methods, but am not sure how to deal with getting > >__del__ to invoke close(). Naturally, I can add a "__del__" entry to its > >methods list easily enough, but the 'has_finalizer()' function in > >gcmodule.c only checks for a __del__ attribute on instance objects, and for > >tp_del on heap types. > > > >It looks to me like the correct fix would be to check for tp_del always, > >not just on heap types. However, when I tried this, I started getting > >warnings from the tests, saying that 22 uncollectable objects were being > >created (all generators, in test_generators). > > > >It seems that the tests create cycles via globals(), since they define a > >bunch of generator functions and then call them, saving the generator > >iterators (or objects that reference them) in global variables > > > >after investigating this a bit, it seems to me that either has_finalizer() > >needs to > >Whoops. I hit send by accident. Anyway, the issue seems to mostly be that >the tests create generator-iterators in global variables. With a bit of >effort, I've been able to stomp most of the cycles. From pje at telecommunity.com Sun Jun 19 01:16:25 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 18 Jun 2005 19:16:25 -0400 Subject: [Python-Dev] Generator enhancements patch available Message-ID: <5.1.1.6.0.20050618191302.03424200@mail.telecommunity.com> I've just submitted patch 1223381 (http://python.org/sf/1223381), which implements code and test changes for: * yield expressions * bare yield (short for yield None) * yield in try/finally * generator.send(value) (send value into generator; substituted for PEP 342's next(arg)) * generator.throw(typ[,val[,tb]]) (raise error in generator) * generator.close() * GeneratorExit built-in exception type * generator.__del__ (well, the C equivalent) * All necessary mods to the compiler, parser module, and Python 'compiler' package to support these changes. It was necessary to change a small part of the eval loop (well, the initialization, not the loop) and the gc module's has_finalizer() logic in order to support a C equivalent to __del__. Specialists in these areas should probably scrutinize this patch! There is one additional implementation detail that was not contemplated in either PEP. in order to prevent used-up generators from retaining unnecessary references to their frame's contents, I set the generator's gi_frame member to None whenever the generator finishes normally or with an error. Thus, an exhausted generator cannot be part of a cycle, and it releases its frame object sooner than in previous Python versions. For generators used only in a direct "for" loop, this makes no difference, but for generators used with the iterator protocol (i.e. "gen.next()") from Python, this avoids stranding the generator's frame in a traceback cycle. Anyway, your comments/questions/feedback/bug reports are welcome. From bjourne at gmail.com Sun Jun 19 01:12:11 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Sun, 19 Jun 2005 01:12:11 +0200 Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral Literals to Python In-Reply-To: References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> Message-ID: <740c3aec05061816127acb2469@mail.gmail.com> *cough* Would it also be possible for the PEP-maintainers not to accept PEPs that are obvious jokes unless thedate is April I? *uncough* -- mvh Bj?rn From pje at telecommunity.com Sun Jun 19 00:24:48 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 18 Jun 2005 18:24:48 -0400 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects In-Reply-To: <5.1.1.6.0.20050618175716.01e413a8@mail.telecommunity.com> References: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com> <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> One more note; I tried changing generators to set their gi_frame to None whenever the generator finishes normally or with an error; this eliminated most of the reference cycles, and I was able to make test_generators work correctly with only 3 explicit close() calls, for the "fun" tests that use objects which hold references to generators that in turn reference the object. So, I think I've got this sorted out, assuming that I'm not doing something hideously insane by having 'has_finalizer()' always check tp_del even for non-heap types, and defining a tp_del slot for generators to call close() in. I ended up having to copy a bunch of stuff from typeobject.c in order to make this work, as there doesn't appear to be any way to share stuff like subtype_del and subtype_dealloc in a meaningful way with the generator code. At 06:00 PM 6/18/2005 -0400, Phillip J. Eby wrote: >Argh! My email client's shortcut for Send is Ctrl-E, which is the same as >end-of-line in the editor I've been using all day. Anyway, the problem is >that it seems to me as though actually checking for tp_del is too >aggressive (conservative?) for generators, because sometimes a generator >object is finished or un-started, and therefore can't resurrect objects >during close(). However, I don't really know how to implement another >strategy; gcmodule isn't exactly my forte. :) Any input from the GC gurus >would be appreciated. Thanks! > >At 05:56 PM 6/18/2005 -0400, Phillip J. Eby wrote: > >At 05:50 PM 6/18/2005 -0400, Phillip J. Eby wrote: > > >Working on the PEP 342/343 generator enhancements, I've got working > > >send/throw/close() methods, but am not sure how to deal with getting > > >__del__ to invoke close(). Naturally, I can add a "__del__" entry to its > > >methods list easily enough, but the 'has_finalizer()' function in > > >gcmodule.c only checks for a __del__ attribute on instance objects, > and for > > >tp_del on heap types. > > > > > >It looks to me like the correct fix would be to check for tp_del always, > > >not just on heap types. However, when I tried this, I started getting > > >warnings from the tests, saying that 22 uncollectable objects were being > > >created (all generators, in test_generators). > > > > > >It seems that the tests create cycles via globals(), since they define a > > >bunch of generator functions and then call them, saving the generator > > >iterators (or objects that reference them) in global variables > > > > > >after investigating this a bit, it seems to me that either has_finalizer() > > >needs to > > > >Whoops. I hit send by accident. Anyway, the issue seems to mostly be that > >the tests create generator-iterators in global variables. With a bit of > >effort, I've been able to stomp most of the cycles. > >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com From martin at v.loewis.de Sun Jun 19 01:42:20 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Jun 2005 01:42:20 +0200 Subject: [Python-Dev] PyPI: no space left on device In-Reply-To: <20050618212021.GA23867@burma.localdomain> References: <20050618203942.GA15154@burma.localdomain> <20050618210901.GB848@panix.com> <20050618212021.GA23867@burma.localdomain> Message-ID: <42B4B15C.1030206@v.loewis.de> Gustavo Niemeyer wrote: > Before sending the message I thought, "let's see how much time it > takes until someone mentions the right place to deliver the message". > Adding that address to the PyPI page itself would be valueable, and > will probably save python-dev from further misinformed reporters. > > Thanks for forwarding it this time, Unfortunately, the "right place" depends on the nature of the problem: could be a PyPI problem, could be a pydotorg problem, could be a distutils problem. As for "adding (that) address to the PyPI page itself": How did you miss the "Get help" and "Bug reports" links below "Contact Us" on the PyPI page? They would have brought you to the PyPI SF trackers, but that would also have been the right place. Regards, Martin From martin at v.loewis.de Sun Jun 19 01:47:45 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 19 Jun 2005 01:47:45 +0200 Subject: [Python-Dev] Propose to reject PEP 313 -- Adding Roman Numeral Literals to Python In-Reply-To: <740c3aec05061816127acb2469@mail.gmail.com> References: <000a01c57307$4c5a9fa0$bb07a044@oemcomputer> <740c3aec05061816127acb2469@mail.gmail.com> Message-ID: <42B4B2A1.5090903@v.loewis.de> BJ?rn Lindqvist wrote: > Would it also be possible for the PEP-maintainers not to accept PEPs > that are obvious jokes unless thedate is April I? I believe this is the current policy. Why do you think the PEP editor works differently? Regards, Martin From abo at minkirri.apana.org.au Sun Jun 19 09:48:19 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Sun, 19 Jun 2005 00:48:19 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> Message-ID: <42B52343.3050902@minkirri.apana.org.au> Kay Schluehr wrote: > Josiah Carlson wrote: > > > Kay Schluehr wrote: > > > > > >> Maybe anonymus function closures should be pushed forward right now > not only syntactically? Personally I could live with lambda or several > >> of the alternative syntaxes listed on the wiki page. I must admit I ended up deleting most of the "alternative to lambda" threads after they flooded my in box. So it is with some dread I post this, contributing to it... As I see it, a lambda is an anonymous function. An anonymous function is a function without a name. We already have a syntax for a function... why not use it. ie: f = filter(def (a): return a > 1, [1,2,3]) The implications of this are that both functions and procedures can be anonymous. This also implies that unlike lamba's, anonymous functions can have statements, not just expressions. You can even do compound stuff like; f = filter(def (a): b=a+1; return b>1, [1,2,3]) or if you want you can use indenting; f = filter(def (a): b=a+1 return b>1, [1,2,3]) It also means the following becomes valid syntax; f = def (a,b): return a>b I'm not sure if there are syntactic ambiguities to this. I'm not sure if the CS boffins are disturbed by "side effects" from statements. Perhaps both can be resolved by limiting annonymous functions to expressions. Or require use of brackets or ";" to resolve ambiguity. This must have been proposed already and shot down in flames... sorry for re-visiting old stuff and contributing noise. -- Donovan Baarda From simonwittber at gmail.com Sun Jun 19 02:49:02 2005 From: simonwittber at gmail.com (Simon Wittber) Date: Sun, 19 Jun 2005 08:49:02 +0800 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <17076.21429.501499.914085@montanaro.dyndns.org> References: <4e4a11f805061723192888915e@mail.gmail.com> <7168d65a0506180644367753b9@mail.gmail.com> <4e4a11f805061807127f39700b@mail.gmail.com> <17076.21429.501499.914085@montanaro.dyndns.org> Message-ID: <4e4a11f805061817495428952d@mail.gmail.com> > Why this discussion of yet another serialization format? Pickle is stated to be unsafe. Marshal is also stated to be unsafe. XML can be bloated, and XML+gzip is quite slow. Do size,speed, and security features have to mutually exclusive? No, that possibly is why people have had to invent their own formats. I can list four off the top of my head: bencode (bittorrent) jelly (twisted) banana (twisted) tofu (soya3d, looks like it is using twisted now... hmmm) XML is simply not suitable for database appplications, real time data capture and game/entertainment applications. I'm sure other people have noticed this... or am I alone on this issue? :-) Have a look at this contrived example: import time value = (("this is a record",1,2,1000001,"(08)123123123","some more text")*10000) import gherkin t = time.clock() s = gherkin.dumps(value) print 'Gherkin encode', time.clock() - t, 'seconds' t = time.clock() gherkin.loads(s) print 'Gherkin decode', time.clock() - t, 'seconds' import xmlrpclib t = time.clock() s = xmlrpclib.dumps(value) print 'XMLRPC encode', time.clock() - t, 'seconds' t = time.clock() xmlrpclib.loads(s) print 'XMLRPC decode', time.clock() - t, 'seconds' Which produces the output: >pythonw -u "bench.py" Gherkin encode 0.120689361357 seconds Gherkin decode 0.395871262968 seconds XMLRPC encode 0.528666352847 seconds XMLRPC decode 9.01307819849 seconds From ncoghlan at gmail.com Sun Jun 19 02:49:56 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 19 Jun 2005 10:49:56 +1000 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B52343.3050902@minkirri.apana.org.au> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> Message-ID: <42B4C134.8010308@gmail.com> Donovan Baarda wrote: > As I see it, a lambda is an anonymous function. An anonymous function is > a function without a name. And here we see why I'm such a fan of the term 'deferred expression' instead of 'anonymous function'. Python's lambda expressions *are* the former, but they are emphatically *not* the latter. Anyway, the AlternateLambdaSyntax Wiki page has a couple of relevant entries under 'real closures'. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From nas at arctrix.com Sun Jun 19 02:50:24 2005 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 18 Jun 2005 18:50:24 -0600 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects In-Reply-To: <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> References: <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com> <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> Message-ID: <20050619005024.GA14709@mems-exchange.org> On Sat, Jun 18, 2005 at 06:24:48PM -0400, Phillip J. Eby wrote: > So, I think I've got this sorted out, assuming that I'm not doing > something hideously insane by having 'has_finalizer()' always > check tp_del even for non-heap types, and defining a tp_del slot > for generators to call close() in. That sounds like the right thing to do. I suspect the "uncollectable cycles" problem will not be completely solvable. With this change, all generators become objects with finalizers. In reality, a 'file' object, for example, has a finalizer as well but it gets away without telling the GC that because its finalizer doesn't do anything "evil". Since generators can do arbitrary things, the GC must assume the worst. Most cycles involving enhanced generators can probably be broken by the GC because the generator is not in the strongly connected part of cycle. The GC will have to work a little harder to figure that out but that's probably not too significant. The real problem is that some cycles involving enhanced generators will not be breakable by the GC. I think some programs that used to work okay are now going to start leaking memory because objects will accumulate in gc.garbage. Now, I could be wrong about all this. I've have not been following the PEP 343 discussion too closely. Maybe Guido has some clever idea. Also, I find it difficult to hold in my head a complete model of how the GC now works. It's an incredibly subtle piece of code. Perhaps Tim can comment. Neil From jcarlson at uci.edu Sun Jun 19 02:51:19 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 18 Jun 2005 17:51:19 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <20050618011502.728D.JCARLSON@uci.edu> Message-ID: <20050618171515.72A5.JCARLSON@uci.edu> Kay Schluehr wrote: > Josiah Carlson wrote: > > > Kay Schluehr wrote: > >> The arrow is a straightforward punctuation for function definitions. > >> Reusing existing keywords for different semantics seems to me as a kind > >> of inbreeding. > > > > That's starting to look like the pseudocode from old algorithms > > textbooks, which is very similar to bad pseudocode from modern CS theory > > papers. Punctuation as a replacement for words does not always win > > (perfect examples being 'and' vs. &&, 'or' vs. ||, 'not' vs. !, ...) > > Writing functions as arrows is very convenient not only in CS but also > in mathematics. Looking like pseudo-code was not one of Guidos Python > regrets if I remember them correctly. Apparently you weren't reading what I typed. I don't find '->' notation for functions to be readable, either in code, papers, or otherwise. In fact, when I teach my CS courses, I use Python, and in the case where I need to describe mathematical functions, I use a standard mathematical notation for doing so: 'f(x) = ...'. > >> For pushing anymus functions forward I propose to enable explizit > >> partial evaluation as a programming technique: > > > > If I remember correctly, we've got rightcurry and leftcurry for that (or > > rightpartial and leftpartial, or something). > > > Currying usually does not perform a function evaluation in order to > create another more special function. Partial evaluation is a dynamic > programming and optimization technique. Psyco uses specialization and > caching implicitely. I propose to use it explicitely but in a more > restricted context. I never claimed that currying was partial function evaluation. In fact, if I remember correctly (I don't use curried functions), the particular currying that was implemented and shipped for Python 2.4 was a simple function that kept a list of args and kwargs that were already passed to the curried function. When it got enough arguments, it would go ahead and evaluate it. > > I'll assume that you don't actually want it to rewrite the source, or > > actually return the source representation of the anonymous function > > (those are almost non-starters). > > > > Computer algebra systems store expressions in internal tree form, > manipulate them efficiently and pretty-print them in textual or latex > output on demand. There would be much more involved than a tree to tree > translation starting with Pythons internal parse tree and an evaluator > dedicated to it. Right, and Python is not a CAS. You want a CAS? Use Mathematica, Maple, or some CAS addon to Python. Keeping the parse tree around when it is not needed is a waste. In the code that I write, that would be 100% of the time. > > As for all anonymous functions allowing partial evaluation via keywords: > > it would hide errors. Right now, if you forget an argument or add too > > many arguments, you get a TypeError. Your proposal would make > > forgetting an argument in certain ways return a partially evaluated > > function. > > > That's why I like to dump the function in a transparent mode. Personally > I could dispense a little security in favor for cheap metainformation. It's not about security, it's about practicality and backwards compatibility. Here is an example: >>> x = lambda a,b: a+b+1 >>> x(a=1) Traceback (most recent call last): File "", line 1, in ? TypeError: () takes exactly 2 non-keyword arguments (1 given) >>> According to you, that should instead produce... lambda b: 1+b+1 New and old users alike have gotten used to the fact that calling a function without complete arguments is an error. You are proposing to make calling a function with less than the required number of arguments not be an error. For the sake of sanity, I have to be -1000 on this. I'm also not aware of an automatic mechanism for code compilation (in Python) that both attaches the parse tree to the compiled function, or allows for the conversion of that parse tree back to a source form, or allows one to replace variable references with constants. If you would care to write the pieces necessary, I'm sure you will find a use for it. - Josiah From jcarlson at uci.edu Sun Jun 19 03:03:58 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 18 Jun 2005 18:03:58 -0700 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <4e4a11f805061817495428952d@mail.gmail.com> References: <17076.21429.501499.914085@montanaro.dyndns.org> <4e4a11f805061817495428952d@mail.gmail.com> Message-ID: <20050618175824.72A8.JCARLSON@uci.edu> Simon Wittber wrote: > > Why this discussion of yet another serialization format? > > Pickle is stated to be unsafe. Marshal is also stated to be unsafe. > XML can be bloated, and XML+gzip is quite slow. > > Do size,speed, and security features have to mutually exclusive? No, > that possibly is why people have had to invent their own formats. I > can list four off the top of my head: ... Looks to me like the eval(repr(obj)) loop spanks XMLRPC. It likely also spanks Gherkin, but I'm not one to run untrusted code. Give it a shot on your own machine. As for parsing, repr() of standard Python objects is pretty easy to parse, and if you want something a bit easier to read, there's always pprint (though it may not be quite as fast). - Josiah Python 2.3.4 (#53, May 25 2004, 21:17:02) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> value = (("this is a record",1,2,1000001,"(08)123123123","some moretext")*10000) >>> import time >>> t = time.time();y = repr(value);time.time()-t 0.030999898910522461 >>> t = time.time();z = eval(y);time.time()-t 0.26600003242492676 >>> import xmlrpclib >>> t = time.time();n = xmlrpclib.dumps(value);time.time()-t 0.42100000381469727 >>> t = time.time();m = xmlrpclib.loads(n);time.time()-t 4.4529998302459717 >>> From python at rcn.com Sun Jun 19 04:09:24 2005 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Jun 2005 22:09:24 -0400 Subject: [Python-Dev] Recommend accepting PEP 312 -- SimpleImplicit Lambda In-Reply-To: <42B52343.3050902@minkirri.apana.org.au> Message-ID: <000901c57473$ea3610c0$7c03a044@oemcomputer> [Donovan Baarda] > As I see it, a lambda is an anonymous function. An anonymous function is > a function without a name. We already have a syntax for a function... > why not use it. ie: > > f = filter(def (a): return a > 1, [1,2,3]) This approach is entirely too obvious. If we want to be on the leading edge, we can't be copying what was done years ago in Lua. ;-) Raymond From pje at telecommunity.com Sun Jun 19 04:15:54 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 18 Jun 2005 22:15:54 -0400 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects In-Reply-To: <20050619005024.GA14709@mems-exchange.org> References: <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com> <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050618212627.01e22658@mail.telecommunity.com> At 06:50 PM 6/18/2005 -0600, Neil Schemenauer wrote: >On Sat, Jun 18, 2005 at 06:24:48PM -0400, Phillip J. Eby wrote: > > So, I think I've got this sorted out, assuming that I'm not doing > > something hideously insane by having 'has_finalizer()' always > > check tp_del even for non-heap types, and defining a tp_del slot > > for generators to call close() in. > >That sounds like the right thing to do. > >I suspect the "uncollectable cycles" problem will not be completely >solvable. With this change, all generators become objects with >finalizers. In reality, a 'file' object, for example, has a >finalizer as well but it gets away without telling the GC that >because its finalizer doesn't do anything "evil". Since generators >can do arbitrary things, the GC must assume the worst. Yep. It's too bad that there's no simple way to guarantee that the generator won't resurrect anything. On the other hand, close() is guaranteed to give the generator at most one chance to do this. So, perhaps there's some way we could have the GC close() generators in unreachable cycles. No, wait, that would mean they could resurrect things, right? Argh. >Most cycles involving enhanced generators can probably be broken by >the GC because the generator is not in the strongly connected part >of cycle. The GC will have to work a little harder to figure that >out but that's probably not too significant. Yep; by setting the generator's frame to None, I was able to significantly reduce the number of generator cycles in the tests. >The real problem is that some cycles involving enhanced generators >will not be breakable by the GC. I think some programs that used to >work okay are now going to start leaking memory because objects will >accumulate in gc.garbage. Yep, unless we .close() generators after adding them to gc.garbage(), which *might* be an option. Although, I suppose if it *were* an option, then why doesn't GC already have some sort of ability to do this? (i.e. run __del__ methods on items in gc.garbage, then remove them if their refcount drops to 1 as a result). [...pause to spend 5 minutes working it out in pseudocode...] Okay, I think I see why you can't do it. You could guarantee that all relevant __del__ methods get called, but it's bloody difficult to end up with only unreachable items in gc.garbage afterwards. I think gc would have to keep a new list for items reachable from finalizers, that don't themselves have finalizers. Then, before creating gc.garbage, you walk the finalizers and call their finalization (__del__) methods. Then, you put any remaining items that are in either the finalizer list or the reachable-from-finalizers list into gc.garbage. This approach might need a new type slot, but it seems like it would let us guarantee that finalizers get called, even if the object ends up in garbage as a result. In the case of generators, however, close() guarantees that the generator releases all its references, and so can no longer be part of a cycle. Thus, it would guarantee eventual cleanup of all generators. And, it would lift the general limitation on __del__ methods. Hm. Sounds too good to be true. Surely if this were possible, Uncle Timmy would've thought of it already, no? Guess we'll have to wait and see what he thinks. >Now, I could be wrong about all this. I've have not been following >the PEP 343 discussion too closely. Maybe Guido has some clever >idea. Also, I find it difficult to hold in my head a complete model >of how the GC now works. It's an incredibly subtle piece of code. >Perhaps Tim can comment. I'm hoping Uncle Timmy can work his usual algorithmic magic here and provide us with a brilliant but impossible-for-mere-mortals-to-understand solution. (The impossible-to-understand part being optional, of course. :) ) From python at rcn.com Sun Jun 19 05:28:51 2005 From: python at rcn.com (Raymond Hettinger) Date: Sat, 18 Jun 2005 23:28:51 -0400 Subject: [Python-Dev] Recommend accepting PEP 312 --SimpleImplicit Lambda In-Reply-To: <000901c57473$ea3610c0$7c03a044@oemcomputer> Message-ID: <000001c5747f$040c22e0$7c03a044@oemcomputer> > [Donovan Baarda] > > As I see it, a lambda is an anonymous function. An anonymous function > is > > a function without a name. We already have a syntax for a function... > > why not use it. ie: > > > > f = filter(def (a): return a > 1, [1,2,3]) [Me] > This approach is entirely too obvious. If we want to be on the leading > edge, we can't be copying what was done years ago in Lua. ;-) Despite the glib comment, the idea as presented doesn't work because it mixes statement and expression semantics (the inner 'return' is at best unpleasant). Also, the naming is off -- 'def' defines some variable name. In Lua, the 'def' is called 'function' which doesn't imply a variable assignment. Raymond From skip at pobox.com Sun Jun 19 06:07:57 2005 From: skip at pobox.com (Skip Montanaro) Date: Sat, 18 Jun 2005 23:07:57 -0500 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <4e4a11f805061817462238af59@mail.gmail.com> References: <4e4a11f805061723192888915e@mail.gmail.com> <7168d65a0506180644367753b9@mail.gmail.com> <4e4a11f805061807127f39700b@mail.gmail.com> <17076.21429.501499.914085@montanaro.dyndns.org> <4e4a11f805061817462238af59@mail.gmail.com> Message-ID: <17076.61341.558478.28695@montanaro.dyndns.org> Simon> XML is simply not suitable for database appplications, real time Simon> data capture and game/entertainment applications. I use XML-RPC as the communications protocol between an Apache web server and a middleware piece that talks to a MySQL database. The web server contains a mixture of CGI scripts written in Python and two websites written in Mason (Apache+mod_perl). Performance is fine. Give either of these a try: http://www.mojam.com/ http://www.musi-cal.com/ Simon> I'm sure other people have noticed this... or am I alone on this Simon> issue? :-) Probably not. XML-RPC is commonly thought of as slow, and if you operate with it in a completely naive fashion, I don't doubt that it can be. It doesn't have to be though. Do you have to be intelligent about caching frequently used information and returning results in reasonably sized chunks? Sure, but that's probably true of any database-backed application. Simon> Have a look at this contrived example: ... Simon> Which produces the output: >> pythonw -u "bench.py" Simon> Gherkin encode 0.120689361357 seconds Simon> Gherkin decode 0.395871262968 seconds Simon> XMLRPC encode 0.528666352847 seconds Simon> XMLRPC decode 9.01307819849 seconds That's fine, so XML-RPC is slower than Gherkin. I can't run the Gherkin code, but my XML-RPC numbers are a bit different than yours: XMLRPC encode 0.65 seconds XMLRPC decode 2.61 seconds That leads me to believe you're not using any sort of C XML decoder. (I mentioned sgmlop in my previous post. I'm sure /F has some other super-duper accelerator that's even faster.) I'm not saying that XML-RPC is the fastest thing on Earth. I'd be willing to bet it's a lot more interoperable than Gherkin is though: http://xmlrpc.scripting.com/directory/1568/implementations and probably will be for the forseeable future. Also, as you indicated, your example was a bit contrived. XML-RPC seems to be fast enough for many real-world applications. Here's a somewhat less-contrived example from the above websites: >>> orca >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 200, 50, 10000) ; print time.time() - t 1.28429102898 >>> len(x[1]) 200 >>> x[1][0] ['MusicEntry', {'venue': 'Jazz Showcase', 'address': '', 'price': '', 'keywords': ['.jz.1369'], 'event': '', 'city': 'Chicago', 'end': datetime.datetime(2005, 6, 19, 0, 0), 'zip': '', 'start': datetime.datetime(2005, 6, 14, 0, 0), 'state': 'IL', 'program': '', 'email': 'skip at mojam.com', 'info': '', 'update_time': datetime.datetime(2005, 6, 8, 0, 0), 'address1': '59 West Grand Avenue', 'address2': '', 'address3': '', 'venueid': 2630, 'key': 816875, 'submit_time': datetime.datetime(2005, 6, 8, 0, 0), 'active': 1, 'merchandise': '', 'tickets': '', 'name': '', 'addressid': 17056, 'performers': ['Cedar Walton Quartet'], 'country': '', 'venueurl': '', 'time': '', 'performerids': [174058]}] >>> orca.checkpoint() 'okay' >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 200, 50, 10000) ; print time.time() - t 1.91681599617 orca is an xmlrpclib proxy to the aforementioned middleware component. (These calls are being made to a production server.) The search() call gets the first 200 concert listings within a 50-mile radius of Chicago. x[1][0] is the first item returned. All 200 returned items are of the same complexity. 1.28 seconds certainly isn't Earth-shattering performance. Even worse is the 1.92 seconds after the checkpoint (which flushes the caches forcing the MySQL database to be queried for everything). Returning 200 items is also contrived. Real users can't ask for that many items at a time through the web interface. Cutting it down to 20 items (which is what users get by default) shows a different story: >>> orca.checkpoint() 'okay' >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 20, 50, 10000) ; print time.time() - t 0.29478096962 >>> t = time.time() ; x = orca.search(('MusicEntry', {'city': 'Chicago, IL'}), 0, 20, 50, 10000) ; print time.time() - t 0.0978591442108 The first query after a checkpoint is slow because we have to go to MySQL a few times, but once everything's cached, things are fast. The 0.1 second time for the last call is going to be almost all XML-RPC overhead, because all the data's already been cached. I find that acceptable. If you go to the Mojam website and click "Chicago", the above query is pretty much what's performed (several other queries are also run to get corollary info). I still find it hard to believe that yet another serialization protocol is necessary. XML is certainly overkill for almost everything. I'll be the first to admit that. (I have to struggle with it as a configuration file format at work.) However, it is certainly widely available. Skip From simonwittber at gmail.com Sun Jun 19 06:37:56 2005 From: simonwittber at gmail.com (Simon Wittber) Date: Sun, 19 Jun 2005 12:37:56 +0800 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <17076.61341.558478.28695@montanaro.dyndns.org> References: <4e4a11f805061723192888915e@mail.gmail.com> <7168d65a0506180644367753b9@mail.gmail.com> <4e4a11f805061807127f39700b@mail.gmail.com> <17076.21429.501499.914085@montanaro.dyndns.org> <4e4a11f805061817462238af59@mail.gmail.com> <17076.61341.558478.28695@montanaro.dyndns.org> Message-ID: <4e4a11f805061821378cd84b6@mail.gmail.com> > I use XML-RPC as the communications protocol between an Apache web server > and a middleware piece that talks to a MySQL database. The web server > contains a mixture of CGI scripts written in Python and two websites written > in Mason (Apache+mod_perl). Performance is fine. Give either of these a > try: I've also implemented some middleware between Apache and SQL Server, using XMLRPC. Unfortunately, large queries were the order of the day, and things started moving like treacle. I profiled the code, and discovered that 30 seconds of the 35 second response time was being chewed up in building and parsing XML. I hacked things a bit, and instead of sending XML, sent pickles inside the XML response. This dropped response times to around the 8 second mark. This was still not good enough, as multiple queries could still choke up the system. Typical queries were about 1000-10000 records of 3-15 fields of varying datatypes, including unicode. Don't ask me why people needed these kind of reports, I'm just the programmer, and don't always get to control design decisions. :-) This demonstrates that sometimes, in real world apps, large amounts of data need to be shipped around, and it needs to happen fast. I guess when interoperability counts, XMLRPC is the best solution. When controlling both ends of the connection, pickle _should_ be best, except it is stated as being unsecure. > That's fine, so XML-RPC is slower than Gherkin. I can't run the Gherkin > code, but my XML-RPC numbers are a bit different than yours: > > XMLRPC encode 0.65 seconds > XMLRPC decode 2.61 seconds > > That leads me to believe you're not using any sort of C XML decoder. I used the standard library xmlrpclib only. > I'm not saying that XML-RPC is the fastest thing on Earth. I'd be willing > to bet it's a lot more interoperable than Gherkin is though: Agreed. Interoperability was a request in the aforementioned RFE, though it was not an explicit goal of mine. I needed a very fast serialization routine, which could safely unserialize messages from untrusted sources, for sending big chunks of data over TCP. I also needed it to be suitable for use in real-time multiplayer games (my part time hobby). XMLRPC doesn't suit this application. When I control the server, and the client applications, I shouldn't have to use XML. Thanks for the feedback. Simon. From raymond.hettinger at verizon.net Sun Jun 19 07:43:27 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 19 Jun 2005 01:43:27 -0400 Subject: [Python-Dev] Propose to reject PEP 294 -- Type Names in the types Module Message-ID: <000001c57491$d1dc8fe0$25fccc97@oemcomputer> Introducing a new set of duplicate type names and deprecating old ones causes a certain amount of disruption. Given the age of the types module, the disruption is likely to be greater than any potential benefit that could be realized. Plenty of people will have to incur the transition costs, but no one will likely find the benefit to be perceptible. Suggest rejecting this PEP and making a note for Py3.0 to either sync-up the type names or abandon the types module entirely. Raymond From raymond.hettinger at verizon.net Sun Jun 19 08:50:54 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 19 Jun 2005 02:50:54 -0400 Subject: [Python-Dev] Question about PEP 330 -- Python Bytecode Verification Message-ID: <000101c5749b$3d5a2d00$25fccc97@oemcomputer> Do we have *any* known use cases where we would actually run bytecode that was suspicious enough to warrant running a well-formedness check? In assessing security risks, the PEP notes, "Practically, it would be difficult for a malicious user to 'inject' invalid bytecode into a PVM for the purposes of exploitation, but not impossible." Can that ever occur without there being a far greater risk of malicious, but well-formed bytecode? If you download a file, foo.pyc, from an untrusted source and run it in a susceptible environment, does its well-formedness give you *any* feeling of security. I think not. There isn't anything wrong with having a verifier module, but I can't think of any benefit that would warrant changing the bytecode semantics just to facilitate one of the static stack checks. Raymond From kay.schluehr at gmx.net Sun Jun 19 08:52:19 2005 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sun, 19 Jun 2005 08:52:19 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B52343.3050902@minkirri.apana.org.au> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> Message-ID: Donovan Baarda wrote: > I must admit I ended up deleting most of the "alternative to lambda" > threads after they flooded my in box. So it is with some dread I post > this, contributing to it... I must admit you are right. And I will stop defending my proposal because it seems to create nothing more than pointless polemics. Suggesting elements of FP to python-dev may have the same chances to be accepted than raising a Buddha statue in a catholic church. > As I see it, a lambda is an anonymous function. An anonymous function is > a function without a name. We already have a syntax for a function... > why not use it. ie: > > f = filter(def (a): return a > 1, [1,2,3]) You mix expressions with statements. This is a no-go in Python. Permitting those constructs is a radical language change not just a simple syntax replacement. Kay From abo at minkirri.apana.org.au Sun Jun 19 16:55:29 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Sun, 19 Jun 2005 07:55:29 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B4C134.8010308@gmail.com> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> Message-ID: <42B58761.7090701@minkirri.apana.org.au> Nick Coghlan wrote: > Donovan Baarda wrote: > >>As I see it, a lambda is an anonymous function. An anonymous function is >>a function without a name. > > > And here we see why I'm such a fan of the term 'deferred expression' > instead of 'anonymous function'. But isn't a function just a deferred expression with a name :-) As a person who started out writing assembler where every "function" I wrote was a macro that got expanded inline, the distiction is kinda blurry to me. > Python's lambda expressions *are* the former, but they are > emphatically *not* the latter. Isn't that because lambda's have the limitation of not allowing statements, only expressions? I know this limitation avoids side-effects and has significance in some formal (functional?) languages... but is that what Python is? In the Python I use, lambda's are always used where you are too lazy to define a function to do it's job. To me, anonymous procedures/functions would be a superset of "deferred expressions", and if the one stone fits perfectly in the slingshot we have and can kill multiple birds... why hunt for another stone? Oh yeah Raymond: on the "def defines some variable name"... are you joking? You forgot the smiley :-) I don't get what the problem is with mixing statement and expression semantics... from a practial point of view, statements just offer a superset of expression functionality. If there really is a serious practical reason why they must be limited to expressions, why not just raise an exception or something if the "anonymous function" is too complicated... I did some fiddling and it seems lambda's can call methods and stuff that can have side effects, which kinda defeats what I thought was the point of "statements vs expressions"... I guess I just don't understand... maybe I'm just thick :-) > Anyway, the AlternateLambdaSyntax Wiki page has a couple of relevant > entries under 'real closures'. Where is that wiki BTW? I remember looking at it ages ago but can't find the link anymore. -- Donovan Baarda From raymond.hettinger at verizon.net Sun Jun 19 09:03:50 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun, 19 Jun 2005 03:03:50 -0400 Subject: [Python-Dev] Propose to close PEP 254 -- Making Classes Look More Like Types Message-ID: <000201c5749d$0c133500$25fccc97@oemcomputer> This PEP is an empty stub that is unlikely to ever get filled-out in a way that adds anything beyond what is already implemented and documented. Raymond From ncoghlan at gmail.com Sun Jun 19 09:32:39 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 19 Jun 2005 17:32:39 +1000 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B58761.7090701@minkirri.apana.org.au> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> <42B58761.7090701@minkirri.apana.org.au> Message-ID: <42B51F97.3080206@gmail.com> Donovan Baarda wrote: > Nick Coghlan wrote: >> And here we see why I'm such a fan of the term 'deferred expression' >> instead of 'anonymous function'. > > But isn't a function just a deferred expression with a name :-) According to the specific meaning of 'expression' in the Python grammar? No. > Isn't that because lambda's have the limitation of not allowing > statements, only expressions? I know this limitation avoids side-effects > and has significance in some formal (functional?) languages... but is > that what Python is? Personally, I believe it has to do with two things, one practical and one more stylistic/philosophical. Firstly, Python uses significant whitespace between statements, and to delineate sections of compound statements. Within expressions however, whitespace is essentially insignificant (except where it affects the tokenisation). Trying to embed suites (where whitespace can be significant) inside expressions (where whitespace is generally insiginificant) tends to result in a horrible mess, or else some rather unPythonic concepts (e.g. using something other than indentation to demarcate the suite). Secondly, trying to cram too much functionality into a single line is a questionable activity. The 'deferred expression' approach taken with lambdas is a hint that if the thing you're trying to defer is more complex than a single expression, it may be worth giving it a proper name via 'def'. > In the Python I use, lambda's are always used where > you are too lazy to define a function to do it's job. In simple cases, naming the expression to be deferred results in it gaining a prominence it doesn't deserve, possibly obscuring the purpose of the main statement. In such cases, deferred expressions are a better solution - it's merely unfortunate that Python's current syntax for the practice anything to write home about. > To me, anonymous procedures/functions would be a superset of "deferred > expressions", and if the one stone fits perfectly in the slingshot we > have and can kill multiple birds... why hunt for another stone? Because there is already a solution for the large birds - *named* functions. Unfortunately, using a named function for a simple deferred expression is like using a catapult to kill a canary. > If there really is a serious practical reason why they must be limited > to expressions, why not just raise an exception or something if the > "anonymous function" is too complicated... You'd still need a definition of 'too complicated' - which we already have for deferred expressions. Specifically "unable to be expressed as a single Python expression". >> Anyway, the AlternateLambdaSyntax Wiki page has a couple of relevant >> entries under 'real closures'. > > Where is that wiki BTW? I remember looking at it ages ago but can't find > the link anymore. http://wiki.python.org/moin/AlternateLambdaSyntax Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From kdart at kdart.com Sun Jun 19 09:45:27 2005 From: kdart at kdart.com (Keith Dart) Date: Sun, 19 Jun 2005 00:45:27 -0700 (PDT) Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: <2m1x6z7qu6.fsf@starship.python.net> References: <000301c5738b$6afc5fa0$bb07a044@oemcomputer> <42B3E41A.2020608@kdart.com> <2m1x6z7qu6.fsf@starship.python.net> Message-ID: On Sat, 18 Jun 2005, Michael Hudson wrote: > > The shortest way I know of going from 2149871625L to -2145095671 is > the still-fairly-gross: > >>>> v = 2149871625L >>>> ~int(~v&0xFFFFFFFF) > -2145095671 > >> I suppose the best thing is to introduce an "unsignedint" type for this >> purpose. > > Or some kind of bitfield type, maybe. > > C uses integers both as bitfields and to count things, and at least in > my opinion the default assumption in Python should be that this is > what an integer is being used for, but when you need a bitfield it can > all get a bit horrible. > > That said, I think in this case we can just make fcntl_ioctl use the > (new-ish) 'I' format argument to PyArg_ParseTuple and then you'll just > be able to use 2149871625L and be happy (I think, haven't tried this). Thanks for the reply. I think I will go ahead and add some extension types to Python. Thankfully, Python is extensible with new objects. It is also useful (to me, anyway) to be able to map, one to one, external primitives from other systems to Python primitives. For example, CORBA and SNMP have a set of types (signed ints, unsigned ints, etc.) defined that I would like to interface to Python (actually I have already done this to some degree). But Python makes it a bit more difficult without that one-to-one mapping of basic types. Having an unsigned int type, for example, would make it easier to interface Python to SNMP or even some C libraries. In other words, Since the "Real World" has these types that I must sometimes interface to, it is useful to have these same (predictable) types in Python. So, it is worth extending the basic set of data types, and I will add it to my existing collection of Python extensions. Therefore, I would like to ask here if anyone has already started something like this? If not, I will go ahead and do it (if I have time). -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: F3D288E4 ===================================================================== From jcarlson at uci.edu Sun Jun 19 10:01:47 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 19 Jun 2005 01:01:47 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 --Simple Implicit Lambda In-Reply-To: <42B58761.7090701@minkirri.apana.org.au> References: <42B4C134.8010308@gmail.com> <42B58761.7090701@minkirri.apana.org.au> Message-ID: <20050619002931.72AB.JCARLSON@uci.edu> Donovan Baarda wrote: > Nick Coghlan wrote: > > Donovan Baarda wrote: > > > >>As I see it, a lambda is an anonymous function. An anonymous function is > >>a function without a name. > > > > > > And here we see why I'm such a fan of the term 'deferred expression' > > instead of 'anonymous function'. > > But isn't a function just a deferred expression with a name :-) A function in Python is actually a deferred sequence of statements and expressions. An anonymous function in Python (a lambda) is a deferred expression. > > Python's lambda expressions *are* the former, but they are > > emphatically *not* the latter. > > Isn't that because lambda's have the limitation of not allowing > statements, only expressions? I know this limitation avoids side-effects > and has significance in some formal (functional?) languages... but is > that what Python is? In the Python I use, lambda's are always used where > you are too lazy to define a function to do it's job. I've generally seen people use lambdas for things that don't require names in the current context; i.e. callbacks with simple executions. > To me, anonymous procedures/functions would be a superset of "deferred > expressions", and if the one stone fits perfectly in the slingshot we > have and can kill multiple birds... why hunt for another stone? Are "deferred expressions" perhaps another way of spelling "function closure"? > Oh yeah Raymond: on the "def defines some variable name"... are you > joking? You forgot the smiley :-) 'def' happens to bind the name that follows the def to the function with the arguments and body following the name. > I don't get what the problem is with mixing statement and expression > semantics... from a practial point of view, statements just offer a > superset of expression functionality. Statements don't have a return value. To be more precise, what is the value of "for i in xrange(10): z.append(...)"? Examine the selection of statements available to Python, and ask that question. The only one that MAY have a return value, is 'return' itself, which really requires an expression to the right (which passes the expression to the right to the caller's frame). When you have statements that ultimately need a 'return' for a return value; you may as well use a standard function definition. > If there really is a serious practical reason why they must be limited > to expressions, why not just raise an exception or something if the > "anonymous function" is too complicated... Define "too complicated"? > I did some fiddling and it seems lambda's can call methods and stuff > that can have side effects, which kinda defeats what I thought was the > point of "statements vs expressions"... I guess I just don't > understand... maybe I'm just thick :-) There is nothing stopping anyone from modifying anything in a lambda... a = list(...) pop = lambda:a.pop() lcycle = lambda: a and a.append(a.pop(0)) rcycle = lambda: a and a.insert(0, a.pop()) popany = lambda: a and a.pop(random.randrange(len(a))) - Josiah From jcarlson at uci.edu Sun Jun 19 10:07:57 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 19 Jun 2005 01:07:57 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> Message-ID: <20050619010354.72AE.JCARLSON@uci.edu> Keith Dart wrote: > Therefore, I would like to ask here if anyone has already started > something like this? If not, I will go ahead and do it (if I have time). If all you need to do is read or write C-like types to or from memory, you should spend some time looking through the 'struct' module if you haven't already. - Josiah From kdart at kdart.com Sun Jun 19 10:25:55 2005 From: kdart at kdart.com (Keith Dart) Date: Sun, 19 Jun 2005 01:25:55 -0700 (PDT) Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: <20050619010354.72AE.JCARLSON@uci.edu> References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: On Sun, 19 Jun 2005, Josiah Carlson wrote: > > Keith Dart wrote: > >> Therefore, I would like to ask here if anyone has already started >> something like this? If not, I will go ahead and do it (if I have time). > > If all you need to do is read or write C-like types to or from memory, > you should spend some time looking through the 'struct' module if you > haven't already. I know about 'struct'. However, it will just convert to Python "native" types. C unsigned become Python longs. u = struct.pack("I", 0xfffffffe) struct.unpack("I", u) (4294967294L,) In SNMP, for example, a Counter32 is basically an unsigned int, defined as "IMPLICIT INTEGER (0..4294967295)". One cannot efficiently translate and use that type in native Python. Currently, I have defined an "unsigned" type as a subclass of long, but I don't think that would be speed or storage efficient. On the other hand, adding my own type won't help with the ioctl() problem, since it won't know about it. -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: F3D288E4 ===================================================================== From abo at minkirri.apana.org.au Sun Jun 19 18:29:21 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Sun, 19 Jun 2005 09:29:21 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 --Simple Implicit Lambda In-Reply-To: <20050619002931.72AB.JCARLSON@uci.edu> References: <42B4C134.8010308@gmail.com> <42B58761.7090701@minkirri.apana.org.au> <20050619002931.72AB.JCARLSON@uci.edu> Message-ID: <42B59D61.1000009@minkirri.apana.org.au> Josiah Carlson wrote: > Donovan Baarda wrote: > >>Nick Coghlan wrote: >> >>>Donovan Baarda wrote: [...] >>But isn't a function just a deferred expression with a name :-) > > > A function in Python is actually a deferred sequence of statements and > expressions. An anonymous function in Python (a lambda) is a deferred > expression. in the end though, a sequence of statements that completes with a "return value" is, when treated as a black box, indistinguishable from an expression. Originally I thought that this also had to be qualified with "and has no side-effects", but I see now that is not the case. [...] >>Oh yeah Raymond: on the "def defines some variable name"... are you >>joking? You forgot the smiley :-) > > > 'def' happens to bind the name that follows the def to the function with > the arguments and body following the name. Yeah, but we don't use "def" to bind arbitary variables, only functions/procedures. So in python, they are intimately identified with functions and procedures. >>I don't get what the problem is with mixing statement and expression >>semantics... from a practial point of view, statements just offer a >>superset of expression functionality. > > > Statements don't have a return value. To be more precise, what is the > value of "for i in xrange(10): z.append(...)"? Examine the selection of > statements available to Python, and ask that question. The only one > that MAY have a return value, is 'return' itself, which really requires > an expression to the right (which passes the expression to the right to > the caller's frame). When you have statements that ultimately need a > 'return' for a return value; you may as well use a standard function > definition. Hmmm. For some reason I thought that these kind of things would have a return value of None, the same as a function without an explicit return. I see now that this is not true... >>If there really is a serious practical reason why they must be limited >>to expressions, why not just raise an exception or something if the >>"anonymous function" is too complicated... > > > Define "too complicated"? I was thinking that this is up to the interpreter... depending on what the practical limitations are that cause the limitation in the first place. For example... if it can't be reduced to an "expression" through simple transforms. But look... I've gone and created another monster thread on "alternatives to lambda"... I'm going to shut up now. -- Donovan Baarda From kay.schluehr at gmx.net Sun Jun 19 13:47:54 2005 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sun, 19 Jun 2005 13:47:54 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B58761.7090701@minkirri.apana.org.au> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> <42B58761.7090701@minkirri.apana.org.au> Message-ID: Donovan Baarda wrote: > I don't get what the problem is with mixing statement and expression > semantics... from a practial point of view, statements just offer a > superset of expression functionality. > > If there really is a serious practical reason why they must be limited > to expressions, why not just raise an exception or something if the > "anonymous function" is too complicated... > > I did some fiddling and it seems lambda's can call methods and stuff > that can have side effects, which kinda defeats what I thought was the > point of "statements vs expressions"... I guess I just don't > understand... maybe I'm just thick :-) The whole point is that you are able to do all the basic control flow operations like IF, FOR and WHILE using simple expressions ( like in Pythons lambda ), the lambda statement itself and recursion. Therefore lambda expressions constitute a Turing complete language and they also do so in Python. Different to many FP languages lambda plays no central role in Python because statements won't be reduced to lambda expressions ( or some kind of ). They are merely an add-on. Reduction provides often the advantage to make expressions/statements scriptable what they are not in Python. Python is strong in scripting classes/objects ( a big plus of the language ) but you can't simply use the language to prove that lambda x,y: x+y*y lambda x,y: y**2+x are essentialy the same functions with different implementations [1]. I think this is a severe lack of expressibility and has nothing to do with the silly objection that one has to write one more line for a simple callback - o.k. I admit that I'm lazy too ;) Regards, Kay [1] Not without hacking the parse tree. Doing so one might finally end up accessing the expression in a simple modifieable manner: >>> (lambda x,y: x+y*y).expr ('+',(x,'*',(y,y))) >>> (lambda x,y: y**2+x).expr ('+',(('**',(y,2)),x)) From reinhold-birkenfeld-nospam at wolke7.net Sun Jun 19 14:44:32 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Sun, 19 Jun 2005 14:44:32 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> <42B58761.7090701@minkirri.apana.org.au> Message-ID: Kay Schluehr wrote: > Reduction provides often the advantage to make expressions/statements > scriptable what they are not in Python. Python is strong in scripting > classes/objects ( a big plus of the language ) but you can't simply use > the language to prove that > > lambda x,y: x+y*y > lambda x,y: y**2+x > > are essentialy the same functions with different implementations [1]. Except that they are not. Think of __pow__, think of __add__ and __radd__. Reinhold -- Mail address is perfectly valid! From skip at pobox.com Sun Jun 19 15:39:48 2005 From: skip at pobox.com (Skip Montanaro) Date: Sun, 19 Jun 2005 08:39:48 -0500 Subject: [Python-Dev] PEP for RFE 46738 (first draft) In-Reply-To: <4e4a11f805061821378cd84b6@mail.gmail.com> References: <4e4a11f805061723192888915e@mail.gmail.com> <7168d65a0506180644367753b9@mail.gmail.com> <4e4a11f805061807127f39700b@mail.gmail.com> <17076.21429.501499.914085@montanaro.dyndns.org> <4e4a11f805061817462238af59@mail.gmail.com> <17076.61341.558478.28695@montanaro.dyndns.org> <4e4a11f805061821378cd84b6@mail.gmail.com> Message-ID: <17077.30116.594374.702068@montanaro.dyndns.org> Simon> I hacked things a bit, and instead of sending XML, sent pickles Simon> inside the XML response. I've done the same thing (I think I may have used marshal). It works fine as long as you know both ends are Python. Skip From skip at pobox.com Sun Jun 19 15:40:46 2005 From: skip at pobox.com (Skip Montanaro) Date: Sun, 19 Jun 2005 08:40:46 -0500 Subject: [Python-Dev] Propose to reject PEP 294 -- Type Names in the types Module In-Reply-To: <000001c57491$d1dc8fe0$25fccc97@oemcomputer> References: <000001c57491$d1dc8fe0$25fccc97@oemcomputer> Message-ID: <17077.30174.836873.22778@montanaro.dyndns.org> Raymond> Suggest rejecting this PEP and making a note for Py3.0 to Raymond> either sync-up the type names or abandon the types module Raymond> entirely. I thought the types module was already deprecated, at least verbally if not officially. Skip From skip at pobox.com Sun Jun 19 15:45:24 2005 From: skip at pobox.com (Skip Montanaro) Date: Sun, 19 Jun 2005 08:45:24 -0500 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> Message-ID: <17077.30452.331966.648854@montanaro.dyndns.org> >> As I see it, a lambda is an anonymous function. An anonymous function >> is a function without a name. We already have a syntax for a >> function... why not use it. ie: >> >> f = filter(def (a): return a > 1, [1,2,3]) Kay> You mix expressions with statements. You could remove the "return" and restrict the body of the def to an expression: f = filter(def (a): a > 1, [1,2,3]) That looks almost exactly like a lambda, but uses "def" and parenthesizes the argument list. It seems to me that would remind people "this is a function". Skip From kay.schluehr at gmx.net Sun Jun 19 16:45:07 2005 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sun, 19 Jun 2005 16:45:07 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> <42B58761.7090701@minkirri.apana.org.au> Message-ID: Reinhold Birkenfeld wrote: >> >> lambda x,y: x+y*y >> lambda x,y: y**2+x >> >> are essentialy the same functions with different implementations [1]. > > > Except that they are not. Think of __pow__, think of __add__ and __radd__. You know the difference between the concept of a function and it's infinitely many representations? That's why formal definitions exist. > > Reinhold > > Just for refresh: "Formally, a function f from a set X of input values to a set Y of possible output values (written as f : X -> Y) is a relation between X and Y which satisfies: 1. f is total, or entire: for all x in X, there exists a y in Y such that x f y (x is f-related to y), i.e. for each input value, there is at least one output value in Y. 2. f is many-to-one, or functional: if x f y and x f z, then y = z. i.e., many input values can be related to one output value, but one input value cannot be related to many output values. A more concise expression of the above definition is the following: a function from X to Y is a subset f of the cartesian product X ? Y, such that for each x in X, there is a unique y in Y such that the ordered pair (x, y) is in f." http://en.wikipedia.org/wiki/Function_%28mathematics%29 Kay From kay.schluehr at gmx.net Sun Jun 19 17:22:47 2005 From: kay.schluehr at gmx.net (Kay Schluehr) Date: Sun, 19 Jun 2005 17:22:47 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <17077.30452.331966.648854@montanaro.dyndns.org> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <17077.30452.331966.648854@montanaro.dyndns.org> Message-ID: Skip Montanaro wrote: > >> As I see it, a lambda is an anonymous function. An anonymous function > >> is a function without a name. We already have a syntax for a > >> function... why not use it. ie: > >> > >> f = filter(def (a): return a > 1, [1,2,3]) > > Kay> You mix expressions with statements. > > You could remove the "return" and restrict the body of the def to an > expression: > > f = filter(def (a): a > 1, [1,2,3]) > > That looks almost exactly like a lambda, but uses "def" and parenthesizes > the argument list. It seems to me that would remind people "this is a > function". Yes, but skipping the name of a function ( anonymizing it ) is not a strong reason to disallow statements in the anonymus function body. The crucial issue is the notation of callable expressions that are not statements but can be defined inside of other expressions ( e.g. inside a filter() call as in the example ). That's why I prefer notations that emphasize the expression character. Using the arrow notation ( (args) -> expr ) would be fine for me but I would not deselect Python in favor for Java if ( expr from (args) ) is used instead. To me it's a "judean popular front" vs "popular front of judea" kind of thing. Kay From rrr at ronadam.com Sun Jun 19 18:21:40 2005 From: rrr at ronadam.com (Ron Adam) Date: Sun, 19 Jun 2005 12:21:40 -0400 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B4C134.8010308@gmail.com> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> Message-ID: <42B59B94.5070906@ronadam.com> Nick Coghlan wrote: > Donovan Baarda wrote: > >>As I see it, a lambda is an anonymous function. An anonymous function is >>a function without a name. > > > And here we see why I'm such a fan of the term 'deferred expression' > instead of 'anonymous function'. > > Python's lambda expressions *are* the former, but they are > emphatically *not* the latter. Lambda's have inputs and outputs like a function, so they can be used in place of functions. A simple deferred expression wouldn't be the same thing since it would use local names directly, but not at the time the expression is created, and couldn't (easily) be used in place of a function call. addxy = defer x+y # do this later for x in range(10): for y in range(10): print addxy # prints sums of x,y This is basically a limited single expression macro. (Which I believe isn't wanted.) A better example might be as used in a dictionary in place of a case statement. ie.. case['add'] = defer x+y, etc... Regards, Ron (Reply to this in python-list if it would be better discussed there.) From jcarlson at uci.edu Sun Jun 19 18:45:00 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 19 Jun 2005 09:45:00 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 --Simple Implicit Lambda In-Reply-To: <42B59D61.1000009@minkirri.apana.org.au> References: <20050619002931.72AB.JCARLSON@uci.edu> <42B59D61.1000009@minkirri.apana.org.au> Message-ID: <20050619082907.72B1.JCARLSON@uci.edu> Donovan Baarda wrote: > Josiah Carlson wrote: > > Donovan Baarda wrote: > > > >>Nick Coghlan wrote: > >> > >>>Donovan Baarda wrote: > [...] > >>But isn't a function just a deferred expression with a name :-) > > > > > > A function in Python is actually a deferred sequence of statements and > > expressions. An anonymous function in Python (a lambda) is a deferred > > expression. > > in the end though, a sequence of statements that completes with a > "return value" is, when treated as a black box, indistinguishable from > an expression. Originally I thought that this also had to be qualified > with "and has no side-effects", but I see now that is not the case. Lambdas/anonymous functions are generally used when they are simple and are either to be applied _now_, or passed as a callback to someone else (sometimes people use the x = lambda..., but that is laziness, def/return is clearer). Are either of the following two examples easy to read? rslt = (def (arg): #body that handles arg ... )(argument_passed_to_fcn) foo((def (arg): #body that handles arg ... ), foo_arg, ...) I personally think they are horrible. Both of the above would be easier to read as... def _throw_away(arg): #body that handles arg ... rslt = _throw_away(argument_passed_to_fcn) foo(_throw_away, ...) > [...] > >>Oh yeah Raymond: on the "def defines some variable name"... are you > >>joking? You forgot the smiley :-) > > > > > > 'def' happens to bind the name that follows the def to the function with > > the arguments and body following the name. > > Yeah, but we don't use "def" to bind arbitary variables, only > functions/procedures. So in python, they are intimately identified with > functions and procedures. Being that this entire thread is about functions, perhaps there was an implied "function" in there? Also, with the right decorator, it becomes an arbitrary assignment method in Python 2.4 ... >>> def assign(val): ... return lambda a:val ... >>> @assign(5) ... def foo(arg): pass ... >>> foo 5 >>> > >>If there really is a serious practical reason why they must be limited > >>to expressions, why not just raise an exception or something if the > >>"anonymous function" is too complicated... > > > > > > Define "too complicated"? > > I was thinking that this is up to the interpreter... depending on what > the practical limitations are that cause the limitation in the first > place. For example... if it can't be reduced to an "expression" through > simple transforms. Care to write the transformation library for all Python statements? Care to see that multi-line anonymous function I described above? I certainly hope not. - Josiah From reinhold-birkenfeld-nospam at wolke7.net Sun Jun 19 19:33:11 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Sun, 19 Jun 2005 19:33:11 +0200 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> <42B58761.7090701@minkirri.apana.org.au> Message-ID: Kay Schluehr wrote: > Reinhold Birkenfeld wrote: > >>> >>> lambda x,y: x+y*y >>> lambda x,y: y**2+x >>> >>> are essentialy the same functions with different implementations [1]. >> >> >> Except that they are not. Think of __pow__, think of __add__ and __radd__. > > You know the difference between the concept of a function and it's > infinitely many representations? That's why formal definitions exist. I must say that I don't understand what you're after. Reinhold -- Mail address is perfectly valid! From facundobatista at gmail.com Sun Jun 19 20:31:54 2005 From: facundobatista at gmail.com (Facundo Batista) Date: Sun, 19 Jun 2005 15:31:54 -0300 Subject: [Python-Dev] Propose to reject PEP 276 -- Simple iterator for ints In-Reply-To: <000201c572fc$30d94e80$bb07a044@oemcomputer> References: <000201c572fc$30d94e80$bb07a044@oemcomputer> Message-ID: On 6/17/05, Raymond Hettinger wrote: > The principal use case was largely met by enumerate(). From PEP 276's +1 for reject it. . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From pje at telecommunity.com Sun Jun 19 22:16:30 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 19 Jun 2005 16:16:30 -0400 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects In-Reply-To: <5.1.1.6.0.20050618212627.01e22658@mail.telecommunity.com> References: <20050619005024.GA14709@mems-exchange.org> <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com> <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050618222502.01e444e8@mail.telecommunity.com> At 10:15 PM 6/18/2005 -0400, Phillip J. Eby wrote: >Okay, I think I see why you can't do it. You could guarantee that all >relevant __del__ methods get called, but it's bloody difficult to end up >with only unreachable items in gc.garbage afterwards. I think gc would >have to keep a new list for items reachable from finalizers, that don't >themselves have finalizers. Then, before creating gc.garbage, you walk the >finalizers and call their finalization (__del__) methods. Then, you put >any remaining items that are in either the finalizer list or the >reachable-from-finalizers list into gc.garbage. > >This approach might need a new type slot, but it seems like it would let us >guarantee that finalizers get called, even if the object ends up in garbage >as a result. In the case of generators, however, close() guarantees that >the generator releases all its references, and so can no longer be part of >a cycle. Thus, it would guarantee eventual cleanup of all >generators. And, it would lift the general limitation on __del__ methods. > >Hm. Sounds too good to be true. Surely if this were possible, Uncle Timmy >would've thought of it already, no? Guess we'll have to wait and see what >he thinks. Or maybe not. After sleeping on it, I realized that the problems are all in when and how often __del__ is called. The idea I had above would end up calling __del__ twice on non-generator objects. For generators it's not a problem because the first call ends up ensuring that the second call is a no-op. However, the *order* of __del__ calls makes a difference, even for generators. What good is a finally: clause if all the objects reachable from it have been finalized already, anyway? Ultimately, I'm thinking that maybe we were right not to allow try-finally to cross yield boundaries in generators. It doesn't seem like you can guarantee anything about the behavior in the presence of cycles, so what's the point? For a while I played around with the idea that maybe we could still support 'with:' in generators, though, because to implement that we could make frames call __exit__ on any pending 'with' blocks as part of their tp_clear operation. This would only work, however, if the objects with __exit__ methods don't have any references back to the frame. In essence, you'd need a way to put the __exit__ objects on a GC-managed list that wouldn't run until after all the tp_clear calls had finished. But even that is tough to make guarantees about. For example, can you guarantee in that case that a generator's 'with:' blocks are __exit__-ed in the proper order? Really, if we do allow 'with' and 'try-finally' to surround yield, I think we're going to have to tell people that it only works if you use a with or try-finally in some non-generator code to ensure that the generator.close() gets called, and that if you end up creating a garbage cycle, we either have to let it end up in gc.garbage, or just not execute its finally clause or __exit__ methods. Of course, this sort of happens right now for other things with __del__; if it's part of a cycle the __del__ method never gets called. The only difference is that it hangs around in gc.garbage, doing nothing useful. If it's garbage, it's not reachable from anywhere else, so it does nobody any good to have it around. So, maybe we should just say, "sucks to be you" and tp_clear anything that we'd otherwise have put in gc.garbage. :) In other words, since we're not going to call those __del__ methods anyway, maybe it just needs to be part of the language semantics that __del__ isn't guaranteed to be called, and a garbage collector that can't find a safe way to call it, doesn't have to. tp_dealloc for classic classes and heap types could then just skip calling __del__ if they've already been cleared... oh wait, how do you know you've been cleared? Argh. Another nice idea runs up on the rocks of reality. On the other hand, if you go ahead and run __del__ after tp_clear, the __del__ method will quickly run afoul of an AttributeError and die with only a minor spew to sys.stderr, thus encouraging people to get rid of their silly useless __del__ methods on objects that normally end up in cycles. :) From pje at telecommunity.com Sun Jun 19 23:04:48 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 19 Jun 2005 17:04:48 -0400 Subject: [Python-Dev] gcmodule issue w/adding __del__ to generator objects In-Reply-To: <5.1.1.6.0.20050618222502.01e444e8@mail.telecommunity.com> References: <5.1.1.6.0.20050618212627.01e22658@mail.telecommunity.com> <20050619005024.GA14709@mems-exchange.org> <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> <5.1.1.6.0.20050618175100.01e42c68@mail.telecommunity.com> <5.1.1.6.0.20050618153727.01e2f008@mail.telecommunity.com> <5.1.1.6.0.20050618181647.01e2f008@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050619163017.01e189c8@mail.telecommunity.com> Sigh. Looks like Guido already used the time machine to bring up these ideas five years ago: http://mail.python.org/pipermail/python-dev/2000-March/002514.html And apparently you went back with him: http://mail.python.org/pipermail/python-dev/2000-March/002478.html So I give up, 'cause there's no way I can compete with you time travellers. :) Although I do wonder -- why was __cleanup__ never implemented? The only clue seems to be Guido's comment that he "[finds] having a separate __cleanup__ protocol cumbersome." It certainly seems to me that having a __cleanup__ that allows an object to handle itself being garbage would be handy, although it's only meaningful to have a __cleanup__ if you also have a __del__; otherwise, there would never be a reason to call it. Maybe that's the reason it was considered cumbersome. At 04:16 PM 6/19/2005 -0400, Phillip J. Eby wrote: >At 10:15 PM 6/18/2005 -0400, Phillip J. Eby wrote: > >Okay, I think I see why you can't do it. You could guarantee that all > >relevant __del__ methods get called, but it's bloody difficult to end up > >with only unreachable items in gc.garbage afterwards. I think gc would > >have to keep a new list for items reachable from finalizers, that don't > >themselves have finalizers. Then, before creating gc.garbage, you walk the > >finalizers and call their finalization (__del__) methods. Then, you put > >any remaining items that are in either the finalizer list or the > >reachable-from-finalizers list into gc.garbage. > > > >This approach might need a new type slot, but it seems like it would let us > >guarantee that finalizers get called, even if the object ends up in garbage > >as a result. In the case of generators, however, close() guarantees that > >the generator releases all its references, and so can no longer be part of > >a cycle. Thus, it would guarantee eventual cleanup of all > >generators. And, it would lift the general limitation on __del__ methods. > > > >Hm. Sounds too good to be true. Surely if this were possible, Uncle Timmy > >would've thought of it already, no? Guess we'll have to wait and see what > >he thinks. > >Or maybe not. After sleeping on it, I realized that the problems are all >in when and how often __del__ is called. The idea I had above would end up >calling __del__ twice on non-generator objects. For generators it's not a >problem because the first call ends up ensuring that the second call is a >no-op. > >However, the *order* of __del__ calls makes a difference, even for >generators. What good is a finally: clause if all the objects reachable >from it have been finalized already, anyway? > >Ultimately, I'm thinking that maybe we were right not to allow try-finally >to cross yield boundaries in generators. It doesn't seem like you can >guarantee anything about the behavior in the presence of cycles, so what's >the point? > >For a while I played around with the idea that maybe we could still support >'with:' in generators, though, because to implement that we could make >frames call __exit__ on any pending 'with' blocks as part of their tp_clear >operation. This would only work, however, if the objects with __exit__ >methods don't have any references back to the frame. In essence, you'd >need a way to put the __exit__ objects on a GC-managed list that wouldn't >run until after all the tp_clear calls had finished. > >But even that is tough to make guarantees about. For example, can you >guarantee in that case that a generator's 'with:' blocks are __exit__-ed in >the proper order? > >Really, if we do allow 'with' and 'try-finally' to surround yield, I think >we're going to have to tell people that it only works if you use a with or >try-finally in some non-generator code to ensure that the generator.close() >gets called, and that if you end up creating a garbage cycle, we either >have to let it end up in gc.garbage, or just not execute its finally clause >or __exit__ methods. > >Of course, this sort of happens right now for other things with __del__; if >it's part of a cycle the __del__ method never gets called. The only >difference is that it hangs around in gc.garbage, doing nothing useful. If >it's garbage, it's not reachable from anywhere else, so it does nobody any >good to have it around. So, maybe we should just say, "sucks to be you" >and tp_clear anything that we'd otherwise have put in gc.garbage. :) > >In other words, since we're not going to call those __del__ methods anyway, >maybe it just needs to be part of the language semantics that __del__ isn't >guaranteed to be called, and a garbage collector that can't find a safe way >to call it, doesn't have to. tp_dealloc for classic classes and heap types >could then just skip calling __del__ if they've already been cleared... oh >wait, how do you know you've been cleared? Argh. Another nice idea runs >up on the rocks of reality. > >On the other hand, if you go ahead and run __del__ after tp_clear, the >__del__ method will quickly run afoul of an AttributeError and die with >only a minor spew to sys.stderr, thus encouraging people to get rid of >their silly useless __del__ methods on objects that normally end up in >cycles. :) > >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: >http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com From nicksjacobson at yahoo.com Mon Jun 20 01:22:04 2005 From: nicksjacobson at yahoo.com (Nick Jacobson) Date: Sun, 19 Jun 2005 16:22:04 -0700 (PDT) Subject: [Python-Dev] misplaced PEP Message-ID: <20050619232204.21175.qmail@web53906.mail.yahoo.com> At the www.python.org/peps page, PEP 281 is erroneously listed in the "Finished PEPs (done, implemented in CVS)" section. ____________________________________________________ Yahoo! Sports Rekindle the Rivalries. Sign up for Fantasy Football http://football.fantasysports.yahoo.com From luisa.valbusa at tiscali.it Sun Jun 19 12:46:00 2005 From: luisa.valbusa at tiscali.it (Luisa) Date: Sun, 19 Jun 2005 12:46:00 +0200 Subject: [Python-Dev] Problem with embedded python Message-ID: <002101c574bc$15c6f040$0100a8c0@LUISA> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050619/1f1fdaae/attachment.htm From nicksjacobson at yahoo.com Mon Jun 20 06:36:45 2005 From: nicksjacobson at yahoo.com (Nick Jacobson) Date: Sun, 19 Jun 2005 21:36:45 -0700 (PDT) Subject: [Python-Dev] misplaced PEP Message-ID: <20050620043645.56407.qmail@web53901.mail.yahoo.com> Well, it's fixed now. Thanks to whomever took care of it. --- Nick Jacobson wrote: > At the www.python.org/peps page, PEP 281 is > erroneously listed in the "Finished PEPs (done, > implemented in CVS)" section. > > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From gvanrossum at gmail.com Mon Jun 20 12:10:26 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon, 20 Jun 2005 03:10:26 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B4C134.8010308@gmail.com> References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> Message-ID: [Nick Coghlan] > And here we see why I'm such a fan of the term 'deferred expression' > instead of 'anonymous function'. > > Python's lambda expressions *are* the former, but they are > emphatically *not* the latter. Let me emphatically disagree. Your POV is entirely syntactical, which IMO is a shallow perspective. Semantically, lambdas ARE anonymous functions, and that's what counts. Since "deferred expression" is not defined in Python, you can't reasonably argue about whether that's what lambdas are, but intuitively for me the term refers to something more lightweight than lambdas. Now, whether the use cases for lambdas are better served by anonymous functions or by something else that might be called deferred expression, I don't know yet. But as long as we are describing the present state we should call a spade a spade, etc. (Alas, no time to follow the rest of this thread -- vacationing with little connectivity until EuroPython starts next week.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Mon Jun 20 16:06:22 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon, 20 Jun 2005 07:06:22 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: [Keith Dart] > In SNMP, for example, a Counter32 is basically an unsigned int, defined > as "IMPLICIT INTEGER (0..4294967295)". One cannot efficiently translate > and use that type in native Python. Currently, I have defined an > "unsigned" type as a subclass of long, but I don't think that would be > speed or storage efficient. In my experience you can just use Python longs whenever a C API needs an "unsigned" long. There's no need to subtype, and your assumption that it would not be efficient enough is mistaken (unless you are manipulating arrays with millions of them, in which case you should be using Numeric, which has its own types for this purpose). (Want to argue about the efficiency? Write a typical use case and time it.) By far the easiest way to do arithmetic mod 2**32 is to just add "& 0xFFFFFFFF" to the end of your expression. For example, simulating the effect of multiplying an unsigned long by 3 would be x = (x * 3) & 0xFFFFFFFF. If there is a problem with ioctl() not taking long ints, that would be a bug in ioctl, not a lacking data type or a problem with long ints. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mcherm at mcherm.com Mon Jun 20 17:31:58 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Mon, 20 Jun 2005 08:31:58 -0700 Subject: [Python-Dev] Multiple expression eval in compound if statement? Message-ID: <20050620083158.kqfxwkg63qhccc80@login.werra.lunarpages.com> Gerrit Holl writes: > What would happen if... Raymond replies: > Every molecule in your body would simultaneously implode at the speed of > light. So you're saying it triggers C-language "undefined behavior"? -- Michael Chermside From mwh at python.net Mon Jun 20 18:59:19 2005 From: mwh at python.net (Michael Hudson) Date: Mon, 20 Jun 2005 17:59:19 +0100 Subject: [Python-Dev] Multiple interpreters not compatible with current thread module In-Reply-To: <2m4qby9qzl.fsf@starship.python.net> (Michael Hudson's message of "Thu, 16 Jun 2005 18:00:14 +0100") References: <93dc9c320505170425392e3d80@mail.gmail.com> <2m4qby9qzl.fsf@starship.python.net> Message-ID: <2mzmtl6k2g.fsf@starship.python.net> Michael Hudson writes: > I'm not expecting anyone else to think hard about this on recent form, > so I'll think about it for a bit and then fix it in the way that seems > best after that. Feel free to surprise me. And so that's what I did. I think I got this right, but threads are tricky buggers so x-platform testing of CVS HEAD would be appreciated (heck, that would be appreciated anyway, I don't get the impression much of it is happening at the moment...). Cheers, mwh -- glyph: I don't know anything about reality. -- from Twisted.Quotes From jimjjewett at gmail.com Mon Jun 20 19:42:09 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 20 Jun 2005 13:42:09 -0400 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda Message-ID: >>> lambda x,y: x+y*y >>> lambda x,y: y**2+x >>> are essentialy the same functions with different implementations [1]. >> Except that they are not. Think of __pow__, think of __add__ and __radd__. > You know the difference between the concept of a function and it's > infinitely many representations? That's why formal definitions exist. To clear up the archives, the problem is that Kay and Reinhold were talking about different things when they said "function". In arithmetic (and most sane extensions), those two lines are essentially the same function written different ways. In python those two lines represent two different functions, because x and y might not be (bound to) sane numbers. The variables could be bound to objects that redefine the addition, multiplication, and exponentiation operators. So y*y might not be the same as y**2, and x+y might not be the same as y+x. -jJ From kdart at kdart.com Tue Jun 21 05:49:24 2005 From: kdart at kdart.com (Keith Dart) Date: Mon, 20 Jun 2005 20:49:24 -0700 (PDT) Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: On Mon, 20 Jun 2005, Guido van Rossum wrote: > [Keith Dart] >> In SNMP, for example, a Counter32 is basically an unsigned int, defined >> as "IMPLICIT INTEGER (0..4294967295)". One cannot efficiently translate >> and use that type in native Python. Currently, I have defined an >> "unsigned" type as a subclass of long, but I don't think that would be >> speed or storage efficient. > > In my experience you can just use Python longs whenever a C API needs > an "unsigned" long. There's no need to subtype, and your assumption > that it would not be efficient enough is mistaken (unless you are > manipulating arrays with millions of them, in which case you should be > using Numeric, which has its own types for this purpose). (Want to > argue about the efficiency? Write a typical use case and time it.) Ok, I'll take your word for it. I don't have any performance problems now, in my usage, but I wanted to make sure that Python "shows well" in certain "bake offs" ;-) > By far the easiest way to do arithmetic mod 2**32 is to just add "& > 0xFFFFFFFF" to the end of your expression. For example, simulating the > effect of multiplying an unsigned long by 3 would be x = (x * 3) & > 0xFFFFFFFF. But then I wouldn't know if it overflowed 32 bits. In my usage, the integer will be translated to an unsigned (32 bit) integer in another system (SNMP). I want to know if it will fit, and I want to know early if there will be a problem, rather than later (at conversion time). One of the "selling points" of Python in previous versions was that you would get an OverFlowError on overflow, where other languages did not (they overflowed silently). So I subclassed long in 2.3, to get the same overflow exception: class unsigned(long): floor = 0L ceiling = 4294967295L bits = 32 _mask = 0xFFFFFFFFL def __new__(cls, val): return long.__new__(cls, val) def __init__(self, val): if val < self.floor or val > self.ceiling: raise OverflowError, "value %s out of range for type %s" % (val, self.__class__.__name__) def __repr__(self): return "%s(%sL)" % (self.__class__.__name__, self) def __add__(self, other): return self.__class__(long.__add__(self, other)) ... Again, because I want to catch the error early, before conversion to the external type. BTW, the conversion is done in pure Python (to a BER representation), so using a C type (via ctypes, or pyrex, or whatever) is not possible. > If there is a problem with ioctl() not taking long ints, that would be > a bug in ioctl, not a lacking data type or a problem with long ints. That must be it, then. Shall I file a bug somewhere? -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: F3D288E4 ===================================================================== From kdart at kdart.com Tue Jun 21 06:58:24 2005 From: kdart at kdart.com (Keith Dart) Date: Mon, 20 Jun 2005 21:58:24 -0700 (PDT) Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: On Mon, 20 Jun 2005, Keith Dart wrote: > But then I wouldn't know if it overflowed 32 bits. In my usage, the > integer will be translated to an unsigned (32 bit) integer in another > system (SNMP). I want to know if it will fit, and I want to know early if > there will be a problem, rather than later (at conversion time). > > class unsigned(long): I guess I just clarify this more. My "unsigned" type really is an object that represents a type of number from the external system. Previously, there was a nice, clean mapping between external types and Python types. Now there is not so clean a mapping. Not that that makes it a problem with Python itself. However, since it is sometimes necessary to interface to other systems with Python, I see no reason why Python should not have a full set of built in numeric types corresponding to the machine types and, in turn, other system types. Then it would be easier (and probaby a bit faster) to interface to them. Perhaps Python could have an "integer" type for long/int unified types, and just "int" type as "normal" integers? -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: F3D288E4 ===================================================================== From dmitry at targeted.org Tue Jun 21 07:25:18 2005 From: dmitry at targeted.org (Dmitry Dvoinikov) Date: Tue, 21 Jun 2005 11:25:18 +0600 Subject: [Python-Dev] Explicitly declaring expected exceptions for a block Message-ID: <1355231971.20050621112518@targeted.org> Excuse me if I couldn't find that in the existing PEPs, but wouldn't that be useful to have a construct that explicitly tells that we know an exception of specific type could happen within a block, like: ---------------------------------- ignore TypeError: do stuff [else: do other stuff] being essintially identical to try: do stuff except TypeError: pass [else: do other stuff] ---------------------------------- The reason for that being self-tests with lots and lots of little code snippets like this: try: c().foo() except TypeError: pass which could be written as ignore TypeError: c().foo() Sincerely, Dmitry Dvoinikov http://www.targeted.org/ From paul.dubois at gmail.com Tue Jun 21 07:56:10 2005 From: paul.dubois at gmail.com (Paul Du Bois) Date: Mon, 20 Jun 2005 22:56:10 -0700 Subject: [Python-Dev] Explicitly declaring expected exceptions for a block In-Reply-To: <1355231971.20050621112518@targeted.org> References: <1355231971.20050621112518@targeted.org> Message-ID: <85f6a31f05062022564619f4f6@mail.gmail.com> On 6/20/05, Dmitry Dvoinikov wrote: > Excuse me if I couldn't find that in the existing PEPs, but > wouldn't that be useful to have a construct that explicitly > tells that we know an exception of specific type could happen > within a block, like: > ignore TypeError: > do stuff > [else: > do other stuff] If I understand PEP 343 correctly, it allows for easy implementation of part of your request. It doesn't implement the else: clause, but you don't give a use case for it either. class ignored_exceptions(object): def __init__(self, *exceptions): self.exceptions = exceptions def __enter__(self): return None def __exit__(self, type, value, traceback): try: raise type, value, traceback except self.exceptions: pass with ignored_exceptions(SomeError): do stuff I don't see the use, but it's possible. > The reason for that being self-tests with lots and lots of > little code snippets like this: If you're trying to write tests, perhaps a better use-case would be something like: with required_exception(SomeError): do something that should cause SomeError paul From dmitry at targeted.org Tue Jun 21 08:22:54 2005 From: dmitry at targeted.org (Dmitry Dvoinikov) Date: Tue, 21 Jun 2005 12:22:54 +0600 Subject: [Python-Dev] Explicitly declaring expected exceptions for a block In-Reply-To: <85f6a31f0506202255128b39e2@mail.gmail.com> References: <1355231971.20050621112518@targeted.org> <85f6a31f0506202255128b39e2@mail.gmail.com> Message-ID: <1341515918.20050621122254@targeted.org> > If you're trying to write tests, perhaps a better use-case would be > something like: > with required_exception(SomeError): > do something that should cause SomeError Yes, you are right, that'd be a better and more flexible way, thank you. Sincerely, Dmitry Dvoinikov http://www.targeted.org/ --- Original message follows --- > On 6/20/05, Dmitry Dvoinikov wrote: >> Excuse me if I couldn't find that in the existing PEPs, but >> wouldn't that be useful to have a construct that explicitly >> tells that we know an exception of specific type could happen >> within a block, like: >> ignore TypeError: >> do stuff >> [else: >> do other stuff] > If I understand PEP 343 correctly, it allows for easy implementation > of part of your request. It doesn't implement the else: clause, but > you don't give a use case for it either. > class ignored_exceptions(object): > def __init__(self, *exceptions): > self.exceptions = exceptions > def __enter__(self): > return None > def __exit__(self, type, value, traceback): > try: > raise type, value, traceback > except self.exceptions: > pass > with ignored_exceptions(SomeError): > do stuff > I don't see the use, but it's possible. >> The reason for that being self-tests with lots and lots of >> little code snippets like this: > If you're trying to write tests, perhaps a better use-case would be > something like: > with required_exception(SomeError): > do something that should cause SomeError > paul From shane at hathawaymix.org Tue Jun 21 09:49:19 2005 From: shane at hathawaymix.org (Shane Hathaway) Date: Tue, 21 Jun 2005 01:49:19 -0600 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: <42B7C67F.1060304@hathawaymix.org> Keith Dart wrote: > I guess I just clarify this more. My "unsigned" type really is an object > that represents a type of number from the external system. Previously, > there was a nice, clean mapping between external types and Python types. > Now there is not so clean a mapping. Not that that makes it a problem > with Python itself. > > However, since it is sometimes necessary to interface to other systems > with Python, I see no reason why Python should not have a full set of > built in numeric types corresponding to the machine types and, in turn, > other system types. Then it would be easier (and probaby a bit faster) > to interface to them. Perhaps Python could have an "integer" type for > long/int unified types, and just "int" type as "normal" integers? For your purposes, would it work to use the struct module to detect overflows early? >>> import struct >>> struct.pack('i', 2 ** 33) Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int Another possibility would be to add to the "struct" module a full set of integer types with a fixed width: int8, uint8, int16, uint16, int32, uint32, int64, and uint64. Code that's focused on integration with other languages might benefit. Shane From gvanrossum at gmail.com Tue Jun 21 12:16:17 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue, 21 Jun 2005 03:16:17 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: On 6/20/05, Keith Dart wrote: > On Mon, 20 Jun 2005, Guido van Rossum wrote: [...] > > By far the easiest way to do arithmetic mod 2**32 is to just add "& > > 0xFFFFFFFF" to the end of your expression. For example, simulating the > > effect of multiplying an unsigned long by 3 would be x = (x * 3) & > > 0xFFFFFFFF. > > But then I wouldn't know if it overflowed 32 bits. Huh? C unsigned ints don't flag overflow either -- they perform perfect arithmetic mod 2**32. > In my usage, the > integer will be translated to an unsigned (32 bit) integer in another > system (SNMP). I want to know if it will fit, and I want to know early if > there will be a problem, rather than later (at conversion time). So check if it is >= 2**32 (or < 0, of course). > One of the "selling points" of Python in previous versions was that you > would get an OverFlowError on overflow, where other languages did not > (they overflowed silently). So I subclassed long in 2.3, to get the same > overflow exception: > ... > Again, because I want to catch the error early, before conversion to the > external type. This is a very specialized application. Your best approach is to check for overflow before passing into the external API -- ideally the wrappers for that API should do so. > > If there is a problem with ioctl() not taking long ints, that would be > > a bug in ioctl, not a lacking data type or a problem with long ints. > > That must be it, then. Shall I file a bug somewhere? SourceForge. (python.org/dev for more info) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Tue Jun 21 12:19:04 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue, 21 Jun 2005 03:19:04 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: On 6/20/05, Keith Dart wrote: > However, since it is sometimes necessary to interface to other systems > with Python, I see no reason why Python should not have a full set of > built in numeric types corresponding to the machine types and, in turn, > other system types. Then it would be easier (and probaby a bit faster) > to interface to them. Perhaps Python could have an "integer" type for > long/int unified types, and just "int" type as "normal" integers? Strongly disagree. (a) Stop worrying about speed. The overhead of a single Python bytecode execution is probably more than the cost of an integer operation in most cases. (b) I don't know what you call a "normal" integer any more; to me, unified long/int is as normal as they come. Trust me, that's the case for most users. Worrying about 32 bits becomes less and less normal. (c) The right place to do the overflow checks is in the API wrappers, not in the integer types. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mcherm at mcherm.com Tue Jun 21 13:12:23 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Tue, 21 Jun 2005 04:12:23 -0700 Subject: [Python-Dev] Explicitly declaring expected exceptions for ablock Message-ID: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com> Dmitry Dvoinikov writes: > The reason for that being self-tests with lots and lots of > little code snippets like this: > > try: > c().foo() > except TypeError: > pass Paul Du Boise already responded explaining that PEP 343 probably handles the task you want. I just wanted to mention that you may need to reconsider the task. The above snippet is almost certainly incorrect. I suspect that you wanted either: try: c().foo() fail('Should have raised TypeError') except TypeError: pass # expected or perhaps this: try: c().foo() except TypeError: fail('Should not have raised TypeError') There ARE situations when you want to allow an exception (but not necessarily expect it) and do nothing when it occurs, but I don't find them all that common, and I certainly don't find them arising in unit tests. -- Michael Chermside From dmitry at targeted.org Tue Jun 21 13:30:00 2005 From: dmitry at targeted.org (Dmitry Dvoinikov) Date: Tue, 21 Jun 2005 17:30:00 +0600 Subject: [Python-Dev] Explicitly declaring expected exceptions for ablock In-Reply-To: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com> References: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com> Message-ID: <315080598.20050621173000@targeted.org> > I suspect that you wanted either: > try: > c().foo() > fail('Should have raised TypeError') > except TypeError: > pass # expected Right, of course I use something along these lines: try: c().foo() except TypeError: pass else: assert False, "c.foo() should have thrown TypeError" and so if foo throws anything but TypeError I get traceback, otherwise it's an assertion error and is reported as such. This is probably just one of the many possible wording of the same thing though. Sincerely, Dmitry Dvoinikov http://www.targeted.org/ --- Original message follows --- > Dmitry Dvoinikov writes: >> The reason for that being self-tests with lots and lots of >> little code snippets like this: >> >> try: >> c().foo() >> except TypeError: >> pass > Paul Du Boise already responded explaining that PEP 343 probably handles > the task you want. I just wanted to mention that you may need to > reconsider the task. The above snippet is almost certainly incorrect. > I suspect that you wanted either: > try: > c().foo() > fail('Should have raised TypeError') > except TypeError: > pass # expected > or perhaps this: > try: > c().foo() > except TypeError: > fail('Should not have raised TypeError') > There ARE situations when you want to allow an exception (but not > necessarily expect it) and do nothing when it occurs, but I don't > find them all that common, and I certainly don't find them arising > in unit tests. > -- Michael Chermside From dmitry at targeted.org Tue Jun 21 13:43:16 2005 From: dmitry at targeted.org (Dmitry Dvoinikov) Date: Tue, 21 Jun 2005 17:43:16 +0600 Subject: [Python-Dev] Explicitly declaring expected exceptions for ablock In-Reply-To: <315080598.20050621173000@targeted.org> References: <20050621041223.zajsnzwn1tcs48c4@login.werra.lunarpages.com> <315080598.20050621173000@targeted.org> Message-ID: <7810152903.20050621174316@targeted.org> > try: > c().foo() > except TypeError: > pass > else: > assert False, "c.foo() should have thrown TypeError" In fact, the above code actually expects foo to throw particular exception, not exactly the same as the original requirement. More of expect TypeError: c().foo() and still can be implemented under PEP 343, that's what I meant under "more flexible". Sincerely, Dmitry Dvoinikov http://www.targeted.org/ --- Original message follows --- >> I suspect that you wanted either: >> try: >> c().foo() >> fail('Should have raised TypeError') >> except TypeError: >> pass # expected > Right, of course I use something along these lines: > try: > c().foo() > except TypeError: > pass > else: > assert False, "c.foo() should have thrown TypeError" > and so if foo throws anything but TypeError I get traceback, > otherwise it's an assertion error and is reported as such. > This is probably just one of the many possible wording of the > same thing though. > Sincerely, > Dmitry Dvoinikov > http://www.targeted.org/ > --- Original message follows --- >> Dmitry Dvoinikov writes: >>> The reason for that being self-tests with lots and lots of >>> little code snippets like this: >>> >>> try: >>> c().foo() >>> except TypeError: >>> pass >> Paul Du Boise already responded explaining that PEP 343 probably handles >> the task you want. I just wanted to mention that you may need to >> reconsider the task. The above snippet is almost certainly incorrect. >> I suspect that you wanted either: >> try: >> c().foo() >> fail('Should have raised TypeError') >> except TypeError: >> pass # expected >> or perhaps this: >> try: >> c().foo() >> except TypeError: >> fail('Should not have raised TypeError') >> There ARE situations when you want to allow an exception (but not >> necessarily expect it) and do nothing when it occurs, but I don't >> find them all that common, and I certainly don't find them arising >> in unit tests. >> -- Michael Chermside > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/dmitry%40targeted.org From ncoghlan at gmail.com Tue Jun 21 16:51:07 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 22 Jun 2005 00:51:07 +1000 Subject: [Python-Dev] Explicitly declaring expected exceptions for a block In-Reply-To: <85f6a31f05062022564619f4f6@mail.gmail.com> References: <1355231971.20050621112518@targeted.org> <85f6a31f05062022564619f4f6@mail.gmail.com> Message-ID: <42B8295B.8030107@gmail.com> Paul Du Bois wrote: > If I understand PEP 343 correctly, it allows for easy implementation > of part of your request. It doesn't implement the else: clause, but > you don't give a use case for it either. > > class ignored_exceptions(object): > def __init__(self, *exceptions): > self.exceptions = exceptions > def __enter__(self): > return None > def __exit__(self, type, value, traceback): > try: > raise type, value, traceback > except self.exceptions: > pass > > with ignored_exceptions(SomeError): > do stuff > > I don't see the use, but it's possible. It was possible in PEP 340 and in early drafts of PEP 346, but it isn't possible in PEP 343. In PEP 343, the statement template *cannot* suppress exceptions - it can react to them, and it can turn them into different exceptions, but that's all. And yes, this is deliberate - the control flow is too murky otherwise: with some_template(): raise TypeError print "Hi!" Does that code print "Hi!" or not? Under PEP 343, you know it doesn't, because the TypeError isn't trapped. If templates could actually suppress exceptions, you'd have no idea what the code does without looking up the definition of 'some_template' - this is a bad thing, which is why PEP 343 defines the semantics the way it does. However, I'm with Michael - every time I've needed something like this, I have had non-trivial code in either the 'except' or the 'else' blocks of the try statement. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Tue Jun 21 17:13:01 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 22 Jun 2005 01:13:01 +1000 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: References: <42B35DC8.6040308@gmail.com> <20050618011502.728D.JCARLSON@uci.edu> <42B52343.3050902@minkirri.apana.org.au> <42B4C134.8010308@gmail.com> Message-ID: <42B82E7D.4070906@gmail.com> Guido van Rossum wrote: > [Nick Coghlan] > >>And here we see why I'm such a fan of the term 'deferred expression' >>instead of 'anonymous function'. >> >>Python's lambda expressions *are* the former, but they are >>emphatically *not* the latter. > > Let me emphatically disagree. Your POV is entirely syntactical, which > IMO is a shallow perspective. Semantically, lambdas ARE anonymous > functions, and that's what counts. Interesting. Knowing this, I think I better understand your desire to get rid of lambda expressions for Py3K. > Since "deferred expression" is not defined in Python, you can't > reasonably argue about whether that's what lambdas are, but > intuitively for me the term refers to something more lightweight than > lambdas. As you say, it is a matter of looking at lambdas based on what the current syntax restricts them to (i.e. single expressions), rather than what the underlying machinery is capable of (i.e. full-fledged functions). > Now, whether the use cases for lambdas are better served by anonymous > functions or by something else that might be called deferred > expression, I don't know yet. Like you (judging by your stated goals for Py3K), I don't have any use cases for full anonymous functions - named functions serve that role quite happily for me. Where I find lambda expressions useful is the role that Python's current syntax restricts them too - functions which consist of a single (usually simple) expression. For those, pulling the expression out and naming it would just end up hiding the meaningful sections of code behind a few pieces of boilerplate > But as long as we are describing the > present state we should call a spade a spade, etc. I guess I take a syntactic view of the status quo, because, while lambdas may be implemented as anonymous functions, the current syntax doesn't let me *write* an arbitrary function as a lambda. Regardless, I believe the balance will eventually tip in some direction - either lambdas disappear entirely, become able support full anonymous functions, or else the idea of a 'deferred expression' becomes a defining characteristic, rather than a syntactic quirk. I figure it's all Py3K type stuff though, without any great urgency associated with it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From rrr at ronadam.com Tue Jun 21 18:14:49 2005 From: rrr at ronadam.com (Ron Adam) Date: Tue, 21 Jun 2005 12:14:49 -0400 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: <42B83CF9.2010908@ronadam.com> Keith Dart wrote: > On Mon, 20 Jun 2005, Keith Dart wrote: > > >>But then I wouldn't know if it overflowed 32 bits. In my usage, the >>integer will be translated to an unsigned (32 bit) integer in another >>system (SNMP). I want to know if it will fit, and I want to know early if >>there will be a problem, rather than later (at conversion time). >> >>class unsigned(long): > > > I guess I just clarify this more. My "unsigned" type really is an object > that represents a type of number from the external system. Previously, > there was a nice, clean mapping between external types and Python types. > Now there is not so clean a mapping. Not that that makes it a problem > with Python itself. > > However, since it is sometimes necessary to interface to other systems > with Python, I see no reason why Python should not have a full set of > built in numeric types corresponding to the machine types and, in turn, > other system types. Then it would be easier (and probaby a bit faster) > to interface to them. Perhaps Python could have an "integer" type for > long/int unified types, and just "int" type as "normal" integers? It seems to me, that maybe a single "byte_buffer" type, that can be defined to the exact needed byte lengths and have possible other characteristics to aid in interfacing to other languages or devices, would be a better choice. Then pythons ints, floats, etc... can uses what ever internal lengths is most efficient for the system it's compiled on and then the final result can be stored in the 'byte_buffer' for interfacing purposes. It would also be a good choice for bit manipulation when someone needs that, instead of trying to do it in an integer. Would something like that fulfill your need? Regards, Ron From python at rcn.com Wed Jun 22 00:08:07 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue, 21 Jun 2005 18:08:07 -0400 Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops In-Reply-To: <7592058.1119361866@dhcp31-171.ics.uci.edu> Message-ID: <007b01c576ad$b46d2a00$d922c797@oemcomputer> [Raymond] > > The above recommendations should get the PEP ready for judgement day. [David Eppstein] > I thought judgement day already happened for this PEP in the "Parade of > PEPs". No? The parade's text said the main gripe was having the index in the middle, rather than right after the keyword. However, even after addressed the gripe, Guido still hated it. So, the PEP is now dead as a doornail. Raymond From kdart at kdart.com Wed Jun 22 00:18:13 2005 From: kdart at kdart.com (Keith Dart) Date: Tue, 21 Jun 2005 15:18:13 -0700 (PDT) Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: On Tue, 21 Jun 2005, Guido van Rossum wrote: [two messages mixed] > Huh? C unsigned ints don't flag overflow either -- they perform > perfect arithmetic mod 2**32. I was talking about signed ints. Sorry about the confusion. Other scripting languages (e.g. perl) do not error on overflow. > >> In my usage, the >> integer will be translated to an unsigned (32 bit) integer in another >> system (SNMP). I want to know if it will fit, and I want to know early if >> there will be a problem, rather than later (at conversion time). > > So check if it is >= 2**32 (or < 0, of course). That's exactly what I do. ;-) The "unsigned" subclass of long is part of the API, and checks the range when it is created (and they get created implicitly when operated on). > (a) Stop worrying about speed. The overhead of a single Python > bytecode execution is probably more than the cost of an integer > operation in most cases. I am not thinking of the integer operation, but the extra Python bytecode necessary to implement the extra checks for overflow. >> Again, because I want to catch the error early, before conversion to the >> external type. > > This is a very specialized application. Your best approach is to check > for overflow before passing into the external API -- ideally the > wrappers for that API should do so. > (c) The right place to do the overflow checks is in the API wrappers, > not in the integer types. That would be the "traditional" method. I was trying to keep it an object-oriented API. What should "know" the overflow condition is the type object itself. It raises OverFlowError any time this occurs, for any operation, implicitly. I prefer to catch errors earlier, rather than later. > (b) I don't know what you call a "normal" integer any more; to me, > unified long/int is as normal as they come. Trust me, that's the case > for most users. Worrying about 32 bits becomes less and less normal. By "normal" integer I mean the mathematical definition. Most Python users don't have to worry about 32 bits now, that is a good thing when you are dealing only with Python. However, if one has to interface to other systems that have definite types with limits, then one must "hack around" this feature. I was just thinking how nice it would be if Python had, in addition to unified ("real", "normal") integers it also had built-in types that could be more easily mapped to external types (the typical set of signed, unsigned, short, long, etc.).Yes, you can check it at conversion time, but that would mean extra Python bytecode. It seems you think this is a special case, but I think Python may be used as a "glue language" fairly often, and some of us would benefit from having those extra types as built-ins. -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: F3D288E4 ===================================================================== From kdart at kdart.com Wed Jun 22 00:23:20 2005 From: kdart at kdart.com (Keith Dart) Date: Tue, 21 Jun 2005 15:23:20 -0700 (PDT) Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: <42B83CF9.2010908@ronadam.com> References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> <42B83CF9.2010908@ronadam.com> Message-ID: On Tue, 21 Jun 2005, Ron Adam wrote: > It seems to me, that maybe a single "byte_buffer" type, that can be > defined to the exact needed byte lengths and have possible other > characteristics to aid in interfacing to other languages or devices, > would be a better choice. > > Then pythons ints, floats, etc... can uses what ever internal lengths is > most efficient for the system it's compiled on and then the final result > can be stored in the 'byte_buffer' for interfacing purposes. > > It would also be a good choice for bit manipulation when someone needs > that, instead of trying to do it in an integer. > > Would something like that fulfill your need? Sounds interresting. Not exactly stright-forward. What i have now is functional, but if speed becomes a problem then this might be useful. -- -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Keith Dart public key: ID: F3D288E4 ===================================================================== From dmitry at targeted.org Wed Jun 22 06:09:49 2005 From: dmitry at targeted.org (Dmitry Dvoinikov) Date: Wed, 22 Jun 2005 10:09:49 +0600 Subject: [Python-Dev] Explicitly declaring expected exceptions for a block In-Reply-To: <42B8295B.8030107@gmail.com> References: <1355231971.20050621112518@targeted.org> <85f6a31f05062022564619f4f6@mail.gmail.com> <42B8295B.8030107@gmail.com> Message-ID: <1259444531.20050622100949@targeted.org> > It was possible in PEP 340 and in early drafts of PEP 346, but it > isn't possible in PEP 343. > In PEP 343, the statement template *cannot* suppress exceptions - it > can react to them, and it can turn them into different exceptions, but > that's all. ...doing homework... The following code, combined from PEP 343 and Mr. Du Bois suggestion: -------------------------------------------------------------- from sys import exc_info class ignored_exceptions(object): def __init__(self, *exceptions): self.exceptions = exceptions def __enter__(self): return None def __exit__(self, type, value, traceback): try: raise type, value, traceback except self.exceptions: pass try: with = ignored_exceptions(TypeError) with.__enter__() try: try: raise TypeError() except: exc = exc_info() raise finally: with.__exit__(*exc) except: print exc_info() -------------------------------------------------------------- still yields exceptions.TypeError. Now, back to original question then, do you think it'd be beneficial to have some sort of exception ignoring or expecting statement ? Sincerely, Dmitry Dvoinikov http://www.targeted.org/ --- Original message follows --- > Paul Du Bois wrote: >> If I understand PEP 343 correctly, it allows for easy implementation >> of part of your request. It doesn't implement the else: clause, but >> you don't give a use case for it either. >> >> class ignored_exceptions(object): >> def __init__(self, *exceptions): >> self.exceptions = exceptions >> def __enter__(self): >> return None >> def __exit__(self, type, value, traceback): >> try: >> raise type, value, traceback >> except self.exceptions: >> pass >> >> with ignored_exceptions(SomeError): >> do stuff >> >> I don't see the use, but it's possible. > It was possible in PEP 340 and in early drafts of PEP 346, but it > isn't possible in PEP 343. > In PEP 343, the statement template *cannot* suppress exceptions - it > can react to them, and it can turn them into different exceptions, but > that's all. > And yes, this is deliberate - the control flow is too murky otherwise: > with some_template(): > raise TypeError > print "Hi!" > Does that code print "Hi!" or not? Under PEP 343, you know it doesn't, > because the TypeError isn't trapped. If templates could actually > suppress exceptions, you'd have no idea what the code does without > looking up the definition of 'some_template' - this is a bad thing, > which is why PEP 343 defines the semantics the way it does. > However, I'm with Michael - every time I've needed something like > this, I have had non-trivial code in either the 'except' or the 'else' > blocks of the try statement. > Regards, > Nick. From kbk at shore.net Wed Jun 22 06:55:15 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed, 22 Jun 2005 00:55:15 -0400 (EDT) Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200506220455.j5M4tFZf007346@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 338 open ( +0) / 2866 closed ( +5) / 3204 total ( +5) Bugs : 914 open ( +5) / 5060 closed (+13) / 5974 total (+18) RFE : 188 open ( +0) / 170 closed ( +0) / 358 total ( +0) New / Reopened Patches ______________________ update the binhex module for Mach-O (2005-06-14) http://python.org/sf/1220874 opened by Bob Ippolito ftplib storbinary callback (2005-06-15) http://python.org/sf/1221598 opened by Phil Schwartz C++ compilation support for distutils (2005-06-17) http://python.org/sf/1222585 opened by Antti Honkela PEP 342/343 Generator enhancements (2005-06-18) http://python.org/sf/1223381 opened by Phillip J. Eby HP-UX ia64 64-bit Python executable (2005-06-21) http://python.org/sf/1225212 opened by Peter Kropf Patches Closed ______________ _csv.c leaks when dialect creation fails (2005-06-14) http://python.org/sf/1220242 closed by montanaro test_curses bugs (2004-08-09) http://python.org/sf/1005892 closed by akuchling no html file for modulefinder (2005-04-10) http://python.org/sf/1180012 closed by birkenfeld Restore GC support to set objects (2005-05-11) http://python.org/sf/1200018 closed by rhettinger change recall in IDLE shell to not overwrite current command (2005-05-06) http://python.org/sf/1196917 closed by kbk New / Reopened Bugs ___________________ buffer() object broken. (2005-06-15) CLOSED http://python.org/sf/1221424 opened by Ray Schumacher Incorrect documentation for InPlace Number operators (2005-06-15) CLOSED http://python.org/sf/1221477 opened by Daniel Stutzbach float() not accurate (2005-06-16) CLOSED http://python.org/sf/1222098 opened by Brian Dols Bad optparse help wrapping with multiple paragraphs (2005-06-16) http://python.org/sf/1222235 opened by Barry A. Warsaw Typo in section 5.3 of tutorial (2005-06-17) CLOSED http://python.org/sf/1222459 opened by uman tk + setlocale problems... (2005-06-17) http://python.org/sf/1222721 opened by Alex A. Naanou SimpleXMLRPCServer does not set FD_CLOEXEC (2005-06-17) http://python.org/sf/1222790 opened by Winfried Harbecke it's -> its (2005-06-17) CLOSED http://python.org/sf/1222928 opened by Ed Swierk 2.4.1 .msi file won't install to different disk (2005-06-17) CLOSED http://python.org/sf/1222978 opened by Ray Trent race in os.makedirs() (2005-06-18) http://python.org/sf/1223238 opened by Mattias Engdeg?rd subprocess.py abuse of errno (2005-06-20) http://python.org/sf/1223937 reopened by astrand subprocess.py abuse of errno (2005-06-20) http://python.org/sf/1223937 opened by Oren Tirosh error locale.getlocale() with LANGUAGE=eu_ES (2005-06-20) http://python.org/sf/1223976 opened by Zunbeltz Izaola Len too large with national characters (2005-06-20) CLOSED http://python.org/sf/1224047 opened by Henrik Winther Jensen int/long unification and hex() (2005-06-20) http://python.org/sf/1224347 opened by Josiah Carlson tokenize module does not detect inconsistent dedents (2005-06-21) CLOSED http://python.org/sf/1224621 opened by Danny Yoo rsplit introduces spurious CR (2005-06-21) http://python.org/sf/1224672 opened by Richard Prosser Line endings problem with Python 2.4.1 cause imports to fail (2005-06-21) http://python.org/sf/1225059 opened by Nicolas Lehuen inspect.isclass() fails with custom __getattr__ (2005-06-21) http://python.org/sf/1225107 opened by Jeremy Kloth Bugs Closed ___________ float issue for NaN type in .pyc file (2004-12-07) http://python.org/sf/1080440 closed by mwh buffer() object broken. (2005-06-15) http://python.org/sf/1221424 closed by nascheme Incorrect documentation for InPlace Number operators (2005-06-15) http://python.org/sf/1221477 closed by rhettinger Replace MSVC memory allocator with ptmalloc2 (2005-06-07) http://python.org/sf/1216562 closed by loewis Sub threads execute in restricted mode (2005-03-15) http://python.org/sf/1163563 closed by mwh float() not accurate (2005-06-16) http://python.org/sf/1222098 closed by nascheme Typo in section 5.3 of tutorial (2005-06-17) http://python.org/sf/1222459 closed by rhettinger it's -> its (2005-06-17) http://python.org/sf/1222928 closed by nascheme 2.4.1 .msi file won't install to different disk (2005-06-17) http://python.org/sf/1222978 closed by hacksoncode spurious blank page in dist.pdf (2005-05-27) http://python.org/sf/1209560 closed by birkenfeld subprocess.py abuse of errno (2005-06-20) http://python.org/sf/1223937 closed by astrand Len too large with national characters (2005-06-20) http://python.org/sf/1224047 closed by mwh tokenize module does not detect inconsistent dedents (2005-06-21) http://python.org/sf/1224621 closed by rhettinger subprocess call() helper should close stdin if PIPE (2005-06-14) http://python.org/sf/1220113 closed by astrand From jcarlson at uci.edu Wed Jun 22 07:16:34 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 21 Jun 2005 22:16:34 -0700 Subject: [Python-Dev] Recommend accepting PEP 312 -- Simple Implicit Lambda In-Reply-To: <42B82E7D.4070906@gmail.com> References: <42B82E7D.4070906@gmail.com> Message-ID: <20050621221018.44F9.JCARLSON@uci.edu> Nick Coghlan wrote: > > Guido van Rossum wrote: > > But as long as we are describing the > > present state we should call a spade a spade, etc. > > I guess I take a syntactic view of the status quo, because, while > lambdas may be implemented as anonymous functions, the current syntax > doesn't let me *write* an arbitrary function as a lambda. You can write anything as a lambda, but it may not be easy. > Regardless, I believe the balance will eventually tip in some > direction - either lambdas disappear entirely, become able support > full anonymous functions, or else the idea of a 'deferred expression' > becomes a defining characteristic, rather than a syntactic quirk. I would put my money on the latter rather than the former. The moment functions start moving beyond a line or so is when they usually start begging for a name. - Josiah From stuart at stuartbishop.net Wed Jun 22 08:20:02 2005 From: stuart at stuartbishop.net (Stuart Bishop) Date: Wed, 22 Jun 2005 16:20:02 +1000 Subject: [Python-Dev] subprocess.call() and stdin Message-ID: <42B90312.5070809@stuartbishop.net> Redirecting to python-dev for discussion. When I invoke subprocess.call(), I often want to ensure that the subprocess' stdin is closed. This ensures it will die if the subprocess attempts to read from stdin rather than block. This could be done if the subprocess.call() helper closes the input if stdin=subprocess.PIPE, which may be the only sane way to handle this argument (I can't think of any use cases for spawning a subprocess with an input stream that nothing can write too). It could also be done by adding a subprocess.CLOSED constant, which if passed to Popen causes a new closed file descriptor to be given to the subprocess. -------- Original Message -------- Bugs item #1220113, was opened at 2005-06-14 15:04 Message generated for change (Comment added) made by zenzen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Rejected Priority: 5 Submitted By: Stuart Bishop (zenzen) Assigned to: Peter ?strand (astrand) Summary: subprocess call() helper should close stdin if PIPE Initial Comment: The following code snippet should die instead of hang. >>> from subprocess import call, PIPE >>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=PIPE) It makes no sense not to close stdin if it is PIPE because the stream cannot be accessed. The use case for this is ensuring a subprocess that detects if it is connected to a terminal or not runs in 'batch' mode, and that it will die instead of hang if it unexpectidly attempts to read from stdin. Workaround is to use Popen instead. ---------------------------------------------------------------------- >Comment By: Stuart Bishop (zenzen) Date: 2005-06-22 16:12 Message: Logged In: YES user_id=46639 I can't think of any uses cases for wanting to create an inaccessible pipe and give it to the child. Wanting to pass a closed file handle is common. It is needed when calling a program that behaves differently if its stdin is a terminal or not. Or when you simply would prefer the subprocess to die if it attempts to read from its stdin rather than block. Using Popen instead of call is s simpler workaround than creating and closing a file descriptor and passing it in. Perhaps what is needed is a new constant, subprocess.CLOSED which creates a new file descriptor and closes it? This would be useful for Popen too, allowing call() to remain a think and trivially documented wrapper? ---------------------------------------------------------------------- Comment By: Peter ?strand (astrand) Date: 2005-06-22 02:08 Message: Logged In: YES user_id=344921 >It makes no sense not to close stdin if it is PIPE >because the stream cannot be accessed True, but what if you actually *want* to create an inaccessible pipe, and give it to the child? Currently, the call() wrapper is *very* short and simple. I think this is very good. For example, it allows us to document it in two sentences. You get what you ask for: If you use call() with strange arguments, you'll get a somewhat strange behav?our. I see no point in introducing lots of sanity checks in the wrapper functions. >The use case for this is ensuring a subprocess that >detects if it is connected to a terminal or not runs in >batch' mode, and that it will die instead of hang if >it unexpectidly attempts to read from stdin I'm not sure I understand what you want, but if you want to have stdin connected to a closed file descriptor, just pass one: >>> from subprocess import call, PIPE >>> rv = call(['/usr/bin/bzip2', '-c'], stdout=PIPE, stdin=4711) (Of course, you should verify that 4711 is unused.) If you don't agree with me, post to python-dev for discussion. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1220113&group_id=5470 -- Stuart Bishop http://www.stuartbishop.net/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-dev/attachments/20050622/8173968d/signature-0001.pgp From fredrik at pythonware.com Wed Jun 22 10:48:47 2005 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 22 Jun 2005 10:48:47 +0200 Subject: [Python-Dev] PEP for RFE 46738 (first draft) References: <4e4a11f805061723192888915e@mail.gmail.com><7168d65a0506180644367753b9@mail.gmail.com><4e4a11f805061807127f39700b@mail.gmail.com><17076.21429.501499.914085@montanaro.dyndns.org><4e4a11f805061817462238af59@mail.gmail.com> <17076.61341.558478.28695@montanaro.dyndns.org> Message-ID: Skip Montanaro wrote: > That's fine, so XML-RPC is slower than Gherkin. I can't run the Gherkin > code, but my XML-RPC numbers are a bit different than yours: > > XMLRPC encode 0.65 seconds > XMLRPC decode 2.61 seconds > > That leads me to believe you're not using any sort of C XML decoder. (I > mentioned sgmlop in my previous post. I'm sure /F has some other > super-duper accelerator that's even faster.) the CET/iterparse-based decoder described here http://effbot.org/zone/element-iterparse.htm is 3-4 times faster (but I don't recall if I used sgmlop or just plain expat in those tests). From t-meyer at ihug.co.nz Wed Jun 22 12:54:26 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Wed, 22 Jun 2005 22:54:26 +1200 Subject: [Python-Dev] python-dev Summary for 2005-05-16 through 2005-05-31 [draft] Message-ID: You may have noticed that the summaries have been absent for the last month - apologies for that; Steve has been dutifully doing his part, but I've been caught up with other things. Anyway, Steve will post the May 01-15 draft shortly, and here's May 16-31. We should be able to get the first June one done fairly shortly, too. If anyone has time to flick over this and let me/Steve/Tim know if you have corrections, that would be great; thanks! =Tony.Meyer ============= Announcements ============= ---- QOTF ---- We have our first ever Quote of the Fortnight (QOTF), thanks to the wave of discussion over `PEP 343`_ and Jack Diederich: I still haven't gotten used to Guido's heart-attack inducing early enthusiasm for strange things followed later by a simple proclamation I like. Some day I'll learn that the sound of fingernails on the chalkboard is frequently followed by candy for the whole class. See, even threads about anonymous block statements can end happily! ;) .. _PEP 343: http://www.python.org/peps/pep-0343.html Contributing thread: - `PEP 343 - Abstract Block Redux `__ [SJB] ------------------ First PyPy Release ------------------ The first release of `PyPy`_, the Python implementation of Python, is finally available. The PyPy team has made impressive progress, and the current release of PyPy now passes around 90% of the Python language regression tests that do not depend deeply on C-extensions. The PyPy interpreter still runs on top of a CPython interpreter though, so it is still quite slow due to the double-interpretation penalty. .. _PyPy: http://codespeak.net/pypy Contributing thread: - `First PyPy (preview) release `__ [SJB] -------------------------------- Thesis: Type Inference in Python -------------------------------- Brett C. successfully defended his masters thesis `Localized Type Inference of Atomic Types in Python`_, which investigates some of the issues of applying type inference to the current Python language, as well as to the Python language augmented with type annotations. Congrats Brett! .. _Localized Type Inference of Atomic Types in Python: http://www.drifty.org/thesis.pdf Contributing thread: - `Localized Type Inference of Atomic Types in Python `__ [SJB] ========= Summaries ========= --------------------------- PEP 343 and With Statements --------------------------- The discussion on "anonymous block statements" brought itself closer to a real conclusion this fortnight, with the discussion around `PEP 343`_ and `PEP 3XX`_ converging not only on the semantics for "with statements", but also on semantics for using generators as with-statement templates. To aid in the adaptation of generators to with-statements, Guido proposed adding close() and throw() methods to generator objects, similar to the ones suggested by `PEP 325`_ and `PEP 288`_. The throw() method would cause an exception to be raised at the point where the generator is currently suspended, and the close() method would use throw() to signal the generator to clean itself up by raising a GeneratorExit exception. People seemed generally happy with this proposal and -- believe it or not -- we actually went an entire eight days without an email about anonymous block statements!! It looked as if an updated `PEP 343`_, including the new generator functionality, would be coming early the next month. So stay tuned. ;) .. _PEP 288: http://www.python.org/peps/pep-0288.html .. _PEP 325: http://www.python.org/peps/pep-0325.html .. _PEP 343: http://www.python.org/peps/pep-0343.html .. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html Contributing threads: - `PEP 343 - Abstract Block Redux `__ - `Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux) `__ - `Example for PEP 343 `__ - `Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup `__ - `PEP 343 - New kind of yield statement? `__ - `PEP 342/343 status? `__ - `PEP 346: User defined statements (formerly known as PEP 3XX) `__ [SJB] ----------- Decimal FAQ ----------- Raymond Hettinger suggested that a decimal FAQ would shorten the module's learning curve, and drafted one. There were no objections, but few adjustments (to the list, at least). Raymond will probably make the FAQ available at some point. Contributing thread: - `Decimal FAQ `__ [TAM] --------------------- Constructing Decimals --------------------- A long discussion took place regarding whether the decimal constructor should or should not respect context settings, and whether matching the standard (and what the standard says) should be a priority. Raymond Hettinger took the lead in the status-quo (does not) corner, with Tim Peters leading the opposition. Tim and Guido eventually called in the standard's expert, Mike Cowlishaw. He gave a very thorough explanation of the history behind his decisions in this matter, and eventually weighed in on Raymond's side. As such, it seems that the status-quo has won (not that it was a competition, of course ). For those that need to know, the unary plus operation, as strange as it looks, forces a rounding using the current context. As such, context-aware construction can be written:: val = +Decimal(string_repr) Contributing threads: - `Adventures with Decimal `__ - `Decimal construction `__ - `[Python-checkins] python/nondist/peps pep-0343.txt, 1.8, 1.9 `__ [TAM] ------------------------ Handling old bug reports ------------------------ Facundo Batista continued with his progress checking the open bug reports, looking for bugs that are specific to 2.2.1 or 2.2.2. The aim is to verify whether these bugs exist in current CVS, or are old-of-date. There are no longer any bugs in the 2.1.x or 2.2.x categories, and Facundo wondered whether removing those categories would be a good idea. The consensus was that there was no harm in leaving the categories there, but that changing the text to indicate that those versions are unmaintained would be a good idea. Raymond Hettinger reminded us that care needs to be taken in closing old bug reports. Particularly, a bug report should only be closed if (a) there are no means of reproducing the error, (b) it is impossible to tell what the poster meant, and they are no longer contactable, or (c) the bug is no longer present in current CVS. Contributing threads: - `Deprecating old bugs, now from 2.2.2 `__ - `Closing old bugs `__ - `Old Python version categories in Bug Tracker `__ [TAM] ------------------ Exception chaining ------------------ Ka-Ping Yee has submitted `PEP 344`_, which is a concrete proposal for exception chaining. It proposes three standard attributes on trackback objects: __context__ for implicit chaining, __cause__ for explicit chaining, and __traceback__ to point to the traceback. Guido likes the motivation and rationale, but feels that the specification needs more work. A lot of discussion about the specifics of the PEP took place, and Ka-Ping is working these into a revised version. One of the major questions was whether there is no need for both __context__ and __cause__ (to differentiate between explicit and implicit chaining). Guido didn't feel that there was, but others disagreed. Discussion branched off into whether which attributes should be double-underscored, or not. Guido's opinion is that it depends who "owns" the namespace, and with "magic" behaviour caused (or indicated) by the presence of the attribute. He felt that the underscores in the proposed exception attributes should remain. .. _PEP 344: http://www.python.org/peps/pep-0344.html Contributing threads: - `PEP 344: Exception Chaining and Embedded Tracebacks `__ - `PEP 344: Implicit Chaining Semantics `__ [Python-Dev] PEP 344: Implicit Chaining Semantics - `PEP 344: Explicit vs. Implicit Chaining `__ [Python-Dev] PEP 344: Explicit vs. Implicit Chaining - `Tidier Exceptions `__ [TAM] ------------------------------------ Adding content to exception messages ------------------------------------ Nicolas Fleury suggested that there should be a standard method of adding information to an existing exception (to re-raise it). Nick Coghlan suggested that this would be reasonably simple to do with PEP 344, if all exceptions were also new-style classes, but Nicolas indicated that this wouldn't work in some cases. Contributing threads: - `Adding content to exception messages `__ [Python-Dev] Adding content to exception messages [TAM] =============== Skipped Threads =============== - `Loading compiled modules under MSYS/MingGW? `__ - `RFC: rewrite fileinput module to use itertools. `__ - `Multiple interpreters not compatible with current thread module `__ - `Weekly Python Patch/Bug Summary `__ - `Request for dev permissions `__ - `python-dev Summary for 2005-05-01 through 2005-05-15 [draft] `__ - `AST manipulation and source code generation `__ - `Weekly Python Patch/Bug Summary `__ - `AST branch patches (was Re: PEP 342/343 status?) `__ - `[Python-checkins] python/dist/src/Lib/test test_site.py, 1.6, 1.7 `__ - `Split MIME headers into multiple lines near a space `__ From gmccaughan at synaptics-uk.com Wed Jun 22 13:27:32 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Wed, 22 Jun 2005 12:27:32 +0100 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> Message-ID: <200506221227.33222.gmccaughan@synaptics-uk.com> [GvR:] > > Huh? C unsigned ints don't flag overflow either -- they perform > > perfect arithmetic mod 2**32. [Keith Dart:] > I was talking about signed ints. Sorry about the confusion. Other > scripting languages (e.g. perl) do not error on overflow. C signed ints also don't flag overflow, nor do they -- as you point out -- in various other languages. > > (c) The right place to do the overflow checks is in the API wrappers, > > not in the integer types. > > That would be the "traditional" method. > > I was trying to keep it an object-oriented API. What should "know" the > overflow condition is the type object itself. It raises OverFlowError any > time this occurs, for any operation, implicitly. I prefer to catch errors > earlier, rather than later. Why "should"? Sure, catch errors earlier. But *are* the things you'd catch earlier by having an unsigned-32-bit-integer type actually errors? Is it, e.g., an "error" to move the low 16 bits into the high part by writing x = (y<<16) & 0xFFFF0000 instead of x = (y&0xFFFF) << 16 or to add 1 mod 2^32 by writing x = (y+1) & 0xFFFFFFFF instead of if y == 0xFFFFFFFF: x = 0 else: x = y+1 ? Because it sure doesn't seem that way to me. Why is it better, or more "object-oriented", to have the checking done by a fixed-size integer type? > > (b) I don't know what you call a "normal" integer any more; to me, > > unified long/int is as normal as they come. Trust me, that's the case > > for most users. Worrying about 32 bits becomes less and less normal. > > By "normal" integer I mean the mathematical definition. Then you aren't (to me) making sense. You were distinguishing this from a unified int/long. So far as I can see, a unified int/long type *does* implement (modulo implementation limits and bugs) the "mathematical definition". What am I missing? > Most Python users > don't have to worry about 32 bits now, that is a good thing when you are > dealing only with Python. However, if one has to interface to other > systems that have definite types with limits, then one must "hack around" > this feature. Why is checking the range of a parameter with a restricted range a "hack"? Suppose some "other system" has a function in its interface that expects a non-zero integer argument, or one with its low bit set. Do we need a non-zero-integer type and an odd-integer type? > I was just thinking how nice it would be if Python had, in > addition to unified ("real", "normal") integers it also had built-in > types that could be more easily mapped to external types (the typical > set of signed, unsigned, short, long, etc.). Yes, you can check it at > conversion time, but that would mean extra Python bytecode. It seems you > think this is a special case, but I think Python may be used as a "glue > language" fairly often, and some of us would benefit from having those > extra types as built-ins. Well, which extra types? One for each of 8, 16, 32, 64 bit and for each of signed, unsigned? Maybe also "non-negative signed" of each size? That's 12 new builtin types, so perhaps you'd be proposing a subset; what subset? And how are they going to be used? - If the conversion to one of these new limited types occurs immediately before calling whatever function it is that uses it, then what you're really doing is a single range-check. Why disguise it as a conversion? - If the conversion occurs earlier, then you're restricting the ways in which you can calculate the parameter values in question. What's the extra value in that? I expect I'm missing something important. Could you provide some actual examples of how code using this new feature would look? -- g From fredrik.johansson at gmail.com Wed Jun 22 13:41:39 2005 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Wed, 22 Jun 2005 13:41:39 +0200 Subject: [Python-Dev] Decimal floats as default (was: discussion about PEP239 and 240) Message-ID: <3d0cebfb05062204414026741b@mail.gmail.com> Hi all, raymond.hettinger at verizon.net Fri Jun 17 10:36:01 2005 wrote: > The future direction of the decimal module likely entails literals in > the form of 123.45d with binary floats continuing to have the form > 123.45. This conflicts with the rational literal proposal of having > 123.45 interpreted as 123 + 45/100. Decimal literals are a wonderful idea, especially if it means that decimals and floats can be made to interact with each other directly. But why not go one step further, making 123.45 decimal and 123.45b binary? In fact, I think a good case can be made for replacing the default float type with a decimal type. Decimal floats make life easier for humans accustomed to base 10, so they should be easy to use. This is particularly relevant given Python's relatively large user base of "non-programmers", but applies to many domains. Easy-to-use, correct rounding is essential in many applications that need to process human-readable data (round() would certainly be more meaningful if it operated on decimals). Not to mention that arbitrary precision arithmetic just makes the language more powerful. Rationals are inappropriate except in highly specialized applications because of the non-constant size and processing time, but decimals would only slow down programs by a (usually small) constant factor. I suspect most Python programs do not demand the performance hardware floats deliver, nor require the limited precision or particular behaviour of IEEE 754 binary floats (the need for machine-precision integers might be greater -- I've written "& 0xffffffffL" many times). Changing to decimal would not significantly affect users who really need good numeric performance either. The C interface would convert Python floats to C doubles as usual, and numarray would function accordingly. Additionally, "hardware" could be a special value for the precision in the decimal (future float) context. In that case, decimal floats could be phased in without breaking compatibility, by leaving hardware as the default precision. 123.45d is better than Decimal("123.45"), but appending "d" to specify a quantity with high precision is as illogical as appending "L" to an integer value to bypass the machine word size limit. I think the step from hardware floats to arbitrary-precision decimals would be as natural as going from short to unlimited-size integers. I've thought of the further implications for complex numbers and the math library, but I'll stop writing here to listen to feedback in case there is some obvious technical flaw or people just don't like the idea :-) Sorry if this has been discussed and/or rejected before (this is my first post to python-dev, though I've occasionally read the list since I started using Python extensively about two years ago). Fredrik Johansson From ncoghlan at gmail.com Wed Jun 22 14:32:25 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 22 Jun 2005 22:32:25 +1000 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: <200506221227.33222.gmccaughan@synaptics-uk.com> References: <2m1x6z7qu6.fsf@starship.python.net> <200506221227.33222.gmccaughan@synaptics-uk.com> Message-ID: <42B95A59.3070700@gmail.com> Gareth McCaughan wrote: > [Keith Dart:] >>By "normal" integer I mean the mathematical definition. > > Then you aren't (to me) making sense. You were distinguishing > this from a unified int/long. So far as I can see, a unified int/long > type *does* implement (modulo implementation limits and bugs) > the "mathematical definition". What am I missing? Hmm, a 'mod_int' type might be an interesting concept (i.e. a type that performs integer arithmetic, only each operation is carried out modulo some integer). Then particular bit sizes would be simple ints, modulo the appropriate power of two. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Wed Jun 22 14:36:54 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 22 Jun 2005 22:36:54 +1000 Subject: [Python-Dev] Explicitly declaring expected exceptions for a block In-Reply-To: <1259444531.20050622100949@targeted.org> References: <1355231971.20050621112518@targeted.org> <85f6a31f05062022564619f4f6@mail.gmail.com> <42B8295B.8030107@gmail.com> <1259444531.20050622100949@targeted.org> Message-ID: <42B95B66.6090502@gmail.com> Dmitry Dvoinikov wrote: > Now, back to original question then, do you think it'd be > beneficial to have some sort of exception ignoring or expecting > statement ? Not really - as I said earlier, I usually have something non-trivial in the except or else clause, so I'm not simply ignoring the exceptions. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From ncoghlan at gmail.com Wed Jun 22 14:45:07 2005 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 22 Jun 2005 22:45:07 +1000 Subject: [Python-Dev] python-dev Summary for 2005-05-16 through 2005-05-31 [draft] In-Reply-To: References: Message-ID: <42B95D53.9050608@gmail.com> Tony Meyer wrote: > .. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html Now that PEP 346 is on python.org, it would be best to reference that, rather than the PEP 3XX URL on my ISP web space (that URL is now just a redirect to PEP 346 on python.org). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.blogspot.com From eppstein at ics.uci.edu Tue Jun 21 22:51:06 2005 From: eppstein at ics.uci.edu (David Eppstein) Date: Tue, 21 Jun 2005 13:51:06 -0700 Subject: [Python-Dev] Propose updating PEP 284 -- Integer for-loops In-Reply-To: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> References: <000401c573e2$16cd8ce0$bb07a044@oemcomputer> Message-ID: <7592058.1119361866@dhcp31-171.ics.uci.edu> On 6/18/05 4:45 AM -0400 Raymond Hettinger wrote: > The above recommendations should get the PEP ready for judgement day. I thought judgement day already happened for this PEP in the "Parade of PEPs". No? > Also, I recommend tightening the PEP's motivation. There are only two > advantages, encoding and readability. The former is only a minor gain > because all it saves is a function call, an O(1) savings in an O(n) > context. The latter is where the real benefits lay. The readability part is really my only motivation for this. In a nutshell, I would like a syntax that I could use with little or no explanation with people who already understand some concepts of imperative programming but have never seen Python before (e.g. when I use Python-like syntax for the pseudocode in my algorithms lectures). The current for x in range(...) syntax is not quite there. In practice, I have been using for x in 0, 1, 2, ... n-1: which does not work as actual programming language syntax but seems to communicate my point better than the available syntaxes. I have to admit, among your proposed options > for i between 2 < i <= 10: ... > for i over 2 < i <= 10: ... # chained comparison style > for i over [2:11]: ... # Slice style > for i = 3 to 10: ... # Basic style I don't really care for the repetition of the variable name in the first two, the third is no more readable to me than the current range syntax, and the last one only looks ok to me because I used to program in Pascal, long ago. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ From fredrik.johansson at gmail.com Wed Jun 22 16:50:12 2005 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Wed, 22 Jun 2005 16:50:12 +0200 Subject: [Python-Dev] Decimal floats as default (was: discussion about PEP239 and 240) In-Reply-To: <200506220942.52768.mclay@python.net> References: <3d0cebfb05062204414026741b@mail.gmail.com> <200506220942.52768.mclay@python.net> Message-ID: <3d0cebfb05062207503f3256b2@mail.gmail.com> On 6/22/05, Michael McLay wrote: > This idea is dead on arrival. The change would break many applications and > modules. A successful proposal cannot break backwards compatibility. Adding a > dpython interpreter to the current code base is one possiblity. Is there actually much code around that relies on the particular precision of 32- or 64-bit binary floats for arithmetic, and ceases working when higher precision is available? Note that functions like struct.pack would be unaffected. If compatibility is a problem, this could still be a possibility for Python 3.0. In either case, compatibility can be ensured by allowing both n-digit decimal and hardware binary precision for floats, settable via a float context. Then the backwards compatible binary mode can be default, and "decimal mode" can be set with one line of code. d-suffixed literals create floats with decimal precision. There is the alternative of providing decimal literals by using separate decimal and binary float base types, but in my eyes this would be redundant. The primary use of binary floats is performance and compatibility, and both can be achieved with my proposal without sacrificing the simplicity and elegance of having a single type to represent non-integral numbers. It makes more sense to extend the float type with the power and versatility of the decimal module than to have a special type side by side with a default type that is less capable. Fredrik From gmccaughan at synaptics-uk.com Wed Jun 22 17:10:27 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Wed, 22 Jun 2005 16:10:27 +0100 Subject: [Python-Dev] =?iso-8859-1?q?Is_PEP_237_final_--_Unifying_Long_Int?= =?iso-8859-1?q?egers_and=09Integers?= In-Reply-To: <42B95A59.3070700@gmail.com> References: <2m1x6z7qu6.fsf@starship.python.net> <200506221227.33222.gmccaughan@synaptics-uk.com> <42B95A59.3070700@gmail.com> Message-ID: <200506221610.28305.gmccaughan@synaptics-uk.com> On Wednesday 2005-06-22 13:32, Nick Coghlan wrote: > Gareth McCaughan wrote: > > [Keith Dart:] > >>By "normal" integer I mean the mathematical definition. > > > > Then you aren't (to me) making sense. You were distinguishing > > this from a unified int/long. So far as I can see, a unified int/long > > type *does* implement (modulo implementation limits and bugs) > > the "mathematical definition". What am I missing? > > Hmm, a 'mod_int' type might be an interesting concept (i.e. a type > that performs integer arithmetic, only each operation is carried out > modulo some integer). > > Then particular bit sizes would be simple ints, modulo the appropriate > power of two. It might indeed, but it would be entirely the opposite of what (I think) Keith wants, namely something that raises an exception any time a value goes out of range :-). -- g From skip at pobox.com Wed Jun 22 21:22:16 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 22 Jun 2005 14:22:16 -0500 Subject: [Python-Dev] Decimal floats as default (was: discussion about PEP239 and 240) In-Reply-To: <3d0cebfb05062207503f3256b2@mail.gmail.com> References: <3d0cebfb05062204414026741b@mail.gmail.com> <200506220942.52768.mclay@python.net> <3d0cebfb05062207503f3256b2@mail.gmail.com> Message-ID: <17081.47720.970132.531615@montanaro.dyndns.org> Fredrik> Is there actually much code around that relies on the Fredrik> particular precision of 32- or 64-bit binary floats for Fredrik> arithmetic, and ceases working when higher precision is Fredrik> available? Umm, yeah... The path you take from one or more string literals representing real numbers through a series of calculations and ending up in a hardware double-precision floating point number is probably going to be different at different precisions. >>> x = Decimal("1.0000000000001") >>> y = Decimal("1.000000000000024567") >>> x Decimal("1.0000000000001") >>> y Decimal("1.000000000000024567") >>> float(x) 1.0000000000000999 >>> float(y) 1.0000000000000246 >>> x/y Decimal("1.000000000000075432999999998") >>> float(x)/float(y) 1.0000000000000753 >>> float(x/y) 1.0000000000000755 Performance matters too: % timeit.py -s 'from decimal import Decimal ; x = Decimal("1.0000000000001") ; y = Decimal("1.000000000000024567")' 'x/y' 1000 loops, best of 3: 1.39e+03 usec per loop % timeit.py -s 'from decimal import Decimal ; x = float(Decimal("1.0000000000001")) ; y = float(Decimal("1.000000000000024567"))' 'x/y' 1000000 loops, best of 3: 0.583 usec per loop I imagine a lot of people would be very unhappy if their fp calculations suddenly began taking 2000x longer, even if their algorithms didn't break. (For all I know, Raymond might have a C version of Decimal waiting for an unsuspecting straight man to complain about Decimal's performance and give him a chance to announce it.) If nothing else, extension module code that executes f = PyFloat_AsDouble(o); or if (PyFloat_Check(o)) { ... } would either have to change or those functions would have to be rewritten to accept Decimal objects and convert them to doubles (probably silently, because otherwise there would be so many warnings). For examples of packages that might make large use of these functions, take a look at Numeric, SciPy, ScientificPython, MayaVi, and any other package that does lots of floating point arithmetic. Like Michael wrote, I think this idea is DOA. Skip From fredrik.johansson at gmail.com Wed Jun 22 22:19:39 2005 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Wed, 22 Jun 2005 22:19:39 +0200 Subject: [Python-Dev] Decimal floats as default (was: discussion about PEP239 and 240) In-Reply-To: <17081.47720.970132.531615@montanaro.dyndns.org> References: <3d0cebfb05062204414026741b@mail.gmail.com> <200506220942.52768.mclay@python.net> <3d0cebfb05062207503f3256b2@mail.gmail.com> <17081.47720.970132.531615@montanaro.dyndns.org> Message-ID: <3d0cebfb05062213196393d9d0@mail.gmail.com> On 6/22/05, Skip Montanaro wrote: > If nothing else, extension module code that executes > > f = PyFloat_AsDouble(o); > > or > > if (PyFloat_Check(o)) { > ... > } > > would either have to change or those functions would have to be rewritten to > accept Decimal objects and convert them to doubles (probably silently, > because otherwise there would be so many warnings). > Silent conversion was the idea. > Like Michael wrote, I think this idea is DOA. Granted, then. However, keeping binary as default does not kill the other idea in my proposal, which is to extend the float type to cover decimals instead of having a separate decimal type. I consider this a more important issue (contradicting the thread title :-) than whether "d" should be needed to specify decimal precision. Fredrik From steven.bethard at gmail.com Wed Jun 22 23:07:07 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 22 Jun 2005 15:07:07 -0600 Subject: [Python-Dev] python-dev Summary for 2005-05-01 through 2005-05-16 [draft] Message-ID: Here's the May 01-15 draft. Sorry for the delay. Please check the Unicode summary at the end especially closely; I'm not entirely sure I got that one all right. Thanks! As always, please let us know if you have any corrections! Steve ====================== Summary Announcements ====================== ---------------------------------------------- PEP 340 Episode 2: Revenge of the With (Block) ---------------------------------------------- This fornight's Python-Dev was dominated again by another nearly 400 messages on the topic of anonymous block statements. The discussion was a little more focused than the last thanks mainly to Guido's introduction of `PEP 340`_. Discussion of this PEP resulted in a series of other PEPs, including * `PEP 342`_: Enhanced Iterators, which broke out into a separate PEP the parts of `PEP 340`_ that allowed code to pass values into iterators using ``continue EXPR`` and yield-expressions. * `PEP 343`_: Anonymous Block Redux, a dramatically simplified version of `PEP 340`_, which removed the looping nature of the anonymous blocks and the injection-of-exceptions semantics for generators. * `PEP 3XX`_: User Defined ("with") Statements, which proposed non-looping anonymous blocks accompanied by finalization semantics for iterators and generators in for loops. Various details of each of these proposals are discussed below in the sections: 1. `Enhanced Iterators`_ 2. `Separate APIs for Iterators and Anonymous Blocks`_ 3. `Looping Anonymous Blocks`_ 4. `Loop Finalization`_ At the time of this writing, it looked like the discussion was coming very close to a final agreement; `PEP 343`_ and `PEP 3XX`_ both agreed upon the same semantics for the block-statement, the keyword had been narrowed down to either ``do`` or ``with``, and Guido had agreed to add back in to `PEP 343`_ some form of exception-injection semantics for generators. .. _PEP 340: http://www.python.org/peps/pep-0340.html .. _PEP 342: http://www.python.org/peps/pep-0342.html .. _PEP 343: http://www.python.org/peps/pep-0343.html .. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html [SJB] ========= Summaries ========= ------------------ Enhanced Iterators ------------------ `PEP 340`_ incorporated a variety of orthogonal features into a single proposal. To make the PEP somewhat less monolithic, the method for passing values into an iterator was broken off into `PEP 342`_. This method includes: * updating the iterator protocol to use .__next__() instead of .next() * introducing a new builtin next() * allowing continue-statements to pass values into iterators * allowing generators to receive values with a yield-expression Though these features had seemed mostly uncontroversial, Guido seemed inclined to wait for a little more motivation from the co-routiney people before accepting the proposal. Contributing threads: - `Breaking off Enhanced Iterators PEP from PEP 340 `__ [SJB] ------------------------------------------------ Separate APIs for Iterators and Anonymous Blocks ------------------------------------------------ `PEP 340`_ had originally proposed to treat the anonymous block protocol as an extension of the iterator protocol. Several problems with this approach were raised, including: * for-loops could accidentally be used with objects requiring blocks, meaning that resources would not get cleaned up properly * blocks could be used instead of for-loops, violating TOOWTDI As a result, both `PEP 343`_ and `PEP 3XX`_ propose decorators for generator functions that will wrap the generator object appropriately to match the anonymous block protocol. Generator objects without the proposed decorators would not be usable in anonymous block statements. Contributing threads: - `PEP 340 -- loose ends `__ - `PEP 340 -- concept clarification `__ [SJB] ------------------------ Looping Anonymous Blocks ------------------------ A few issues arose as a result of `PEP 340`_'s formulation of anonymous blocks as a variation on a loop. Because the anonymous blocks of `PEP 340`_ were defined in terms of while-loops, there was some discussion as to whether they should have an ``else`` clause like Python ``for`` and ``while`` loops do. There didn't seem to be one obvious interpretation of an ``else`` block though, so Guido rejected the ``else`` block proposal. The big issue with looping anonymous blocks, however, was in the handling of ``break`` and ``continue`` statements. Many use cases for anonymous blocks did not require loops. However, because `PEP 340`_ anonymous blocks were implemented in terms of loops, ``break`` and ``continue`` acted much like they would in a loop. This meant that in code like:: for item in items: with lock: if handle(item): break the ``break`` statement would only break out of the anonymous block (the ``with`` statement) instead of breaking out of the for-loop. This pretty much shot-down `PEP 340`_; there were too many cases where an anonymous block didn't look like a loop, and having it behave like one would have been a major stumbling block in learning the construct. As a result, both `PEP 343`_ and `PEP 3XX`_ were proposed as non-looping versions of `PEP 340`_. Contributing threads: - `PEP 340: Else clause for block statements `__ - `PEP 340 -- loose ends `__ - `PEP 340 -- concept clarification `__ - `PEP 340: Breaking out. `__ - `PEP 340: Non-looping version (aka PEP 310 redux) `__ - `PEP 340 - Remaining issues `__ - `PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340) `__ - `Merging PEP 310 and PEP 340-redux? `__ - `PEP 343 - Abstract Block Redux `__ [SJB] ----------------- Loop Finalization ----------------- Greg Ewing pointed out that a generator with a yield inside a block-statement would require additional work to guarantee its finalization. For example, if the generator:: def all_lines(filenames): for name in filenames: with open(name) as f: for line in f: yield line were used in code like:: for line in all_lines(filenames): if some_cond(line): break then unless the for-loop performed some sort of finalization on the all_lines generator, the last-opened file could remain open indefinitiely. As a result, `PEP 3XX`_ proposes that for-loops check for a __finish__() method on their iterators, and if one exists, call that method when the for-loop completes. Generators like all_lines above, that put a yield inside a block-statement, would then acquire a __finish__() method that would raise a TerminateIteration exception at the point of the last yield. The TerminateIteration exception would thus cause the block-statement to complete, guaranteeing that the generator was properly finalized. Contributing threads: - `PEP 340 - For loop cleanup, and feature separation `__ - `PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340) `__ - `PEP 343 - Abstract Block Redux `__ [SJB] ---------------------------- Breaking out of Nested Loops ---------------------------- As a result of some of the issues of looping anonymous blocks, a few threads discussed options for breaking out of nested loops. These mainly worked by augmenting the ``break`` statement with another keyword (or keywords) that would indicate which loop to break out of. One proposal suggested that ``break`` be followed with ``for`` or ``while`` to indicate which loop to break out of. But ``break for`` would only really be useful in a while-loop nested within a for-loop, and ``break while`` would only really be useful in a for-loop nested within a while-loop. That is, because loops could only be named by type, the proposal was only useful when loops of different types were mixed. This suggestion was thus discarded as not being general enough. A few other suggestions were briefly discussed: adding labels to loops, using an integer to indicate which "stack level" to break at, and pushing breaks onto a "break buffer", but Guido killed the discussion, saying, `"Stop all discussion of breaking out of multiple loops. It ain't gonna happen before my retirement." `__ Contributing threads: - `PEP 340: Breaking out. `__ - `PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340) `__ [SJB] ------------------------ The future of exceptions ------------------------ Ka-Ping Yee suggested that instead of passing (type, value, traceback) tuples in exceptions it would be better to put the traceback in value.traceback. Guido had also suggested this (in the `PEP 340`_ murk) but pointed out that this would not work as long as string exceptions exist (as there is nowhere to put the traceback). Guido noted that there are no concrete plans as to when string exceptions will be deprecated and removed (other than in 3.0 at the latest); he indicated that it could be sooner, if someone wrote a PEP with a timeline (e.g. deprecated in 2.5, gone in 2.6). Brett C. volunteered to write a PEP targetted at Python 3000 covering exception changes (base inheritance, standard attributes (e.g. .traceback), reworking the built-in exception inheritance hierarchy, and the future of bare except statements). Contributing threads: - `Tidier Exceptions `__ .. _PEP 340: http://www.python.org/peps/pep-0340.html [TAM] ----------------------------------- Unifying try/except and try/finally ----------------------------------- Reinhold Birkenfeld submitted a Pre-PEP to allow both except and finally clauses in try blocks. For example, a construction like:: try: except Ex1: else: finally: would be exactly the same as the legacy:: try: try: except Ex1: else: finally: Guido liked this idea (so much that he wanted to accept it immediately), and recommended that it was checked in as a PEP. However, Tim Peters pointed out that this functionality was removed from Python (by Guido) way back in 0.9.6, seemingly because there was confusion about exactly when the finally clause would be called (explicit is better than implicit!). Guido clarified that control would only pass forward, and indicated that he felt that since this is now available in Java (and C#) fewer people would be confused. The main concern about this change was that, while the cost was low, it seemed to add very little value. Contributing threads: - `Pre-PEP: Unifying try-except and try-finally `__ [TAM] ----------------- Decorator Library ----------------- Michele Simionato asked whether a module for commonly used decorators, or utilities to create decorators, was planned. Raymond Hettinger indicated that while this was likely in the long term, he felt that it was better if these first evolved via wikis, recipes, or mailing lists, so that a module would only be added once best practices and proven winners had emerged. In the meantime, there is both a `Decorator Library wiki page`_ and you can try out `Michele's library`_ [zip]. To assist with decorator creation, Michele would like a facility to copy a function. Phillip J. Eby noted that the informally-discussed proposal is to add a mutable __signature__ to functions to assist with signature preserving decorators. Raymond suggested a patch adding a __copy__ method to functions or a patch for the copy module, and Michele indicated that he would also like to subclass FunctionType with an user-defined __copy__ method. Contributing threads: - `my first post: asking about a "decorator" module `__ - `The decorator module `__ .. _Decorator Library wiki page: http://www.python.org/moin/PythonDecoratorLibrary .. _Michele's library: http://www.phyast.pitt.edu/~micheles/python/decorator.zip [TAM] --------------------- Hooking Py_FatalError --------------------- Errors that invoke Py_FatalError generally signify that the internal state of Python is in such a poor state that continuing (including raising an exception) is impossible or unwise; as a result, Py_FatalError outputs the error to stderr and calls abort(). m.u.k. would like to have a callback to hook Py_FatalError to avoid this call to abort(). The general consensus was that effort would be better directed to fixing the causes of fatal errors than hooking Py_FatalError. m.u.k.'s use case was for generating additional logging information; a `callback system patch`_ (revised by James William Pye) is available for those interested. Contributing threads: - `Need to hook Py_FatalError `__ .. _callback system patch: http://python.org/sf/1195571 ------------------- Chaining Exceptions ------------------- Ka-Ping Yee suggested adding information to exceptions when they are raised in the handler for another exception. For example:: def a(): try: raise AError except: raise BError raises an exception which is an instance of BError. This instance could have an attribute which is instance of AError, containing information about the original exception. Use cases include catching a low-level exception (e.g. socket.error) and turning it into a high-level exception (e.g. an HTTPRequestFailed exception) and handling problems in exception handling code. Guido liked the idea, and discussion fleshed out a tighter definition; however it was unclear whether adding this now was feasible - this would perhaps be best added in Python 3000. Contributing threads: - `Chained Exceptions `__ [TAM] ------------------------ Py_UNICODE Documentation ------------------------ Nicholas Bastin started a series of threads discussing an inconsistency between the Py_UNICODE docs and the behavior on some RedHat systems. The docs say that Py_UNICODE should be an alias for wchar_t when wchar_t is available and has 16 bits, but Nick found that pyconfig.h still reports PY_UNICODE_TYPE as wchar_t, even when PY_UNICODE_SIZE is 4. An extensive discussion between Nick, Marc-Andre Lemburg and Martin v. L?wis suggests that the possible Python-internal representations for Py_UNICODE are: * 4-byte wchar_t encoded as UTF-32 (UCS-4) * 2-byte wchar_t encoded as UTF-16 * unsigned short encoded as UTF-16 Python defaults to 2-byte mode, using wchar_t if available (and has 16 bits) and using unsigned short otherwise. You may end up with the 4-byte mode if TCL was built for UCS-4 (this overrides the defaults) or if you explicitly request it with --enable-unicode=ucs4. To get UCS-2 when TCL was built for UCS-4, you must explicitly request --enable-unicode=ucs2. Of course, this will mean that _tkinter can't be built anymore. Also noted by this discussion was that even with --enable-unicode=ucs2, Python continues to support surrogate pairs in the BMP. So for example, even with a UCS-2 build, u"\U00012345" encodes as a sequence of two characters; it does not produce a UnicodeError. At the time of this posting, it did not appear that there was a documentation patch available yet. Contributing threads: - `Py_UNICODE madness `__ - `New Py_UNICODE doc `__ - `Python's Unicode width default (New Py_UNICODE doc) `__ [SJB] =============== Skipped Threads =============== - `Keyword for block statements `__ - `PEP 340 - possible new name for block-statement `__ - `Generating nested data structures with blocks `__ - `PEP 340 -- Clayton's keyword? `__ - `PEP 340: Only for try/finally? `__ - `2 words keyword for block `__ - `anonymous blocks `__ - `"begin" as keyword for pep 340 `__ - `PEP 340: propose to get rid of 'as' keyword `__ - `PEP 340 keyword: after `__ - `PEP 340 keyword: Extended while syntax `__ - `PEP 340 - Remaining issues - keyword `__ - `PEP 340: Examples as class's. `__ - `Proposed alternative to __next__ and __exit__ `__ - `"with" use case: exception chaining `__ - `PEP 343: Resource Composition and Idempotent __exit__ `__ - `[Python-checkins] python/nondist/peps pep-0343.txt, 1.8, 1.9 `__ - `the current behavior of try: ... finally: `__ - `a patch to inspect and a non-feature request `__ - `Python 2.4 set objects and cyclic garbage `__ - `CHANGE BayPIGgies: May *THIRD* Thurs `__ - `Python continually calling sigprocmask() on FreeBSD 5 `__ - `Weekly Python Patch/Bug Summary `__ - `problems with memory management `__ - `Adding DBL_MANTISSA and such to Python `__ - `python-dev Summary for 2005-04-16 through 2005-04-30 [draft] `__ - `Python Language track at Europython, still possibilities to submit talks `__ - `(no subject) `__ - `Kernel panic writing to /dev/dsp with cmpci driver `__ From skip at pobox.com Thu Jun 23 01:01:51 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 22 Jun 2005 18:01:51 -0500 Subject: [Python-Dev] PEP 304 - is anyone really interested? Message-ID: <17081.60895.24439.92843@montanaro.dyndns.org> I wrote PEP 304, "Controlling Generation of Bytecode Files": http://www.python.org/peps/pep-0304.html quite awhile ago. The first version appeared in January 2003 in response to questions from people about controlling/suppressing bytecode generation in certain situations. It sat idle for a long while, though from time-to-time people would ask about the functionality and I'd respond or update the PEP. In response to another recent question about this topic: http://mail.python.org/pipermail/python-list/2005-June/284775.html and a wave of recommendations by Raymond Hettinger regarding several other PEPs, I updated the patch to work with current CVS. Aside from one response by Thomas Heller noting that my patch upload failed (and which has been corrected since), I've seen no response either on python-dev or comp.lang.python. I really have no personal use for this functionality. I control all the computers on which I use Python and don't use any exotic hardware (which includes Windows as far with its multi-rooted file system as far as I'm concerned), don't run from read-only media or think that in-memory file systems are much of an advantage over OS caching. The best I will ever do with it is respond to people's inputs. I'd hate to see it sit for another two years. If someone out there is interested in this functionality and would benefit more from its incorporation into the core, I'd be happy to hand it off to you. So speak up folks, otherwise my recommendation is that it be put out of its misery. Skip From trentm at ActiveState.com Thu Jun 23 01:42:54 2005 From: trentm at ActiveState.com (Trent Mick) Date: Wed, 22 Jun 2005 16:42:54 -0700 Subject: [Python-Dev] PEP 304 - is anyone really interested? In-Reply-To: <17081.60895.24439.92843@montanaro.dyndns.org> References: <17081.60895.24439.92843@montanaro.dyndns.org> Message-ID: <20050622234254.GB13632@ActiveState.com> [Skip Montanaro wrote] > > I wrote PEP 304, "Controlling Generation of Bytecode Files": > > http://www.python.org/peps/pep-0304.html > > ... > So speak up folks, otherwise my recommendation is that it be put out of its > misery. I've had use for it before, but have managed to work around the problems. I think it is a good feature, but I wouldn't have the time to shepherd the patch. Trent -- Trent Mick TrentM at ActiveState.com From theller at python.net Thu Jun 23 11:53:27 2005 From: theller at python.net (Thomas Heller) Date: Thu, 23 Jun 2005 11:53:27 +0200 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers References: <2m1x6z7qu6.fsf@starship.python.net> <200506221227.33222.gmccaughan@synaptics-uk.com> <42B95A59.3070700@gmail.com> Message-ID: Nick Coghlan writes: > Gareth McCaughan wrote: >> [Keith Dart:] >>>By "normal" integer I mean the mathematical definition. >> >> Then you aren't (to me) making sense. You were distinguishing >> this from a unified int/long. So far as I can see, a unified int/long >> type *does* implement (modulo implementation limits and bugs) >> the "mathematical definition". What am I missing? > > Hmm, a 'mod_int' type might be an interesting concept (i.e. a type > that performs integer arithmetic, only each operation is carried out > modulo some integer). > > Then particular bit sizes would be simple ints, modulo the appropriate > power of two. ctypes provides mutable platform integers and floats of various sizes (and much more). Currently they don't have any methods, only the value attribute. Maybe it would be useful to implement the standard numeric methods on them. Thomas From paolo_veronelli at libero.it Thu Jun 23 19:07:13 2005 From: paolo_veronelli at libero.it (Paolino) Date: Thu, 23 Jun 2005 19:07:13 +0200 Subject: [Python-Dev] is type a usable feature? Message-ID: <42BAEC41.7090202@libero.it> Hello developers,I noticed my application was growing strangely while I was using type, then I tried this: while True: type('A',(),{}) and saw memory filling up.Is there a clean solution to that? I see it as a bug in python engeneering,that is why I wrote to you. Thanks a lot Paolino From tim.peters at gmail.com Thu Jun 23 18:25:14 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 23 Jun 2005 12:25:14 -0400 Subject: [Python-Dev] is type a usable feature? In-Reply-To: <42BAEC41.7090202@libero.it> References: <42BAEC41.7090202@libero.it> Message-ID: <1f7befae050623092570e2b6a0@mail.gmail.com> [Paolino ] > Hello developers,I noticed my application was growing strangely while I > was using type, then I tried this: > > while True: > type('A',(),{}) > > and saw memory filling up.Is there a clean solution to that? > I see it as a bug in python engeneering,that is why I wrote to you. Python bugs should be reported on SourceForge: http://sourceforge.net/tracker/?group_id=5470&atid=105470 Please specify the Python version and OS. I do not see memory growth running the above under Python 2.3.5 or 2.4.1 on Windows, so I don't have any evidence of a bug here in the Pythons I usually use. From paolo_veronelli at libero.it Thu Jun 23 20:15:30 2005 From: paolo_veronelli at libero.it (Paolino) Date: Thu, 23 Jun 2005 20:15:30 +0200 Subject: [Python-Dev] is type a usable feature? In-Reply-To: <1f7befae050623092570e2b6a0@mail.gmail.com> References: <42BAEC41.7090202@libero.it> <1f7befae050623092570e2b6a0@mail.gmail.com> Message-ID: <42BAFC42.5090505@libero.it> Tim Peters wrote: > [Paolino ] > >>Hello developers,I noticed my application was growing strangely while I >>was using type, then I tried this: >> >>while True: >> type('A',(),{}) >> >>and saw memory filling up.Is there a clean solution to that? >>I see it as a bug in python engeneering,that is why I wrote to you. > > > Python bugs should be reported on SourceForge: > > http://sourceforge.net/tracker/?group_id=5470&atid=105470 > > Please specify the Python version and OS. I do not see memory growth > running the above under Python 2.3.5 or 2.4.1 on Windows, so I don't > have any evidence of a bug here in the Pythons I usually use. > Sorry, the growth happens using ipython shell.With python original shell there is nothing wrong. From paolo_veronelli at tiscali.it Thu Jun 23 20:08:34 2005 From: paolo_veronelli at tiscali.it (Paolino) Date: Thu, 23 Jun 2005 20:08:34 +0200 Subject: [Python-Dev] is type a usable feature? In-Reply-To: <1f7befae050623092570e2b6a0@mail.gmail.com> References: <42BAEC41.7090202@libero.it> <1f7befae050623092570e2b6a0@mail.gmail.com> Message-ID: <42BAFAA2.7060002@tiscali.it> Tim Peters wrote: > [Paolino ] > >>Hello developers,I noticed my application was growing strangely while I >>was using type, then I tried this: >> >>while True: >> type('A',(),{}) >> >>and saw memory filling up.Is there a clean solution to that? >>I see it as a bug in python engeneering,that is why I wrote to you. > > > Python bugs should be reported on SourceForge: > > http://sourceforge.net/tracker/?group_id=5470&atid=105470 > > Please specify the Python version and OS. I do not see memory growth > running the above under Python 2.3.5 or 2.4.1 on Windows, so I don't > have any evidence of a bug here in the Pythons I usually use. > Sorry, the growth happens using ipython shell.With python original shell there is nothing wrong. From tim.peters at gmail.com Fri Jun 24 00:26:00 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 23 Jun 2005 18:26:00 -0400 Subject: [Python-Dev] refcounting vs PyModule_AddObject In-Reply-To: <42B341E3.5000609@v.loewis.de> References: <2mzmtrbxwo.fsf@starship.python.net> <42B0725D.9040200@v.loewis.de> <2m4qbybpf2.fsf@starship.python.net> <42B341E3.5000609@v.loewis.de> Message-ID: <1f7befae05062315266801564c@mail.gmail.com> [Michael Hudson] >> I've been looking at this area partly to try and understand this bug: >> >> [ 1163563 ] Sub threads execute in restricted mode >> >> but I'm not sure the whole idea of multiple interpreters isn't >> inherently doomed :-/ [Martin v. L?wis] > That's what Tim asserts, saying that people who want to use the > feature should fix it themselves. I haven't said they're doomed, although I have said that people who want to use them are on their own. I think the latter is simply an obvious truth, since (a) multiple interpreters have been entirely unexercised by the Python test suite ("if it's not tested, it's broken" rules); (b) Python developers generally pay no attention to them; and (c), as the teensy bit of docs for them imply, they're an "80% solution", but to a problem that's probably more sensitive than most to glitches in the other 20%. I've also said that Mark's thread state PEP explicitly disavowed responsibility for working nicely with multiple interpreters. I said that only because that's what his PEP said . From greg.ewing at canterbury.ac.nz Fri Jun 24 04:11:03 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 24 Jun 2005 14:11:03 +1200 Subject: [Python-Dev] PEP 304 - is anyone really interested? In-Reply-To: <17081.60895.24439.92843@montanaro.dyndns.org> References: <17081.60895.24439.92843@montanaro.dyndns.org> Message-ID: <42BB6BB7.5020109@canterbury.ac.nz> Skip Montanaro wrote: > I wrote PEP 304, "Controlling Generation of Bytecode Files": > > http://www.python.org/peps/pep-0304.html I would like to see some way of having bytecode files put into platform/version dependent subdirectories, which would make it easier e.g. to have Python code shared over a network between machines of different architectures or running different Python versions. But the scheme this PEP proposes (a single session-wide location) is too coarse-grained for what I have in mind. So I would not support this particular PEP without sufficiently large changes that it might as well become a new PEP. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing at canterbury.ac.nz +--------------------------------------+ From bob at redivi.com Fri Jun 24 08:00:03 2005 From: bob at redivi.com (Bob Ippolito) Date: Fri, 24 Jun 2005 02:00:03 -0400 Subject: [Python-Dev] PEP 304 - is anyone really interested? In-Reply-To: <42BB6BB7.5020109@canterbury.ac.nz> References: <17081.60895.24439.92843@montanaro.dyndns.org> <42BB6BB7.5020109@canterbury.ac.nz> Message-ID: On Jun 23, 2005, at 10:11 PM, Greg Ewing wrote: > Skip Montanaro wrote: > >> I wrote PEP 304, "Controlling Generation of Bytecode Files": >> >> http://www.python.org/peps/pep-0304.html >> > > I would like to see some way of having bytecode files put > into platform/version dependent subdirectories, which > would make it easier e.g. to have Python code shared > over a network between machines of different architectures > or running different Python versions. But the scheme this > PEP proposes (a single session-wide location) is too > coarse-grained for what I have in mind. So I would not > support this particular PEP without sufficiently large > changes that it might as well become a new PEP. You could implement that in your sitecustomize.py, unless you have Python interpreters that change platform or version mid-session. I don't see why it needs to be subdirectories, you could make the bytecode cache local or somewhere on the server that you have write access to. -bob From t-meyer at ihug.co.nz Fri Jun 24 11:16:44 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Fri, 24 Jun 2005 21:16:44 +1200 Subject: [Python-Dev] python-dev Summary for 2005-06-01 through 2005-06-15 [draft] Message-ID: You've just read two summaries, but here is another one, as we come back up to speed. If at all possible, it would be great if we could send this out in time to catch people for the bug day (very tight, we know), so if anyone has a chance to check this straight away, that would be great. Please send any amendments to Steve (steven.bethard at gmail.com) as that probably gives us the best chance of getting it out on time. As always, many thanks for the proofreading! ===================== Summary Announcements ===================== --------------------------------- Bug Day: Saturday, June 25th 2005 --------------------------------- AMK is organizing another Python Bug Day this Saturday, June 25th. If you're looking to help out with Python, this is a great way to get started! Contributing Threads: - `Bug day on the 25th? `__ [SJB] ---------------------- FishEye for Python CVS ---------------------- Peter Moore has kindly set up `Fish Eye for the Python CVS repository`_. FishEye is a repository browsing, searching, analysis and monitoring tool, with great features like RSS feeds, Synthetic changesets, Pretty ediffs and SQL like searches. Check it out! .. _Fish Eye for the Python CVS repository: http://fisheye.cenqua.com/viewrep/python/ Contributing Threads: - `FishEye on Python CVS Repository `__ [SJB] -------------------------------- PyPy Sprint: July 1st - 7th 2005 -------------------------------- The next `PyPy`_ sprint is scheduled right after EuroPython 2005 in Gothenborg, Sweden. It will focus mainly on translating PyPy to lower level backends, so as to move away from running PyPy on top of the CPython interpreter. There will be newcomer-friendly introductions, and other topics are possible, so if you have any interest in PyPy, now is the time to help out! .. _PyPy: http://codespeak.net/pypy Contributing Threads: - `Post-EuroPython 2005 PyPy Sprint 1st - 7th July 2005 `__ [SJB] --------------------------------- Reminder: Google's Summer of Code --------------------------------- Just a reminder that the friendly folks at Python have set up a `wiki`_ and a `mailing list`_ for questions about `Google's Summer of Code`_. For specific details on particular projects (e.g. what needs done to complete Python SSL support) participants may also ask questions to the Python-Dev list. .. _wiki: http://wiki.python.org/moin/CodingProjectIdeas .. _mailing list: http://mail.python.org/mailman/listinfo/summerofcode .. _Google's Summer of Code: http://code.google.com/summerofcode.html [SJB] ---------------------- hashlib Review Request ---------------------- Gregory P. Smith noted that he has finished up the hashlib work he started on a few months ago for patches `935454`_ and `1121611`_ (where the final patch is). He feels that the patch is ready, and would like anyone interested to review it; the patch incorporates both OpenSSL hash support and SHA256+SHA512 support in a single module. `The documentation`_ can be accessed separately, for convenience. .. _935454: http://python.org/sf/935454 .. _1121611: http://python.org/sf/1121611 .. _The documentation: http://electricrain.com/greg/hashlib-py25-doc/module-hashlib.html Contributing Threads: - `hashlib - faster md5/sha, adds sha256/512 support `__ [TAM] ========= Summaries ========= ---------------- PEP 343 Progress ---------------- The PEP 343 discussions were mostly concluded. Guido posted the newest version of the PEP to both Python-Dev and Python-List and the discussions that followed were brief and mostly in agreement with the proposal. The PEP 343 syntax was modified slightly to require parentheses if VAR is a comma-separated list of variables. This made the proposal forward-compatible to extending the with-block for multiple resources. In the favored extension, the with-block would take multiple expressions in a manner analogous to import statements:: with EXPR1 [as VAR1], EXPR2 [as VAR2], ...: BLOCK There were also some brief discussions about how with-blocks should behave in the presence of async exceptions like the KeyboardInterrupt generated from a ^C. While it seemed like it would be a nice property for with-blocks to guarantee that the __exit__ methods would still be called in the presence of async exceptions, making such a guarantee proved to be too complicated. Thus the final conclusion, as summarized by Nick Coghlan, was that "with statements won't make any more guarantees about this than try/finally statements do". Contributing Threads: - `PEP 343 rewrite complete `__ - `For review: PEP 343: Anonymous Block Redux and Generator Enhancements `__ - `PEP 343 - next steps `__ - `PEP 343 question `__ [SJB] -------------- Do-While Loops -------------- Raymond Hettinger asked for a "dowhile" loop of the form:: dowhile : which would run once before testing the , and then proceed as a normal while-loop. He was subsequently referred to `PEP 315`_, which proposes a slightly different syntax for a similar purpose. The discussion expanded to not only do-while loops, but also loops with break conditions at locations other than the beginning and the end of a loop. A variety of syntax proposals were suggested, but none seemed compellingly better than the current syntax:: while True: ... if : break ... which supports putting the condition(s) at any location in the loop. .. _PEP 315: http://www.python.org/peps/pep-0315.html Contributing Threads: - `Wishlist: dowhile `__ [SJB] ------------------------------------ Reference Counting in Module Globals ------------------------------------ Both Michael Hudson and Skip Montanaro noticed that Py_INCREFs appeared to be unnecessary when adding an object to a module's globals. Armin Rigo explained that after a module is initialized, the import mechanism makes a "hidden" copy of the module's dict so that the module can be reloaded. This means that objects added as module globals will always have an extra reference count in this hidden dict. However, Armin Rigo agreed with Michael Hudson that this explanation was no longer applicable after an interpreter shutdown. The best conclusion he could draw in this a situation: "it's all quite obscure". Contributing Threads: - `refcounting vs PyModule_AddObject `__ - `[Python-checkins] python/dist/src/Modules _csv.c, 1.37, 1.38 `__ [SJB] ----------------------------------------- Reorganising the standard library (again) ----------------------------------------- The ever-popular topic of reorganising the standard library came up again this fortnight, courtesy of Reinhold Birkenfeld. The questions posed included hierarchy (flat/nested), third party modules, size (batteries included or not), and the standard GUI toolkit. As usual, there was a great deal of discussion, but not a great deal of consensus about any of these (other than that including ElementTree in the standard library would be good), and given the amount of breakage this would involve (and that Guido didn't weigh in at all), it seems unlikely that much will change before Python 3000; although Josiah Carlson indicated that he had a patch that would avoid a lot of breakage. Contributing Threads: - `Thoughts on stdlib evolvement `__ [TAM] -------------------------- First Mac-tel, now Py-tel? -------------------------- Guido mentioned that Intel has a free (as in beer) C `compiler for Linux`_, and that a friend of his (who is involved in its production and marketing) would like to see how it performs with Python. The compiler wasn't news to some of the -dev crowd, though, with Martin v. L?wis pointing out a `bug report on the compiler`_, as well as a `patch`_, and a `message indicating that some people had problems`_ with the resulting interpreter. Martin pointed out that there were some old (2002 and 2004) results indicating that the Intel compiler was slightly faster, but couldn't find any results for the latest version. Michael Hoffman gave summaries of more testing, which gave a 16% speed increase. He felt that, while this was significant, he wasted a lot of time dealing with resulting problems with extension modules, and so doesn't use as much any more. .. _compiler for Linux: http://www.intel.com/software/products/compilers/clin/index.htm .. _bug report on the compiler: http://python.org/sf/1162001 .. _patch: http://python.org/sf/1162023 .. _message indicating that some people had problems: http://mail.python.org/pipermail/python-list/2005- March/270672.html Contributing Threads: - `Compiling Python with Intel compiler? `__ [TAM] ------------------ sys.path Behaviour ------------------ Reinhold Birkenfeld noticed that sys.path's first element is '' in interactive sessions, but the directory containing the script otherwise, and wondered if this was intentional. Guido clarified that he's always liked it this way, so that if you use os.path.join to join it with a script name you don't get a spurious ".\" preprended. The "absolutizing" of sys.path entries, however, is reasonably new; Bob Ippolito pointed out that is also `problematic with regards to path hooks`_. He has a `patch to fix it`_, but hasn't had a chance to commit it; Phillip J. Eby noted that the patch doesn't fix completely fix it, however, and indicated that fixing site.py with respect to `PEP 302`_ will be quite challenging. .. _problematic with regards to path hooks: http://mail.python.org/pipermail/python-dev/2005-April/052885.html .. _patch to fix it: http://python.org/sf/1174614 .. _PEP 302: http://python.org/peps/pep-0302.html Contributing Threads: - `sys.path in interactive session `__ [TAM] ---------------- More on old bugs ---------------- The discussion about what to do with old bugs continued this fortnight. Against the concern about prematurely closing old bugs, there was the suggestion that given that there are such a huge number of open bug reports, and since closed bugs can be reopened, this wasn't such a problem. It was suggested that the act of closing a bug might trigger activity to get it fixed, if necessary. The thread died off before a consensus was reached, unfortunately. Contributing Threads: - `Closing old bugs `__ [TAM] --------------------------------- Improved pseudo-switch statements --------------------------------- Skip Montanaro has been playing around with getting the Python compiler to recognize switch-like statements and generate O(1) code out of them. The rules are that the object being compared ('x') can be any expression, but must be precisely the same in each elif clause, the comparison operator must be "==", and the right-hand-side of the test must evaluate to a simple hashable constant. However, if evaluating 'x' has side-effects, then this would break code. Various people felt that it was unwise to allow 'x' to be any expression; Anthony Baxter suggested that one could allow any local object that didn't define a comparison operator. Contributing Threads: - `Multiple expression eval in compound if statement? `__ [TAM] =============== Skipped Threads =============== - `Adventures with Decimal `__ - `Weekly Python Patch/Bug Summary `__ - `AST manipulation and source code generation `__ - `Vestigial code in threadmodule? `__ - `[Python-checkins] python/dist/src/Lib sre_compile.py, 1.57, 1.58 `__ - `Split MIME headers into multiple lines near a space `__ - `python running in several threads `__ - `problem installing current cvs `__ - `b32encode and NUL bytes `__ - `Example workaround classes for using Unicode with csv module... `__ - `Weekly Python Patch/Bug Summary `__ - `[Python-checkins] python/dist/src/Doc/lib libtokenize.tex, 1.5, 1.6 `__ - `Five patch reviews & patch request `__ - `AIX 4.3, Python 2.4.1 fails in test_exceptions with a core dump `__ - `PEP 342 - Enhanced Iterators `__ - `A bug in pyconfig.h under Linux? `__ - `Dynamic class inheritance && something else `__ - `Weekly Python Patch/Bug Summary `__ From rhamph at gmail.com Sun Jun 26 11:34:07 2005 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 26 Jun 2005 03:34:07 -0600 Subject: [Python-Dev] Adding Python-Native Threads Message-ID: There are some very serious problems with threads today. Events are often proposed as a solution to these problems, but they have their own issues. A detailed analysis of the two options and some solutions (most of which are used in my proposal) can be found in Why Events Are A Bad Idea [0]. I'm going to skip the event aspects and sum up the thread problems here: * Expensive (resident memory, address space, time to create and to switch between) * Finely-grained atomicity (python is coarser than C, but still finer than we need) * Unpredictable (switching between them is not deterministic) * Uninterruptible (no way to kill them, instead you must set a flag and make sure they check it regularly) * Fail silently (if they die with an exception it gets printed to the console but the rest of the program is left uninformed) To resolve these problems I propose adding lightweight cooperative threads to Python. They can be built around a Frame object, suspending and resuming like generators do. That much is quite easy. Avoiding the C stack is a bit harder. However, it can be done if we introduce a new operator for threaded calls (which I refer to as non-atomic calls), which I propose be "func@()". As a bonus this prevents any operation which doesn't use the non-atomic call syntax from triggering a thread switch accidentally. After that you need a way of creating new threads. I propose we use a "sibling func@()" statement (note that the "@()" would be parsed as part of the statement itself, and not as an expression). It would create a new thread and run func in it, but more importantly it would also "prejoin" the new thread to the current function. This means that for the current function to return to its parent, all of the threads it created must first return themselves. It also means that an exception in any of them will be propagated to the parent of the creator, after interrupting all of its siblings (with an Interrupted exception) so that it is permitted to return. Now for an example of how this would work in practice, using an echo server. Note that I am assuming a "con" module (short for concurrent) that would contain standard functions to support these python-native threads. from con import bootstrap, tcplisten, ipv6addr, ipv4addr def main(): main_s = tcplisten@(ipv6addr('::1', 6839), ipv4addr('127.0.0.1', 6839)) while 1: sibling echo@(main_s.accept@()) def echo(s): try: try: while 1: s.write@(s.read@()) except (EOFError, IOError): pass finally: s.close() if __name__ == '__main__': try: bootstrap(main) except KeyboardInterrupt: pass And here is a diagram of the resulting cactus stack, assuming three clients, two of which are reading and the other is writing. bootstrap - main - accept | echo - read | echo - read | echo - write Some notes: * An idle@() function would be used for all thread switches. I/O functions such as read@() and write@() would use it internally, and idle would internally call a function to do poll() on all file descriptors. * idle is the function that will raise Interrupted * It should be illegal to attempt a non-atomic call once your thread is in an interrupted state. This ensures you cannot get "interrupted twice", by having something further up the call stack fail and interrupt you while you are already handling an interruption. * Being a cooperative system, it does not allow you to use multiple CPUs simultaneously. I don't regard this as a significant problem, and in any case there are better ways use multiple CPUs if you need to. References: [0] http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/index.html -- Adam Olsen, aka Rhamphoryncus From ronaldoussoren at mac.com Sun Jun 26 12:17:53 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 26 Jun 2005 12:17:53 +0200 Subject: [Python-Dev] Adding Python-Native Threads In-Reply-To: References: Message-ID: <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com> On 26-jun-2005, at 11:34, Adam Olsen wrote: > > To resolve these problems I propose adding lightweight cooperative > threads to Python. They can be built around a Frame object, > suspending > and resuming like generators do. > > That much is quite easy. Avoiding the C stack is a bit harder. > However, it can be done if we introduce a new operator for threaded > calls (which I refer to as non-atomic calls), which I propose be > "func@()". As a bonus this prevents any operation which doesn't use > the non-atomic call syntax from triggering a thread switch > accidentally. Have a look at stackless python. http://www.stackless.com/ Ronald From florian.proff.schulze at gmx.net Sun Jun 26 12:26:07 2005 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Sun, 26 Jun 2005 12:26:07 +0200 Subject: [Python-Dev] Adding Python-Native Threads References: Message-ID: Also look at greenlets, they are in the py lib http://codespeak.net/py Regards, Florian Schulze From rhamph at gmail.com Sun Jun 26 13:04:17 2005 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 26 Jun 2005 05:04:17 -0600 Subject: [Python-Dev] Adding Python-Native Threads In-Reply-To: <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com> References: <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com> Message-ID: On 6/26/05, Ronald Oussoren wrote: > Have a look at stackless python. http://www.stackless.com/ On 6/26/05, Florian Schulze wrote: > Also look at greenlets, they are in the py lib http://codespeak.net/py While internally Stackless and Greenlets may (or may not) share a lot with my proposed python-native threads, they differ greatly in their intent and the resulting interface they expose. Stackless and Greenlets are both tools for building an architecture that uses threads, while my python-native threads IS that resulting architecture. For example, with Greenlets you would use the .switch() method of a specific greenlet instance to switch to it, and with my python-native threads you would use the global idle() function which would decide for itself which thread to switch to. -- Adam Olsen, aka Rhamphoryncus From rjohnsto at sfu.ca Thu Jun 23 21:05:52 2005 From: rjohnsto at sfu.ca (Robert W. Johnstone) Date: Thu, 23 Jun 2005 12:05:52 -0700 (PDT) Subject: [Python-Dev] Subclassing in 'C' Message-ID: <4330.199.60.7.247.1119553552.squirrel@webmail.fas.sfu.ca> Hello, I'm trying to develop an extension module in C, but am having some difficulties. I thought about posting to the general python mailling list, but since it has to do with the C API I figured this was the correct location. My difficulty lies with subclassing a class provided by another external module. I have access to the structure definition, so I can determine the size fo the tp_basicsize member of the PyTypeObject structure. What I can't seem to determine is value for the tp_base element. This is my current "best guess". PyObject* superclass; superclass = PyObject_GetAttrString( module, "SuperClass" ); if ( !superclass ) return; type.tp_base = superclass->ob_type; Py_DECREF( superclass ); type.tp_new = PyType_GenericNew; if ( PyType_Ready( &type ) < 0 ) return; Unfortunately, I seem to be inheriting from the type object for SuperClass, and not from SuperClass itself. I would greatly appreciate some direction on this issue. Thank-you, ---------------------------------------- Robert Johnstone Simon Fraser University http://www.sfu.ca/~rjohnsto/ From ronaldoussoren at mac.com Sun Jun 26 15:26:35 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 26 Jun 2005 15:26:35 +0200 Subject: [Python-Dev] Subclassing in 'C' In-Reply-To: <4330.199.60.7.247.1119553552.squirrel@webmail.fas.sfu.ca> References: <4330.199.60.7.247.1119553552.squirrel@webmail.fas.sfu.ca> Message-ID: <939D3972-F16D-4E6E-8356-A1A16EC2FC2B@mac.com> On 23-jun-2005, at 21:05, Robert W. Johnstone wrote: > Hello, > > I'm trying to develop an extension module in C, but am having some > difficulties. I thought about posting to the general python mailling > list, but since it has to do with the C API I figured this was the > correct > location. The python list would have been the right location. > > My difficulty lies with subclassing a class provided by another > external > module. I have access to the structure definition, so I can > determine the > size fo the tp_basicsize member of the PyTypeObject structure. What I > can't seem to determine is value for the tp_base element. > > This is my current "best guess". > > PyObject* superclass; > superclass = PyObject_GetAttrString( module, "SuperClass" ); > if ( !superclass ) return; > type.tp_base = superclass->ob_type; type.tp_base = (PyTypeObject*)superclass; > Py_DECREF( superclass ); And don't decref the superclass. You may want to add a typecheck that checks if the superclass is a type. Ronald From gvanrossum at gmail.com Sun Jun 26 16:56:47 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun, 26 Jun 2005 07:56:47 -0700 Subject: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers In-Reply-To: References: <2m1x6z7qu6.fsf@starship.python.net> <20050619010354.72AE.JCARLSON@uci.edu> Message-ID: [me] > > (c) The right place to do the overflow checks is in the API wrappers, > > not in the integer types. [Keith Dart] > That would be the "traditional" method. > > I was trying to keep it an object-oriented API. What should "know" the > overflow condition is the type object itself. It raises OverFlowError any > time this occurs, for any operation, implicitly. I prefer to catch errors > earlier, rather than later. Isn't clear to me at all. I might compute a value using some formula that exceeds 2**32 in some intermediate result but produces an in-range value in the end. That should be acceptable as an argument. I also don't see why one approach is more OO than another; sounds like you have a case of buzzworditis. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Sun Jun 26 17:03:36 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 26 Jun 2005 11:03:36 -0400 Subject: [Python-Dev] Adding Python-Native Threads In-Reply-To: References: <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com> <4CCF4B06-8920-42F6-B62E-B44D0360AEA9@mac.com> Message-ID: <5.1.1.6.0.20050626104826.01fa7f88@mail.telecommunity.com> At 05:04 AM 6/26/2005 -0600, Adam Olsen wrote: >On 6/26/05, Ronald Oussoren wrote: > > Have a look at stackless python. http://www.stackless.com/ > >On 6/26/05, Florian Schulze wrote: > > Also look at greenlets, they are in the py lib http://codespeak.net/py > >While internally Stackless and Greenlets may (or may not) share a lot >with my proposed python-native threads, they differ greatly in their >intent and the resulting interface they expose. Stackless and >Greenlets are both tools for building an architecture that uses >threads, while my python-native threads IS that resulting >architecture. > >For example, with Greenlets you would use the .switch() method of a >specific greenlet instance to switch to it, and with my python-native >threads you would use the global idle() function which would decide >for itself which thread to switch to. See PEP 342 for a competing proposal, that includes a short (<50 lines) co-operative threading example. It uses the 'yield' keyword instead of your '@' proposal, and already has a Sourceforge patch implementing it. It does not propose a specific built-in idler or I/O handler, but such things could be added later. In the short term, I think it's enough that Python be *able* to have such things (without using Stackless or Greenlets), and maybe include an example of a run loop. I don't currently plan to push for a "standard" or "official" run loop implementation in Python 2.5, however, unless all the relevant stakeholders just happen to come to some kind of agreement that makes it happen. Note that while having a universal run loop is a worthy goal, actually implementing one is hard. The experts in this are definitely in the Twisted camp, as they have implemented run loops for numerous platforms to support compatibility w/various GUI frameworks' event loops. To some extent, this means that the run loop needs to be an interface with multiple implementations, and I think that we would need a standard library implementation of PEP 246 in order to support it properly. Anyway, my point is that even deciding on standards for a universal run loop is a fairly big project, one that possibly even deserves its own SIG to put together. By contrast, the implementation of simulated threads and non-blocking function calls was a weekend project. From reinhold-birkenfeld-nospam at wolke7.net Sun Jun 26 18:57:39 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Sun, 26 Jun 2005 18:57:39 +0200 Subject: [Python-Dev] Some RFE for review Message-ID: Hi, while bugs and patches are sometimes tricky to close, RFE can be very easy to decide whether to implement in the first place. So what about working a bit on this front? Here are several RFE reviewed, perhaps some can be closed ("should" is always from submitter's point of view): 1193128: str.translate(None, delchars) should be allowed and only delete delchars from the string. Review: may be a good feature, although string._idmap can be passed as the first parameter too. 1226256: The "path" module by Jason Orendorff should be in the standard library. http://www.jorendorff.com/articles/python/path/ Review: the module is great and seems to have a large user base. On c.l.py there are frequent praises about it. 1216944: urllib(2) should gain a dict mapping HTTP status codes to the correspondig status/error text. Review: I can't see anything speaking against it. 1214675: warnings should get a removefilter() method. An alternative would be to fully document the "filters" attribute to allow direct tinkering with it. Review: As mwh said in a comment, this isn't Java, so the latter may be the way to go. 1205239: Shift operands should be allowed to be negative integers, so e.g. a << -2 is the same as a >> 2. Review: Allowing this would open a source of bugs previously well identifiable. 1152248: In order to read "records" separated by something other than newline, file objects should either support an additional parameter (the separator) to (x)readlines(), or gain an additional method which does this. Review: The former is a no-go, I think, because what is read won't be lines. The latter is further complicating the file interface, so I would follow the principle that not every 3-line function should be builtin. 1110010: A function "attrmap" should be introduced which is used as follows: attrmap(x)['att'] == getattr(x, 'att') The submitter mentions the use case of new-style classes without a __dict__ used at the right of %-style string interpolation. Review: I don't know whether this is worth it. 1052098: An environment variable should be supported to set the default encoding. Review: If one wants this for a single application, he can still implement it himself. 985094: getattr should support a callable as the second argument, used as follows: getattr(obj, func) == func(obj) Review: Highly unintuitive to me. That's all for today; sorry if it was too much ;) Reinhold -- Mail address is perfectly valid! From pje at telecommunity.com Sun Jun 26 20:17:15 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 26 Jun 2005 14:17:15 -0400 Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review) In-Reply-To: Message-ID: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> At 06:57 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote: >1226256: >The "path" module by Jason Orendorff should be in the standard library. >http://www.jorendorff.com/articles/python/path/ >Review: the module is great and seems to have a large user base. On c.l.py >there are frequent praises about it. I would note that there are some things in the interface that should be cleaned up before it becomes a stdlib module. It has many ways to do the same thing, and many of its property and method names are confusing because they either do the same thing as a standard function, but have a different name (like the 'parent' property that is os.path.dirname in disguise), or they have the same name as a standard function but do something different (like the 'listdir()' method that returns full paths rather than just filenames). I'm also not keen on the fact that it makes certain things properties whose value can change over time; i.e. ctime/mtime/atime and size really shouldn't be properties, but rather methods. I'm also not sure how I feel about all the read/write methods that hide the use of file objects; these seem like they should remain file object methods, especially since PEP 343 will allow easy closing with something like: with closing(some_path.open('w')) as f: f.write(data) Granted, this is more verbose than: some_path.write_bytes(data) But brevity isn't always everything. If these methods are kept I would suggest using different names, like "set_bytes()", "set_text()", and "set_lines()", because "write" sounds like something you do on an ongoing basis to a stream, while these methods just replace the file's entire contents. Aside from all these concerns, I'm +1 on adding the module. Here's my list of suggested changes: * path.joinpath(*args) -> path.subpath(*args) * path.listdir() -> path.subpaths() * path.splitall() -> path.parts() * path.parent -> path.dirname (and drop dirname() method) * path.name -> path.filename (and drop basename() method) * path.namebase -> path.filebase (maybe something more descriptive?) * path.atime/mtime/ctime -> path.atime(), path.mtime(), path.ctime() * path.size -> path.filesize() * drop getcwd(); it makes no sense on a path instance * add a samepath() method that compares the absolute, case and path-normalized versions of two paths, and a samerealpath() method that does the same but with symlinks resolved. And, assuming these file-content methods are kept: * path.bytes() -> path.get_file_bytes() * path.write_bytes() -> path.set_file_bytes() and path.append_file_bytes() * path.text() -> path.get_file_text() * path.write_text() -> path.set_file_text() and path.append_file_text() * path.lines() -> path.get_file_lines() * path.write_lines() -> path.set_file_lines() and path.append_file_lines() From oren.tirosh at gmail.com Sun Jun 26 20:52:56 2005 From: oren.tirosh at gmail.com (Oren Tirosh) Date: Sun, 26 Jun 2005 21:52:56 +0300 Subject: [Python-Dev] Adding Python-Native Threads In-Reply-To: References: Message-ID: <7168d65a05062611525e5868b5@mail.gmail.com> On 6/26/05, Adam Olsen wrote: ... > To resolve these problems I propose adding lightweight cooperative > threads to Python. Speaking of lightweight cooperative threads - has anyone recently tried to build Python with the pth option? It doesn't quite work out of the box. How much maintenance would be required to make it work again? Oren From reinhold-birkenfeld-nospam at wolke7.net Sun Jun 26 21:00:21 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Sun, 26 Jun 2005 21:00:21 +0200 Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review) In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> Message-ID: Phillip J. Eby wrote: > At 06:57 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote: >>1226256: >>The "path" module by Jason Orendorff should be in the standard library. >>http://www.jorendorff.com/articles/python/path/ >>Review: the module is great and seems to have a large user base. On c.l.py >>there are frequent praises about it. [...] > Aside from all these concerns, I'm +1 on adding the module. > > Here's my list of suggested changes: [...] I agree with your changes list. One more issue is open: the one of naming. As "path" is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? Reinhold -- Mail address is perfectly valid! From hoffman at ebi.ac.uk Sun Jun 26 21:19:29 2005 From: hoffman at ebi.ac.uk (Michael Hoffman) Date: Sun, 26 Jun 2005 20:19:29 +0100 Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review) In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> Message-ID: On Sun, 26 Jun 2005, Phillip J. Eby wrote: > * drop getcwd(); it makes no sense on a path instance Personally I use path.getcwd() as a class method all the time. It makes as much sense as fromkeys() does on a dict instance, which is technically possible but non-sensical. > And, assuming these file-content methods are kept: > > * path.bytes() -> path.get_file_bytes() > * path.write_bytes() -> path.set_file_bytes() and path.append_file_bytes() > * path.text() -> path.get_file_text() > * path.write_text() -> path.set_file_text() and path.append_file_text() > * path.lines() -> path.get_file_lines() > * path.write_lines() -> path.set_file_lines() and path.append_file_lines() I don't know how often these are used. I don't use them myself. I am mainly interested in this module so that I don't have to use os.path anymore. Reinhold Birkenfeld wrote: > One more issue is open: the one of naming. As "path" is already the > name of a module, what would the new object be called to avoid > confusion? pathobj? objpath? Path? I would argue for Path. It fits with the recent cases of: from sets import Set from decimal import Decimal -- Michael Hoffman European Bioinformatics Institute From skip at pobox.com Sun Jun 26 21:31:21 2005 From: skip at pobox.com (Skip Montanaro) Date: Sun, 26 Jun 2005 14:31:21 -0500 Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review) In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> Message-ID: <17087.649.730816.972325@montanaro.dyndns.org> Phillip> It has many ways to do the same thing, and many of its property Phillip> and method names are confusing because they either do the same Phillip> thing as a standard function, but have a different name (like Phillip> the 'parent' property that is os.path.dirname in disguise), or Phillip> they have the same name as a standard function but do something Phillip> different (like the 'listdir()' method that returns full paths Phillip> rather than just filenames). To the extent that the path module tries to provide a uniform abstraction that's not saddled with a particular way of doing things (e.g., the Unix way or the Windows way), I don't think this is necessarily a bad thing. Skip From Jack.Jansen at cwi.nl Sun Jun 26 23:48:54 2005 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Sun, 26 Jun 2005 23:48:54 +0200 Subject: [Python-Dev] [Python-checkins] python/dist/src/Tools/bgen/bgen bgenGenerator.py, 1.17, 1.18 bgenObjectDefinition.py, 1.29, 1.30 bgenType.py, 1.15, 1.16 bgenVariable.py, 1.6, 1.7 scantools.py, 1.37, 1.38 In-Reply-To: References: Message-ID: <74A681D5-F0EE-4E08-A6FC-264158C6A5E8@cwi.nl> On 24-jun-2005, at 21:46, tim_one at users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18095/Tools/ > bgen/bgen > > Modified Files: > bgenGenerator.py bgenObjectDefinition.py bgenType.py > bgenVariable.py scantools.py > Log Message: > Normalize whitespace to avoid offending Bug Day volunteers. Argh, sorry... I thought I had all my machines educated to do tab expansion for Python, but apparently not... -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From walter at livinglogic.de Mon Jun 27 00:22:46 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?D=F6rwald_Walter?=) Date: Mon, 27 Jun 2005 00:22:46 +0200 Subject: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review) In-Reply-To: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> References: <5.1.1.6.0.20050626133610.03652688@mail.telecommunity.com> Message-ID: <1BCEF0D1-5E33-4485-80F6-010E8C7EC2AC@livinglogic.de> Phillip J. Eby wrote: > [...] > I'm also not keen on the fact that it makes certain things > properties whose value can change over time; i.e. ctime/mtime/atime > and > size really shouldn't be properties, but rather methods. I think ctime, mtime and atime should be (or return) datetime.datetime objects instead of integer timestamps. Bye, Walter D?rwald From tim.peters at gmail.com Mon Jun 27 00:59:35 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 26 Jun 2005 18:59:35 -0400 Subject: [Python-Dev] [Python-checkins] python/dist/src/Tools/bgen/bgen bgenGenerator.py, 1.17, 1.18 bgenObjectDefinition.py, 1.29, 1.30 bgenType.py, 1.15, 1.16 bgenVariable.py, 1.6, 1.7 scantools.py, 1.37, 1.38 In-Reply-To: <74A681D5-F0EE-4E08-A6FC-264158C6A5E8@cwi.nl> References: <74A681D5-F0EE-4E08-A6FC-264158C6A5E8@cwi.nl> Message-ID: <1f7befae05062615594fb89b7d@mail.gmail.com> [tim_one at users.sourceforge.net] >> Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen >> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18095/Tools/ >> bgen/bgen >> >> Modified Files: >> bgenGenerator.py bgenObjectDefinition.py bgenType.py >> bgenVariable.py scantools.py >> Log Message: >> Normalize whitespace to avoid offending Bug Day volunteers. [Jack Jansen] > Argh, sorry... I thought I had all my machines educated to do tab > expansion for Python, but apparently not... You probably do. I run Tools/scripts/reindent.py over the whole source tree from time to time, and check in whatever it changes. Converting tabs to 4-space indents is one of the things it does, but not the only thing. Most often these days it finds trailing whitespace at the end of a line, and snips it off. Once in a great while, that catches code that *depends* on trailing whitespace (typically in a long string literal), which is viciously obscure (nobody looking at the file can see the dependence, and editors may remove it). It also removes empty lines at the end of a .py file, ensures that the final line ends with a newline, and changes "oddball" indent levels to 4-space indents (curiously, that one triggers 2nd-most often these days). In any case, it's not like this is work -- no need to feel apologetic. From tim.peters at gmail.com Mon Jun 27 01:15:47 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 26 Jun 2005 19:15:47 -0400 Subject: [Python-Dev] [Python-checkins] python/dist/src/Lib Cookie.py, 1.17, 1.18 In-Reply-To: References: Message-ID: <1f7befae050626161574adbe07@mail.gmail.com> [birkenfeld at users.sourceforge.net] > Update of /cvsroot/python/python/dist/src/Lib > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4891/Lib > > Modified Files: > Cookie.py > Log Message: > bug [ 1108948 ] Cookie.py produces invalid code > > > > Index: Cookie.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/Cookie.py,v > retrieving revision 1.17 > retrieving revision 1.18 > diff -u -d -r1.17 -r1.18 > --- Cookie.py 20 Oct 2003 14:01:49 -0000 1.17 > +++ Cookie.py 26 Jun 2005 21:02:49 -0000 1.18 > @@ -470,9 +470,9 @@ > def js_output(self, attrs=None): > # Print javascript > return """ > - > """ % ( self.OutputString(attrs), ) I assume this accounts for the current failure of test_cookie: test_cookie test test_cookie produced unexpected output: ********************************************************************** *** mismatch between line 19 of expected output and line 19 of actual output: -