From scav at blueyonder.co.uk Tue Mar 1 00:05:15 2005 From: scav at blueyonder.co.uk (Peter Harris) Date: Mon Feb 28 23:59:57 2005 Subject: [Python-Dev] Re: [Python Dev] PEP 309 In-Reply-To: <20050228110042.1852D1E4012@bag.python.org> References: <20050228110042.1852D1E4012@bag.python.org> Message-ID: <4223A3AB.60606@blueyonder.co.uk> >>Overall, I have no major objections to the PEP or the patch. Before it >>goes in on auto-pilot, it would be darned nice if the proponents said >>that they've found it helpful in real code and that they are satisfied >>with the timings. >> >> > >I guess "darned nice" is the best you can hope for. Not sure if Peter >Harris is still around. > >Regards, >Martin > > Yes, I'm still lurking, slightly aghast that my little PEP is getting such ferocious scrutiny. I would have like some of that in time for it to go into 2.4, but I suppose you should be careful what you wish for. I'll answer a few points from this thread, in no particular order: My original desire was a built-in, but it was suggested that the first step would be a Python implementation in the standard library to try it out. That was the basis for the PEP, and in fact a C implementation would have been beyond my expertise. However, I sympathise with anyone who feels unhappy about a new module just for what amounts to one function. I'd be happy to go back to the built-in, now someone cleverer than I am has written one. Sorry I can't rememeber your name, whoever you are. I'm having trouble with my e-mails. I was never too bothered about efficiency, and still am not. For me it was always primarily a way to save typing or build call-back functions on the fly. The discussion about using it to make instancemethods and classmethods -- way over my head! I would count that as something weird enough to be worth spelling out in "plain Python", in my code anyway. The PEP was scattered over a few patches because I wasn't too sure how to go about it, so there was my Python module, the C implementation, the unit tests and the docs all in separate patches. 3/4 of that was my fault - sorry! Once the PEP had been accepted, I didn't like to mess with it, which is why I went quiet for a while. It's gone past the point where I can personally contribute much, so I'd just like to say thanks and I look forward to the day when I can just use it. Peter Harris From steven.bethard at gmail.com Tue Mar 1 00:18:30 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue Mar 1 00:18:34 2005 Subject: [Python-Dev] Re: [Python Dev] PEP 309 In-Reply-To: <4223A3AB.60606@blueyonder.co.uk> References: <20050228110042.1852D1E4012@bag.python.org> <4223A3AB.60606@blueyonder.co.uk> Message-ID: Peter Harris wrote: > However, I sympathise with anyone who feels unhappy about a new module > just for what amounts to one function. Well, since it seems that the emphasis in Python development is now moving more towards expanding the standard library, a module that has only one function now might grow more functions in the not so distant future, so I have to say that I'm on the side of not being unhappy with the added module. (I'm also secretly hoping that map, filter, reduce, etc. will be moved to the functional module in the future, maybe in Python 3.0. But that's probably just my love for list comprehensions and generator expressions speaking.) ;-) Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From python at rcn.com Tue Mar 1 00:17:02 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue Mar 1 00:21:14 2005 Subject: [Python-Dev] Re: [Python Dev] PEP 309 In-Reply-To: <4223A3AB.60606@blueyonder.co.uk> Message-ID: <000001c51deb$9d777960$8401a044@oemcomputer> [Peter Harris] > I look forward to the day when I can just use it. You PEP is marked as final. The code has been checked in to CVS and will be in Py2.5. Congrats, Raymond Hettinger From martin at v.loewis.de Tue Mar 1 00:51:42 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Mar 1 00:51:45 2005 Subject: [Python-Dev] Re: PEP 754 In-Reply-To: <915D2D65A9986440A277AC5C98AA466F978B71@groamrexm02.amer.pfizer.com> References: <915D2D65A9986440A277AC5C98AA466F978B71@groamrexm02.amer.pfizer.com> Message-ID: <4223AE8E.7030408@v.loewis.de> Warnes, Gregory R wrote: > What else needs to be done to allow fpconst to go into the Python library? See PEP 1. First, the PEP must be Accepted. Regards, Martin From kbk at shore.net Tue Mar 1 05:11:28 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Mar 1 05:11:41 2005 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200503010411.j214BShi009050@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 303 open ( -5) / 2764 closed ( +9) / 3067 total ( +4) Bugs : 849 open (+11) / 4837 closed ( +3) / 5686 total (+14) RFE : 169 open ( +1) / 148 closed ( +0) / 317 total ( +1) New / Reopened Patches ______________________ New fpconst module (2005-02-24) http://python.org/sf/1151323 opened by Gregory Warnes PyXxx_Check() speed-up (2005-02-27) http://python.org/sf/1153056 opened by Armin Rigo os.remove error on Solaris (2005-02-28) http://python.org/sf/1153417 opened by Richard Philips Patches Closed ______________ rlcompleter does not expand on [ ] (2002-04-22) http://python.org/sf/547176 closed by mwh Non-blocking Socket Server (2004-05-02) http://python.org/sf/946207 closed by loewis add urldecode() method to urllib (2003-05-21) http://python.org/sf/740827 closed by loewis adding bool support to xdrlib.py (2004-10-18) http://python.org/sf/1049151 closed by loewis PEP309 Partial implementation (2004-04-25) http://python.org/sf/941881 closed by rhettinger sanity check for readline remove/replace (2004-12-31) http://python.org/sf/1093585 closed by loewis PEP 309 LaTeX documentation (2004-04-07) http://python.org/sf/931007 closed by rhettinger Build Patch #941881 (PEP 309 in C) on windows (2004-08-10) http://python.org/sf/1006948 closed by rhettinger PEP 309 unit tests (2004-04-07) http://python.org/sf/931010 closed by rhettinger New / Reopened Bugs ___________________ hotshot.runctx: builtins missing (2005-02-23) http://python.org/sf/1149798 opened by Jurjen N.E. Bos macostools.mkdirs: not thread-safe (2005-02-23) http://python.org/sf/1149804 opened by Jurjen N.E. Bos (XMLRPC) multitude of sockets ending up in TIME_WAIT (2005-02-25) http://python.org/sf/1151968 opened by Jonas Wid?n Dict docstring error Python-2.3.5 (2005-02-26) http://python.org/sf/1152424 opened by Colin J. Williams urllib2 dont respect debuglevel in httplib (2005-02-27) http://python.org/sf/1152723 opened by abbatini Default class args get clobbered by prior instances. (2005-02-26) CLOSED http://python.org/sf/1152726 opened by Simon Drabble curses.textpad raises error (2005-02-27) http://python.org/sf/1152762 opened by John McPherson Setting socket timeout crashes SSL (2005-02-27) http://python.org/sf/1153016 opened by pristine777 http_error_302() crashes with 'HTTP/1.1 400 Bad Request (2005-02-27) http://python.org/sf/1153027 opened by pristine777 PyXxx_Check(x) trusts x->ob_type->tp_mro (2005-02-27) http://python.org/sf/1153075 opened by Armin Rigo reflected operator not used when operands have the same type (2005-02-27) http://python.org/sf/1153163 opened by HughSW reflected operator not used when operands have the same type (2005-02-27) CLOSED http://python.org/sf/1153171 opened by HughSW string interpolation breaks with %d and large float (2005-02-28) http://python.org/sf/1153226 opened by Stephen Thorne eval does not bind variables in lambda bodies correctly (2005-02-28) http://python.org/sf/1153622 opened by Mattias Engdeg?rd String interpolation needs PEP 237 update (2005-02-28) http://python.org/sf/1153769 opened by Richard Brodie Bugs Closed ___________ Python24.dll crashes, EXAMPLE ATTACHED (2005-02-12) http://python.org/sf/1121201 closed by complex Default class args get clobbered by prior instances. (2005-02-26) http://python.org/sf/1152726 closed by rhettinger reflected operator not used when operands have the same type (2005-02-27) http://python.org/sf/1153171 closed by rhettinger New / Reopened RFE __________________ Enhance file.readlines by making line separator selectable (2005-02-26) http://python.org/sf/1152248 opened by Nick Coghlan From ncoghlan at iinet.net.au Tue Mar 1 11:16:58 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 1 11:17:02 2005 Subject: [Python-Dev] Re: [Python Dev] PEP 309 In-Reply-To: References: <20050228110042.1852D1E4012@bag.python.org> <4223A3AB.60606@blueyonder.co.uk> Message-ID: <4224411A.6060302@iinet.net.au> Steven Bethard wrote: > (I'm also secretly hoping that map, filter, > reduce, etc. will be moved to the functional module in the future, > maybe in Python 3.0. But that's probably just my love for list > comprehensions and generator expressions speaking.) ;-) Given Guido's stated desire to get rid of those three, but also given the fact that sometimes they're just plain clearer than the equivalent list comp (e.g. performing a type conversion on an entire list), having the functional module as a place to eventually put them seemed like a fine idea to me. I'm actually half-tempted to suggest that those three functions should be aliased in the functional module for 2.5 (in addition to being in builtins - ala the contents of the exceptions module). I also agree with your other point - with a functional module in place, it becomes more feasible to give the functional programming folks a few extra tools without impacting too badly on those people that *aren't* interested in FP related stuff. Cheers, Nick. Maybe we should open a book on the next method to make it into functional. Compose, perhaps? -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From steve at holdenweb.com Tue Mar 1 12:14:59 2005 From: steve at holdenweb.com (Steve Holden) Date: Tue Mar 1 12:15:14 2005 Subject: [Python-Dev] Re: python-dev Summary for 2005-01-16 through 2005-01-31 In-Reply-To: <1109649779.890523.249560@o13g2000cwo.googlegroups.com> References: <1109649779.890523.249560@o13g2000cwo.googlegroups.com> Message-ID: <42244EB3.5070406@holdenweb.com> Michele Simionato wrote [on c.l.py]: > Brett Cannon: [... python-dev summary ... boilerplate change ...] > > +1 for this idea. The summary looks much better now :) > Keep the good work going, > Sorry, but i have to disagree. I hope you won't take this reply personally, Michele, since it's directed to all c.l.py readers, as well as (particularly) at Python users who [unlike you] are mostly take and rather less give. Although this is inherently the nature of open source, in certain cases this can be taken too far. I have a long history of doing things, and an equally long history giving up doing them. This stems from a personal belief that organic growth (IMHO the healthiest type) will only be engendered by variety. I was the Chairman of the Sun UK User Group once. When I was elected I said I would serve for two years, and when I resigned after two years many people said to me "Steve, please reconsider your decision". I observed, perhaps somewhat cynically, that most of the people who said this were motivated by the wish to avoid the pain of locating and electing a new chairman. Guess what ... when I refused to reconsider they found a new chairman, who was at least as good as me (I thought he was better), and life carried on. If you were to ask a member of the Sun UK User Group now the name of their second chairman I'd be very surprised if they had any idea who the hell Steve Holden was. (Historical note: the first chairman was Chris Brown, and nobody will remember him either). Now, the reason for this specific rant is this: I can tell a cry for help when I see one. Brett has done a magnificent job of providing python-dev summaries since Andrew decided he'd had enough, and he is to be congratulated for it. I managed to offload another bunch of work on him (moderation of various troublesome PyCon mailing lists), but at least I was able to recompense him by letting him into PyCon for nothing. I can say this because I am confident that nobody will even think of suggesting that Brett's contribution to the Python community doesn't entitle him to a free place at PyCon. I suspect most readers of this list would feel the same about Guido (I certainly hope so, because he too is a free-loader this year :-). I would actually like a free place at PyCon to represent recognition of significant contributions to the Python community, but there is a conflict here with another of my goals (raising funds for the PSF). But frankly, I think it's time someone else stood up and said "Brett, you've done a magnificent job. Hesitant though I am about replacing you, I would like to volunteer for the task, because only when you are free from the burden of writing the python-dev summaries will we see what else you are capable of". Since I am at best an intermittent reader of python-dev I can say this without fear of having to stand up myself. Oops, I'm rambling. I guess what I'm trying to say boils down to "Ask not what the Python community can do for you ...", and anyone who can't provide the remainder of the analogy is too young to consider themselves a victim of this post, and can claim a free ticket until they are old enough ti understand what history is. I like to think that although I don't make frequent checkins to the code base I do *something* to engender the Python community spirit (though I don't consider my own interpretation of that spirit to uniquely define it), and I'm damned sure Brett has done his share. It would be great if just a *few* more people who are currently consuming the fruits of our labors would stop sitting on the sidelines shouting "great job!" and roll their sleeves up. I hope I'll be able to put these remarks in a corporate context for PyCon - which astute readers will have noticed will be my last PyCon as chairman. I am happy to say that Andrew Kuchling has finally admitted his lust for power and confirmed that he is prepared to act as chairman for 2006, and I wish him well. More later one-more-thing-given-up-ly y'rs - steve From ncoghlan at iinet.net.au Tue Mar 1 14:12:01 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 1 14:12:05 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) Message-ID: <42246A21.4030104@iinet.net.au> A recent question on c.l.p pointed out that the 2.4 Decimal implementation raises TypeError directly for operator arguments it doesn't understand, instead of returning NotImplemented. Obviously, this creates problems for anyone trying to define a class that 'plays nicely' with Decimal (but does not inherit from Decimal), since their __rop__ methods never get called - Decimal's TypeError gets in the way. I had another look at the PEP and didn't see anything on this topic, and I can't recall any discussion on the topic either on this list, or directly with Raymond and Facundo (Google doesn't have anything, either). My current thoughts run as follows: 1. The standard binary operations (that is, '/', '*', '+', '-', '%', '**', '//', divmod(), cmp()) should be returning NotImplemented instead of raising TypeError directly. 2. The method-only operations should continue to raise TypeErrors 3. Invocations via Context methods should continue to raise TypeErrors I was going to say this couldn't be fixed for Python 2.4 because it was a semantic change. But I modified my local copy to work this way, and the full Decimal test suite passed without any issue. For syntax-based access, the bad call still results in a TypeError from the PyNumber_* machinery, and there isn't anything in the docs that says anything about whether a bad operand in a direct call to a special method results in NotImplemented being returned or TypeError being raised. So, 2 questions: 1. Should Python 2.5's C implementation of Decimal follow the standard numeric coercion rules as described above? 2. Is it reasonable to class this as a bugfix and fix the Python version for 2.4.2? (2.4.1's a bit too soon for my liking) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From python at rcn.com Tue Mar 1 14:22:51 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue Mar 1 14:27:06 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <42246A21.4030104@iinet.net.au> Message-ID: <001101c51e61$c57b99c0$ef26c797@oemcomputer> > A recent question on c.l.p pointed out that the 2.4 Decimal implementation > raises TypeError directly for operator arguments it doesn't understand, > instead > of returning NotImplemented. > > Obviously, this creates problems for anyone trying to define a class that > 'plays > nicely' with Decimal (but does not inherit from Decimal), since their > __rop__ > methods never get called - Decimal's TypeError gets in the way. Try to address this in a larger context than decimal. The same sort of logic is present in sets.py and in datetime objects. Raymond From ncoghlan at iinet.net.au Tue Mar 1 14:45:43 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 1 14:45:46 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <001101c51e61$c57b99c0$ef26c797@oemcomputer> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> Message-ID: <42247207.4050402@iinet.net.au> Raymond Hettinger wrote: > Try to address this in a larger context than decimal. The same sort of > logic is present in sets.py and in datetime objects. Interesting. In that case, my other suggestion was to have raising NotImplementedError from a special method be the equivalent of returning NotImplemented (which makes life much easier for a class like Decimal which has an internal method for doing the type conversion). Then a class ("NotImplementedTypeError"?) that inherited from both NotImplementedError and TypeError could be used to fix the problem in a fairly backward compatible way - the PyNumber machinery would see the NotImplemented error, and try the other available operators before falling back to generating its own TypeError, while direct calls with invalid arguments would still be raising a subclass of TypeError, so existing code to catch TypeError from direct calls would continue to function. I didn't suggest this initially, since I didn't realise Decimal wasn't the only class with the problem, and I'm sure messing with PyNumber_* isn't possible for the 2.4 series :) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From reinhold-birkenfeld-nospam at wolke7.net Tue Mar 1 19:57:04 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Tue Mar 1 20:04:26 2005 Subject: [Python-Dev] Legacy bugs on SF Message-ID: Hello, still in 2004, this comment was added to old bugs with groups Python2.*: """ Please, could you verify if this problem persists in Python 2.3.4 or 2.4? If yes, in which version? Can you provide a test case? If the problem is solved, from which version? Note that if you fail to answer in one month, I'll close this bug as "Won't fix". """ The month is over now, so what to do with them? Note that there are other very old bugs, e.g. in the "Platform-specific" group, which I think may be closed too. Reinhold -- Mail address is perfectly valid! From FBatista at uniFON.com.ar Tue Mar 1 21:06:04 2005 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Mar 1 21:10:41 2005 Subject: [Python-Dev] Legacy bugs on SF Message-ID: [Reinhold Birkenfeld] #- The month is over now, so what to do with them? The "month" is like a minimum time limit. I got separated and moved to my parent house. I still don't have internet connection, so I have this work a bit overdue. But I'll continue it, and eventually will reach the bugs where I alerted, and will close it, and will review the others (be aware that I'm not alerting every bug, I try to check every one when possible to see if it's still a bug, if was fixed, if the context changed to it has no sense any more, etc...). #- Note that there are other very old bugs, e.g. in the #- "Platform-specific" #- group, which I think may be closed too. Ok. . Facundo Bit?cora De Vuelo: http://www.taniquetil.com.ar/plog PyAr - Python Argentina: http://pyar.decode.com.ar/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA. La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050301/6a70eeb4/attachment.htm From nas at arctrix.com Tue Mar 1 23:55:50 2005 From: nas at arctrix.com (Neil Schemenauer) Date: Tue Mar 1 23:55:54 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <42247207.4050402@iinet.net.au> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> Message-ID: <20050301225550.GB11698@mems-exchange.org> On Tue, Mar 01, 2005 at 11:45:43PM +1000, Nick Coghlan wrote: > Interesting. In that case, my other suggestion was to have raising > NotImplementedError from a special method be the equivalent of returning > NotImplemented (which makes life much easier for a class like Decimal which > has an internal method for doing the type conversion). NotImplementedError has nothing to do with the NotImplemented singleton. It's unfortunate about the naming. IMO, Decimal should be returning NotImplemented instead of raising TypeError. That could be considered a bug but I'd file it under new functionality for the purposes of backporting (i.e. fix it in 2.5 only). Neil From martin at v.loewis.de Wed Mar 2 00:18:53 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 2 00:18:56 2005 Subject: [Python-Dev] Re: python-dev Summary for 2005-01-16 through 2005-01-31 In-Reply-To: <42244EB3.5070406@holdenweb.com> References: <1109649779.890523.249560@o13g2000cwo.googlegroups.com> <42244EB3.5070406@holdenweb.com> Message-ID: <4224F85D.2070906@v.loewis.de> Steve Holden wrote: > Now, the reason for this specific rant is this: I can tell a cry for > help when I see one. Brett has done a magnificent job of providing > python-dev summaries since Andrew decided he'd had enough, and he is to > be congratulated for it. I managed to offload another bunch of work on > him (moderation of various troublesome PyCon mailing lists), but at > least I was able to recompense him by letting him into PyCon for nothing. The more I participate, the more I can relate to Eric Raymond's notion of a "gift society". Volunteers give their contributions to the community just because they want to, and they may get recognition in return. But because these are gifts, you can just stop giving them away at any time, and nobody should feel bad about doing so. The community only is only entitled to the contributor saying so - finding somebody else to step in is indeed optional. I don't actually know whether Brett would prefer to hand over the python-dev summaries to somebody else, but if he wants to, he could just *stop* publishing them, with nobody taking over, and my appreciation of this contribution would not change at all. Continuing it until a new volunteer steps forward is, as I said, truly optional. I still recall when Tim Peters reappeared in the net (even though I haven't been around long enough to remember him disappear), and I know I didn't understand all the cheering and praising (I do now, of course). So their isn't anything wrong with taking a vacation from a project for some time, not even if the vacation takes a few years :-) Enough ranting. Regards, Martin From bac at OCF.Berkeley.EDU Wed Mar 2 04:52:06 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Mar 2 04:52:14 2005 Subject: [Python-Dev] OK, time to retire (was: Re: python-dev Summary for 2005-01-16 through 2005-01-31) In-Reply-To: <42244EB3.5070406@holdenweb.com> References: <1109649779.890523.249560@o13g2000cwo.googlegroups.com> <42244EB3.5070406@holdenweb.com> Message-ID: <42253866.6030002@ocf.berkeley.edu> Steve Holden wrote: > Michele Simionato wrote [on c.l.py]: > >> Brett Cannon: > > [... python-dev summary ... boilerplate change ...] > >> >> +1 for this idea. The summary looks much better now :) >> Keep the good work going, >> > Sorry, but i have to disagree. I hope you won't take this reply > personally, Michele, since it's directed to all c.l.py readers, as well > as (particularly) at Python users who [unlike you] are mostly take and > rather less give. Although this is inherently the nature of open source, > in certain cases this can be taken too far. > [SNIP] > Now, the reason for this specific rant is this: I can tell a cry for > help when I see one. Brett has done a magnificent job of providing > python-dev summaries since Andrew decided he'd had enough, and he is to > be congratulated for it. I managed to offload another bunch of work on > him (moderation of various troublesome PyCon mailing lists), but at > least I was able to recompense him by letting him into PyCon for nothing. > [SNIP] > But frankly, I think it's time someone else stood up and said "Brett, > you've done a magnificent job. Hesitant though I am about replacing you, > I would like to volunteer for the task, because only when you are free > from the burden of writing the python-dev summaries will we see what > else you are capable of". Since I am at best an intermittent reader of > python-dev I can say this without fear of having to stand up myself. > [SNIP] [I am going to use this to reply to both Steve and Martin] As Steve mentioned above, he can spot a cry for help when he sees one. I think the problem is that I am a total sucker when it comes to the Python community and python-dev. Anyone who has been on the python-dev list for as long as I have been a participant has most likely seen my almost yearly "thank you" emails I send the list (which there will probably be another one of once I choose where I am going to pursue my doctorate; I have acceptances but I am still waiting to here back from 9 more schools). Usually it is just me gushing to python-dev, thanking the list for how Python has gotten me where I am today. And that statement is completely sincere; python-dev has sculpted me into the programmer that I am (does this mean I can blame python-dev for my own buggy code? =). And for that I will be eternally grateful to all of the wonderful people I have gotten to work with and know on this list. It has also made me want to help people to get involved on python-dev in hopes others would benefit from python-dev the same way I have. Granted, python-dev tends not to attract people like I was when I started getting involved (a philosophy degree and 4 CS courses does not equal a good programmer by default =), but I have always hoped that through my efforts some other people could come to enjoy hacking on Python, learn some things, and advance the language. But I think the big problem is that the Summaries have become a "gift" in the truest sense of the word. I lost all personal benefit from the Summaries over a year ago. Initially I learned a ton from all of the reading I was doing and the research required to understand what the heck people were talking about. But I have graduated from "The School of Hard Knocks". At this point I do the Summaries entirely altruistically, giving back what I can to the community in a way that I know benefits many people which happens to have zero benefit to me now. The Summaries consume what little free time I do have for Python which is unfortunate. I have always hoped I would get to the point in my programming abilities that I would be a larger asset to python-dev as a programmer than as a writer. I would like to think I have reached that point finally after my over two and a half years on the list (I can't believe I first posted to the list on June 17, 2002!). So, to make sure I don't squander what time I do have for Python waiting for a possible replacement that might never come, I have decided that I am going to stop doing the python-dev Summaries after PyCon; the Summary covering the last half of March 2005 will be it for me. Hopefully I will be more valuable as an active participant on python-dev again instead of as a passive listener who just happens to chime in on occasion and squash a simple bug when I am procrastinating from doing my homework. This has been a long time coming and I needed a swift kick in the ass to finally get me to stop. I thank you, Steve, for giving me that kick like the English gentleman you are. =) -Brett From anthony at interlink.com.au Wed Mar 2 05:03:18 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 2 05:03:29 2005 Subject: [Python-Dev] 2.4.1c1 March 10th, 2.4.1 March 17th Message-ID: <200503021503.18654.anthony@interlink.com.au> Ok - Fred and Martin and I will be cutting 2.4.1c1 on March 10th, and (assuming no problems) 2.4.1 on March 17th. Apologies for the delay - my time's been all consumed lately with doing lighting and sound design for a show. (Any Melbournites who're interested, we're putting on Michael Palin's play "The Weekend" from next Thursday - see www.stagtheatre.org for more details ) -- Anthony Baxter It's never too late to have a happy childhood. From nick at craig-wood.com Wed Mar 2 10:55:15 2005 From: nick at craig-wood.com (Nick Craig-Wood) Date: Wed Mar 2 10:55:19 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <20050301225550.GB11698@mems-exchange.org> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> Message-ID: <20050302095515.GA14982@craig-wood.com> On Tue, Mar 01, 2005 at 05:55:50PM -0500, Neil Schemenauer wrote: > On Tue, Mar 01, 2005 at 11:45:43PM +1000, Nick Coghlan wrote: > > Interesting. In that case, my other suggestion was to have raising > > NotImplementedError from a special method be the equivalent of returning > > NotImplemented (which makes life much easier for a class like Decimal which > > has an internal method for doing the type conversion). > > NotImplementedError has nothing to do with the NotImplemented > singleton. It's unfortunate about the naming. IMO, Decimal should > be returning NotImplemented instead of raising TypeError. That seems very un-pythonic to me - return an error? If you look at the patched Decimal module you'll see all sorts of contortions necessary to accomodate it, wheras if it could just raise NotImplementedError or NotImplementedTypeError it would make the code a lot cleaner. I have to say when I read the python language docs, I assumed there was a mistake in them and they meant to say "raise NotImplementedError" instead of "return NotImplemented". Why is it like that? And could it be changed (Nick Coghlan's proposal seems good to me)? -- Nick Craig-Wood -- http://www.craig-wood.com/nick From ncoghlan at iinet.net.au Wed Mar 2 11:23:34 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Mar 2 11:23:38 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <20050301225550.GB11698@mems-exchange.org> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> Message-ID: <42259426.9010401@iinet.net.au> Neil Schemenauer wrote: > IMO, Decimal should > be returning NotImplemented instead of raising TypeError. I agree, but I'm also interested in the fact that nobody commented on this before Python 2.4 went out. I read the reference page on numeric coercion more times than I care to count, and still didn't register that there might be an issue with raising the TypeError - and that's only me. Relative to people like Raymond and Facundo, I spent comparatively little time working on Decimal :) I think the problem arose because the natural thing to do in Python code is to push the type inspection for operations into a common method, and have that method raise TypeError if the other argument isn't acceptable. Trying to convert that exception to a 'NotImplemented' return value is a pain (it's less of a pain in C, since you're already checking for error returns instead of using exceptions). The error return idiom is really unnatural in Python code. Mixing the error return for the special methods with raising an exception for non-special methods is exceedingly ugly (particularly for Decimal, since operations through Context objects should *always* raise a TypeError for bad arguments. If the special methods return NotImplemented instead of raising an exception, then Context has to convert those returns to TypeErrors). Additionally, returning NotImplemented means that direct calls to special methods may not raise an exception in response to a bad argument. Compare: Py> 1 .__add__("") NotImplemented Py> "".__add__(1) Traceback (most recent call last): File "", line 1, in ? TypeError: cannot concatenate 'str' and 'int' objects Hmm, perhaps a decorator would help here. Something like: class OperatorTypeError(TypeError): pass class operatormethod(object): def __init__(self, method): self._method = method self._obj = None def __get__(self, obj, type=None): self._obj = obj return self def __call__(*args, **kwds): self = args[0] obj = self._obj try: if obj is None: return self._method(*args[1:], **kwds) return self._method(obj, *args[1:], **kwds) except OperatorTypeError: return NotImplemented Then do: Py> class C: ... def _add(self, other): ... raise OperatorTypeError ... __add__ = operatormethod(_add) ... __radd__ = __add__ ... Py> C()._add(1) Traceback (most recent call last): File "", line 1, in ? File "", line 3, in _add __main__.OperatorTypeError Py> C().__add__(1) NotImplemented Py> C() + 1 Traceback (most recent call last): File "", line 1, in ? TypeError: unsupported operand type(s) for +: 'instance' and 'int' Py> 1 + C() Traceback (most recent call last): File "", line 1, in ? TypeError: unsupported operand type(s) for +: 'int' and 'instance' That makes it easy to write natural Python code (raising a specific subclass of type error), while still returning NotImplemented so the operator coercion works correctly. > That > could be considered a bug but I'd file it under new functionality > for the purposes of backporting (i.e. fix it in 2.5 only). Given that I'm already ambivalent about changing this for 2.4, it won't take much to convince me that this should be left to 2.5. Particularly if we actually try to find a way to make it easier to 'do the right thing', rather than just changing Decimal. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Wed Mar 2 11:28:58 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Mar 2 11:29:20 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <20050302095515.GA14982@craig-wood.com> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> <20050302095515.GA14982@craig-wood.com> Message-ID: <4225956A.9060805@iinet.net.au> Nick Craig-Wood wrote: > Why is it like that? Speed, I assume - if it was exception based, then *every* operation effectively gets wrapped in a try/except block. Yikes! Also, for C implementations, the error return is fairly natural. It's only when implementing operations in Python that it bites. > And could it be changed (Nick Coghlan's proposal > seems good to me)? Take a look at my latest suggestion using OperatorTypeError and operatormethod. (which makes it easy to put the try/catch block in place if you want it, while still leaving it out for the common case). Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From mwh at python.net Wed Mar 2 13:21:08 2005 From: mwh at python.net (Michael Hudson) Date: Wed Mar 2 13:21:10 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <42259426.9010401@iinet.net.au> (Nick Coghlan's message of "Wed, 02 Mar 2005 20:23:34 +1000") References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> <42259426.9010401@iinet.net.au> Message-ID: <2mmztmqlvv.fsf@starship.python.net> Nick Coghlan writes: > Neil Schemenauer wrote: >> IMO, Decimal should >> be returning NotImplemented instead of raising TypeError. > > I agree, but I'm also interested in the fact that nobody commented on > this before Python 2.4 went out. I read the reference page on numeric > coercion more times than I care to count, and still didn't register > that there might be an issue with raising the TypeError - and that's > only me. Relative to people like Raymond and Facundo, I spent > comparatively little time working on Decimal :) > > I think the problem arose because the natural thing to do in Python > code is to push the type inspection for operations into a common > method, and have that method raise TypeError if the other argument > isn't acceptable. > > Trying to convert that exception to a 'NotImplemented' return value is > a pain (it's less of a pain in C, since you're already checking for > error returns instead of using exceptions). It seems to me that the most straightforward thing would be for _convert_other to return NotImplemented, or to split it into two functions, one that returns NotImplemented and one that raises and use the former in __methods__. This would mean a certain amount of other = _convert_other_ni(other) if other is NotImplemented: return other but this seems mild in comparisons to the contortions the module already performs in the name of correctness. Cheers, mwh PS: this beahviour seems odd already: >>> decimal.getcontext().abs(1) Traceback (most recent call last): File "", line 1, in ? File "/Users/mwh/Source/python/dist/src/Lib/decimal.py", line 2321, in abs return a.__abs__(context=self) TypeError: wrapper __abs__ doesn't take keyword arguments couldn't/shouldn't the context methods do a _convert_other (of the erroring out kind) of their arguments? -- Gullible editorial staff continues to post links to any and all articles that vaguely criticize Linux in any way. -- Reason #4 for quitting slashdot today, from http://www.cs.washington.edu/homes/klee/misc/slashdot.html From steven.bethard at gmail.com Wed Mar 2 18:20:37 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed Mar 2 18:20:40 2005 Subject: [Python-Dev] assigning a custom mapping type to __dict__ Message-ID: I think the behavior when supplying a custom mapping type to __dict__ is a little confusing. I'd like to supply a patch, but I'm not sure what the preferred solution is. Let me explain the problem a little first. For a custom mapping type that does not inherit from dict, Python gives a nice error message when it's assigned to __dict__: py> class M(object): ... def __getitem__(self, key): ... return 42 ... py> class O(object): ... pass ... py> o = O() py> o.__dict__ = M() Traceback (most recent call last): File "", line 1, in ? TypeError: __dict__ must be set to a dictionary However, a subclass of dict will still be accepted, but has some confusing behavior -- a redefined __getitem__ is not actually invoked: py> class M(dict): ... def __getitem__(self, key): ... return 42 ... py> class O(object): ... pass ... py> o = O() py> o.__dict__ = M() py> o.a Traceback (most recent call last): File "", line 1, in ? AttributeError: 'O' object has no attribute 'a' By overriding __getattribute__ in the object's class, you can get the expected behavior: py> class M(dict): ... def __getitem__(self, key): ... return 42 ... py> class O(object): ... def __getattribute__(self, name): ... getattribute = super(O, self).__getattribute__ ... if not name.startswith('__') or not name.endswith('__'): ... try: ... return getattribute('__dict__')[name] ... except KeyError: ... raise AttributeError(name) ... return getattribute(name) ... py> o = O() py> o.__dict__ = M() py> o.a 42 py> o.a = 1 py> o.a 42 So, I see this as a problem, mainly because an error (assiging a mapping to __dict__ and expecting its methods to be called) passes silently. I guess the first question is, do others agree this is also an problem? If so, I see a few solutions, and I'd like some input on which is most desirable. (1) Raise an exception when a subclass of dict is assigned to __dict__. This is probably the simplest solution (I assume it's basically just changing a PyDict_Check to a PyDict_CheckExact), and it keeps the error from passing silently, but it is backwards-incompatible. (Classes like my last versions of M and O above will be broken.) (2) Allow sublcasses of dict to be assigned to __dict__, but change the implementation to call the subclass methods, not the dict methods. I don't know exactly what needs to be changed to make this work. (3) Allow any mapping type to be assigned to __dict__, and change the implementation to use Py_Mapping* calls instead. I don't know the implementation well enough to know the consequences of the last two, so I'm hoping for some feedback here. Thanks! Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From Scott.Daniels at Acm.Org Wed Mar 2 20:44:20 2005 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Wed Mar 2 20:56:17 2005 Subject: [Python-Dev] Re: OK, time to retire In-Reply-To: <42253866.6030002@ocf.berkeley.edu> References: <1109649779.890523.249560@o13g2000cwo.googlegroups.com> <42244EB3.5070406@holdenweb.com> <42253866.6030002@ocf.berkeley.edu> Message-ID: Brett C. wrote: > I have decided that I am going to stop doing the python-dev Summaries > after PyCon; the Summary covering the last half of March 2005 will be > it for me. I (as well as most, I'd guess) have enjoyed your voice in the summaries. Thanks for a great series of summaries. Perhaps your final summary could be a personal view of PyCon for those of us unable to get there. If you make no more contribution to Python than you have so far, you will have done us a great service. Hip-hip-hooray-ly y'rs --Scott David Daniels Scott.Daniels@Acm.Org From gvanrossum at gmail.com Thu Mar 3 04:58:00 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Mar 3 04:58:07 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <4225956A.9060805@iinet.net.au> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> <20050302095515.GA14982@craig-wood.com> <4225956A.9060805@iinet.net.au> Message-ID: > Nick Craig-Wood wrote: > > Why is it like that? Nick Coghlan replied: > Speed, I assume - if it was exception based, then *every* operation effectively > gets wrapped in a try/except block. Yikes! No, the reason is that if we did this with exceptions, it would be liable to mask errors; an exception does not necessarily originate immediately with the code you invoked, it could have been raised by something else that was invoked by that code. The special value NotImplemented must pretty much originate directly in the invoked code, since the implementation guarantees that e.g. a+b can never *return* NotImplemented: if a.__add__(b) and b.__radd__(a) both return NotImplemented, TypeError is raised. (Then why is StopIteration an exception instead of a special value, you may ask? That's because we didn't want to have a value that you could stick in a list to stop iteration over the list prematurely, which would cause problems similar to null bytes in C strings. For binary operators this isn't a problem, since binary operators (at least by convention) always return an object that itself supports the same binary operator that produced it, and NotImplemented itself doesn't implement any binary operators.) > Also, for C implementations, the error return is fairly natural. It's only when > implementing operations in Python that it bites. You went on to great lengths later about how it's less natural in Python to return an error value, but I disagree. I've written a lot of code like this and it's quite natural to write things like this: def __add__(self, other): if not isinstance(other, ThisClass): return NotImplemented return ...implementation of self + other... If you want to factor out the type check, it makes just as much sense to define a Boolean function: def __add__(self, other): if not self.acceptableArgument(other): return NotImplemented ...etc... I'm guessing this wasn't caught before 2.4 was released because most people reviewing the code were more interested in correct computations than into correct integration in the Python framework. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Mar 3 12:11:45 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Mar 3 12:11:49 2005 Subject: [Python-Dev] Patch Reviewing In-Reply-To: References: Message-ID: <4226F0F1.10208@v.loewis.de> Reinhold Birkenfeld wrote: > just felt a little bored and tried to review a few (no-brainer) patches. Thanks! They are now all closed. Please understand that there is a chance that a review posted *only* to python-dev might get lost/overlooked by the committers, and then somebody may need to duplicate your work later. So it is better to include the review in the patch, and only post a summary to python-dev. Regards, Martin From irmen at xs4all.nl Thu Mar 3 18:17:46 2005 From: irmen at xs4all.nl (Irmen de Jong) Date: Thu Mar 3 18:17:42 2005 Subject: [Python-Dev] a bunch of Patch reviews In-Reply-To: <421E4238.70702@v.loewis.de> References: <41EA9196.1020709@xs4all.nl> <421E4238.70702@v.loewis.de> Message-ID: <422746BA.3010404@xs4all.nl> Martin v. L?wis wrote: > Irmen de Jong wrote: > >> I've looked at one bug and a bunch of patches and >> added a comment to them: > > > Thanks! I have now processed the ones for which I found guidance. Thank you > As for the remaining ones: > >> [ 756021 ] Allow socket.inet_aton('255.255.255.255') on Windows >> Looks good but added suggestion about when to test for special case > > > So what to do about this? Wait whether he revises the patch? > Accept anyway? Update the patch myself? I think I will revise the patch myself. I was just waiting for some more input, but since nobody replied, I'll just go ahead with it. >> [ 1103350 ] send/recv SEGMENT_SIZE should be used more in socketmodule > > > So what do you propose to do? AFAICT, there is no definition of > SEGMENT_SIZE in a TCP implementation, and I think we should not try > to make up a value. Well, for the VMS platform a value has been made up... Why make an exception for that one and not for Win32? > IMO, Python should expose sockets more or less "as-is". If the system > has a flaw, Python should expose it instead of working around it. Is this the default way of treating system flaws, or should they be considered on a per-case basis? I can imagine that some system flaws are just plain stupid and are easy to hide (or circumvent) in the Python implementation. The recv/send segment size issue can have nasty results on Win32, see the referenced bug report 853507 "socket.recv() raises MemoryError exception" which I think is related to this. (yes, I know that Tim marked that bug as wontfix) Note: I'm not too experienced with Win32 programming and so I don't have a very good argumentation for the buffer size issue on this platform. If there is somebody with better understanding of the issues involved here, please advise. (it's just empirical knowledge that I have that leads me to believe that win32's tcp implementation suffers from similar recv/send size problems as VMS does-- for which a special case was made in the code) >> [ 1062014 ] fix for 764437 AF_UNIX socket special linux socket names > > > Can you please elaborate the problem? What is a "special linux socket > name"? See bug report 764437. AF_UNIX sockets on Linux can have so-called 'kernel' socket names that don't show up in the filesystem (regular unix domain sockets do). The current socketmodule doesn't treat this special kind of unix domain sockets well. This fix was submitted during the last python bug day you can find some more info here too: http://www.python.org/moin/PythonBugDayStatus > Regardless, the comment of the other reviewer is also valid: any patch > needs documentation and test cases. Yes, Johannes is right. I should have made docs and test cases but didn't get around to doing that yet. When my home network is fixed (router failure) I'll try to spend some time improving the patches. Regards Irmen de Jong From mcherm at mcherm.com Thu Mar 3 22:12:25 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Thu Mar 3 22:12:31 2005 Subject: [Python-Dev] Re: [Python Dev] PEP 309 Message-ID: <20050303131225.p3ifya9ua96ogs4c@mcherm.com> Nick Coghlan writes: > I'm actually half-tempted to suggest that [map, filter and reduce] should be > aliased in the functional module for 2.5 (in addition to being in builtins - ala > the contents of the exceptions module). Well, I think it's a good idea, so I'll formally propose it! Let's alias map, filter, and reduce into the functional module now that it exists. This doesn't create any need or pressure to remove these as builtins, but it does contemplate a day when we might choose to make such a choice. -- Michael Chermside From python at rcn.com Thu Mar 3 22:22:40 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu Mar 3 22:26:56 2005 Subject: [Python-Dev] Re: [Python Dev] PEP 309 In-Reply-To: <20050303131225.p3ifya9ua96ogs4c@mcherm.com> Message-ID: <000d01c52037$221ad0c0$dd23a044@oemcomputer> > Nick Coghlan writes: > > I'm actually half-tempted to suggest that [map, filter and reduce] > should be > > aliased in the functional module for 2.5 (in addition to being in > builtins - > ala > > the contents of the exceptions module). > > Well, I think it's a good idea, so I'll formally propose it! Let's alias > map, filter, and reduce into the functional module now that it exists. > This doesn't create any need or pressure to remove these as builtins, but > it does contemplate a day when we might choose to make such a choice. -1 map(), filter(), and zip() have already evolved into imap(), ifilter(), ifilterfalse(), and izip() which we already have in the itertools module. Introducing extra copies of the old list/builtin versions doesn't improve anyone's life. It adds bloat without adding new functionality. Also, someone else (Alex maybe) was proposing another module for functions that consume an iterator and return a scalar (like sum(), min(), max(), etc). The idea was to make some of the itertools recipes into a module (any, all, no, take, etc). If that happens, then there would be yet another possible home for reduce. If you want to add something new and useful to the functional module, try a compose() function. Raymond From anthony at interlink.com.au Fri Mar 4 03:45:20 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Mar 4 03:45:33 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.27, 1.1193.2.28 In-Reply-To: References: Message-ID: <200503041345.20965.anthony@interlink.com.au> On Friday 04 March 2005 03:56, rhettinger@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Misc > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6024/Misc > > Modified Files: > Tag: release24-maint > NEWS > Log Message: > SF bug #1155938: Missing None check for __init__(). Won't this break working (but erroneous) code? If so, should it be applied to the 2.4 branch? >>> class f(object): ... def __init__(self): ... self.a = 1 ... return True ... >>> >>> a=f() >>> -- Anthony Baxter It's never too late to have a happy childhood. From python at rcn.com Fri Mar 4 04:35:18 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri Mar 4 04:39:41 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.27, 1.1193.2.28 In-Reply-To: <200503041345.20965.anthony@interlink.com.au> Message-ID: <002201c5206b$302be620$dd23a044@oemcomputer> > > SF bug #1155938: Missing None check for __init__(). > > Won't this break working (but erroneous) code? If so, > should it be applied to the 2.4 branch? Hmm, more than one poster found the error message to be helpful in detecting buggy code. The non-None return value was a pretty good indicator that the code was doing something other than what the programmer intended. OTOH, there is almost certainly some existing, working code that would stop working. How about changing this to a warning in Py2.4? Raymond From pycon at python.org Fri Mar 4 04:46:41 2005 From: pycon at python.org (Steve Holden) Date: Fri Mar 4 04:46:43 2005 Subject: [Python-Dev] Final PyCon Keynote from Google Speaker Message-ID: <20050304034641.EE7181E4004@bag.python.org> Dear Python Colleague: I am happy to say that we have now completed the PyCon DC 2005 keynote speaker lineup, and that the final keynote will be given by Greg Stein, an engineering manager wirh Google's Blogger group, who will be speaking on "Python at Google" Greg is also known to many in the open source world as the chairman of the Apache Foundation, and he is a significant contributor to Subversion, WebDAV and Python. With the opening keynote from a Microsoft speaker, the second keynote from Guido van Rossum (the inventor of Python) and the last keynote from Google, PyCon is beginning to demonstrate just how significant Python has become in the modern world of software and Internet services. I hope you will join me in DC from March 23-25, and possibly attend the four-day sprint being held immediately before the conference. Pre- conference registration is available via http://www.python.org//pycon/2005/register.html This is going to be a great opportunity for all those with an interest in Python to see just how far the language has come. regards Steve Holden Chairman, PyCON DC 2005 -- PyCon DC 2005: The third Python Community Conference http://www.pycon.org/ http://www.python.org/pycon/ The scoop on Python implementations and applications From anthony at interlink.com.au Fri Mar 4 04:56:08 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Mar 4 04:56:38 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.27, 1.1193.2.28 In-Reply-To: <002201c5206b$302be620$dd23a044@oemcomputer> References: <002201c5206b$302be620$dd23a044@oemcomputer> Message-ID: <200503041456.09414.anthony@interlink.com.au> On Friday 04 March 2005 14:35, Raymond Hettinger wrote: > Hmm, more than one poster found the error message to be helpful in > detecting buggy code. The non-None return value was a pretty good > indicator that the code was doing something other than what the > programmer intended. > > OTOH, there is almost certainly some existing, working code that would > stop working. > > How about changing this to a warning in Py2.4? +1 That would seem to be the prudent choice. -- Anthony Baxter It's never too late to have a happy childhood. From Cameron at Phaseit.net Tue Mar 1 13:08:24 2005 From: Cameron at Phaseit.net (Cameron Laird) Date: Fri Mar 4 05:13:09 2005 Subject: [Python-Dev] Re: Moving towards Python 3.0 (was Re: Speed up Message-ID: <20050301120824.GA24187@lairds.us> functioncalls) Reply-To: claird-dated-1109937599.27a97c@phaseit.net A month ago, Nathan Binkert wrote: > > Wouldn't it be nicer to have a facility that let you send messages > > between processes and manage concurrency properly instead? You'll > > need > > most of this anyway to do multithreading sanely, and the benefit to > > the > > multiple process model is that you can scale to multiple machines, not > > just processors. For brokering data between processes on the same > > machine, you can use mapped memory if you can't afford to copy it > > around, which gives you basically all the benefits of threads with > > fewer pitfalls. > > I don't think this is an answered problem. There are plenty of > researchers on both sides of this fence. It is not been proven at all > that threads are a bad model. > > http://capriccio.cs.berkeley.edu/pubs/threads-hotos-2003.pdf or even > http://www.python.org/~jeremy/weblog/030912.html > I want to add: me, too. That is, while all my instincts incline toward message-passing and "bulkier" concurrency which emphasizes clusters more than multiprocessors, what's most certain to me is that we--computing people, academics, all of us together--really don't know yet what the right answers are. *My* personal desire: Python as a healthy industrial-strength language strong enough to support such radical experiments as Stackless, PyPy, a sophisticated multithreader, and so on. It has been; I expect it will be. Cameron Laird http://www.phaseit.net Phaseit, Inc. claird@phaseit.net +1 281 996 8546 FAX From bac at OCF.Berkeley.EDU Fri Mar 4 07:55:47 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Mar 4 07:55:57 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] Message-ID: <42280673.5010201@ocf.berkeley.edu> OK, so this one is really short. Beyond the fact that I am probably doing this too late at night I also want to give the three people who have stepped forward to take over for me when I stopped writing the Summaries multiple chances to pick up on any of the Skipped Threads or even flesh out any of my thread summaries more. I have asked them to post their writing here to make the choosing of a successor a little on the open side. Hope to send this out Sunday night. ------------------------------- ===================== Summary Announcements ===================== -------------------------- Giving myself a gold watch -------------------------- As some of you may have already heard or read, I am retiring from writing the python-dev Summaries after sending out the March 16 - 31 summary. It has been a long time coming and it required a kick in the ass (graciously supplied by Steve Holden) to finally make me let go of doing this and let someone else take over. The joy of the Summaries has dwindled over the 2.5 years I have been doing this. I was only doing them to be helpful. But now I would rather put my time and effort I have for Python into coding work rather than the Summaries. I would like to think I can be more productive and helpful as a programmer than a writer. And so there will only be three more regular Summaries after this written by yours truly. But do not worry about the Summaries dying! When I announced this (see http://mail.python.org/pipermail/python-dev/2005-March/051823.html for the thread that led to this), three individuals stepped forward to pick up the work once I step down. Steven Bethard, Tony Meyer, and Tim Lesher are being considered. I honestly have no clue how the heck I am going to choose between the three of them. As for my last Summary, expect a more expository one with my random thoughts on PyCon, Python, and anything else that comes to mind that I feel like using this space to abuse. You have Scott David Daniels to thank for that format idea for my final hurrah. ------------ Go to PyCon! ------------ I just booked my hotel room for PyCon_. You going to be there so I can shake your hand, thanking you for supporting Python? .. _PyCon: http://www.pycon.org/ ========= Summaries ========= ------------------------------------- Python Security Response Team founded ------------------------------------- For those of you who don't know, a security hole was found in XML-RPC servers in the stdlib that use register_instance; details at http://www.python.org/security/PSF-2005-001/ . In response to this, Guido has now formed the 'Python Security Response Team`_. This group's job is to respond to any and all security issues that come up as quickly as possible and to issue a patch in a timely fashion. .. _Python Security Response Team: http://www.python.org/security/ Contributing threads: - `Wanted: members for Python Security Response Team <>`__ ------------------------------ Licensing issues in the stdlib ------------------------------ It was reported to python-dev that 'profiler' and 'md5 do not conform to the Debian Free Software Guidelines. The specific issue was derived works using Python. This is obviously not a good thing. Things are in the works, though. The hope is to get the copyrights signed over and all of this cleared up. At worst the modules will be replaced with code licensed to the Python Software Foundation. If you care to accelerate this by writing replacements please do so. Contributing threads: - `license issues with profiler.py and md5.h/md5c.c <>`__ =============== Skipped Threads =============== + complex I/O problem + Is msvcr71.dll re-redistributable? + Son of PEP 246, redux + Re: PEP 246: LiskovViolation as a name + 2.3.5 and 2.4.1 release plans + Re: [Python-checkins] python/dist/src/Python future.c, 2.14, 2.15 + list of constants -> tuple of constants + Other library updates + Re: python/dist/src/Lib rfc822.py,1.78,1.79 + Patch review: [ 1098732 ] Enhance tracebacks and stack traces with vars + Re: [Python-checkins] python/dist/src/Python compile.c, 2.343, 2.344 + Re: [Python-checkins] python/dist/src/Lib xmlrpclib.py, 1.38, 1.39 + ViewCVS on SourceForge is broken + builtin_id() returns negative numbers + Clarification sought about including a multidimensional array object into Python core From ncoghlan at iinet.net.au Fri Mar 4 16:33:01 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Mar 4 16:33:06 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> <20050302095515.GA14982@craig-wood.com> <4225956A.9060805@iinet.net.au> Message-ID: <42287FAD.70908@iinet.net.au> Guido van Rossum wrote: > No, the reason is that if we did this with exceptions, it would be > liable to mask errors; an exception does not necessarily originate > immediately with the code you invoked, it could have been raised by > something else that was invoked by that code. The special value > NotImplemented must pretty much originate directly in the invoked > code, since the implementation guarantees that e.g. a+b can never > *return* NotImplemented: if a.__add__(b) and b.__radd__(a) both return > NotImplemented, TypeError is raised. That makes sense - although for that reasoning, a TypeError subclass that the binary operation machinery promotes to a standard TypeError would seem to work too (with the advantage of also raising at least some sort of exception when the method is called directly) > You went on to great lengths later about how it's less natural in > Python to return an error value, but I disagree. I've written a lot of > code like this and it's quite natural to write things like this: > > def __add__(self, other): > if not isinstance(other, ThisClass): > return NotImplemented > return ...implementation of self + other... It wasn't so much this part that struck me as ugly, as the subsequent usage of the methods directly in the Decimal Context implementation. The inconsistency of the interface was the main irritation. All of the methods that implemented standard binary operations returned NotImplemented on an argument error, while other methods raised TypeError directly. So for some methods Context was checking the return value and raising TypeError, but for others it was just making the call. The mixture of the two styles didn't look nice :) > If you want to factor out the type check, it makes just as much sense > to define a Boolean function: > > def __add__(self, other): > if not self.acceptableArgument(other): > return NotImplemented > ...etc... I'm considering an approach that involves the simple wrapper function: def raiseIfNotImplemented(func, message): def wrapper(*args, **kwds): result = func(*args, **kwds) if result is NotImplemented: raise TypeError(message) return result After rewriting _convert_other and all the binary special methods to return NotImplemented for bad arguments (as you suggest), the wrapper above can be used to provide alternate functions that raise the TypeError instead (making it easy to provide a consistent API for use by Context). Obviously, such a strategy makes this a 2.5 only solution, as it involves adding methods. A potentially-2.4-compatible solution is the one I used in 'friendly decimal' which simply duplicates the code of the above wrapper wherever it is needed by Decimal or Context. > I'm guessing this wasn't caught before 2.4 was released because most > people reviewing the code were more interested in correct computations > than into correct integration in the Python framework. This is quite likely to be true :) Although I find it interesting that string objects share the same characteristic of not respecting __rop__ when it is provided by another class that is not a subclass of string. Regards, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Fri Mar 4 16:36:47 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Mar 4 16:36:49 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <2mmztmqlvv.fsf@starship.python.net> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> <42259426.9010401@iinet.net.au> <2mmztmqlvv.fsf@starship.python.net> Message-ID: <4228808F.3080802@iinet.net.au> Michael Hudson wrote: > PS: this beahviour seems odd already: > > >>>>decimal.getcontext().abs(1) > > Traceback (most recent call last): > File "", line 1, in ? > File "/Users/mwh/Source/python/dist/src/Lib/decimal.py", line 2321, in abs > return a.__abs__(context=self) > TypeError: wrapper __abs__ doesn't take keyword arguments > > couldn't/shouldn't the context methods do a _convert_other (of the > erroring out kind) of their arguments? You'll have to ask Facundo that one - it never occurred to me to question it until looking into the __rop__ behaviour :) But I do agree - if nothing else, the _convert_other gives a far more meaningful error message when things do go wrong. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From tlesher at gmail.com Fri Mar 4 18:44:02 2005 From: tlesher at gmail.com (Tim Lesher) Date: Fri Mar 4 18:44:05 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] In-Reply-To: <42280673.5010201@ocf.berkeley.edu> References: <42280673.5010201@ocf.berkeley.edu> Message-ID: <9613db6005030409443ea55f74@mail.gmail.com> Here are sample summaries of two "skipped threads". If anyone has comments, criticisms, or rotten tomatoes, let me know. --------------------------------------------------- Replacing list of constants with tuple of constants --------------------------------------------------- Skip Montanaro noticed that Raymond Hettinger had made a number of library checkins replacing lists with tuples in constructs like ``for a in [1, 2, 3]:`` and ``if a in [1, 2, 3]:``. He agreed with the motivation (the peephole optimizer can convert a tuple of constants into a single constant, eliminating construction time), but questioned hardcoding the optimization. Instead, he suggested teaching the optimizer about "throwaway" lists in ``for`` and ``if`` statements. Guido agreed with the sentiment. Raymond accepted the suggestion, and checked in code to implement it. Contributing threads: - `list of constants -> tuple of constants `__ ------------------- Complex I/O problem ------------------- Neal Becker asked why the result of printing a ``complex`` is not an acceptable input to constructing a complex. For example, ``print complex(2,2)`` yields ``'(2+2j)'``; but ``complex('(2+2j)')`` throws a ``ValueError``. A. M. Kuchling responded that many types, including ``str`` and ``file`` share this behavior, and that in any event, parsing string representations is a job more suited to ``eval`` than to classes themselves. Guido `reiterated the rules`_ for ``str()`` (used by ``print``) and ``repr()``. Since both ``complex.__str__`` and ``complex.__repr__`` pass the ``eval()`` test, he pronounced it fine. .. _reiterated the rules: http://mail.python.org/pipermail/python-dev/2005-February/051390.html Contributing threads: - `complex I/O problem `__ -- Tim Lesher From martin at v.loewis.de Fri Mar 4 20:46:00 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 4 20:46:04 2005 Subject: [Python-Dev] Outdating import.c 2.241 Message-ID: <4228BAF8.1010709@v.loewis.de> I just mistakenly checked in import.c; I have "outdated" this checkin (cvs admin -o 2.241 import.c). If you happened to have checked-out import.c in-between, don't be surprised if the version numbers go backwards. Regards, Martin From mwh at python.net Fri Mar 4 21:42:46 2005 From: mwh at python.net (Michael Hudson) Date: Fri Mar 4 21:42:49 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python import.c, 2.240, 2.241 In-Reply-To: (loewis@users.sourceforge.net's message of "Fri, 04 Mar 2005 11:40:38 -0800") References: Message-ID: <2m6507qh15.fsf@starship.python.net> loewis@users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Python > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22139/Python > > Modified Files: > import.c > Log Message: > Patch #1043890: tarfile: add extractall() method. Uh, did you mean to check import.c in here? Cheers, mwh -- > So what does "abc" / "ab" equal? cheese -- Steve Holden defends obscure semantics on comp.lang.python From mwh at python.net Fri Mar 4 21:44:05 2005 From: mwh at python.net (Michael Hudson) Date: Fri Mar 4 21:44:07 2005 Subject: [Python-Dev] Outdating import.c 2.241 In-Reply-To: <4228BAF8.1010709@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Fri, 04 Mar 2005 20:46:00 +0100") References: <4228BAF8.1010709@v.loewis.de> Message-ID: <2m1xavqgyy.fsf@starship.python.net> "Martin v. L?wis" writes: > I just mistakenly checked in import.c; I have "outdated" > this checkin (cvs admin -o 2.241 import.c). If you happened > to have checked-out import.c in-between, don't be surprised > if the version numbers go backwards. Oh! Oops. python-checkins sorts before python-dev... Cheers, mwh -- Remember - if all you have is an axe, every problem looks like hours of fun. -- Frossie -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From tim.peters at gmail.com Fri Mar 4 22:01:42 2005 From: tim.peters at gmail.com (Tim Peters) Date: Fri Mar 4 22:01:58 2005 Subject: [Python-Dev] Useful thread project for 2.5? Message-ID: <1f7befae050304130154cd25b7@mail.gmail.com> Florent Guillaume recently wrote a valuable addin for Zope: http://www.zope.org/Members/nuxeo/Products/DeadlockDebugger When a Zope has threads that are hung, this can give a report of Python's current state (stack trace) across all threads -- even the ones that are hung (the deadlocked threads don't have to cooperate). The same flavor of thing would (of course) be handy outside of Zope too -- debugging deadlocked Python threads is a PITA regardless of context. Florent's DeadlockDebugger in turn builds on an external C threadframe module: http://www.majid.info/mylos/stories/2004/06/10/threadframe.html Folding the functionality of that (or similar functionality) into the core would, IMO, be a valuable addition for 2.5, and would make an excellent intro project for an aspiring contributor interested in how threads work in CPython (what this module does is conceptually simple). It belongs in the core because it's not safe to chase the tstate chain without holding pystate.c's internal head_mutex lock (holding the GIL isn't enough -- it's normal practice to call PyThreadState_Delete() while not holding the GIL). I'd do it myself (and maybe I will anyway), but this really would make a good (finite; conceptually simple) project for someone who wants to gain Python developer experience. From pje at telecommunity.com Fri Mar 4 22:17:38 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Mar 4 22:15:03 2005 Subject: [Python-Dev] Useful thread project for 2.5? In-Reply-To: <1f7befae050304130154cd25b7@mail.gmail.com> Message-ID: <5.1.1.6.0.20050304161554.029f7ec0@mail.telecommunity.com> At 04:01 PM 3/4/05 -0500, Tim Peters wrote: >Florent's DeadlockDebugger in turn builds on an external C threadframe module: > > http://www.majid.info/mylos/stories/2004/06/10/threadframe.html > >Folding the functionality of that (or similar functionality) into the >core would, IMO, be a valuable addition for 2.5, and would make an >excellent intro project for an aspiring contributor interested in how >threads work in CPython (what this module does is conceptually >simple). It belongs in the core because it's not safe to chase the >tstate chain without holding pystate.c's internal head_mutex lock >(holding the GIL isn't enough -- it's normal practice to call >PyThreadState_Delete() while not holding the GIL). > >I'd do it myself (and maybe I will anyway), but this really would make >a good (finite; conceptually simple) project for someone who wants to >gain Python developer experience. What would you suggest calling it? sys._current_frames(), returning a dictionary? From steven.bethard at gmail.com Fri Mar 4 23:02:39 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri Mar 4 23:02:42 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] In-Reply-To: <42280673.5010201@ocf.berkeley.edu> References: <42280673.5010201@ocf.berkeley.edu> Message-ID: Here's another two skipped threads. Ditto Tim Lesher's "comments, criticisms, or rotten tomatoes" request. =) ------------------------------------- 2.3.5 and 2.4.1 release plans ------------------------------------- Anthony Baxter, Alex Martelli and Tim Peters squelched a bug where deepcopy failed on instances of types that lacked an ``__mro__`` attribute. The patch was pretty straight-forward (use ``inspect.getmro`` instead of ``cls.__mro__``), but coming up with a test case was hard -- creating a Python object that doesn't have an ``__mro__`` takes some complicated C code like that of Zope's ExtensionClass. Fortunately, John Lenton's c.l.py suggestion to simply raise an AttributeError for ``__mro__`` in ``__getattribute__`` properly ticked the bug, and 2.3.5 was cleared for release. Contributing Threads: - `2.3.5 and 2.4.1 release plans `__ ------------------------------------- Clarification sought about including a multidimensional array object into Python core ------------------------------------- Travis Oliphant and others looked into the issues of including an array object (like that of Numeric or numarray) in Python core. Guido seemed hesitant, concerned that as Numeric and numarray continue to co-exist, the array object wouldn't be the "best of breed" (one of the expectations for inclusion in Python core). Travis explained that most of the disagreements are over ufunc objects, not the basic array object itself, so it wouldn't be unreasonable to include the array object without the ufunc object if necessary. There was also some suggestion that, at least for basic arithmetic operations, Numeric and numarray mostly agree, so a stripped-down ufunc object for these operations might also be inclusion-worthy. In an aside that grew decidedly un-aside-ish, Paul F. Dubois, Guido and David Ascher explained why matrix should not inherit from arrayobject -- this would complicate __new__ and cause confusion when mixed operands (arrays and matrices) are given to a binary op like multiplication. Contributing Threads: - `Clarification sought about including a multidimensional array object into Python core `__ - `Numeric life as I see it `__ Steve Bethard -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From aahz at pythoncraft.com Fri Mar 4 23:45:12 2005 From: aahz at pythoncraft.com (Aahz) Date: Fri Mar 4 23:45:14 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] In-Reply-To: References: <42280673.5010201@ocf.berkeley.edu> Message-ID: <20050304224512.GA11693@panix.com> Both entries so far look very good. Perhaps writing python-dev summaries could be a rotating position? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From jjl at pobox.com Fri Mar 4 15:29:51 2005 From: jjl at pobox.com (John J Lee) Date: Sat Mar 5 02:09:51 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] In-Reply-To: <20050304224512.GA11693@panix.com> References: <42280673.5010201@ocf.berkeley.edu> <20050304224512.GA11693@panix.com> Message-ID: On Fri, 4 Mar 2005, Aahz wrote: > Both entries so far look very good. Perhaps writing python-dev summaries > could be a rotating position? Or even a joint effort? It's up to the contributors, of course: just a thought... John From aahz at pythoncraft.com Sat Mar 5 02:15:44 2005 From: aahz at pythoncraft.com (Aahz) Date: Sat Mar 5 02:15:46 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] In-Reply-To: References: <42280673.5010201@ocf.berkeley.edu> <20050304224512.GA11693@panix.com> Message-ID: <20050305011544.GA5921@panix.com> On Fri, Mar 04, 2005, John J Lee wrote: > On Fri, 4 Mar 2005, Aahz wrote: >> >> Both entries so far look very good. Perhaps writing python-dev summaries >> could be a rotating position? > > Or even a joint effort? It's up to the contributors, of course: just > a thought... That was my original thought, too, then I remembered just how much work coordinating is.... ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From gward at python.net Sat Mar 5 03:57:46 2005 From: gward at python.net (Greg Ward) Date: Sat Mar 5 03:57:50 2005 Subject: [Python-Dev] Annoying $Id$ in Misc/NEWS Message-ID: <20050305025746.GA24022@cthulhu.gerg.ca> This entry in Misc/NEWS """ - SF patch 995225: The test file testtar.tar accidentally contained CVS keywords (like $Id: NEWS,v 1.1266 2005/03/05 02:53:17 gward Exp $), which could cause spurious failures in test_tarfile.py depending on how the test file was checked out. """ just caused a conflict when I merged something from 2.4 onto the trunk. (It was "Id: ... 1.265 ... mloewis" on the trunk, but "Id: ... 1.1193.2.32 ... gward" on the branch.) This is deliciously ironic: you can't talk about accidental CVS keywords in testtar.tar without having accidental CVS keywords in the text that talks about accidental CVS keywords!! Aieee!!! The obvious fix is to disable keyword expansion for Misc/NEWS: cvs admin -ko Misc/NEWS Anyone mind if I do that? wondering-what-will-happen-to-my-subject-line'ly, Greg -- Greg Ward http://www.gerg.ca/ Never try to outstubborn a cat. From bac at OCF.Berkeley.EDU Sat Mar 5 04:17:08 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Mar 5 04:17:16 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] In-Reply-To: <20050304224512.GA11693@panix.com> References: <42280673.5010201@ocf.berkeley.edu> <20050304224512.GA11693@panix.com> Message-ID: <422924B4.2070501@ocf.berkeley.edu> Aahz wrote: > Both entries so far look very good. Perhaps writing python-dev summaries > could be a rotating position? Good idea that several people have now suggested to me. I have emailed Tim, Steve, and Tony to see what they think. It wouldn't surprise me if this happens if for any other reason than the shock at how long these Summaries can take. =) -Brett From ncoghlan at iinet.net.au Sat Mar 5 04:24:08 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 5 04:24:11 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14 [draft] In-Reply-To: <20050305011544.GA5921@panix.com> References: <42280673.5010201@ocf.berkeley.edu> <20050304224512.GA11693@panix.com> <20050305011544.GA5921@panix.com> Message-ID: <42292658.3050708@iinet.net.au> Aahz wrote: > On Fri, Mar 04, 2005, John J Lee wrote: > >>On Fri, 4 Mar 2005, Aahz wrote: >> >>>Both entries so far look very good. Perhaps writing python-dev summaries >>>could be a rotating position? >> >>Or even a joint effort? It's up to the contributors, of course: just >>a thought... > > > That was my original thought, too, then I remembered just how much work > coordinating is.... ;-) Maybe the contributors could claim threads to summarise shortly after the threads start? That might reduce the burden of needing to follow *every* thread. . . then Skipped Threads would contain any threads that nobody claimed. JJL's last comment applies to me, too, naturally :) Regards, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Sat Mar 5 05:06:38 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 5 05:06:41 2005 Subject: [Python-Dev] Documentation for __new__ Message-ID: <4229304E.5030609@iinet.net.au> Steven Bethard has put together some text to add __new__ to the list of Basic Customisation methods in the language reference. Would one of the documentation folks care to take a look at it? The relevant SF tracker item is http://www.python.org/sf/1156412 Regards, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From bac at OCF.Berkeley.EDU Sat Mar 5 05:55:23 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Mar 5 05:55:32 2005 Subject: [Python-Dev] Memory Allocator Part 2: Did I get it right? In-Reply-To: References: <8b28704b4465e03002fc70db5facedb6@uwaterloo.ca> <1f7befae05021514524d0a35ec@mail.gmail.com> <4c0d14b0b08390d046e1220b6f360745@uwaterloo.ca> <1f7befae05021520263d77a2a3@mail.gmail.com> <4212FB5B.1030209@v.loewis.de> Message-ID: <42293BBB.1010000@ocf.berkeley.edu> Evan Jones wrote: > Sorry for taking so long to get back to this thread, it has been one of > those weeks for me. > > On Feb 16, 2005, at 2:50, Martin v. L?wis wrote: > >> Evan then understood the feature, and made it possible. > > > This is very true: it was a very useful exercise. > >> I can personally accept breaking the code that still relies on the >> invalid APIs. The only problem is that it is really hard to determine >> whether some code *does* violate the API usage. > > > Great. Please ignore the patch on SourceForge for a little while. I'll > produce a "revision 3" this weekend, without the compatibility hack. > Evan uploaded the newest version (@ http://www.python.org/sf/1123430) and he says it is "complete". Assuming a code review says the patch is sane, do we want to go with this garbage collection change? From past discussions I don't remember a consensus on acceptance or rejection, just lots of discussion about ripping out the hacks to allow freeing memory w/o holding the GIL (I assume Evan's patch rips that code out). -Brett From bac at OCF.Berkeley.EDU Sat Mar 5 07:49:38 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Mar 5 07:49:41 2005 Subject: [Python-Dev] Requesting that a class be a new-style class In-Reply-To: References: <4216C89F.3040400@iinet.net.au> <2mpsywxplq.fsf@starship.python.net> Message-ID: <42295682.9040909@ocf.berkeley.edu> Guido van Rossum wrote: >>>This is something I've typed way too many times: >>> >>>Py> class C(): >>> File "", line 1 >>> class C(): >>> ^ >>>SyntaxError: invalid syntax >>> >>>It's the asymmetry with functions that gets to me - defining a >>>function with no arguments still requires parentheses in the >>>definition statement, but defining a class with no bases requires the >>>parentheses to be omitted. > > > It's fine to fix this in 2.5. I guess I can add this to my list of > early oopsies -- although to the very bottom. :-) > > It's *not* fine to make C() mean C(object). (We already have enough > other ways to declaring new-style classes.) > OK, this is now in thanks to the following revisions: Checking in Grammar/Grammar; /cvsroot/python/python/dist/src/Grammar/Grammar,v <-- Grammar new revision: 1.53; previous revision: 1.52 done Checking in Python/graminit.c; /cvsroot/python/python/dist/src/Python/graminit.c,v <-- graminit.c new revision: 2.39; previous revision: 2.38 done Checking in Python/compile.c; /cvsroot/python/python/dist/src/Python/compile.c,v <-- compile.c new revision: 2.349; previous revision: 2.348 done Checking in Misc/NEWS; /cvsroot/python/python/dist/src/Misc/NEWS,v <-- NEWS new revision: 1.1267; previous revision: 1.1266 done -Brett From bac at OCF.Berkeley.EDU Sat Mar 5 08:39:43 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Mar 5 08:39:50 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-15 through 2005-02-28 [draft] Message-ID: <4229623F.1060902@ocf.berkeley.edu> Decided to just plow through the next Summary so that I was finally caught up. I am not expecting the candidates for taking of the Summaries to write stuff for this one (although I wouldn't mind it =). The idea of having them work together to write the Summaries has been proposed, but this is going out before I have heard back. Depending on the number of people who send in edits, this could go out the same time as the 2005-02-01 Summary or I might wait until Monday night so people who only check mail on weekdays have a chance to look at this. ------------------------------------- ===================== Summary Announcements ===================== ------------------------ Status of the candidates ------------------------ XXX ----------- PyCon Looms ----------- PyCon_ is coming! Be there or be a Java or Perl coder! .. _PyCon: http://www.pycon.org/ ========= Summaries ========= ------------- PEP movements ------------- `PEP 309`_ is now final since the 'functional' module has now been checked into Python. .. _PEP 309: http://www.python.org/peps/pep-0309.html Contributing threads: - `PEP 309 enhancements <>`__ - `PEP 309 <>`__ ------------------------------------------------------ Indices for slices other objects with __int__ not okay ------------------------------------------------------ Travis Oliphant asked if it would be possible to patch slicing so that any object that defines __int__ could be used. Guido didn't like this idea, though. Float, for instance, has __int__ defined. Guido admitted he "unfortunately copied a design mistake from C here". He said he might add a __trunc__ magic method in Python 3000 for objects that really can't be viewed as an int but are willing to have data loss to give one. Contributing threads: - `Fixing _PyEval_SliceIndex so that integer-like objects can be used <>`__ - `Fix _PyEval_SliceIndex (Take two) <>`__ -------------------------------------------- Why can't ``class C(): pass`` be acceptable? -------------------------------------------- No reason. =) So as of Python 2.5 it is acceptable to have empty parentheses for class definitions. It does create a classic class and not a new-style one. Contributing threads: - `Requesting that a class be a new-style class <>`__ ---------------------------------- What basestring is truly meant for ---------------------------------- What is basestring for? According to Guido it is purely for unicode and str to inherit from to help with checks in code where either type is acceptable. It is *not* meant to be used as a base class for any other classes. Contributing threads: - `UserString <>`__ ------------------------------------------------------ Quickly opening an SF bug/patch in Firefox/Thunderbird ------------------------------------------------------ Martin v. L?wis posted a way to use the DictionarySearch_ plug-in for Mozilla to launch a browser with the highlighted patch/bug #. See the email for the thread on how to get it to work. .. _DictionarySearch: http://dictionarysearch.mozdev.org/download.php/http://downloads.mozdev.org/dictionarysearch/dictionarysearch_0.8.xpi Contributing threads: - `Quick access to Python bug reports in Thunderbird <>`__ -------------------------------- Optimizing ``x in [1, 2, 3]`` -------------------------------- Raymond Hettinger has been trying to teach the peepholer some new tricks to optimize ``x in [1, 2, 3]`` and the like into a faster operation. Initially he got it to change the list to a tuple. He then tried turning the list into a frozenset, but that had the unforeseen issue of breaking semantics since it then required the object being checked for to be hashable. So Raymond suggested introducing a SearchSet that tried the comparison as a frozenset first, and upon failure of hashing, to old way of just looking at each item in the list. But this seemed like overkill since most lists would be small; probably usually under 4 items. But Fredrik Lundh suggested expanding it to ``x == 1 or x == 2 or x == 3``. This seemed like a performance boost when the items of the list were lists since the COMPARE_OP opcode special-cases comparing ints. But for other instances it probably isn't worth it unless more special-casing is done in the opcodes. Contributing threads: - `Prospective Peephole Transformation <>`__ ------------------ A DupStore opcode? ------------------ Raymond Hettinger suggested a new opcode called DupStore that would replace load;store opcode pairs. Guido questioned if this was leading down a road of adding too much extra code for little benefit. Off this a discussion about speeding up frame allocation, an area viewed as needing some optimization, started up. Contributing threads: - `Store x Load x --> DupStore <>`__ =============== Skipped Threads =============== + pymalloc on 2.1.3 + Exceptions *must*? be old-style classes? + subclassing PyCFunction_Type + Windows Low Fragementation Heap yields speedup of ~15% + string find(substring) vs. substring in string + [ python-Bugs-1124637 ] test_subprocess is far too slow (fwd) + Five review rule on the /dev/ page? + Some old patches + Comment regarding PEP 328 From martin at v.loewis.de Sat Mar 5 09:07:21 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 5 09:07:25 2005 Subject: [Python-Dev] Memory Allocator Part 2: Did I get it right? In-Reply-To: <42293BBB.1010000@ocf.berkeley.edu> References: <8b28704b4465e03002fc70db5facedb6@uwaterloo.ca> <1f7befae05021514524d0a35ec@mail.gmail.com> <4c0d14b0b08390d046e1220b6f360745@uwaterloo.ca> <1f7befae05021520263d77a2a3@mail.gmail.com> <4212FB5B.1030209@v.loewis.de> <42293BBB.1010000@ocf.berkeley.edu> Message-ID: <422968B9.1010902@v.loewis.de> Brett C. wrote: > Assuming a code review says the patch is sane, do we want to go with > this garbage collection change? From past discussions I don't remember > a consensus on acceptance or rejection, just lots of discussion about > ripping out the hacks to allow freeing memory w/o holding the GIL (I > assume Evan's patch rips that code out). I think the consensus is that the feature is desirable. So if the code is correct, it should be checked in. Regards, Martin From gward at python.net Sat Mar 5 17:39:42 2005 From: gward at python.net (Greg Ward) Date: Sat Mar 5 17:39:46 2005 Subject: [Python-Dev] Documentation for __new__ In-Reply-To: <4229304E.5030609@iinet.net.au> References: <4229304E.5030609@iinet.net.au> Message-ID: <20050305163942.GA3242@cthulhu.gerg.ca> On 05 March 2005, Nick Coghlan said: > Steven Bethard has put together some text to add __new__ to the list of > Basic Customisation methods in the language reference. Would one of the > documentation folks care to take a look at it? I've tried to tighten up the text there and hopefully make it a smidgeon clearer and more accurate. Here's my best effort: __new__(cls[, ...]) Called to create a new instance of class 'cls'. __new__() is a static method (special-cased so you need not declare it as such) that takes the class to create an instance of as the first argument. The remaining arguments are those passed to the object constructor expression. The return value of __new__() should be the new object instance. Typical usage is to create a new instance of the class by invoking the superclass's __new__() method using "super(currentclass, cls).__new__([...])" with appropriate arguments, modifying the returned instance if necessary, and then returning it. If the returned value is an instance of 'cls', its __init__() will be invoked like "__init__(self[, ...])", where the extra arguments are the same as were passed to __new__(). You do need not to return an instance of 'cls', but if you do not, the new instance's __init__() method will not be invoked. __new__() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. Feedback welcome. Has anyone volunteered to render this in LaTeX yet? If not, I might. Greg -- Greg Ward http://www.gerg.ca/ I hope something GOOD came in the mail today so I have a REASON to live!! From gward at python.net Sat Mar 5 20:09:28 2005 From: gward at python.net (Greg Ward) Date: Sat Mar 5 20:09:31 2005 Subject: [Python-Dev] ossaudiodev test failure Message-ID: <20050305190928.GA8248@cthulhu.gerg.ca> I just discovered that the ossaudiodev test fails on Linux 2.6 with ALSA's OSS emulation layer. I'm pretty sure this can be blamed on ALSA (The difference is this: if you pass bogus sampling parameters to setparameters(), OSS will accept the request and set the hardware to something reasonable, while ALSA will reject the request by returning -1 and setting errno, which becomes IOException.) (IMHO, test_ossaudiodev.py should not test this feature if it's not reliably emulated by ALSA.) Anyways, if you're running Linux or FreeBSD, or any other OS that contains OSS drivers or OSS emulation, can you please run ./python ./Lib/test/test_ossaudiodev.py from your CVS tree and email me the following info: * whether the test passed or not * what version of what OS you're running * what sound hardware you have * (Linux only) are you using ALSA or OSS? * whether other audio software (eg. xmms, xine, mpg123, audacity) works for you A passing test looks like this: playing test sound file... elapsed time: 3.1 sec and has a uniquely Pythonesque sound. ;-) The failure I've been seeing is this (you'll see a slightly different traceback, because I've refactored the code a bit in my working dir): playing test sound file... elapsed time: 3.1 sec Traceback (most recent call last): File "Lib/test/test_ossaudiodev.py", line 147, in ? test() File "Lib/test/test_ossaudiodev.py", line 142, in test test_bad_setparameters(dsp) File "Lib/test/test_ossaudiodev.py", line 125, in test_bad_setparameters result = dsp.setparameters(fmt, channels, rate, False) IOError: [Errno 22] Invalid argument Oh, to find out if you're using ALSA or OSS: * run "lsmod" and look for a bunch of modules starting with "snd_" -- this is ALSA * look for device files like /dev/snd/pcmC0D* -- ALSA again If you're using ALSA and /dev/dsp doesn't work, try "modprobe snd_pcm_oss" -- that's the OSS emulation layer. Thanks! Greg -- Greg Ward http://www.gerg.ca/ I used to be a FUNDAMENTALIST, but then I heard about the HIGH RADIATION LEVELS and bought an ENCYCLOPEDIA!! From steven.bethard at gmail.com Sat Mar 5 21:32:26 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat Mar 5 21:32:29 2005 Subject: [Python-Dev] Documentation for __new__ In-Reply-To: <20050305163942.GA3242@cthulhu.gerg.ca> References: <4229304E.5030609@iinet.net.au> <20050305163942.GA3242@cthulhu.gerg.ca> Message-ID: On Sat, 5 Mar 2005 11:39:42 -0500, Greg Ward wrote: > On 05 March 2005, Nick Coghlan said: > > Steven Bethard has put together some text to add __new__ to the list of > > Basic Customisation methods in the language reference. Would one of the > > documentation folks care to take a look at it? [snip] > Typical usage is to create a new instance of the class by > invoking the superclass's __new__() method using > "super(currentclass, cls).__new__([...])" Sorry I didn't catch this originally, but this should be "super(currentclass, cls).__new__(cls[, ...])" since __new__ is a staticmethod. Steve Bethard -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From martin at v.loewis.de Sun Mar 6 20:16:08 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Mar 6 20:16:12 2005 Subject: [Python-Dev] Migrating to subversion Message-ID: <422B56F8.9040708@v.loewis.de> I don't know whether anybody has done this before, but I just tried to run cvs2svn on the Python repository. The conversion took 7 hours, and the result is now available at http://www.dcl.hpi.uni-potsdam.de/python/branches/ Because of the load that the conversion produces on the machine, I cannot run the entire conversion every day. Reportedly, cvs2svn is able to run in incremental mode, but I haven't tried this out yet. On close inspection, you might notice a few things: - A few branches/tags are missing, namely r16b1|cnri-16-start|descr-branch|release152p1-patches|after-c all-reorg|r22a1|before-call-reorg|release16|r161|r161-branch|r16a2|release152p2 I had to manually exclude these tags, because cvs2svn complained that they (some of them) are tags on some files, and branches on other files. When I excluded these, it then complained that some other tags depend on the excluded ones, so I had to exclude these as well. I suspect that this can be fixed by modifying the CVS repository before the conversion, typically by converting the version tags to branch tags. cvs2svn did not report what files specifically caused the problems, and I don't know the proper cvs/rcs incantation to fix these. So if anybody has done that before, or knows how to, please let me know. - A few tags are useless, most notably the "vendor" branches. I think they should be excluded in the conversion. I don't know where the "unlabeled" branches come from, but they appear to be useless as well. - It has imported the CVSROOT directory as well. I don't know whether this is deliberate/useful. Anyway, this repository is now online for anonymous read-only access. Anybody interested in playing with it is welcome to do so. For those interested in server side issues: the repository is an 1.1.1 fsfs repository. The CVS repository consumes 368MiB; the SVN repository 797MiB. Regards, Martin From gward at python.net Sun Mar 6 22:57:11 2005 From: gward at python.net (Greg Ward) Date: Sun Mar 6 22:57:15 2005 Subject: [Python-Dev] Migrating to subversion In-Reply-To: <422B56F8.9040708@v.loewis.de> References: <422B56F8.9040708@v.loewis.de> Message-ID: <20050306215711.GA9397@cthulhu.gerg.ca> On 06 March 2005, "Martin v. L?wis" said: > - It has imported the CVSROOT directory as well. I don't > know whether this is deliberate/useful. This is just an artifact of how SourceForge CVS repositories are organized. When I converted Optik's CVS to Subversion, I just did this: cvs2svn -s /home/svn/optik /home/cvs/optik/optik where /home/cvs/optik is what I got after unpacking the tarball downloaded from SF. Presumably for Python's repository, this would work: cvs2svn -s /home/svn/python /home/cvs/python/python ...except, umm, isn't distutils a separate top-level directory in the Python repository or something? That'll probably have to be fixed manually. Greg -- Greg Ward http://www.gerg.ca/ The NSA. We care: we listen to our customers. From gward at python.net Sun Mar 6 23:33:01 2005 From: gward at python.net (Greg Ward) Date: Sun Mar 6 23:33:04 2005 Subject: [Python-Dev] Documentation for __new__ In-Reply-To: References: <4229304E.5030609@iinet.net.au> <20050305163942.GA3242@cthulhu.gerg.ca> Message-ID: <20050306223301.GB9397@cthulhu.gerg.ca> I've LaTeX'ified the proposed documentation for __new__() and uploaded the patch to SF: http://sourceforge.net/tracker/download.php?group_id=5470&atid=105470&file_id=124422&aid=1156412 Someone who really understands new-style classes should give this a look. (Guido?) I've tested that Python 2.3.5 and 2.4 (CVS) actually implement what's described in this doc patch, but it wouldn't hurt if someone read it from the point of view of what the promised constract is supposed to be! Greg -- Greg Ward http://www.gerg.ca/ I'd like some JUNK FOOD ... and then I want to be ALONE -- From python-dev at fromemail.com Sun Mar 6 23:42:03 2005 From: python-dev at fromemail.com (Fazal Majid) Date: Sun Mar 6 23:42:06 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? Message-ID: <9c074500619cad8ce1824f940e03ad87@fromemail.com> Tim Peters wrote: > Florent's DeadlockDebugger in turn builds on an external C threadframe > module: > > http://www.majid.info/mylos/stories/2004/06/10/threadframe.html > > Folding the functionality of that (or similar functionality) into the > core would, IMO, be a valuable addition for 2.5, and would make an > excellent intro project for an aspiring contributor interested in how > threads work in CPython (what this module does is conceptually > simple). It belongs in the core because it's not safe to chase the > tstate chain without holding pystate.c's internal head_mutex lock > (holding the GIL isn't enough -- it's normal practice to call > PyThreadState_Delete() while not holding the GIL). > > I'd do it myself (and maybe I will anyway), but this really would make > a good (finite; conceptually simple) project for someone who wants to > gain Python developer experience. Since I started this, I might as well finish it. I do have some Python developer experience (hey, I even voted for comp.lang.python back when...) but not in the core interpreter itself. I suspect integrating this feature (let's call it sys._current_frames() for the sake of argument, although it probably belongs in the threads module) in the core is not going to be quite as trivial as you say, as there are potential memory leaks. If the function returns a dictionary, it should probably be a weak dict to avoid a circular reference between the frame of the thread that calls _current_frames and its locals that contain the returned dictionary. But that would also mean the references to other threads' frames are going to be weak, thus not allowing the current thread to inspect their locals and backtraces at will as those weak references may be broken. -- Fazal Majid Email: python-dev@fromemail.com Home: +1 415 359-0918 1111 Jones Apt. 1 Cell: +1 415 244-1337 San Francisco, CA 94109, USA Web: www.majid.info From gward at python.net Mon Mar 7 00:11:26 2005 From: gward at python.net (Greg Ward) Date: Mon Mar 7 00:11:29 2005 Subject: [Python-Dev] Failing tests: marshal, warnings Message-ID: <20050306231126.GA13378@cthulhu.gerg.ca> Anyone else seeing test failures on the 2.4 branch right now? I started seeing this failure: test_warnings test test_warnings failed -- Traceback (most recent call last): File "/scratch/src/python-2.4/Lib/test/test_warnings.py", line 57, in test_warn_specific_category warnings.warn(text, category) File "/scratch/src/python-2.4/Lib/warnings.py", line 57, in warn warn_explicit(message, category, filename, lineno, module, registry) File "/scratch/src/python-2.4/Lib/warnings.py", line 92, in warn_explicit raise message RuntimeWarning: unfiltered RuntimeWarning yesterday, so I "cvs up"'d today to see if it had been fixed. Now, not only is test_warnings failing, so is test_marshal: test_marshal test test_marshal failed -- Traceback (most recent call last): File "/scratch/src/python-2.4/Lib/test/test_marshal.py", line 88, in test_floats got = marshal.load(file(test_support.TESTFN, "rb")) IOError: [Errno 2] No such file or directory: '@test' Naturally, they only fail when running the full test suite. ;-( I sure hope my recent fix to textwrap isn't to blame! I'll fiddle around a bit in my working dir and see if I can figure out what's going on, but I'd like to know if I'm the only one seeing these problems... Greg -- Greg Ward http://www.gerg.ca/ Never underestimate the power of human stupidity. From bac at OCF.Berkeley.EDU Mon Mar 7 00:43:52 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Mar 7 00:44:02 2005 Subject: [Python-Dev] Failing tests: marshal, warnings In-Reply-To: <20050306231126.GA13378@cthulhu.gerg.ca> References: <20050306231126.GA13378@cthulhu.gerg.ca> Message-ID: <422B95B8.5060307@ocf.berkeley.edu> Greg Ward wrote: > Anyone else seeing test failures on the 2.4 branch right now? I started > seeing this failure: > Both are passing for me on OS X 10.3.8 w/ a fresh cvs up as of 15:41 PT. But I am getting failures for test_socket (ignoring the usual test__locale failure on OS X): ====================================================================== FAIL: testHostnameRes (__main__.GeneralModuleTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "Lib/test/test_socket.py", line 273, in testHostnameRes self.fail("Error testing host resolution mechanisms.") AssertionError: Error testing host resolution mechanisms. Don't know if this is an "OS X" issue or a "my system" issue. -Brett From gward at python.net Mon Mar 7 01:28:50 2005 From: gward at python.net (Greg Ward) Date: Mon Mar 7 01:28:52 2005 Subject: [Python-Dev] Failing tests: marshal, warnings In-Reply-To: <20050306231126.GA13378@cthulhu.gerg.ca> References: <20050306231126.GA13378@cthulhu.gerg.ca> Message-ID: <20050307002850.GC9397@cthulhu.gerg.ca> On 06 March 2005, I said: > Anyone else seeing test failures on the 2.4 branch right now? I started > seeing this failure: > > test_warnings > test test_warnings failed -- Traceback (most recent call last): > File "/scratch/src/python-2.4/Lib/test/test_warnings.py", line 57, in test_warn_specific_category > warnings.warn(text, category) > File "/scratch/src/python-2.4/Lib/warnings.py", line 57, in warn > warn_explicit(message, category, filename, lineno, module, registry) > File "/scratch/src/python-2.4/Lib/warnings.py", line 92, in warn_explicit > raise message > RuntimeWarning: unfiltered RuntimeWarning [...] > Naturally, they only fail when running the full test suite. OK, I've narrowed it down: test_warnings fails when run after test_descr: $ ./python -E Lib/test/regrtest.py test_descr test_warnings test_descr test_warnings test test_warnings failed -- Traceback (most recent call last): File "/scratch/src/python-2.4/Lib/test/test_warnings.py", line 57, in test_warn_specific_category warnings.warn(text, category) File "/scratch/src/python-2.4/Lib/warnings.py", line 57, in warn warn_explicit(message, category, filename, lineno, module, registry) File "/scratch/src/python-2.4/Lib/warnings.py", line 92, in warn_explicit raise message RuntimeWarning: unfiltered RuntimeWarning Does this ring alarm bells with anyone? Greg -- Greg Ward http://www.gerg.ca/ Gee, I feel kind of LIGHT in the head now, knowing I can't make my satellite dish PAYMENTS! From gward at python.net Mon Mar 7 01:56:23 2005 From: gward at python.net (Greg Ward) Date: Mon Mar 7 01:56:29 2005 Subject: [Python-Dev] Failing tests: marshal, warnings In-Reply-To: <20050307002850.GC9397@cthulhu.gerg.ca> References: <20050306231126.GA13378@cthulhu.gerg.ca> <20050307002850.GC9397@cthulhu.gerg.ca> Message-ID: <20050307005623.GD9397@cthulhu.gerg.ca> On 06 March 2005, I said: > OK, I've narrowed it down: test_warnings fails when run after > test_descr: Raymond Hettinger, step right up! You're the next contestant on The Tests Are Failing! Your recent checkins include... working file: Lib/test/test_descr.py; sticky tag: release24-maint revision: 1.202.2.1 date: 2005/03/03 16:55:53; author: rhettinger; lines: +13 -0 SF bug #1155938: Missing None check for __init__(). ---------------------------- revision: 1.202.2.2 date: 2005/03/04 04:47:04; author: rhettinger; lines: +7 -1 Convert "__init__ should return None" from an exception to a warning. If I revert test_descr.py to 1.202 (the branch point for 2.4), then running ./python -E Lib/test/regrtest.py test_descr test_warnings works just fine. If I revert to 1.202.2.1, then test_descr fails, but test_warnings passes. If I update to the latest, 1.202.2.2, then test_desc passes, but test_warnings fails. [...a few minutes of tinkering and reading up on warning filters...] A-ha! I get it. There are two mistakes in test_descr.py:test_init(): lack of "finally" clause, and failure to make a copy of warnings.filters. This patch fixes both: """ --- Lib/test/test_descr.py 4 Mar 2005 04:47:04 -0000 1.202.2.2 +++ Lib/test/test_descr.py 7 Mar 2005 00:54:00 -0000 @@ -3973,15 +3973,18 @@ def __init__(self): return 10 - oldfilters = warnings.filters - warnings.filterwarnings("error", category=RuntimeWarning) + oldfilters = warnings.filters[:] try: - Foo() - except RuntimeWarning: pass - else: - raise TestFailed, "did not test __init__() for None return" - warnings.filters = oldfilters + warnings.filterwarnings("error", category=RuntimeWarning) + try: + Foo() + except RuntimeWarning: + pass + else: + raise TestFailed, "did not test __init__() for None return" + finally: + warnings.filters = oldfilters def test_main(): """ I'll check this in and merge to the trunk once I see all tests passing. Greg -- Greg Ward http://www.gerg.ca/ I hope something GOOD came in the mail today so I have a REASON to live!! From bac at OCF.Berkeley.EDU Mon Mar 7 02:14:22 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Mar 7 02:14:30 2005 Subject: [Python-Dev] Failing tests: marshal, warnings In-Reply-To: <20050307005623.GD9397@cthulhu.gerg.ca> References: <20050306231126.GA13378@cthulhu.gerg.ca> <20050307002850.GC9397@cthulhu.gerg.ca> <20050307005623.GD9397@cthulhu.gerg.ca> Message-ID: <422BAAEE.8020308@ocf.berkeley.edu> Greg Ward wrote: [SNIP] > A-ha! I get it. There are two mistakes in test_descr.py:test_init(): > lack of "finally" clause, and failure to make a copy of > warnings.filters. This patch fixes both: > > """ > --- Lib/test/test_descr.py 4 Mar 2005 04:47:04 -0000 1.202.2.2 > +++ Lib/test/test_descr.py 7 Mar 2005 00:54:00 -0000 > @@ -3973,15 +3973,18 @@ > def __init__(self): > return 10 > > - oldfilters = warnings.filters > - warnings.filterwarnings("error", category=RuntimeWarning) > + oldfilters = warnings.filters[:] > try: > - Foo() > - except RuntimeWarning: > pass > - else: > - raise TestFailed, "did not test __init__() for None return" > - warnings.filters = oldfilters > + warnings.filterwarnings("error", category=RuntimeWarning) > + try: > + Foo() > + except RuntimeWarning: > + pass > + else: > + raise TestFailed, "did not test __init__() for None return" > + finally: > + warnings.filters = oldfilters > > > def test_main(): > """ > > I'll check this in and merge to the trunk once I see all tests passing. > Well, I just checked in the list copy fix, so you only have to worry about adding the 'finally' clause to the 'try' statement. -Brett From bac at OCF.Berkeley.EDU Mon Mar 7 02:15:44 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Mar 7 02:15:51 2005 Subject: [Python-Dev] Failing tests: marshal, warnings In-Reply-To: <422BAAEE.8020308@ocf.berkeley.edu> References: <20050306231126.GA13378@cthulhu.gerg.ca> <20050307002850.GC9397@cthulhu.gerg.ca> <20050307005623.GD9397@cthulhu.gerg.ca> <422BAAEE.8020308@ocf.berkeley.edu> Message-ID: <422BAB40.1030808@ocf.berkeley.edu> Brett C. wrote: > Greg Ward wrote: > [SNIP] > >> A-ha! I get it. There are two mistakes in test_descr.py:test_init(): >> lack of "finally" clause, and failure to make a copy of >> warnings.filters. This patch fixes both: >> >> """ >> --- Lib/test/test_descr.py 4 Mar 2005 04:47:04 -0000 1.202.2.2 >> +++ Lib/test/test_descr.py 7 Mar 2005 00:54:00 -0000 >> @@ -3973,15 +3973,18 @@ >> def __init__(self): >> return 10 >> >> - oldfilters = warnings.filters >> - warnings.filterwarnings("error", category=RuntimeWarning) >> + oldfilters = warnings.filters[:] >> try: >> - Foo() >> - except RuntimeWarning: >> pass >> - else: >> - raise TestFailed, "did not test __init__() for None return" >> - warnings.filters = oldfilters >> + warnings.filterwarnings("error", category=RuntimeWarning) >> + try: >> + Foo() >> + except RuntimeWarning: >> + pass >> + else: >> + raise TestFailed, "did not test __init__() for None return" >> + finally: >> + warnings.filters = oldfilters >> >> >> def test_main(): >> """ >> >> I'll check this in and merge to the trunk once I see all tests passing. >> > > Well, I just checked in the list copy fix, so you only have to worry > about adding the 'finally' clause to the 'try' statement. > nm, the commit failed because Greg beat me to the checkin by like a second. -Brett From gward at python.net Mon Mar 7 02:18:47 2005 From: gward at python.net (Greg Ward) Date: Mon Mar 7 02:18:51 2005 Subject: [Python-Dev] Failing tests: marshal, warnings In-Reply-To: <20050307005623.GD9397@cthulhu.gerg.ca> References: <20050306231126.GA13378@cthulhu.gerg.ca> <20050307002850.GC9397@cthulhu.gerg.ca> <20050307005623.GD9397@cthulhu.gerg.ca> Message-ID: <20050307011846.GE9397@cthulhu.gerg.ca> On 06 March 2005, I said: > I'll check this in and merge to the trunk once I see all tests passing. Checked in on 2.4 branch. Not merged to trunk since Raymond hasn't merged his stuff to the trunk yet. Greg -- Greg Ward http://www.gerg.ca/ I repeat myself when under stress I repeat myself when under stress I repeat--- From ta-meyer at ihug.co.nz Mon Mar 7 03:07:44 2005 From: ta-meyer at ihug.co.nz (Tony Meyer) Date: Mon Mar 7 03:07:50 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-01 through 2005-02-14[draft] Message-ID: Somewhat slower, but here are two more threads from me (email is mostly a weekday thing for me, and the last few days were full of sun, wine, food and jazz. Well, and work. But working with sun, wine, food and jazz, so it's hard to complain too much). Feedback will not be ignored :) -------------------------------------- More licensing issues - redistribution -------------------------------------- As most people know, one of the major changes between the Windows builds of Python 2.3 and 2.4 is that 2.4 is built with VC7, rather than VC6. One of the consequences of this change is that 2.4 links with the Microsoft DLL msvcr71.dll, which only some people have, rather than msvcr.dll, which pretty much all Windows users have. The Windows Python 2.4 distribution installs msvcr71.dll, so it's there when needed. However, those building frozen applications (e.g. with py2exe) need to ensure that their users have msvcr71.dll. After going through the EULA's for both the commercial and free-to-use Microsoft compilers, it seems that redistributing mscvr71.dll is acceptable, if the re-distributor owns a copy of the commercial (not free) compiler, includes an EULA agreement in one of various forms (e.g. 'click-wrap'), and follows various other minor conditions (note that just about every message in this thread contains "IANAL, but"). This leaves those without VC7 unable to redistribute msvcr71, unless, as some suggested, distributing a frozen Python application can be considered as redistributing Python (and the various other minor conditions are followed). In an interesting twist, it appears that the official Windows Python 2.4 distribution is in breach of the EULA, as a 'click-wrap' license is required, and is not present. This element of the thread died without reaching a conclusion, however. If you *are* a lawyer (with expertise in this area), and would like to comment, then please do! Contributing threads: - `Is msvcr71.dll re-redistributable?`__ ---------------------------------- Avoiding signs in memory addresses ---------------------------------- Troels Walsted Hansen pointed out that builtin_id() can return a negative number in Python 2.4 (and can generate a warning in 2.3). Some 2.3 modules (but no 2.4 ones) have code to work around this, but Troels suggested that a better solution would be to simply have builtin_id() return an unsigned long integer. The consensus was that this would be a good idea, although nothing has been checked in yet, and so this will probably stagnate without someone submitting a patch (or at least a bug report). Contributing threads: - `builtin_id() returns negative numbers`__ From pje at telecommunity.com Mon Mar 7 03:34:00 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Mar 7 03:31:08 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? In-Reply-To: <9c074500619cad8ce1824f940e03ad87@fromemail.com> Message-ID: <5.1.1.6.0.20050306213134.036b7310@mail.telecommunity.com> At 02:42 PM 3/6/05 -0800, Fazal Majid wrote: >I suspect integrating this feature (let's call it sys._current_frames() >for the sake of argument, although it probably belongs in the threads >module) in the core is not going to be quite as trivial as you say, as >there are potential memory leaks. If the function returns a dictionary, it >should probably be a weak dict to avoid a circular reference between the >frame of the thread that calls _current_frames and its locals that contain >the returned dictionary. Given the primary use case as a debugging tool, I don't think the circularity will have any significant problems. It would be simpler to just document that a caller should do this: try: framedict = sys._current_frames() # do stuff here finally: framedict = None # break the cycle, allowing GC From t-meyer at ihug.co.nz Mon Mar 7 03:48:21 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Mon Mar 7 03:48:27 2005 Subject: [Python-Dev] python-dev Summary for 2005-02-15 through 2005-02-28[draft] In-Reply-To: Message-ID: > I am not expecting the candidates for taking of the Summaries > to write stuff for this one (although I wouldn't mind it =). In penance for being late with the other ones, here are a summaries for a couple of skipped threads for this period: --------------------------------------- Slow unit tests should be distinguished --------------------------------------- Guido clarified that unit tests should distinguish between "regular" tests and slow ones by use of the unit test 'resource' keys, as a result of Peter ?strand asking for comments about bug #1124637, which complained that test_subprocess is too slow. The suggested solution was to add another resource for subprocess, so that generally a quick version would run, but a longer, more thorough test would run with -uall or -usubprocess. Along the way, it was discovered that the reason that Windows already ran test_subprocess quickly was because there was code special-casing it to be fast. The resource solution was checked in, although Windows was left special-cased. Contributing threads: - `[ python-Bugs-1124637 ] test_subprocess is far too slow (fwd) `__ ----------------------------------- Clarification of the '5 for 1' deal ----------------------------------- It seems that the offer that some python-dev'ers have made to review a patch in exchange for reviews of five (originally ten) other patches is finally being taken up by various people. However, python-dev traffic has increased with patch and bug reviews, and the question was posed whether reviews should be posted in general, or only for this specific deal. The answer is that the comments should also be entered via the SourceForge tracking system, but that a brief message covering a batch (rather than individual) of reviews is acceptable for python-dev, at least for now. New reports should almost never be posted to python-dev, however, and should be entered via the tracking system. This offer isn't official policy, but a reference to it will be added to Brett's summary of the development process. However, people should also remember that it may take developers some time to find time to deal with reviews, and so have patience after posting their review. Contributing threads: - `discourage patch reviews to the list? `__ - `Some old patches `__ - `Five review rule on the /dev/ page? `__ From tim.peters at gmail.com Mon Mar 7 03:49:33 2005 From: tim.peters at gmail.com (Tim Peters) Date: Mon Mar 7 03:49:36 2005 Subject: [Python-Dev] Useful thread project for 2.5? In-Reply-To: <5.1.1.6.0.20050304161554.029f7ec0@mail.telecommunity.com> References: <1f7befae050304130154cd25b7@mail.gmail.com> <5.1.1.6.0.20050304161554.029f7ec0@mail.telecommunity.com> Message-ID: <1f7befae050306184969c677f8@mail.gmail.com> [Phillip J. Eby] > What would you suggest calling it? sys._current_frames(), returning a > dictionary? I don't fight about names -- anything that doesn't make Guido puke works . I channel that sys._current_frames() would be fine. A dict mapping thread id to current thread frame would be lovely! From tim.peters at gmail.com Mon Mar 7 04:02:38 2005 From: tim.peters at gmail.com (Tim Peters) Date: Mon Mar 7 04:02:42 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? In-Reply-To: <9c074500619cad8ce1824f940e03ad87@fromemail.com> References: <9c074500619cad8ce1824f940e03ad87@fromemail.com> Message-ID: <1f7befae0503061902522ab847@mail.gmail.com> [Fazal Majid] > Since I started this, I might as well finish it. I do have some Python > developer experience (hey, I even voted for comp.lang.python back > when...) but not in the core interpreter itself. Cool! WRT your current module, it would need changes to follow Python's C coding style, to check _every_ C API call for an error return, and to grab head_mutex while crawling over the tstate chain. > I suspect integrating this feature (let's call it sys._current_frames() > for the sake of argument, although it probably belongs in the threads > module) I expect that Phillip's thought here is that the sys module already has a _getframe() function, so everyone who knows that would likely expect a new frame-retrieval function to be exposed in sys too. > in the core is not going to be quite as trivial as you say, as there are > potential memory leaks. Good news: I don't think so. > If the function returns a dictionary, it should probably be a weak dict to avoid > a circular reference between the frame of the thread that calls _current_frames > and its locals that contain the returned dictionary. But that would also mean the > references to other threads' frames are going to be weak, thus not allowing the > current thread to inspect their locals and backtraces at will as those weak > references may be broken. dicts and frames both participate in cyclic gc in recent Pythons. For example, you can run this all day in 2.4 and not see memory grow (provided you don't disable cyclic gc): def f(): import sys myframe = sys._getframe() while 1: f() Frames aren't weakly referenceable anyway: >>> import weakref >>> import sys >>> f = sys._getframe() >>> weakref.ref(f) Traceback (most recent call last): File "", line 1, in ? TypeError: cannot create weak reference to 'frame' object From anthony at interlink.com.au Mon Mar 7 04:13:04 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Mar 7 04:13:29 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36 In-Reply-To: References: Message-ID: <200503071413.05194.anthony@interlink.com.au> Greg, Um, unless I misread this, you added new attributes to the ossaudiodev objects in the 2.4 branch. Please don't do this - this is a new feature, and suddenly people who want to use those new attributes have to either test for version >= 2.4.1, or else do a hasattr test. Please see the bugfix PEP for more rationale on this... Anthony -- Anthony Baxter It's never too late to have a happy childhood. From mwh at python.net Mon Mar 7 09:51:46 2005 From: mwh at python.net (Michael Hudson) Date: Mon Mar 7 09:51:47 2005 Subject: [Python-Dev] Failing tests: marshal, warnings In-Reply-To: <20050307011846.GE9397@cthulhu.gerg.ca> (Greg Ward's message of "Sun, 6 Mar 2005 20:18:47 -0500") References: <20050306231126.GA13378@cthulhu.gerg.ca> <20050307002850.GC9397@cthulhu.gerg.ca> <20050307005623.GD9397@cthulhu.gerg.ca> <20050307011846.GE9397@cthulhu.gerg.ca> Message-ID: <2mvf83q1nh.fsf@starship.python.net> Greg Ward writes: > On 06 March 2005, I said: >> I'll check this in and merge to the trunk once I see all tests passing. > > Checked in on 2.4 branch. Not merged to trunk since Raymond hasn't > merged his stuff to the trunk yet. I don't think that code is going onto HEAD. Cheers, mwh -- Our lecture theatre has just crashed. It will currently only silently display an unexplained line-drawing of a large dog accompanied by spookily flickering lights. -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year) From ncoghlan at iinet.net.au Mon Mar 7 10:01:50 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Mar 7 10:01:58 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <422BD867.601@canterbury.ac.nz> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> <20050302095515.GA14982@craig-wood.com> <4225956A.9060805@iinet.net.au> <42287FAD.70908@iinet.net.au> <422BD867.601@canterbury.ac.nz> Message-ID: <422C187E.90607@iinet.net.au> Greg Ewing wrote: > Is there some reason why Context couldn't invoke the binary > operator methods using binary operators, rather than calling > their methods directly? I had a similar idea, but then I remembered that the whole point of invoking the methods through a specific Context is to override the default context that the binary operators pick up. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From barry at python.org Mon Mar 7 14:57:11 2005 From: barry at python.org (Barry Warsaw) Date: Mon Mar 7 14:57:16 2005 Subject: [Python-Dev] Migrating to subversion In-Reply-To: <422B56F8.9040708@v.loewis.de> References: <422B56F8.9040708@v.loewis.de> Message-ID: <1110203831.362.53.camel@presto.wooz.org> On Sun, 2005-03-06 at 14:16, "Martin v. L?wis" wrote: > I don't know whether anybody has done this before, > but I just tried to run cvs2svn on the Python repository. > The conversion took 7 hours, and the result is now > available at > > http://www.dcl.hpi.uni-potsdam.de/python/branches/ > > Because of the load that the conversion produces on > the machine, I cannot run the entire conversion every > day. Reportedly, cvs2svn is able to run in incremental > mode, but I haven't tried this out yet. I personally have had no success doing this, but the last time I tried was with a fairly old version of svn. BTW, it looks like you just pulled in python/dist right? Did you pull in the trunk? What about nondist, or as Greg mentioned, distutils which is a sibling of the top-level python directory. > - It has imported the CVSROOT directory as well. I don't > know whether this is deliberate/useful. Almost certainly not useful. Even the diff checkin messages are generated with a much simpler script (owing largely to svn being designed to make this kind of thing possible without disgusting hacks). > Anyway, this repository is now online for anonymous read-only > access. Anybody interested in playing with it is welcome to > do so. > For those interested in server side issues: the repository > is an 1.1.1 fsfs repository. The CVS repository consumes > 368MiB; the SVN repository 797MiB. Just curious - why did you choose fsfs instead of the BerkeleyDB backend? Thanks for doing this Martin. I've heard that SF may be offering svn as early as May or June and I would love to see us convert when that's available. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050307/cd9bb9ad/attachment.pgp From martin at v.loewis.de Mon Mar 7 21:01:32 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Mar 7 21:01:38 2005 Subject: [Python-Dev] Migrating to subversion In-Reply-To: <1110203831.362.53.camel@presto.wooz.org> References: <422B56F8.9040708@v.loewis.de> <1110203831.362.53.camel@presto.wooz.org> Message-ID: <422CB31C.1030700@v.loewis.de> Barry Warsaw wrote: > I personally have had no success doing this, but the last time I tried > was with a fairly old version of svn. It gives an error message when you try. You then need to interpret the error message, retry, and it gives you another error message. You do this three times, and end up with the command line cvs2svn -q --fs-type=fsfs --encoding=latin1 "--exclude=r16b1|cnri-16-start|descr-branch|release152p1-patches|after-call-reorg|r22a1|before-call-reorg|release16|r161|r161-branch|r16a2|release152p2" -s py.svn.new python > BTW, it looks like you just pulled in python/dist right? Did you pull > in the trunk? What about nondist, or as Greg mentioned, distutils which > is a sibling of the top-level python directory. I posted the wrong URL. The right one is http://www.dcl.hpi.uni-potsdam.de/python/ It has distutils right in http://www.dcl.hpi.uni-potsdam.de/python/trunk/distutils/ > Just curious - why did you choose fsfs instead of the BerkeleyDB > backend? I had hoped that it would consume less disk space - but I really should try with bdb again. In our operational repositories, I have now migrated everything to fsfs because it is so much more friendly to backups. You can backup the on-disk state, and trust that is consistent. With bdb, you need to use hot-backup.py or some such, and this gives you an entire copy which then goes into all incremental backups. With fsfs, the incremental backups really pickup new commits only. (for the Python svn, it doesn't matter much, since that is excluded from backup, anyway) > Thanks for doing this Martin. I've heard that SF may be offering svn as > early as May or June and I would love to see us convert when that's > available. So do I. However, I now believe that it is unlikely that SF will provide automatic conversion (or the automatic conversion will be useless/fail); instead, we likely need to provide our hand-tuned conversion. I'll keep collecting issues/complaints about this specific conversion, and try to integrate them if I can. Regards, Martin From martin at v.loewis.de Mon Mar 7 21:07:51 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Mar 7 21:07:54 2005 Subject: [Python-Dev] Migrating to subversion In-Reply-To: <20050306215711.GA9397@cthulhu.gerg.ca> References: <422B56F8.9040708@v.loewis.de> <20050306215711.GA9397@cthulhu.gerg.ca> Message-ID: <422CB497.4000705@v.loewis.de> Greg Ward wrote: > Presumably for Python's repository, this would work: > > cvs2svn -s /home/svn/python /home/cvs/python/python > > ...except, umm, isn't distutils a separate top-level directory in the > Python repository or something? Ok. Removing the CVSROOT before the conversion (from CVS) or after the conversion (from svn) would probably work. OTOH, I wonder whether the distutils CVS needs to be converted at all, or whether it would be sufficient to only migrate the python "module" (in which case your approach would be sufficient). Regards, Martin From skip at pobox.com Mon Mar 7 21:43:01 2005 From: skip at pobox.com (Skip Montanaro) Date: Mon Mar 7 21:39:09 2005 Subject: [Python-Dev] Urllib code or the docs appear wrong Message-ID: <16940.48341.344618.535840@montanaro.dyndns.org> It seems to me that either urllib's docs are wrong or its code is wrong w.r.t. how the User-agent header is handled. In part, the docs say: By default, the URLopener class sends a User-Agent: header of "urllib/VVV", where VVV is the urllib version number. Applications can define their own User-Agent: header by subclassing URLopener or FancyURLopener and setting the instance attribute version to an appropriate string value before the open() method is called. Looking at the code it seems to me that the User-agent header is fixed at instantiation time: version = "Python-urllib/%s" % __version__ # Constructor def __init__(self, proxies=None, **x509): ... self.addheaders = [('User-agent', self.version)] ... and that when open_http() is called, it simply calls putheader() for each element of addheaders. Setting the version instance attribute will have no effect. If I managed to add another User-agent header before open_http() was called, the request would wind up with two copies which is probably not desirable either. I can see a couple ways around this: * Just change the docs to match the current implementation. Users wishing to override the User-agent header would then have to subclass FancyURLopener and set the version class attribute. * Defer decisions about the value of the User-agent until open_http() is called. It appears the OpenerDirector class in urllib2 has a similar "early binding" problem. I don't particularly care how this is solved, but it appears to need solving. Skip From steve at holdenweb.com Mon Mar 7 21:53:15 2005 From: steve at holdenweb.com (Steve Holden) Date: Mon Mar 7 21:53:42 2005 Subject: [Python-Dev] Re: Documentation for __new__ In-Reply-To: <20050305163942.GA3242@cthulhu.gerg.ca> References: <4229304E.5030609@iinet.net.au> <20050305163942.GA3242@cthulhu.gerg.ca> Message-ID: <422CBF3B.9040506@holdenweb.com> Greg Ward wrote: > On 05 March 2005, Nick Coghlan said: > >>Steven Bethard has put together some text to add __new__ to the list of >>Basic Customisation methods in the language reference. Would one of the >>documentation folks care to take a look at it? > > > I've tried to tighten up the text there and hopefully make it a smidgeon > clearer and more accurate. Here's my best effort: > > __new__(cls[, ...]) > > Called to create a new instance of class 'cls'. __new__() > is a static method (special-cased so you need not declare it > as such) that takes the class to create an instance of as > the first argument. The remaining arguments are those > passed to the object constructor expression. The return > value of __new__() should be the new object instance. > Just to offer alternatives: __new__(cls[, ...]) __new__() is a static method (special-cased so you need not declare it as such) whose fist argumen is the class of which a new instance is required. The remaining arguments are those passed to the object constructor expression (the call to the class). The return value of __new__() should be the new instance object, which is not constrained to be of type 'cls'. > Typical usage is to create a new instance of the class by > invoking the superclass's __new__() method using > "super(currentclass, cls).__new__([...])" with appropriate > arguments, modifying the returned instance if necessary, and > then returning it. If the returned value is an instance of > 'cls', its __init__() will be invoked like > "__init__(self[, ...])", where the extra arguments are the > same as were passed to __new__(). > Typical usage creates a new instance of the required class by invoking the superclass's __new__() method using "super(currentclass, cls).__new__(...)" with appropriate argumnents and then modifying the newly- created instance as necessary before returning it. If __new__() returns an instance of 'cls' then the new instance's __init__() method will be called with the instance itself as the first argument and the remaining arguments being the second and subsequent arguments to __new__(). > You do need not to return an instance of 'cls', but if you > do not, the new instance's __init__() method will not be > invoked. > If __new__() does not return an instance of 'cls' then the new instance's __init__() method will not be invoked. > __new__() is intended mainly to allow subclasses of > immutable types (like int, str, or tuple) to customize > instance creation. > > Feedback welcome. Has anyone volunteered to render this in LaTeX yet? > If not, I might. > > Greg I decided some time ago that documenting Python in LaTex wasn't my forte ... regards Steve From gward at python.net Tue Mar 8 02:06:37 2005 From: gward at python.net (Greg Ward) Date: Tue Mar 8 02:06:39 2005 Subject: [Python-Dev] Re: Documentation for __new__ In-Reply-To: <422CBF3B.9040506@holdenweb.com> References: <4229304E.5030609@iinet.net.au> <20050305163942.GA3242@cthulhu.gerg.ca> <422CBF3B.9040506@holdenweb.com> Message-ID: <20050308010637.GA28943@cthulhu.gerg.ca> On 07 March 2005, Steve Holden said: > Just to offer alternatives: Cool, I liked some of your changes. I'm happy with this doc change. Will checkin Doc/ref/ref3.tex on 2.4 branch and merge to trunk shortly. Greg -- Greg Ward http://www.gerg.ca/ "What do you mean -- a European or an African swallow?" From gward at python.net Tue Mar 8 02:14:43 2005 From: gward at python.net (Greg Ward) Date: Tue Mar 8 02:14:46 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36 In-Reply-To: <200503071413.05194.anthony@interlink.com.au> References: <200503071413.05194.anthony@interlink.com.au> Message-ID: <20050308011443.GB28943@cthulhu.gerg.ca> On 07 March 2005, Anthony Baxter said: > Um, unless I misread this, you added new attributes to the ossaudiodev > objects in the 2.4 branch. Please don't do this - this is a new > feature, and suddenly people who want to use those new attributes have > to either test for version >= 2.4.1, or else do a hasattr test. Please > see the bugfix PEP for more rationale on this... D'ohhh -- busted! My only excuse is that the lack of these attributes was originally filed as a bug report, and I suddenly realized "oops! new attributes == feature request" just as I was checking the change in on 2.4. I'll revert the change on 2.4 if you (or anyone) really wants me to. Otherwise, I'd rather leave it as-is and go fix more bugs. Greg -- Greg Ward http://www.gerg.ca/ I don't believe there really IS a GAS SHORTAGE.. I think it's all just a BIG HOAX on the part of the plastic sign salesmen -- to sell more numbers!! From gward at python.net Tue Mar 8 02:16:59 2005 From: gward at python.net (Greg Ward) Date: Tue Mar 8 02:17:01 2005 Subject: [Python-Dev] Migrating to subversion In-Reply-To: <422CB497.4000705@v.loewis.de> References: <422B56F8.9040708@v.loewis.de> <20050306215711.GA9397@cthulhu.gerg.ca> <422CB497.4000705@v.loewis.de> Message-ID: <20050308011659.GC28943@cthulhu.gerg.ca> On 07 March 2005, "Martin v. L?wis" said: > OTOH, I wonder whether the distutils CVS needs to be converted at all, > or whether it would be sufficient to only migrate the python "module" > (in which case your approach would be sufficient). The last time I looked (late 2000), there was useful content in /distutils that was not in /python/dist/src/Lib/distutils. Greg -- Greg Ward http://www.gerg.ca/ I can never remember how to spell "mnemonic". From gward at python.net Tue Mar 8 02:23:32 2005 From: gward at python.net (Greg Ward) Date: Tue Mar 8 02:23:34 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? In-Reply-To: <9c074500619cad8ce1824f940e03ad87@fromemail.com> References: <9c074500619cad8ce1824f940e03ad87@fromemail.com> Message-ID: <20050308012332.GD28943@cthulhu.gerg.ca> On 06 March 2005, Fazal Majid said: > Since I started this, I might as well finish it. I do have some Python > developer experience (hey, I even voted for comp.lang.python back > when...) but not in the core interpreter itself. What would be *really* spiffy is to provide a way for externally-triggered thread dumps. This is one of my top two Java features [1]. The way this works in Java is a bit awkward -- "kill -QUIT" the Java process and it writes a traceback for every running thread to stdout -- but it works. Something similar ought to be possible for Python, although optional (because Python apps can handle signals themselves, unlike Java apps). It could be as simple as this: calling sys.enablethreaddump(signal=signal.SIGQUIT) from the program enables externally-triggered thread dumps via the specified signal. Greg [1] The other is compiler recognition of "@deprecated" in doc comments. -- Greg Ward http://www.gerg.ca/ Think honk if you're a telepath. From aahz at pythoncraft.com Tue Mar 8 03:10:47 2005 From: aahz at pythoncraft.com (Aahz) Date: Tue Mar 8 03:10:49 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36 In-Reply-To: <20050308011443.GB28943@cthulhu.gerg.ca> References: <200503071413.05194.anthony@interlink.com.au> <20050308011443.GB28943@cthulhu.gerg.ca> Message-ID: <20050308021047.GA8957@panix.com> On Mon, Mar 07, 2005, Greg Ward wrote: > > D'ohhh -- busted! My only excuse is that the lack of these attributes > was originally filed as a bug report, and I suddenly realized "oops! new > attributes == feature request" just as I was checking the change in on > 2.4. > > I'll revert the change on 2.4 if you (or anyone) really wants me to. > Otherwise, I'd rather leave it as-is and go fix more bugs. Please revert. I've spent more time than I'd like dealing with the introduction of booleans in Python 2.2, and helping other people avoid similar problems seems like a Good Thing to me. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From pje at telecommunity.com Tue Mar 8 03:22:42 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Mar 8 03:20:00 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? In-Reply-To: <20050308012332.GD28943@cthulhu.gerg.ca> References: <9c074500619cad8ce1824f940e03ad87@fromemail.com> <9c074500619cad8ce1824f940e03ad87@fromemail.com> Message-ID: <5.1.1.6.0.20050307211533.027a6e10@mail.telecommunity.com> At 08:23 PM 3/7/05 -0500, Greg Ward wrote: >On 06 March 2005, Fazal Majid said: > > Since I started this, I might as well finish it. I do have some Python > > developer experience (hey, I even voted for comp.lang.python back > > when...) but not in the core interpreter itself. > >What would be *really* spiffy is to provide a way for >externally-triggered thread dumps. This is one of my top two Java >features [1]. The way this works in Java is a bit awkward -- >"kill -QUIT" the Java process and it writes a traceback for every >running thread to stdout -- but it works. Something similar ought to be >possible for Python, although optional (because Python apps can handle >signals themselves, unlike Java apps). > >It could be as simple as this: calling > > sys.enablethreaddump(signal=signal.SIGQUIT) > >from the program enables externally-triggered thread dumps via the >specified signal. Couldn't this just be done with traceback.print_stack(), given the _current_frames() facility? About the only real problem with it is that the other threads can keep running while you're trying to print the stack dumps. I suppose you could set the check interval really high and then restore it afterwards as a sneaky way of creating a critical section. Unfortunately, there's no getcheckinterval(). Which reminds me, btw, it would be nice while we're adding more execution control functions to have a way to get the current trace hook and profiling hook, not to mention ways to set them on non-current threads. You can do these things from C of course; I mean accessible as part of the Python API. From tim.peters at gmail.com Tue Mar 8 04:13:54 2005 From: tim.peters at gmail.com (Tim Peters) Date: Tue Mar 8 04:13:56 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? In-Reply-To: <5.1.1.6.0.20050307211533.027a6e10@mail.telecommunity.com> References: <9c074500619cad8ce1824f940e03ad87@fromemail.com> <20050308012332.GD28943@cthulhu.gerg.ca> <5.1.1.6.0.20050307211533.027a6e10@mail.telecommunity.com> Message-ID: <1f7befae050307191354957c84@mail.gmail.com> [Greg Ward] >> What would be *really* spiffy is to provide a way for >> externally-triggered thread dumps. This is one of my top two Java >> features [1]. The way this works in Java is a bit awkward -- >> "kill -QUIT" the Java process and it writes a traceback for every >> running thread to stdout -- but it works. Something similar ought to be >> possible for Python, although optional (because Python apps can handle >> signals themselves, unlike Java apps). >> >> It could be as simple as this: calling >> >> sys.enablethreaddump(signal=signal.SIGQUIT) >> >> from the program enables externally-triggered thread dumps via the >> specified signal. See the link in my original post to Florent's Zope deadlock debugger. Things like the above are easy enough to create _given_ a bit of C code in the core to build on. [Phillip J. Eby] > Couldn't this just be done with traceback.print_stack(), given the > _current_frames() facility? Right. > About the only real problem with it is that the other threads can keep > running while you're trying to print the stack dumps. I don't know that it matters. If you're debugging a deadlocked thread, its stack isn't going to change. If you're trying to find out where unexpected time is getting swallowed, statements in the offending loop(s) are still going to show up in the stack trace. > I suppose you could set the check interval really high and then restore it > afterwards as a sneaky way of creating a critical section. Unfortunately, there's > no getcheckinterval(). sys.getcheckinterval() was added in Python 2.3. > Which reminds me, btw, it would be nice while we're adding more execution > control functions to have a way to get the current trace hook and profiling > hook, not to mention ways to set them on non-current threads. You can do > these things from C of course; I mean accessible as part of the Python API. Huh. It didn't remind me of those at all . From gvanrossum at gmail.com Tue Mar 8 04:32:25 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue Mar 8 04:32:29 2005 Subject: [Python-Dev] Urllib code or the docs appear wrong In-Reply-To: <16940.48341.344618.535840@montanaro.dyndns.org> References: <16940.48341.344618.535840@montanaro.dyndns.org> Message-ID: On Mon, 7 Mar 2005 14:43:01 -0600, Skip Montanaro wrote: > > It seems to me that either urllib's docs are wrong or its code is wrong > w.r.t. how the User-agent header is handled. In part, the docs say: > > By default, the URLopener class sends a User-Agent: header of > "urllib/VVV", where VVV is the urllib version number. Applications can > define their own User-Agent: header by subclassing URLopener or > FancyURLopener and setting the instance attribute version to an > appropriate string value before the open() method is called. > > Looking at the code it seems to me that the User-agent header is fixed at > instantiation time: > > version = "Python-urllib/%s" % __version__ > > # Constructor > def __init__(self, proxies=None, **x509): > ... > self.addheaders = [('User-agent', self.version)] > ... > > and that when open_http() is called, it simply calls putheader() for each > element of addheaders. Setting the version instance attribute will have no > effect. If I managed to add another User-agent header before open_http() > was called, the request would wind up with two copies which is probably not > desirable either. > > I can see a couple ways around this: > > * Just change the docs to match the current implementation. Users > wishing to override the User-agent header would then have to subclass > FancyURLopener and set the version class attribute. > > * Defer decisions about the value of the User-agent until open_http() is > called. > > It appears the OpenerDirector class in urllib2 has a similar "early binding" > problem. > > I don't particularly care how this is solved, but it appears to need > solving. Good catch. I propose fixing the docs; "fixing" the code after so many year of being out of sync with the doc might cause more surprises. (Unless you can find evidence in CVS that this *used* to work and someone introduced an unfortunate optimization that disabled the feature.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony at interlink.com.au Tue Mar 8 07:41:19 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Mar 8 07:42:12 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36 In-Reply-To: <20050308011443.GB28943@cthulhu.gerg.ca> References: <200503071413.05194.anthony@interlink.com.au> <20050308011443.GB28943@cthulhu.gerg.ca> Message-ID: <200503081741.20335.anthony@interlink.com.au> On Tuesday 08 March 2005 12:14, Greg Ward wrote: > D'ohhh -- busted! My only excuse is that the lack of these attributes > was originally filed as a bug report, and I suddenly realized "oops! new > attributes == feature request" just as I was checking the change in on > 2.4. > > I'll revert the change on 2.4 if you (or anyone) really wants me to. > Otherwise, I'd rather leave it as-is and go fix more bugs. I really would like to see it reverted, please. Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From martin at v.loewis.de Tue Mar 8 10:17:58 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Mar 8 10:18:01 2005 Subject: [Python-Dev] os.access and Unicode Message-ID: <422D6DC6.5010001@v.loewis.de> Apparently, os.access was forgotten when the file system encoding was introduced in Python 2.2, and then it was again forgotten in PEP 277. I've now fixed it in the trunk (posixmodule.c:2.334), and I wonder whether this is a backport candidate. People who try to invoke os.access with a non-ASCII filename on non-NT+ systems will get a UnicodeError; with the patch, the operation will succeed (assuming the characters are all supported in the file system encoding). Should this be backported? Regards, Martin From mwh at python.net Tue Mar 8 11:03:19 2005 From: mwh at python.net (Michael Hudson) Date: Tue Mar 8 11:03:23 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? In-Reply-To: <5.1.1.6.0.20050307211533.027a6e10@mail.telecommunity.com> (Phillip J. Eby's message of "Mon, 07 Mar 2005 21:22:42 -0500") References: <9c074500619cad8ce1824f940e03ad87@fromemail.com> <9c074500619cad8ce1824f940e03ad87@fromemail.com> <5.1.1.6.0.20050307211533.027a6e10@mail.telecommunity.com> Message-ID: <2mr7iqpi8o.fsf@starship.python.net> "Phillip J. Eby" writes: > Which reminds me, btw, it would be nice while we're adding more > execution control functions to have a way to get the current trace > hook and profiling hook, Well, there's the f_trace member of frame objects, but I honestly can't remember what it's for... > not to mention ways to set them on non-current threads. You can do > these things from C of course; I mean accessible as part of the > Python API. Again, given sys._current_frames(), this shouldn't be very hard. Cheers, mwh -- And then the character-only displays went away (leading to increasingly silly graphical effects and finally to ads on web pages). -- John W. Baxter, comp.lang.python From mcherm at mcherm.com Tue Mar 8 15:55:52 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Mar 8 15:55:56 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? Message-ID: <20050308065552.ef6k5vbzrln3sw8@mcherm.com> Greg writes: > This is one of my top two Java features [1]. ... > [1] The other is compiler recognition of "@deprecated" in doc comments. Hmm... what a good idea! Let's see... what exactly does "@deprecated" DO in Java? Why it causes the compiler to emit a warning if you use the function in question. Not a bad idea. Amusingly enough, the syntax is even the same in Python: ===== code ===== import warnings def deprecated(func): """This is a decorator which can be used to mark functions as deprecated. It will result in a warning being emmitted when the function is used.""" def newFunc(*args, **kwargs): warnings.warn("Call to deprecated function.") return func(*args, **kwargs) return newFunc ===== example ===== >>> @deprecated def func(x,y): return x + y >>> func(3,4) Warning (from warnings module): File "g:/Documents/Personal/Python/deprecated.py", line 10 warnings.warn("Call to deprecated function.") UserWarning: Call to deprecated function. 7 So... shall I go add this to the cookbook? -- Michael Chermside From pje at telecommunity.com Tue Mar 8 16:56:40 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Mar 8 16:54:00 2005 Subject: [Python-Dev] Re: Useful thread project for 2.5? In-Reply-To: <2mr7iqpi8o.fsf@starship.python.net> References: <5.1.1.6.0.20050307211533.027a6e10@mail.telecommunity.com> <9c074500619cad8ce1824f940e03ad87@fromemail.com> <9c074500619cad8ce1824f940e03ad87@fromemail.com> <5.1.1.6.0.20050307211533.027a6e10@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050308105522.02115020@mail.telecommunity.com> At 10:03 AM 3/8/05 +0000, Michael Hudson wrote: >"Phillip J. Eby" writes: > > > Which reminds me, btw, it would be nice while we're adding more > > execution control functions to have a way to get the current trace > > hook and profiling hook, > >Well, there's the f_trace member of frame objects, but I honestly >can't remember what it's for... It sets the *local* trace function, which is only active if a *global* trace function is set for the thread. The global trace function is the thing you can't get at from Python. From bac at OCF.Berkeley.EDU Tue Mar 8 19:58:45 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Mar 8 19:58:53 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <422D6DC6.5010001@v.loewis.de> References: <422D6DC6.5010001@v.loewis.de> Message-ID: <422DF5E5.1000406@ocf.berkeley.edu> Martin v. L?wis wrote: > Apparently, os.access was forgotten when the file system encoding > was introduced in Python 2.2, and then it was again forgotten in > PEP 277. > > I've now fixed it in the trunk (posixmodule.c:2.334), and I wonder > whether this is a backport candidate. People who try to invoke > os.access with a non-ASCII filename on non-NT+ systems will get > a UnicodeError; with the patch, the operation will succeed > (assuming the characters are all supported in the file system > encoding). > Should this be backported? > If there was no other way to get os.access-like functionality, I would say it should be backported. But since there are other ways to figure out everything that os.access can tell you I say don't backport and amend the docs to state it is not Unicode-aware. If one was adventurous enough the docs could even include other ways to get the same info when Unicode had to be used. -Brett From mal at egenix.com Tue Mar 8 21:50:11 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Mar 8 21:50:14 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <422DF5E5.1000406@ocf.berkeley.edu> References: <422D6DC6.5010001@v.loewis.de> <422DF5E5.1000406@ocf.berkeley.edu> Message-ID: <422E1003.50804@egenix.com> Brett C. wrote: > Martin v. L?wis wrote: > >> Apparently, os.access was forgotten when the file system encoding >> was introduced in Python 2.2, and then it was again forgotten in >> PEP 277. >> >> I've now fixed it in the trunk (posixmodule.c:2.334), and I wonder >> whether this is a backport candidate. People who try to invoke >> os.access with a non-ASCII filename on non-NT+ systems will get >> a UnicodeError; with the patch, the operation will succeed >> (assuming the characters are all supported in the file system >> encoding). >> Should this be backported? +1; it's a bug, not a new feature. > If there was no other way to get os.access-like functionality, I would > say it should be backported. But since there are other ways to figure > out everything that os.access can tell you I say don't backport and > amend the docs to state it is not Unicode-aware. If one was adventurous > enough the docs could even include other ways to get the same info when > Unicode had to be used. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 08 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From raymond.hettinger at verizon.net Tue Mar 8 23:43:54 2005 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue Mar 8 23:48:14 2005 Subject: [Python-Dev] itemgetter/attrgetter extension Message-ID: <000e01c52430$4eff1fe0$7eb99d8d@oemcomputer> Any objections to extending itemgetter() and attrgetter() to be able to extract multiple fields at a time? # SELECT name, rank, serialnum FROM soldierdata map(attrgetter('name', 'rank', 'serialnum'), soldierdata) # SELECT * FROM soldierdata ORDER BY unit, rank, proficiency sorted(soldierdata, key=attrgetter('unit', 'rank', 'proficiency')) Raymond Hettinger From jimjjewett at gmail.com Wed Mar 9 00:05:05 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed Mar 9 00:05:08 2005 Subject: [Python-Dev] @deprecated (was: Useful thread project for 2.5?) Message-ID: Greg wished for: ... compiler recognition of "@deprecated" in doc comments. Michael Chermside suggested: ===== code ===== import warnings def deprecated(func): """This is a decorator which can be used to mark functions as deprecated. It will result in a warning being emmitted when the function is used.""" def newFunc(*args, **kwargs): warnings.warn("Call to deprecated function.") return func(*args, **kwargs) return newFunc ===== example ===== ... UserWarning: Call to deprecated function. I agree that it should go in the cookbook, but I think you should set the category to a DeprecationWarning and give the offending function's name. (Whether to worry about suppression of calls to *other* deprecated functions -- I think is out of scope for a recipe.) def newFunc(*args, **kwargs): warnings.warn("Call to deprecated function %s" % func.__name__, category=DeprecationWarning) return func(*args, **kwargs) -jJ From python at rcn.com Wed Mar 9 00:09:04 2005 From: python at rcn.com (Raymond Hettinger) Date: Wed Mar 9 00:13:30 2005 Subject: [Python-Dev] @deprecated (was: Useful thread project for 2.5?) In-Reply-To: Message-ID: <000f01c52433$d6f5eb60$7eb99d8d@oemcomputer> > Michael Chermside suggested: > import warnings > > def deprecated(func): > """This is a decorator which can be used to mark functions > as deprecated. It will result in a warning being emmitted > when the function is used.""" > def newFunc(*args, **kwargs): > warnings.warn("Call to deprecated function.") > return func(*args, **kwargs) > return newFunc Decorators like this should preserve information about the underlying function: > def deprecated(func): > """This is a decorator which can be used to mark functions > as deprecated. It will result in a warning being emmitted > when the function is used.""" > def newFunc(*args, **kwargs): > warnings.warn("Call to deprecated function.") > return func(*args, **kwargs) newFunc.__name__ = func.__name__ newFunc.__doc__ = func.__doc__ newFunc.__dict__.update(func.__dict__) > return newFunc Raymond From tdelaney at avaya.com Wed Mar 9 00:19:36 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Wed Mar 9 00:19:38 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> The perennial "how do I remove duplicates from a list" topic came up on c.l.py and in the discussion I mentioned the java 1.5 LinkedHashSet and LinkedHashMap. I'd thought about proposing these before, but couldn't think of where to put them. It was pointed out that the obvious place would be the collections module. For those who don't know, LinkedHashSet and LinkedHashMap are simply hashed sets and maps that iterate in the order that the keys were added to the set/map. I almost invariably use them for the above scenario - removing duplicates without changing order. Does anyone else think it would be worthwhile adding these to collections, or should I just make a cookbook entry? Cheers. Tim Delaney From steven.bethard at gmail.com Wed Mar 9 00:43:34 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed Mar 9 00:43:37 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> Message-ID: Delaney, Timothy C (Timothy) wrote: > The perennial "how do I remove duplicates from a list" topic came up on > c.l.py and in the discussion I mentioned the java 1.5 LinkedHashSet and > LinkedHashMap. I'd thought about proposing these before, but couldn't > think of where to put them. It was pointed out that the obvious place > would be the collections module. > > For those who don't know, LinkedHashSet and LinkedHashMap are simply > hashed sets and maps that iterate in the order that the keys were added > to the set/map. I almost invariably use them for the above scenario - > removing duplicates without changing order. > > Does anyone else think it would be worthwhile adding these to > collections, or should I just make a cookbook entry? I guess I'm -0 on this. Though I was the one that suggested that collections is the right place to put them, I'm not really certain how much we gain by including them. I too would only ever use them for removing duplicates from a list. But if we're trying to provide a solution to this problem, I'd rather see an iterable-friendly one. See a previous thread on this issue[1] where I suggest something like: def filterdups(iterable): seen = set() for item in iterable: if item not in seen: seen.add(item) yield item Adding this to, say, itertools would cover all my use cases. And as long as you don't have too many duplicates, filterdups as above should keep memory consumption down better. On the other hand, if someone could give me a few other reasonable use cases for LinkedHashSet and LinkedHashMap, I wouldn't object to their inclusion. Steve [1] http://mail.python.org/pipermail/python-list/2005-February/264179.html -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From gward at python.net Wed Mar 9 01:56:46 2005 From: gward at python.net (Greg Ward) Date: Wed Mar 9 01:56:49 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36 In-Reply-To: <200503081741.20335.anthony@interlink.com.au> References: <200503071413.05194.anthony@interlink.com.au> <20050308011443.GB28943@cthulhu.gerg.ca> <200503081741.20335.anthony@interlink.com.au> Message-ID: <20050309005646.GB30204@cthulhu.gerg.ca> On 08 March 2005, Anthony Baxter said: > I really would like to see it reverted, please. Done. Greg -- Greg Ward http://www.gerg.ca/ I just forgot my whole philosophy of life!!! From bac at OCF.Berkeley.EDU Wed Mar 9 02:03:04 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Mar 9 02:03:11 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> Message-ID: <422E4B48.8030003@ocf.berkeley.edu> Steven Bethard wrote: > Delaney, Timothy C (Timothy) wrote: > >>The perennial "how do I remove duplicates from a list" topic came up on >>c.l.py and in the discussion I mentioned the java 1.5 LinkedHashSet and >>LinkedHashMap. I'd thought about proposing these before, but couldn't >>think of where to put them. It was pointed out that the obvious place >>would be the collections module. >> >>For those who don't know, LinkedHashSet and LinkedHashMap are simply >>hashed sets and maps that iterate in the order that the keys were added >>to the set/map. I almost invariably use them for the above scenario - >>removing duplicates without changing order. >> >>Does anyone else think it would be worthwhile adding these to >>collections, or should I just make a cookbook entry? > > > I guess I'm -0 on this. > > Though I was the one that suggested that collections is the right > place to put them, I'm not really certain how much we gain by > including them. I too would only ever use them for removing > duplicates from a list. But if we're trying to provide a solution to > this problem, I'd rather see an iterable-friendly one. See a previous > thread on this issue[1] where I suggest something like: > > def filterdups(iterable): > seen = set() > for item in iterable: > if item not in seen: > seen.add(item) > yield item > > Adding this to, say, itertools would cover all my use cases. And as > long as you don't have too many duplicates, filterdups as above should > keep memory consumption down better. > I am -1 on adding LinkedHash*. While I can understand wanting to get rid of duplicates easily and wanting a good solutoin, Steven's snippet of code shows rolling your own solution is easy. Plus this can even be simplified down to a one-liner using itertools already:: itertools.ifilterfalse(lambda item, _set=set(): (item in _set) or (_set.add(item) and False), iterable) I don't think it is the prettiest solution, but it does show that coming up with one is not hard nor restricted to only a single solution that requires knowing some Python idiom (well, mine does for the default arg to the lambda, but Steven's doesn't). The last thing I want to happen is for Python's stdlib to grow every possible data structure out there like Java seems to have. I don't ever want to have to think about what *variant* of a data structure I should use, only what *type* of data structure (and no, I don't count collections.deque and Queue.Queue an overlap since the latter is meant more for thread communication, at least in my mind). -Brett From gward at python.net Wed Mar 9 02:21:52 2005 From: gward at python.net (Greg Ward) Date: Wed Mar 9 02:21:54 2005 Subject: No new features (was Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36) In-Reply-To: <200503091208.38519.anthony@interlink.com.au> References: <200503081741.20335.anthony@interlink.com.au> <20050309005646.GB30204@cthulhu.gerg.ca> <200503091208.38519.anthony@interlink.com.au> Message-ID: <20050309012152.GC30204@cthulhu.gerg.ca> On 09 March 2005, Anthony Baxter said (privately): > Thanks! I really want to keep the no-new-features thing going for > as long as possible, pending any AoG (Acts of Guido), of course. Grumble. How do you feel about upgrading optparse to sync with Optik 1.5.1? I'm a bit embarassed that Python 2.4's optparse has __version__ == "1.5a2" because I didn't release Optik 1.5 in time. And yes, there were some tiny new features in 1.5 and a few more coming in 1.5.1: * SF patch #870807: allow users to specify integer option arguments in hexadecimal, octal, or binary with leading "0x", "0", or "0b" (1.5 final). * SF feature #1050184: add 'append_const' action (patch by Andrea 'fwyzard' Bocci) (1.5 final). * Keep going if importing gettext fails (so optparse can be used in the Python build process) (1.5 final). * Fix so the 'merge' script works again (bugs spotted, and mostly fixed, by Andrea 'fwyzard' Bocci). (1.5.1) * SF bug #1145594: add 'finish()' method to OptionParser so applications can explicitly break reference cycles, making life easier for Python's garbage collector. (1.5.1) * SF feature #988126: add 'epilog' attribute to OptionParser (and constructor arg): paragraph of text to print after the main option help. (1.5.1) * Corrected French translation (po/optik/fr.po) (Laurent Laporte). (1.5.1) Every one of these is useful to someone, and none of them are even remotely destabilizing. But they all add functionality that would be present in 2.4.1 and not in 2.4. That doesn't bother me in the slightest, but I guess it bothers some people. I'd like to check this in for 2.4.1. But I won't if anyone says "don't do it". Nor will I if no one else says "do it". Burnt once... Greg -- Greg Ward http://www.gerg.ca/ Any priest or shaman must be presumed guilty until proven innocent. From nas at arctrix.com Wed Mar 9 03:08:50 2005 From: nas at arctrix.com (Neil Schemenauer) Date: Wed Mar 9 03:08:52 2005 Subject: [Python-Dev] unicode inconsistency? In-Reply-To: <1f7befae04090912007305d532@mail.gmail.com> Message-ID: On Sat, Apr 04, 1998 at 07:04:02AM +0000, Tim Peters wrote: > [Martin v. L?wis] > > I can't see any harm by supporting this operation also if __str__ returns > > a Unicode object. > > It doesn't sound like a good idea to me, at least in part because it > would be darned messy to implement short of saying "OK, we don't give > a rip anymore about what type of objects PyObject_{Str,Repr} return", It's about 10 lines of code. See http://python.org/sf/1159501 . Neil From gward at python.net Wed Mar 9 03:24:35 2005 From: gward at python.net (Greg Ward) Date: Wed Mar 9 03:24:40 2005 Subject: @deprecated (was Re: [Python-Dev] Re: Useful thread project for 2.5?) In-Reply-To: <20050308065552.ef6k5vbzrln3sw8@mcherm.com> References: <20050308065552.ef6k5vbzrln3sw8@mcherm.com> Message-ID: <20050309022435.GA3553@cthulhu.gerg.ca> On 08 March 2005, Michael Chermside said: > Greg writes: > > This is one of my top two Java features [1]. > ... > > [1] The other is compiler recognition of "@deprecated" in doc comments. > > Hmm... what a good idea! Let's see... what exactly does "@deprecated" DO in > Java? Why it causes the compiler to emit a warning if you use the function in > question. Not a bad idea. Isn't it, though? > Amusingly enough, the syntax is even the same in > Python: [...code omitted...] Cooool. So simple, and yet so powerful. One might even call it Pythonic! > So... shall I go add this to the cookbook? Hell yes! Greg -- Greg Ward http://www.gerg.ca/ Always look on the bright side of life. From gward at python.net Wed Mar 9 03:32:09 2005 From: gward at python.net (Greg Ward) Date: Wed Mar 9 03:32:15 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> Message-ID: <20050309023209.GB3553@cthulhu.gerg.ca> On 09 March 2005, Delaney, Timothy C (Timothy) said: > For those who don't know, LinkedHashSet and LinkedHashMap are simply > hashed sets and maps that iterate in the order that the keys were added > to the set/map. I almost invariably use them for the above scenario - > removing duplicates without changing order. > > Does anyone else think it would be worthwhile adding these to > collections, or should I just make a cookbook entry? +1 on a cookbook entry. +0 on adding to stdlib. I'll attach another approach to the same problem, an ordered dictionary object. I believe the semantics of adding/readding/deleting keys is the same as java.util.LinkedHashMap -- certainly it seems the most sensible and easy-to-implement semantics. Greg -- Greg Ward http://www.gerg.ca/ I brought my BOWLING BALL -- and some DRUGS!! -------------- next part -------------- A non-text attachment was scrubbed... Name: odict.py Type: text/x-python Size: 2266 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050308/8767e4c4/odict.py From tdelaney at avaya.com Wed Mar 9 03:40:11 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Wed Mar 9 03:40:09 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE721255@au3010avexu1.global.avaya.com> Greg Ward wrote: > I'll attach another approach to the same problem, an ordered > dictionary object. I believe the semantics of > adding/readding/deleting keys is the same as java.util.LinkedHashMap > -- certainly it seems the most sensible and easy-to-implement > semantics. That's essentially the same approach I used - I didn't get around to properly implementing a doubly-linked list between the keys - I just maintained a list with the correct order. If I'm going to write a Cookbook recipe though I should probably do it properly ... Personally, I think it would be quite nice if all python dictionaries had these semantics - it would be nice to be able to say that items will be returned in the order they're inserted - but I wouldn't want to incur the additional cost in such a fundamental data structure. One thing I did notice - dict.__repr__ and dict.__str__ don't produce strings in the iteration order of subclasses - they always use the arbitrary dict iteration order. Is this intentional? Tim Delaney From kbk at shore.net Wed Mar 9 04:14:29 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Mar 9 04:14:53 2005 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200503090314.j293ETbm005561@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 279 open (-24) / 2797 closed (+33) / 3076 total ( +9) Bugs : 851 open ( +2) / 4853 closed (+16) / 5704 total (+18) RFE : 173 open ( +4) / 150 closed ( +2) / 323 total ( +6) New / Reopened Patches ______________________ Fix for win32 proxy bypass support (no_proxy) in urllib(2) (2005-03-01) http://python.org/sf/1154227 opened by mullendr Fix for win32 proxy bypass support (no_proxy) in urllib(2) (2005-03-02) CLOSED http://python.org/sf/1155083 opened by mullendr exception doc updates (2005-03-03) CLOSED http://python.org/sf/1156102 opened by Michael Hudson patch for bug 1158490 (locale breakage) (2005-03-08) http://python.org/sf/1158909 opened by Wummel cookielib mis-handles RFC 2109 cookies in Netscape mode (2005-03-04) http://python.org/sf/1157027 opened by John J Lee fix for a deadlock in the logging module (2005-03-07) http://python.org/sf/1158052 opened by Sebastian Prause Names conflict with QT (on linux) (2005-03-08) http://python.org/sf/1158926 opened by CyberRan Handle corrupted gzip files with unexpected EOF (2005-03-08) http://python.org/sf/1159051 opened by Wummel cgi.py invalid REQUEST_METHOD set (2005-03-08) http://python.org/sf/1159139 opened by Joe Improve %s support for unicode (2005-03-09) http://python.org/sf/1159501 opened by Neil Schemenauer Patches Closed ______________ Reference count bug fix (2005-02-12) http://python.org/sf/1121234 closed by loewis PEP 309 reference implementation (2004-04-07) http://python.org/sf/931005 closed by rhettinger Fix for win32 proxy bypass support (no_proxy) in urllib(2) (2005-03-02) http://python.org/sf/1155083 closed by loewis Flush stdout/stderr if closed (fix for bug 1074011) (2004-11-29) http://python.org/sf/1075147 closed by loewis setup.py --help and --help-commands altered. (2005-01-17) http://python.org/sf/1104111 closed by loewis tarfile.ExFileObject iterators (2005-01-23) http://python.org/sf/1107973 closed by loewis patch for gzip.GzipFile.flush() (2005-01-26) http://python.org/sf/1110248 closed by loewis PyArg_ParseTuple problem with 'L' format (2003-04-17) http://python.org/sf/723201 closed by loewis crach link c++ extension by mingw (2004-06-29) http://python.org/sf/981773 closed by loewis Patch for Lib/bsddb/__init__.py to work with modulefinder (2005-01-31) http://python.org/sf/1112812 closed by loewis Changes to cookielib.py & friends for 2.4b1 (2004-09-16) http://python.org/sf/1028908 closed by loewis cookielib and cookies with special names (2005-02-06) http://python.org/sf/1117339 closed by loewis cookielib.LWPCookieJar incorrectly loads value-less cookies (2005-02-06) http://python.org/sf/1117454 closed by loewis Consistent retrieval of Python version. (2004-10-14) http://python.org/sf/1046831 closed by loewis allow UNIX mmap size to default to current file size (new) (2005-02-19) http://python.org/sf/1144555 closed by loewis allow UNIX mmap size to default to current file size (2003-06-06) http://python.org/sf/749830 closed by loewis better timer resolution for profile.py (2002-11-30) http://python.org/sf/645894 closed by loewis better parser error message for non-EOL following line cont. (2003-09-08) http://python.org/sf/802188 closed by loewis Updated "Working on Cygwin" section (2005-01-22) http://python.org/sf/1107221 closed by loewis Non-ascii in non-unicode __credits__ in Lib/pydoc.py (2004-08-15) http://python.org/sf/1009389 closed by mwh exception doc updates (2005-03-03) http://python.org/sf/1156102 closed by mwh support PY_LONGLONG in structmember (2005-02-02) http://python.org/sf/1115086 closed by loewis HEAD/PUT/DELETE support for urllib2.py (2005-01-28) http://python.org/sf/1111653 closed by loewis tarfile.py: fix for bug #1100429 (2005-01-16) http://python.org/sf/1103407 closed by loewis Patch for bug 1088077 (2004-12-19) http://python.org/sf/1088078 closed by loewis gcc compiler on Windows (2004-11-30) http://python.org/sf/1075887 closed by loewis tarfile: add extractall() method (2004-10-10) http://python.org/sf/1043890 closed by loewis Arrange 64bits detection for ReliantUnix (2004-05-24) http://python.org/sf/959534 closed by loewis tarfile.py enhancements (2004-03-17) http://python.org/sf/918101 closed by loewis Allow compilation of C/C++ Python extensions without MSVC (2004-05-19) http://python.org/sf/957033 closed by loewis Fix crash in xmlprase_GetInputContext in pyexpat.c (2005-02-08) http://python.org/sf/1118602 closed by loewis Minor improvement on 1116583 (2005-02-06) http://python.org/sf/1117114 closed by jjlee New / Reopened Bugs ___________________ Duplicate output with unbuffered IO after fork (2005-03-02) CLOSED http://python.org/sf/1154811 opened by Michael Aivazis getElementsbyTagName('*') in dom.minidom (2005-03-02) CLOSED http://python.org/sf/1155207 opened by bugmenot Bugs in parsedate_tz (2005-03-02) http://python.org/sf/1155362 opened by TH self.length shield exception in httplib (2005-03-03) http://python.org/sf/1155638 opened by Andrew P. Lentvorski, Jr. yield in __init__ causes broken new-style class (2005-03-03) CLOSED http://python.org/sf/1155938 opened by Steve Alexander Calls from VBScript clobber passed args (2005-03-03) http://python.org/sf/1156179 opened by Erik Rose [2.4 regression] seeking in codecs.reader broken (2005-03-03) http://python.org/sf/1156259 opened by Matthias Klose cmd.Cmd().cmdloop() can't read from file (2005-03-03) http://python.org/sf/1156280 opened by jamesthiele add documentation for __new__ (2005-03-03) CLOSED http://python.org/sf/1156412 opened by Steven Bethard doctest should support -m (2005-03-03) http://python.org/sf/1156430 opened by Ian Bicking Solaris and Python/pykota (2005-03-04) http://python.org/sf/1156441 opened by Kelly Ormsby csv Sniffer returns bad dialect? (2005-03-05) http://python.org/sf/1157169 opened by Neil Schemenauer PEP 328 and Python 2.4 error (2005-03-05) http://python.org/sf/1157255 opened by Kent Johnson pickle errors (2005-03-05) CLOSED http://python.org/sf/1157576 opened by Laxori666 RuntimeWarning: unfiltered RuntimeWarning (2005-03-06) CLOSED http://python.org/sf/1157665 opened by Grzegorz Makarewicz xml.dom.minidom.Node.removeChild() doesn't remove (2005-03-06) http://python.org/sf/1157901 opened by Matthias Kempka unixccompiler.py should deal with env in linker (2005-03-06) http://python.org/sf/1158005 opened by Edward Moy string.Template does not allow step-by-step replacements (2005-03-07) http://python.org/sf/1158231 opened by Stefan Behnel heapq should provide iterators: itersmallest, iterlargest (2005-03-07) CLOSED http://python.org/sf/1158313 opened by Stefan Behnel locale fails if LANGUAGE has multiple locales (2005-03-07) http://python.org/sf/1158490 opened by mixedpuppy --disable-unicode fails when linking (2005-03-07) CLOSED http://python.org/sf/1158607 opened by Frank Baumgart 2.4 crashes when try to exit app and mulitple threads active (2005-03-08) http://python.org/sf/1159425 opened by hugendian Bugs Closed ___________ Python 2.4.0 crashes with a segfault, EXAMPLE ATTACHED (2005-02-11) http://python.org/sf/1120452 closed by nascheme Duplicate output with unbuffered IO after fork (2005-03-01) http://python.org/sf/1154811 closed by aivazis getElementsbyTagName('*') in dom.minidom (2005-03-02) http://python.org/sf/1155207 closed by bugmenot gzip.GzipFile.flush() does not flush all internal buffers (2005-01-26) http://python.org/sf/1110242 closed by loewis Bugs in rfc822.parseaddr() (2002-03-18) http://python.org/sf/531205 closed by loewis Subtle bug in os.path.realpath on Cygwin (2003-07-09) http://python.org/sf/768419 closed by loewis uu.decode prints to stderr (2003-09-09) http://python.org/sf/803413 closed by loewis segfault in readline (2004-12-16) http://python.org/sf/1086603 closed by loewis yield in __init__ causes broken new-style class (2005-03-03) http://python.org/sf/1155938 closed by rhettinger test_subprocess is far too slow (2005-02-17) http://python.org/sf/1124637 closed by astrand subprocess example missing "stdout=PIPE" (2005-02-13) http://python.org/sf/1121579 closed by astrand TarFile iteration can break (on Windows) if file has links (2005-01-11) http://python.org/sf/1100429 closed by loewis add documentation for __new__ (2005-03-03) http://python.org/sf/1156412 closed by gward buffer problem in pyexpat.c(xmlparse_GetInputContext) (2004-03-29) http://python.org/sf/925152 closed by loewis pickle errors (2005-03-05) http://python.org/sf/1157576 closed by tim_one RuntimeWarning: unfiltered RuntimeWarning (2005-03-06) http://python.org/sf/1157665 closed by rhettinger --disable-unicode fails when linking (2005-03-07) http://python.org/sf/1158607 closed by loewis New / Reopened RFE __________________ add get_current_dir_name() to os module (2005-03-01) http://python.org/sf/1154351 opened by Michael Hoffman file() on a file (2005-03-02) http://python.org/sf/1155485 opened by Felix Lee __getattr__ and __setattr__ methods for modules (2005-03-04) http://python.org/sf/1156499 opened by Reinhold Birkenfeld RFE Closed __________ ossaudiodev object does not support common readonly attrs (2003-10-05) http://python.org/sf/818006 closed by gward heapq should provide iterators: itersmallest, iterlargest (2005-03-07) http://python.org/sf/1158313 closed by rhettinger From aahz at pythoncraft.com Wed Mar 9 05:30:29 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed Mar 9 05:30:32 2005 Subject: No new features (was Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36) In-Reply-To: <20050309012152.GC30204@cthulhu.gerg.ca> References: <200503081741.20335.anthony@interlink.com.au> <20050309005646.GB30204@cthulhu.gerg.ca> <200503091208.38519.anthony@interlink.com.au> <20050309012152.GC30204@cthulhu.gerg.ca> Message-ID: <20050309043028.GA14832@panix.com> On Tue, Mar 08, 2005, Greg Ward wrote: > On 09 March 2005, Anthony Baxter said (privately): >> >> Thanks! I really want to keep the no-new-features thing going for >> as long as possible, pending any AoG (Acts of Guido), of course. > > Grumble. How do you feel about upgrading optparse to sync with Optik > 1.5.1? I'm a bit embarassed that Python 2.4's optparse has __version__ > == "1.5a2" because I didn't release Optik 1.5 in time. -1, sorry -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From mwh at python.net Wed Mar 9 10:16:28 2005 From: mwh at python.net (Michael Hudson) Date: Wed Mar 9 10:16:30 2005 Subject: [Python-Dev] Re: @deprecated In-Reply-To: <20050309022435.GA3553@cthulhu.gerg.ca> (Greg Ward's message of "Tue, 8 Mar 2005 21:24:35 -0500") References: <20050308065552.ef6k5vbzrln3sw8@mcherm.com> <20050309022435.GA3553@cthulhu.gerg.ca> Message-ID: <2my8cxnpqr.fsf@starship.python.net> Greg Ward writes: > On 08 March 2005, Michael Chermside said: >> Greg writes: >> > This is one of my top two Java features [1]. >> ... >> > [1] The other is compiler recognition of "@deprecated" in doc comments. >> >> Hmm... what a good idea! Let's see... what exactly does "@deprecated" DO in >> Java? Why it causes the compiler to emit a warning if you use the function in >> question. Not a bad idea. > > Isn't it, though? > >> Amusingly enough, the syntax is even the same in >> Python: > [...code omitted...] > > Cooool. So simple, and yet so powerful. One might even call it Pythonic! One difference: I imagine (hope!) the java compiler would complain if the deprecated function is referenced, whereas the python version only complains if it is called... Cheers, mwh -- ROOSTA: Ever since you arrived on this planet last night you've been going round telling people that you're Zaphod Beeblebrox, but that they're not to tell anyone else. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From mal at egenix.com Wed Mar 9 11:10:59 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Mar 9 11:11:01 2005 Subject: [Python-Dev] unicode inconsistency? In-Reply-To: References: Message-ID: <422ECBB3.90400@egenix.com> Neil Schemenauer wrote: > On Sat, Apr 04, 1998 at 07:04:02AM +0000, Tim Peters wrote: > >>[Martin v. L?wis] >> >>>I can't see any harm by supporting this operation also if __str__ returns >>>a Unicode object. >> >>It doesn't sound like a good idea to me, at least in part because it >>would be darned messy to implement short of saying "OK, we don't give >>a rip anymore about what type of objects PyObject_{Str,Repr} return", > > > It's about 10 lines of code. See http://python.org/sf/1159501 . The patch implements the PyObjbect_Text() idea (an API that returns a basestring instance, ie. string or unicode) and then uses this in '%s' (the string version) to properly propogate to u'%s' (the unicode version). Maybe we should also expose the C API as suggested in the patch, e.g. as text(obj). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 09 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From anthony at interlink.com.au Wed Mar 9 12:07:20 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 9 12:07:49 2005 Subject: No new features (was Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36) In-Reply-To: <20050309012152.GC30204@cthulhu.gerg.ca> References: <200503091208.38519.anthony@interlink.com.au> <20050309012152.GC30204@cthulhu.gerg.ca> Message-ID: <200503092207.22313.anthony@interlink.com.au> On Wednesday 09 March 2005 12:21, Greg Ward wrote: > On 09 March 2005, Anthony Baxter said (privately): > > Thanks! I really want to keep the no-new-features thing going for > > as long as possible, pending any AoG (Acts of Guido), of course. > > Grumble. How do you feel about upgrading optparse to sync with Optik > 1.5.1? I'm a bit embarassed that Python 2.4's optparse has __version__ > == "1.5a2" because I didn't release Optik 1.5 in time. The version string update I don't see being a problem. > And yes, there were some tiny new features in 1.5 and a few more coming > in 1.5.1: > > * SF patch #870807: allow users to specify integer option arguments > in hexadecimal, octal, or binary with leading "0x", "0", or "0b" > (1.5 final). Again, this is a new feature, and having it behave differently in 2.4.1 to 2.4 could be confusing - but I'm not absolutely against this. > * SF feature #1050184: add 'append_const' action (patch by > Andrea 'fwyzard' Bocci) (1.5 final). New feature? > * Keep going if importing gettext fails (so optparse can be used > in the Python build process) (1.5 final). Bug fix. > * Fix so the 'merge' script works again (bugs spotted, and mostly > fixed, by Andrea 'fwyzard' Bocci). (1.5.1) Bug fix. > * SF bug #1145594: add 'finish()' method to OptionParser so > applications can explicitly break reference cycles, making life > easier for Python's garbage collector. (1.5.1) Is this purely an internal method, or something that people would use as part of the API? If it's an API change, it shouldn't be included. > * SF feature #988126: add 'epilog' attribute to OptionParser > (and constructor arg): paragraph of text to print after the > main option help. (1.5.1) API change. > * Corrected French translation (po/optik/fr.po) (Laurent Laporte). > (1.5.1) Bug fix. > Every one of these is useful to someone, and none of them are even > remotely destabilizing. But they all add functionality that would be > present in 2.4.1 and not in 2.4. That doesn't bother me in the > slightest, but I guess it bothers some people. Initially, I was inclined to be much less anal about the no-new-features thing. But since doing it, I've had a quite large number of people tell me how much they appreciate this approach - vendors, large companies with huge installed bases of Python, and also from people releasing software written in Python. Very few people offer the counter argument as a general case - with the obvious exception that everyone has their "just this one little backported feature, pleeeease!" (I'm the same - there's been times where I've had new features I'd have loved to see in a bugfix release, just so I could use them sooner). > I'd like to check this in for 2.4.1. But I won't if anyone says "don't > do it". Nor will I if no one else says "do it". Well, 2.4.1rc1 is out in about 12 hours time. If you want to check in the bugfixes before then, please feel free. If you don't want to apply bits and pieces, then don't worry about it. I don't want anything but critical fixes landing between 2.4.1rc1 and 2.4.1 final. Remember that 2.4.2 will turn up inevitably, it's not like this is the last release between now and 2.5... I should also add that the above is really just policy as it's evolved - if people want to discuss this (am I being too strict?) I'm happy to talk about it. I'd even suggest a BOF at PyCon, except I won't be there :-( -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Wed Mar 9 12:12:17 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 9 12:46:51 2005 Subject: [Python-Dev] BRANCH FREEZE for 2.4.1rc1, 0000 UTC, 2005-03-10 Message-ID: <200503092212.18408.anthony@interlink.com.au> So we're cutting 2.4.1c1 in about 12 hours time. Please leave the branch alone from 0000 UTC (that's 8pm US Eastern, unless my timezones are all screwed up). Once the release candidate is done, we'll have a week to shake out any embarrassing bugs, and then a 2.4.1 final. Please be _extraordinarily_ conservative with checkins to the release24-maint branch until 2.4.1 final is out. If in doubt, feel free to email me, or contact on any one of AIM: anthonyatekit, jabber: anthonyatekit@jabber.org, or IRC: #python-dev on Freenode. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Wed Mar 9 13:17:06 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 9 13:17:21 2005 Subject: [Python-Dev] rationale for the no-new-features approach Message-ID: <200503092317.07909.anthony@interlink.com.au> So it's only fair that I write down my rationale for why I'm being anal about the no-new-features approach. Comments are more than welcome - ideally, after discussion, I'll put some more words in the bugfix PEP. Goal 1: When we cut a bugfix release, people will upgrade to it. - This helps the users (they have bugs fixed) - This helps us (python-dev) because people won't be logging bugs against already-fixed-bugs. - This makes us (Python) look good, because people using Python have the most bug-free experience possible. Goal 2: Packagers of Python will package up releases of Python that are as close to the "official" release as possible. - This, to me, is a huge win. If we don't have to worry about whether someone is running 2.4.1, or DistroFoo's version of 2.4.1 with a couple of backported fixes from 2.4.2, or some other horrible frankenstein's monster of a release, it makes our lives much easier. (This is also why I've been on a bit of a stomping exercise to attempt to get distros that broke Python into pieces to stop doing this (notably, Debian/Ubuntu's distutils- in-python-devel pain)) - And yes, we're always going to have the problem of some distros stuck on old Python releases (Redhat -> Python 1.5.2, Debian stable -> Python 2.1, &c) but we can hopefully encourage them to at least roll out the latest bugfix version of whatever version they're stuck on. Goal 3: Good PR. - I've read a disturbing amount of words about other languages (*cough*Java*cough*) that have no sanity about minor and major releases. This seems to piss people off a great deal. One of Python's selling points is that we're very cautious and a suitable choice for grownups to use in business. - This is good for us (Python community) because it makes it less likely we'll be stuck in a job where we're forced to code Perl, or C++, or Java, or some other horrible fate. It also boosts the size of the community, meaning that there will be more volunteers to work on Python (hurrah!) Goal 4: Try and prevent something like try: True, False except NameError: True, False = 1, 0 from ever ever happening again. - This, I hope, needs no further explanation Anthony -- Anthony Baxter It's never too late to have a happy childhood. From abo at minkirri.apana.org.au Wed Mar 9 13:38:15 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Wed Mar 9 13:41:35 2005 Subject: No new features (was Re: [Python-Dev] Re: [Python-checkins]python/dist/src/Modules ossaudiodev.c, 1.35, 1.36) References: <200503091208.38519.anthony@interlink.com.au><20050309012152.GC30204@cthulhu.gerg.ca> <200503092207.22313.anthony@interlink.com.au> Message-ID: <007d01c524a4$dd17fb20$24ed0ccb@apana.org.au> G'day, From: "Anthony Baxter" > On Wednesday 09 March 2005 12:21, Greg Ward wrote: > > On 09 March 2005, Anthony Baxter said (privately): > > > Thanks! I really want to keep the no-new-features thing going for > > > as long as possible, pending any AoG (Acts of Guido), of course. [...] > Initially, I was inclined to be much less anal about the no-new-features > thing. But since doing it, I've had a quite large number of people tell me how > much they appreciate this approach - vendors, large companies with huge > installed bases of Python, and also from people releasing software written > in Python. Very few people offer the counter argument as a general case - > with the obvious exception that everyone has their "just this one little > backported feature, pleeeease!" (I'm the same - there's been times where > I've had new features I'd have loved to see in a bugfix release, just so I > could use them sooner). Just my 2c; I don't mind new features in minor releases, provided they meet the following two criteria; 1) Don't break the old API! The new features must be pure extensions that in no way change the old API. Any existing code should be un-effected in any way by the change. 2) The new features must be clearly documented as "New in version X.Y.Z". This way people using these features will know the minium Python version required for their application. Any change that breaks rule 1) must be delayed until a major release. Any change that breaks rule 2) is a documentation bug that needs fixing. ---------------------------------------------------------------- Donovan Baarda http://minkirri.apana.org.au/~abo/ ---------------------------------------------------------------- From irmen at xs4all.nl Wed Mar 9 13:53:01 2005 From: irmen at xs4all.nl (Irmen de Jong) Date: Wed Mar 9 13:53:03 2005 Subject: [Python-Dev] Re: @deprecated In-Reply-To: <2my8cxnpqr.fsf@starship.python.net> References: <20050308065552.ef6k5vbzrln3sw8@mcherm.com> <20050309022435.GA3553@cthulhu.gerg.ca> <2my8cxnpqr.fsf@starship.python.net> Message-ID: <6641.213.34.50.11.1110372781.squirrel@213.34.50.11> mwh wrote: > One difference: I imagine (hope!) the java compiler would complain if > the deprecated function is referenced, whereas the python version only > complains if it is called... The java compiler actually already produces deprecation warnings during the *compilation phase* (when used with the -deprecation option). During runtime, no warnings are produced. Grtz, Irmen. From mwh at python.net Wed Mar 9 13:53:48 2005 From: mwh at python.net (Michael Hudson) Date: Wed Mar 9 13:53:51 2005 Subject: [Python-Dev] Re: No new features In-Reply-To: <007d01c524a4$dd17fb20$24ed0ccb@apana.org.au> (Donovan Baarda's message of "Wed, 9 Mar 2005 23:38:15 +1100") References: <200503091208.38519.anthony@interlink.com.au> <20050309012152.GC30204@cthulhu.gerg.ca> <200503092207.22313.anthony@interlink.com.au> <007d01c524a4$dd17fb20$24ed0ccb@apana.org.au> Message-ID: <2mr7ipnfoj.fsf@starship.python.net> "Donovan Baarda" writes: > > Just my 2c; > > I don't mind new features in minor releases, provided they meet the > following two criteria; > > 1) Don't break the old API! The new features must be pure extensions that in > no way change the old API. Any existing code should be un-effected in any > way by the change. > > 2) The new features must be clearly documented as "New in version X.Y.Z". > This way people using these features will know the minium Python version > required for their application. No no no! The point of what Anthony is saying, as I read it, is that experience suggests it is exactly this sort of change that should be avoided. Consider the case of Mac OS X 10.2 which came with Python 2.2.0: this was pretty broken anyway because of some apple snafus but it was made even more useless by the fact that people out in the wild were writing code for 2.2.1 and using True/False/bool. Going from 2.x.y to 2.x.y+1 shouldn't break anything, going from 2.x.y+1 to 2.x.y shouldn't break anything that doesn't whack into a bug in 2.x.y -- and "not having bool" isn't a bug in this sense. My 2p. Cheers, mwh -- Well, you pretty much need Microsoft stuff to get misbehaviours bad enough to actually tear the time-space continuum. Luckily for you, MS Internet Explorer is available for Solaris. -- Calle Dybedahl, alt.sysadmin.recovery From mwh at python.net Wed Mar 9 13:55:42 2005 From: mwh at python.net (Michael Hudson) Date: Wed Mar 9 13:55:45 2005 Subject: [Python-Dev] Re: @deprecated In-Reply-To: <6641.213.34.50.11.1110372781.squirrel@213.34.50.11> (Irmen de Jong's message of "Wed, 9 Mar 2005 13:53:01 +0100 (CET)") References: <20050308065552.ef6k5vbzrln3sw8@mcherm.com> <20050309022435.GA3553@cthulhu.gerg.ca> <2my8cxnpqr.fsf@starship.python.net> <6641.213.34.50.11.1110372781.squirrel@213.34.50.11> Message-ID: <2mmztdnfld.fsf@starship.python.net> "Irmen de Jong" writes: > mwh wrote: > > >> One difference: I imagine (hope!) the java compiler would complain if >> the deprecated function is referenced, whereas the python version only >> complains if it is called... > > The java compiler actually already produces deprecation warnings > during the *compilation phase* (when used with the -deprecation option). > During runtime, no warnings are produced. Yeah, that's what I meant to say, even if it wasn't what I said :) Cheers, mwh -- LINTILLA: You could take some evening classes. ARTHUR: What, here? LINTILLA: Yes, I've got a bottle of them. Little pink ones. -- The Hitch-Hikers Guide to the Galaxy, Episode 12 From mcherm at mcherm.com Wed Mar 9 15:08:06 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Mar 9 15:08:09 2005 Subject: [Python-Dev] Re: @deprecated Message-ID: <20050309060806.yob2cln2eudcw8ko@mcherm.com> [I followed Greg's suggestion and proposed a "deprecated" decorator.] Raymond writes: > Decorators like this should preserve information about the underlying > function Of course! The version I posted was intended to illustrate the idea, not to be a clean implementation. A long time ago, I proposed a decorator-maker-decorator (see "Creating Well-Behaved Decorators" in http://www.python.org/moin/PythonDecoratorLibrary), and I still think it's probably a wise approach since it's easy for people to be careless and forget to preserve these sorts of features. Jim Jewett writes: > I agree that it should go in the cookbook, but I think you should > set the category to a DeprecationWarning and give the offending > function's name. I had wondered about that, but wasn't familiar with how people actually use categories. DeprecationWarning certainly sounds right, or is there some reason why I should use a custom subclass of DeprecationWarning? Michael Hudson and Irmen point out: > One difference: I imagine (hope!) the java compiler would complain if > the deprecated function is referenced, whereas the python version only > complains if it is called... True enough. And java doesn't complain at all if the deprecated function is invoked via reflection. It's a fundamental difference in style between the two languages: Python barely even HAS a "compile phase", and Python programs tend to be far more dynamic than Java. -- Michael Chermside From srichter at cosmos.phy.tufts.edu Wed Mar 9 15:11:15 2005 From: srichter at cosmos.phy.tufts.edu (Stephan Richter) Date: Wed Mar 9 15:11:18 2005 Subject: [Python-Dev] @deprecated (was: Useful thread project for 2.5?) In-Reply-To: References: Message-ID: <200503090911.15625.srichter@cosmos.phy.tufts.edu> On Tuesday 08 March 2005 18:05, Jim Jewett wrote: > ? ? ... compiler recognition of "@deprecated" in doc comments. > > Michael Chermside suggested: > > ? ? ===== code ===== > ? ? import warnings > > ? ? def deprecated(func): > ? ? ? ? """This is a decorator which can be used to mark functions > ? ? ? ? as deprecated. It will result in a warning being emmitted > ? ? ? ? when the function is used.""" > ? ? ? ? def newFunc(*args, **kwargs): > ? ? ? ? ? ? warnings.warn("Call to deprecated function.") > ? ? ? ? ? ? return func(*args, **kwargs) > ? ? ? ? return newFunc > > ? ? ===== example ===== > ... > ? ? UserWarning: Call to deprecated function. This is a recipe for disaster. Creating a new function from the old can have unwanted side effects, since you effectively change the object. For example, if someone is monkey patching this function, then the deprecation warning gets lost. In Zope 3's deprecation package, we have decided to put a special deprecation proxy around the module (instead of the function) and register a list of attribute names (note that it does not matter whether its a class, function or other type of object) that are deprecated. The advantage is that you get a deprecation warning just by looking up the object. See: http://svn.zope.org/Zope3/trunk/src/zope/deprecation/ Disclaimer: This code is a bit experimental and might not be the best solution. It is currently used in the trunk and does a pretty good job, though. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training From aahz at pythoncraft.com Wed Mar 9 15:48:17 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed Mar 9 15:48:20 2005 Subject: [Python-Dev] Re: No new features In-Reply-To: <2mr7ipnfoj.fsf@starship.python.net> References: <200503091208.38519.anthony@interlink.com.au> <20050309012152.GC30204@cthulhu.gerg.ca> <200503092207.22313.anthony@interlink.com.au> <007d01c524a4$dd17fb20$24ed0ccb@apana.org.au> <2mr7ipnfoj.fsf@starship.python.net> Message-ID: <20050309144817.GA9198@panix.com> On Wed, Mar 09, 2005, Michael Hudson wrote: > > No no no! The point of what Anthony is saying, as I read it, is that > experience suggests it is exactly this sort of change that should be > avoided. Consider the case of Mac OS X 10.2 which came with Python > 2.2.0: this was pretty broken anyway because of some apple snafus but > it was made even more useless by the fact that people out in the wild > were writing code for 2.2.1 and using True/False/bool. Going from > 2.x.y to 2.x.y+1 shouldn't break anything, going from 2.x.y+1 to 2.x.y > shouldn't break anything that doesn't whack into a bug in 2.x.y -- and > "not having bool" isn't a bug in this sense. Yes, exactly. That's why I wrote PEP 6 in the first place, and experience has only led to tightening things up further. The specific example of 2.2.1 and bool is even worse than you're noting, because 2.3's bool works a bit differently, so you have to actually code for three different problems in two major versions. Sehr schlecht. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From anthony at interlink.com.au Wed Mar 9 16:07:48 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 9 16:08:03 2005 Subject: [Python-Dev] Re: No new features In-Reply-To: <2mr7ipnfoj.fsf@starship.python.net> References: <007d01c524a4$dd17fb20$24ed0ccb@apana.org.au> <2mr7ipnfoj.fsf@starship.python.net> Message-ID: <200503100207.50401.anthony@interlink.com.au> On Wednesday 09 March 2005 23:53, Michael Hudson wrote: > No no no! The point of what Anthony is saying, as I read it, Your reading of my message is correct. > is that > experience suggests it is exactly this sort of change that should be > avoided. Consider the case of Mac OS X 10.2 which came with Python > 2.2.0: this was pretty broken anyway because of some apple snafus but > it was made even more useless by the fact that people out in the wild > were writing code for 2.2.1 and using True/False/bool. Going from > 2.x.y to 2.x.y+1 shouldn't break anything, going from 2.x.y+1 to 2.x.y > shouldn't break anything that doesn't whack into a bug in 2.x.y -- and > "not having bool" isn't a bug in this sense. This is exactly right, and, assuming people are in agreement, I plan to add text to this effect to PEP 6. One of the slightly negative side effects of Python's growth is that a lot of people depend on it now, and I feel it's in everyone's interests to try and make the best releases I can. We now do more regular bugfix releases - once every 6 months is the goal. Ideally, people shouldn't be _forced_ to upgrade (although obviously I'd prefer if they did ). Assuming that none of the bugs fixed are actually biting you, you should be able to stick with that version of 2.4.0 that's rolled out across hundreds of machines, and not have to worry that someone's writing code against some 2.4.2-specific features. Or, to put it more concisely -- if we allow new features (however minor) in bugfix releases, we're effectively shipping a "new" Python release every 6 months. This is insane. I want to have my cake and eat it too - I want the bugs fixed, but I don't want to have to mess about with worrying about new features each time I need to roll out a new release (in my job). [Disclaimer: It should be obvious here that I'm speaking for myself, and in doing so, I'm wearing a number of different hats - while I'm currently the guy who does the cvs-tag-and-release, I'm also an author of a whole pile of Python software, some open source, some closed (for work). In addition, I need to think about deployment and rollout strategies for new code and new systems in my job. I'm trying to optimise the release process to suit all these hats, and at the same time thinking about what I've been told by other people (distro vendors, people running _very_ large sets of machines with Python software)] In an attempt to stir up some discussion - if you have a reason _for_ allowing new features in a minor release, suggest it! I'm genuinely interested in having this discussion and working out the best solution... Anthony -- Anthony Baxter It's never too late to have a happy childhood. From barry at python.org Wed Mar 9 16:13:27 2005 From: barry at python.org (Barry Warsaw) Date: Wed Mar 9 16:13:31 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <200503092317.07909.anthony@interlink.com.au> References: <200503092317.07909.anthony@interlink.com.au> Message-ID: <1110381207.22370.45.camel@presto.wooz.org> On Wed, 2005-03-09 at 07:17, Anthony Baxter wrote: > So it's only fair that I write down my rationale for why I'm being anal > about the no-new-features approach. Comments are more than welcome - > ideally, after discussion, I'll put some more words in the bugfix PEP. I applaud your strictness Anthony (I was going to use a different phrase using some words from your original message, but then I thought better of that. :). I think we learned our lesson from the 2.2 True/False fiasco. It's a natural instinct to want to get those new features in earlier than the minor release lifecycle, but resisting that temptation demonstrates our maturity as a community, IMO. Having to wait for those new features in the mainline Python releases is the cost of our "batteries included" approach. If the users of a package want a shorter lifecycle, then the package maintainer will just have to make independent releases, sync'ing up with Python development in the latter's trunk. I've had to do this occasionally with the email package, for example. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050309/f4515295/attachment.pgp From jim at zope.com Wed Mar 9 16:38:12 2005 From: jim at zope.com (Jim Fulton) Date: Wed Mar 9 16:38:28 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <200503092317.07909.anthony@interlink.com.au> References: <200503092317.07909.anthony@interlink.com.au> Message-ID: <422F1864.3040305@zope.com> +1 (wish * could say +sys.maxint). To emphasize: - We want people to update to bug fix releases quickly. For people to do that, they need to know the risk of breakage is low. Reducing the scope of the release is important for that. - Python has gotten much better at this than it used to be. I remember the days when major features in 3rd-dot releases caused major headaches for us. I think that made Python look bad. Jim Anthony Baxter wrote: > So it's only fair that I write down my rationale for why I'm being anal > about the no-new-features approach. Comments are more than welcome - > ideally, after discussion, I'll put some more words in the bugfix PEP. > > Goal 1: When we cut a bugfix release, people will upgrade to it. > - This helps the users (they have bugs fixed) > - This helps us (python-dev) because people won't be logging > bugs against already-fixed-bugs. > - This makes us (Python) look good, because people using > Python have the most bug-free experience possible. > > Goal 2: Packagers of Python will package up releases of Python > that are as close to the "official" release as possible. > - This, to me, is a huge win. If we don't have to worry about > whether someone is running 2.4.1, or DistroFoo's version of > 2.4.1 with a couple of backported fixes from 2.4.2, or some > other horrible frankenstein's monster of a release, it makes > our lives much easier. (This is also why I've been on a bit of > a stomping exercise to attempt to get distros that broke Python > into pieces to stop doing this (notably, Debian/Ubuntu's distutils- > in-python-devel pain)) > - And yes, we're always going to have the problem of some distros > stuck on old Python releases (Redhat -> Python 1.5.2, Debian stable > -> Python 2.1, &c) but we can hopefully encourage them to at least > roll out the latest bugfix version of whatever version they're stuck on. > > Goal 3: Good PR. > - I've read a disturbing amount of words about other > languages (*cough*Java*cough*) that have no sanity about > minor and major releases. This seems to piss people off a > great deal. One of Python's selling points is that we're very > cautious and a suitable choice for grownups to use in business. > - This is good for us (Python community) because it makes it > less likely we'll be stuck in a job where we're forced to code > Perl, or C++, or Java, or some other horrible fate. It also boosts > the size of the community, meaning that there will be more > volunteers to work on Python (hurrah!) > > Goal 4: Try and prevent something like > try: > True, False > except NameError: > True, False = 1, 0 > from ever ever happening again. > - This, I hope, needs no further explanation > > Anthony > -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From nik at linuxforen.de Wed Mar 9 17:08:41 2005 From: nik at linuxforen.de (Niklas Fischer) Date: Wed Mar 9 16:57:22 2005 Subject: [Python-Dev] scrambling python in a microsoft .exe container Message-ID: <20050309160841.26143.qmail@mail.kemm.de> hi all! i lost the fight against google to find a wrapper/scrambler for python - boxed in an exe file. even the mandatory '-monty' string didn t help to find any tools. anyone got experience with such a tool? thanks for any hint. greets, Nik From gvanrossum at gmail.com Wed Mar 9 17:01:09 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed Mar 9 17:01:13 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <422F1864.3040305@zope.com> References: <200503092317.07909.anthony@interlink.com.au> <422F1864.3040305@zope.com> Message-ID: On Wed, 09 Mar 2005 10:38:12 -0500, Jim Fulton wrote: > +1 (wish * could say +sys.maxint). Anthony gets my +1 too, which then adds up to sys.maxint. :-) > To emphasize: > > - We want people to update to bug fix releases quickly. For people > to do that, they need to know the risk of breakage is low. Reducing the > scope of the release is important for that. > > - Python has gotten much better at this than it used to be. I remember the > days when major features in 3rd-dot releases caused major headaches for us. For those who only remember bool(), Python 1.5.2 was probably the worst offender here (it had nothing to do with 1.5.1). Mea culpa etcetera. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony at interlink.com.au Wed Mar 9 17:40:13 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 9 17:40:26 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: References: <200503092317.07909.anthony@interlink.com.au> <422F1864.3040305@zope.com> Message-ID: <200503100340.15327.anthony@interlink.com.au> My google-fu returned, and I found the piece I was looking for earlier. This discusses (from an internal Sun perspective) some of the problems with Java. They make quite a big deal about the problem of changing code across minor releases. I recall (re)reading this at some point and it helped me clarify some ideas floating around in my head. http://www.internalmemos.com/memos/memodetails.php?memo_id=1321 On Thursday 10 March 2005 03:01, Guido van Rossum wrote: > For those who only remember bool(), Python 1.5.2 was probably the > worst offender here (it had nothing to do with 1.5.1). Mea culpa > etcetera. That was a heck of a long time ago, and Python was a heck of a lot younger then. Hell, I can still remember the interpreter-prints-all-return-values-to-stdout being turned off sometime during the 0.9.x series (best minor-release-change ever!). And while the True/False issue was a complete pain, it at least serves as a good example (in the http://www.despair.com/demotivators/mis24x30prin.html sense of the word ) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From fdrake at acm.org Wed Mar 9 17:48:50 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Mar 9 17:48:54 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 In-Reply-To: References: Message-ID: <200503091148.50639.fdrake@acm.org> On Wednesday 09 March 2005 06:54, anthonybaxter@users.sourceforge.net wrote: > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/NEWS.txt,v ... > -What's New in IDLE 1.1.1? > -======================= > +What's New in IDLE 1.1.1c1? ... > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/idlever.py,v > -IDLE_VERSION = "1.1.1" > +IDLE_VERSION = "1.1.1c1" Shouldn't this progress from 1.1.1 to 1.1.2c1? Otherwise it's moving backward. -Fred -- Fred L. Drake, Jr. From anthony at interlink.com.au Wed Mar 9 18:06:08 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 9 18:06:23 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 In-Reply-To: <200503091148.50639.fdrake@acm.org> References: <200503091148.50639.fdrake@acm.org> Message-ID: <200503100406.09997.anthony@interlink.com.au> On Thursday 10 March 2005 03:48, Fred L. Drake, Jr. wrote: > > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/idlever.py,v > > -IDLE_VERSION = "1.1.1" > > +IDLE_VERSION = "1.1.1c1" > > Shouldn't this progress from 1.1.1 to 1.1.2c1? Otherwise it's moving > backward. No - Py2.4 shipped with idle 1.1. I must've updated it to 1.1.1 sometime after the 2.4 release (I vaguely recall doing this). -- Anthony Baxter It's never too late to have a happy childhood. From janssen at parc.com Wed Mar 9 18:12:27 2005 From: janssen at parc.com (Bill Janssen) Date: Wed Mar 9 18:12:54 2005 Subject: [Python-Dev] Re: No new features In-Reply-To: Your message of "Wed, 09 Mar 2005 04:53:48 PST." <2mr7ipnfoj.fsf@starship.python.net> Message-ID: <05Mar9.091232pst."58617"@synergy1.parc.xerox.com> Hear, hear! > Going from > 2.x.y to 2.x.y+1 shouldn't break anything, going from 2.x.y+1 to 2.x.y > shouldn't break anything that doesn't whack into a bug in 2.x.y -- and > "not having bool" isn't a bug in this sense. Micro releases are all about bug fixes. Every micro release of the same minor release should be compatible with every other, except for bugs and fixes to those bugs. Bill From nas at arctrix.com Wed Mar 9 19:16:30 2005 From: nas at arctrix.com (Neil Schemenauer) Date: Wed Mar 9 19:16:33 2005 Subject: [Python-Dev] unicode inconsistency? In-Reply-To: <422ECBB3.90400@egenix.com> References: <422ECBB3.90400@egenix.com> Message-ID: <20050309181630.GB4739@mems-exchange.org> On Wed, Mar 09, 2005 at 11:10:59AM +0100, M.-A. Lemburg wrote: > The patch implements the PyObjbect_Text() idea (an API that > returns a basestring instance, ie. string or unicode) and > then uses this in '%s' (the string version) to properly propogate > to u'%s' (the unicode version). > > Maybe we should also expose the C API as suggested in the patch, > e.g. as text(obj). Perhaps the right thing to do is introduce a new format code that means insert text(obj) instead of str(obj), e.g %t. If we do that though then we should make "'%s' % u'xyz'" return a string instead of a unicode object. I suspect that would break a lot of code. OTOH, having %s mean text(obj) instead of str(obj) may work just fine. People who want it to mean str() generally don't have any unicode strings floating around so text() has the same effect. People who are using unicode probably would find text() to be more useful behavior. I think that's why someone hacked PyString_Format to sometimes return unicode strings. Regarding the use of __str__, to return a unicode object: we could introduce a new slot (e.g. __text__) instead. However, I can't see any advantage to that. If someone really wants a str object then they call str() or PyObject_Str(). Neil From martin at v.loewis.de Wed Mar 9 19:37:19 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 9 19:37:22 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <422DF5E5.1000406@ocf.berkeley.edu> References: <422D6DC6.5010001@v.loewis.de> <422DF5E5.1000406@ocf.berkeley.edu> Message-ID: <422F425F.8010701@v.loewis.de> Brett C. wrote: > If there was no other way to get os.access-like functionality, I would > say it should be backported. But since there are other ways to figure > out everything that os.access can tell you I believe this is not really true, atleast not on Windows, and perhaps not in certain NFS cases, either. If there are ACLs on the file, access will consider them (or atleast it should), but no other method will. Regards, Martin From fdrake at acm.org Wed Mar 9 19:42:37 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Mar 9 19:42:42 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 In-Reply-To: <200503100406.09997.anthony@interlink.com.au> References: <200503091148.50639.fdrake@acm.org> <200503100406.09997.anthony@interlink.com.au> Message-ID: <200503091342.37208.fdrake@acm.org> On Wednesday 09 March 2005 12:06, Anthony Baxter wrote: > No - Py2.4 shipped with idle 1.1. I must've updated it to 1.1.1 sometime > after the 2.4 release (I vaguely recall doing this). Ah, ok. I guess I should have looked. -Fred -- Fred L. Drake, Jr. From aahz at pythoncraft.com Wed Mar 9 19:44:57 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed Mar 9 19:45:00 2005 Subject: [Python-Dev] scrambling python in a microsoft .exe container In-Reply-To: <20050309160841.26143.qmail@mail.kemm.de> References: <20050309160841.26143.qmail@mail.kemm.de> Message-ID: <20050309184457.GA19962@panix.com> On Wed, Mar 09, 2005, Niklas Fischer wrote: > > i lost the fight against google to find a wrapper/scrambler for python - > boxed in an exe file. > even the mandatory '-monty' string didn t help to find any tools. > > anyone got experience with such a tool? It's not clear what you're looking for, but it's pretty clear that python-dev is the wrong place to ask. Please switch to comp.lang.python. Thanks. (python-dev should only be used for people working on Python releases.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From mwh at python.net Wed Mar 9 20:04:19 2005 From: mwh at python.net (Michael Hudson) Date: Wed Mar 9 20:04:21 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <200503100340.15327.anthony@interlink.com.au> (Anthony Baxter's message of "Thu, 10 Mar 2005 03:40:13 +1100") References: <200503092317.07909.anthony@interlink.com.au> <422F1864.3040305@zope.com> <200503100340.15327.anthony@interlink.com.au> Message-ID: <2mis40od3g.fsf@starship.python.net> Anthony Baxter writes: > My google-fu returned, and I found the piece I was looking for earlier. > This discusses (from an internal Sun perspective) some of the problems > with Java. They make quite a big deal about the problem of changing > code across minor releases. I recall (re)reading this at some point and it > helped me clarify some ideas floating around in my head. > > http://www.internalmemos.com/memos/memodetails.php?memo_id=1321 Thanks for posting that, it was interesting. Cheers, mwh -- NUTRIMAT: That drink was individually tailored to meet your personal requirements for nutrition and pleasure. ARTHUR: Ah. So I'm a masochist on a diet am I? -- The Hitch-Hikers Guide to the Galaxy, Episode 9 From mal at egenix.com Wed Mar 9 20:04:40 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Mar 9 20:04:46 2005 Subject: [Python-Dev] unicode inconsistency? In-Reply-To: <20050309181630.GB4739@mems-exchange.org> References: <422ECBB3.90400@egenix.com> <20050309181630.GB4739@mems-exchange.org> Message-ID: <422F48C8.7010007@egenix.com> Neil Schemenauer wrote: > On Wed, Mar 09, 2005 at 11:10:59AM +0100, M.-A. Lemburg wrote: > >>The patch implements the PyObjbect_Text() idea (an API that >>returns a basestring instance, ie. string or unicode) and >>then uses this in '%s' (the string version) to properly propogate >>to u'%s' (the unicode version). >> >>Maybe we should also expose the C API as suggested in the patch, >>e.g. as text(obj). > > > Perhaps the right thing to do is introduce a new format code that > means insert text(obj) instead of str(obj), e.g %t. If we do that > though then we should make "'%s' % u'xyz'" return a string instead of > a unicode object. I suspect that would break a lot of code. It would result in lots of UnicodeErrors due to failing conversion of the Unicode string to a string. Plus it would break with the general rule of always coercing to Unicode (see below) and lose us the ability to write polymorphic code. > OTOH, having %s mean text(obj) instead of str(obj) may work just > fine. People who want it to mean str() generally don't have any > unicode strings floating around so text() has the same effect. > People who are using unicode probably would find text() to be more > useful behavior. I think that's why someone hacked PyString_Format > to sometimes return unicode strings. That wasn't a hack: it's part of the Unicode integration logic which always coerces to Unicode if strings and Unicode meet. In the above case a string format string meets a Unicode object as argument which then results in a Unicode object to be returned. > Regarding the use of __str__, to return a unicode object: we could > introduce a new slot (e.g. __text__) instead. However, I can't see > any advantage to that. If someone really wants a str object then > they call str() or PyObject_Str(). Right. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 09 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From martin at v.loewis.de Wed Mar 9 21:01:40 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 9 21:01:43 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> Message-ID: <422F5624.5020901@v.loewis.de> Delaney, Timothy C (Timothy) wrote: > Does anyone else think it would be worthwhile adding these to > collections, or should I just make a cookbook entry? -0 for the addition to the collections module, -1 on the specific name. Java made a *big* mistake by putting "Hash" into the names of these things. From the outside, it is primarily a Dictionary; only when you look closer you see that this specific dictionary requires hashable keys (as opposed to, say, the C++ std::map, which requires sortable keys). So consequently, the data type should be called OrderedDictionary, and its cookbook recipe is http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/107747 Regards, Martin From glyph at divmod.com Wed Mar 9 21:08:27 2005 From: glyph at divmod.com (Glyph Lefkowitz) Date: Wed Mar 9 21:07:05 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: References: <200503092317.07909.anthony@interlink.com.au> <422F1864.3040305@zope.com> Message-ID: <422F57BB.4070206@divmod.com> Guido van Rossum wrote: > On Wed, 09 Mar 2005 10:38:12 -0500, Jim Fulton wrote: > >>+1 (wish * could say +sys.maxint). > > > Anthony gets my +1 too, which then adds up to sys.maxint. :-) +1! I hope this mailing list runs on python2.2+, or the discussion has to stop now due to OverflowError... From theller at python.net Wed Mar 9 21:42:32 2005 From: theller at python.net (Thomas Heller) Date: Wed Mar 9 21:42:34 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <20050309023209.GB3553@cthulhu.gerg.ca> (Greg Ward's message of "Tue, 8 Mar 2005 21:32:09 -0500") References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> Message-ID: <3bv4wnyf.fsf@python.net> [About an ordered dictionary] > Delaney, Timothy C (Timothy) wrote: >> Does anyone else think it would be worthwhile adding these to >> collections, or should I just make a cookbook entry? Greg Ward writes: > +1 on a cookbook entry. +0 on adding to stdlib. "Martin v. L?wis" writes: > -0 for the addition to the collections module, -1 on the specific > name. I cannot understand why people are against adding it to stdlib (after the name, the implementation, and the exact place have been decided). It's certainly a useful data type, isn't it? Thomas From steven.bethard at gmail.com Wed Mar 9 22:01:44 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed Mar 9 22:01:47 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <3bv4wnyf.fsf@python.net> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> Message-ID: Thomas Heller wrote: > [About an ordered dictionary] [snip] > I cannot understand why people are against adding it to stdlib (after > the name, the implementation, and the exact place have been decided). > It's certainly a useful data type, isn't it? Well, that was basically the question I posed. So far I've seen only one use for it, and that one is better served by adding a function to itertools. What use do you have for it other than filtering duplicates from a list while retaining order? Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From bac at OCF.Berkeley.EDU Wed Mar 9 22:15:06 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Mar 9 22:15:25 2005 Subject: [Python-Dev] @deprecated (was: Useful thread project for 2.5?) In-Reply-To: <200503090911.15625.srichter@cosmos.phy.tufts.edu> References: <200503090911.15625.srichter@cosmos.phy.tufts.edu> Message-ID: <422F675A.7090207@ocf.berkeley.edu> Stephan Richter wrote: [SNIP - Michael's deprecated decorator] > > This is a recipe for disaster. Creating a new function from the old can have > unwanted side effects, since you effectively change the object. For example, > if someone is monkey patching this function, then the deprecation warning > gets lost. > That's the idiom for decorators. They will almost always be wrappers for other functions by returning a new function that uses closures to access the original. Besides, you no longer have the original function directly available in the global namespace of the module since the name of the decorated function gets rebound before you have a chance to see the original function. So there really is no issue with the function not being the exact same since you can't really see it undecorated without a lot of effort; no real change of the object that the user can ever tell. And as for monkey-patching breaking something, that's there fault for monkey-patching. Python doesn't prevent you from doing something stupid which why it is the language four out of five adults choose to code in (the fifth one, the hold-out, just can't let go of Java because he/she is a recent college grad and it is all they have ever known; they also think that EAFP is "evil" =). -Brett From theller at python.net Wed Mar 9 22:19:39 2005 From: theller at python.net (Thomas Heller) Date: Wed Mar 9 22:19:41 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: (Steven Bethard's message of "Wed, 9 Mar 2005 14:01:44 -0700") References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> Message-ID: Steven Bethard writes: > Thomas Heller wrote: >> [About an ordered dictionary] > [snip] >> I cannot understand why people are against adding it to stdlib (after >> the name, the implementation, and the exact place have been decided). >> It's certainly a useful data type, isn't it? > > Well, that was basically the question I posed. So far I've seen only > one use for it, and that one is better served by adding a function to > itertools. Hm, removing duplicates from a list is an algorithm, not a data structure. And the code you posted (no offense intended) is, also imo, faster written by an experienced programmer than located in some module. OTOH, I see no problem adding it to itertools. > What use do you have for it other than filtering > duplicates from a list while retaining order? If this were the only use case, you are right. I cannot remember the use case I once had right now, and probably the code has been rewritten anyway. But I assume use cases exist, otherwise there weren't so many recipes for the ordered dictionary. Thomas From steven.bethard at gmail.com Wed Mar 9 22:30:18 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed Mar 9 22:30:21 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> Message-ID: Thomas Heller wrote: > Steven Bethard writes: > > > What use do you have for it other than filtering > > duplicates from a list while retaining order? > > If this were the only use case, you are right. I cannot remember the > use case I once had right now, and probably the code has been rewritten > anyway. But I assume use cases exist, otherwise there weren't so many > recipes for the ordered dictionary. Sorry, I didn't mean to come across as suggesting that there weren't other use cases for it. I'm only asking for people who know these other use cases to present them here. At the moment, we have only one use case, and for that use case, OrderedDict/OrderedSet isn't really the best solution. (That's all my code was intended to point out -- I don't really care if it makes it into itertools or not.) If people could present a few reasonable problems where OrderedDict/OrderedSet really do provide the best solutions, and then make the case that these use cases are reasonably frequent, I think there would be much better support for adding such types to the standard library. Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From foom at fuhm.net Wed Mar 9 23:15:58 2005 From: foom at fuhm.net (James Y Knight) Date: Wed Mar 9 23:16:15 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> Message-ID: <4c0eb0ae19d775d9efa65bb96c5599a2@fuhm.net> On Mar 9, 2005, at 4:19 PM, Thomas Heller wrote: > If this were the only use case, you are right. I cannot remember the > use case I once had right now, and probably the code has been rewritten > anyway. But I assume use cases exist, otherwise there weren't so many > recipes for the ordered dictionary. I use ordered dictionaries for testing. With an ordered dict I can string compare the output of my program to what is expected. Without an ordered dict, I'd have to re-parse the output and order it, which would require some complicated code that's just as likely to be wrong as the code I'm trying to test. James From martin at v.loewis.de Wed Mar 9 23:31:45 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 9 23:31:47 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <3bv4wnyf.fsf@python.net> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> Message-ID: <422F7951.3060809@v.loewis.de> Thomas Heller wrote: > I cannot understand why people are against adding it to stdlib (after > the name, the implementation, and the exact place have been decided). > It's certainly a useful data type, isn't it? It depends on the precise semantics. You often want a dictionary where the keys come out in some order - but that is rarely the order in which they were added to the dictionary. Most frequently, you want the keys sorted, according to some criteria. If not that, I would assume that you typically have the order of keys determined before even filling the dictionary, in which case you can do for k in keys_in_preferred_order: v = hashtable[k] process(k,v) I remember having needed that once in the past 15 years (in Smalltalk at the time), so I wrote an OrderedDictionary for Smalltalk/V (which didn't have it). It took me an hour or so. I don't recall what precisely it was that I needed it for, and I cannot think of any use case for the data type right now. So I'm -0 on adding the data type: I have a vague feeling it is needed, but rarely, and I don't know precisely what for. Regards, Martin From martin at v.loewis.de Wed Mar 9 23:34:23 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 9 23:34:27 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <4c0eb0ae19d775d9efa65bb96c5599a2@fuhm.net> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> <4c0eb0ae19d775d9efa65bb96c5599a2@fuhm.net> Message-ID: <422F79EF.7000207@v.loewis.de> James Y Knight wrote: > I use ordered dictionaries for testing. With an ordered dict I can > string compare the output of my program to what is expected. Without an > ordered dict, I'd have to re-parse the output and order it, which would > require some complicated code that's just as likely to be wrong as the > code I'm trying to test. I see. I would argue that you were better off if the test cases were sorted (according to some total, stable-across-releases, order), rather than being ordered. Regards, Martin From abo at minkirri.apana.org.au Wed Mar 9 23:41:29 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Wed Mar 9 23:41:40 2005 Subject: [Python-Dev] Re: No new features References: <200503091208.38519.anthony@interlink.com.au><20050309012152.GC30204@cthulhu.gerg.ca><200503092207.22313.anthony@interlink.com.au><007d01c524a4$dd17fb20$24ed0ccb@apana.org.au> <2mr7ipnfoj.fsf@starship.python.net> Message-ID: <00e501c524f9$2271ff00$24ed0ccb@apana.org.au> G'day again, From: "Michael Hudson" > "Donovan Baarda" writes: > > > > > Just my 2c; > > > > I don't mind new features in minor releases, provided they meet the > > following two criteria; > > > > 1) Don't break the old API! The new features must be pure extensions that in > > no way change the old API. Any existing code should be un-effected in any > > way by the change. > > > > 2) The new features must be clearly documented as "New in version X.Y.Z". > > This way people using these features will know the minium Python version > > required for their application. > > No no no! The point of what Anthony is saying, as I read it, is that > experience suggests it is exactly this sort of change that should be > avoided. Consider the case of Mac OS X 10.2 which came with Python > 2.2.0: this was pretty broken anyway because of some apple snafus but > it was made even more useless by the fact that people out in the wild > were writing code for 2.2.1 and using True/False/bool. Going from > 2.x.y to 2.x.y+1 shouldn't break anything, going from 2.x.y+1 to 2.x.y > shouldn't break anything that doesn't whack into a bug in 2.x.y -- and > "not having bool" isn't a bug in this sense. You missed the "minor releases" bit in my post. major releases, ie 2.x -> 3.0, are for things that can break existing code. They change the API so that things that run on 2.x may not work with 3.x. minor releases, ie 2.2.x ->2.3.0, are for things that cannot break existing code. They can extend the API such that code for 2.3.x may not work on 2.2.x, but code that runs on 2.2.x must work on 2.3.x. micro releases, ie 2.2.1 ->2.2.2, are for bug fixes only. There can be no changes to the API, so that all code that runs on 2.2.2 should work with 2.2.1, barring the bugs fixed. The example you cited of adding bool was an extension to the API, and hence should have been a minor release, not a micro release. I just read the PEP-6, and it doesn't seem to use this terminology, or make this distinction... does something else do this anywhere? I thought this approach was common knowledge... ---------------------------------------------------------------- Donovan Baarda http://minkirri.apana.org.au/~abo/ ---------------------------------------------------------------- From jrw at pobox.com Thu Mar 10 00:14:35 2005 From: jrw at pobox.com (John Williams) Date: Thu Mar 10 00:14:39 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents Message-ID: <422F835B.6040904@pobox.com> Steven Bethard wrote: > Thomas Heller wrote: > >> [About an ordered dictionary] > > > Well, that was basically the question I posed. So far I've seen only > one use for it, and that one is better served by adding a function to > itertools. What use do you have for it other than filtering > duplicates from a list while retaining order? > > Steve Using a LinkedHashMap generally cuts down in the amount of apparent randomness in a program. This is especially helpful when it comes time to debug a really complicated program by diffing log files, since it prevents slightly different maps from having wildly different iteration orders. Often using a plain HashMap can introduce enough randomness to make two otherwise similar log files nearly impossible to compare. The other use case I have is for dealing with data where the iteration order doesn't matter to the program but it does matter to users. For instance, consider the ConfigParser's write method. Any ordering of values in the output is functionally equivalent, but the original data is likely to have come from a file that was arranged in some meaningful order, and it would be nice to preserve that order, especially if it can be done with no extra effort. --jw From aahz at pythoncraft.com Thu Mar 10 00:23:10 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu Mar 10 00:23:13 2005 Subject: [Python-Dev] Re: No new features In-Reply-To: <00e501c524f9$2271ff00$24ed0ccb@apana.org.au> References: <2mr7ipnfoj.fsf@starship.python.net> <00e501c524f9$2271ff00$24ed0ccb@apana.org.au> Message-ID: <20050309232310.GA1531@panix.com> On Thu, Mar 10, 2005, Donovan Baarda wrote: > > major releases, ie 2.x -> 3.0, are for things that can break existing code. > They change the API so that things that run on 2.x may not work with 3.x. > > minor releases, ie 2.2.x ->2.3.0, are for things that cannot break existing > code. They can extend the API such that code for 2.3.x may not work on > 2.2.x, but code that runs on 2.2.x must work on 2.3.x. > > micro releases, ie 2.2.1 ->2.2.2, are for bug fixes only. There can be no > changes to the API, so that all code that runs on 2.2.2 should work with > 2.2.1, barring the bugs fixed. > > The example you cited of adding bool was an extension to the API, and hence > should have been a minor release, not a micro release. > > I just read the PEP-6, and it doesn't seem to use this terminology, or make > this distinction... does something else do this anywhere? I thought this > approach was common knowledge... Functionally speaking, Python has only major releases and micro releases. We don't have the resources to support minor releases. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From martin at v.loewis.de Thu Mar 10 00:45:08 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Mar 10 00:45:14 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <422F835B.6040904@pobox.com> References: <422F835B.6040904@pobox.com> Message-ID: <422F8A84.5070005@v.loewis.de> John Williams wrote: > The other use case I have is for dealing with data where the iteration > order doesn't matter to the program but it does matter to users. For > instance, consider the ConfigParser's write method. Any ordering of > values in the output is functionally equivalent, but the original data > is likely to have come from a file that was arranged in some meaningful > order, and it would be nice to preserve that order, especially if it can > be done with no extra effort. That is a good example, IMO. But then, in the specific case, the dictionary for each section is created deeply inside ConfigParser, with the lines cursect = {'__name__': sectname} self._sections[sectname] = cursect So this uses a builtin dict, and there is no easy way to override it for the application. Of course, given your reasoning, ConfigParser *should* use an OrderedDictionary (probably both for cursect and for self._sections), in which case this would all be transparent to the application. Regards, Martin From listsub at wickedgrey.com Thu Mar 10 01:17:49 2005 From: listsub at wickedgrey.com (Eli Stevens (WG.c)) Date: Thu Mar 10 01:17:52 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> Message-ID: <422F922D.4060409@wickedgrey.com> Steven Bethard wrote: > Thomas Heller wrote: > >>[About an ordered dictionary] > > Well, that was basically the question I posed. So far I've seen only > one use for it, and that one is better served by adding a function to > itertools. What use do you have for it other than filtering > duplicates from a list while retaining order? The primary use case I have deals with DB result sets*. The ordering of the rows returned from a query is important, so keeping the iteration order is nice. Most of the tables I deal with have keys of some kind, and being able to pull out a result row by key is also nice. Granted, I rarely use /both/ at the same time, but it is nice to not have to specify how the result set will be used when I retrieve it. To me, this seems to be the same concept as the config file parsing previously mentioned. I don't feel qualified to have an opinion** about inclusion in the stdlib, much less vote. relurkin'ly yrs, Eli [*] - In my case, it's actually coded in Java, for work. There might be a reason that this problem isn't language-generic, but the 1.5 minutes I spent thinking about it were not illuminating. [**] - Yet I have one anyway. This kind of datatype seems one of those easy-to-get-half-right things that could benefit from a solid implementation. It also doesn't strike me as controversial in terms of API or internal structure. From tommy at ilm.com Thu Mar 10 01:39:42 2005 From: tommy at ilm.com (Tommy Burnette) Date: Thu Mar 10 01:39:48 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <422F7951.3060809@v.loewis.de> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> <422F7951.3060809@v.loewis.de> Message-ID: <16943.38734.34295.653190@evoke.lucasdigital.com> I'd say I'm +0. fwiw- I've been using a locally-rolled OrderedDict implementation for the last 5-6 years in which insertion order is the only order respected. I use it all over the place (in a code base of ~60k lines of python code). so there's another use case for you. bust as you say, easy to do yourself... =?ISO-8859-1?Q?"Martin_v._L=F6wis"?= writes: | Thomas Heller wrote: | > I cannot understand why people are against adding it to stdlib (after | > the name, the implementation, and the exact place have been decided). | > It's certainly a useful data type, isn't it? | | It depends on the precise semantics. You often want a dictionary where | the keys come out in some order - but that is rarely the order in which | they were added to the dictionary. Most frequently, you want the keys | sorted, according to some criteria. If not that, I would assume that you | typically have the order of keys determined before even filling the | dictionary, in which case you can do | | for k in keys_in_preferred_order: | v = hashtable[k] | process(k,v) | | I remember having needed that once in the past 15 years (in Smalltalk | at the time), so I wrote an OrderedDictionary for Smalltalk/V (which | didn't have it). It took me an hour or so. | | I don't recall what precisely it was that I needed it for, and I cannot | think of any use case for the data type right now. | | So I'm -0 on adding the data type: I have a vague feeling it is needed, | but rarely, and I don't know precisely what for. | | Regards, | Martin | _______________________________________________ | Python-Dev mailing list | Python-Dev@python.org | http://mail.python.org/mailman/listinfo/python-dev | Unsubscribe: http://mail.python.org/mailman/options/python-dev/tommy%40ilm.com From gvanrossum at gmail.com Thu Mar 10 02:28:57 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Mar 10 02:28:59 2005 Subject: [Python-Dev] @deprecated (was: Useful thread project for 2.5?) In-Reply-To: <200503090911.15625.srichter@cosmos.phy.tufts.edu> References: <200503090911.15625.srichter@cosmos.phy.tufts.edu> Message-ID: > This is a recipe for disaster. Creating a new function from the old can have > unwanted side effects, since you effectively change the object. For example, > if someone is monkey patching this function, then the deprecation warning > gets lost. That's a rather extreme use case, and not one that IMO should block the @deprecated decorator from being used. I hope that monkey patching is rare enough that you shouldn't mind checking once a year of so if the thing you're monkey-patching might have been deprecated (in which case you shouldn't be monkey-patching it but instead rewrite your code to avoid it altogether). > In Zope 3's deprecation package, we have decided to put a special deprecation > proxy around the module (instead of the function) and register a list of > attribute names (note that it does not matter whether its a class, function > or other type of object) that are deprecated. The advantage is that you get a > deprecation warning just by looking up the object. Yeah, but not everybody has Zope's proxying machinery. I think you're overreacting. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Thu Mar 10 02:35:48 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu Mar 10 02:35:51 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <16943.38734.34295.653190@evoke.lucasdigital.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> <422F7951.3060809@v.loewis.de> <16943.38734.34295.653190@evoke.lucasdigital.com> Message-ID: <20050310013548.GA14791@panix.com> On Wed, Mar 09, 2005, Tommy Burnette wrote: > > I'd say I'm +0. fwiw- I've been using a locally-rolled OrderedDict > implementation for the last 5-6 years in which insertion order is the > only order respected. I use it all over the place (in a code base of > ~60k lines of python code). > > so there's another use case for you. bust as you say, easy to do > yourself... Gee, I just found out I could have used an OrderedDict today. (We're using a dict that we're now having to add an auxilliary list to to track when keys are added.) (This isn't particularly an argument in favor of adding OrderedDict to stdlib, but it's another use case. Each dict key contains a dict value; the subkeys from later-added keys are supposed to override earlier subkeys. The original implementation relied on subkeys being unique, but that doesn't work for our new business requirements.) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From python at rcn.com Thu Mar 10 04:21:18 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu Mar 10 04:26:08 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <20050310013548.GA14791@panix.com> Message-ID: <002301c52520$39da0da0$163cc797@oemcomputer> [Aahz] > Gee, I just found out I could have used an OrderedDict today. (We're > using a dict that we're now having to add an auxilliary list to to track > when keys are added.) (This isn't particularly an argument in favor of > adding OrderedDict to stdlib, but it's another use case. Each dict key > contains a dict value; the subkeys from later-added keys are supposed to > override earlier subkeys. The original implementation relied on subkeys > being unique, but that doesn't work for our new business requirements.) If I read the proposal correctly, order would be determined by when the key was first encountered. Presumably, that would mean that the related value would also be the first encountered (not overridden by later-added keys as dictated by your business requirements). Raymond From aahz at pythoncraft.com Thu Mar 10 05:16:36 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu Mar 10 05:16:37 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <002301c52520$39da0da0$163cc797@oemcomputer> References: <20050310013548.GA14791@panix.com> <002301c52520$39da0da0$163cc797@oemcomputer> Message-ID: <20050310041636.GA26264@panix.com> On Wed, Mar 09, 2005, Raymond Hettinger wrote: > [Aahz] >> >> Gee, I just found out I could have used an OrderedDict today. (We're >> using a dict that we're now having to add an auxilliary list to to track >> when keys are added.) (This isn't particularly an argument in favor of >> adding OrderedDict to stdlib, but it's another use case. Each dict key >> contains a dict value; the subkeys from later-added keys are supposed to >> override earlier subkeys. The original implementation relied on subkeys >> being unique, but that doesn't work for our new business requirements.) > > If I read the proposal correctly, order would be determined by when the > key was first encountered. Presumably, that would mean that the related > value would also be the first encountered (not overridden by later-added > keys as dictated by your business requirements). Hmmmmm.... Guess this means we need a PEP! -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From jbauer at rubic.com Thu Mar 10 05:33:12 2005 From: jbauer at rubic.com (Jeff Bauer) Date: Thu Mar 10 05:33:13 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents Message-ID: <422FCE08.5020804@rubic.com> Thomas Heller wrote: > Steven Bethard writes: > >> What use do you have for it other than filtering >> duplicates from a list while retaining order? > > If this were the only use case, you are right. I cannot > remember the use case I once had right now, and probably > the code has been rewritten anyway. But I assume use > cases exist, otherwise there weren't so many recipes > for the ordered dictionary. I'm not specifically lobbying for its inclusion in stdlib, but I often find an ordered dict useful when I want both ordered and random access, e.g. situations: - database table fields/attributes - drop down field selectors In both cases, I could get by with other techniques, but I would use stdlib ordered dicts if they were available. I also prefer the term "ordered dict" to LinkedHashXXX. Jeff Bauer Rubicon, Inc. From tdelaney at avaya.com Thu Mar 10 05:55:45 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Mar 10 05:55:41 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE721257@au3010avexu1.global.avaya.com> Jeff Bauer wrote: > I'm not specifically lobbying for its inclusion in > stdlib, but I often find an ordered dict useful when I > want both ordered and random access, e.g. situations: > > - database table fields/attributes > - drop down field selectors Yep - these are also cases that are familiar to me - it's the type of thing you don't think about until you actually need it. > In both cases, I could get by with other techniques, but I > would use stdlib ordered dicts if they were available. > I also prefer the term "ordered dict" to LinkedHashXXX. You may notice the subject is LinkedHashXXX *equivalents* ;) There is no way I would ever propose adding them with those names. OTOH, "ordered set" and "ordered dict" implies different things to different people - usually "sorted" rather than "the order things were put in". Perhaps "temporally-ordered" ;) BTW, just to clarify the semantics: Set: Items are iterated over in the order that they are added. Adding an item that compares equal to one that is already in the set does not replace the item already in the set, and does not change the iteration order. Removing an item, then re-adding it moves the item to the end of the iteration order. Dict: Keys are iterated over in the order that they are added. Setting a value using a key that compares equal to one already in the dict replaces the value, but not the key, and does not change the iteration order. Removing a key (and value) then re-adding it moves the key to the end of the iteration order. Tim Delaney From tdelaney at avaya.com Thu Mar 10 06:00:20 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Mar 10 06:00:18 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE025203FD@au3010avexu1.global.avaya.com> Steven Bethard wrote: > def filterdups(iterable): > seen = set() > for item in iterable: > if item not in seen: > seen.add(item) > yield item > > Adding this to, say, itertools would cover all my use cases. And as > long as you don't have too many duplicates, filterdups as above should > keep memory consumption down better. Thinking about this further - memory usage would be almost identical. By the time you completed the iterable, you would have built up exactly the same set internally - although probably not as memory efficient since it would be being built piecemeal. OTOH, an ordered set has a bit of extra memory for maintaining the order, so it's going to be pretty close. The only thing this gains you (and it's significant) is the ability to work on any iterable lazily. Tim Delaney From stephan.richter at tufts.edu Wed Mar 9 15:07:06 2005 From: stephan.richter at tufts.edu (Stephan Richter) Date: Thu Mar 10 06:36:39 2005 Subject: [Python-Dev] @deprecated (was: Useful thread project for 2.5?) In-Reply-To: References: Message-ID: <200503090907.07130.stephan.richter@tufts.edu> On Tuesday 08 March 2005 18:05, Jim Jewett wrote: > ? ? ... compiler recognition of "@deprecated" in doc comments. > > Michael Chermside suggested: > > ? ? ===== code ===== > ? ? import warnings > > ? ? def deprecated(func): > ? ? ? ? """This is a decorator which can be used to mark functions > ? ? ? ? as deprecated. It will result in a warning being emmitted > ? ? ? ? when the function is used.""" > ? ? ? ? def newFunc(*args, **kwargs): > ? ? ? ? ? ? warnings.warn("Call to deprecated function.") > ? ? ? ? ? ? return func(*args, **kwargs) > ? ? ? ? return newFunc > > ? ? ===== example This is a recipe for disaster. Creating a new function from the old can have unwanted side effects, since you effectively change the object. For example, if someone is monkey patching this function, then the deprecation warning gets lost. In Zope 3's deprecation package, we have decided to put a special deprecation proxy around the module (instead of the function) and register a list of attribute names (note that it does not matter whether its a class, function or other type of object) that are deprecated. The advantage is that you get a deprecation warning just by looking up the object. See: http://svn.zope.org/Zope3/trunk/src/zope/deprecation/ Disclaimer: This code is a bit experimental and might not be the best solution. It is currently used in the trunk and does a pretty good job, though. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training From steven.bethard at gmail.com Thu Mar 10 06:46:16 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu Mar 10 06:46:19 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE025203FD@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE025203FD@au3010avexu1.global.avaya.com> Message-ID: Delaney, Timothy C (Timothy) wrote: > Steven Bethard wrote: > > > def filterdups(iterable): > > seen = set() > > for item in iterable: > > if item not in seen: > > seen.add(item) > > yield item > > > > Adding this to, say, itertools would cover all my use cases. And as > > long as you don't have too many duplicates, filterdups as above should > > keep memory consumption down better. > > Thinking about this further - memory usage would be almost identical. By > the time you completed the iterable, you would have built up exactly the > same set internally - although probably not as memory efficient since it > would be being built piecemeal. OTOH, an ordered set has a bit of extra > memory for maintaining the order, so it's going to be pretty close. Definitely true that if you iterate through your entire iterable, it doesn't gain you anything over an OrderedSet. OTOH, if you only end up looking at the first N elements of the iterable, then this would be more efficient than putting the entire iterable into an OrderedSet and taking the first N from there. Of course you can put only the first elements into the OrderedSet, but note that you can't just put in the first N; you have to add elements of the iterable into the OrderedSet until it has len(obj) == N. Not that this should be more than a few lines of code, but it's code that you wouldn't have to write with fitlerdups. That said, you're right of course that filterdups is certainly not a big win over OrderedSet. I was really just trying to point out that if we were just trying to provide a solution to the filtering-duplicates-while-maintaining-order problem that OrderedSet wasn't the only path to that goal. I think since then there have been a number of other reasonable cases suggested for wanting an OrderedSet, e.g.: * A DB result set that is indexed by a key but also maintains row order [Eli Stevens, Jeff Bauer] * A config-file data structure that indexes by option names but maintains the order of elements read from a config file [John Williams] * Drop-down field selectors indexed by both name and sequence position [Jeff Bauer] So I'm now probably +0.5 on adding an OrderedSet to collections. Not that my opinion is particularly influential, of course. ;-) Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From python at rcn.com Thu Mar 10 07:29:51 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu Mar 10 07:34:49 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <20050310041636.GA26264@panix.com> Message-ID: <000301c5253a$910ff480$163cc797@oemcomputer> > > If I read the proposal correctly, order would be determined by when the > > key was first encountered. Presumably, that would mean that the related > > value would also be the first encountered (not overridden by later-added > > keys as dictated by your business requirements). > > Hmmmmm.... Guess this means we need a PEP! Or the implementation can have a switch to choose between keep-first logic or replace logic. The latter seems a bit odd to me. The key position would be determined by the first encountered while the value would be determined by the last encountered. Starting with [(10, v1), (20, v2), (10.0, v3)], the ordered dictionary's items would look like [(10, v3), (20, v2)]. Raymond From nbastin at opnet.com Thu Mar 10 07:38:35 2005 From: nbastin at opnet.com (Nicholas Bastin) Date: Thu Mar 10 07:38:49 2005 Subject: [Python-Dev] SWT PyCon Sprint? Message-ID: I realize that this is exceedingly late in the game, but is anybody interested in doing a Write-Python-Bindings-for-SWT sprint? It's been brought up before in various places, and PyCon seems the likely place to get enough concentrated knowledge to actually get it kicked off and somewhat working... -- Nick From mwh at python.net Thu Mar 10 09:19:18 2005 From: mwh at python.net (Michael Hudson) Date: Thu Mar 10 09:19:20 2005 Subject: [Python-Dev] Re: No new features In-Reply-To: <00e501c524f9$2271ff00$24ed0ccb@apana.org.au> (Donovan Baarda's message of "Thu, 10 Mar 2005 09:41:29 +1100") References: <200503091208.38519.anthony@interlink.com.au> <20050309012152.GC30204@cthulhu.gerg.ca> <200503092207.22313.anthony@interlink.com.au> <007d01c524a4$dd17fb20$24ed0ccb@apana.org.au> <2mr7ipnfoj.fsf@starship.python.net> <00e501c524f9$2271ff00$24ed0ccb@apana.org.au> Message-ID: <2macpboqux.fsf@starship.python.net> "Donovan Baarda" writes: > G'day again, [...] > You missed the "minor releases" bit in my post. > > major releases, ie 2.x -> 3.0, are for things that can break existing code. > They change the API so that things that run on 2.x may not work with 3.x. > > minor releases, ie 2.2.x ->2.3.0, are for things that cannot break existing > code. They can extend the API such that code for 2.3.x may not work on > 2.2.x, but code that runs on 2.2.x must work on 2.3.x. > > micro releases, ie 2.2.1 ->2.2.2, are for bug fixes only. There can be no > changes to the API, so that all code that runs on 2.2.2 should work with > 2.2.1, barring the bugs fixed. > > The example you cited of adding bool was an extension to the API, and hence > should have been a minor release, not a micro release. > > I just read the PEP-6, and it doesn't seem to use this terminology, or make > this distinction... does something else do this anywhere? I thought this > approach was common knowledge... I see. You were proposing a much larger change to the way Python releases work than I (and perhaps you? :) realised. Not breaking any code 2.x to 2.x+1 is a nice idea, but doesn't really seem feasible in practice. Cheers, mwh -- nonono, while we're making wild conjectures about the behavior of completely irrelevant tasks, we must not also make serious mistakes, or the data might suddenly become statistically valid. -- Erik Naggum, comp.lang.lisp From mwh at python.net Thu Mar 10 09:42:55 2005 From: mwh at python.net (Michael Hudson) Date: Thu Mar 10 09:42:57 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE721257@au3010avexu1.global.avaya.com> (Timothy C. Delaney's message of "Thu, 10 Mar 2005 15:55:45 +1100") References: <338366A6D2E2CA4C9DAEAE652E12A1DE721257@au3010avexu1.global.avaya.com> Message-ID: <2m64zzoprk.fsf@starship.python.net> "Delaney, Timothy C (Timothy)" writes: > Set: Items are iterated over in the order that they are added. Adding an > item that compares equal to one that is already in the set does not > replace the item already in the set, and does not change the iteration > order. Removing an item, then re-adding it moves the item to the end of > the iteration order. Well, this could be satisfied by an append_new operation on lists, right (thinking of Common Lisps #'cl:pushnew)? Complexity not that great, of course, but I've written code like: if a not in l: l.append(a) and not suffered that badly for it before now... > Dict: Keys are iterated over in the order that they are added. Setting a > value using a key that compares equal to one already in the dict > replaces the value, but not the key, and does not change the iteration > order. Removing a key (and value) then re-adding it moves the key to the > end of the iteration order. And these are what CL types call association lists, in effect. Cheers, mwh -- #ifndef P_tmpdir printf( "Go buy a better computer" ); exit( ETHESKYISFALLINGANDIWANTMYMAMA ); -- Dimitri Maziuk on writing secure code, asr From mal at egenix.com Thu Mar 10 11:15:42 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Mar 10 11:15:46 2005 Subject: [Python-Dev] Decimal & returning NotImplemented (or not) In-Reply-To: <42287FAD.70908@iinet.net.au> References: <001101c51e61$c57b99c0$ef26c797@oemcomputer> <42247207.4050402@iinet.net.au> <20050301225550.GB11698@mems-exchange.org> <20050302095515.GA14982@craig-wood.com> <4225956A.9060805@iinet.net.au> <42287FAD.70908@iinet.net.au> Message-ID: <42301E4E.6020209@egenix.com> Nick Coghlan wrote: > Guido van Rossum wrote: > >> No, the reason is that if we did this with exceptions, it would be >> liable to mask errors; an exception does not necessarily originate >> immediately with the code you invoked, it could have been raised by >> something else that was invoked by that code. The special value >> NotImplemented must pretty much originate directly in the invoked >> code, since the implementation guarantees that e.g. a+b can never >> *return* NotImplemented: if a.__add__(b) and b.__radd__(a) both return >> NotImplemented, TypeError is raised. > > > That makes sense - although for that reasoning, a TypeError subclass > that the binary operation machinery promotes to a standard TypeError > would seem to work too (with the advantage of also raising at least some > sort of exception when the method is called directly) I have to correct Guido's explanation a bit: the original reason for using a special singleton instead of an exception was indeed a significant difference in performance (raising exceptions at the Python level is expensive, not so much at the C level). I don't remember the details, but I did some measurements at the time which made the decision to use a singleton rather easy :-) The NotImplemented singleton was part of the coercion proposal: http://www.egenix.com/files/python/CoercionProposal.html (pretty old stuff, now that I look at it again ;-) That said, Guido's point is just valid. We would have had to add code that saves the current exception to avoid masking any pending errors - code like that always turns into a mess sooner or later. Cheers, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 10 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ncoghlan at iinet.net.au Thu Mar 10 12:19:16 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 10 12:19:23 2005 Subject: [Python-Dev] itemgetter/attrgetter extension In-Reply-To: <000e01c52430$4eff1fe0$7eb99d8d@oemcomputer> References: <000e01c52430$4eff1fe0$7eb99d8d@oemcomputer> Message-ID: <42302D34.60100@iinet.net.au> Raymond Hettinger wrote: > Any objections to extending itemgetter() and attrgetter() to be able to > extract multiple fields at a time? > > # SELECT name, rank, serialnum FROM soldierdata > map(attrgetter('name', 'rank', 'serialnum'), soldierdata) > > # SELECT * FROM soldierdata ORDER BY unit, rank, proficiency > sorted(soldierdata, key=attrgetter('unit', 'rank', 'proficiency')) Breaking the symmetry with getattr() bothers me a little, as does the signature change that occurs when multiple arguments are given (returning a tuple, whereas the single argument returns just the retrieved attribute). For itemgetter, I'd like to see multiple arguments eventually map to multi-dimensional slices (to preserve symmetry with indexing syntax). Call it -1 for itemgetter and -0 for attrgetter. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Thu Mar 10 12:28:22 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 10 12:28:28 2005 Subject: [Python-Dev] @deprecated (was: Useful thread project for 2.5?) In-Reply-To: <000f01c52433$d6f5eb60$7eb99d8d@oemcomputer> References: <000f01c52433$d6f5eb60$7eb99d8d@oemcomputer> Message-ID: <42302F56.7060702@iinet.net.au> Raymond Hettinger wrote: > Decorators like this should preserve information about the underlying > function: > > >> def deprecated(func): >> """This is a decorator which can be used to mark functions >> as deprecated. It will result in a warning being emmitted >> when the function is used.""" >> def newFunc(*args, **kwargs): >> warnings.warn("Call to deprecated function.") >> return func(*args, **kwargs) > > newFunc.__name__ = func.__name__ > newFunc.__doc__ = func.__doc__ > newFunc.__dict__.update(func.__dict__) > >> return newFunc A utility method on function objects could simplify this: newFunc.update_info(func) The main benefit I see is that an 'update_info' method is more future-proof. If some other attributes are added to function objects that should be preserved, update_info() can be updated in parallel to transfer them, and any code using the method continues to be correct. Regards, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Thu Mar 10 12:49:34 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 10 12:49:41 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE721257@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721257@au3010avexu1.global.avaya.com> Message-ID: <4230344E.9090905@iinet.net.au> Delaney, Timothy C (Timothy) wrote: > OTOH, "ordered set" and "ordered dict" implies different things to > different people - usually "sorted" rather than "the order things were > put in". Perhaps "temporally-ordered" ;) OTGH*, I would expect an OrderedDict / OrderedSet to have 'add to the end' semantics, but also provide a 'sort()' method so that the ordering could be changed at a later date. IOW, by default the ordering is temporal. Sorting the ordered dict/set changes the iteration order for the current contents. Further additions are still added in temporal order until such time as the dict/set is sorted again. The parallels are to using list.append() to build a list, and list.sort() to order the current contents (in fact, a simplistic approach could use that exact technique to remember the order of keys, at the cost of doubling key storage requirements). Cheers, Nick. *OTGH: On the gripping hand :) -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From barry at python.org Thu Mar 10 13:53:52 2005 From: barry at python.org (Barry Warsaw) Date: Thu Mar 10 13:53:56 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <16943.38734.34295.653190@evoke.lucasdigital.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721254@au3010avexu1.global.avaya.com> <20050309023209.GB3553@cthulhu.gerg.ca> <3bv4wnyf.fsf@python.net> <422F7951.3060809@v.loewis.de> <16943.38734.34295.653190@evoke.lucasdigital.com> Message-ID: <1110459232.7498.3.camel@presto.wooz.org> On Wed, 2005-03-09 at 19:39, Tommy Burnette wrote: > I'd say I'm +0. fwiw- I've been using a locally-rolled OrderedDict > implementation for the last 5-6 years in which insertion order is the > only order respected. I use it all over the place (in a code base of > ~60k lines of python code). > > so there's another use case for you. bust as you say, easy to do > yourself... I'm -0 on adding it to the stdlib, but mostly because I don't like the name, and I suspect it's going to be one of those nuggets lurking in the standard library that few people will use, tending either to overlook it or just roll their own anyway because the standard one doesn't have quite the right semantics. FWIW, email.Message.Message /exposes/ an ordered dictionary-like interface even though it's implemented as a simple list. It was considered at the time that the number of headers in an email message wouldn't be so large that anything else would be worth the complexity. I think that still holds, for the original uses cases at least. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050310/f6ace154/attachment.pgp From barry at python.org Thu Mar 10 13:57:02 2005 From: barry at python.org (Barry Warsaw) Date: Thu Mar 10 13:57:04 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <000301c5253a$910ff480$163cc797@oemcomputer> References: <000301c5253a$910ff480$163cc797@oemcomputer> Message-ID: <1110459422.7493.7.camel@presto.wooz.org> On Thu, 2005-03-10 at 01:29, Raymond Hettinger wrote: > Or the implementation can have a switch to choose between keep-first > logic or replace logic. This is what I meant by my previous follow up: while the concept of "an ordered dictionary" is nice and seemingly generic enough, in practice I suspect that exact semantics and other design factors will either tend to push the stdlib implementation into ever more complexity, or won't prevent people from continuing to roll their own because the stdlib version "isn't quite right". -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050310/c51807ec/attachment.pgp From bjourne at gmail.com Thu Mar 10 14:07:04 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Thu Mar 10 14:07:07 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <4230344E.9090905@iinet.net.au> References: <338366A6D2E2CA4C9DAEAE652E12A1DE721257@au3010avexu1.global.avaya.com> <4230344E.9090905@iinet.net.au> Message-ID: <740c3aec05031005072d3d34f8@mail.gmail.com> I would LOVE for **kwargs to be an ordered dict. It would allow me to write code like this: .class MyTuple: . def __init__(self, **kwargs): . self.__dict__ = ordereddict(kwargs) . . def __iter__(self): . for k, v in self.__dict__.items(): . yield v . .t = MyTuple(r = 99, g = 12, b = 4) .r, g, b = t .print r, g, b I know it goes beyond the original intention of the proposal, but I figure I'd mention it anyway because the unorder of **kwargs has been bugging me alot. -- mvh Bj?rn From anthony at interlink.com.au Thu Mar 10 14:15:03 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Mar 10 14:15:25 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <000301c5253a$910ff480$163cc797@oemcomputer> References: <000301c5253a$910ff480$163cc797@oemcomputer> Message-ID: <200503110015.04590.anthony@interlink.com.au> On Thursday 10 March 2005 17:29, Raymond Hettinger wrote: > Or the implementation can have a switch to choose between keep-first > logic or replace logic. > > The latter seems a bit odd to me. The key position would be determined > by the first encountered while the value would be determined by the last > encountered. Starting with [(10, v1), (20, v2), (10.0, v3)], the > ordered dictionary's items would look like [(10, v3), (20, v2)]. Or, alternately, we keep the last occurence, and move it to the new position. There's a wide number of different use cases, each with a slightly different final result, and for this reason alone I'm -0 on it for the library. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From theller at python.net Thu Mar 10 14:30:24 2005 From: theller at python.net (Thomas Heller) Date: Thu Mar 10 14:30:30 2005 Subject: [Python-Dev] Re: Python 2.4, distutils, and pure python packages In-Reply-To: <1110456568.493624.48080@o13g2000cwo.googlegroups.com> (fuzzyman@gmail.com's message of "10 Mar 2005 04:09:28 -0800") References: <1110456568.493624.48080@o13g2000cwo.googlegroups.com> Message-ID: The following message is a courtesy copy of an article that has been posted to comp.lang.python as well. [CC to python-dev] "Fuzzyman" writes: > Python 2.4 is built with Microsoft Visiual C++ 7. This means that it > uses msvcr7.dll, which *isn't* a standard part of the windows operating > system. Nitpicking - it's MSVC 7.1, aka MS Visual Studio .NET 2003, and it's msvcr71.dll. > This means that if you build a windows installer using > distutils - it *requires* msvcr7.dll in order to run. This is true even > if your package is a pure python package. This means that when someone > tries to use a windows installer created with Python 2.4, on a machine > with only python 2.3 - it will fail. Bummer. > It's likely that nothing can be done about this (although for a pure > python package there's no reason not to use the 'source distribution' > and the setup.py). It does mean that I have to build my windows > installer on a machine with python 2.3. There's a workaround, although ugly. bdist_wininst has a --target-version flag which allows to build an installer for another Python version. It works both for pure Python packages, and for (how are they called? non-pure?) packages containing compiled extensions. The 2.4 distutils package has all that is needed to create a installer running with python 2.3 or maybe even 2.2 (which uses msvcrt.dll instead of msvcr71.dll). The result, of course, is that the installer can only install for the version that you specified at build time. Because distutils cannot cross-compile extensions for other versions, you must have build extensions (if there are any to include) with the other Python version before - distutils will simply pick up the extensions it finds in build subdirectories for the other version. Anyway, whether you have extensions or not, you must also specify the --skip-build command line flag, distutils will complain if you don't. So, for a pure distribution you would typically run these commands to build separate installers for 2.3 and 2.4: \python24\python setup.py --skip-build --target-version 2.3 bdist_wininst \python24\python setup.py --skip-build --target-version 2.4 bdist_wininst and for non-pure packages you must compile with each version before building the installer (if you want for some reason to use python 2.4 to build the installer for 2.3): \python24\python setup.py build_ext \python23\python setup.py build_ext \python24\python setup.py --skip-build --target-version 2.3 bdist_wininst \python24\python setup.py --skip-build --target-version 2.4 bdist_wininst OTOH, it's no problem to install both python 2.3 and python 2.4 in separate directories on the same machine and always make native builds. -- To make this story even more complete, there have been also other bugs caused by the different runtime dlls used in 2.3 and 2.4, most only showing when you use the --install-script option. The installer was using msvcrt.dll and msvcr71.dll at the same time, which led to crashes when the install script was started - the PythonCard installer suffered from that, at least. The bug only occurred with pure python distributions, because then the dll problem occurred. The bug is solved in Python 2.3.5, and also in the 2.4.1 release which will be out soon, with one exception: if the install-script prints something the output will be lost instead of displayed on the last screen. At least that's better than crashing the process. Thomas From python at rcn.com Thu Mar 10 15:03:35 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu Mar 10 15:08:47 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <740c3aec05031005072d3d34f8@mail.gmail.com> Message-ID: <001801c52579$f3f33500$163cc797@oemcomputer> [BJ?rn Lindqvist] > I would LOVE for **kwargs to be an ordered dict. It would allow me to > write code like this: > > .class MyTuple: > . def __init__(self, **kwargs): > . self.__dict__ = ordereddict(kwargs) This doesn't work. The kwargs are already turned into a regular dictionary before ordereddict sees it. Raymond Hettinger From anthony at python.org Thu Mar 10 15:37:45 2005 From: anthony at python.org (Anthony Baxter) Date: Thu Mar 10 15:38:21 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 Message-ID: <200503110138.02569.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.1 (release candidate 1). Python 2.4.1 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of the bugs squished in this release. Assuming no major problems crop up, a final release of Python 2.4.1 will follow in about a week's time. For more information on Python 2.4.1, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.4.1/ Highlights of this new release include: - Bug fixes. According to the release notes, several dozen bugs have been fixed, including a fix for the SimpleXMLRPCServer security issue (PSF-2005-001). Highlights of the previous major Python release (2.4) are available from the Python 2.4 page, at http://www.python.org/2.4/highlights.html Enjoy the new release, Anthony Anthony Baxter anthony@python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050311/5a0061f4/attachment-0001.pgp From p.f.moore at gmail.com Thu Mar 10 16:35:10 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Thu Mar 10 16:35:13 2005 Subject: [Python-Dev] itemgetter/attrgetter extension In-Reply-To: <42302D34.60100@iinet.net.au> References: <000e01c52430$4eff1fe0$7eb99d8d@oemcomputer> <42302D34.60100@iinet.net.au> Message-ID: <79990c6b050310073578d94bc0@mail.gmail.com> On Thu, 10 Mar 2005 21:19:16 +1000, Nick Coghlan wrote: > Raymond Hettinger wrote: > > Any objections to extending itemgetter() and attrgetter() to be able to > > extract multiple fields at a time? > > > > # SELECT name, rank, serialnum FROM soldierdata > > map(attrgetter('name', 'rank', 'serialnum'), soldierdata) > > > > # SELECT * FROM soldierdata ORDER BY unit, rank, proficiency > > sorted(soldierdata, key=attrgetter('unit', 'rank', 'proficiency')) > > Breaking the symmetry with getattr() bothers me a little, as does the signature > change that occurs when multiple arguments are given (returning a tuple, whereas > the single argument returns just the retrieved attribute). I don't think that the signature change is much of an issue in practice, as the arguments will be specified explicitly in all sensible use, and so the signature is going to be fixed for any particular use. > For itemgetter, I'd like to see multiple arguments eventually map to > multi-dimensional slices (to preserve symmetry with indexing syntax). Hmm, I can see the point. Would this impact on the performance of itemgetter? If so, maybe requiring use of an explicit lambda for multidimensional slices would be OK. > Call it -1 for itemgetter and -0 for attrgetter. I'm probably -0 on both. The idea seems neat, but what do you gain over map((lambda obj: obj.name, obj.rank, obj.serialnum), soldierdata) sorted(soldierdata, key=(lambda obj: obj.unit, obj.rank, obj.proficiency)) ? And what when you want to change obj.rank to getattr(obj, rank, 'Civilian')? Paul. From pje at telecommunity.com Thu Mar 10 17:00:39 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Mar 10 16:57:29 2005 Subject: [Python-Dev] SWT PyCon Sprint? In-Reply-To: Message-ID: <5.1.1.6.0.20050310105540.03f45410@mail.telecommunity.com> At 01:38 AM 3/10/05 -0500, Nicholas Bastin wrote: >I realize that this is exceedingly late in the game, but is anybody >interested in doing a Write-Python-Bindings-for-SWT sprint? It's been >brought up before in various places, and PyCon seems the likely place to >get enough concentrated knowledge to actually get it kicked off and >somewhat working... I'm certainly interested in the concept in general, though I'm curious whether the planned approach is a GCJ/SWIG wrapper, or a javaclass (bytecode translation)+ctypes dynamic approach. I'm somewhat more interested in the latter approach, as I find C++ a bit of a pain with respect to buildability. An additional complication is that SWT is a different package on each platform, so it's not so much "port SWT to Python" as "port SWT-windows to Python", "port SWT-Mac to Python", etc. (I assume that you're talking about porting to a JVM-less CPython extension, since if you were to leave it in Java you could just use Jython or one of the binary Python-Java bridges to access SWT as-is.) From tim.peters at gmail.com Thu Mar 10 17:31:27 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 10 17:31:30 2005 Subject: [Python-Dev] Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <200503110138.02569.anthony@python.org> References: <200503110138.02569.anthony@python.org> Message-ID: <1f7befae0503100831ed6939c@mail.gmail.com> I don't know how far I'll get with this. Using the current Zope-2_7-branch of the Zope module at cvs.zope.org:/cvs-repository, building Zope via python setup.py build_ext -i worked fine when I got up today, using the released Python 2.4. One of its tests fails, because of a Python bug that should be fixed in 2.4.1. So I wanted to test that. After uninstalling 2.4, then installing 2.4.1c1, then deleting Zope's lib\python\build directory, the attempt to build Zope works fine for quite a while (successfully compiles many chunks of Zope C code), but dies here: ... building 'ZODB.cPickleCache' extension C:\Program Files\Microsoft Visual Studio .NET 2003\Vc\bin\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -IC:\Code\Zope-2_7-branch\lib\Components\ExtensionClass\src -IC:\python24\include -IC:\python24\PC /TcZODB/cPickleCache.c /Fobuild\temp.win32-2.4 \Release\ZODB/cPickleCache.obj cPickleCache.c error: No such file or directory I don't know which file it's complaining about. In contrast, output I saved from building Zope with 2.4 this morning appears identical up to this point: ... building 'ZODB.cPickleCache' extension C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -IC:\Code\Zope-2_7-branch\lib\Components\ExtensionClass\src -IC:\python24\include -IC:\python24\PC /TcZODB/cPickleCache.c /Fobuild\temp.win32-2.4 \Release\ZODB/cPickleCache.obj cPickleCache.c but then, instead of an error, it goes on to link (and build more C stuff): C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\python24\libs /LIBPATH:C:\python24\PCBuild /EXPORT:initcPickleCache build\temp.win32-2.4\Release\ZODB/cPickleCache.obj /OUT:ZODB\cPickleCache.pyd /IMPLIB:build\temp.win32-2.4\Release\ZODB\cPickleCache.lib Creating library build\temp.win32-2.4\Release\ZODB\cPickleCache.lib and object build\temp.win32-2.4\Release\ZODB\cPickleCache.exp building 'ZODB.TimeStamp' extension .... Gets stranger: cPickleCache.c is really part of ZODB, and python setup.py build continues to work fine with 2.4.1c1 from a ZODB3 checkout (and using the same branch tag as was used for Zope). Anyone change anything here for 2.4.1 that rings a bell? Can anyone confirm that they can or can't build current Zope 2.7 on Windows with 2.4.1c1 too? If it's not unique to my box, is it unique to Windows (e.g., can anyone build current Zope on Linux with 2.4.1c1)? From anthony at interlink.com.au Thu Mar 10 18:00:11 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Mar 10 18:00:26 2005 Subject: [Python-Dev] Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <1f7befae0503100831ed6939c@mail.gmail.com> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> Message-ID: <200503110400.12648.anthony@interlink.com.au> It works on Linux, with Zope 2.7.4. Just as a note to others (I've mentioned this to Tim already) if you set an environment variable DISTUTILS_DEBUG before running a setup.py, you get very verbose information about what's going on, and, more importantly, full tracebacks rather than terse error messages. error: No such file or directory looks like a distutils error message. -- Anthony Baxter It's never too late to have a happy childhood. From tim.peters at gmail.com Thu Mar 10 18:07:50 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 10 18:08:24 2005 Subject: [Python-Dev] Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <200503110400.12648.anthony@interlink.com.au> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> Message-ID: <1f7befae0503100907dcc3b41@mail.gmail.com> [Anthony Baxter] > It works on Linux, with Zope 2.7.4. Thanks! > Just as a note to others (I've mentioned this to Tim already) if you set an > environment variable DISTUTILS_DEBUG before running a setup.py, you get > very verbose information about what's going on, and, more importantly, full > tracebacks rather than terse error messages. > > error: No such file or directory > > looks like a distutils error message. This helped a lot, although I'm even more confused now: building 'ZODB.cPickleCache' extension C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -IC:\Code\Zope-2_7-branch\lib\Components\ExtensionClass\src -IC:\python24\include -IC:\python24\PC /TcZODB/cPickleCache.c /Fobuild\temp.win32-2.4\Release\ZODB/cPickleCache.obj cPickleCache.c error: No such file or directory Traceback (most recent call last): File "C:\Code\Zope-2_7-branch\setup.py", line 1094, in ? distclass=ZopeDistribution, File "C:\python24\lib\distutils\core.py", line 149, in setup dist.run_commands() File "C:\python24\lib\distutils\dist.py", line 946, in run_commands self.run_command(cmd) File "C:\python24\lib\distutils\dist.py", line 966, in run_command cmd_obj.run() File "C:\python24\lib\distutils\command\build_ext.py", line 279, in run self.build_extensions() File "C:\python24\lib\distutils\command\build_ext.py", line 405, in build_extensions self.build_extension(ext) File "C:\python24\lib\distutils\command\build_ext.py", line 502, in build_extension target_lang=language) File "C:\python24\lib\distutils\ccompiler.py", line 847, in link_shared_object extra_preargs, extra_postargs, build_temp, target_lang) File "C:\python24\lib\distutils\msvccompiler.py", line 422, in link if not self.initialized: self.initialize() File "C:\python24\lib\distutils\msvccompiler.py", line 239, in initialize os.environ['path'] = string.join(self.__paths, ';') File "C:\python24\lib\os.py", line 419, in __setitem__ putenv(key, item) OSError: [Errno 2] No such file or directory LOL -- or something . Before going to sleep, Anthony suggested that Bug #1110478: Revert os.environ.update to do putenv again. might be relevant. HTF can we get a "no such file or directly" out of a putenv()?! Don't be shy. From tim.peters at gmail.com Thu Mar 10 18:07:50 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 10 18:10:17 2005 Subject: [Python-Dev] Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <200503110400.12648.anthony@interlink.com.au> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> Message-ID: <1f7befae0503100907dcc3b41@mail.gmail.com> [Anthony Baxter] > It works on Linux, with Zope 2.7.4. Thanks! > Just as a note to others (I've mentioned this to Tim already) if you set an > environment variable DISTUTILS_DEBUG before running a setup.py, you get > very verbose information about what's going on, and, more importantly, full > tracebacks rather than terse error messages. > > error: No such file or directory > > looks like a distutils error message. This helped a lot, although I'm even more confused now: building 'ZODB.cPickleCache' extension C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -IC:\Code\Zope-2_7-branch\lib\Components\ExtensionClass\src -IC:\python24\include -IC:\python24\PC /TcZODB/cPickleCache.c /Fobuild\temp.win32-2.4\Release\ZODB/cPickleCache.obj cPickleCache.c error: No such file or directory Traceback (most recent call last): File "C:\Code\Zope-2_7-branch\setup.py", line 1094, in ? distclass=ZopeDistribution, File "C:\python24\lib\distutils\core.py", line 149, in setup dist.run_commands() File "C:\python24\lib\distutils\dist.py", line 946, in run_commands self.run_command(cmd) File "C:\python24\lib\distutils\dist.py", line 966, in run_command cmd_obj.run() File "C:\python24\lib\distutils\command\build_ext.py", line 279, in run self.build_extensions() File "C:\python24\lib\distutils\command\build_ext.py", line 405, in build_extensions self.build_extension(ext) File "C:\python24\lib\distutils\command\build_ext.py", line 502, in build_extension target_lang=language) File "C:\python24\lib\distutils\ccompiler.py", line 847, in link_shared_object extra_preargs, extra_postargs, build_temp, target_lang) File "C:\python24\lib\distutils\msvccompiler.py", line 422, in link if not self.initialized: self.initialize() File "C:\python24\lib\distutils\msvccompiler.py", line 239, in initialize os.environ['path'] = string.join(self.__paths, ';') File "C:\python24\lib\os.py", line 419, in __setitem__ putenv(key, item) OSError: [Errno 2] No such file or directory LOL -- or something . Before going to sleep, Anthony suggested that Bug #1110478: Revert os.environ.update to do putenv again. might be relevant. HTF can we get a "no such file or directly" out of a putenv()?! Don't be shy. From tim.peters at gmail.com Thu Mar 10 18:46:23 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 10 18:46:25 2005 Subject: [Python-Dev] Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <1f7befae0503100907dcc3b41@mail.gmail.com> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> <1f7befae0503100907dcc3b41@mail.gmail.com> Message-ID: <1f7befae05031009467a3d2852@mail.gmail.com> This is going to need someone who understands distutils internals. The strings we end up passing to putenv() grow absurdly large, and sooner or later Windows gets very unhappy with them. os.py has a elif name in ('os2', 'nt'): # Where Env Var Names Must Be UPPERCASE class controlling introduction of a _Environ class. I changed its __setitem__ like so: def __setitem__(self, key, item): if key.upper() == "PATH": # new line print "len(item)", len(item) # new line putenv(key, item) self.data[key.upper()] = item As "setup.py build_ext -i" goes on while compiling Zope, this is the output before putenv barfs: len(item) 1025 len(item) 1680 len(item) 2335 len(item) 2990 len(item) 3645 len(item) 4300 len(item) 4955 len(item) 5610 len(item) 6265 len(item) 6920 len(item) 7575 len(item) 8230 len(item) 8885 len(item) 9540 len(item) 10195 len(item) 10850 len(item) 11505 len(item) 12160 len(item) 12815 len(item) 13470 len(item) 14125 len(item) 14780 len(item) 15435 len(item) 16090 len(item) 16745 len(item) 17400 len(item) 18055 len(item) 18710 len(item) 19365 len(item) 20020 len(item) 20675 len(item) 21330 len(item) 21985 len(item) 22640 len(item) 23295 len(item) 23950 len(item) 24605 len(item) 25260 len(item) 25915 len(item) 26570 len(item) 27225 len(item) 27880 len(item) 28535 len(item) 29190 len(item) 29845 len(item) 30500 len(item) 31155 len(item) 31810 len(item) 32465 The PATH isn't gibberish at this point -- it just keeps adding the MSVC directories (like C:\\Program Files\\Microsoft Visual Studio .NET 2003\\Vc7\\bin ) again and again and again. I don't know what the environ limits are on various flavors of Windows; empirically, on my Win XP Pro SP2 box, the values have to be < 32K or putenv() dies. But there's surely no need for distutils to make PATH grow without bound, so I think this is a distutils bug. A workaround for building Zope is easy but embarrassing : kill setup.py before it hits this error, then start it again. Lather, rinse, repeat. After a few iterations, everything builds fine. From amk at amk.ca Thu Mar 10 19:06:22 2005 From: amk at amk.ca (A.M. Kuchling) Date: Thu Mar 10 19:06:32 2005 Subject: [Python-Dev] Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <1f7befae05031009467a3d2852@mail.gmail.com> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> <1f7befae0503100907dcc3b41@mail.gmail.com> <1f7befae05031009467a3d2852@mail.gmail.com> Message-ID: <20050310180622.GA14309@rogue.amk.ca> On Thu, Mar 10, 2005 at 12:46:23PM -0500, Tim Peters wrote: > This is going to need someone who understands distutils internals. > The strings we end up passing to putenv() grow absurdly large, and > sooner or later Windows gets very unhappy with them. In distutils.msvccompiler: def __init__ (self, verbose=0, dry_run=0, force=0): ... self.initialized = False def compile(self, sources, output_dir=None, macros=None, include_dirs=None, debug=0, extra_preargs=None, extra_postargs=None, depends=None): if not self.initialized: self.initialize() ... def initialize(self): ... does not seem to set self.initialized to True! I think the fix is to add 'self.initialized = True' to the initialize() method, but can't test it (no Windows). This fix should also go into 2.4.1-final, I expect. --amk From tim.peters at gmail.com Thu Mar 10 19:12:34 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 10 19:12:38 2005 Subject: [Python-Dev] Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <20050310180622.GA14309@rogue.amk.ca> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> <1f7befae0503100907dcc3b41@mail.gmail.com> <1f7befae05031009467a3d2852@mail.gmail.com> <20050310180622.GA14309@rogue.amk.ca> Message-ID: <1f7befae05031010126037451f@mail.gmail.com> [ A.M. Kuchling] > In distutils.msvccompiler: > > def __init__ (self, verbose=0, dry_run=0, force=0): > ... > self.initialized = False > > def compile(self, sources, > output_dir=None, macros=None, include_dirs=None, debug=0, > extra_preargs=None, extra_postargs=None, depends=None): > > if not self.initialized: self.initialize() > ... > > def initialize(self): > ... does not seem to set self.initialized to True! > > I think the fix is to add 'self.initialized = True' to the > initialize() method, but can't test it (no Windows). This fix should > also go into 2.4.1-final, I expect. Dunno, but sounds good. We certainly ought to "fix it" for 2.4.1 final, whatever the fix is. I opened a bug report: http://www.python.org/sf/1160802 From jcarlson at uci.edu Thu Mar 10 20:18:24 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Mar 10 20:20:30 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <001801c52579$f3f33500$163cc797@oemcomputer> References: <740c3aec05031005072d3d34f8@mail.gmail.com> <001801c52579$f3f33500$163cc797@oemcomputer> Message-ID: <20050310111208.A440.JCARLSON@uci.edu> "Raymond Hettinger" wrote: > > [BJ?rn Lindqvist] > > I would LOVE for **kwargs to be an ordered dict. It would allow me to > > write code like this: > > > > .class MyTuple: > > . def __init__(self, **kwargs): > > . self.__dict__ = ordereddict(kwargs) > > This doesn't work. The kwargs are already turned into a regular > dictionary before ordereddict sees it. From what I understand, he was saying that it would be nice if kwargs were an ordered dict /instead of/ a standard dict. Whether or not he realizes it will not happen due to the 2x memory overhead, 2x speed hit, etc., every time kwargs are used, is another matter. Alternatively, BJorn could use a list of tuples and *args to preserve order, but that is an off-list discussion for another day. - Josiah From tseaver at zope.com Thu Mar 10 21:02:31 2005 From: tseaver at zope.com (Tres Seaver) Date: Thu Mar 10 21:21:50 2005 Subject: [Python-Dev] Re: Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <200503110400.12648.anthony@interlink.com.au> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> Message-ID: <4230A7D7.6050801@zope.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Anthony Baxter wrote: | It works on Linux, with Zope 2.7.4. Just as a note to others (I've mentioned | this to Tim already) if you set an environment variable DISTUTILS_DEBUG | before running a setup.py, you get very verbose information about what's going | on, and, more importantly, full tracebacks rather than terse error messages. | | error: No such file or directory | | looks like a distutils error message. Unit tests for Zope 2.7.4's 'zdaemon' package, which passed under Python 2.4, now fail under 2.4.1c1: $ uname -a Linux secretariat 2.6.8.1-5-686 #1 Sat Feb 12 00:50:37 UTC 2005 i686 \ GNU/Linux $ ../Python-2.4/python test.py -v zdaemon Running unit tests at level 1 Running unit tests from /home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python /home/tseaver/projects/Zope-CVS/Python-2.4/Lib/whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module ~ DeprecationWarning) ............................ - ---------------------------------------------------------------------- Ran 28 tests in 2.278s OK $ ../Python-2.4.1c1/python test.py -v zdaemon Running unit tests at level 1 Running unit tests from /home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python /home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module ~ DeprecationWarning) ...................Traceback (most recent call last): ~ File "test.py", line 918, in ? ~ process_args() ~ File "test.py", line 908, in process_args ~ bad = main(module_filter, test_filter, libdir) ~ File "test.py", line 698, in main ~ runner(files, test_filter, debug) ~ File "test.py", line 599, in runner ~ r = runner.run(suite) ~ File "test.py", line 366, in run ~ return unittest.TextTestRunner.run(self, test) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 696, in run ~ test(result) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 428, in __call__ ~ return self.run(*args, **kwds) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 424, in run ~ test(result) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 428, in __call__ ~ return self.run(*args, **kwds) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 424, in run ~ test(result) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 428, in __call__ ~ return self.run(*args, **kwds) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 424, in run ~ test(result) ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", line 281, in __call__ ~ return self.run(*args, **kwds) ~ File "/home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python/zdaemon/tests/testzdrun.py", line 97, in run ~ zdctl.main(["-s", self.zdsock] + args) AttributeError: 'ZDaemonTests' object has no attribute 'zdsock' By staring at the code of the failing test, it looks like the MRO of the testcase class has changed: it declares a 'run' method, which is supposed to run the external process, which clashes with the 'run' method of unittest.TestCase. I don't know what change in the 2.4 -> 2.4.1c1 update would have mucked with the MRO (if a MRO issue is involved). Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCMKfXGqWXf00rNCgRAn8sAJ40b9THGi81tRJItjl1kTo+7kV86QCffBKu +7CnIOgjvNQ3jlFl4PRDJ9c= =oyGX -----END PGP SIGNATURE----- From tim.peters at gmail.com Thu Mar 10 22:05:06 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 10 22:05:11 2005 Subject: [Python-Dev] Re: Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <4230A7D7.6050801@zope.com> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> <4230A7D7.6050801@zope.com> Message-ID: <1f7befae05031013053994c080@mail.gmail.com> [Tres Seaver] > Unit tests for Zope 2.7.4's 'zdaemon' package, which passed under Python > 2.4, now fail under 2.4.1c1: Are you sure they passed under 2.4? Derrick Hudson changed run() to _run() in the SVN version of zdaemon way back on Jan 19, with this checkin comment: Log message for revision 28881: Renamed run() method to _run(). Python 2.4's unittest.TestCase class defines all of the test logic in the run() method instead of in __call__() (as 2.3 did). The rename prevents overriding the base implementation and causing the tests to fail. Changed: U Zope3/trunk/src/zdaemon/tests/testzdrun.py I then ported that to where it belonged (zdaemon isn't part of Zope3 under SVN, it's stitched in to Zope3, from time to time, from the distinct zdaemon SVN trunk). I suppose that never got ported to the CVS version -- well, until today, cuz it looks like Stefan Holek checked in the same kinds of changes to testzdrun.py about 2.5 hours ago. [BTW, the zdaemon tests don't run at all on Windows, so I'll never notice anything wrong with them] > $ uname -a > Linux secretariat 2.6.8.1-5-686 #1 Sat Feb 12 00:50:37 UTC 2005 i686 \ > GNU/Linux > $ ../Python-2.4/python test.py -v zdaemon > Running unit tests at level 1 > Running unit tests from > /home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python > /home/tseaver/projects/Zope-CVS/Python-2.4/Lib/whrandom.py:38: > DeprecationWarning: the whrandom module is deprecated; please use the > random module > ~ DeprecationWarning) > ............................ > - ---------------------------------------------------------------------- > Ran 28 tests in 2.278s > > OK > $ ../Python-2.4.1c1/python test.py -v zdaemon > Running unit tests at level 1 > Running unit tests from > /home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python > /home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/whrandom.py:38: > DeprecationWarning: the whrandom module is deprecated; please use the > random module > ~ DeprecationWarning) > ...................Traceback (most recent call last): > ~ File "test.py", line 918, in ? > ~ process_args() > ~ File "test.py", line 908, in process_args > ~ bad = main(module_filter, test_filter, libdir) > ~ File "test.py", line 698, in main > ~ runner(files, test_filter, debug) > ~ File "test.py", line 599, in runner > ~ r = runner.run(suite) > ~ File "test.py", line 366, in run > ~ return unittest.TextTestRunner.run(self, test) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 696, in run > ~ test(result) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 428, in __call__ > ~ return self.run(*args, **kwds) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 424, in run > ~ test(result) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 428, in __call__ > ~ return self.run(*args, **kwds) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 424, in run > ~ test(result) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 428, in __call__ > ~ return self.run(*args, **kwds) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 424, in run > ~ test(result) > ~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", > line 281, in __call__ > ~ return self.run(*args, **kwds) > ~ File > "/home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python/zdaemon/tests/testzdrun.py", > line 97, in run > ~ zdctl.main(["-s", self.zdsock] + args) > AttributeError: 'ZDaemonTests' object has no attribute 'zdsock' > > By staring at the code of the failing test, it looks like the MRO of the > testcase class has changed: it declares a 'run' method, which is > supposed to run the external process, which clashes with the 'run' > method of unittest.TestCase. I don't know what change in the 2.4 -> > 2.4.1c1 update would have mucked with the MRO (if a MRO issue is involved). > > Tres. Pretty baffling. I assume that if you did "cvs up", the test would pass now (because of Stefan's recent checkin). Sounds anyway like an unintended unittest incompatibility snuck in somewhere between 2.3 and 2.4. From nbastin at opnet.com Thu Mar 10 22:06:54 2005 From: nbastin at opnet.com (Nicholas Bastin) Date: Thu Mar 10 22:07:11 2005 Subject: [Python-Dev] SWT PyCon Sprint? In-Reply-To: <5.1.1.6.0.20050310105540.03f45410@mail.telecommunity.com> References: <5.1.1.6.0.20050310105540.03f45410@mail.telecommunity.com> Message-ID: <2d3e2b4c9fdc0533901b55d8c676b017@opnet.com> On Mar 10, 2005, at 11:00 AM, Phillip J. Eby wrote: > At 01:38 AM 3/10/05 -0500, Nicholas Bastin wrote: >> I realize that this is exceedingly late in the game, but is anybody >> interested in doing a Write-Python-Bindings-for-SWT sprint? It's >> been brought up before in various places, and PyCon seems the likely >> place to get enough concentrated knowledge to actually get it kicked >> off and somewhat working... > > I'm certainly interested in the concept in general, though I'm curious > whether the planned approach is a GCJ/SWIG wrapper, or a javaclass > (bytecode translation)+ctypes dynamic approach. I'm somewhat more > interested in the latter approach, as I find C++ a bit of a pain with > respect to buildability. I'm open to either approach. I don't know a lot about JNI, so I was hoping somebody would come along for the ride who could answer certain questions about how SWT is implemented. A third option would be to grovel over SWT and implement an identical functionality in Python (pure-python plus ctypes), and make a mirror implementation, rather than a wrapper. What approach we take depends largely on who shows up and what they feel comfortable with. > An additional complication is that SWT is a different package on each > platform, so it's not so much "port SWT to Python" as "port > SWT-windows to Python", "port SWT-Mac to Python", etc. The API is identical for each platform, however, so depending on the level at which you wrapped it, this is only a build problem. > (I assume that you're talking about porting to a JVM-less CPython > extension, since if you were to leave it in Java you could just use > Jython or one of the binary Python-Java bridges to access SWT as-is.) Yes, JVM-less CPython extension would be the plan. -- Nick From tseaver at zope.com Thu Mar 10 22:09:56 2005 From: tseaver at zope.com (Tres Seaver) Date: Thu Mar 10 22:12:05 2005 Subject: [Python-Dev] Re: Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <1f7befae05031013053994c080@mail.gmail.com> References: <200503110138.02569.anthony@python.org> <1f7befae0503100831ed6939c@mail.gmail.com> <200503110400.12648.anthony@interlink.com.au> <4230A7D7.6050801@zope.com> <1f7befae05031013053994c080@mail.gmail.com> Message-ID: <4230B7A4.2080700@zope.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tim Peters wrote: | [Tres Seaver] | |>Unit tests for Zope 2.7.4's 'zdaemon' package, which passed under Python |>2.4, now fail under 2.4.1c1: | | | Are you sure they passed under 2.4? Yep. I showed output from that in the original post (and below). | Derrick Hudson changed run() to | _run() in the SVN version of zdaemon way back on Jan 19, with this | checkin comment: | | Log message for revision 28881: | Renamed run() method to _run(). Python 2.4's unittest.TestCase class | defines all of the test logic in the run() method instead of in __call__() | (as 2.3 did). The rename prevents overriding the base implementation and | causing the tests to fail. | | Changed: | U Zope3/trunk/src/zdaemon/tests/testzdrun.py | | I then ported that to where it belonged (zdaemon isn't part of Zope3 | under SVN, it's stitched in to Zope3, from time to time, from the | distinct zdaemon SVN trunk). | | I suppose that never got ported to the CVS version -- well, until | today, cuz it looks like Stefan Holek checked in the same kinds of | changes to testzdrun.py about 2.5 hours ago. Right. In fact, he beat me to the commit, and gave me a nice CVS conflict as lagniappe. | [BTW, the zdaemon tests don't run at all on Windows, so I'll never | notice anything wrong with them] | | |>$ uname -a |>Linux secretariat 2.6.8.1-5-686 #1 Sat Feb 12 00:50:37 UTC 2005 i686 \ |>GNU/Linux |>$ ../Python-2.4/python test.py -v zdaemon |>Running unit tests at level 1 |>Running unit tests from |>/home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python |>/home/tseaver/projects/Zope-CVS/Python-2.4/Lib/whrandom.py:38: |>DeprecationWarning: the whrandom module is deprecated; please use the |>random module |>~ DeprecationWarning) |>............................ |>- ---------------------------------------------------------------------- |>Ran 28 tests in 2.278s |> |>OK |>$ ../Python-2.4.1c1/python test.py -v zdaemon |>Running unit tests at level 1 |>Running unit tests from |>/home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python |>/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/whrandom.py:38: |>DeprecationWarning: the whrandom module is deprecated; please use the |>random module |>~ DeprecationWarning) |>...................Traceback (most recent call last): |>~ File "test.py", line 918, in ? |>~ process_args() |>~ File "test.py", line 908, in process_args |>~ bad = main(module_filter, test_filter, libdir) |>~ File "test.py", line 698, in main |>~ runner(files, test_filter, debug) |>~ File "test.py", line 599, in runner |>~ r = runner.run(suite) |>~ File "test.py", line 366, in run |>~ return unittest.TextTestRunner.run(self, test) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 696, in run |>~ test(result) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 428, in __call__ |>~ return self.run(*args, **kwds) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 424, in run |>~ test(result) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 428, in __call__ |>~ return self.run(*args, **kwds) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 424, in run |>~ test(result) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 428, in __call__ |>~ return self.run(*args, **kwds) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 424, in run |>~ test(result) |>~ File "/home/tseaver/projects/Zope-CVS/Python-2.4.1c1/Lib/unittest.py", |>line 281, in __call__ |>~ return self.run(*args, **kwds) |>~ File |>"/home/tseaver/projects/Zope-CVS/Zope-2.7.4/lib/python/zdaemon/tests/testzdrun.py", |>line 97, in run |>~ zdctl.main(["-s", self.zdsock] + args) |>AttributeError: 'ZDaemonTests' object has no attribute 'zdsock' |> |>By staring at the code of the failing test, it looks like the MRO of the |>testcase class has changed: it declares a 'run' method, which is |>supposed to run the external process, which clashes with the 'run' |>method of unittest.TestCase. I don't know what change in the 2.4 -> |>2.4.1c1 update would have mucked with the MRO (if a MRO issue is involved). |> |>Tres. | | | Pretty baffling. I assume that if you did "cvs up", the test would | pass now (because of Stefan's recent checkin). | | Sounds anyway like an unintended unittest incompatibility snuck in | somewhere between 2.3 and 2.4. I think somebody tweaked either the base classes of unittest.TestCase or else the MRO algoritm (if it *is* algorithmic, that is ;) Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCMLekGqWXf00rNCgRAofeAJ9nEbAMmklXhH3BpRzihinTVFuiiQCfZA2q EwBLgXghI8WLJVmwoRMQxxA= =nAjP -----END PGP SIGNATURE----- From pje at telecommunity.com Thu Mar 10 22:40:42 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Mar 10 22:37:34 2005 Subject: [Python-Dev] SWT PyCon Sprint? In-Reply-To: <2d3e2b4c9fdc0533901b55d8c676b017@opnet.com> References: <5.1.1.6.0.20050310105540.03f45410@mail.telecommunity.com> <5.1.1.6.0.20050310105540.03f45410@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050310161223.03005920@mail.telecommunity.com> At 04:06 PM 3/10/05 -0500, Nicholas Bastin wrote: >On Mar 10, 2005, at 11:00 AM, Phillip J. Eby wrote: > >>At 01:38 AM 3/10/05 -0500, Nicholas Bastin wrote: >>>I realize that this is exceedingly late in the game, but is anybody >>>interested in doing a Write-Python-Bindings-for-SWT sprint? It's been >>>brought up before in various places, and PyCon seems the likely place to >>>get enough concentrated knowledge to actually get it kicked off and >>>somewhat working... >> >>I'm certainly interested in the concept in general, though I'm curious >>whether the planned approach is a GCJ/SWIG wrapper, or a javaclass >>(bytecode translation)+ctypes dynamic approach. I'm somewhat more >>interested in the latter approach, as I find C++ a bit of a pain with >>respect to buildability. > >I'm open to either approach. I don't know a lot about JNI, so I was >hoping somebody would come along for the ride who could answer certain >questions about how SWT is implemented. By design, SWT is pure Java with an as-thin-as-possible JNI wrapping of the native platform GUI. It's akin to writing a pure Python GUI for Windows using ctypes or simple Pyrex wrappers to directly call Windows C APIs. The important thing to be aware of here is that this means that you can't just take the SWT .dll's or .so's and put a Python binding on them, because all you'd end up with is a wrapper to the Windows or Mac or GTK API. I'll repeat this, because it's a common source of confusion about SWT: the SWT implementation is *all* Java. The native dynamic libraries for SWT are just the glue to be able to call platform API's, they do not contain *any* high-level code. This is both good and bad. It's good in that you don't have to worry about the JNI issue if you want to port SWT to another platform, you just have to have a way to call the platform's APIs. It's bad in that you can't just put a new language binding on the native libraries and have an SWT port. > A third option would be to grovel over SWT and implement an identical > functionality in Python (pure-python plus ctypes), and make a mirror > implementation, rather than a wrapper. If you're going to do that, you might as well use javaclass to do the grunt work of the translation, and simply replace the native library with a ctypes-based module. >>An additional complication is that SWT is a different package on each >>platform, so it's not so much "port SWT to Python" as "port SWT-windows >>to Python", "port SWT-Mac to Python", etc. > >The API is identical for each platform, however, so depending on the level >at which you wrapped it, this is only a build problem. I'm just pointing out that the source code for each platform is different. The native library for each platform is different. The native libraries also expose different APIs on each platform, because they just expose that platform's native API. So sure, there's probably some code that's similar or identical, but the whole point of SWT is that each one is a custom implementation for its platform, as distinct from approaches like AWT, Swing, wxWidgets, et al. >>(I assume that you're talking about porting to a JVM-less CPython >>extension, since if you were to leave it in Java you could just use >>Jython or one of the binary Python-Java bridges to access SWT as-is.) > >Yes, JVM-less CPython extension would be the plan. Then here are the possible architectures, as I understand them: Layer 1: Native Platform Interface Option 1: Use the JNI libraries supplied with SWT (forces GCJ at layer 2) Option 2: Create a Python wrapper for the corresponding APIs * Possibly semi-autogenerated from JNI metadata * ctypes, Pyrex, or C Layer 2: SWT Java Implementations Option 1: Compile with GCJ (requires JNI approach at layer 1) Option 2: Translate Java bytecode to Python bytecode w/javaclass Option 3: Translate Java bytecode to Pyrex or C (by modifying javaclass) Option 4: Translate Java source to Python or Pyrex by hand (for *each* platform!) Layer 3: Python wrapper (only needed for GCJ scenario) Option 1: SWIG (needs a C++/SWIG guru) Option 2: Boost::Python Option 3: Plain C++ So as you can see, the choices are basically GCJ versus everything else, but the GCJ approach forces C++ into the mix and requires good knowledge of either C++ or SWIG, maybe both. But it only needs to be done once, and should then be buildable on all platforms. OTOH, if I had the necessary skills, I'd have probably already taken on the effort. It would be cool if somebody created a SWIG wrapper generator that could pull the necessary info from Java class files to make a GCJ wrapper. But as far as I know, everybody who's wrapped Java libraries for Python using GCJ wrote their wrappers by hand. Anyway, this is now getting *way* off topic for Python-Dev, so if there aren't any other interested parties we should take this off-list. From aahz at pythoncraft.com Fri Mar 11 01:23:41 2005 From: aahz at pythoncraft.com (Aahz) Date: Fri Mar 11 01:23:43 2005 Subject: [Python-Dev] FWD: SD MAgazine.com - Jolt Awards Winners Message-ID: <20050311002341.GA25716@panix.com> Guido may not be able to go. Anyone else already going? ----- Forwarded message from RLum@cmp.com ----- > Subject: Request - SD MAgazine.com - Jolt Awards Winners > To: webmaster@python.org > From: RLum@cmp.com > Date: Thu, 10 Mar 2005 16:02:35 -0800 > > HI Python.org, > > You may or may not be aware that Python is a finalist in the 15th > Annual Jolt Awards in the Languages and Development Category.The > awards ceremony will take place at SD West, in the Santa Clara > Convention Center, Auditorium, on March 16th from 6:30-7:30 p.m., when > the winners will be announced. Is there anyway that a representative > of Python.org be present to accept the award if they should win? I > will have badges for these individuals at the registration desk to > attend the awards ceremony. > > I understand the Guido van Rossum will be attending SD West. Is there > anyway that he can be present? > > All finalists and conference alumni are invited to a poolside party > with food, drinks and entertainment after the ceremony at the > Westin, which is adjacent to the Conference Center. Hope to see you > there. Dress should be business casual. > > Looking forward to hearing from you. > > Rosalyn > (415) 947-6182 > > > ===================================================== > Rosalyn Lum > Technical Editor > Software Development Magazine > CMP Media > 600 Harrison St., 6th Floor > San Francisco, CA 94107 > www.sdmagazine.com ----- End forwarded message ----- -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From david.ascher at gmail.com Fri Mar 11 01:58:32 2005 From: david.ascher at gmail.com (David Ascher) Date: Fri Mar 11 01:58:38 2005 Subject: [Python-Dev] FWD: SD MAgazine.com - Jolt Awards Winners In-Reply-To: <20050311002341.GA25716@panix.com> References: <20050311002341.GA25716@panix.com> Message-ID: On Thu, 10 Mar 2005 19:23:41 -0500, Aahz wrote: > Guido may not be able to go. Anyone else already going? I may, but only on the 18th, not the 16th. So that doesn't really work =). > > ----- Forwarded message from RLum@cmp.com ----- > > > Subject: Request - SD MAgazine.com - Jolt Awards Winners > > To: webmaster@python.org > > From: RLum@cmp.com > > Date: Thu, 10 Mar 2005 16:02:35 -0800 > > > > HI Python.org, > > > > You may or may not be aware that Python is a finalist in the 15th > > Annual Jolt Awards in the Languages and Development Category.The > > awards ceremony will take place at SD West, in the Santa Clara > > Convention Center, Auditorium, on March 16th from 6:30-7:30 p.m., when > > the winners will be announced. Is there anyway that a representative > > of Python.org be present to accept the award if they should win? I > > will have badges for these individuals at the registration desk to > > attend the awards ceremony. > > > > I understand the Guido van Rossum will be attending SD West. Is there > > anyway that he can be present? > > > > All finalists and conference alumni are invited to a poolside party > > with food, drinks and entertainment after the ceremony at the > > Westin, which is adjacent to the Conference Center. Hope to see you > > there. Dress should be business casual. > > > > Looking forward to hearing from you. > > > > Rosalyn > > (415) 947-6182 > > > > > > ===================================================== > > Rosalyn Lum > > Technical Editor > > Software Development Magazine > > CMP Media > > 600 Harrison St., 6th Floor > > San Francisco, CA 94107 > > www.sdmagazine.com > > ----- End forwarded message ----- > > -- > Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ > > "The joy of coding Python should be in seeing short, concise, readable > classes that express a lot of action in a small amount of clear code -- > not in reams of trivial code that bores the reader to death." --GvR > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/david.ascher%40gmail.com > From gvanrossum at gmail.com Fri Mar 11 02:33:28 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri Mar 11 02:33:32 2005 Subject: [Python-Dev] Adding any() and all() Message-ID: See my blog: http://www.artima.com/forums/flat.jsp?forum=106&thread=98196 Do we even need a PEP or is there a volunteer who'll add any() and all() for me? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Fri Mar 11 02:55:49 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 11 02:55:52 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <200503110138.02569.anthony@python.org> References: <200503110138.02569.anthony@python.org> Message-ID: <4230FAA5.4070309@v.loewis.de> Anthony Baxter wrote: > On behalf of the Python development team and the Python community, I'm > happy to announce the release of Python 2.4.1 (release candidate 1). > > Python 2.4.1 is a bug-fix release. See the release notes at the website > (also available as Misc/NEWS in the source distribution) for details of > the bugs squished in this release. I'd like to encourage feedback on whether the Windows installer works for people. It replaces the VBScript part in the MSI package with native code, which ought to drop the dependency on VBScript, but might introduce new incompatibilities. Regards, Martin From janssen at parc.com Fri Mar 11 03:38:27 2005 From: janssen at parc.com (Bill Janssen) Date: Fri Mar 11 03:38:53 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: Your message of "Thu, 10 Mar 2005 17:33:28 PST." Message-ID: <05Mar10.183828pst."58617"@synergy1.parc.xerox.com> Guido, I think there should be a PEP. For instance, I think I'd want them to be: def any(S): for x in S: if x: return x return S[-1] def all(S): for x in S: if not x: return x return S[-1] Or perhaps these should be called "first" and "last". Bill From python at rcn.com Fri Mar 11 03:49:21 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri Mar 11 03:53:45 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: Message-ID: <000301c525e4$ef687a20$13b12c81@oemcomputer> > See my blog: http://www.artima.com/forums/flat.jsp?forum=106&thread=98196 > > Do we even need a PEP or is there a volunteer who'll add any() and all() > for me? I'll volunteer for this one. Will leave it open for discussion for a bit so that folks can voice any thoughts on the design. Raymond Hettinger From pje at telecommunity.com Fri Mar 11 04:05:02 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Mar 11 04:01:53 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <05Mar10.183828pst."58617"@synergy1.parc.xerox.com> References: Message-ID: <5.1.1.6.0.20050310215726.02435ac0@mail.telecommunity.com> At 06:38 PM 3/10/05 -0800, Bill Janssen wrote: >Guido, > >I think there should be a PEP. For instance, I think I'd want them to be: > >def any(S): > for x in S: > if x: > return x > return S[-1] > >def all(S): > for x in S: > if not x: > return x > return S[-1] > >Or perhaps these should be called "first" and "last". More precisely (since "S" might be an iterator, or empty): def any(S): x = False for x in S: if x: break return x def all(S): x = True for x in S: if not x: break return x However, "first" and "last" don't make sense to me as names. First what? Last what? Actually, "any" and "all" don't make that much sense to me either. I find myself wanting to call them "or" and "and", or maybe "or_iter" and "and_iter". Or perhaps "until_false" or "until_true". Nah, the and/or names make more sense, as they exactly match the behavior of the corresponding logical operators, if you could call them as a function. At least, if they took *args anyway. From python at rcn.com Fri Mar 11 04:22:45 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri Mar 11 04:27:35 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <05Mar10.183828pst."58617"@synergy1.parc.xerox.com> Message-ID: <000701c525e9$a88dcc40$bf20c797@oemcomputer> [Bill Janssen] > I think I'd want them to be: > > def any(S): > for x in S: > if x: > return x > return S[-1] > > def all(S): > for x in S: > if not x: > return x > return S[-1] > > Or perhaps these should be called "first" and "last". -1 Over time, I've gotten feedback about these and other itertools recipes. No one has objected to the True/False return values in those recipes or in Guido's version. Guido's version matches the normal expectation of any/all being a predicate. Also, it avoids the kind of errors/confusion that people currently experience with Python's unique implementation of "and" and "or". Returning the last element is not evil; it's just weird, unexpected, and non-obvious. Resist the urge to get tricky with this one. Raymond Hettinger From anthony at interlink.com.au Fri Mar 11 04:31:37 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Mar 11 04:32:08 2005 Subject: [Python-Dev] Re: Can't build Zope on Windows w/ 2.4.1c1 In-Reply-To: <4230B7A4.2080700@zope.com> References: <200503110138.02569.anthony@python.org> <1f7befae05031013053994c080@mail.gmail.com> <4230B7A4.2080700@zope.com> Message-ID: <200503111431.38198.anthony@interlink.com.au> On Friday 11 March 2005 08:09, Tres Seaver wrote: > |>By staring at the code of the failing test, it looks like the MRO of the > |>testcase class has changed: it declares a 'run' method, which is > |>supposed to run the external process, which clashes with the 'run' > |>method of unittest.TestCase. I don't know what change in the 2.4 -> > |>2.4.1c1 update would have mucked with the MRO (if a MRO issue is > |>involved). Looking in Misc/NEWS, I suspect that this fix is responsible. - unittest.TestCase.run() and unittest.TestSuite.run() can now be successfully extended or overridden by subclasses. Formerly, the subclassed method would be ignored by the rest of the module. (Bug #1078905). -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Fri Mar 11 04:33:02 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Fri Mar 11 04:33:25 2005 Subject: [Python-Dev] branch release24-maint is unfrozen, 2.4.1rc2? Message-ID: <200503111433.02408.anthony@interlink.com.au> Ok, the branch is unfrozen. At the current point in time, I think we're going to need an rc2. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From janssen at parc.com Fri Mar 11 04:40:50 2005 From: janssen at parc.com (Bill Janssen) Date: Fri Mar 11 04:41:16 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: Your message of "Thu, 10 Mar 2005 19:22:45 PST." <000701c525e9$a88dcc40$bf20c797@oemcomputer> Message-ID: <05Mar10.194053pst."58617"@synergy1.parc.xerox.com> > Over time, I've gotten feedback about these and other itertools recipes. > No one has objected to the True/False return values in those recipes or > in Guido's version. > > Guido's version matches the normal expectation of any/all being a > predicate. Also, it avoids the kind of errors/confusion that people > currently experience with Python's unique implementation of "and" and > "or". > > Returning the last element is not evil; it's just weird, unexpected, and > non-obvious. Resist the urge to get tricky with this one. Fine, but then let's keep reduce(), which has this nice property. Bill From jack at performancedrivers.com Fri Mar 11 04:58:22 2005 From: jack at performancedrivers.com (Jack Diederich) Date: Fri Mar 11 04:58:27 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <000701c525e9$a88dcc40$bf20c797@oemcomputer> References: <05Mar10.183828pst."58617"@synergy1.parc.xerox.com> <000701c525e9$a88dcc40$bf20c797@oemcomputer> Message-ID: <20050311035822.GD1839@performancedrivers.com> On Thu, Mar 10, 2005 at 10:22:45PM -0500, Raymond Hettinger wrote: > [Bill Janssen] > > I think I'd want them to be: > > > > def any(S): > > for x in S: > > if x: > > return x > > return S[-1] > > > > def all(S): > > for x in S: > > if not x: > > return x > > return S[-1] > > > > Or perhaps these should be called "first" and "last". > > -1 > > Over time, I've gotten feedback about these and other itertools recipes. > No one has objected to the True/False return values in those recipes or > in Guido's version. > > Guido's version matches the normal expectation of any/all being a > predicate. Also, it avoids the kind of errors/confusion that people > currently experience with Python's unique implementation of "and" and > "or". > > Returning the last element is not evil; it's just weird, unexpected, and > non-obvious. Resist the urge to get tricky with this one. Perl returns the last true/false value as well, and it is a subtle trap. I worked at a perl shop that had a long style guide which outlawed using that side effect but people did anyway. I'm +1 on having it return a true boolean for the same reason "if (x = calc_foo()):" isn't supported, if there is a quirky side effect available someone will use it and someone else won't notice. [in fact we had a guy who was "downsized" for doing things like calling database queries on the return value of a function that was documented as returning True/False] Perl shops require long style guides becuase perl is, ummm, perl. Python is a boon because YAGN (a style guide). The fewer side effects the better. -Jack From skip at pobox.com Tue Mar 8 20:29:48 2005 From: skip at pobox.com (Skip Montanaro) Date: Fri Mar 11 05:22:19 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <422DF5E5.1000406@ocf.berkeley.edu> References: <422D6DC6.5010001@v.loewis.de> <422DF5E5.1000406@ocf.berkeley.edu> Message-ID: <16941.64812.923315.785089@montanaro.dyndns.org> Brett> If there was no other way to get os.access-like functionality, I Brett> would say it should be backported. But since there are other Brett> ways to figure out everything that os.access can tell you I say Brett> don't backport... I don't think you can tell (certainly not easily) what the real user's permissions are on a file without a lot of work. access is almost never the right tool for the job (most of the time euid == ruid), but when it is the right tool it's a lot better than the alternative. I say backport. If people were trying to call os.access with unicode filenames it would have been failing and they were either avoiding unicode filenames as a result or working around it some other way. I can't see how making os.access work with unicode filenames is going to break existing code. Skip From skip at pobox.com Wed Mar 9 03:59:25 2005 From: skip at pobox.com (Skip Montanaro) Date: Fri Mar 11 05:22:31 2005 Subject: [Python-Dev] Urllib code or the docs appear wrong In-Reply-To: References: <16940.48341.344618.535840@montanaro.dyndns.org> Message-ID: <16942.26253.609060.629239@montanaro.dyndns.org> >> It seems to me that either urllib's docs are wrong or its code is >> wrong w.r.t. how the User-agent header is handled. Guido> I propose fixing the docs... Done (also backported to 2.4 branch). Skip From skip at pobox.com Wed Mar 9 13:59:42 2005 From: skip at pobox.com (Skip Montanaro) Date: Fri Mar 11 05:22:33 2005 Subject: No new features (was Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36) In-Reply-To: <200503092207.22313.anthony@interlink.com.au> References: <200503091208.38519.anthony@interlink.com.au> <20050309012152.GC30204@cthulhu.gerg.ca> <200503092207.22313.anthony@interlink.com.au> Message-ID: <16942.62270.916741.720953@montanaro.dyndns.org> Anthony> Initially, I was inclined to be much less anal about the Anthony> no-new-features thing. But since doing it, I've had a quite Anthony> large number of people tell me how much they appreciate this Anthony> approach - vendors, large companies with huge installed bases Anthony> of Python, and also from people releasing software written in Anthony> Python. Same here. People are amazed at work when I tell them I can just install a micro release without any breakage. Anthony> I should also add that the above is really just policy as it's Anthony> evolved - if people want to discuss this (am I being too Anthony> strict?) I'm happy to talk about it. Current policy gets a big +1 from me. Skip From skip at pobox.com Wed Mar 9 14:03:52 2005 From: skip at pobox.com (Skip Montanaro) Date: Fri Mar 11 05:22:35 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <200503092317.07909.anthony@interlink.com.au> References: <200503092317.07909.anthony@interlink.com.au> Message-ID: <16942.62520.281370.469617@montanaro.dyndns.org> Anthony> Goal 4: Try and prevent something like Anthony> try: Anthony> True, False Anthony> except NameError: Anthony> True, False = 1, 0 Anthony> from ever ever happening again. I will point out that in the transition from 2.3 to 2.4 our code that uses sets has try: x = set except NameError: from sets import Set as set else: del x Rather ugly. I suppose I could just put the necessary incantation in sitecustomize to dump the set name in builtins, but it would be kinda nice if this sort of thing could be avoided in the future. Skip From tim.peters at gmail.com Fri Mar 11 05:22:27 2005 From: tim.peters at gmail.com (Tim Peters) Date: Fri Mar 11 05:22:42 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <4230FAA5.4070309@v.loewis.de> References: <200503110138.02569.anthony@python.org> <4230FAA5.4070309@v.loewis.de> Message-ID: <1f7befae05031020227b12870c@mail.gmail.com> [Martin v. L?wis] > I'd like to encourage feedback on whether the Windows installer works > for people. It replaces the VBScript part in the MSI package with native > code, which ought to drop the dependency on VBScript, but might > introduce new incompatibilities. Worked fine here. Did an all-default "all users" install, WinXP Pro SP2, from local disk, and under an account with Admin rights. I uninstalled 2.4 first. I suppose that's the least stressful set of choices I could possibly have made, but at least it confirms a happy baseline. From bob at redivi.com Fri Mar 11 05:34:28 2005 From: bob at redivi.com (Bob Ippolito) Date: Fri Mar 11 05:34:21 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <16942.62520.281370.469617@montanaro.dyndns.org> References: <200503092317.07909.anthony@interlink.com.au> <16942.62520.281370.469617@montanaro.dyndns.org> Message-ID: <91a5753c78295cb177214d74dbe8da40@redivi.com> On Mar 9, 2005, at 8:03 AM, Skip Montanaro wrote: > > Anthony> Goal 4: Try and prevent something like > Anthony> try: > Anthony> True, False > Anthony> except NameError: > Anthony> True, False = 1, 0 > Anthony> from ever ever happening again. > > I will point out that in the transition from 2.3 to 2.4 our code that > uses > sets has > > try: > x = set > except NameError: > from sets import Set as set > else: > del x > > Rather ugly. I suppose I could just put the necessary incantation in > sitecustomize to dump the set name in builtins, but it would be kinda > nice > if this sort of thing could be avoided in the future. try: set except NameError: from sets import Set as set You don't need the rest. -bob From glyph at divmod.com Fri Mar 11 05:46:02 2005 From: glyph at divmod.com (Glyph Lefkowitz) Date: Fri Mar 11 05:46:07 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <91a5753c78295cb177214d74dbe8da40@redivi.com> References: <200503092317.07909.anthony@interlink.com.au> <16942.62520.281370.469617@montanaro.dyndns.org> <91a5753c78295cb177214d74dbe8da40@redivi.com> Message-ID: <4231228A.6050805@divmod.com> Bob Ippolito wrote: > try: > set > except NameError: > from sets import Set as set Syntactical variations notwithstanding, I think it's a common desire to want to run on at least the last few versions of Python, but take advantage of improvements and not emit deprecation warnings on the latest and greatest. I am considering releasing a forward-compatibility library for installation alongside applications that need to use multiple versions of Twisted, so they can do it in a style which is closer to the new versions than the old ones: perhaps it might be good practice to start doing this for Python as well? That way instead of multi-line "except NameError" tests all over the place you can simply have one-liner boilerplate for every module in your project: 'from py24compat import *' Easy to grep/sed for when you're ready to stop supporting old versions, too, for those of you without a copy of the refactoring editor from the 2009 release of PyDev/Eclipse. The only problem with this idea is that the 2.3 -> 2.4 transition has been extremely smooth for me - there are no new features I desperately want to use, and there are no old features that were deprecated or altered (that I've found yet - knock on wood). Still, future incompatibilties are inevitable. Of course I'd also like to echo the thanks to Anthony for having relegated this problem to major releases. From bob at redivi.com Fri Mar 11 05:57:56 2005 From: bob at redivi.com (Bob Ippolito) Date: Fri Mar 11 05:57:46 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <4231228A.6050805@divmod.com> References: <200503092317.07909.anthony@interlink.com.au> <16942.62520.281370.469617@montanaro.dyndns.org> <91a5753c78295cb177214d74dbe8da40@redivi.com> <4231228A.6050805@divmod.com> Message-ID: <288045f1e76f6f30321ec704cd1d229f@redivi.com> On Mar 10, 2005, at 11:46 PM, Glyph Lefkowitz wrote: > Bob Ippolito wrote: >> try: >> set >> except NameError: >> from sets import Set as set > > Syntactical variations notwithstanding, I think it's a common desire > to want to run on at least the last few versions of Python, but take > advantage of improvements and not emit deprecation warnings on the > latest and greatest. > > I am considering releasing a forward-compatibility library for > installation alongside applications that need to use multiple versions > of Twisted, so they can do it in a style which is closer to the new > versions than the old ones: perhaps it might be good practice to start > doing this for Python as well? > > That way instead of multi-line "except NameError" tests all over the > place you can simply have one-liner boilerplate for every module in > your project: > > 'from py24compat import *' > > Easy to grep/sed for when you're ready to stop supporting old > versions, too, for those of you without a copy of the refactoring > editor from the 2009 release of PyDev/Eclipse. > > The only problem with this idea is that the 2.3 -> 2.4 transition has > been extremely smooth for me - there are no new features I desperately > want to use, and there are no old features that were deprecated or > altered (that I've found yet - knock on wood). Still, future > incompatibilties are inevitable. > > Of course I'd also like to echo the thanks to Anthony for having > relegated this problem to major releases. I'm definitely +1 on such a library. I've cobbled together something like it myself: http://svn.red-bean.com/bob/py2app/trunk/src/altgraph/compat.py And this monkey-patches readline support in for UTF-16 streams: http://svn.red-bean.com/bob/unicode/trunk/utf16reader.py -bob From aahz at pythoncraft.com Fri Mar 11 06:53:22 2005 From: aahz at pythoncraft.com (Aahz) Date: Fri Mar 11 06:53:24 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <05Mar10.194053pst."58617"@synergy1.parc.xerox.com> References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <05Mar10.194053pst."58617"@synergy1.parc.xerox.com> Message-ID: <20050311055322.GA20294@panix.com> On Thu, Mar 10, 2005, Bill Janssen wrote: >Raymond Hettinger: >> >> Over time, I've gotten feedback about these and other itertools recipes. >> No one has objected to the True/False return values in those recipes or >> in Guido's version. >> >> Guido's version matches the normal expectation of any/all being a >> predicate. Also, it avoids the kind of errors/confusion that people >> currently experience with Python's unique implementation of "and" and >> "or". >> >> Returning the last element is not evil; it's just weird, unexpected, and >> non-obvious. Resist the urge to get tricky with this one. +1 > Fine, but then let's keep reduce(), which has this nice property. -1 -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From t-meyer at ihug.co.nz Fri Mar 11 07:06:33 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Fri Mar 11 07:06:38 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: Message-ID: [Martin v. L?wis] >> I'd like to encourage feedback on whether the Windows >> installer works for people. It replaces the VBScript part in the >> MSI package with native code, which ought to drop the dependency on >> VBScript, but might introduce new incompatibilities. [Tim Peters] > Worked fine here. Did an all-default "all users" install, > WinXP Pro SP2, from local disk, and under an account with > Admin rights. I uninstalled 2.4 first. I suppose that's the > least stressful set of choices I could possibly have made, > but at least it confirms a happy baseline. Also works fine for me with: * WinXP Pro SP2, from local disk, with admin rights, all defaults, over the top of 2.4.0 * Win2k SP4, from network disk, without admin rights, all defaults, with no previous 2.4 * Win2k SP4 (different machine), from local disk, with admin rights, defaults apart from skipped test suite, over the top of 2.4.0 =Tony.Meyer From martin at v.loewis.de Fri Mar 11 10:49:35 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 11 10:49:39 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <16941.64812.923315.785089@montanaro.dyndns.org> References: <422D6DC6.5010001@v.loewis.de> <422DF5E5.1000406@ocf.berkeley.edu> <16941.64812.923315.785089@montanaro.dyndns.org> Message-ID: <423169AF.6030807@v.loewis.de> Skip Montanaro wrote: > I say backport. If people were trying to call os.access with unicode > filenames it would have been failing and they were either avoiding unicode > filenames as a result or working around it some other way. I can't see how > making os.access work with unicode filenames is going to break existing > code. The question is whether it would encourage conditional work-arounds. If people will put into their code if sys.version_info < (2,4,2): import os, sys def new_access(name, mode, old_access = os.access): try: return old_access(name, mode) except UnicodeError: return old_access(name.encode( sys.getfilesystemencoding()), mode) os.access = new_access then backporting does not improve anything. OTOH, if people are likely to say "yes, this was a bug in 2.4.0 and 2.4.1, you need atleast 2.4.2", backporting will help. Regards, Martin From mal at egenix.com Fri Mar 11 11:23:21 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 11 11:23:26 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <423169AF.6030807@v.loewis.de> References: <422D6DC6.5010001@v.loewis.de> <422DF5E5.1000406@ocf.berkeley.edu> <16941.64812.923315.785089@montanaro.dyndns.org> <423169AF.6030807@v.loewis.de> Message-ID: <42317199.8090600@egenix.com> Martin v. L?wis wrote: > Skip Montanaro wrote: > > I say backport. If people were trying to call os.access with unicode > >> filenames it would have been failing and they were either avoiding >> unicode >> filenames as a result or working around it some other way. I can't >> see how >> making os.access work with unicode filenames is going to break existing >> code. > > > The question is whether it would encourage conditional work-arounds. -1. That only makes the code more complicated. > If > people will put into their code > > if sys.version_info < (2,4,2): > import os, sys > def new_access(name, mode, old_access = os.access): > try: > return old_access(name, mode) > except UnicodeError: > return old_access(name.encode( > sys.getfilesystemencoding()), mode) > os.access = new_access > > then backporting does not improve anything. OTOH, if people are likely > to say "yes, this was a bug in 2.4.0 and 2.4.1, you need atleast 2.4.2", > backporting will help. +1. Application writers can add tests for the correct version of Python to their application and give a warning to the user in case the version doesn't match. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 11 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bjourne at gmail.com Fri Mar 11 11:30:38 2005 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Fri Mar 11 11:30:40 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <20050311055322.GA20294@panix.com> References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> Message-ID: <740c3aec05031102301697d845@mail.gmail.com> Not sure this is pertinent but anyway: "any" and "all" are often used as variable names. "all" especially often and then almost always as a list of something. It would not be good to add "all" to the list of words to watch out for. Also, "all" is usually thought of as a list of (all) things. In my mind it doesn't make sense (yet) that all(seq) returns true if all elements of seq is true and false otherwise, I would have expected "all" to return a list. "any" is better because it is very obvious it can only return one thing. -- mvh Bj?rn From martin at v.loewis.de Fri Mar 11 12:05:52 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 11 12:05:55 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <42317199.8090600@egenix.com> References: <422D6DC6.5010001@v.loewis.de> <422DF5E5.1000406@ocf.berkeley.edu> <16941.64812.923315.785089@montanaro.dyndns.org> <423169AF.6030807@v.loewis.de> <42317199.8090600@egenix.com> Message-ID: <42317B90.5030100@v.loewis.de> M.-A. Lemburg wrote: >> The question is whether it would encourage conditional work-arounds. > > > -1. That only makes the code more complicated. You misunderstand. I'm not proposing that the work-around is added to Python. I'm saying that Python *users* might introduce such work-arounds to their code. > +1. Application writers can add tests for the correct version > of Python to their application and give a warning to the user > in case the version doesn't match. Application writers typically don't do that if they can find a work-around, because that means less hassle for their users. They accept that the code will get more complicated because of the work-around, and blame Python for having this bug in the first place. In any case, I can't backport the os.access change without explicit permission from Anthony. Regards, Martin From p.f.moore at gmail.com Fri Mar 11 12:18:47 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri Mar 11 12:18:51 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <740c3aec05031102301697d845@mail.gmail.com> References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> Message-ID: <79990c6b0503110318cf36d1c@mail.gmail.com> On Fri, 11 Mar 2005 11:30:38 +0100, BJ?rn Lindqvist wrote: > Not sure this is pertinent but anyway: "any" and "all" are often used > as variable names. "all" especially often and then almost always as a > list of something. It would not be good to add "all" to the list of > words to watch out for. Also, "all" is usually thought of as a list of > (all) things. In my mind it doesn't make sense (yet) that all(seq) > returns true if all elements of seq is true and false otherwise, I > would have expected "all" to return a list. "any" is better because it > is very obvious it can only return one thing. Using "any" and "all" as variables hides the builtins, but doesn't disallow their use elsewhere. Personally, though, I wouldn't use "any" or "all" as variable names, so that's a style issue. As far as the names making sense is concerned, they are perfect in context: if all(i > 0 for i in int_list): if any(invalid(s) for s in input_values): While you may think that use in any other context looks a little less natural (something I'm not convinced of), in the intended context, the names seem ideal to me. Paul. From mal at egenix.com Fri Mar 11 12:21:52 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 11 12:21:57 2005 Subject: [Python-Dev] os.access and Unicode In-Reply-To: <42317B90.5030100@v.loewis.de> References: <422D6DC6.5010001@v.loewis.de> <422DF5E5.1000406@ocf.berkeley.edu> <16941.64812.923315.785089@montanaro.dyndns.org> <423169AF.6030807@v.loewis.de> <42317199.8090600@egenix.com> <42317B90.5030100@v.loewis.de> Message-ID: <42317F50.2070805@egenix.com> Martin v. L?wis wrote: > M.-A. Lemburg wrote: > >>> The question is whether it would encourage conditional work-arounds. >> >> >> >> -1. That only makes the code more complicated. > > > You misunderstand. I'm not proposing that the work-around is added > to Python. I'm saying that Python *users* might introduce such > work-arounds to their code. Ah, ok. Either way, the code get's more complicated :-) >> +1. Application writers can add tests for the correct version >> of Python to their application and give a warning to the user >> in case the version doesn't match. > > > Application writers typically don't do that if they can find > a work-around, because that means less hassle for their users. > They accept that the code will get more complicated because > of the work-around, and blame Python for having this bug in the > first place. Hmm, eGenix always suggests to people to use the latest patch level release, esp. for production systems, not only for Python itself, but also for the products. Zope Corp. does the same, as do many other application writers. The only time where we do add work-arounds is for changes between minor Python versions in order to make the same code base work for multiple Python versions, but that's rare. > In any case, I can't backport the os.access change without explicit > permission from Anthony. Agreed. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 11 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From astrand at lysator.liu.se Fri Mar 11 12:43:01 2005 From: astrand at lysator.liu.se (Peter Astrand) Date: Fri Mar 11 12:43:14 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <79990c6b0503110318cf36d1c@mail.gmail.com> References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: On Fri, 11 Mar 2005, Paul Moore wrote: > > Not sure this is pertinent but anyway: "any" and "all" are often used > > as variable names. "all" especially often and then almost always as a > > list of something. It would not be good to add "all" to the list of > > words to watch out for. Also, "all" is usually thought of as a list of > Using "any" and "all" as variables hides the builtins, but doesn't > disallow their use elsewhere. Personally, though, I wouldn't use "any" > or "all" as variable names, so that's a style issue. Even though you can use them as variables (and shadow the builtins), you will still get warnings from "pychecker". The code will also be harder to read: When you see "all" in the middle of some code, you don't know if it's referring to the builtin or a variable. Personally, I think Python has too many builtins already. /Peter ?strand From pedronis at strakt.com Fri Mar 11 13:23:48 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Fri Mar 11 13:24:02 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: <42318DD4.6020201@strakt.com> Peter Astrand wrote: >On Fri, 11 Mar 2005, Paul Moore wrote: > > > >>>Not sure this is pertinent but anyway: "any" and "all" are often used >>>as variable names. "all" especially often and then almost always as a >>>list of something. It would not be good to add "all" to the list of >>>words to watch out for. Also, "all" is usually thought of as a list of >>> >>> > > > >>Using "any" and "all" as variables hides the builtins, but doesn't >>disallow their use elsewhere. Personally, though, I wouldn't use "any" >>or "all" as variable names, so that's a style issue. >> >> > >Even though you can use them as variables (and shadow the builtins), you >will still get warnings from "pychecker". The code will also be harder to >read: When you see "all" in the middle of some code, you don't know if >it's referring to the builtin or a variable. > >Personally, I think Python has too many builtins already. > > > to extend the naming pool, FWIW CL calls similar things EVERY and SOME. From pierre.barbier at cirad.fr Fri Mar 11 14:09:07 2005 From: pierre.barbier at cirad.fr (Pierre Barbier de Reuille) Date: Fri Mar 11 14:07:42 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: <42319873.9080102@cirad.fr> And why not use the names already in use in numarray/Numeric ? They are called "sometrue" and "alltrue" ... IMHO, it explicits more what it means : alltrue(i<5 for i in l) sometrue(i<5 for i in l) Another point is: as I agree there is already a huge lot of builtins, shouldn't it be in some module ? Perhaps in itertools ? Pierre PS: It's my first post on the list, even if I'm reading it for a few months now ^_^ Peter Astrand a ?crit : > On Fri, 11 Mar 2005, Paul Moore wrote: > > >>>Not sure this is pertinent but anyway: "any" and "all" are often used >>>as variable names. "all" especially often and then almost always as a >>>list of something. It would not be good to add "all" to the list of >>>words to watch out for. Also, "all" is usually thought of as a list of > > >>Using "any" and "all" as variables hides the builtins, but doesn't >>disallow their use elsewhere. Personally, though, I wouldn't use "any" >>or "all" as variable names, so that's a style issue. > > > Even though you can use them as variables (and shadow the builtins), you > will still get warnings from "pychecker". The code will also be harder to > read: When you see "all" in the middle of some code, you don't know if > it's referring to the builtin or a variable. > > Personally, I think Python has too many builtins already. > > > /Peter ?strand > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/pierre.barbier%40cirad.fr > -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77 fax : (33) 4 67 61 56 68 From mcherm at mcherm.com Fri Mar 11 14:21:51 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Fri Mar 11 14:21:54 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 Message-ID: <20050311052151.luqfnbc148gscgoc@mcherm.com> [Martin v. L?wis] > I'd like to encourage feedback on whether the Windows installer works > for people. It replaces the VBScript part in the MSI package with native > code, which ought to drop the dependency on VBScript, but might > introduce new incompatibilities. [Tim Peters] > Worked fine here. Did an all-default "all users" install, WinXP Pro > SP2, from local disk, and under an account with Admin rights. I > uninstalled 2.4 first. I suppose that's the least stressful set of > choices I could possibly have made, but at least it confirms a happy > baseline. I tried several stranger things, like installing over 2.4.0 but in a different directory. Everything worked like clockwork. I did NOT try anything that would have involved a system with various things missing (like lack of VBScript), but I did play around with alternate install locations, repairs, uninstalls, single-user and all-user installs, and I found no problems anywhere. Nice work! -- Michael Chermside From ncoghlan at iinet.net.au Fri Mar 11 15:43:10 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Mar 11 15:43:15 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: <4231AE7E.5060406@iinet.net.au> Peter Astrand wrote: > Personally, I think Python has too many builtins already. A suggestion was made on c.l.p a while back to have a specific module dedicated to reductive operations. That is, just as itertools is oriented towards manipulating iterables and creating iterators, this module would be oriented towards consuming iterators in a reductive fashion. product(), anytrue() and alltrue() were obvious candidates for inclusion ([1]). The combination of explicit for loops and a standard toolkit of reductive operations was designed to eliminate the need for reduce() ([2]). Cheers, Nick. [1] While any()/all() read well in the context of an if statement, I think anytrue()/alltrue() better convey the reductive nature of the operations, read nearly as well in the if context, and read significantly better when isolated from the if context (e.g. assigned to a variable). I also think the names are less likely to collide with existing variable names. [2] I'm firmly in Guido's camp on this one - whenever I encounter code that uses reduce(), I have to rewrite it (either mentally or literally) to use a for loop before I can understand it. Getting rid of the function would benefit me because I would no longer have to waste time figuring out what such code was doing - it would already be an explicit loop, or it would be using one of the standard reductive operations. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Fri Mar 11 15:50:44 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Mar 11 15:50:49 2005 Subject: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents In-Reply-To: <200503110015.04590.anthony@interlink.com.au> References: <000301c5253a$910ff480$163cc797@oemcomputer> <200503110015.04590.anthony@interlink.com.au> Message-ID: <4231B044.9080909@iinet.net.au> Anthony Baxter wrote: > On Thursday 10 March 2005 17:29, Raymond Hettinger wrote: > >>Or the implementation can have a switch to choose between keep-first >>logic or replace logic. >> >>The latter seems a bit odd to me. The key position would be determined >>by the first encountered while the value would be determined by the last >>encountered. Starting with [(10, v1), (20, v2), (10.0, v3)], the >>ordered dictionary's items would look like [(10, v3), (20, v2)]. > > > Or, alternately, we keep the last occurence, and move it to the new position. > There's a wide number of different use cases, each with a slightly different > final result, and for this reason alone I'm -0 on it for the library. > > Anthony > But surely such use cases could be more easily handled by subclassing from collections.OrderedDict and tweaking the semantics than by having to implement an ordered mapping from scratch. Would the default semantics below really be that suprising? "An ordered dictionary remembers the order in which keys are first seen and, when iterated over, returns entries based on that order. This applies to direct iteration, iteration over values and (key, value) pairs, to the list-producing methods (i.e. keys(), values() and items()) and to any other operations that involve implicit iteration (e.g. converting to a string representation). Overwriting an entry replaces its value, but does not affect its position in the key order. Removing an entry (using 'del') _does_ remove it from the key order. Accordingly, if the entry is later recreated, it will then occur last in the key order. This behaviour is analagous to that of a list constructed using only list.append() to add items (indeed, the key order can be thought of as a list constructed in this fashion, with keys appended to the list when they are first encountered). An ordered dictionary provides a sort() method. The sort operation is applied to the key ordering and affects future iteration over the dictionary. Again, this is analagous to sorting a list." For instance, to convert a standard dictionary to an ordered dictionary using a specific key function: x = collections.OrderedDict(sorted(d.itervalues(), key=keyfunc)) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From rodsenra at gpr.com.br Fri Mar 11 15:58:47 2005 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Fri Mar 11 15:58:51 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <42319873.9080102@cirad.fr> References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> <42319873.9080102@cirad.fr> Message-ID: <4231B227.8030803@gpr.com.br> [ Pierre Barbier de Reuille ]: > They are called "sometrue" and "alltrue" ... IMHO, it explicits more > what it means : > > alltrue(i<5 for i in l) > sometrue(i<5 for i in l) +1 [ from a comment in GvR's blog ] > > Why not, > > if True in (x > 42 for x in S): > > instead of "any" and why not > > if not False in (x > 42 for x in S): > > instead of "all"? > > Because "any" and "all" have shortcut semantics (they > return as soon as they can determine the final result). [ Guido ]: ---------- > See my blog: > http://www.artima.com/forums/flat.jsp?forum=106&thread=98196 > Do we even need a PEP ? In the absence of a PEP, soon will see in c.l.p discussions like: """ For completeness sake shouldn't there be a optimiztion version for nonetrue() ? def nonetrue(S): for x in S: if x: return False return True why not allfalse() ? Due to the previous use of sometrue(), guess it becomes easier to remeber nonetrue() than allfalse(). One may argue for aliasing(nonetrue, allfalse), and we are back to _builtin pollution_. """ So, people might fallback to any() and all(),realising that: '''not all()''' meaning somefalse() '''not any()''' meaning nonetrue()==allfalse() All I'm saying: +1 for the PEP. OFF-TOPIC: It is curious though that we choose to read an *implicit* True in [all(), any()] instead of an implicit False. I guess that is a moral or ethical choice coming from the Human realm, favouring Truth instead of Falsity. But that difference does not hold in the Boolean realm . best regards, Senra -- Rodrigo Senra MSc Computer Engineer rodsenra@gpr.com.br GPr Sistemas Ltda http://www.gpr.com.br From ncoghlan at iinet.net.au Fri Mar 11 16:16:34 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Mar 11 16:16:40 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <4231B227.8030803@gpr.com.br> References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> <42319873.9080102@cirad.fr> <4231B227.8030803@gpr.com.br> Message-ID: <4231B652.50702@iinet.net.au> Rodrigo Dias Arruda Senra wrote: > OFF-TOPIC: > > It is curious though that we choose to read an *implicit* > True in [all(), any()] instead of an implicit False. Actually, I think it's predicate logic's fault: if p(x) for any x then conclusion 1 if q(x) for all x then conclusion 2 Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From gvanrossum at gmail.com Fri Mar 11 16:19:56 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri Mar 11 16:20:00 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: > Even though you can use them as variables (and shadow the builtins), you > will still get warnings from "pychecker". So? > The code will also be harder to > read: When you see "all" in the middle of some code, you don't know if > it's referring to the builtin or a variable. Yes you do. Builtins will be followed by a '('; the variables won't. (Obviously there can be exceptions, but in practice very, very few.) > Personally, I think Python has too many builtins already. It has fewer than most dynamic languages; and remember that I'm trading product(), any(), all() for reduce(), map() and filter(). There are others slated for disappearance, too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jhylton at gmail.com Fri Mar 11 16:39:54 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Mar 11 16:39:59 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: On Fri, 11 Mar 2005 07:19:56 -0800, Guido van Rossum wrote: > > Personally, I think Python has too many builtins already. > > It has fewer than most dynamic languages; and remember that I'm > trading product(), any(), all() for reduce(), map() and filter(). > There are others slated for disappearance, too. I think I'd miss reduce, but I'd be happy to have it moved to an extension module. There's no need for reduce to be a builtin. I'd be very happy to have any and all around. Jeremy From reinhold-birkenfeld-nospam at wolke7.net Fri Mar 11 17:06:33 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Fri Mar 11 17:17:21 2005 Subject: [Python-Dev] Re: Adding any() and all() In-Reply-To: References: Message-ID: Guido van Rossum wrote: > See my blog: http://www.artima.com/forums/flat.jsp?forum=106&thread=98196 > > Do we even need a PEP or is there a volunteer who'll add any() and all() for me? There is an interesting proposal in the forum: """ He proposes that [x in list if x > 0] (which is currently undefined) be exactly equal to: [x for x in list if x > 0] """ What about that? And, don't you want to save any() and all() for junctions, like those in Perl 6? Reinhold -- Mail address is perfectly valid! From gvanrossum at gmail.com Fri Mar 11 17:28:24 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Fri Mar 11 17:28:30 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: Here's my take on the key issues brought up: Alternative names anytrue(), alltrue(): before I posted to my blog I played with these names (actually anyTrue(), allTrue(), anyFalse(), allFalse()). But I realized (1) any() and all() read much better in their natural context (an if statement), and there's no confusion there; (2) anyFalse() and allFalse() are redundant, so there's no need to distinguish between anyTrue() and anyFalse(). (To the person who wondered why we default to True: because 'if' tests for True, duh. :-) Whether to always return True and False or the first faling / passing element? I played with that too before blogging, and realized that the end case (if the sequence is empty or if all elements fail the test) can never be made to work satisfactory: picking None feels weird if the argument is an iterable of bools, and picking False feels weird if the argument is an iterable of non-bool objects. Whether to use each() and some() instead of all() and any(), to preserve variable namespace: each() in particular (and some() to some extent) emphasizes the individual element, while all() emphasizes the whole set. As long as we accept that the return value should not return one of the arguments elements, I prefer emphasizing the group. The namespace argument doesn't weigh much in my mind; there's no backwards incompatibility, and there are plenty of builtins that are routinely reused as variables (e.g. str, file, id, type) without apparently affecting readability. Context helps a lot! Whether to segregate these into a separate module: they are really a very small amount of syntactic sugat, and I expect that in most cases, instead of importing that module (which totally makes me lose my context while editing) I would probably just write the extra line that it takes to get the same effect with an explicit for loop and if statement. What worries me a bit about doing a PEP for this simple proposal is that it might accidentally have the wrong outcome: a compromise that can carry a majority rather than the "right" solution because nobody could "sell" it. Yet, if people really feel strongly that there are more issues that need to be discussed, and they think it's worth their time, let someone stand up now to own the PEP and guide the discussion. But personally, I think it's more efficient if Raymond just checks in his code now. :-) BTW I definitely expect having to defend removing map/filter/reduce/lambda with a PEP; that's much more controversial because it's *removing* something and hence by definition breaking code. The PEP, in addition to making a really strong case for each of the removals, will have to deal with a deprecation period, possibly moving the functions to a "functional programming" module, etc. PS in the blog responses, a problem with sum() was pointed out -- unless you use the second argument, it will only work for numbers. Now that string concatenation is apparently O(N) rather than O(N**2) (is that right, Raymond?) maybe this could be revised. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri Mar 11 17:39:40 2005 From: barry at python.org (Barry Warsaw) Date: Fri Mar 11 17:39:48 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <4231228A.6050805@divmod.com> References: <200503092317.07909.anthony@interlink.com.au> <16942.62520.281370.469617@montanaro.dyndns.org> <91a5753c78295cb177214d74dbe8da40@redivi.com> <4231228A.6050805@divmod.com> Message-ID: <1110559180.10082.37.camel@geddy.wooz.org> On Thu, 2005-03-10 at 23:46, Glyph Lefkowitz wrote: > That way instead of multi-line "except NameError" tests all over the > place you can simply have one-liner boilerplate for every module in your > project: > > 'from py24compat import *' > > Easy to grep/sed for when you're ready to stop supporting old versions, > too, for those of you without a copy of the refactoring editor from the > 2009 release of PyDev/Eclipse. I do something very similar, both for Mailman and for the commercial products I'm working on. There are lots of ways to handle this kind of thing, and there might be some value in a community effort to work together and standardize this. IWBNI there were some common idioms and packages that people could use. Having said that... > The only problem with this idea is that the 2.3 -> 2.4 transition has > been extremely smooth for me - there are no new features I desperately > want to use, and there are no old features that were deprecated or > altered (that I've found yet - knock on wood). Still, future > incompatibilties are inevitable. ...I agree. We just upgraded to 2.4 internally and it went embarrassingly smoothly. Had to find something else to do with the four days (cough - warsaw's first law - cough) we'd scheduled for that. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050311/6135f928/attachment.pgp From reinhold-birkenfeld-nospam at wolke7.net Fri Mar 11 17:23:43 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Fri Mar 11 17:42:07 2005 Subject: [Python-Dev] Re: @deprecated (was: Useful thread project for 2.5?) In-Reply-To: <42302F56.7060702@iinet.net.au> References: <000f01c52433$d6f5eb60$7eb99d8d@oemcomputer> <42302F56.7060702@iinet.net.au> Message-ID: Nick Coghlan wrote: > Raymond Hettinger wrote: >> Decorators like this should preserve information about the underlying >> function: >> >> >>> def deprecated(func): >>> """This is a decorator which can be used to mark functions >>> as deprecated. It will result in a warning being emmitted >>> when the function is used.""" >>> def newFunc(*args, **kwargs): >>> warnings.warn("Call to deprecated function.") >>> return func(*args, **kwargs) >> >> newFunc.__name__ = func.__name__ >> newFunc.__doc__ = func.__doc__ >> newFunc.__dict__.update(func.__dict__) >> >>> return newFunc > > A utility method on function objects could simplify this: > newFunc.update_info(func) +1. This is really good for 90% of all decorator uses. But maybe a better name should be found, perhaps "update_meta". Reinhold -- Mail address is perfectly valid! From barry at python.org Fri Mar 11 17:48:58 2005 From: barry at python.org (Barry Warsaw) Date: Fri Mar 11 17:49:04 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: <1110559738.10081.43.camel@geddy.wooz.org> On Fri, 2005-03-11 at 06:43, Peter Astrand wrote: > Even though you can use them as variables (and shadow the builtins), you > will still get warnings from "pychecker". The code will also be harder to > read: When you see "all" in the middle of some code, you don't know if > it's referring to the builtin or a variable. > Personally, I think Python has too many builtins already. I agree. Personally, I'd rather see 'all' called 'every' (I'm less sure about 'any' being called 'some'), and I'd rather see these put in some module other than builtin. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050311/0bb2e5b8/attachment-0001.pgp From python at rcn.com Fri Mar 11 18:18:17 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri Mar 11 18:22:39 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: Message-ID: <001e01c5265e$516ff440$dabb958d@oemcomputer> > BTW I definitely expect having to defend removing > map/filter/reduce/lambda with a PEP; that's much more controversial > because it's *removing* something and hence by definition breaking > code. I suspect that lambda will be the only bone of contention. The reduce() function can be moved to the functional module. The map() and filter() functions are already covered by the itertools module. Lambda will be more difficult. Eric Raymond adapted an anti-gun control slogan and said "you can pry lambda out of my cold dead hands." A bunch of folks will sorely miss the ability to create anonymous functions on the fly. When lambda is used for deferred argument evaluation (a la PEP 312), the def syntax is a crummy substitute. > PS in the blog responses, a problem with sum() was pointed out -- > unless you use the second argument, it will only work for numbers. Now > that string concatenation is apparently O(N) rather than O(N**2) (is > that right, Raymond?) maybe this could be revised. str.join() is still the best practice for string concatenation. Armin's optimization doesn't appear in other implementations of Python. In CPython, it has a set of pre-conditions that are usually but not always True. IIRC, it also doesn't apply to Unicode and ASCII mixed with Unicode. Also, the optimization is part of ceval.c and would not be triggered by sum()'s call to PyNumber_Add(). That limitation is not easily removed because the optimization depends on an interaction between the stack, variable refcounts, and the sequence of opcodes. IOW, your original pronouncement on the subject should remain in effect. Raymond From mal at egenix.com Fri Mar 11 18:40:19 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Mar 11 18:40:33 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <001e01c5265e$516ff440$dabb958d@oemcomputer> References: <001e01c5265e$516ff440$dabb958d@oemcomputer> Message-ID: <4231D803.9010000@egenix.com> Raymond Hettinger wrote: >>BTW I definitely expect having to defend removing >>map/filter/reduce/lambda with a PEP; that's much more controversial >>because it's *removing* something and hence by definition breaking >>code. +1 on the PEP -1 on removing those tools - breaks too much code. > I suspect that lambda will be the only bone of contention. The reduce() > function can be moved to the functional module. The map() and filter() > functions are already covered by the itertools module. Nope. Think of the side-effects that can occur as a result of calling the function argument n times with different arguments. itertools only help if the functions don't have side-effects. Iteration is nice, but not the solution to everything :-) > Lambda will be more difficult. Eric Raymond adapted an anti-gun control > slogan and said "you can pry lambda out of my cold dead hands." A bunch > of folks will sorely miss the ability to create anonymous functions on > the fly. When lambda is used for deferred argument evaluation (a la PEP > 312), the def syntax is a crummy substitute. While I'm no fan of lambdas either, the removal would break too much code (e.g. I have 407 hits in the code base I use regularly alone). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 11 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From barry at python.org Fri Mar 11 18:49:04 2005 From: barry at python.org (Barry Warsaw) Date: Fri Mar 11 18:49:09 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <001e01c5265e$516ff440$dabb958d@oemcomputer> References: <001e01c5265e$516ff440$dabb958d@oemcomputer> Message-ID: <1110563344.10079.54.camel@geddy.wooz.org> On Fri, 2005-03-11 at 12:18, Raymond Hettinger wrote: > I suspect that lambda will be the only bone of contention. The reduce() > function can be moved to the functional module. The map() and filter() > functions are already covered by the itertools module. I'm fine seeing reduce() eventually go; I've used it maybe a handful of times in all my years of Python. Using a list comprehension instead of map() is fine too, but I'll miss the filter(None, seq) idiom. Ping's suggested list comprehension abbreviation, i.e.: [x in seq if x] == [x for x in seq if x] might help. > Lambda will be more difficult. Eric Raymond adapted an anti-gun control > slogan and said "you can pry lambda out of my cold dead hands." A bunch > of folks will sorely miss the ability to create anonymous functions on > the fly. When lambda is used for deferred argument evaluation (a la PEP > 312), the def syntax is a crummy substitute. Yeah, I'm with you here. As warty as lambda is, it just is so damn convenient some times. I've recently been using it as a companion to property(), providing concise definitions of read-only attributes. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050311/b4e0befc/attachment.pgp From barry at python.org Fri Mar 11 18:54:07 2005 From: barry at python.org (Barry Warsaw) Date: Fri Mar 11 18:54:10 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <4231D803.9010000@egenix.com> References: <001e01c5265e$516ff440$dabb958d@oemcomputer> <4231D803.9010000@egenix.com> Message-ID: <1110563647.10082.60.camel@geddy.wooz.org> On Fri, 2005-03-11 at 12:40, M.-A. Lemburg wrote: > While I'm no fan of lambdas either, the removal would break too > much code (e.g. I have 407 hits in the code base I use regularly > alone). We /are/ talking Python 3000 here, right? I'm expecting all manner of code breakage in moving from Python 2 to Python 3. So while I would lament the loss of lambda too, I think its impact on my code will be in the noise compared to dealing with "optional" static typing and the truly saddening loss of <>. there's-a-wink-in-there-somewhere-ly y'rs, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050311/21365ac2/attachment.pgp From aleaxit at yahoo.com Fri Mar 11 19:35:04 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Fri Mar 11 19:33:07 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: On Mar 11, 2005, at 17:28, Guido van Rossum wrote: > PS in the blog responses, a problem with sum() was pointed out -- > unless you use the second argument, it will only work for numbers. Now Why is that a *problem*? It solves the "end case (if the sequence is empty" which you mention for any() and all() [[indeed, I think a similar approach to any and all might be best: have them return an item from the sequence, an empty sequence uses the optional second item defaulting to something reasonable -- 0 for sum, False for any, True for all, for example]] in a neatly explicit way. As I recall, I had tried to handle non-empty sequences by picking the first item and doing following + on it (with internal implementation specialcasing for strings, to avoid an "attractive nuisance" situation), and you cut the gordian knot by pronouncing about the second argument with a default of 0 (I immediately thought your pronouncement was excellent and I still can't see why it wouldn't be). Forcing the user to provide a 2nd arg when summing a sequence of non-numbers (say, datetime.timedelta instances, that's one example that ended up in the 2nd ed Cookbook) is good because it ensures a good return value when the sequence is empty (say, timedelta(0) rather than 0 as a number). If you're considering revisions to sum's design, my suggestion would be to find a way to let the user tell sum to use a more accurate approach when summing floats. Summing a million floats with a loop of total+=item loses a LOT of accuracy (several digits' worth!) -- but doing the summation the right way takes O(N) auxiliary temporary space, so both approaches (the naive, performance-optimized accuracy disaster, and the expensive but accurate one) should ideally be available. An optional use_partials keyword argument defaulting to False, for example, might allow that... (again, I've hopefully clarified the issues in another 2nd ed Cookbook recipe, I guess I can post that if it helps). Alex From python at rcn.com Fri Mar 11 19:39:59 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri Mar 11 19:44:21 2005 Subject: [Python-Dev] sum() In-Reply-To: Message-ID: <000c01c52669$bb0dea00$8b25c797@oemcomputer> [Alex] > If you're considering revisions to sum's design, my suggestion would be > to find a way to let the user tell sum to use a more accurate approach > when summing floats. FWIW, when accuracy is an issue, I use: sum(sorted(data, key=abs)) Raymond From aleaxit at yahoo.com Fri Mar 11 19:48:45 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Fri Mar 11 19:46:51 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <001e01c5265e$516ff440$dabb958d@oemcomputer> References: <001e01c5265e$516ff440$dabb958d@oemcomputer> Message-ID: On Mar 11, 2005, at 18:18, Raymond Hettinger wrote: > str.join() is still the best practice for string concatenation. ...except you actually need unicode.join if the strings are of that kind. Fortunately, ''.join intrinsically compensates: >>> x=[u'\u0fe0']*2 >>> ''.join(x) u'\u0fe0\u0fe0' *without* (as one would expect) the GD nuisance of converting x's items to str (hellishly hard to document accurately and clearly, but extremely convenient!-). Which reminds me -- could we have a methodcaller relative to attrgetter and itemgetter? "Sort a list of strings in a case-insensitive way" would become *SO* easy with sort(dalist, key=methodcaller('lower'))... can't REALLY recommend sort(dalist, key=str.lower) then the items of dalist MIGHT be either str or unicode items... (the relevance of ''.join is that it proves SOMEbody considered it important to deal with a list of either str or unicode in the joining context... why not in the sorting context then?). Alex From steven.bethard at gmail.com Fri Mar 11 20:02:53 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri Mar 11 20:02:57 2005 Subject: [Python-Dev] operator.methodcaller In-Reply-To: References: <001e01c5265e$516ff440$dabb958d@oemcomputer> Message-ID: On Fri, 11 Mar 2005 19:48:45 +0100, Alex Martelli wrote: > Which reminds me -- could we have a methodcaller relative to attrgetter > and itemgetter? "Sort a list of strings in a case-insensitive way" > would become *SO* easy with sort(dalist, key=methodcaller('lower'))... > can't REALLY recommend sort(dalist, key=str.lower) then the items of > dalist MIGHT be either str or unicode items... I'd like to second this suggestion -- I've run into this problem a few times. When you're using a listcomp or genexp, you can inline it of course, but especially with a lot of functions growing the incredibly convenient key= arguments in 2.5 (e.g. min, max and the deque functions if I recall correctly), methodcaller would be a much more duck-typing friendly way to create such callables. Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From jimjjewett at gmail.com Fri Mar 11 20:03:17 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri Mar 11 20:03:20 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) Message-ID: Barry: > Ping's suggested list comprehension abbreviation, i.e.: > [x in seq if x] == [x for x in seq if x] > might help. Note that the last x shouldn't have to be x. [x in seq if f(x)] is by far my most common syntax error, and [x for x in seq if f(x)] is always what I want instead. From aleaxit at yahoo.com Fri Mar 11 20:08:09 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Fri Mar 11 20:06:12 2005 Subject: [Python-Dev] sum() In-Reply-To: <000c01c52669$bb0dea00$8b25c797@oemcomputer> References: <000c01c52669$bb0dea00$8b25c797@oemcomputer> Message-ID: <948ec66a275dd5895036bebc9846b9fd@yahoo.com> On Mar 11, 2005, at 19:39, Raymond Hettinger wrote: > [Alex] >> If you're considering revisions to sum's design, my suggestion would > be >> to find a way to let the user tell sum to use a more accurate approach >> when summing floats. > > FWIW, when accuracy is an issue, I use: > > sum(sorted(data, key=abs)) ...and you may still lose a LOT of accuracy that way:-(. Raymond, you technically reviewed the 2nd ed Cookbook -- don't you recall the sidebar about why partials are the RIGHT way to do summations? I was SO proud of that one, thought I'd made the issue clearest than anywhere previously in print... To summarize: as we all know, a+b loses significant digits from (say) b when abs(a) is much larger than abs(b). Now, say we're summing a million numbers, all positive and roughly the same magnitude. If we do it by total += item (which is basically what sum does), by the time we've summed the first 100,000 or so total IS *much larger than item* in absolute value -- we're systematically tossing away about 5 digits from each new item by that time!!! To avoid such a massacre of digits, one uses partials -- summing items two at a time to get half as many partials, then iterating that idea about log2(N) times when summing N items. Problem is, one needs O(N) auxiliary space (assuming one can't just overwrite the incoming list/array/whatever). There's all other kinds of accuracy issues with a+b, but the one partials-based summation deals with is numero uno in many real-life situations, e.g. when one needs to get (say) the total area from N regions, each of roughly the same magnitude, whose areas were separately measured -- total length from N segments ditto -- total mass of N items ditto -- and so forth. Alex From jimjjewett at gmail.com Fri Mar 11 20:29:51 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri Mar 11 20:29:54 2005 Subject: [Python-Dev] Adding any() and all() Message-ID: Guido van Rossum: > Whether to segregate these into a separate module: they are really a > very small amount of syntactic sugat, and I expect that in most cases, > instead of importing that module (which totally makes me lose my > context while editing) I would probably just write the extra line that > it takes to get the same effect with an explicit for loop and if > statement. Is that so bad? If you plan to use them often, then from itertools import any, every is reasonable. If you only use them once and weren't expecting it (and want your imports at the top) ... well how awful is it to have an extra line or two in your code? These aren't *really* replacing map/filter/reduce because you're adding the new functions now, but the old-function removal is waiting until (forever?) A compromise (which may not be worth doing) would be to put them in another module, and then import them into __builtins__ if desired. map/filter/reduce could be moved out (but reimported for at least 2.5) in the same way, which would make their inclusion look more like "standard policy" than "part of the language". [And yes, I know that itertools might be the "wrong" module; I just don't remember the name of the module for iteration-related tools that happen to return something other than an iterator.] -jJ From barry at python.org Fri Mar 11 20:35:35 2005 From: barry at python.org (Barry Warsaw) Date: Fri Mar 11 20:35:39 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: Message-ID: <1110569735.10086.96.camel@geddy.wooz.org> On Fri, 2005-03-11 at 14:29, Jim Jewett wrote: > Is that so bad? > > If you plan to use them often, then > > from itertools import any, every +1 -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050311/d00aaed8/attachment.pgp From pinard at iro.umontreal.ca Fri Mar 11 21:10:40 2005 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Fri Mar 11 21:11:39 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: <20050311201040.GA29161@phenix.progiciels-bpi.ca> [Guido van Rossum] > But I realized (1) any() and all() read much better in their natural > context (an if statement), and there's no confusion there; I do not think builtins should read good in some statement contexts and bad in the others, or designed to be legible in only a few contexts. This would be a waste. In other contexts dans `if' or `while', `any(...)' might be read as "pick one of", in which case a True/False result might be surprising. `allTrue' (or such) is clearer in all contexts, even if not as nice in some of them. For `any(...)' to be clear in all contexts, it should return one of its arguments. That would surely do the proper thing in `if' or `while' context. We've read various arguments about this idea. Some (pro or con) arguments are only valid outside `if' and `while' context, other (pro and con) arguments are only valid within `if' and `while' context. If we are going to dismiss contexts outside `if' and `while', we should not retain arguments which only apply outside those contexts. This is my only complain. The overall idea and principle are good, especially if they succeed in cleaning out `reduce' and the others. However, if for compatibility reasons, `reduce' stays, than we are merely adding other ways, without sparing any, and this means yet another tiny bloat in Python, that might be better avoided. > What worries me a bit about doing a PEP for this simple proposal is > that it might accidentally have the wrong outcome: Isn't that true of any PEP? I thought going through a PEP for changes, and additions just as well, was not far from being a principle. Depending on opinions, the outcome is right or wrong. The principle of PEPs is not necessarily a good one in practice, as some PEPs are forever incomplete or unupdated, miss their role, and then sum up to having nearly been an annoyance. Good if PEPs may be avoided! :-) -- Fran?ois Pinard http://pinard.progiciels-bpi.ca From jrw at pobox.com Fri Mar 11 21:42:46 2005 From: jrw at pobox.com (John Williams) Date: Fri Mar 11 21:42:49 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: Message-ID: <423202C6.1020309@pobox.com> Jim Jewett wrote: > Guido van Rossum: >>[Why any() and all() shouldn't need to be imported.] > > Is that so bad? > > If you plan to use them often, then > > from itertools import any, every > > is reasonable. If you only use them once and weren't expecting it > (and want your imports at the top) ... well how awful is it to have > an extra line or two in your code? The problem with this approach is that any() and all() are so fundamental* that you should just use them without thinking about it, just as when you use "+" to conctenate strings, you don't have to stop and think to yourself, "Ah, this program needs to be able to manipulate strings. I'd better make sure string operations as available in this module." Thinking such thoughts takes you away from thinking about the problem you're trying to solve by manipulating strings. Likewise, programmers solve a lot of problems with boolean expressions, and it seems silly to require a special declaration just to make the full complement of boolean operations available. I can think of three ways of coping with any() and all() being in a module: First, I could just not use them. In that case all the effort here is wasted, and my code becomes less readable than it would have been otherwise. This is the approach I usually take with modules like "operator", where I can just as easily write a lambda expression (for now at least). Second, I could break my concentration to think about import statements every time I have a use for these particular functions. Third, I could import them at the top of every module. Since one of the distinguishing features of Python in a lack of gratuitous boilerplate code everywhere, I would find it very sad to add even a little bit. So while putting any() and all() into a module isn't that bad in itself, it seems like the start of a slippery slope that has Python at the top and C++ at the bottom. -- jw *I appreciate the irony of calling something "fundamental" when we've all gotten by just fine without it for many years--I'm trying to think from the perspective of someone used to dealing with a later (and hopefully better) version of Python. From sabbey at u.washington.edu Fri Mar 11 23:42:25 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Fri Mar 11 23:42:29 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators Message-ID: I would like to get some feedback on a proposal to introduce smalltalk/ruby-like "code blocks" to python. Code blocks are, among other things, a clean way to solve the "acquire/release" problem [1][2]. This proposal also touches on some of the problems PEP 288 [3] deals with. The best discussion I have been able to find on this topic is in the thread of ref. [4]. Since generators, when used with 'for' loops, are so similar to code blocks [5], one can imagine two ways to implement code blocks in python: (1, parallel) keep them as similar as possible to 'for' loops so that a normal user doesn't distinguish the two, or (2, orthogonal) make them so much different than 'for' loops that the normal user doesn't notice the similarities. Anything between these extremes will probably be confusing. Here I will describe an attempt at (1). (I) Give generators a __call__ method as an alternative to 'next'. Method __call__ should take a single parameter: a function. Using __call__ will cause the generator to start executing normally, but when a 'yield' is reached, the generator will invoke the function passed to __call__ instead of activating the currently implemented 'yield' mechanism. Since __call__ will be an alternative to 'next', it will raise an exception if 'next' has already been called and vice-versa. Also, any calls to 'next' will raise an exception if there is a 'yield' in the try of a try/finally. Such yields will no longer trigger a SyntaxError because they will not a problem when using __call__. (II) Have a method of generators, __blockcall__, which will be equal to __call__, but will only exist if (1) the generator contains a "try/finally try" yield, or (2) the user explicitly defines it, for example, with a function decorator (@completion_required would be a descriptive name). Have 'for' loops use __blockcall__ if it is available, and __iter__ otherwise. Pass to __blockcall__ the block of code in the 'for' loop. Scope rules for the passed block of code should mimic current 'for' loop behavior. Behavior of 'break' and 'return' should be mimicked, perhaps with special exceptions catchable only by the 'for' loop. Mimicking 'yield' will be a problem unless/until multi-level yields are allowed. (performance and implementation difficulties for all of this? I don't know). The thunk shouldn't be savable for later use because the 'for' loop will no longer be around to deal with 'break' and 'return'. This means that __blockcall__ will not be implementable as a function that takes a function as an argument. (III) Allow 'continue' to pass values to 'yield' (something similar previously rejected here [6]). As far as I know, all other control statements that transfer execution to a different frame (yield, return, raise) pass values, and I don't see why this case should be any different. I do not see such a mechanism as gimmicky; being able to cleanly pass values when changing scope is an inherent part of nearly every programming language. As an example of the syntax I am suggesting, here is something I was desiring recently, a generator to open, unpickle, repickle and close a file: def pickled_file(name): f = open(name, 'r') l yield pickle.load(f) f.close() f = open(name, 'w') pickle.dump(l, f) f.close() The unpickled object is sent to the caller at the yield statement, and the modified object is received back at the same statement. Note the suggested 'yield' syntax and the conspicuous absence of '='. This syntax is backwardly compatible with current yield syntax. Also, this syntax does not require yield to appear as a function; it is still clear that this is a unique control-flow statement. This function would be used like this: for l in pickled_file('greetings.pickle'): l.append('hello') l.append('howdy') continue l The above code would have the same effect as: def named(l): l.append('hello') l.append('howdy') return l pickled_file('greetings.pickle')(named) (IV) Allow 'yield' to return no value; in this case a new keyword, 'with', will be required instead of an awkward 'for': "with f():" instead of "for in f():" (V) For the same reasons as in (III), allow generators to return values. These values can be sent with the StopIteration exception if 'next' is being used for iteration. An obvious syntax for receiving these values is shown by this example: with dt = stopwatch(): f() g() print 'it took', dt, 'seconds' Although "with stopwatch() result dt:" might not be so bad. [1] PEP 310 Reliable Acquisition/Release Pairs http://www.python.org/peps/pep-0310.html [2] PEP 325 Resource-Release Support for Generators http://www.python.org/peps/pep-0325.html [3] PEP 288 Generators Attributes and Exceptions http://www.python.org/peps/pep-0288.html [4] http://mail.python.org/pipermail/python-dev/2003-February/032800.html [5] http://mail.python.org/pipermail/python-dev/2003-February/032826.html [6] http://mail.python.org/pipermail/python-dev/2002-March/021923.html -Brian From foom at fuhm.net Fri Mar 11 23:48:02 2005 From: foom at fuhm.net (James Y Knight) Date: Fri Mar 11 23:48:12 2005 Subject: [Python-Dev] operator.methodcaller In-Reply-To: References: <001e01c5265e$516ff440$dabb958d@oemcomputer> Message-ID: On Mar 11, 2005, at 2:02 PM, Steven Bethard wrote: > On Fri, 11 Mar 2005 19:48:45 +0100, Alex Martelli > wrote: >> Which reminds me -- could we have a methodcaller relative to >> attrgetter >> and itemgetter? "Sort a list of strings in a case-insensitive way" >> would become *SO* easy with sort(dalist, key=methodcaller('lower'))... >> can't REALLY recommend sort(dalist, key=str.lower) then the items of >> dalist MIGHT be either str or unicode > > I'd like to second this suggestion -- I've run into this problem a few > times. When you're using a listcomp or genexp, you can inline it of > course, but especially with a lot of functions growing the incredibly > convenient key= arguments in 2.5 (e.g. min, max and the deque > functions if I recall correctly), methodcaller would be a much more > duck-typing friendly way to create such callables. Yeah, if *only* we had some way to create callables in a nice, short, generic, and easy to understand way. How about this proposal: >> sort(dalist, key=lambda x: x.lower()) Who wants to make a PEP for it? Maybe we can add it in Python 3000. James From python at rcn.com Fri Mar 11 23:52:45 2005 From: python at rcn.com (Raymond Hettinger) Date: Fri Mar 11 23:57:07 2005 Subject: [Python-Dev] sum() Message-ID: <001c01c5268d$0ad802a0$8b25c797@oemcomputer> [Alex] > > FWIW, when accuracy is an issue, I use: > > > > sum(sorted(data, key=abs)) > > ...and you may still lose a LOT of accuracy that way:-(. > > Raymond, you technically reviewed the 2nd ed Cookbook -- don't you > recall the sidebar about why partials are the RIGHT way to do > summations? I was SO proud of that one, thought I'd made the issue > clearest than anywhere previously in print... No doubt about it. Partials are superior. My little snippet meets my needs with no fuss -- my usual situation is many small values whose accuracy gets clobbered upon encountering a handful of large values. In the draft statistics module, nondist/sandbox/statistics/statistics.py , I used yet another approach and tracked a separate error term. The advantage is adding precision while still keeping O(n) summation speed. Of course, we can always use the decimal module to add as much precision as needed. Raymond From trentm at ActiveState.com Sat Mar 12 00:07:48 2005 From: trentm at ActiveState.com (Trent Mick) Date: Sat Mar 12 00:11:11 2005 Subject: [Python-Dev] distutils fix for building Zope against Python 2.4.1c1 Message-ID: <20050311230747.GA18655@ActiveState.com> http://mail.python.org/pipermail/python-checkins/2005-March/045185.html Note that I also could not build PyWin32 against 2.4.1c1 and I suspect this was the same problem. (Still checking to see if this change fixes the PyWin32 build for me.) In any case, this change should also make it to the trunk, right? Trent -- Trent Mick TrentM@ActiveState.com From trentm at ActiveState.com Sat Mar 12 00:15:50 2005 From: trentm at ActiveState.com (Trent Mick) Date: Sat Mar 12 00:19:11 2005 Subject: [Python-Dev] distutils fix for building Zope against Python 2.4.1c1 In-Reply-To: <20050311230747.GA18655@ActiveState.com> References: <20050311230747.GA18655@ActiveState.com> Message-ID: <20050311231549.GB18655@ActiveState.com> [Trent Mick wrote] > > http://mail.python.org/pipermail/python-checkins/2005-March/045185.html > > Note that I also could not build PyWin32 against 2.4.1c1 and I suspect > this was the same problem. (Still checking to see if this change fixes > the PyWin32 build for me. > ... It doesn't. Investigating... Trent -- Trent Mick TrentM@ActiveState.com From tim.peters at gmail.com Sat Mar 12 00:22:12 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sat Mar 12 00:22:14 2005 Subject: [Python-Dev] Re: distutils fix for building Zope against Python 2.4.1c1 In-Reply-To: <20050311230747.GA18655@ActiveState.com> References: <20050311230747.GA18655@ActiveState.com> Message-ID: <1f7befae050311152210d8cd07@mail.gmail.com> [Trent Mick] > http://mail.python.org/pipermail/python-checkins/2005-March/045185.html > > Note that I also could not build PyWin32 against 2.4.1c1 and I suspect > this was the same problem. (Still checking to see if this change fixes > the PyWin32 build for me.) Be sure to speak up if it doesn't! Any build that invokes the C compiler often on Windows would have the same problem. > In any case, this change should also make it to the trunk, right? Don't think it's needed on HEAD. As the checkin comment said: This doesn't appear to be an issue on the HEAD (MSVCCompiler initializes itself via __init__() on the HEAD). I verified by building Zope with unaltered HEAD too, and had no problem with that. From skip at pobox.com Fri Mar 11 20:26:40 2005 From: skip at pobox.com (Skip Montanaro) Date: Sat Mar 12 00:40:48 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <91a5753c78295cb177214d74dbe8da40@redivi.com> References: <200503092317.07909.anthony@interlink.com.au> <16942.62520.281370.469617@montanaro.dyndns.org> <91a5753c78295cb177214d74dbe8da40@redivi.com> Message-ID: <16945.61680.296812.366920@montanaro.dyndns.org> Bob> try: Bob> set Bob> except NameError: Bob> from sets import Set as set Bob> You don't need the rest. Sure, but then pychecker bitches about a statement that appears to have no effect. ;-) Skip From bob at redivi.com Sat Mar 12 00:47:11 2005 From: bob at redivi.com (Bob Ippolito) Date: Sat Mar 12 00:46:47 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: <16945.61680.296812.366920@montanaro.dyndns.org> References: <200503092317.07909.anthony@interlink.com.au> <16942.62520.281370.469617@montanaro.dyndns.org> <91a5753c78295cb177214d74dbe8da40@redivi.com> <16945.61680.296812.366920@montanaro.dyndns.org> Message-ID: On Mar 11, 2005, at 2:26 PM, Skip Montanaro wrote: > > Bob> try: > Bob> set > Bob> except NameError: > Bob> from sets import Set as set > > Bob> You don't need the rest. > > Sure, but then pychecker bitches about a statement that appears to > have no > effect. ;-) Well then fix PyChecker to look for this pattern :) -bob From mal at egenix.com Sat Mar 12 01:32:31 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Sat Mar 12 01:32:33 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> Message-ID: <4232389F.2050009@egenix.com> Guido van Rossum wrote: > Here's my take on the key issues brought up: > > Alternative names anytrue(), alltrue(): before I posted to my blog I > played with these names (actually anyTrue(), allTrue(), anyFalse(), > allFalse()). But I realized (1) any() and all() read much better in > their natural context (an if statement), and there's no confusion > there; (2) anyFalse() and allFalse() are redundant, so there's no need > to distinguish between anyTrue() and anyFalse(). (To the person who > wondered why we default to True: because 'if' tests for True, duh. :-) I've been using exists() and forall() in mxTools for years. The names originate from math usage, where you often use these qualifiers. A note on builtins: Most tools in mxTools automatically get registered as builtins when you load the module. The motivation for this was that wanted to have easy access to these often used APIs. However, a few years down the line I realized that this kind of usage just causes confusion when you read code: * it is not clear where the APIs originate * the code dependencies are not longer easily visible Ever since, I've switched to making all use of these functions explicit with reference to the mx.Tools module. http://www.egenix.com/files/python/mxTools.html -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 12 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ncoghlan at iinet.net.au Sat Mar 12 01:43:48 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 12 01:43:54 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: References: Message-ID: <42323B44.5080201@iinet.net.au> Jim Jewett wrote: > Note that the last x shouldn't have to be x. > > [x in seq if f(x)] > > is by far my most common syntax error, and > > [x for x in seq if f(x)] > > is always what I want instead. That 'x in seq' bit still shouts "containment" to me rather than iteration, though. Perhaps repurposing 'from': (x from seq if f(x)) That rather breaks TOOWTDI though (since it is essentially new syntax for a for loop). And I have other hopes for the meaning of (x from ()). . . Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From tim.leeuwvander at nl.unisys.com Thu Mar 10 22:19:52 2005 From: tim.leeuwvander at nl.unisys.com (Leeuw van der, Tim) Date: Sat Mar 12 01:49:34 2005 Subject: [Python-Dev] Python2.4.1c1 and win32com Message-ID: Hi, Win32com generates Python-files for use with com interfaces, using the make-py.py utility. The generated files are OK with Python2.3.5 The generated files crash the Python interpreter with Python 2.4 Under Python 2.4.1c1, They give a syntax error!? The files unfortunately are very big, nearly 2Mb each, although they compress very well (270Kb). The errors given are for one file: "D:\Python24\lib\site-packages\win32com\gen_py\00020905-0000-0000-C000-000000000046x0x8x1.py", line 3025 "StyleName": (3, 2, (8, 0), (), "StyleName", None), "Value": (0, 2, (8, 0), (), "Value", None), ^ SyntaxError: invalid syntax (The marker is in the middle of 'None') And for the other file: File "D:\Python24\lib\site-packages\win32com\gen_py\00020813-0000-0000-C000-000000000046x0x1x3.py", line 2591 return ret ^ SyntaxError: invalid syntax At least the interpreter no longer crashes, but I'm very surprised that this gives syntax errors when the exact same files do compile with Python 2.3.5 (To be sure, I generated the files using Python2.3.5, then copied them to the 2.4.1c1 installation, and then tried to have them compiled.) Eyeballing the files, I can definately not see any syntax errors. Anyone who can help or offer any suggestions? Please CC me on any replies b/c I'm not subscribed to the list. regards, --Tim From ncoghlan at iinet.net.au Sat Mar 12 01:55:25 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 12 01:55:30 2005 Subject: [Python-Dev] Lambda & deferred evaluation (was: Adding any() and all()) In-Reply-To: <1110563344.10079.54.camel@geddy.wooz.org> References: <001e01c5265e$516ff440$dabb958d@oemcomputer> <1110563344.10079.54.camel@geddy.wooz.org> Message-ID: <42323DFD.4040105@iinet.net.au> Barry Warsaw wrote: > On Fri, 2005-03-11 at 12:18, Raymond Hettinger wrote: >>Lambda will be more difficult. Eric Raymond adapted an anti-gun control >>slogan and said "you can pry lambda out of my cold dead hands." A bunch >>of folks will sorely miss the ability to create anonymous functions on >>the fly. When lambda is used for deferred argument evaluation (a la PEP >>312), the def syntax is a crummy substitute. > > > Yeah, I'm with you here. As warty as lambda is, it just is so damn > convenient some times. I've recently been using it as a companion to > property(), providing concise definitions of read-only attributes. I'm hoping that "lambda goes in Python3K" will get rid of the *existing* lambda, so that a more Pythonic syntax for deferred evaluation can be found. At the moment, any such attempt to come up with an alternate syntax tends to get shouted down on the basis that "you can already use lambda, TOOWTDI, stop wasting time". I see the *feature* as useful, but the current syntax as lousy. If it was a PEP, the latter would be grounds for rejection of lambda expressions (such as happened to PEP 308). A collection of alternate suggestions did get posted to the Wiki though (my personal favourite is "( from )"): http://www.python.org/moin/AlternateLambdaSyntax Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From kbk at shore.net Sat Mar 12 01:59:25 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Mar 12 02:00:03 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 In-Reply-To: <200503100406.09997.anthony@interlink.com.au> (Anthony Baxter's message of "Thu, 10 Mar 2005 04:06:08 +1100") References: <200503091148.50639.fdrake@acm.org> <200503100406.09997.anthony@interlink.com.au> Message-ID: <87psy5znki.fsf@hydra.bayview.thirdcreek.com> Anthony Baxter writes: > On Thursday 10 March 2005 03:48, Fred L. Drake, Jr. wrote: >> > RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/idlever.py,v >> > -IDLE_VERSION = "1.1.1" >> > +IDLE_VERSION = "1.1.1c1" >> >> Shouldn't this progress from 1.1.1 to 1.1.2c1? Otherwise it's moving >> backward. > > No - Py2.4 shipped with idle 1.1. I must've updated it to 1.1.1 sometime > after the 2.4 release (I vaguely recall doing this). To eliminate this confusion I'd propose either 1. Eliminating the separate IDLE versioning now that it's installed with Python when Tcl/Tk is available. or 2. Bringing its version in sync with Python 2.5. I'm in favor of the latter, to handle the situation where a customized IDLE is decoupled from Python. -- KBK From trentm at ActiveState.com Sat Mar 12 02:24:28 2005 From: trentm at ActiveState.com (Trent Mick) Date: Sat Mar 12 02:27:56 2005 Subject: [Python-Dev] distutils fix for building Zope against Python 2.4.1c1 In-Reply-To: <20050311231549.GB18655@ActiveState.com> References: <20050311230747.GA18655@ActiveState.com> <20050311231549.GB18655@ActiveState.com> Message-ID: <20050312012427.GA3266@ActiveState.com> [Trent Mick wrote] > [Trent Mick wrote] > > > > http://mail.python.org/pipermail/python-checkins/2005-March/045185.html > > > > Note that I also could not build PyWin32 against 2.4.1c1 and I suspect > > this was the same problem. (Still checking to see if this change fixes > > the PyWin32 build for me. > > ... > > It doesn't. Investigating... Investigation has turned up that I cannot keep my Python trees straight. That patch *does* fix building PyWin32 against 2.4.1c1. Cheers, Trent -- Trent Mick TrentM@ActiveState.com From tim.peters at gmail.com Sat Mar 12 02:35:32 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sat Mar 12 02:35:36 2005 Subject: [Python-Dev] distutils fix for building Zope against Python 2.4.1c1 In-Reply-To: <20050312012427.GA3266@ActiveState.com> References: <20050311230747.GA18655@ActiveState.com> <20050311231549.GB18655@ActiveState.com> <20050312012427.GA3266@ActiveState.com> Message-ID: <1f7befae0503111735cf8663e@mail.gmail.com> [Trent Mick] > Investigation has turned up that I cannot keep my Python trees straight. > That patch *does* fix building PyWin32 against 2.4.1c1. Good! Please send a check for US$1000.00 to the PSF by Monday . From t-meyer at ihug.co.nz Sat Mar 12 03:31:03 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Sat Mar 12 03:31:13 2005 Subject: [Python-Dev] Python2.4.1c1 and win32com In-Reply-To: Message-ID: > Win32com generates Python-files for use with com interfaces, > using the make-py.py utility. > > The generated files are OK with Python2.3.5 > > The generated files crash the Python interpreter with Python 2.4 > > Under Python 2.4.1c1, They give a syntax error!? > > The files unfortunately are very big, nearly 2Mb each, > although they compress very well (270Kb). [...] > Anyone who can help or offer any suggestions? I believe this is a pywin32 bug, which has been fixed in (pywin32) CVS and will be fixed for the next build. It's certainly a probably with 2.4.0 as well as 2.4.1. The pywin32 mailing list archives have more details, as do the tracker for the project: . =Tony.Meyer From ejones at uwaterloo.ca Sat Mar 12 03:44:18 2005 From: ejones at uwaterloo.ca (Evan Jones) Date: Sat Mar 12 04:13:54 2005 Subject: [Python-Dev] Memory Allocator Part 2: Did I get it right? In-Reply-To: <42293BBB.1010000@ocf.berkeley.edu> References: <8b28704b4465e03002fc70db5facedb6@uwaterloo.ca> <1f7befae05021514524d0a35ec@mail.gmail.com> <4c0d14b0b08390d046e1220b6f360745@uwaterloo.ca> <1f7befae05021520263d77a2a3@mail.gmail.com> <4212FB5B.1030209@v.loewis.de> <42293BBB.1010000@ocf.berkeley.edu> Message-ID: <422cc7d350ef750f9c11e8a2015d5b22@uwaterloo.ca> On Mar 4, 2005, at 23:55, Brett C. wrote: > Evan uploaded the newest version (@ http://www.python.org/sf/1123430) > and he says it is "complete". I should point out (very late! It's been a busy couple of weeks) that I fully expect that there may be some issues with the patch that will arise when it is reviewed. I am still lurking on Python-Dev, and I will strive to correct any defects ASAP. A few users have contacted me privately and have tested the patch, so it works for a few people at least. Evan Jones From tim.peters at gmail.com Sat Mar 12 05:02:55 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sat Mar 12 05:02:58 2005 Subject: [Python-Dev] sum() In-Reply-To: <001c01c5268d$0ad802a0$8b25c797@oemcomputer> References: <001c01c5268d$0ad802a0$8b25c797@oemcomputer> Message-ID: <1f7befae05031120024dba870c@mail.gmail.com> FYI, there are a lot of ways to do accurate fp summation, but in general people worry about it too much (except for those who don't worry about it at all -- they're _really_ in trouble <0.5 wink>). One clever way is to build on that whenever |x| and |y| are within a factor of 2 of each other, x+y is exact in 754 arithmetic. So you never lose information (not even one bit) when adding two floats with the same binary exponent. That leads directly to this kind of code: from math import frexp class Summer: def __init__(self): self.exp2sum = {} def add(self, x): while 1: exp = frexp(x)[1] if exp in self.exp2sum: x += self.exp2sum.pop(exp) # exact! else: break self.exp2sum[exp] = x # trivially exact def total(self): items = self.exp2sum.items() items.sort() return sum((x for dummy, x in items), 0.0) exp2sum maps a binary exponent to a float having that exponent. If you pass a sequence of fp numbers to .add(), then ignoring underflow/overflow endcases, the key invariant is that the exact (mathematical) sum of all the numbers passed to add() so far is equal to the exact (mathematical) sum of exp2sum.values(). While it's not obvious at first, the total number of additions performed inside add() is typically a bit _less_ than the total number of times add() is called. More importantly, len(exp2sum) can never be more than about 2K. The worst case for that is having one input with each possible binary exponent, like 2.**-1000 + 2.**-999 + ... + 2.**999 + 2.**1000. No inputs are like that in real life, and exp2sum typically has no more than a few dozen entries. total() then adds those, in order of increasing exponent == in order of increasing absolute value. This can lose information, but there is no bad case for it, in part because there are typically so few addends, and in part because that no two addends have the same binary exponent implies massive cancellation can never occur. Playing with this can show why most fp apps shouldn't worry most often. Example: Get a million floats of about the same magnitude: >>> xs = [random.random() for dummy in xrange(1000000)] Sum them accurately: >>> s = Summer() >>> for x in xs: ... s.add(x) No information has been lost yet (if you could look at your FPU's "inexact result" flag, you'd see that it hasn't been set yet), and the million inputs have been squashed into just a few dozen buckets: >>> len(s.exp2sum) 22 >>> from pprint import pprint >>> pprint(s.exp2sum) {-20: 8.8332388070710977e-007, -19: 1.4206079529399673e-006, -16: 1.0065260162672729e-005, -15: 2.4398649189794064e-005, -14: 5.3980784313178987e-005, -10: 0.00074737138777436485, -9: 0.0014605198999595448, -8: 0.003361820812962546, -7: 0.0063811680318408559, -5: 0.016214300821313588, -4: 0.044836286041944229, -2: 0.17325355843673518, -1: 0.46194788522906305, 3: 6.4590200674982423, 4: 11.684394209886134, 5: 24.715676913177944, 6: 49.056084672323166, 10: 767.69329043309051, 11: 1531.1560084859361, 13: 6155.484212371357, 17: 98286.760386143636, 19: 393290.34884990752} Add those 22, smallest to largest: >>> s.total() 500124.06621686369 Add the originals, "left to right": >>> sum(xs) 500124.06621685845 So they're the same to about 14 significant decimal digits. No good if exact pennies are important, but far more precise than real-world measurements. How much does sorting first help? >>> xs.sort() >>> sum(xs) 500124.06621685764 Not much! It actually moved a bit in the wrong direction (which is unusual, but them's the breaks). Using decimal as a (much more expensive) sanity check: >>> import decimal >>> t = decimal.Decimal(0) >>> for x in xs: ... t += decimal.Decimal(repr(x)) >>> print t 500124.0662168636972127329455 Of course Summer's result is very close to that. From tim.peters at gmail.com Sat Mar 12 05:23:43 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sat Mar 12 05:23:45 2005 Subject: [Python-Dev] sum() In-Reply-To: <1f7befae05031120024dba870c@mail.gmail.com> References: <001c01c5268d$0ad802a0$8b25c797@oemcomputer> <1f7befae05031120024dba870c@mail.gmail.com> Message-ID: <1f7befae05031120232815581c@mail.gmail.com> [Tim Peters] ... > One clever way is to build on that whenever |x| and |y| are within a > factor of 2 of each other, x+y is exact in 754 arithmetic. Ack, I'm fried. Forget that, it's wrong. The correct statement is that x-y is always exact whenever x and y are within a factor of two of each other. Summer.add() _can_ lose info -- it needs additional trickery to make it loss-free, and because x+y can lose (at most one bit) when x and y have the same binary exponent. Exercise for the abused reader . From ncoghlan at iinet.net.au Sat Mar 12 05:25:55 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 12 05:26:00 2005 Subject: [Python-Dev] func.update_meta (was: @deprecated) In-Reply-To: References: <000f01c52433$d6f5eb60$7eb99d8d@oemcomputer> <42302F56.7060702@iinet.net.au> Message-ID: <42326F53.3060106@iinet.net.au> Reinhold Birkenfeld wrote: > Nick Coghlan wrote: >>A utility method on function objects could simplify this: >> newFunc.update_info(func) > > > +1. This is really good for 90% of all decorator uses. But maybe a > better name should be found, perhaps "update_meta". I like "update_meta" Patch against current CVS added to SF with the behaviour: def update_meta(self, other): self.__name__ = other.__name__ self.__doc__ = other.__doc__ self.__dict__.update(other.__dict__) http://www.python.org/sf/1161819 Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From briandorsey at gmail.com Sat Mar 12 07:14:24 2005 From: briandorsey at gmail.com (Brian Dorsey) Date: Sat Mar 12 07:14:28 2005 Subject: [Python-Dev] Python2.4.1c1 and win32com In-Reply-To: References: Message-ID: <66e877b705031122147601a210@mail.gmail.com> On Sat, 12 Mar 2005 15:31:03 +1300, Tony Meyer wrote: > > Win32com generates Python-files for use with com interfaces, > > using the make-py.py utility. > > > > The generated files are OK with Python2.3.5 > > > > The generated files crash the Python interpreter with Python 2.4 > I believe this is a pywin32 bug, which has been fixed in (pywin32) CVS and > will be fixed for the next build. It's certainly a probably with 2.4.0 as > well as 2.4.1. > > The pywin32 mailing list archives have more details, as do the tracker for > the project: . Here is a link to the specific (closed) bug: http://sourceforge.net/tracker/?func=detail&aid=1085454&group_id=78018&atid=551954 And, in case you can't update to the CVS pywin32 for some reason, there is a work-around listed in the bug. Take care, -Brian From jcarlson at uci.edu Sat Mar 12 07:15:07 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat Mar 12 07:16:39 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: Message-ID: <20050311215619.A458.JCARLSON@uci.edu> Brian Sabbey wrote: > I would like to get some feedback on a proposal to introduce > smalltalk/ruby-like "code blocks" to python. Code blocks are, among other > things, a clean way to solve the "acquire/release" problem [1][2]. This > proposal also touches on some of the problems PEP 288 [3] deals with. [...] My first reaction to the proposal was "ick". Why? Because every time I've seen a mechanism for modifying the internals of generators constructed using yield, the syntax has been awful; in my opinion (whether my opinion means anything is another matter), the syntax you propose is quite awful, and the functionality you desire does not require syntax modification to be possible. Strawman 1: the syntax def pickled_file(name): f = open(name, 'r') l yield pickle.load(f) ^ ------ | f.close() | f = open(name, 'w') | pickle.dump(l, f) | f.close() | While this is currently a syntax error, it is not clear that an assignment is happening /at all/, and I don't believe it would be /even if/ if were documented, and every time someone opened up Python it said "This is an assignment!" with an example. It looks too magical to me (and from a guy who had previously proposed 'Some' as the opposite of 'None', that's saying something). Strawman 2: putting data back into a generator def pickled_file(name): f = open(name, 'r') yield pickle.load(f) f.close() f = open(name, 'w') pickle.dump(l, f) f.close() Keep your code, except toss that leading 'l'. for l in pickled_file('greetings.pickle'): l.append('hello') l.append('howdy') Toss that 'continue l', and your code will work (as long as both the function and the for are sitting in the same namespace). Hrm, not good enough? Use a Queue, or use another variable in a namespace accessable to both your function and your loop. Strawman 3: there is no strawman 3, but all lists need to have at least 3 items I'm sorry if this email comes off harsh, - Josiah From hugh at hodain.net Sat Mar 12 07:18:54 2005 From: hugh at hodain.net (Hugh Secker-Walker) Date: Sat Mar 12 07:17:10 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: <000301c525e4$ef687a20$13b12c81@oemcomputer> (python@rcn.com) References: <000301c525e4$ef687a20$13b12c81@oemcomputer> Message-ID: <200503120618.BAA29980@hub.hodain.net> Hello Folks, I've been lurking here for several months. I've been using Python for several years for prototyping fun. And I'm one of the architects of its extensive use in research, engineering development, and testing on a large, commercial speech-recognition system. I know a lot about modeling mammalian auditory physiology. And I have a relatively-large collection of photos of Tim Peters. > Raymond Hettinger: > Will leave it open for discussion for a bit so that folks can voice > any thoughts on the design. Ignoring the important naming issues, I have two comments about the any() and all() ideas: 1. I'm surprised that no one has suggested adding supporting unary methods to object, returning a boolean. If __any__ and __all__ existed, then containers that observed that their contents were immutable could rely on a single counter, self._numtrue, to implement the any and all operations in O(0) time via appropriate comparisons with __len__() (plus the amortized cost of maintaining self._numtrue). Of course builtin containers could or will do this anyway. 2. In an offline discussion about using any() and all() a couple of us noticed their similarity to the containment operator 'in'. Taking this idea as far as it can go in Python as we know it, you could introduce new keywords, 'any' and 'all', and corresponding byte codes dispatching to __any__ and __all__, giving the emminently readable usages: if any x: ... if not all y: ... This latter idea may be far-fetched. But if there's any chance it would happen, it adds a twist to the naming issue. Were this syntax introduced after any() and all() were available as builtins, simple use of 'any(blah)' would still work, but references to the no-longer-rare tokens 'any' and 'all', e.g. as functional objects, would, of course, no longer compile. -Hugh From bac at OCF.Berkeley.EDU Sat Mar 12 07:25:23 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Mar 12 07:25:30 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: Message-ID: <42328B53.8080806@ocf.berkeley.edu> Jim Jewett wrote: > Guido van Rossum: > > >>Whether to segregate these into a separate module: they are really a >>very small amount of syntactic sugat, and I expect that in most cases, >>instead of importing that module (which totally makes me lose my >>context while editing) I would probably just write the extra line that >>it takes to get the same effect with an explicit for loop and if >>statement. > > > Is that so bad? > > If you plan to use them often, then > > from itertools import any, every > > is reasonable. If you only use them once and weren't expecting it > (and want your imports at the top) ... well how awful is it to have > an extra line or two in your code? > Basically the question (at least in my mind) when it comes to built-ins is whether the usage is so basic and fundamental that sticking them in a module is done more for rigid organization than for convenience. Personally, I think any() and all() meet this requirement. With their ties to basic boolean testing they should be built into the language and not just a part of the stdlib. If I am banging something out at a interpreter prompt I will want these functions easily accessible and I won't think of them as something to import. This probably comes off as wishy-washy, but it is just what my mind is spitting out at the moment. They also have the feeling as something that could be useful as a syntax (although I am just suggesting syntactic support!). Which could be an even better way to measure whether something should be a built-in. Would the function be useful as a syntactic addition to the language, but just can't quite reach the need of a new keyword? Once again any() and all() feel like that to me. > These aren't *really* replacing map/filter/reduce because you're > adding the new functions now, but the old-function removal is > waiting until (forever?) > Even if Python 3000 comes out a while from now, why wait? Two more is not that much. Besides, it isn't like we are adding functions as some crazy rate. And Guido has stated that the 2.x branch stops once 2.9 (if it goes that long) comes out. So at worst you only have to worry about 5 more releases to worry. =) -Brett From martin at v.loewis.de Sat Mar 12 11:38:50 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 12 11:38:53 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <20050311052151.luqfnbc148gscgoc@mcherm.com> References: <20050311052151.luqfnbc148gscgoc@mcherm.com> Message-ID: <4232C6BA.9000908@v.loewis.de> Michael Chermside wrote: > I tried several stranger things, like installing over 2.4.0 but in a > different directory. Everything worked like clockwork. I did NOT try > anything that would have involved a system with various things missing > (like lack of VBScript), but I did play around with alternate install > locations, repairs, uninstalls, single-user and all-user installs, and > I found no problems anywhere. Nice work! Thanks! Somebody reported that it failed to update python24.dll in an update installation; not sure why this would be. Regards, Martin From python at rcn.com Sat Mar 12 13:17:36 2005 From: python at rcn.com (Raymond Hettinger) Date: Sat Mar 12 13:22:21 2005 Subject: [Python-Dev] sum() In-Reply-To: <1f7befae05031120232815581c@mail.gmail.com> Message-ID: <000b01c526fd$8666e9c0$3426a044@oemcomputer> > [Tim Peters] > Summer.add() _can_ lose info -- it needs additional > trickery to make it loss-free, and because x+y can lose (at most one > bit) when x and y have the same binary exponent. Computing an error term can get the bit back and putting that term back in the input queue restores the overall sum. Once the inputs are exhausted, the components of exp2sum should be exact. from math import frexp from itertools import chain def summer2(iterable): exp2sum = {} queue = [] for x in chain(iterable, queue): mant, exp = frexp(x) while exp in exp2sum: y = exp2sum.pop(exp) z = x + y d = x - z + y # error term assert z + d == x + y if d: queue.append(d) x = z mant, exp = frexp(x) exp2sum[exp] = x return sum(sorted(exp2sum.itervalues(), key=abs), 0.0) The implementation can be tweaked to consume the error term right away so the queue won't grow by more than few pending items. Also, the speed can be boosted by localizing frexp, exp2sum.pop, and queue.append. Raymond Hettinger From 2005a at usenet.alexanderweb.de Sat Mar 12 13:46:23 2005 From: 2005a at usenet.alexanderweb.de (Alexander Schremmer) Date: Sat Mar 12 13:50:51 2005 Subject: [Python-Dev] Re: RELEASED Python 2.4.1, release candidate 1 References: <20050311052151.luqfnbc148gscgoc@mcherm.com> <4232C6BA.9000908@v.loewis.de> Message-ID: <1rmdoxvs0mzws.dlg@usenet.alexanderweb.de> On Sat, 12 Mar 2005 11:38:50 +0100, "Martin v. L?wis" wrote: > Somebody reported that it failed to update python24.dll in > an update installation; not sure why this would be. Because it was in use? Kind regards, Alexander From martin at v.loewis.de Sat Mar 12 14:50:51 2005 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 12 14:50:54 2005 Subject: [Python-Dev] Re: RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <1rmdoxvs0mzws.dlg@usenet.alexanderweb.de> References: <20050311052151.luqfnbc148gscgoc@mcherm.com> <4232C6BA.9000908@v.loewis.de> <1rmdoxvs0mzws.dlg@usenet.alexanderweb.de> Message-ID: <4232F3BB.10702@v.loewis.de> Alexander Schremmer wrote: >>Somebody reported that it failed to update python24.dll in >>an update installation; not sure why this would be. > > > Because it was in use? Perhaps. In that case, Installer should schedule a reboot, and ask for the reboot when the installation is otherwise complete. Regards, Martin From martin at v.loewis.de Sat Mar 12 15:05:23 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 12 15:05:26 2005 Subject: [Python-Dev] Re: distutils fix for building Zope against Python 2.4.1c1 In-Reply-To: <1f7befae050311152210d8cd07@mail.gmail.com> References: <20050311230747.GA18655@ActiveState.com> <1f7befae050311152210d8cd07@mail.gmail.com> Message-ID: <4232F723.1050304@v.loewis.de> Tim Peters wrote: > Don't think it's needed on HEAD. As the checkin comment said: > > This doesn't appear to be an issue on the HEAD (MSVCCompiler initializes > itself via __init__() on the HEAD). > > I verified by building Zope with unaltered HEAD too, and had no > problem with that. Are you sure your HEAD is current? My copy of msvccompiler.py 1.67 reads def __init__ (self, verbose=0, dry_run=0, force=0): CCompiler.__init__ (self, verbose, dry_run, force) self.__version = get_build_version() if self.__version >= 7: self.__root = r"Software\Microsoft\VisualStudio" self.__macros = MacroExpander(self.__version) else: self.__root = r"Software\Microsoft\Devstudio" self.initialized = False def initialize(self): self.__paths = self.get_msvc_paths("path") ... Regards, Martin From martin at v.loewis.de Sat Mar 12 15:11:42 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 12 15:11:46 2005 Subject: [Python-Dev] Python2.4.1c1 and win32com In-Reply-To: References: Message-ID: <4232F89E.4040200@v.loewis.de> Leeuw van der, Tim wrote: > The generated files crash the Python interpreter with Python 2.4 > > Under Python 2.4.1c1, They give a syntax error!? Even though the bug was fixed in pywin, it is interesting to observe that Mark's analysis says "Cause of the underling crash is that the generated .py causes an overflow as the byte-code is generated - something in 2.4 bloated the byte-code for these modules." There also seems to be an interaction with PEP 263, for which patch 1101726 might provide a solution. So I think this needs to be investigated; please submit a bug report, including the Python file that causes the crash. Regards, Martin From martin at v.loewis.de Sat Mar 12 15:19:42 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 12 15:19:45 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 In-Reply-To: <87psy5znki.fsf@hydra.bayview.thirdcreek.com> References: <200503091148.50639.fdrake@acm.org> <200503100406.09997.anthony@interlink.com.au> <87psy5znki.fsf@hydra.bayview.thirdcreek.com> Message-ID: <4232FA7E.1030202@v.loewis.de> Kurt B. Kaiser wrote: > To eliminate this confusion I'd propose either > > 1. Eliminating the separate IDLE versioning now that it's installed with > Python when Tcl/Tk is available. > > or > > 2. Bringing its version in sync with Python 2.5. > > I'm in favor of the latter, to handle the situation where a customized > IDLE is decoupled from Python. I think there is a 3rd option: 3. Decouple Python and IDLE version numbers. I.e. IDLE version numbers should change only when the IDLE maintainer thinks they should, not when the Python release manager bumps the Python versions. For 2.4.1, the IDLE version number should just be rolled back (or forward) to "1.1.1". Unless you plan to include IDLE 1.1.2 with Python 2.4.2, it will stay at 1.1.1 in the 2.4 branch. Whether or not updating the IDLE code base to include new features is another issue - it might be that an exception can be made for IDLE; OTOH, it might be reasonable to apply the same strict requirements for idlelib as we do for the rest of the Python library, i.e. no new features. Regards, Martin From jhylton at gmail.com Sat Mar 12 16:53:06 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Sat Mar 12 16:59:49 2005 Subject: [Python-Dev] Re: RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <4232F3BB.10702@v.loewis.de> References: <20050311052151.luqfnbc148gscgoc@mcherm.com> <4232C6BA.9000908@v.loewis.de> <1rmdoxvs0mzws.dlg@usenet.alexanderweb.de> <4232F3BB.10702@v.loewis.de> Message-ID: I seem to have a problem with the install on XP SP1. Python itself is installed, but IDLE won't start. The error says: "IDLE's subprocess didn't make connection. Either IDLE can't start a subprocess or personal firewall software is blocking the connection." I believe the problem is the firewall, but I'm not sure if it is related to the install. The previous install (Python 2.3) worked fine. Jeremy From aahz at pythoncraft.com Sat Mar 12 17:09:38 2005 From: aahz at pythoncraft.com (Aahz) Date: Sat Mar 12 17:09:40 2005 Subject: [Python-Dev] Re: RELEASED Python 2.4.1, release candidate 1 In-Reply-To: References: <20050311052151.luqfnbc148gscgoc@mcherm.com> <4232C6BA.9000908@v.loewis.de> <1rmdoxvs0mzws.dlg@usenet.alexanderweb.de> <4232F3BB.10702@v.loewis.de> Message-ID: <20050312160938.GB2581@panix.com> On Sat, Mar 12, 2005, Jeremy Hylton wrote: > > I seem to have a problem with the install on XP SP1. Python itself is > installed, but IDLE won't start. The error says: "IDLE's subprocess > didn't make connection. Either IDLE can't start a subprocess or > personal firewall software is blocking the connection." I believe the > problem is the firewall, but I'm not sure if it is related to the > install. The previous install (Python 2.3) worked fine. http://www.python.org/2.4/bugs.html -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From martin at v.loewis.de Sat Mar 12 17:10:30 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 12 17:10:33 2005 Subject: [Python-Dev] Re: RELEASED Python 2.4.1, release candidate 1 In-Reply-To: References: <20050311052151.luqfnbc148gscgoc@mcherm.com> <4232C6BA.9000908@v.loewis.de> <1rmdoxvs0mzws.dlg@usenet.alexanderweb.de> <4232F3BB.10702@v.loewis.de> Message-ID: <42331476.5050808@v.loewis.de> Jeremy Hylton wrote: > I seem to have a problem with the install on XP SP1. Python itself is > installed, but IDLE won't start. The error says: "IDLE's subprocess > didn't make connection. Either IDLE can't start a subprocess or > personal firewall software is blocking the connection." I believe the > problem is the firewall, but I'm not sure if it is related to the > install. The previous install (Python 2.3) worked fine. What firewall software are you using? Any exceptions in the console when starting IDLE? Regards, Martin From anthony at interlink.com.au Sat Mar 12 17:14:37 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat Mar 12 17:15:28 2005 Subject: [Python-Dev] Open issues for 2.4.1 Message-ID: <200503130314.39322.anthony@interlink.com.au> So here's a list of open items I'm thinking about for the 2.4.1 release. - os.access handling unicode filenames I'm still thinking over whether this is going to cause more problems for people who find it works for some Python 2.4 and not others. I'm leaning towards saying that this is a bug fix, but I'd appreciate any comments (pro or con). If no-one comments, I'll go with whatever my gut feeling says is the right answer. - The unitest changes Changes to unitest to fix subclassing broke Zope's unittests. Should this change be reverted before 2.4.1, or was the Zope test suite doing something particularly dodgy? I'm talking about - unittest.TestCase.run() and unittest.TestSuite.run() can now be successfully extended or overridden by subclasses. Formerly, the subclassed method would be ignored by the rest of the module. (Bug #1078905). I've not looked too closely at the broken Zope code - can someone from ZC comment? How likely is it that other programs will also have been broken by this? At this point, I'm leaning (very slightly) towards the feeling that this should probably be backed out before 2.4.1, mostly because it seems to me that this is an incompatibility, rather than a pure bug fix. I'm now pretty sure we need a 2.4.1rc2 for this week, and a 2.4.1 final the week after. There's been a few too many changes for my tastes to say that going straight to a 2.4.1 final is a prudent course of action. -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Sat Mar 12 17:23:43 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Sat Mar 12 17:24:44 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 In-Reply-To: <4232FA7E.1030202@v.loewis.de> References: <87psy5znki.fsf@hydra.bayview.thirdcreek.com> <4232FA7E.1030202@v.loewis.de> Message-ID: <200503130323.44961.anthony@interlink.com.au> On Sunday 13 March 2005 01:19, "Martin v. L?wis" wrote: > Kurt B. Kaiser wrote: > > 1. Eliminating the separate IDLE versioning now that it's installed with > > Python when Tcl/Tk is available. > > 2. Bringing its version in sync with Python 2.5. > 3. Decouple Python and IDLE version numbers. I.e. IDLE version numbers > should change only when the IDLE maintainer thinks they should, not > when the Python release manager bumps the Python versions. This is only a brief note here - I have a longer post coming on the subject of uncoupled libraries in the Python core (triggered by Greg's recent Optik post), but I doing mind any of the options above. I'm wondering if we shouldn't have a centralised place in the CVS for these version numbers? > For 2.4.1, the IDLE version number should just be rolled back (or > forward) to "1.1.1". Unless you plan to include IDLE 1.1.2 with Python > 2.4.2, it will stay at 1.1.1 in the 2.4 branch. > > Whether or not updating the IDLE code base to include new features > is another issue - it might be that an exception can be made for IDLE; I would be very strongly against this. If the code is distributed as part of the stdlib, it should conform to the same rules as the rest of the stdlib. > OTOH, it might be reasonable to apply the same strict requirements > for idlelib as we do for the rest of the Python library, i.e. no new > features. Exactly right. Expecting users to understand that "no new features, except for this or that or the other package" is unreasonable, particularly when the delimiter is the (from a user's point of view) arbitrary distinction based on whether the source of the module is also separately available. See back to my earlier post of the subject of the rationale behind no-new-features - if we're going to keep to this, we should do it right. -- Anthony Baxter It's never too late to have a happy childhood. From p.f.moore at gmail.com Sat Mar 12 18:17:16 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Sat Mar 12 18:24:02 2005 Subject: [Python-Dev] Re: RELEASED Python 2.4.1, release candidate 1 In-Reply-To: References: <20050311052151.luqfnbc148gscgoc@mcherm.com> <4232C6BA.9000908@v.loewis.de> <1rmdoxvs0mzws.dlg@usenet.alexanderweb.de> <4232F3BB.10702@v.loewis.de> Message-ID: <79990c6b050312091715c23028@mail.gmail.com> On Sat, 12 Mar 2005 10:53:06 -0500, Jeremy Hylton wrote: > I seem to have a problem with the install on XP SP1. Python itself is > installed, but IDLE won't start. The error says: "IDLE's subprocess > didn't make connection. Either IDLE can't start a subprocess or > personal firewall software is blocking the connection." I believe the > problem is the firewall, but I'm not sure if it is related to the > install. The previous install (Python 2.3) worked fine. Did your firewall have an exception for Python 2.3, which doesn't apply to 2.4 (for example, it included the full path to the exempted executable)? Paul From tim.peters at gmail.com Sat Mar 12 19:06:29 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sat Mar 12 19:06:33 2005 Subject: [Python-Dev] sum() In-Reply-To: <000b01c526fd$8666e9c0$3426a044@oemcomputer> References: <1f7befae05031120232815581c@mail.gmail.com> <000b01c526fd$8666e9c0$3426a044@oemcomputer> Message-ID: <1f7befae0503121006710d76e1@mail.gmail.com> [Raymond Hettinger] > Computing an error term can get the bit back and putting that term back > in the input queue restores the overall sum. Right! > Once the inputs are exhausted, the components of exp2sum should be exact. Ditto. I'll cover some subtleties below: > from math import frexp > from itertools import chain > > def summer2(iterable): > exp2sum = {} > queue = [] > for x in chain(iterable, queue): > mant, exp = frexp(x) > while exp in exp2sum: > y = exp2sum.pop(exp) > z = x + y > d = x - z + y # error term > assert z + d == x + y Much more is true there, but hard to assert in Python: the mathematical sum z+d is exactly equal to the mathematical sum x+y there, and the floating-point |d| is less than 1 unit in the last place relative to the floating-point |z|. It would be clearer to name "z" as "hi" and "d" as "lo". More, that's not _generally_ true: it relies on that x and y have the same binary exponent. For example, pick x=1 and y=1e100, and you end up with hi=1e100 and lo=0. The mathematical x+y isn't equal to the mathematical hi+lo then. It's a real weakness of "Kahan summation" that most spellings of it don't bother to account for this; it's sufficient to normalize first, swapping x and y if necessary so that abs(x) >= abs(y) (although that isn't needed _here_ because we know they have the same binary exponent). There's another way to handle the general case that doesn't require test, branch, or abs(), but at the cost of several more addition/subtractions. > if d: > queue.append(d) If x and y have different signs, this part doesn't trigger. If all inputs are positive, then we expect it to trigger about half the time (the cases where exactly one of x's and y's least-significant bits is set). So the queue can be expected to grow to about half the length of the original iterable by the time the original iterable is exhausted. > x = z > mant, exp = frexp(x) > exp2sum[exp] = x > return sum(sorted(exp2sum.itervalues(), key=abs), 0.0) > > The implementation can be tweaked to consume the error term right away > so the queue won't grow by more than few pending items. Theorem 10 in Shewchuk's "Adaptive Precision Floating-Point Arithmetic and Fast Robust Geometric Predicates" gives the obvious correct way to do that, although as a practical matter it greatly benifits from a bit more logic to eliminate zero entries as they're produced (Shewchuk doesn't because it's not his goal to add a bunch of same-precision floats). BTW, extracting binary exponents isn't necessary in his approach (largely specializations to "perfect" 754 arithmetic of Priest's earlier less-demanding methods). > Also, the speed can be boosted by localizing frexp, exp2sum.pop, and > queue.append. I'm very glad you quit while it was still interesting . People who are paranoid about fp sums should use something like this. Even pairing is prone to disaster, given sufficiently nasty input. For example: >>> xs = [1, 1e100, 1, -1e100] * 10000 >>> sum(xs) # and the obvious pairing method gives the same 0.0 >>> sum(sorted(xs)) # the result is nearly incomprehensible -8.0076811737544552e+087 >>> sum(sorted(xs, reverse=True)) 8.0076811737544552e+087 >>> summer2(xs) # exactly right 20000.0 From tim.peters at gmail.com Sat Mar 12 19:26:21 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sat Mar 12 19:26:24 2005 Subject: [Python-Dev] Re: distutils fix for building Zope against Python 2.4.1c1 In-Reply-To: <4232F723.1050304@v.loewis.de> References: <20050311230747.GA18655@ActiveState.com> <1f7befae050311152210d8cd07@mail.gmail.com> <4232F723.1050304@v.loewis.de> Message-ID: <1f7befae05031210262461d069@mail.gmail.com> [Tim] >> Don't think it's needed on HEAD. As the checkin comment said: >> >> This doesn't appear to be an issue on the HEAD (MSVCCompiler initializes >> itself via __init__() on the HEAD). >> >> I verified by building Zope with unaltered HEAD too, and had no >> problem with that. [Martin] > Are you sure your HEAD is current? Of course I was sure. I was also wrong about that, but I resent the implication that I wasn't sure . I'll port the fix sometime this weekend. Thanks! From martin at v.loewis.de Sat Mar 12 20:11:27 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Mar 12 20:11:30 2005 Subject: [Python-Dev] Open issues for 2.4.1 In-Reply-To: <200503130314.39322.anthony@interlink.com.au> References: <200503130314.39322.anthony@interlink.com.au> Message-ID: <42333EDF.6020106@v.loewis.de> Anthony Baxter wrote: > I'm now pretty sure we need a 2.4.1rc2 for this week, and a 2.4.1 final the > week after. There's been a few too many changes for my tastes to say that > going straight to a 2.4.1 final is a prudent course of action. "The week after" will by the PyCon week, in which I won't be able to roll a release. The week after that, I'll have to give exams from Wednesday on, so I would prefer to produce my MSI file on March 29. Regards, Martin From tim.peters at gmail.com Sat Mar 12 20:31:57 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sat Mar 12 20:32:00 2005 Subject: [Python-Dev] Open issues for 2.4.1 In-Reply-To: <200503130314.39322.anthony@interlink.com.au> References: <200503130314.39322.anthony@interlink.com.au> Message-ID: <1f7befae050312113117ae4036@mail.gmail.com> [Anthony Baxter] > So here's a list of open items I'm thinking about for the 2.4.1 > release. > > [... sorry, but my editor automatically deletes all paragraphs > mentioning problems with Unicode ...] > - The unitest changes> Changes to unitest to fix subclassing broke Zope's > unittests. Should this change be reverted before 2.4.1, No. > or was the Zope test suite doing something particularly dodgy? Yes, it was overriding a documented, public method, but with no awareness that it was overriding anything -- a subclass just happened to define a method with the same name as an advertised unittest base class method. > I'm talking about > - unittest.TestCase.run() and unittest.TestSuite.run() can now be > successfully extended or overridden by subclasses. Formerly, > the subclassed method would be ignored by the rest of the module. > (Bug #1078905). Since there was no _intent_ in the zdaemon tests to override run(), it's just an accident that unittest used to ignore run() overrides. The fix in zdaemon source was s/run/_run/g. It shouldn't have used "run" to begin with. > I've not looked too closely at the broken Zope code - can someone from > ZC comment? Not officially . > How likely is it that other programs will also have been broken by this? Approximately equal to the odds that someone else defined a method named "run()" in a TestCase or TestSuite subclass without realizing they were actually overriding an advertised base class method (but one that didn't actually work as advertised, so there was no visible consequence before). ZC has a relatively huge number of test suites, and it should be noted that only the zdaemon suite was affected by this. Since the zdaemon tests don't run at all on Windows, I never noticed this; if I had, I would have changed the zdaemon test source as a matter of course (i.e., the thought "Python bug" wouldn't have crossed my mind in this case). > At this point, I'm leaning (very slightly) towards the feeling that this should > probably be backed out before 2.4.1, mostly because it seems to me that > this is an incompatibility, rather than a pure bug fix. It looked like pilot error on zdaemon's side. > I'm now pretty sure we need a 2.4.1rc2 for this week, and a 2.4.1 final the > week after. There's been a few too many changes for my tastes to say that > going straight to a 2.4.1 final is a prudent course of action. OK by me! From kbk at shore.net Sat Mar 12 21:35:21 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Mar 12 21:35:55 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 In-Reply-To: <4232FA7E.1030202@v.loewis.de> (Martin v. =?iso-8859-1?q?L=F6wis's?= message of "Sat, 12 Mar 2005 15:19:42 +0100") References: <200503091148.50639.fdrake@acm.org> <200503100406.09997.anthony@interlink.com.au> <87psy5znki.fsf@hydra.bayview.thirdcreek.com> <4232FA7E.1030202@v.loewis.de> Message-ID: <87ll8szjp2.fsf@hydra.bayview.thirdcreek.com> "Martin v. L?wis" writes: > Kurt B. Kaiser wrote: >> To eliminate this confusion I'd propose either 1. Eliminating the >> separate IDLE versioning now that it's installed with Python when >> Tcl/Tk is available. or 2. Bringing its version in sync with >> Python 2.5. I'm in favor of the latter, to handle the situation >> where a customized IDLE is decoupled from Python. > > I think there is a 3rd option: > > 3. Decouple Python and IDLE version numbers. I.e. IDLE version numbers > should change only when the IDLE maintainer thinks they should, not > when the Python release manager bumps the Python versions. > > For 2.4.1, the IDLE version number should just be rolled back (or > forward) to "1.1.1". Unless you plan to include IDLE 1.1.2 with Python > 2.4.2, it will stay at 1.1.1 in the 2.4 branch. Well, I had to read this several times to understand it :-) While your third approach makes some sense, I believe it's actually the source of the current confusion. IMHO it would be better to bump the micro number to agree with Python's, and have an empty entry in the IDLE NEWS.txt. We tried your suggestion with 2.3 and it got confusing. Part of the problem is, who bumps idlever.py and the header in NEWS.txt? Anthony and I have been stepping on each other a bit. My suggestion: IDLE maintainer starts the new NEWS.txt header if Python release mgr hasn't, but leaves it "What's New in IDLE" i.e. no release number and lets the release manager handle the release numbers. I still prefer making IDLE's version the same as Python's. > Whether or not updating the IDLE code base to include new features > is another issue - it might be that an exception can be made for IDLE; > OTOH, it might be reasonable to apply the same strict requirements > for idlelib as we do for the rest of the Python library, i.e. no new > features. I've always taken that approach: only bug fixes on the maintainance branches. -- KBK From sabbey at u.washington.edu Sat Mar 12 21:36:10 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Sat Mar 12 21:36:15 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <20050311215619.A458.JCARLSON@uci.edu> References: <20050311215619.A458.JCARLSON@uci.edu> Message-ID: On Fri, 11 Mar 2005, Josiah Carlson wrote: > My first reaction to the proposal was "ick". Why? Because every time > I've seen a mechanism for modifying the internals of generators > constructed using yield, the syntax has been awful; in my opinion > (whether my opinion means anything is another matter), the syntax you > propose is quite awful, I find it quite natural. Stuff on the right of 'yield' is going out, stuff on the left is coming in. Since 'yield' is different than return in that it marks a spot at which execution both leaves and re-enters the frame, it makes sense that 'yield' should have a syntax that indicates as much. > and the functionality you desire does not > require syntax modification to be possible. Was the functionality provided by generators or decorators or anything else impossible before those were introduced? Of course not. The point is to make things easier and more natural, not to enable some previously impossible functionality. > Strawman 1: the syntax > > def pickled_file(name): > f = open(name, 'r') > l yield pickle.load(f) > ^ > ------ > | f.close() > | f = open(name, 'w') > | pickle.dump(l, f) > | f.close() > | > While this is currently a syntax error, it is not clear that an > assignment is happening /at all/, and I don't believe it would be /even > if/ if were documented, and every time someone opened up Python it said > "This is an assignment!" with an example. It looks too magical to me > (and from a guy who had previously proposed 'Some' as the opposite of > 'None', that's saying something). Perhaps you are right, I don't know. It seems to me that most people would have to see the syntax once to know exactly what is going on, but I certainly don't know that for sure. Either way, I'd hate to have all my suggestions dismissed because of the syntax of this one piece. > Strawman 2: putting data back into a generator > > def pickled_file(name): > f = open(name, 'r') > yield pickle.load(f) > f.close() > f = open(name, 'w') > pickle.dump(l, f) > f.close() > > Keep your code, except toss that leading 'l'. > > for l in pickled_file('greetings.pickle'): > l.append('hello') > l.append('howdy') > > Toss that 'continue l', and your code will work (as long as both the > function and the for are sitting in the same namespace). But they're *not* in the same namespace necessarily. That is entirely the point. One is changing scope but has no clean way to pass values. How is making 'l' some (more) global variable possibly a clearer way to pass it to the generator? Your argument is like saying one does not need to return values from a function because we could always just use a global variable to do it. > Hrm, not good enough? Use a Queue, or use another variable in a > namespace accessable to both your function and your loop. Again, I entirely realize it's possible to do these things now, but that is not the point. > I'm sorry if this email comes off harsh, I'm just relieved someone responded :) -Brian From jcarlson at uci.edu Sat Mar 12 22:44:40 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat Mar 12 22:46:10 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: <20050311215619.A458.JCARLSON@uci.edu> Message-ID: <20050312125546.A45E.JCARLSON@uci.edu> Brian Sabbey wrote: > > On Fri, 11 Mar 2005, Josiah Carlson wrote: > > > My first reaction to the proposal was "ick". Why? Because every time > > I've seen a mechanism for modifying the internals of generators > > constructed using yield, the syntax has been awful; in my opinion > > (whether my opinion means anything is another matter), the syntax you > > propose is quite awful, > > I find it quite natural. Stuff on the right of 'yield' is going out, > stuff on the left is coming in. Since 'yield' is different than return in > that it marks a spot at which execution both leaves and re-enters the > frame, it makes sense that 'yield' should have a syntax that indicates as > much. Inventors always find their ideas to be quite natural; if they weren't, they wouldn't have come to them. > > and the functionality you desire does not > > require syntax modification to be possible. > > Was the functionality provided by generators or decorators or anything > else impossible before those were introduced? Of course not. The point > is to make things easier and more natural, not to enable some previously > impossible functionality. I don't find the syntax you provide to be 'natural'. Trying to convince me of the 'naturalness' of it is probably not going to be productive, but as I said in my original email, "whether my opinion means anything is another matter". Being that no one else has publically responded to your post, and it has been nearly 24 hours, my feeling is that either there are more important things going on (Python 2.4.1, sum/product/any/all discussion, etc.), people haven't been paying attention in recent days (due to the higher volume), people are not sufficiently one side or another to comment, or some other thing. > > Strawman 1: the syntax > > > > def pickled_file(name): > > f = open(name, 'r') > > l yield pickle.load(f) > > ^ > > ------ > > | f.close() > > | f = open(name, 'w') > > | pickle.dump(l, f) > > | f.close() > > | > > While this is currently a syntax error, it is not clear that an > > assignment is happening /at all/, and I don't believe it would be /even > > if/ if were documented, and every time someone opened up Python it said > > "This is an assignment!" with an example. It looks too magical to me > > (and from a guy who had previously proposed 'Some' as the opposite of > > 'None', that's saying something). > > Perhaps you are right, I don't know. It seems to me that most people > would have to see the syntax once to know exactly what is going on, but I > certainly don't know that for sure. Either way, I'd hate to have all my > suggestions dismissed because of the syntax of this one piece. I say it is magical. Why? The way you propose it, 'yield' becomes an infix operator, with the name provided on the left being assigned a value produced by something that isn't explicitly called, referenced, or otherwise, by the right. In fact, I would say, it would be akin to the calling code modifying gen.gi_frame._f_locals directly. Such "action at a distance", from what I understand, is wholly frowned upon in Python. There also does not exist any other infix operator that does such a thing (=, +=, ... assign values, but where the data comes from is obvious). Bad things to me (so far): 1. Assignment is not obvious. 2. Where data comes from is not obvious. 3. Action at a distance like nothing else in Python. 4. No non-'=' operator assigns to a local namespace. > > Strawman 2: putting data back into a generator > > > > def pickled_file(name): > > f = open(name, 'r') > > yield pickle.load(f) > > f.close() > > f = open(name, 'w') > > pickle.dump(l, f) > > f.close() > > > > Keep your code, except toss that leading 'l'. > > > > for l in pickled_file('greetings.pickle'): > > l.append('hello') > > l.append('howdy') > > > > Toss that 'continue l', and your code will work (as long as both the > > function and the for are sitting in the same namespace). > > But they're *not* in the same namespace necessarily. That is entirely the > point. One is changing scope but has no clean way to pass values. How is > making 'l' some (more) global variable possibly a clearer way to pass it > to the generator? Your argument is like saying one does not need to > return values from a function because we could always just use a global > variable to do it. Or you could even use a class instance. class foo: def pickled_file(self, name): f = open(name, 'r') yield pickle.load(f) f.close() f = open(name, 'w') pickle.dump(self.l, f) f.close() fi = foo() for l in fi.pickled_file('greetings.pickle'): l.append('hello') l.append('howdy') fi.l = l If you can call the function, there is always a shared namespace. It may not be obvious (you may need to place the function as a method of an instance), but it is still there. > > Hrm, not good enough? Use a Queue, or use another variable in a > > namespace accessable to both your function and your loop. > > Again, I entirely realize it's possible to do these things now, but that > is not the point. The point of your proposed syntax is to inject data back into a generator from an external namespace. Right? My point is that if you are writing software, there are already fairly reasonable methods to insert data back into a generator from an external namespace. Furthermore, I would say that using yield semantics in the way that you are proposing, while being discussed in other locations (the IBM article on cooperative multithreading via generators springs to mind), is a clever hack, but not something that should be supported via syntax. Considering the magical nature of how you propose to change yield, I can't see any serious developer of the Python language (those with commit access to Python CVS) saying anything other than "-1". Coupled with the fact that I cannot recall Guido or anyone else here ever having said "it would be nice if we could put stuff back into generators", my feeling is that your syntax proposal is not going to make it (I could be wrong). - Josiah From nidoizo at yahoo.com Sat Mar 12 22:58:29 2005 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Sat Mar 12 22:56:28 2005 Subject: [Python-Dev] Re: Adding any() and all() In-Reply-To: References: Message-ID: Reinhold Birkenfeld wrote: > """ > He proposes that > > [x in list if x > 0] > > (which is currently undefined) be exactly equal to: > > [x for x in list if x > 0] > """ > > What about that? But [x in list] means already something. Supporting [x in list if condition] will complicate the parser and will be error-prone if someone remove the condition. It will make more sense to me to support that syntax instead (unless I miss something): [for x in list if x > 0] Regards, Nicolas From steven.bethard at gmail.com Sat Mar 12 23:15:11 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat Mar 12 23:15:16 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: Message-ID: Brian Sabbey wrote: > (I) Give generators a __call__ method as an alternative to 'next'. > Method __call__ should take a single parameter: a function. Using > __call__ will cause the generator to start executing normally, but when a > 'yield' is reached, the generator will invoke the function passed to > __call__ instead of activating the currently implemented 'yield' > mechanism. The goals behind this seem a lot like the goals of PEP 288[1]. I remember discussions suggesting code like: def gen(): a, b, c=3 = yield 1 yield a + b*c g = gen() print g.next() # prints 1 print g.next(1, 2) # prints 7 But as you can see, this creates a weird asymmetry because the last yield throws away its arguments, and depending on how the generator is written, different calls to next may require a different number of arguments. This means that, unless the code is extremely well documented, you have to read the source code for the generator to know how to call it. Because of these and other complications, I believe the PEP is now lobbying for a way to get the generator instance object and a way to cause an exception to be thrown from outside the generator. Take a look and see if the PEP might meet your needs -- I haven't seen much action on it recently, but it seems much less invasive than your proposal... > As an example of the syntax I am suggesting, here is something I was > desiring recently, a generator to open, unpickle, repickle and close a > file: > > def pickled_file(name): > f = open(name, 'r') > l yield pickle.load(f) > f.close() > f = open(name, 'w') > pickle.dump(l, f) > f.close() I believe with PEP 288, this would look something like: def pickled_file(name): self = mygen.get_instance() f = open(name, 'r') yield pickle.load(f) f.close() f = open(name, 'w') pickle.dump(self.l, f) f.close() > This function would be used like this: > > for l in pickled_file('greetings.pickle'): > l.append('hello') > l.append('howdy') > continue l > And this would be written something like: gen = pickled_file('greetings.pickle') for l in gen: l.append('hello') l.append('howdy') gen.l = l Personally, I find this use of a generator thoroughly confusing, and I don't see what you gain from it. The PEP 288 examples are perhaps somewhat more convincing though... Steve [1] http://www.python.org/peps/pep-0288.html -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From python at rcn.com Sat Mar 12 23:32:39 2005 From: python at rcn.com (Raymond Hettinger) Date: Sat Mar 12 23:37:03 2005 Subject: [Python-Dev] sum() In-Reply-To: <1f7befae0503121006710d76e1@mail.gmail.com> Message-ID: <000401c52753$665459a0$ccb3958d@oemcomputer> > the queue can be expected to grow to about half the length > of the original iterable by the time the original iterable is > exhausted. > > > x = z > > mant, exp = frexp(x) > > exp2sum[exp] = x > > return sum(sorted(exp2sum.itervalues(), key=abs), 0.0) > > > > The implementation can be tweaked to consume the error term right away > > so the queue won't grow by more than few pending items. > > Theorem 10 in Shewchuk's "Adaptive Precision Floating-Point Arithmetic > and Fast Robust Geometric Predicates" gives the obvious correct > way to do that, although as a practical matter it greatly benifits > from a bit more logic to eliminate zero entries as they're produced > (Shewchuk doesn't because it's not his goal to add a bunch of > same-precision floats). BTW, extracting binary exponents isn't > necessary in his approach (largely specializations to "perfect" 754 > arithmetic of Priest's earlier less-demanding methods). The approach I'm using relies on being able to exactly multiply the 0 or 1 bit error term mantissas by an integer (a count of how many times the term occurs). With a Python dictionary keeping the counts, many steps are saved and the tool becomes much more memory friendly than with the previous queue approach. from math import frexp def summer3(iterable): exp2sum = {} # map to a value with a given exponent errdict = {} # map error terms to an occurrence count def addone(x, exppop=exp2sum.pop, errget=errdict.get, frexp=frexp): mant, exp = frexp(x) # extract exponent from float representation while exp in exp2sum: y = exppop(exp) # pair with a value having the same exp z = x + y # sum may be inexact by less than 1 ulp of z d = x - z + y # difference is the error term assert z + d == x + y # sum plus error term is exact! errdict[d] = errget(d, 0) + 1 # count occurrences of each term x = z # continue with new sum mant, exp = frexp(x) exp2sum[exp] = x for x in iterable: addone(x) while errdict: d, q = errdict.popitem() assert frexp(d)[0] in [-0.5, 0.0, 0.5] # error terms are 1 bit # product of 1 bit error term with an integer count is exact addone(d * q) dummy = errdict.pop(0.0, None) # At this point, the errdict is empty, all partial sums are exact, # and each partial sum has a different exponent. They can now be # added smallest absolute value to largest and only lose bits that # cannot fit in the final result. IOW, the final result is accurate # to full precision (assuming no representation error in the inputs). return sum(sorted(exp2sum.itervalues(), key=abs), 0.0) Aside from being a nice recipe, this was a good exercise in seeing what pure Python can do with IEEE-754 guarantees. The modest memory consumption and the typical O(n) runtime are a nice plus (no pun intended). Raymond From sabbey at u.washington.edu Sun Mar 13 00:54:06 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Sun Mar 13 00:54:11 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: Message-ID: On Sat, 12 Mar 2005, Steven Bethard wrote: > The goals behind this seem a lot like the goals of PEP 288[1]. I > remember discussions suggesting code like: > > def gen(): > a, b, c=3 = yield 1 > yield a + b*c > > g = gen() > print g.next() # prints 1 > print g.next(1, 2) # prints 7 > > But as you can see, this creates a weird asymmetry because the last > yield throws away its arguments, and depending on how the generator is > written, different calls to next may require a different number of > arguments. This means that, unless the code is extremely well > documented, you have to read the source code for the generator to know > how to call it. The intention of my proposal was for using generators with 'for' loops. In this case, the generator runs to completion, so the arguments to the last yield are never thrown away. If 'next' were not able to take any arguments, that would be compatible with my proposal. Also, there was the issue that there is an asymmetry because the first call to 'next' does not take any arguments. This asymmetry does not exist, however, when using the generator in a 'for' loop, because there is no "first" call to 'continue' in such a case. > Because of these and other complications, I believe the PEP is now > lobbying for a way to get the generator instance object and a way to > cause an exception to be thrown from outside the generator. Take a > look and see if the PEP might meet your needs -- I haven't seen much > action on it recently, but it seems much less invasive than your > proposal... This PEP solves similar problems, yes. And I would agree that my proposal is much more invasive on python's implementation. From the users' point of view, however, I think it is much less invasive. For example, no doubt there will be many users who write a generator that is to be used in a 'for' loop and are baffled that they receive a syntax error when they try to write some try/finally cleanup code. With the PEP, they would have to figure out that they have to use the 'throw' method of generators to trigger cleanup code (and then have to remember to call it each time they are done with the generator). With this proposal, try/finally would just work as they expect and they would be non the wiser. > def pickled_file(name): > self = mygen.get_instance() > f = open(name, 'r') > yield pickle.load(f) > f.close() > f = open(name, 'w') > pickle.dump(self.l, f) > f.close() > > And this would be written something like: > > gen = pickled_file('greetings.pickle') > for l in gen: > l.append('hello') > l.append('howdy') > gen.l = l > > Personally, I find this use of a generator thoroughly confusing, and I > don't see what you gain from it. The PEP 288 examples are perhaps > somewhat more convincing though... The disadvantage of doing it this way (or with a class wrapping the generator) is that it is implicit. If I were reading the pickled_file code, I would have no idea where the self.l comes from. If it is coming from the 'for' loop, why not just be able to explicitly say that? I agree that this is a confusing way to use generators. But it is the expected way to use "code blocks" as found in other languages. It would take some getting used to that 'for' can be used this way, but I think it would be worth it. -Brian From sabbey at u.washington.edu Sun Mar 13 00:54:10 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Sun Mar 13 00:54:16 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <20050312125546.A45E.JCARLSON@uci.edu> References: <20050311215619.A458.JCARLSON@uci.edu> <20050312125546.A45E.JCARLSON@uci.edu> Message-ID: On Sat, 12 Mar 2005, Josiah Carlson wrote: > I say it is magical. Why? The way you propose it, 'yield' becomes an > infix operator, with the name provided on the left being assigned a > value produced by something that isn't explicitly called, referenced, or > otherwise, by the right. In fact, I would say, it would be akin to the > calling code modifying gen.gi_frame._f_locals directly. Such "action at > a distance", from what I understand, is wholly frowned upon in Python. > There also does not exist any other infix operator that does such a > thing (=, +=, ... assign values, but where the data comes from is > obvious). > > Bad things to me (so far): > 1. Assignment is not obvious. > 2. Where data comes from is not obvious. > 3. Action at a distance like nothing else in Python. > 4. No non-'=' operator assigns to a local namespace. If it is too much action at a distance, one could also use the syntax "a = yield b", as has been suggested before, but I preferred it without the '='. With the '=', I don't see how it is any more action at a distance than "a = yield_func(b)" or "for i in f()". In all of these cases, values are being passed when the scope is changing. If you want to argue that both of these syntaxes are too ugly to be allowed, then ok. > Or you could even use a class instance. > > class foo: > def pickled_file(self, name): > f = open(name, 'r') > yield pickle.load(f) > f.close() > f = open(name, 'w') > pickle.dump(self.l, f) > f.close() > > fi = foo() > for l in fi.pickled_file('greetings.pickle'): > l.append('hello') > l.append('howdy') > fi.l = l > > If you can call the function, there is always a shared namespace. It > may not be obvious (you may need to place the function as a method of an > instance), but it is still there. The disadvantage of this method is that it is not clear where the self.l values is coming from just by reading the generator method definition. If it is coming from the code block, why not be able to explicity write it that way? Similarly, "continue l" explicitly shows that you are passing a value back to the generator. >>> Hrm, not good enough? Use a Queue, or use another variable in a >>> namespace accessable to both your function and your loop. >> >> Again, I entirely realize it's possible to do these things now, but that >> is not the point. > > The point of your proposed syntax is to inject data back into a > generator from an external namespace. Right? The point is to inject data back into a generator in a clean, explicit way; I understand there are other ways in which one can already do this. > Furthermore, I would say that using yield semantics in the way that you > are proposing, while being discussed in other locations (the IBM article > on cooperative multithreading via generators springs to mind), is a > clever hack, but not something that should be supported via syntax. My impression is that being able to use yield in this way is one of the primary reasons for the success of ruby. Therefore, I do not see it as just a clever hack. It is a clean, explicity way to pass values when changing scope. -Brian From david.ascher at gmail.com Sun Mar 13 01:29:12 2005 From: david.ascher at gmail.com (David Ascher) Date: Sun Mar 13 01:29:25 2005 Subject: [Python-Dev] sum() In-Reply-To: <000401c52753$665459a0$ccb3958d@oemcomputer> References: <1f7befae0503121006710d76e1@mail.gmail.com> <000401c52753$665459a0$ccb3958d@oemcomputer> Message-ID: You guys have way too much time on your hands and neurons devoted to this stuff. I'm glad that means that I can spend the afternoon playing w/ my kids and searching python-dev when I need to add numbers =). From kbk at shore.net Sun Mar 13 02:01:25 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Sun Mar 13 02:01:32 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <4230FAA5.4070309@v.loewis.de> (Martin v. =?iso-8859-1?q?L=F6wis's?= message of "Fri, 11 Mar 2005 02:55:49 +0100") References: <200503110138.02569.anthony@python.org> <4230FAA5.4070309@v.loewis.de> Message-ID: <87is3wxst6.fsf@hydra.bayview.thirdcreek.com> "Martin v. L?wis" writes: > I'd like to encourage feedback on whether the Windows installer works > for people. It replaces the VBScript part in the MSI package with native > code, which ought to drop the dependency on VBScript, but might > introduce new incompatibilities. I had some strange experiences. Windows2000: I downloaded the 2.4.1c1 installer to the desktop and clicked on it. It complained that it couldn't access the installer. I then clicked on the 2.4.1b2 installer and that started ok. Cancelled it. Repeated this sequence a second time. Same. The third time I clicked on 2.4.1c1 it started up and installed Python ok. For ducks I installed Python at D:\"Python 2.4". It installed in the correct location, also found C:\Python24 and uninstalled that. Python and IDLE appear to run ok. Shortcuts on Run work, and file association was updated, so right clicking a .py used IDLE from Python 2.4.1c1, in no-subprocess mode. -- KBK From steven.bethard at gmail.com Sun Mar 13 02:01:50 2005 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun Mar 13 02:09:07 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: Message-ID: Brian Sabbey wrote: > I agree that this is a confusing way to use generators. But it is the > expected way to use "code blocks" as found in other languages. It would > take some getting used to that 'for' can be used this way, but I think it > would be worth it. I guess I need some more convincing... I don't find your approach[*], e.g. def pickled_file(name): f = open(name, 'r') data yield pickle.load(f) f.close() f = open(name, 'w') pickle.dump(data, f) f.close() for data in pickled_file('greetings.pickle'): data.append('hello') data.append('howdy') continue data any clearer than, say: class PickledFile(object): def __init__(self, name): self.name = name f = open(self.name, 'r') self.data = pickle.load(f) f.close() def close(self): f = open(self.name, 'w') pickle.dump(self.data, f) f.close() p = PickledFile('greetings.pickle') p.data.extend(['hello', 'howdy']) p.close() Note that I'm not using the iteration construct (for-in) because I'm not really doing an iteration here. Pehaps I could be taught to read for-in otherwise, but without an obvious benefit for doing so, I'm really not inclined to. Steve [*] I've renamed your "l" to "data". The first time I read your post, it looked even more confusing to me because "l" (lower case L) is rendered too similarly to "|" (or-bar) in my browser. -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy From jcarlson at uci.edu Sun Mar 13 02:26:52 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun Mar 13 02:28:17 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: <20050312125546.A45E.JCARLSON@uci.edu> Message-ID: <20050312161509.A464.JCARLSON@uci.edu> Brian Sabbey wrote: > > On Sat, 12 Mar 2005, Josiah Carlson wrote: > > > I say it is magical. Why? The way you propose it, 'yield' becomes an > > infix operator, with the name provided on the left being assigned a > > value produced by something that isn't explicitly called, referenced, or > > otherwise, by the right. In fact, I would say, it would be akin to the > > calling code modifying gen.gi_frame._f_locals directly. Such "action at > > a distance", from what I understand, is wholly frowned upon in Python. > > There also does not exist any other infix operator that does such a > > thing (=, +=, ... assign values, but where the data comes from is > > obvious). > > > > Bad things to me (so far): > > 1. Assignment is not obvious. > > 2. Where data comes from is not obvious. > > 3. Action at a distance like nothing else in Python. > > 4. No non-'=' operator assigns to a local namespace. > > If it is too much action at a distance, one could also use the syntax "a = > yield b", as has been suggested before, but I preferred it without the > '='. With the '=', I don't see how it is any more action at a distance > than "a = yield_func(b)" or "for i in f()". In all of these cases, values > are being passed when the scope is changing. Right, but as I said before, when using any other assignment operator, where the value comes from is obvious: the call or statement offered to the right of the operator. In your example, the data comes from the caller, and even if you were to offer documentation, it is not obvious who or what the caller is, or whether or not they would actually provide such a value (what would happen if they didn't?). Essentially what you are wanting to do is to take what would normally be multiple function calls and place them in a generator to package up setup, body, finalization. l = load_pickle(filename) #setup l.append(...) #body write_pickle(filename, l) #finalization The above is /the/ canonical way to do this in basically every programming language. Sticking it in a generator for a sense of being 'clean' is preposterous. > If you want to argue that both of these syntaxes are too ugly to be > allowed, then ok. I stand by my original "ick". > > Or you could even use a class instance. > > > > class foo: > > def pickled_file(self, name): > > f = open(name, 'r') > > yield pickle.load(f) > > f.close() > > f = open(name, 'w') > > pickle.dump(self.l, f) > > f.close() > > > > fi = foo() > > for l in fi.pickled_file('greetings.pickle'): > > l.append('hello') > > l.append('howdy') > > fi.l = l > > > > If you can call the function, there is always a shared namespace. It > > may not be obvious (you may need to place the function as a method of an > > instance), but it is still there. > > The disadvantage of this method is that it is not clear where the self.l > values is coming from just by reading the generator method definition. If > it is coming from the code block, why not be able to explicity write it > that way? Similarly, "continue l" explicitly shows that you are passing a > value back to the generator. External to pickled_file, it is explicit where your 'l' or 'self.l' thing is coming from. You hit the nail on the head, the problem with /both/ your syntax and the equivalent that I offer above is that inside pickled_file, it is not obvious where self.l (or equivalently the 'l' in "l yield...") is coming from. Why? Because pulling the body out of a setup/body/finalization series messes that up. You would be better off either calling 3 functions like I provided above (saving people the confusion of "what the hell is this for loop doing here?"), or putting everything in one function/method, with a callback like... def pickled_file(filename, callback): #setup... callback(args) #body #finalization > > The point of your proposed syntax is to inject data back into a > > generator from an external namespace. Right? > > The point is to inject data back into a generator in a clean, explicit > way; I understand there are other ways in which one can already do this. But it is neither explicit nor clean. Explicit from the outside, perhaps, but not inside your generator. And it certainly is not clean because it pisses all over the history of "don't to anything too magical" that Python seems to have strived for. What if I were to tell you that "a and b" would start assigning a value to "a" that wasn't "b" or "a", wasn't anything even related to "a" or "b", and would be produced by code that isn't even referenced by "a", "b", or anything else in the local namespace? You'd probably tell me that I'd had a few too many drinks, and you'd probably be right. You are essentially asking for "yield" to behave like this, and I think you've had a few too many drinks *wink*. Steven has pointed out PEP 288, and in my opinion, PEP 288 is the right thing to do in regards to passing exceptions to generators. Use the 'throw' method throw an exception, perfect. It solves a long-standing problem, and if /you/ want to use it to pass values, you can! class Data(Exception): pass def foo(filename): ... try: yield ... except Data, data: l = data ... Assuming PEP 288 were in. Yes, the above smells of a hack, but it is obvious that the value assigned to 'l' is coming from an exception that needs to be triggered by either the statement on the right side of the yield, or offered via the gen.throw() method. I personally would not condone such use of gen.throw(), but I also don't condone the use of "eval" and "exec", and that hasn't stopped thousands from doing so. > > Furthermore, I would say that using yield semantics in the way that you > > are proposing, while being discussed in other locations (the IBM article > > on cooperative multithreading via generators springs to mind), is a > > clever hack, but not something that should be supported via syntax. > > My impression is that being able to use yield in this way is one of the > primary reasons for the success of ruby. Therefore, I do not see it as > just a clever hack. It is a clean, explicity way to pass values when > changing scope. Ah, but Ruby is not Python, Ruby is Ruby. Python only recently got generators; before then we saved state explicitly (thank you Guido and everyone else who made generators happen). As such, while it may be "old hat" to Ruby, it is relatively new to Python. I stand by my "clever hack" statement, and I will agree to disagree with you on it, like I agree to disagree with you about both the necessity of passing arbitrary values back into a generator (I think it is poor style, and too much brain overhead to wonder where data is coming from), and the necessity of a new syntax to make such things "easier" (if (ab)using generators in such a way makes /anything/ "easier"). - Josiah From tim.peters at gmail.com Sun Mar 13 02:30:03 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sun Mar 13 02:30:06 2005 Subject: [Python-Dev] sum() In-Reply-To: <000401c52753$665459a0$ccb3958d@oemcomputer> References: <1f7befae0503121006710d76e1@mail.gmail.com> <000401c52753$665459a0$ccb3958d@oemcomputer> Message-ID: <1f7befae0503121730568b118f@mail.gmail.com> [Raymond Hettinger] > The approach I'm using relies on being able to exactly multiply the 0 or > 1 bit error term mantissas by an integer (a count of how many times the > term occurs). With a Python dictionary keeping the counts, many steps > are saved and the tool becomes much more memory friendly than with the > previous queue approach. Cool! That's a nice approach. For contrast, here's one that doesn't use frexp(), and is probably faster because of that; internally, len(sums) probably won't exceed 5 in real life (but could get as large as 2K for pathological inputs, spanning fp's full dynamic range): def summer4(iterable): sums = [0.0] for x in iterable: sums.append(x) for i in xrange(len(sums)-2, -1, -1): y = sums[i] if abs(x) < abs(y): x, y = y, x hi = x+y lo = y - (hi - x) if lo: sums[i+1] = lo else: del sums[i+1] x = hi sums[0] = x return sum(reversed(sums), 0.0) In effect, that keeps an arbitrarily long list of "error terms" so that no info is lost before the final sum(). I think it's surprising at first how short that list usually is. > ... > Aside from being a nice recipe, this was a good exercise in seeing what > pure Python can do with IEEE-754 guarantees. Now you know I how feel everytime sometime proclaims that fp arithmetic is inherently unreliable <0.3 wink>. Learning how to use it correctly is indeed the blackest of obscure arts, though. > The modest memory consumption and the typical O(n) runtime are a nice > plus (no pun intended). Yup, they're all emminently practical if you don't care too much about speed. The one above is the fastest I tried, and it still takes some 40x longer than plain sum() over a million-element random.random() list. From sabbey at u.washington.edu Sun Mar 13 02:43:23 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Sun Mar 13 02:43:27 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: Message-ID: On Sat, 12 Mar 2005, Steven Bethard wrote: > Brian Sabbey wrote: >> I agree that this is a confusing way to use generators. But it is the >> expected way to use "code blocks" as found in other languages. It would >> take some getting used to that 'for' can be used this way, but I think it >> would be worth it. > > I guess I need some more convincing... I don't find your approach[*], e.g. > > def pickled_file(name): > f = open(name, 'r') > data yield pickle.load(f) > f.close() > f = open(name, 'w') > pickle.dump(data, f) > f.close() > > for data in pickled_file('greetings.pickle'): > data.append('hello') > data.append('howdy') > continue data > > any clearer than, say: > > class PickledFile(object): > def __init__(self, name): > self.name = name > f = open(self.name, 'r') > self.data = pickle.load(f) > f.close() > def close(self): > f = open(self.name, 'w') > pickle.dump(self.data, f) > f.close() > > p = PickledFile('greetings.pickle') > p.data.extend(['hello', 'howdy']) > p.close() > > Note that I'm not using the iteration construct (for-in) because I'm > not really doing an iteration here. Pehaps I could be taught to read > for-in otherwise, but without an obvious benefit for doing so, I'm > really not inclined to. In the real world, I need to lock and unlock the pickled files. So if I forget to call p.close(), then it would be a problem. I would need a try/finally block. If I could use the above 'for' loop approach, I am able to encapsulate the try/finally code. This is the same problem that is addressed in PEP 310. Copied from the PEP, here is the basic idea: The syntax of a 'with' statement is as follows:: 'with' [ var '=' ] expr ':' suite This statement is defined as being equivalent to the following sequence of statements: var = expr if hasattr(var, "__enter__"): var.__enter__() try: suite finally: var.__exit__() I prefer re-using the 'for' loop for this purpose because it allows the problem to be solved in a general way by re-using a structure with which most users are already familiar, and uses generators, which are easier to use in this case than defining a class with __exit__, __enter__, etc. -Brian From sabbey at u.washington.edu Sun Mar 13 03:00:00 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Sun Mar 13 03:00:04 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <20050312161509.A464.JCARLSON@uci.edu> References: <20050312125546.A45E.JCARLSON@uci.edu> <20050312161509.A464.JCARLSON@uci.edu> Message-ID: On Sat, 12 Mar 2005, Josiah Carlson wrote: > I stand by my "clever hack" statement, and I will agree to disagree with > you on it, like I agree to disagree with you about both the necessity of > passing arbitrary values back into a generator (I think it is poor > style, and too much brain overhead to wonder where data is coming from), > and the necessity of a new syntax to make such things "easier" (if > (ab)using generators in such a way makes /anything/ "easier"). I will also agree to disagree. If I may also summarize, you see such syntax as magical, non-obvious and unnecessary, and I find it clean and useful. -Brian From jcarlson at uci.edu Sun Mar 13 05:31:40 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun Mar 13 05:33:22 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: <20050312161509.A464.JCARLSON@uci.edu> Message-ID: <20050312201535.A467.JCARLSON@uci.edu> Brian Sabbey wrote: > On Sat, 12 Mar 2005, Josiah Carlson wrote: > > > I stand by my "clever hack" statement, and I will agree to disagree with > > you on it, like I agree to disagree with you about both the necessity of > > passing arbitrary values back into a generator (I think it is poor > > style, and too much brain overhead to wonder where data is coming from), > > and the necessity of a new syntax to make such things "easier" (if > > (ab)using generators in such a way makes /anything/ "easier"). > > I will also agree to disagree. If I may also summarize, you see such > syntax as magical, non-obvious and unnecessary, and I find it clean and > useful. Small correction: I find the syntax magical, the data flow to be non-obvious, and the functionality unnecessary (for the reasons previously specified). - Josiah From greg.ewing at canterbury.ac.nz Sun Mar 13 07:45:42 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun Mar 13 07:54:17 2005 Subject: [Python-Dev] Adding any() and all() References: <000701c525e9$a88dcc40$bf20c797@oemcomputer> <20050311055322.GA20294@panix.com> <740c3aec05031102301697d845@mail.gmail.com> <79990c6b0503110318cf36d1c@mail.gmail.com> <4231AE7E.5060406@iinet.net.au> Message-ID: <4233E196.3050905@canterbury.ac.nz> Nick Coghlan wrote: > A suggestion was made on c.l.p a while back to have a specific module > dedicated to reductive operations. That is, just as itertools is > oriented towards manipulating iterables and creating iterators, this > module would be oriented towards consuming iterators in a reductive fashion. Is there really any need for another module just for this? The distinction between reductive and non-reductive operations on iterators seems rather too fine to me to deserve a whole new module. Why not just put them all in itertools? > [1] While any()/all() read well in the context of an if statement, I > think anytrue()/alltrue() better convey the reductive nature of the > operations, read nearly as well in the if context, and read > significantly better when isolated from the if context (e.g. assigned to > a variable). I don't agree. I think 'any' and 'all' are fine names for boolean-valued functions in any context. Including 'true' in their names smacks of the same kind of redundancy as if blarg == True: ... which is widely regarded as a naive-newbie style blunder in any language. +1 on 'any' and 'all' -1 on any names including 'true' or 'false' Greg From greg.ewing at canterbury.ac.nz Sun Mar 13 07:51:43 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun Mar 13 08:00:18 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators References: Message-ID: <4233E2FF.5090507@canterbury.ac.nz> Brian Sabbey wrote: > I prefer re-using the 'for' loop for this purpose because it allows the > problem to be solved in a general way by re-using a structure with which > most users are already familiar, and uses generators, which are easier > to use in this case than defining a class with __exit__, __enter__, etc. But this is an abuse of both the for-loop and the generator. It's using a for-loop for something that does not loop and is never intended to loop, and it's using yield to do something other than producing a value for consumption. I'd rather see a new mechanism altogether for this very different use case. > -Brian -- Greg From greg.ewing at canterbury.ac.nz Sun Mar 13 07:57:58 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun Mar 13 08:06:33 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) References: <42323B44.5080201@iinet.net.au> Message-ID: <4233E476.9030705@canterbury.ac.nz> Nick Coghlan wrote: > That 'x in seq' bit still shouts "containment" to me rather than > iteration, though. > > Perhaps repurposing 'from': > > (x from seq if f(x)) > > That rather breaks TOOWTDI though (since it is essentially new syntax > for a for loop). And I have other hopes for the meaning of (x from ()). . . How about: (for x in seq if f(x)) It still has the word 'for' in it and avoids mentioning x more times than necessary. Although I can't help feeling that it should be some other word instead, such as (all x in seq if f(x)) or (every x in seq if f(x)) -- Greg From robey at lag.net Sun Mar 13 08:35:32 2005 From: robey at lag.net (Robey Pointer) Date: Sun Mar 13 08:36:09 2005 Subject: [Python-Dev] Open issues for 2.4.1 In-Reply-To: <200503130314.39322.anthony@interlink.com.au> References: <200503130314.39322.anthony@interlink.com.au> Message-ID: <4233ED44.8050608@lag.net> Anthony Baxter wrote: >So here's a list of open items I'm thinking about for the 2.4.1 >release. > > - os.access handling unicode filenames > I'm still thinking over whether this is going to cause more problems > for people who find it works for some Python 2.4 and not others. I'm > leaning towards saying that this is a bug fix, but I'd appreciate any > comments (pro or con). If no-one comments, I'll go with whatever my > gut feeling says is the right answer. > > (I've been lurking for a while but this is my first post. I maintain paramiko and enjoy harrassing people on sourceforge to fix the new-style Exception bug that vexes me. Nice to meet you all.) :) I agree 100% with the arguments made over the past couple of weeks about strictly refusing to add new features to micro releases (like 2.4.1). But I don't think fixing the os.access bug is anything but a bug fix, and here's my rationale: It's entirely possible that some python users have worked around this bug by special-casing their use of os.access to manually encode the filename, but that doesn't mean that bringing os.access behavior in line with the other filename methods should be considered a "feature add". Pretend there was a bug with "int" such that you could add any number to it except 1. You can work around this bug by replacing "+1" with "+2-1" but it doesn't make it any less of a bug or any less important to fix. In other words, I don't think that the argument "users may have written code to work around the bug" is a valid reason to not fix a bug. robey From aleaxit at yahoo.com Sun Mar 13 08:53:31 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Sun Mar 13 08:51:32 2005 Subject: [Python-Dev] Open issues for 2.4.1 In-Reply-To: <4233ED44.8050608@lag.net> References: <200503130314.39322.anthony@interlink.com.au> <4233ED44.8050608@lag.net> Message-ID: <3e72ee3554de36ff3cb746c9ee5747c2@yahoo.com> On Mar 13, 2005, at 08:35, Robey Pointer wrote: > In other words, I don't think that the argument "users may have > written code to work around the bug" is a valid reason to not fix a > bug. +1, as long as the bugfix doesn't break the workaround (and os.access's bug seems to meet this criterion). Alex From anthony at interlink.com.au Sun Mar 13 09:22:49 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Sun Mar 13 09:23:19 2005 Subject: [Python-Dev] Open issues for 2.4.1 In-Reply-To: <4233ED44.8050608@lag.net> References: <200503130314.39322.anthony@interlink.com.au> <4233ED44.8050608@lag.net> Message-ID: <200503131922.52582.anthony@interlink.com.au> On Sunday 13 March 2005 18:35, Robey Pointer wrote: > [on the os.access unicode fix] Ok, I'm convinced - Martin, can you check this in? -- Anthony Baxter It's never too late to have a happy childhood. From martin at v.loewis.de Sun Mar 13 10:28:19 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Mar 13 10:28:23 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <87is3wxst6.fsf@hydra.bayview.thirdcreek.com> References: <200503110138.02569.anthony@python.org> <4230FAA5.4070309@v.loewis.de> <87is3wxst6.fsf@hydra.bayview.thirdcreek.com> Message-ID: <423407B3.4090407@v.loewis.de> Kurt B. Kaiser wrote: > I had some strange experiences. Weird indeed. > I downloaded the 2.4.1c1 installer to the desktop and clicked on it. > It complained that it couldn't access the installer. Do you happen to remember the precise error message? > I then clicked on the 2.4.1b2 installer and that started ok. Cancelled it. This must have been 2.4b2, right? However, didn't you have 2.4 installed already? > For ducks I installed Python at D:\"Python 2.4". It installed in > the correct location, also found C:\Python24 and uninstalled that. That is supposed to happen; I never tested it, though. > Python and IDLE appear to run ok. Shortcuts on Run work, and file > association was updated, so right clicking a .py used IDLE from > Python 2.4.1c1, in no-subprocess mode. Good! Martin From vwehren at home.nl Sun Mar 13 10:52:06 2005 From: vwehren at home.nl (Vincent Wehren) Date: Sun Mar 13 10:52:08 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <423407B3.4090407@v.loewis.de> Message-ID: <20050313095206.F26EA1E4006@bag.python.org> Martin, This is somewhat of a corner case, but maybe worth investigating: To check what I mentioned on comp.lang.python earlier, I ran the installer again (with 2.4.1 still intact), selected the "Change Python 2.4.1c1" radio button, clicked the "Finish" Button, clicked the "Advanced" button, clicked the "Cancel" button, and clicked "Yes" to the question "Are you sure you want to cancel the Python 2.4.1c1 installation". This crashed msiexec.exe. I was able to reproduce this on Windows XP Professional, Service Pack 2. Regards, -- Vincent Wehren From tim.leeuwvander at nl.unisys.com Sun Mar 13 11:00:12 2005 From: tim.leeuwvander at nl.unisys.com (Leeuw van der, Tim) Date: Sun Mar 13 11:00:19 2005 Subject: [Python-Dev] Python2.4.1c1 and win32com Message-ID: -----Original Message----- From: "Martin v. Lowis" [mailto:martin@v.loewis.de] Sent: Saturday, March 12, 2005 3:12 PM To: Leeuw van der, Tim Cc: python-dev@python.org Subject: Re: [Python-Dev] Python2.4.1c1 and win32com > Leeuw van der, Tim wrote: > > The generated files crash the Python interpreter with Python 2.4 > > > > Under Python 2.4.1c1, They give a syntax error!? > > Even though the bug was fixed in pywin, it is interesting to observe > that Mark's analysis says > > "Cause of the underling crash is that the generated .py > causes an overflow as the byte-code is generated - something > in 2.4 bloated the byte-code for these modules." > > There also seems to be an interaction with PEP 263, for which patch > 1101726 might provide a solution. > > So I think this needs to be investigated; please submit a bug report, > including the Python file that causes the crash. > I agree that this needs to be investigated, b/c valid code shouldn't result in a syntax error. But just to make sure there's no misunderstandings: Python2.4.1rc1 fixed the crashes. And the generated file is 1.5Mb big; I think I should not post it as - is but rather compress it? What's the best advice on attaching such large files? > Regards, > Martin > From martin at v.loewis.de Sun Mar 13 11:21:39 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Mar 13 11:21:43 2005 Subject: [Python-Dev] Python2.4.1c1 and win32com In-Reply-To: References: Message-ID: <42341433.1080208@v.loewis.de> Leeuw van der, Tim wrote: > I agree that this needs to be investigated, b/c valid code shouldn't > result in a syntax error. But just to make sure there's no > misunderstandings: Python2.4.1rc1 fixed the crashes. And the > generated file is 1.5Mb big; I think I should not post it as - is but > rather compress it? What's the best advice on attaching such large > files? If you can store it somewhere on the net, that would be best, putting a link to the file into the SF bug report. SF has a limit on file attachments which is much smaller. Regards, Martin From mwh at python.net Sun Mar 13 13:56:31 2005 From: mwh at python.net (Michael Hudson) Date: Sun Mar 13 13:56:32 2005 Subject: new-style sxceptions (was Re: [Python-Dev] Open issues for 2.4.1) In-Reply-To: <4233ED44.8050608@lag.net> (Robey Pointer's message of "Sat, 12 Mar 2005 23:35:32 -0800") References: <200503130314.39322.anthony@interlink.com.au> <4233ED44.8050608@lag.net> Message-ID: <2m64zvn1q8.fsf_-_@starship.python.net> Robey Pointer writes: > (I've been lurking for a while but this is my first post. I maintain > paramiko and enjoy harrassing people on sourceforge to fix the > new-style Exception bug that vexes me. Nice to meet you all.) :) Well, hey, you can review my patch if you like: http://python.org/sf/1104669 Writing documentation would be even better :) Cheers, mwh -- Also, whenever a programmer thinks, "Hey, skins, what a cool idea", their computer's speakers should create some sort of cock-shaped soundwave and plunge it repeatedly through their skulls. -- http://www.livejournal.com/talkread.bml?journal=jwz&itemid=123070 From gvanrossum at gmail.com Sun Mar 13 17:26:10 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun Mar 13 17:26:13 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <4233E476.9030705@canterbury.ac.nz> References: <42323B44.5080201@iinet.net.au> <4233E476.9030705@canterbury.ac.nz> Message-ID: [Nick Coghlan] > > That 'x in seq' bit still shouts "containment" to me rather than > > iteration, though. > > > > Perhaps repurposing 'from': > > > > (x from seq if f(x)) > > > > That rather breaks TOOWTDI though (since it is essentially new syntax > > for a for loop). And I have other hopes for the meaning of (x from ()). . . [Greg Ewing] > How about: > > (for x in seq if f(x)) > > It still has the word 'for' in it and avoids mentioning > x more times than necessary. > > Although I can't help feeling that it should be some > other word instead, such as > > (all x in seq if f(x)) > > or > > (every x in seq if f(x)) I doubt that we'll find a satisfactory solution for this issue. Here's why: - We're only talking of a savings of 4 characters (plus two spaces) in the best case, assuming the identifier can be a single letter (it's scope is only the current expression anyway). - Ping's proposal ('x in seq if f(x)') IMO loses for several reasons: it would be a hugely ambiguous use of 'in', and would cut off a possible future syntax for conditional expression(*): 'A if T else B'. - The various improvements on Ping's proposal reduce the amount of typing saved even more ('every' is actually longer than 'for x'). - Before anybody asks, I really do think the reason this is requested at all is really just to save typing; there isn't the "avoid double evaluation" argument that helped acceptance for assignment operators (+= etc.), and I find redability is actually improved with 'for'. ____ (*) Pleas stop calling it 'ternary expression'. That doesn't explain what it means. It's as if we were to refer to the + operator as 'binary expression'. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Sun Mar 13 17:38:42 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sun Mar 13 17:38:44 2005 Subject: [Python-Dev] Rationale for sum()'s design? Message-ID: There are a few design choices we could have made for sum(); in particular, for non-empty sequences we could not have used the identity element (the optional second argument). As it is, we get unjustified but understandable complaints that sum() "only supports numbers". An alternative design could have returned: - the identity (defaulting to 0) if the sequence is empty - the first and only element if the sequence only has one element - (...(((A + B) + C) + D) + ...) if the sequence has more than one element This has surprises too (in particular of returning 0 when invoked without overriding the identity argument for a seqence of addable non-numbers) but works without using the second argument for a larger set of inputs I believe it is often used in such a way that the input is known to be non-empty). I'd be happy to be pointed to a past discussion where this was considered and rejected with a good reason; then I can post that to the blog (where the deficiency in sum() is being berated a bit excessively). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From sabbey at u.washington.edu Sun Mar 13 20:23:27 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Sun Mar 13 20:23:32 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <4233E2FF.5090507@canterbury.ac.nz> References: <4233E2FF.5090507@canterbury.ac.nz> Message-ID: On Sun, 13 Mar 2005, Greg Ewing wrote: > Brian Sabbey wrote: >> I prefer re-using the 'for' loop for this purpose because it allows the >> problem to be solved in a general way by re-using a structure with which >> most users are already familiar, and uses generators, which are easier to >> use in this case than defining a class with __exit__, __enter__, etc. > > But this is an abuse of both the for-loop and the generator. > It's using a for-loop for something that does not loop and > is never intended to loop, and it's using yield to do > something other than producing a value for consumption. > > I'd rather see a new mechanism altogether for this very > different use case. The problem with creating a new mechanism is that sometimes you will want to loop. For example, reading a bunch of items from a shared resource, modifying them, and sending them back. A new, non-looping mechanism will not be adequate for this because it cannot loop, and a 'for' loop will not be adequate for the same reason it is not adequate now: it can't automatically unlock the resource at the end of the loop. I was looking for a single mechanism to handle all of these cases, but I can see why you and most people would not be so keen on the idea of abusing (I prefer "expanding the uses of") 'for' loops this way. -Brian From martin at v.loewis.de Sun Mar 13 23:18:18 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Mar 13 23:18:26 2005 Subject: [Python-Dev] Open issues for 2.4.1 In-Reply-To: <200503131922.52582.anthony@interlink.com.au> References: <200503130314.39322.anthony@interlink.com.au> <4233ED44.8050608@lag.net> <200503131922.52582.anthony@interlink.com.au> Message-ID: <4234BC2A.6010207@v.loewis.de> Anthony Baxter wrote: > Ok, I'm convinced - Martin, can you check this in? Done! Martin From eric.nieuwland at xs4all.nl Mon Mar 14 00:02:16 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Mon Mar 14 00:02:20 2005 Subject: [Python-Dev] Adding any() and all() In-Reply-To: References: Message-ID: I think the discussion should separate numeric calculation and truth value calculation. Numeric calculation need to run through all elements, with the order possibly important. Truth value calculation (as per any() and all()) may terminate before all elements have been seen. Finally, any(), all(), and perhaps some should apply to sets as well. --eric From greg.ewing at canterbury.ac.nz Mon Mar 14 01:14:21 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon Mar 14 01:14:37 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: <4233E2FF.5090507@canterbury.ac.nz> Message-ID: <4234D75D.2040703@canterbury.ac.nz> Brian Sabbey wrote: > The problem with creating a new mechanism is that sometimes you will > want to loop. For example, reading a bunch of items from a shared > resource, modifying them, and sending them back. A new, non-looping > mechanism will not be adequate for this because it cannot loop, If there is a mechanism for passing a code block as a thunk to an arbitrary function, the function is free to loop or not as it sees fit. I'd just prefer the spelling of it didn't force you to make it look like it's looping when it's not. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From sabbey at u.washington.edu Mon Mar 14 01:21:19 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Mon Mar 14 01:21:25 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <4234D75D.2040703@canterbury.ac.nz> References: <4233E2FF.5090507@canterbury.ac.nz> <4234D75D.2040703@canterbury.ac.nz> Message-ID: On Mon, 14 Mar 2005, Greg Ewing wrote: > Brian Sabbey wrote: >> The problem with creating a new mechanism is that sometimes you will want >> to loop. For example, reading a bunch of items from a shared resource, >> modifying them, and sending them back. A new, non-looping mechanism will >> not be adequate for this because it cannot loop, > > If there is a mechanism for passing a code block as > a thunk to an arbitrary function, the function is > free to loop or not as it sees fit. I'd just prefer > the spelling of it didn't force you to make it > look like it's looping when it's not. I think you're right. How about something like below? In the same way that "self" is passed "behind the scenes" as the first argument, so can the thunk be. (Many ideas taken from around [1]) def stopwatch(thunk): t = time.time() thunk() return t - time.time() with stopwatch() result dt: a() b() print 'it took', dt, 'seconds to compute' ========================================= def pickled_file(thunk, name): f = open(name) new_data = thunk(pickle.load(f)) f.close() f = open(name, 'w') pickle.dump(new_data, f) f.close() with greetings from pickled_file('greetings.pickle'): greetings.append('hello') greetings.append('howdy') value greetings [1] http://mail.python.org/pipermail/python-dev/2003-February/032732.html -Brian From nyamatongwe at gmail.com Mon Mar 14 00:12:09 2005 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Mon Mar 14 01:22:22 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: References: <42323B44.5080201@iinet.net.au> <4233E476.9030705@canterbury.ac.nz> Message-ID: <50862ebd050313151211fbc478@mail.gmail.com> Guido van Rossum: > - Before anybody asks, I really do think the reason this is requested > at all is really just to save typing; there isn't the "avoid double > evaluation" argument that helped acceptance for assignment operators > (+= etc.), and I find redability is actually improved with 'for'. For me, the main motivation is to drop an unnecessarily repeated identifier. If you repeat something there is a chance that one of the occurrances will be wrong which is one reason behind the Don't Repeat Yourself principle. The reader can more readily see that this is a filter expression rather than a transforming expression. Neil From greg.ewing at canterbury.ac.nz Mon Mar 14 01:22:25 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon Mar 14 01:22:41 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: Message-ID: <4234D941.9040605@canterbury.ac.nz> Guido van Rossum wrote: > > - the identity (defaulting to 0) if the sequence is empty > - the first and only element if the sequence only has one element > - (...(((A + B) + C) + D) + ...) if the sequence has more than one element While this might be reasonable if the identity argument is not specified, I think that if an identity is specified, it should be used even if the sequence is non-empty. The reason being that the user might be relying on that to get the semantics he wants. Think of the second argument as "accumulator object" rather than "identity". -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Mon Mar 14 01:32:33 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon Mar 14 01:32:47 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: <4233E2FF.5090507@canterbury.ac.nz> <4234D75D.2040703@canterbury.ac.nz> Message-ID: <4234DBA1.7010304@canterbury.ac.nz> Brian Sabbey wrote: > How about something like below? In the same way > that "self" is passed "behind the scenes" as the first argument, so can > the thunk be. > > with stopwatch() result dt: > a() > b() > print 'it took', dt, 'seconds to compute' Something like that would be better, yes. Maybe even just dt = stopwatch(): a() b() Whatever keyword is used is bound to not sound right for some usages, so it would be best if no keyword were needed at all. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From python at rcn.com Mon Mar 14 02:11:03 2005 From: python at rcn.com (Raymond Hettinger) Date: Mon Mar 14 02:16:16 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() andall()) In-Reply-To: <50862ebd050313151211fbc478@mail.gmail.com> Message-ID: <002201c52832$ce0902a0$7dbd2c81@oemcomputer> [GvR] > > - Before anybody asks, I really do think the reason this is requested > > at all is really just to save typing; there isn't the "avoid double > > evaluation" argument that helped acceptance for assignment operators > > (+= etc.), and I find redability is actually improved with 'for'. {Neil Hodgson] > For me, the main motivation is to drop an unnecessarily repeated > identifier. If you repeat something there is a chance that one of the > occurrances will be wrong which is one reason behind the Don't Repeat > Yourself principle. It's not actually a repetition. Instead, it is a precise expression of what is to be returned. The proposal adopts a custom syntax to handle one possible return value (identity) as a default. You're trading away explictness, giving up on having a uniform syntax, and introducing more than one way to do it (the current approach would still be valid). The repeated expression argument carried more weight in the context of augmented assignment where complex lvalues are common: self.arr[i] += j. For genexps and listcomps, simple loop variables are the norm. Try applying the proposal to existing code. I think you'll see that you've gained nothing. > The reader can more readily see that this is a > filter expression rather than a transforming expression. Personally, I find the proposed syntax to be difficult to parse. It's a step backwards. -1 Sorry, I deem the proposal to be horrendous and hope it gets trounced before it gets out of hand. Raymond From gvanrossum at gmail.com Mon Mar 14 03:39:33 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon Mar 14 03:39:37 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <4234D941.9040605@canterbury.ac.nz> References: <4234D941.9040605@canterbury.ac.nz> Message-ID: [Guido van Rossum] > > > > - the identity (defaulting to 0) if the sequence is empty > > - the first and only element if the sequence only has one element > > - (...(((A + B) + C) + D) + ...) if the sequence has more than one element [Greg Ewing] > While this might be reasonable if the identity > argument is not specified, I think that if an > identity is specified, it should be used even > if the sequence is non-empty. The reason being > that the user might be relying on that to get > the semantics he wants. > > Think of the second argument as "accumulator > object" rather than "identity". But I think the logical consequence of your approach would be that sum([]) should raise an exception rather than return 0, which would be backwards incompatible. Because if the identity element has a default value, the default value should be used exactly as if it were specified explicitly. Unfortunately my proposal is also backwards incompatible, since currently sum([1,1], 40) equals 42. I guess nobody remembers why we did it the way it is? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From kbk at shore.net Mon Mar 14 01:21:05 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon Mar 14 05:44:11 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <423407B3.4090407@v.loewis.de> (Martin v. =?iso-8859-1?q?L=F6wis's?= message of "Sun, 13 Mar 2005 10:28:19 +0100") References: <200503110138.02569.anthony@python.org> <4230FAA5.4070309@v.loewis.de> <87is3wxst6.fsf@hydra.bayview.thirdcreek.com> <423407B3.4090407@v.loewis.de> Message-ID: <87wtsbw00e.fsf@hydra.bayview.thirdcreek.com> "Martin v. L?wis" writes: >> I downloaded the 2.4.1c1 installer to the desktop and clicked on it. >> It complained that it couldn't access the installer. > > Do you happen to remember the precise error message? "This installation package could not be opened." > >> I then clicked on the 2.4.1b2 installer and that started ok. Cancelled it. > > This must have been 2.4b2, right? However, didn't you have 2.4 installed > already? Right, 2.4b2. I never updated to 2.4 on that box. OK, the problem was: I rushed the install. The package had not finished downloading, but the icon appears on the desktop as soon as the download starts. (With Opera you have to pay attention to the download status, it's rather subtle.) If I wait for the download to complete and the virus check to finish, then it's AOK. That's why the third time was the charm. -- KBK From michael.walter at gmail.com Sun Mar 13 20:19:05 2005 From: michael.walter at gmail.com (Michael Walter) Date: Mon Mar 14 07:26:00 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: Message-ID: <877e9a1705031311194d9701ee@mail.gmail.com> That is like Lisp's +, must be good :P Michael On Sun, 13 Mar 2005 08:38:42 -0800, Guido van Rossum wrote: > There are a few design choices we could have made for sum(); in > particular, for non-empty sequences we could not have used the > identity element (the optional second argument). As it is, we get > unjustified but understandable complaints that sum() "only supports > numbers". An alternative design could have returned: > > - the identity (defaulting to 0) if the sequence is empty > - the first and only element if the sequence only has one element > - (...(((A + B) + C) + D) + ...) if the sequence has more than one element > > This has surprises too (in particular of returning 0 when invoked > without overriding the identity argument for a seqence of addable > non-numbers) but works without using the second argument for a larger > set of inputs I believe it is often used in such a way that the input > is known to be non-empty). > > I'd be happy to be pointed to a past discussion where this was > considered and rejected with a good reason; then I can post that to > the blog (where the deficiency in sum() is being berated a bit > excessively). > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com > From aleaxit at yahoo.com Mon Mar 14 08:31:23 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Mon Mar 14 08:29:23 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <4234D941.9040605@canterbury.ac.nz> References: <4234D941.9040605@canterbury.ac.nz> Message-ID: <26c7faae2413e1044c7c78d7b25ae3ea@yahoo.com> On Mar 14, 2005, at 01:22, Greg Ewing wrote: > Guido van Rossum wrote: >> - the identity (defaulting to 0) if the sequence is empty >> - the first and only element if the sequence only has one element >> - (...(((A + B) + C) + D) + ...) if the sequence has more than one >> element > > While this might be reasonable if the identity > argument is not specified, I think that if an > identity is specified, it should be used even > if the sequence is non-empty. The reason being > that the user might be relying on that to get > the semantics he wants. > > Think of the second argument as "accumulator > object" rather than "identity". +1 to Greg's idea -- I do have cases where the items arrive in irregular bunches and I maintain the running total via this mechanism, initializing as running_total = 0 and updating it as running_total = sum(bunch_of_items, running_total) Back to Guido's request for the history of how the design evolved, I did some googling -- it all happened on this mailing list, April 19 to April 21, 2003. Most of it subject: Fwd: summing a bunch of numbers (or "whatevers"), though some had Re: and other lacked Fwd: decorations, in case you're searching on the month's archives by subject. Reading the whole thread will help with the pro's and con's, but the conclusions are mostly in Guido's post http://mail.python.org/pipermail/python-dev/2003-April/034853.html and my concurring reply http://mail.python.org/pipermail/python-dev/2003-April/034855.html . Alex From gmccaughan at synaptics-uk.com Mon Mar 14 10:57:47 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon Mar 14 10:58:21 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: References: <4233E476.9030705@canterbury.ac.nz> Message-ID: <200503140957.47849.gmccaughan@synaptics-uk.com> > - Before anybody asks, I really do think the reason this is requested > at all is really just to save typing; there isn't the "avoid double > evaluation" argument that helped acceptance for assignment operators > (+= etc.), and I find redability is actually improved with 'for'. I'd like it, and my reason isn't "just to save typing". There are two reasons. 1 Some bit of my brain is convinced that [x in stuff if condition] is the Right Syntax and keeps making me type it even though I know it doesn't work. 2 Seeing [x for x in stuff if condition] triggers my internal duplicated-stuff alarm, and it's distracting, in the same sort of way as it's distracting in C or C++ seeing Thing thing = new Thing(); with the type name appearing three times. Incidentally, while it's quite true that you only save 4 characters if you use "x" for your variable name, some of us do sometimes like to use something more informative even as a very local loop-index variable (which is what this basically is). -- g From ncoghlan at iinet.net.au Mon Mar 14 11:20:56 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Mar 14 11:21:05 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> Message-ID: <42356588.2050508@iinet.net.au> Guido van Rossum wrote: > But I think the logical consequence of your approach would be that > sum([]) should raise an exception rather than return 0, which would be > backwards incompatible. Because if the identity element has a default > value, the default value should be used exactly as if it were > specified explicitly. > > Unfortunately my proposal is also backwards incompatible, since > currently sum([1,1], 40) equals 42. Somewhat ugly, but backwards compatible: sentinel = object() def sum(iterable, initial=sentinel): itr = iter(iterable) if initial is not sentinel: # Initial value provided, so use it value = initial else: try: first = itr.next() except StopIteration: # Empty iterable, return 0 for backwards compatibility # Also correct for standard numerical use return 0 # Assume default constructor returns the additive identity value = type(first)() value += first # Add the elements for item in itr: value += item return value Py> sum([]) 0 Py> seq = ([1], [2], [3]) Py> sum(seq) [1, 2, 3] Py> seq ([1], [2], [3]) Py> seq = ('1', '2', '3') Py> sum(seq) '123' Py> seq ('1', '2', '3') Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From mwh at python.net Mon Mar 14 11:28:42 2005 From: mwh at python.net (Michael Hudson) Date: Mon Mar 14 11:28:44 2005 Subject: [Python-Dev] can we stop pretending _PyTyple_Lookup is internal? Message-ID: <2moedmldwl.fsf@starship.python.net> It's needed to implement things that behave like instance methods, for instance, and I notice that at at least some point in the past Zope3 has used the function with a note attached saying "Guido says this is OK". Also, the context is that I want to submit a patch to PyObjC using the function, and doing a little digging revealed that PyObjC already has a copy & paste copy of _PyType_Lookup, presumably to avoid using an internal API. I don't think this does anyone any good. Thoughts? Cheers, mwh -- I wouldn't trust the Anglo-Saxons for much anything else. Given they way English is spelled, who could trust them on _anything_ that had to do with writing things down, anyway? -- Erik Naggum, comp.lang.lisp From eric.nieuwland at xs4all.nl Mon Mar 14 13:06:01 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Mon Mar 14 13:06:13 2005 Subject: [Python-Dev] func.update_meta (was: @deprecated) In-Reply-To: <42326F53.3060106@iinet.net.au> References: <000f01c52433$d6f5eb60$7eb99d8d@oemcomputer> <42302F56.7060702@iinet.net.au> <42326F53.3060106@iinet.net.au> Message-ID: <3dd07909a9bea923ecd97b8a1b716013@xs4all.nl> Neat! But please add something to the __doc__ so we can also see it was changed. E.g. self.__doc__ = other.__doc__ + os.linesep + "*** deprecated ***" On 12 mrt 2005, at 5:25, Nick Coghlan wrote: > I like "update_meta" > > Patch against current CVS added to SF with the behaviour: > > def update_meta(self, other): > self.__name__ = other.__name__ > self.__doc__ = other.__doc__ > self.__dict__.update(other.__dict__) From aleaxit at yahoo.com Mon Mar 14 13:09:12 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Mon Mar 14 13:07:12 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <200503140957.47849.gmccaughan@synaptics-uk.com> References: <4233E476.9030705@canterbury.ac.nz> <200503140957.47849.gmccaughan@synaptics-uk.com> Message-ID: <1f6e19a38824d30fa3885692c8034b36@yahoo.com> On Mar 14, 2005, at 10:57, Gareth McCaughan wrote: > of way as it's distracting in C or C++ seeing > > Thing thing = new Thing(); > > with the type name appearing three times. I think you can't possibly see this in C:-), you need a star there in C++, and you need to avoid the 'new' (just calling Thing() should do it -- maybe you're commixing with Java?), but still, I do agree it looks uncool... no doubt a subtle ploy by Java and C++ designers to have you use, instead, the preferable interface-and-factory idioms such as: IThing thing* = thingFactory(); rather than declaring and instantiating concrete classes, which is just _so_ three years ago;-) Back to the Python world, I don't particularly love [x for x in ...] by any means, but I surely hope we're not tweaking the syntax for such tiny gains in the 2.4 -> 2.5 transition. Wasn't 2.5 "supposed to be" mostly about standard library reorganizations, enhancements, etc? Were there some MAJOR gains to be had in syntax additions, guess that could be bent, but snipping the [ for ...] leading part seems just such a tiny issue. (If the discussion is about 3.0, and I missed the indication of that, I apologize). Alex From aleaxit at yahoo.com Mon Mar 14 13:12:59 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Mon Mar 14 13:10:59 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <42356588.2050508@iinet.net.au> References: <4234D941.9040605@canterbury.ac.nz> <42356588.2050508@iinet.net.au> Message-ID: <84d9fc0a691684a9870db60d79d725a2@yahoo.com> On Mar 14, 2005, at 11:20, Nick Coghlan wrote: ... > Somewhat ugly, but backwards compatible: I realize you're mostly talking semantics, not implementation, but, as long as we're at it, could we pretty please have back the optimization indicated by...: > # Add the elements if isinstance(value, basestring): return value + ''.join(itr) > for item in itr: > value += item > return value ...? This doesn't break bw compat since currently when value's a string sum would raise an exception... Alex From eric.nieuwland at xs4all.nl Mon Mar 14 13:34:44 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Mon Mar 14 13:34:56 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <4234D941.9040605@canterbury.ac.nz> References: <4234D941.9040605@canterbury.ac.nz> Message-ID: <8f0366d60265d201c50220df4d508b2f@xs4all.nl> Greg Ewing wrote: > Guido van Rossum wrote: >> - the identity (defaulting to 0) if the sequence is empty >> - the first and only element if the sequence only has one element >> - (...(((A + B) + C) + D) + ...) if the sequence has more than one >> element > > While this might be reasonable if the identity > argument is not specified, I think that if an > identity is specified, it should be used even > if the sequence is non-empty. The reason being > that the user might be relying on that to get > the semantics he wants. > > Think of the second argument as "accumulator > object" rather than "identity". +1 for Greg I think of the second argument as a running total which defaults to the operator's neutral element. Perhaps the second argument should not be optional to emphasise this. After all, there's much more to sum() than numbers. --eric From eric.nieuwland at xs4all.nl Mon Mar 14 13:42:29 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Mon Mar 14 13:42:38 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <200503140957.47849.gmccaughan@synaptics-uk.com> References: <4233E476.9030705@canterbury.ac.nz> <200503140957.47849.gmccaughan@synaptics-uk.com> Message-ID: <63551edc689e07481145277c661bd5ec@xs4all.nl> Gareth McCaughan wrote: > I'd like it, and my reason isn't "just to save typing". > There are two reasons. > > 1 Some bit of my brain is convinced that [x in stuff if condition] > is the Right Syntax and keeps making me type it even though > I know it doesn't work. > > 2 Seeing [x for x in stuff if condition] triggers my internal > duplicated-stuff alarm, and it's distracting, in the same sort > of way as it's distracting in C or C++ seeing The full syntax is: [ f(x) for x in seq if pred(x) ] being allowed to write 'x' instead of 'identity(x)' is already a shortcut, just as dropping the conditional part. Remember we're doing set theory stuff here. IMHO we should follow its notation conventions as much as we can. --eric From arigo at tunes.org Mon Mar 14 14:13:39 2005 From: arigo at tunes.org (Armin Rigo) Date: Mon Mar 14 14:19:00 2005 Subject: [Python-Dev] can we stop pretending _PyTyple_Lookup is internal? In-Reply-To: <2moedmldwl.fsf@starship.python.net> References: <2moedmldwl.fsf@starship.python.net> Message-ID: <20050314131339.GA11670@vicky.ecs.soton.ac.uk> Hi Michael, > ... _PyType_Lookup ... There has been discussions about copy_reg.py and at least one other place in the standard library that needs this; it is an essential part of the descriptor model of new-style classes. In my opinion it should be made part not only of the official C API but the Python one too, e.g. as a method of 'type' instances: type(x).lookup('name') Armin From gmccaughan at synaptics-uk.com Mon Mar 14 14:20:10 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon Mar 14 14:20:44 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <1f6e19a38824d30fa3885692c8034b36@yahoo.com> References: <200503140957.47849.gmccaughan@synaptics-uk.com> <1f6e19a38824d30fa3885692c8034b36@yahoo.com> Message-ID: <200503141320.10995.gmccaughan@synaptics-uk.com> On Monday 2005-03-14 12:09, Alex Martelli wrote: > > On Mar 14, 2005, at 10:57, Gareth McCaughan wrote: > > > of way as it's distracting in C or C++ seeing > > > > Thing thing = new Thing(); > > > > with the type name appearing three times. > > I think you can't possibly see this in C:-), you need a star there in > C++, and you need to avoid the 'new' (just calling Thing() should do it > -- maybe you're commixing with Java?), but still, I do agree it looks > uncool... no doubt a subtle ploy by Java and C++ designers to have you > use, instead, the preferable interface-and-factory idioms such as: Er, sorry about the various slips of detail. And I don't even use Java. Bah! (But it looks even worse without the "new" intervening...) > IThing thing* = thingFactory(); > > rather than declaring and instantiating concrete classes, which is just > _so_ three years ago;-) :-) > Back to the Python world, I don't particularly love [x for x in ...] by > any means, but I surely hope we're not tweaking the syntax for such > tiny gains in the 2.4 -> 2.5 transition. Wasn't 2.5 "supposed to be" > mostly about standard library reorganizations, enhancements, etc? Were > there some MAJOR gains to be had in syntax additions, guess that could > be bent, but snipping the [ for ...] leading part seems just such > a tiny issue. (If the discussion is about 3.0, and I missed the > indication of that, I apologize). When I say I'd like it, I don't mean "we should change it now", only that it would be nice for it to be there. Stability matters more than optimality sometimes, and now may well be such a time. -- g From gmccaughan at synaptics-uk.com Mon Mar 14 14:23:59 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon Mar 14 14:24:32 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <63551edc689e07481145277c661bd5ec@xs4all.nl> References: <200503140957.47849.gmccaughan@synaptics-uk.com> <63551edc689e07481145277c661bd5ec@xs4all.nl> Message-ID: <200503141324.00395.gmccaughan@synaptics-uk.com> On Monday 2005-03-14 12:42, Eric Nieuwland wrote: > Gareth McCaughan wrote: > > > I'd like it, and my reason isn't "just to save typing". > > There are two reasons. > > > > 1 Some bit of my brain is convinced that [x in stuff if condition] > > is the Right Syntax and keeps making me type it even though > > I know it doesn't work. > > > > 2 Seeing [x for x in stuff if condition] triggers my internal > > duplicated-stuff alarm, and it's distracting, in the same sort > > of way as it's distracting in C or C++ seeing > > The full syntax is: > [ f(x) for x in seq if pred(x) ] > being allowed to write 'x' instead of 'identity(x)' is already a > shortcut, just as dropping the conditional part. > > Remember we're doing set theory stuff here. IMHO we should follow its > notation conventions as much as we can. I'm well aware of what the full syntax is; being allowed to write "x" instead of "identity(x)" is *not* a "shortcut" but a perfectly straightforward unexceptional instance of the usual syntax; list comprehensions already have neither the syntax nor the semantics of set-theorists' comprehensions; and in fact no set theorist would be at all troubled by seeing { x in S : predicate(x) } which is the nearest equivalent in mathematical notation for the abbreviated comprehension expressions being discussed. Other than that, I quite agree :-). -- g From mwh at python.net Mon Mar 14 14:26:08 2005 From: mwh at python.net (Michael Hudson) Date: Mon Mar 14 14:26:10 2005 Subject: [Python-Dev] can we stop pretending _PyTyple_Lookup is internal? In-Reply-To: <20050314131339.GA11670@vicky.ecs.soton.ac.uk> (Armin Rigo's message of "Mon, 14 Mar 2005 13:13:39 +0000") References: <2moedmldwl.fsf@starship.python.net> <20050314131339.GA11670@vicky.ecs.soton.ac.uk> Message-ID: <2m7jkal5ov.fsf@starship.python.net> Armin Rigo writes: > Hi Michael, > >> ... _PyType_Lookup ... > > There has been discussions about copy_reg.py and at least one other place in > the standard library that needs this; it is an essential part of the > descriptor model of new-style classes. In my opinion it should be made part > not only of the official C API but the Python one too, e.g. as a method of > 'type' instances: type(x).lookup('name') Yes, this would be good too. The patch should be trivial; I guess a little work is required to ensure b/w compat, but not very much really. Cheers, mwh -- how am I expected to quit smoking if I have to deal with NT every day -- Ben Raia From ncoghlan at iinet.net.au Mon Mar 14 14:33:35 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Mar 14 14:33:42 2005 Subject: [Python-Dev] func.update_meta (was: @deprecated) In-Reply-To: <3dd07909a9bea923ecd97b8a1b716013@xs4all.nl> References: <000f01c52433$d6f5eb60$7eb99d8d@oemcomputer> <42302F56.7060702@iinet.net.au> <42326F53.3060106@iinet.net.au> <3dd07909a9bea923ecd97b8a1b716013@xs4all.nl> Message-ID: <423592AF.2080909@iinet.net.au> Eric Nieuwland wrote: > Neat! But please add something to the __doc__ so we can also see it was > changed. E.g. > self.__doc__ = other.__doc__ + os.linesep + "*** deprecated ***" Decorators that alter the signature, or wish to change the docstring can make their modifications after copying from the original (or simply not copy from the original in the first place). E.g: def deprecated(orig): def f(*args, **kwds): #Emit warning here return orig(*args, **kwds) f.update_meta(orig) f.__doc__ = "*** Deprecated *** " + f.__doc__ return f Any such changes are outside the purview of a metadata transfer method, though, since they're highly decorator dependent. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From mcherm at mcherm.com Mon Mar 14 14:34:57 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Mar 14 14:34:59 2005 Subject: [Python-Dev] (no subject) Message-ID: <20050314053457.ujhe44be73aosg4c@mcherm.com> Nick Coghlan writes: > Patch against current CVS added to SF with the behaviour: > > def update_meta(self, other): > self.__name__ = other.__name__ > self.__doc__ = other.__doc__ > self.__dict__.update(other.__dict__) Nice... thanks. But I have to ask: is this really the right set of metadata to be updating? Here are a few things that perhaps ought be copied by update_meta: f.__name__ (already included) f.__doc__ (already included) f.__dict__ (already included) f.__module__ (probably should include) f.func_code.co_filename (to match f.__name__, but I'd leave it alone) there's also the annoying fact that in IDLE (and in some other python-aware IDEs) one can see the argument signature for a function as a "tool tip" or other hint. Very handy that, but if a decorator is applied then all you will see is "func(*args, **kwargs)" which is less than helpful. I'm not sure whether this CAN be duplicated... I believe it is generated by examining the following: f.func_code.co_argcount f.func_code.co_varnames f.func_code.co_flags & 0x4 f.func_code.co_flags & 0x8 ...and I suspect (experimentation seems to confirm this) that if you mangle these then the code object won't work correctly. If anyone's got a suggestion for fixing this, I'd love to hear it. -- Michael Chermside From ncoghlan at iinet.net.au Mon Mar 14 15:05:32 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Mar 14 15:05:38 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <84d9fc0a691684a9870db60d79d725a2@yahoo.com> References: <4234D941.9040605@canterbury.ac.nz> <42356588.2050508@iinet.net.au> <84d9fc0a691684a9870db60d79d725a2@yahoo.com> Message-ID: <42359A2C.4010708@iinet.net.au> Alex Martelli wrote: > > On Mar 14, 2005, at 11:20, Nick Coghlan wrote: > ... > >> Somewhat ugly, but backwards compatible: > > > I realize you're mostly talking semantics, not implementation, but, as > long as we're at it, could we pretty please have back the optimization > indicated by...: It turns out the sentinel value isn't really needed either: def sum(*args): itr = iter(args[0]) try: value = args[1] except IndexError: # No start value, so use the type of the first element try: first = itr.next() except StopIteration: # Empty iterable, return 0 for backwards compatibility # When summing over something other than a sequence of # numbers, giving an initial value is a good idea. return 0 # Use default constructor to get initial value value = type(first)() value += first # Special case optimisation of strings if isinstance(value, basestring): # Rely on ''.join promoting to unicode if needed return value + ''.join(itr) # Add the elements for item in itr: value += item return value I'm not sure how much we really gain though - at the moment, using sum() for anything other than numbers gives an immediate exception. With the behaviour above, it appears to work, but will return 0 for an empty sequence instead of the additive identity of the desired type (e.g. "" or []). So the error will turn up somewhere else, instead of as an explicit exception at the point of origin. Maybe what we really should be doing is trapping the TypeError, and generating a more meaningful error message. E.g. Py> def sum(seq, initial=0): ... itr = iter(seq) ... try: ... first = itr.next() ... except StopIteration: ... return 0 ... value = initial ... try: ... value += first ... except TypeError: ... raise TypeError("Cannot add first element %r to initial value %r" % (fir st, value)) ... for item in itr: ... value += item ... return value ... Py> seq = ([1], [2], [3]) Py> sum(seq) Traceback (most recent call last): File "", line 1, in ? File "", line 11, in sum TypeError: Cannot add first element [1] to initial value 0 Py> sum(seq, []) [1, 2, 3] Py> seq = ('1', '2', '3') Py> sum(seq) Traceback (most recent call last): File "", line 1, in ? File "", line 11, in sum TypeError: Cannot add first element '1' to initial value 0 Py> sum(seq, '') '123' -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From FBatista at uniFON.com.ar Mon Mar 14 15:58:17 2005 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Mon Mar 14 15:58:31 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() an d all()) Message-ID: [Gareth McCaughan] #- 1 Some bit of my brain is convinced that [x in stuff if condition] #- is the Right Syntax and keeps making me type it even though #- I know it doesn't work. My brain says: '"x in stuff" returns a bool, so we have "[bool if condition]"', and then my brain crash generating a core dump... . Facundo Bit?cora De Vuelo: http://www.taniquetil.com.ar/plog PyAr - Python Argentina: http://pyar.decode.com.ar/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA. La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050314/8a99d3ee/attachment.html From edcjones at comcast.net Mon Mar 14 16:25:26 2005 From: edcjones at comcast.net (Edward C. Jones) Date: Mon Mar 14 16:25:22 2005 Subject: [Python-Dev] Exception needed: Not enough arguments to PyArg_ParseTuple Message-ID: <4235ACE6.8000401@comcast.net> I had PyArg_ParseTuple(args, "s#s#i:compare", &p1, &bytes1, &p2, &bytes2) in a program. There are not enough arguments to PyArg_ParseTuple. Does PyArg_ParseTuple know how many arguments it is getting? If so, I suggest that an exception should be raised here. From pje at telecommunity.com Mon Mar 14 16:28:50 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Mar 14 16:25:45 2005 Subject: [Python-Dev] (no subject) In-Reply-To: <20050314053457.ujhe44be73aosg4c@mcherm.com> Message-ID: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> At 05:34 AM 3/14/05 -0800, Michael Chermside wrote: >Nice... thanks. But I have to ask: is this really the right set of metadata to >be updating? Here are a few things that perhaps ought be copied by >update_meta: > > f.__name__ (already included) > f.__doc__ (already included) > f.__dict__ (already included) > f.__module__ (probably should include) > f.func_code.co_filename (to match f.__name__, but I'd leave it alone) Leave __module__ alone, too, unless you want to play havoc with any inspection tools looking for the source code. >there's also the annoying fact that in IDLE (and in some other python-aware >IDEs) one can see the argument signature for a function as a "tool tip" >or other hint. Very handy that, but if a decorator is applied then all >you will see is "func(*args, **kwargs)" which is less than helpful. I'm >not sure whether this CAN be duplicated... I believe it is generated by >examining the following: > > f.func_code.co_argcount > f.func_code.co_varnames > f.func_code.co_flags & 0x4 > f.func_code.co_flags & 0x8 > >...and I suspect (experimentation seems to confirm this) that if you mangle >these then the code object won't work correctly. If anyone's got a >suggestion for fixing this, I'd love to hear it. One solution is to have a __signature__ attribute that's purely documentary. That is, modifying it wouldn't change the function's actual behavior, so it could be copied with update_meta() but then modified by the decorator if need be. __signature__ would basically be a structure like the one returned by inspect.getargspec(). Also, 'instancemethod' would have a __signature__ equal to its im_func.__signature__ with the first argument dropped off, thus making it easy to introspect wrapper chains. I discussed this approach with Guido in private e-mail a few months back during discussion about an article I was writing for DDJ about decorators. We also discussed something very similar to 'update_meta()', but never settled on a name. Originally he wanted me to PEP the whole thing, but he wanted it to include optional type declaration info, so you can probably see why I haven't done anything on that yet. :) However, if we can define a __signature__ format that allows for type declaration, I imagine there'd be little problem with moving forward on it. From mwh at python.net Mon Mar 14 16:28:43 2005 From: mwh at python.net (Michael Hudson) Date: Mon Mar 14 16:28:46 2005 Subject: [Python-Dev] Exception needed: Not enough arguments to PyArg_ParseTuple In-Reply-To: <4235ACE6.8000401@comcast.net> (Edward C. Jones's message of "Mon, 14 Mar 2005 10:25:26 -0500") References: <4235ACE6.8000401@comcast.net> Message-ID: <2mzmx6jlg4.fsf@starship.python.net> "Edward C. Jones" writes: > I had > > PyArg_ParseTuple(args, "s#s#i:compare", &p1, &bytes1, &p2, &bytes2) > > in a program. There are not enough arguments to PyArg_ParseTuple. Does > PyArg_ParseTuple know how many arguments it is getting? I don't think so. > If so, I suggest that an exception should be raised here. I think you'd need to do battle with ISO C first. Cheers, mwh -- Counting lines is probably a good idea if you want to print it out and are short on paper, but I fail to see the purpose otherwise. -- Erik Naggum, comp.lang.lisp From theller at python.net Mon Mar 14 16:35:43 2005 From: theller at python.net (Thomas Heller) Date: Mon Mar 14 16:35:46 2005 Subject: [Python-Dev] (no subject) In-Reply-To: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> (Phillip J. Eby's message of "Mon, 14 Mar 2005 10:28:50 -0500") References: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 05:34 AM 3/14/05 -0800, Michael Chermside wrote: >>Nice... thanks. But I have to ask: is this really the right set of metadata to >> be updating? Here are a few things that perhaps ought be copied by >> update_meta: >> >> f.__name__ (already included) >> f.__doc__ (already included) >> f.__dict__ (already included) >> f.__module__ (probably should include) >> f.func_code.co_filename (to match f.__name__, but I'd leave it alone) > > Leave __module__ alone, too, unless you want to play havoc with any > inspection tools looking for the source code. > > >>there's also the annoying fact that in IDLE (and in some other python-aware >>IDEs) one can see the argument signature for a function as a "tool tip" >>or other hint. Very handy that, but if a decorator is applied then all >>you will see is "func(*args, **kwargs)" which is less than helpful. I'm >>not sure whether this CAN be duplicated... I believe it is generated by >>examining the following: >> >> f.func_code.co_argcount >> f.func_code.co_varnames >> f.func_code.co_flags & 0x4 >> f.func_code.co_flags & 0x8 >> >>...and I suspect (experimentation seems to confirm this) that if you mangle >>these then the code object won't work correctly. If anyone's got a >>suggestion for fixing this, I'd love to hear it. > > One solution is to have a __signature__ attribute that's purely > documentary. That is, modifying it wouldn't change the function's > actual behavior, so it could be copied with update_meta() but then > modified by the decorator if need be. __signature__ would basically > be a structure like the one returned by inspect.getargspec(). Also, > 'instancemethod' would have a __signature__ equal to its > im_func.__signature__ with the first argument dropped off, thus making > it easy to introspect wrapper chains. Another possibility (ugly, maybe) would be to create sourcecode with the function signature that you need, and compile it. inspect.getargspec() and inspect.formatargspec can do most of the work. Thomas From pje at telecommunity.com Mon Mar 14 17:13:04 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Mar 14 17:09:59 2005 Subject: [Python-Dev] (no subject) In-Reply-To: References: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050314110752.03093260@mail.telecommunity.com> At 04:35 PM 3/14/05 +0100, Thomas Heller wrote: >Another possibility (ugly, maybe) would be to create sourcecode with the >function signature that you need, and compile it. inspect.getargspec() and >inspect.formatargspec can do most of the work. I've done exactly that, for generic functions in PyProtocols. It's *very* ugly, and not something I'd wish on anyone needing to write a decorator. IMO, inspect.getargspec() shouldn't need to be so complicated; it should just return an object's __signature__ in future Pythons. Also, the 'object' type should have a __signature__ descriptor that returns the __signature__ of __call__, if present. And types should have a __signature__ that returns the __init__ or __new__ signature of the type. Finally, C methods should have a way to define a __signature__ as well. At that point, any callable object has an introspectable __signature__, which would avoid the need for every introspection framework or documentation tool having to rewrite the same old type dispatching code to check if it's an instancemethod, an instance with a __call__, a type, etc. etc. in order to find the real function and how to modify what getargspec() returns. From gvanrossum at gmail.com Mon Mar 14 18:36:23 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Mon Mar 14 18:36:25 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <42359A2C.4010708@iinet.net.au> References: <4234D941.9040605@canterbury.ac.nz> <42356588.2050508@iinet.net.au> <84d9fc0a691684a9870db60d79d725a2@yahoo.com> <42359A2C.4010708@iinet.net.au> Message-ID: On Tue, 15 Mar 2005 00:05:32 +1000, Nick Coghlan wrote: [...] > Maybe what we really should be doing is trapping the TypeError, and generating a > more meaningful error message. > > E.g. > [...] > ... try: > ... value += first > ... except TypeError: > ... raise TypeError("Cannot add first element %r to initial value %r" % (first, value)) No, no, no! NO! Never catch a general exception like that and replace it with one of your own. That can cause hours of debugging pain later on when the type error is deep down in the bowels of the += operation (or perhaps deep down inside something *it* invoked). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Mon Mar 14 20:42:35 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Mar 14 20:42:58 2005 Subject: [Python-Dev] (no subject) In-Reply-To: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> References: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> Message-ID: <4235E92B.5050605@ocf.berkeley.edu> Phillip J. Eby wrote: [SNIP] > One solution is to have a __signature__ attribute that's purely > documentary. That is, modifying it wouldn't change the function's > actual behavior, so it could be copied with update_meta() but then > modified by the decorator if need be. __signature__ would basically be > a structure like the one returned by inspect.getargspec(). Also, > 'instancemethod' would have a __signature__ equal to its > im_func.__signature__ with the first argument dropped off, thus making > it easy to introspect wrapper chains. > > I discussed this approach with Guido in private e-mail a few months back > during discussion about an article I was writing for DDJ about > decorators. We also discussed something very similar to > 'update_meta()', but never settled on a name. Originally he wanted me > to PEP the whole thing, but he wanted it to include optional type > declaration info, so you can probably see why I haven't done anything on > that yet. :) > > However, if we can define a __signature__ format that allows for type > declaration, I imagine there'd be little problem with moving forward on it. > It could be as simple as just a bunch of tuples. The following (assuming *args and **kwargs are not typed; don't remember if they can be):: def foo(pos1, pos2:int, key1="hi", key2=42:int, *args, **kwargs): pass could be:: ((('pos1', None), ('pos2', int)), (('key1', "hi", None), ('key2', 42, int)), 'args', 'kwargs') In case the format is not obvious, just a bunch of tuples grouped based on whether they are positional, keyword, or one of the collecting arguments. For positional arguments, have a two-item tuple consisting of the argument name and the possible type. For keyword, 3-item tuple with name, default value, and possible type. Lacking *args and/or **kwargs could just be set to None for those tuple positions. Since this is mainly for introspection tools the format can stand to be verbose and not the easiest thing to read by eye, but it must contain all possible info on the arguments. And if actual execution does not use this slot, as Phillip is suggesting, but it is only for informative purposes we could make it optional. It could also be set it to a descriptor that dynamically creates the tuple when called based on the function passed into the descriptor at creation time. This could be rolled into the update_meta (or whatever it ends up being called) function. -Brett From tim.leeuwvander at nl.unisys.com Mon Mar 14 21:20:59 2005 From: tim.leeuwvander at nl.unisys.com (Leeuw van der, Tim) Date: Mon Mar 14 21:21:08 2005 Subject: [Python-Dev] Python2.4.1c1 and win32com Message-ID: Bug has been filed with id 1163244. regards, --Tim -----Original Message----- From: "Martin v. Lowis" [mailto:martin@v.loewis.de] Sent: Saturday, March 12, 2005 3:12 PM To: Leeuw van der, Tim Cc: python-dev@python.org Subject: Re: [Python-Dev] Python2.4.1c1 and win32com Leeuw van der, Tim wrote: > The generated files crash the Python interpreter with Python 2.4 > > Under Python 2.4.1c1, They give a syntax error!? [...] So I think this needs to be investigated; please submit a bug report, including the Python file that causes the crash. Regards, Martin From jcarlson at uci.edu Mon Mar 14 21:58:29 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon Mar 14 22:03:32 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <4234DBA1.7010304@canterbury.ac.nz> References: <4234DBA1.7010304@canterbury.ac.nz> Message-ID: <20050314095844.A470.JCARLSON@uci.edu> Greg Ewing wrote: > > Brian Sabbey wrote: > > > How about something like below? In the same way > > that "self" is passed "behind the scenes" as the first argument, so can > > the thunk be. > > > > with stopwatch() result dt: > > a() > > b() > > print 'it took', dt, 'seconds to compute' > > Something like that would be better, yes. Maybe even just > > dt = stopwatch(): > a() > b() > > Whatever keyword is used is bound to not sound right > for some usages, so it would be best if no keyword > were needed at all. Since PEP 310 was already mentioned, can we just say that the discussion can be boiled down to different ways of spelling __enter__/__exit__ from PEP 310? If so, then if either one of you really want/need this kind of thing, maybe one of you should pick up the PEP, address the issues sufficiently, and make it happen. - Josiah From martin at v.loewis.de Mon Mar 14 23:03:51 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Mar 14 23:03:56 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <87wtsbw00e.fsf@hydra.bayview.thirdcreek.com> References: <200503110138.02569.anthony@python.org> <4230FAA5.4070309@v.loewis.de> <87is3wxst6.fsf@hydra.bayview.thirdcreek.com> <423407B3.4090407@v.loewis.de> <87wtsbw00e.fsf@hydra.bayview.thirdcreek.com> Message-ID: <42360A47.5020609@v.loewis.de> Kurt B. Kaiser wrote: >>Do you happen to remember the precise error message? > > > "This installation package could not be opened." Ah, that you get if the file is already open. Do you run some shell extension that might want to look into the MSI file, or virus scanners? I also recall a KB article that the Indexing Service sometimes prevents files from being opened. Quadruple-click could also cause the problem, if you try to start multiple installers. > If I wait for the download to complete and the virus check to finish, then > it's AOK. That's why the third time was the charm. Ok, so it's likely incomplete download. Regards, Martin From greg.ewing at canterbury.ac.nz Tue Mar 15 00:02:41 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 15 00:04:12 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <20050314095844.A470.JCARLSON@uci.edu> References: <4234DBA1.7010304@canterbury.ac.nz> <20050314095844.A470.JCARLSON@uci.edu> Message-ID: <42361811.5010707@canterbury.ac.nz> Josiah Carlson wrote: > Since PEP 310 was already mentioned, can we just say that the discussion > can be boiled down to different ways of spelling __enter__/__exit__ from > PEP 310? It's not quite the same thing. PEP 310 suggests a mechanism with a fixed control flow -- do something on entry, do the code block, do something on exit. A general block-passing mechanism would give complete control to the function as to when and how to call the block. However, it's possible that if generators were enhanced with some means of allowing yield inside try-finally, then generators plus PEP 310 would cover most use cases: for-loops and generators when you want to loop, and PEP 310 when you don't. So rather than coming up with new syntax at this point, perhaps it would be better to concentrate on the problem of yield inside try-finally. Even if the finally can't be guaranteed to run under all conditions, I think it would be useful if it could be arranged so that for x in somegenerator(): ... raise Blather ... would caused any finallies that the generator was suspended inside to be executed. Then the semantics would be the same as if the for-loop-over-generator were implemented by passing a thunk to a function that calls it repeatedly. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 15 00:11:22 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 15 00:11:40 2005 Subject: [Python-Dev] Exception needed: Not enough arguments to PyArg_ParseTuple In-Reply-To: <4235ACE6.8000401@comcast.net> References: <4235ACE6.8000401@comcast.net> Message-ID: <42361A1A.6030802@canterbury.ac.nz> Edward C. Jones wrote: > I had > > PyArg_ParseTuple(args, "s#s#i:compare", &p1, &bytes1, &p2, &bytes2) > > in a program. There are not enough arguments to PyArg_ParseTuple. Does > PyArg_ParseTuple know how many arguments it is getting? No. There is no standard way for a C function to find out how many parameters it has been passed, and most C implementations don't provide any way at all. That's why the calling interface to varargs functions invariably includes some way for the caller to indicate the number of arguments, such as a format string or a terminating NULL. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 15 00:20:07 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 15 00:20:24 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> Message-ID: <42361C27.7080609@canterbury.ac.nz> Guido van Rossum wrote: > But I think the logical consequence of your approach would be that > sum([]) should raise an exception rather than return 0, which would be > backwards incompatible. Because if the identity element has a default > value, the default value should be used exactly as if it were > specified explicitly. In that case I would argue in favour of keeping it the way it is, since... > currently sum([1,1], 40) equals 42. ...seems quite reasonable to me. Or at least as reasonable as anything else. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From sabbey at u.washington.edu Tue Mar 15 00:41:21 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Tue Mar 15 00:41:28 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <42361811.5010707@canterbury.ac.nz> References: <4234DBA1.7010304@canterbury.ac.nz> <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> Message-ID: > be guaranteed to run under all conditions, I think it > would be useful if it could be arranged so that > > for x in somegenerator(): > ... > raise Blather > ... > > would caused any finallies that the generator was suspended > inside to be executed. Then the semantics would be the > same as if the for-loop-over-generator were implemented by > passing a thunk to a function that calls it repeatedly. One difficulty is that one can never know if the user intends to still use the generator, like so: a = somegenerator() try: for x in a: raise Blather except: a.next() I think they only way you can really be sure .next() will not be called again is if the generator is no longer referenced. Someone submitted a patch once to execute "finallies" when the generator is __del__eted, but it was rejected for various reasons. In my original post in this thread I tried to provide a mechanism such as you describe by providing __call__ as an alternative to 'next', but now I am convinced that it is better to introduce a new syntax instead of re-using generators. Incidentally, passing the thunk "behind the scenes" as the first argument (as mentioned previously) allows one to avoid using lambda to do things such as sort (I hear lambdas are on the way out), while remaining anonymous: with x, y from a.sort(): value cmp(x.k1, y.k1) or (x.k2, y.k2) (or whatever the preferred syntax is) instead of: a.sort(lambda x,y : cmp(x.k1, y.k1) or (x.k2, y.k2)) Not that I find either form better than the other, but I do find both better than have to define a named function. I am going to see if I can make a PEP for this. -Brian From greg.ewing at canterbury.ac.nz Tue Mar 15 01:01:37 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 15 01:01:58 2005 Subject: [Python-Dev] comprehension abbreviation In-Reply-To: <63551edc689e07481145277c661bd5ec@xs4all.nl> References: <4233E476.9030705@canterbury.ac.nz> <200503140957.47849.gmccaughan@synaptics-uk.com> <63551edc689e07481145277c661bd5ec@xs4all.nl> Message-ID: <423625E1.4050409@canterbury.ac.nz> Eric Nieuwland wrote: > > The full syntax is: > [ f(x) for x in seq if pred(x) ] > > being allowed to write 'x' instead of 'identity(x)' is already a > shortcut, That's a really strange way of looking at it. Unless you would also say that x = y is a shorthand for x = identity(y) Not that it's false, it just seems like a completely unnecessary layer of mental gymnastics... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 15 01:06:55 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 15 01:07:12 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <8f0366d60265d201c50220df4d508b2f@xs4all.nl> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> Message-ID: <4236271F.5020407@canterbury.ac.nz> Eric Nieuwland wrote: > Perhaps the second argument should not be optional to emphasise this. > After all, there's much more to sum() than numbers. I think practicality beats purity here. Using it on numbers is surely an extremely common case. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From tim.peters at gmail.com Tue Mar 15 01:16:22 2005 From: tim.peters at gmail.com (Tim Peters) Date: Tue Mar 15 01:16:34 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <4236271F.5020407@canterbury.ac.nz> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> Message-ID: <1f7befae050314161678706275@mail.gmail.com> [Eric Nieuwland] >> Perhaps the second argument should not be optional to emphasise this. >> After all, there's much more to sum() than numbers. [Greg Ewing] > I think practicality beats purity here. Using it on > numbers is surely an extremely common case. I'd personally be delighted if sum() never worked on anything other than numbers. That makes it easy to understand, easy to document, easy to remember, obvious at first sight, and straightforward to implement. Everything a framework isn't, but it's not a bad thing to have *something* that actually means exactly what it looks like it says <0.5 wink>. From pedronis at strakt.com Tue Mar 15 01:36:57 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Tue Mar 15 01:35:47 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: <4234DBA1.7010304@canterbury.ac.nz> <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> Message-ID: <42362E29.5080502@strakt.com> Brian Sabbey wrote: >> be guaranteed to run under all conditions, I think it would be >> useful if it could be arranged so that >> >> for x in somegenerator(): ... raise Blather ... >> >> would caused any finallies that the generator was suspended inside >> to be executed. Then the semantics would be the same as if the >> for-loop-over-generator were implemented by passing a thunk to a >> function that calls it repeatedly. > > > One difficulty is that one can never know if the user intends to > still use the generator, like so: > > a = somegenerator() try: for x in a: raise Blather except: a.next() > > I think they only way you can really be sure .next() will not be > called again is if the generator is no longer referenced. Someone > submitted a patch once to execute "finallies" when the generator is > __del__eted, but it was rejected for various reasons. > > In my original post in this thread I tried to provide a mechanism > such as you describe by providing __call__ as an alternative to > 'next', but now I am convinced that it is better to introduce a new > syntax instead of re-using generators. > > Incidentally, passing the thunk "behind the scenes" as the first > argument (as mentioned previously) allows one to avoid using lambda > to do things such as sort (I hear lambdas are on the way out), while > remaining anonymous: > > with x, y from a.sort(): value cmp(x.k1, y.k1) or (x.k2, y.k2) > > (or whatever the preferred syntax is) instead of: > > a.sort(lambda x,y : cmp(x.k1, y.k1) or (x.k2, y.k2)) > > Not that I find either form better than the other, but I do find both > better than have to define a named function. > > Notice that syntax is the key issue here, it is not like it is hard to think a range of semantics for thunks/anonymous functions. In fact thunks can probably be just closures with some tweaks up to the yield issue. In fact if some unnatural (for Python POV likely) parentheses would be acceptable but they are very likely not, it is simple to devise syntaxes that allows anonymous functions pretty much everywhere. This would allow for some rather unwieldy code. OTOH a suite-based syntax for thunks can likely not work as a substitute of lambda for cases like: f(lambda: ..., ...) where the function is the first argument, and then there are further arguments. (Of course not-so-natural helper functions can be written to swap arguments around). Apart this one very hard problem, it would be nice to be able to define and pass more thunks simultaneously. In particular a more concise syntax for def setx(self, v): self._x = v def getx(self): return self._x x = property(getx,setx) was considered in the past discussions about the topic a worthy goal. regards From sabbey at u.washington.edu Tue Mar 15 02:07:05 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Tue Mar 15 02:07:10 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <42362E29.5080502@strakt.com> References: <4234DBA1.7010304@canterbury.ac.nz> <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> <42362E29.5080502@strakt.com> Message-ID: Samuele Pedroni wrote: > OTOH a suite-based syntax for thunks can likely not work as a substitute of > lambda for cases like: > > f(lambda: ..., ...) > > where the function is the first argument, and then there are further > arguments. I do not see why you say suite-based thunks cannot be used in the case in which the function takes more than one argument. Using the syntax I described earlier, they work in just that way: def pickled_file(thunk, name): ... new_data = thunk(pickle.load(f)) ... with greetings from pickled_file('greetings.pickle'): ... value greetings One can make an analogy with "self". Both the thunk and self can be passed automatically as the first argument, and further arguments can follow. In this way, functions that already take a thunk as a first argument (such as sort) can be re-used without modification. > Apart this one very hard problem, it would be nice to be able to define > and pass more thunks simultaneously. In particular a more concise syntax for > > def setx(self, v): self._x = v > def getx(self): return self._x > x = property(getx,setx) > > was considered in the past discussions about the topic a worthy goal. Here I can make another analogy with self. Just as python does not give syntactic support for multiple dispatch because (I assume) it would require major syntax changes for something that would be rarely used, I do not think it is worth it to give syntactic support for multiple thunks. If only a fraction "epsilon" of functions take a single thunk, then I would guess that "epsilon**2" take two thunks, and it is not worth creating syntax for such a small number of cases, especially if that syntax compromises what would otherwise be a much cleaner syntax for the single thunk case. -Brian From gvanrossum at gmail.com Tue Mar 15 02:57:42 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue Mar 15 02:57:49 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <1f7befae050314161678706275@mail.gmail.com> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> Message-ID: On Mon, 14 Mar 2005 19:16:22 -0500, Tim Peters wrote: > [Eric Nieuwland] > >> Perhaps the second argument should not be optional to emphasise this. > >> After all, there's much more to sum() than numbers. > > [Greg Ewing] > > I think practicality beats purity here. Using it on > > numbers is surely an extremely common case. [Tim Peters] > I'd personally be delighted if sum() never worked on anything other > than numbers. That makes it easy to understand, easy to document, > easy to remember, obvious at first sight, and straightforward to > implement. Everything a framework isn't, but it's not a bad thing to > have *something* that actually means exactly what it looks like it > says <0.5 wink>. Unfortunately this started when I claimed in my blog that sum() was a replacement for 80% of all reduce() uses. This was countered by someone who pointed out that without a 2nd argument it doesn't work for non-numbers that happen to implement __add__, and I'm not sure he was aware you could make it work *with* a 2nd argument (I know *I* had forgotten all about that :-). I think the conclusion should be that sum() is sufficiently constrained by backwards compatibility to make "fixing" it impossible before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is only used for empty lists. Alex's use case for a nonzero 2nd argument: total = 0 for lst in sequence_of_lists: total = sum(lst, total) can be just as easily written like this: total = 0 for lst in sequence_of_lists: total += sum(lst) and I think that's actually clearer (since the reader doesn't have to know about the 2nd argument's meaning). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From kbk at shore.net Tue Mar 15 03:14:31 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Mar 15 03:16:04 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <42360A47.5020609@v.loewis.de> (Martin v. =?iso-8859-1?q?L=F6wis's?= message of "Mon, 14 Mar 2005 23:03:51 +0100") References: <200503110138.02569.anthony@python.org> <4230FAA5.4070309@v.loewis.de> <87is3wxst6.fsf@hydra.bayview.thirdcreek.com> <423407B3.4090407@v.loewis.de> <87wtsbw00e.fsf@hydra.bayview.thirdcreek.com> <42360A47.5020609@v.loewis.de> Message-ID: <87fyyxsliw.fsf@hydra.bayview.thirdcreek.com> "Martin v. L?wis" writes: > Ok, so it's likely incomplete download. Definitely. It's a bit of a misfeature that the icon appears on the desktop before the download is complete. But I'd say there's no real issue here, besides my impatience/inattention. -- KBK From eric.nieuwland at xs4all.nl Tue Mar 15 07:46:38 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Tue Mar 15 07:46:43 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> Message-ID: Guido van Rossum wrote: > I think the conclusion should be that sum() is sufficiently > constrained by backwards compatibility to make "fixing" it impossible > before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is > only used for empty lists. Which is not unlike the get() method of dicts. So may be the default should be None? From ronaldoussoren at mac.com Tue Mar 15 08:03:22 2005 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue Mar 15 08:04:17 2005 Subject: [Python-Dev] can we stop pretending _PyTyple_Lookup is internal? In-Reply-To: <2m7jkal5ov.fsf@starship.python.net> References: <2moedmldwl.fsf@starship.python.net> <20050314131339.GA11670@vicky.ecs.soton.ac.uk> <2m7jkal5ov.fsf@starship.python.net> Message-ID: <88d9bad11e57be6f525c962f1fe6c535@mac.com> On 14-mrt-05, at 14:26, Michael Hudson wrote: > Armin Rigo writes: > >> Hi Michael, >> >>> ... _PyType_Lookup ... >> >> There has been discussions about copy_reg.py and at least one other >> place in >> the standard library that needs this; it is an essential part of the >> descriptor model of new-style classes. In my opinion it should be >> made part >> not only of the official C API but the Python one too, e.g. as a >> method of >> 'type' instances: type(x).lookup('name') > > Yes, this would be good too. The patch should be trivial; I guess a > little work is required to ensure b/w compat, but not very much > really. It should IMHO be possible to override that method, the _PyType_Lookup copy in PyObjC that you mentioned earlier is slightly modified to deal with Objective-C categories. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2105 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050315/543695c9/smime-0001.bin From ncoghlan at iinet.net.au Tue Mar 15 12:50:39 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 15 12:51:39 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <42356588.2050508@iinet.net.au> <84d9fc0a691684a9870db60d79d725a2@yahoo.com> <42359A2C.4010708@iinet.net.au> Message-ID: <4236CC0F.1080305@iinet.net.au> Guido van Rossum wrote: > On Tue, 15 Mar 2005 00:05:32 +1000, Nick Coghlan wrote: >>... try: >>... value += first >>... except TypeError: >>... raise TypeError("Cannot add first element %r to initial value %r" % (first, value)) > > > No, no, no! NO! Never catch a general exception like that and replace > it with one of your own. That can cause hours of debugging pain later > on when the type error is deep down in the bowels of the += operation > (or perhaps deep down inside something *it* invoked). Ouch. Obviously, I hadn't thought about that. . . Wasn't there a concept floating around some time ago to support exception chaining, so additional context information could be added to a thrown exception, without losing the location of the original problem? Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From pedronis at strakt.com Tue Mar 15 12:54:06 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Tue Mar 15 12:52:53 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: References: <4234DBA1.7010304@canterbury.ac.nz> <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> <42362E29.5080502@strakt.com> Message-ID: <4236CCDE.40603@strakt.com> Brian Sabbey wrote: > Samuele Pedroni wrote: > >> OTOH a suite-based syntax for thunks can likely not work as a >> substitute of lambda for cases like: >> >> f(lambda: ..., ...) >> >> where the function is the first argument, and then there are further >> arguments. > > > I do not see why you say suite-based thunks cannot be used in the case > in which the function takes more than one argument. Using the syntax I > described earlier, they work in just that way: > > def pickled_file(thunk, name): > ... > new_data = thunk(pickle.load(f)) > ... > > with greetings from pickled_file('greetings.pickle'): > ... > value greetings > > One can make an analogy with "self". Both the thunk and self can be > passed automatically as the first argument, and further arguments can > follow. In this way, functions that already take a thunk as a first > argument (such as sort) can be re-used without modification. Of course now one has a problem with things that take functions as last arguments, wich if I rembember correctly is the natural position in Ruby. Unless the syntax involve placeholders (likely icky) one is going to have this kind of limitations. My point is that a suite-based syntax can only be a half substitute for lambda and anyway requiring a suite seems overkill and unnatural for the just 1 expression case, for example predicates. IOW a suite-based syntax is not a lambda killer in itself, I would not try to stress that point. >> Apart this one very hard problem, it would be nice to be able to define >> and pass more thunks simultaneously. In particular a more concise >> syntax for >> >> def setx(self, v): self._x = v >> def getx(self): return self._x >> x = property(getx,setx) >> >> was considered in the past discussions about the topic a worthy goal. > > > Here I can make another analogy with self. Just as python does not give > syntactic support for multiple dispatch because (I assume) it would > require major syntax changes multiple dispatch has nothing to do with syntax, in fact usual call syntax is sufficient, and people do use multiple dispatch sometimes, and decorators now can be even used to sugar up the definition side of it. > for something that would be rarely used, I > do not think well that's up to discussion to discover it is worth it to give syntactic support for multiple > thunks. If only a fraction "epsilon" of functions take a single thunk, > then I would guess that "epsilon**2" take two thunks, and it is not > worth creating syntax for such a small number of cases, especially if > that syntax compromises what would otherwise be a much cleaner syntax > for the single thunk case. > well, but this is stated without even trying to come up with a syntax for that case. Notice that the first time around Guido himself would have preferred if achievable a multithunk syntax, he obviously can have changed his mind. But, yes, syntax vs expressivity is the key issue here. From ncoghlan at iinet.net.au Tue Mar 15 13:21:07 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 15 13:21:12 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> Message-ID: <4236D333.3070007@iinet.net.au> Guido van Rossum wrote: > I think the conclusion should be that sum() is sufficiently > constrained by backwards compatibility to make "fixing" it impossible > before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is > only used for empty lists. Two questions about this: 1. When omitting the second argument, would supplying an empty list return 0, None or raise an exception? The last seems most reasonable, as otherwise the function is guessing about what the programmer wants. 2. How would the initial value that forms the basis of summation be built for non-empty sequences? The first element can't be used directly, as that would mutate the first element for a sequence of lists or other mutable objects. Using the default constructor for the type of the first element in the iterable has its own problem. With the second argument being ignored for non-empty iterables, it makes it impossible to use sum() with classes that do have an additive identity, but it isn't created by the default constructor. At the moment, such classes can be used by supplying the additive identity as the second argument to sum(). Both of the above cases can be supported by an approach that says: a. If a second argument is not supplied, and the iterable is empty, raise an exception. b. If a second argument is not supplied, and the iterable is not empty, use the default constructor of the type of the first argument as the initial value c. If a second argument is supplied, and the iterable is empty, return the second argument. d. If a second argument is supplied, and the iterable is not empty, use the second argument as the initial value. This scheme can be made backwards compatible with the current sum() by switching point a. from 'raise an exception' to 'return zero'. With the above compatibility change, getting all the way back to the existing sum() behaviour only requires changing point b. to say "use zero as the initial value" Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Tue Mar 15 13:36:39 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 15 13:36:44 2005 Subject: [Python-Dev] (no subject) In-Reply-To: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> References: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> Message-ID: <4236D6D7.2060601@iinet.net.au> Phillip J. Eby wrote: > I discussed this approach with Guido in private e-mail a few months back > during discussion about an article I was writing for DDJ about > decorators. We also discussed something very similar to > 'update_meta()', but never settled on a name. Originally he wanted me > to PEP the whole thing, but he wanted it to include optional type > declaration info, so you can probably see why I haven't done anything on > that yet. :) > > However, if we can define a __signature__ format that allows for type > declaration, I imagine there'd be little problem with moving forward on it. But one of the reasons for providing 'update_meta' is so that additional metadata (like __signature__) can later be added transparently. Does deciding whether or not to supply the function really need to be dependent on whether or not a format for __signature__ has been chosen? Cheers, Nick. P.S. the patch is currently deficient in the test and documentation departments, so it's formally incomplete, but I figure it makes sense to finish the discussion before sorting those out. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From gmccaughan at synaptics-uk.com Tue Mar 15 13:41:33 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue Mar 15 13:42:05 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <1f7befae050314161678706275@mail.gmail.com> Message-ID: <200503151241.33304.gmccaughan@synaptics-uk.com> On Tuesday 2005-03-15 01:57, Guido van Rossum wrote: > I think the conclusion should be that sum() is sufficiently > constrained by backwards compatibility to make "fixing" it impossible > before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is > only used for empty lists. Why's that better than having the 2nd argument only *needed* for empty lists, but *used* whenever it's given? -- g From mcherm at mcherm.com Tue Mar 15 14:12:57 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Mar 15 14:12:58 2005 Subject: [Python-Dev] Rationale for sum()'s design? Message-ID: <20050315051257.a9dv3dxzbhk4sswo@mcherm.com> Tim writes: > I'd personally be delighted if sum() never worked on anything other > than numbers. Guido writes: > I think the conclusion should be that sum() is sufficiently > constrained by backwards compatibility to make "fixing" it impossible > before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is > only used for empty lists. Guido, I find myself far more convinced by Tim's suggestion than yours. This all started with your claim that sum() was a replacement for most reduce() uses. We all agree that it works fine for summing up numbers. We all agree it DOESN'T work for any case where the operator isn't __add__. So trying to make it work well for non-numbers is just arguing over whether it replaces 75% of all reduce() uses, or 85% of them. Who cares? Remember, *ANY* use of sum() can be replaced by a simple, readable, three line for loop: total = for i in total += i And for that matter, other uses of reduce are similarly easy to write as a loop. I'm not saying reduce() is useless, just that it's not worthwhile to expend effort trying to cover every last use case with things like sum() or product() -- one can always write the loop instead. What I think is MORE important is that sum() have predictable behavior in the degenerate cases. And by "predictable", I mean predictable by a newbie user or one someone who hasn't looked at the docs for sum() for the past year and mostly remembers what it does only because the name is so blatantly obvious. _IF_ we follow Tim's advice and think of sum() as designed to add up numbers (as the name implies), then the corner cases are obvious: summing a list of 1 item should return the item, and summing a list of 0 items should return 0. If we think of sum as a form of reduce() that operates only on the __add__ operator, then we need to make other choices for these corner cases, and the people using it to add numbers will suffer the small indignity of having to recall how the corner case works. Honestly, the only case I even remotely care about is concatenating strings -- as far as _I_ am concerned, everyone else can just write out the loop or provide their own "sum_matrix()" method. And if I understand correctly, there's no plan to make sum() "just work" for concatenating strings (besides which, we'd still need to handle the empty list). So I'm in favor of REMOVING the second argument of sum() for python 3000, unless it was kept purely for "backward compatibility" reasons, which would be defeated by changing it's behavior. (And if this doesn't convince you, then so be it... this isn't a particularly important issue.) -- Michael Chermside From gvanrossum at gmail.com Tue Mar 15 16:47:20 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue Mar 15 16:48:17 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <4236D333.3070007@iinet.net.au> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> Message-ID: On Tue, 15 Mar 2005 22:21:07 +1000, Nick Coghlan wrote: > Guido van Rossum wrote: > > I think the conclusion should be that sum() is sufficiently > > constrained by backwards compatibility to make "fixing" it impossible > > before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is > > only used for empty lists. > > Two questions about this: > > 1. When omitting the second argument, would supplying an empty list return 0, > None or raise an exception? Good question... > The last seems most reasonable, as otherwise the function is guessing about what > the programmer wants. OTOH 0 is more compatible and None is a strong candidate too... > 2. How would the initial value that forms the basis of summation be built for > non-empty sequences? Here's you're way off. There's never any use of "+=", so never any need to create a new object. The algorithm I had in mind was: - if empty, return 2nd arg - if one item, return that - if more than one item (A, B, C, ...) return (...((A + B) + C) + ...) But I'm not so sure now. Thinking ahead to generic types, I'd like the full signature to be: def sum(seq: sequence[T], initial: T = 0) -> T. and that's exactly what it is today. Conclusion: sum() is perfect after all! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From FBatista at uniFON.com.ar Tue Mar 15 16:58:51 2005 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Mar 15 16:59:10 2005 Subject: [Python-Dev] Rationale for sum()'s design? Message-ID: [Guido van Rossum] #- > 1. When omitting the second argument, would supplying an #- empty list return 0, #- > None or raise an exception? #- #- Good question... I'd go for None: - Is a good default for a non-existing argument. - If won't get None from sum() in other case (unless you do sum(None), but you should be aware of that). . Facundo Bit?cora De Vuelo: http://www.taniquetil.com.ar/plog PyAr - Python Argentina: http://pyar.decode.com.ar/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050315/58d3f3e4/attachment.htm From p.f.moore at gmail.com Tue Mar 15 17:26:10 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Tue Mar 15 17:26:42 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> Message-ID: <79990c6b050315082677c1f3e2@mail.gmail.com> On Mon, 14 Mar 2005 17:57:42 -0800, Guido van Rossum wrote: > Unfortunately this started when I claimed in my blog that sum() was a > replacement for 80% of all reduce() uses. That's probably where the error lies, then. When it was introduced, sum() was for summing numbers. Whether summing numbers is 80% of all uses of reduce or not is debatable, but I can't say I care. But I *do* care that this claim was taken as meaning that sum() was *intended* specifically to replace 80% of all reduce() uses - a clear misinterpretation. > I think the conclusion should be that sum() is sufficiently > constrained by backwards compatibility to make "fixing" it impossible > before 3.0. It seems wrong to be talking about "fixing" sum so soon after it was introduced. Paul. From tjreedy at udel.edu Tue Mar 15 17:25:52 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Tue Mar 15 17:37:09 2005 Subject: [Python-Dev] Re: Rationale for sum()'s design? References: <20050315051257.a9dv3dxzbhk4sswo@mcherm.com> Message-ID: "Michael Chermside" wrote in message news:20050315051257.a9dv3dxzbhk4sswo@mcherm.com... > Tim writes: >> I'd personally be delighted if sum() never worked on anything other >> than numbers. > > Guido writes: >> I think the conclusion should be that sum() is sufficiently >> constrained by backwards compatibility to make "fixing" it impossible >> before 3.0. But in 3.0 I'd like to fix it so that the 2nd argument is >> only used for empty lists. > > Guido, I find myself far more convinced by Tim's suggestion than > yours. I am leaning in that direction too. >This all started with your claim that sum() was a replacement > for most reduce() uses. To me, the proper general replacement for the doubly broken builtin reduce would be a properly-defined functional.reduce (a subject for another post), not a bloated sum ;-) > We all agree that it works fine for summing up > numbers. We all agree it DOESN'T work for any case where the operator > isn't __add__. However, __add__ can be defined to mean anything. I'd hate to see people choosing '+' as an operator symbol merely to get reduction disguised as summation. [snip] > What I think is MORE important is that sum() have predictable behavior > in the degenerate cases. And by "predictable", I mean predictable by > a newbie user or one someone who hasn't looked at the docs for sum() > for the past year and mostly remembers what it does only because the > name is so blatantly obvious. Nick's very unmemorable a,b,c,d rules shows the complications that reduction-disguised-as-summation can lead to. Terry J. Reedy From gvanrossum at gmail.com Tue Mar 15 18:07:50 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue Mar 15 18:07:59 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <79990c6b050315082677c1f3e2@mail.gmail.com> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <79990c6b050315082677c1f3e2@mail.gmail.com> Message-ID: [Guido] > > Unfortunately this started when I claimed in my blog that sum() was a > > replacement for 80% of all reduce() uses. [Paul] > That's probably where the error lies, then. When it was introduced, > sum() was for summing numbers. Um, Python doesn't provide a lot of special support for numbers apart from literals -- sum() should support everything that supports the "+" operator, just like min() and max() support everything that supports comparison, etc. > Whether summing numbers is 80% of all > uses of reduce or not is debatable, but I can't say I care. But I *do* > care that this claim was taken as meaning that sum() was *intended* > specifically to replace 80% of all reduce() uses - a clear > misinterpretation. Not intended, but it happens to address many cases. > > I think the conclusion should be that sum() is sufficiently > > constrained by backwards compatibility to make "fixing" it impossible > > before 3.0. > > It seems wrong to be talking about "fixing" sum so soon after it was introduced. 3.0 is soon?!? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From FBatista at uniFON.com.ar Tue Mar 15 18:25:55 2005 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Mar 15 18:26:20 2005 Subject: [Python-Dev] Rationale for sum()'s design? Message-ID: [Guido van Rossum] #- 3.0 is soon?!? *You* should answer this, ;) . Facundo Bit?cora De Vuelo: http://www.taniquetil.com.ar/plog PyAr - Python Argentina: http://pyar.decode.com.ar/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA. La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20050315/24011ad1/attachment.htm From tim.peters at gmail.com Tue Mar 15 18:41:48 2005 From: tim.peters at gmail.com (Tim Peters) Date: Tue Mar 15 18:41:55 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <79990c6b050315082677c1f3e2@mail.gmail.com> Message-ID: <1f7befae05031509416c256ddf@mail.gmail.com> [Guido van Rossum] > Um, Python doesn't provide a lot of special support for numbers apart > from literals -- sum() should support everything that supports the "+" > operator, just like min() and max() support everything that supports > comparison, etc. The discussion in the patch that introduced it may be illuminating: http://www.python.org/sf/724936 >From your (Guido's) first comment there, it seems clear that sum() was originally intended only for numbers. Then it got generalized. Then sequences of strings specifically were disallowed. Then it was suggested that mention of a sequence of lists or tuples be removed from the docs, and that datetime.timedelta() be used in examples where "0" didn't make sense as the identity. Then Alex changed it to disallow any sequence of sequences. Then you suggested either specifically disallowing only sequences of lists, tuples or strings (but allowing all other sequence types as elements), _or_ going back to disallowing only sequences of strings. Alex took the latter suggestion, and that's where it ended. The closest thing to a design rationale I found is Guido's Pronouncement here, and I think it covers most issues raised this time around: http://mail.python.org/pipermail/python-dev/2003-April/034853.html The optional argument was my fault. The rest was Guido's : If we add an optional argument for Tim's use case, it could be used in two different ways: (1) only when the sequence is empty, (2) always used as a starting point. IMO (2) is more useful and more consistent. Nobody opposed that. I remember silently agreeing at the time, just because #2 had precedent in other languages, and seemed easiest to explain: sum(seq, start=0) same-as start + seq[0] + seq[1] + ... From python at rcn.com Tue Mar 15 18:38:03 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue Mar 15 18:42:32 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: Message-ID: <000001c52985$be01b7e0$05b52c81@oemcomputer> > OTOH 0 is more compatible and None is a strong candidate too... None is useless as a default return value for summation. Any code outside the summation would have to explicitly test for that value. Stick with zero. Theoretical musings are no reason to make this function hard to use. > But I'm not so sure now. Thinking ahead to generic types, I'd like the > full signature to be: > > def sum(seq: sequence[T], initial: T = 0) -> T. > > and that's exactly what it is today. Conclusion: sum() is perfect after > all! +1 This works for me! I use sum() quite a bit and have had no disappointments with the current API. Raymond From bac at OCF.Berkeley.EDU Tue Mar 15 20:38:11 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Mar 15 20:38:17 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <4236CC0F.1080305@iinet.net.au> References: <4234D941.9040605@canterbury.ac.nz> <42356588.2050508@iinet.net.au> <84d9fc0a691684a9870db60d79d725a2@yahoo.com> <42359A2C.4010708@iinet.net.au> <4236CC0F.1080305@iinet.net.au> Message-ID: <423739A3.1090308@ocf.berkeley.edu> Nick Coghlan wrote: > Guido van Rossum wrote: > >> On Tue, 15 Mar 2005 00:05:32 +1000, Nick Coghlan >> wrote: >> >>> ... try: >>> ... value += first >>> ... except TypeError: >>> ... raise TypeError("Cannot add first element %r to initial value >>> %r" % (first, value)) >> >> >> >> No, no, no! NO! Never catch a general exception like that and replace >> it with one of your own. That can cause hours of debugging pain later >> on when the type error is deep down in the bowels of the += operation >> (or perhaps deep down inside something *it* invoked). > > > Ouch. Obviously, I hadn't thought about that. . . > > Wasn't there a concept floating around some time ago to support > exception chaining, so additional context information could be added to > a thrown exception, without losing the location of the original problem? > Yes, but it didn't go anywhere. See http://www.python.org/dev/summary/2003-06-01_2003-06-30.html#pep-317 for the summary. -Brett From sabbey at u.washington.edu Tue Mar 15 21:41:03 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Tue Mar 15 21:41:08 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <4236CCDE.40603@strakt.com> References: <4234DBA1.7010304@canterbury.ac.nz> <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> <42362E29.5080502@strakt.com> <4236CCDE.40603@strakt.com> Message-ID: Samuele Pedroni wrote: > My point is that a suite-based syntax > can only be a half substitute for lambda and anyway requiring a suite > seems overkill and unnatural for the just 1 expression case, for example > predicates. IOW a suite-based syntax is not a lambda killer in itself, I > would not try to stress that point. I see your point (also I see Greg Ewing's related point). > multiple dispatch has nothing to do with syntax, in fact usual call > syntax is sufficient, and people do use multiple dispatch sometimes, > and decorators now can be even used to sugar up the definition side > of it. But one needs to use decorators or some other mechanism for the sugar, that is all I intended the phrase "does not give syntactic support" to mean. Perhaps "syntactic sugar" is the correct term to have used. >> for something that would be rarely used, I do not think > > well that's up to discussion to discover ok > well, but this is stated without even trying to come up with a syntax > for that case. Notice that the first time around Guido himself would > have preferred if achievable a multithunk syntax, he obviously can have > changed his mind. But, yes, syntax vs expressivity is the key issue here. Ok. Allow me to try. Up to a choice of (or existence of!) keywords, the simplest to me is: def add(thunk1, thunk2, other): print thunk1(1,2) + thunk2(3,4) + other with x,y from add(100): value x*y also a,b: # yikes?? value a*b # this is thunk2 -Brian From martin at v.loewis.de Tue Mar 15 21:45:58 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Mar 15 21:46:06 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <63551edc689e07481145277c661bd5ec@xs4all.nl> References: <4233E476.9030705@canterbury.ac.nz> <200503140957.47849.gmccaughan@synaptics-uk.com> <63551edc689e07481145277c661bd5ec@xs4all.nl> Message-ID: <42374986.8030502@v.loewis.de> Eric Nieuwland wrote: > The full syntax is: > [ f(x) for x in seq if pred(x) ] > being allowed to write 'x' instead of 'identity(x)' is already a > shortcut, just as dropping the conditional part. That's not the full syntax. The full syntax is [ for in ] where can be an arbitrary expression: and, or, lambda, +, -, ... can be a list of expression, except for boolean and relational expressions (but I think this is further constrained semantically) list a list of tests is optional, and can be another for or if block So a more complex example is [ lambda a: a[x]+y*z for x,y in A for z in B if x > z] (Not that this does what you think it does :-) So that you can write f(x) is just a tiny special case. Regards, Martin From eric.nieuwland at xs4all.nl Tue Mar 15 22:46:36 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Tue Mar 15 22:46:45 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <42374986.8030502@v.loewis.de> References: <4233E476.9030705@canterbury.ac.nz> <200503140957.47849.gmccaughan@synaptics-uk.com> <63551edc689e07481145277c661bd5ec@xs4all.nl> <42374986.8030502@v.loewis.de> Message-ID: <24c8e6d6d66a9c727c5ec0e1eaadec48@xs4all.nl> Martin v. L?wis wrote: > That's not the full syntax. The full syntax is > > [ for in ] > > where > > can be an arbitrary expression: and, or, lambda, +, -, ... > can be a list of expression, except for boolean and > relational expressions (but I think this is further constrained > semantically) > list a list of tests > is optional, and can be another for or if block Aren't these names a bit mixed up w.r.t. what's in that position? As far as I know is not a test but a function as it produces any value not just True/False is a list of names, not arbitrary expressions is anything which will produce an iterator Or so I've understood so far. > So a more complex example is > > [ lambda a: a[x]+y*z for x,y in A for z in B if x > z] I'm schocked ;-) --eric From bac at OCF.Berkeley.EDU Tue Mar 15 22:55:40 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Mar 15 22:55:47 2005 Subject: [Python-Dev] comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <24c8e6d6d66a9c727c5ec0e1eaadec48@xs4all.nl> References: <4233E476.9030705@canterbury.ac.nz> <200503140957.47849.gmccaughan@synaptics-uk.com> <63551edc689e07481145277c661bd5ec@xs4all.nl> <42374986.8030502@v.loewis.de> <24c8e6d6d66a9c727c5ec0e1eaadec48@xs4all.nl> Message-ID: <423759DC.7010609@ocf.berkeley.edu> Eric Nieuwland wrote: > Martin v. L?wis wrote: > >> That's not the full syntax. The full syntax is >> >> [ for in ] >> >> where >> >> can be an arbitrary expression: and, or, lambda, +, -, ... >> can be a list of expression, except for boolean and >> relational expressions (but I think this is further constrained >> semantically) >> list a list of tests >> is optional, and can be another for or if block > > > Aren't these names a bit mixed up w.r.t. what's in that position? The names come directly from the grammar file (Grammar/Grammar) and thus have multiple uses. Basically ignore the actual names and just consider them placeholders. -Brett From aleaxit at yahoo.com Tue Mar 15 23:01:48 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Tue Mar 15 22:59:45 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <1f7befae050314161678706275@mail.gmail.com> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> Message-ID: On Mar 15, 2005, at 01:16, Tim Peters wrote: > [Eric Nieuwland] >>> Perhaps the second argument should not be optional to emphasise this. >>> After all, there's much more to sum() than numbers. > > [Greg Ewing] >> I think practicality beats purity here. Using it on >> numbers is surely an extremely common case. > > I'd personally be delighted if sum() never worked on anything other > than numbers. That makes it easy to understand, easy to document, > easy to remember, obvious at first sight, and straightforward to > implement. Everything a framework isn't, but it's not a bad thing to > have *something* that actually means exactly what it looks like it > says <0.5 wink>. I'm reasonably often using sum on lists of datetime.timedelta instances, "durations", which ARE summable just like numbers even though they aren't numbers. I believe everything else for which I've used sum in production code DOES come under the general concept of "numbers", in particular X+0 == X. Unfortunately, this equation doesn't hold when X is a timedelta, as X+0 raises an exception then. Alex From martin at v.loewis.de Tue Mar 15 23:14:24 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Mar 15 23:14:27 2005 Subject: [Python-Dev] comprehension abbreviation In-Reply-To: <24c8e6d6d66a9c727c5ec0e1eaadec48@xs4all.nl> References: <4233E476.9030705@canterbury.ac.nz> <200503140957.47849.gmccaughan@synaptics-uk.com> <63551edc689e07481145277c661bd5ec@xs4all.nl> <42374986.8030502@v.loewis.de> <24c8e6d6d66a9c727c5ec0e1eaadec48@xs4all.nl> Message-ID: <42375E40.5060302@v.loewis.de> Eric Nieuwland wrote: >> [ for in ] > Aren't these names a bit mixed up w.r.t. what's in that position? It comes more-or-less straight out of Grammar/Grammar, so: no, I don't think so. > As far as I know > is not a test but a function as it produces any value not just > True/False To quote more of Grammar/Grammar, it is test: and_test ('or' and_test)* | lambdef and_test: not_test ('and' not_test)* not_test: 'not' not_test | comparison comparison: expr (comp_op expr)* comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not' expr: xor_expr ('|' xor_expr)* ... power: atom trailer* ['**' factor] trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME So a function call is a power is a factor is a term is an arith_expr is a shift_expr is an and_expr is a xor_expr is an expr is a comparison is a not_test is an and_test is a test. > is a list of names, not arbitrary expressions No, it's more than that. I can also be a, (b, c, (d, e)) But as I said: there are further (semantical) constraints what kind of expression you can have there. Regards, Martin From jimjjewett at gmail.com Wed Mar 16 02:41:41 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed Mar 16 02:41:44 2005 Subject: [Python-Dev] RE: code blocks using 'for' loops and generators Message-ID: It may be time to PEP (or re-PEP), if only to clarify what people are actually asking for. Brian Sabbey's example from message http://mail.python.org/pipermail/python-dev/2005-March/052202.html *seems* reasonably clear, but I don't see how it relates in any way to "for" loops or generators, except as one (but not the only) use case. def add(thunk1, thunk2, other): print thunk1(1,2) + thunk2(3,4) + other with x,y from add(100): value x*y also a,b: # yikes?? value a*b # this is thunk2 To make my own understanding explicit: (1) Calls for "Ruby blocks" or "thunks" are basically calls for placeholders in a function. These placeholders will be filled with code from someplace else, but will execute in the function's own local namespace. (2) A function as a parameter isn't good enough, because the passed-in function can't see bindings in the surrounding larger function. (It still sees the lexical scope it which it was defined.) (3) A function as a parameter isn't good enough, because the passed-in function can't modify bindings in the surrounding larger function. (4) A thunk could be used today be creating a string (rather than a pre-compiled function) and substituting in the thunk's string (rather than the code to which it would compile) before execution, though this would be ugly. (5) A thunk *might* be do-able with exec or eval if a function's locals were still a dictionary. (6) This has nothing whatsoever to do with for loops or generators, except as a common use case. In particular, thunks may be used to lock/unlock or unwind-protect some resource. Generators are not the only place this is needed, but they have made the "what do you mean, __del__ might never get called?" problem even worse, and Ruby often solves these with a Ruby Block. (7) A __leave__ or __exit__ special method really turns into another name for __del__. It might still be useful if it came with semantics of "I explicitly don't care what order cycles are broken; call me as soon as my object is garbage and who cares if it gets resurrected." There have been recipes (from Tim?) for emulating this by using a proxy to the resource, so that no __del__ cycles can form. -jJ From nidoizo at yahoo.com Wed Mar 16 02:49:35 2005 From: nidoizo at yahoo.com (Nicolas Fleury) Date: Wed Mar 16 02:49:52 2005 Subject: [Python-Dev] Re: Rationale for sum()'s design? In-Reply-To: <4236CC0F.1080305@iinet.net.au> References: <4234D941.9040605@canterbury.ac.nz> <42356588.2050508@iinet.net.au> <84d9fc0a691684a9870db60d79d725a2@yahoo.com> <42359A2C.4010708@iinet.net.au> <4236CC0F.1080305@iinet.net.au> Message-ID: Nick Coghlan wrote: > Guido van Rossum wrote: >> No, no, no! NO! Never catch a general exception like that and replace >> it with one of your own. That can cause hours of debugging pain later >> on when the type error is deep down in the bowels of the += operation >> (or perhaps deep down inside something *it* invoked). > > > Ouch. Obviously, I hadn't thought about that. . . > > Wasn't there a concept floating around some time ago to support > exception chaining, so additional context information could be added to > a thrown exception, without losing the location of the original problem? Like that? try : (...) except Exception, exception: # Keep the current exception stack and add information to exception. raise Exception( 'Additional exception info... :\n' + sys.exc_info()[0].__name__ + ': ' + str(exception)), None, sys.exc_info()[-1] I made for myself a reraise function that I use a lot for that purpose, but it has some limitations. A standard way to do it would be great. Regards, Nicolas From jimjjewett at gmail.com Wed Mar 16 02:54:07 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed Mar 16 02:54:10 2005 Subject: [Python-Dev] Re: comprehension abbreviation (was: Adding any() and all()) Message-ID: Gareth McCaughan wrote: > Some bit of my brain is convinced that [x in stuff if condition] > is the Right Syntax and keeps making me type it even though > I know it doesn't work. (and I agree with Gareth) On Monday 2005-03-14 12:42, Eric Nieuwland wrote: > The full syntax is: > [ f(x) for x in seq if pred(x) ] > being allowed to write 'x' instead of 'identity(x)' is already a > shortcut, just as dropping the conditional part. I think this is the heart of the disagreement. Mentally, I'm not collecting some function of x (which happens to be identity). I am filtering an existing set. Being able to collect f(x) instead is just a useful but hackish shortcut. Gareth again: > and in fact no set theorist would be at all troubled by seeing > { x in S : predicate(x) } > which is the nearest equivalent in mathematical notation > for the abbreviated comprehension expressions being discussed. Again, I agree. I think that is what I am unconsciously writing, by translating the ":" into "if" -jJ From pedronis at strakt.com Wed Mar 16 03:24:28 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Wed Mar 16 03:23:20 2005 Subject: [Python-Dev] RE: code blocks using 'for' loops and generators In-Reply-To: References: Message-ID: <423798DC.5080100@strakt.com> Jim Jewett wrote: > It may be time to PEP (or re-PEP), if only to clarify what people are > actually asking for. > > Brian Sabbey's example from message > http://mail.python.org/pipermail/python-dev/2005-March/052202.html > *seems* reasonably clear, but I don't see how it relates in any way to > "for" loops or generators, except as one (but not the only) use case. > > def add(thunk1, thunk2, other): > print thunk1(1,2) + thunk2(3,4) + other > > with x,y from add(100): > value x*y > also a,b: # yikes?? > value a*b # this is thunk2 > > To make my own understanding explicit: > > (1) Calls for "Ruby blocks" or "thunks" are basically calls for > placeholders in a function. These placeholders will be filled > with code from someplace else, but will execute in the function's > own local namespace. > > (2) A function as a parameter isn't good enough, because the > passed-in function can't see bindings in the surrounding larger > function. (It still sees the lexical scope it which it was defined.) > > (3) A function as a parameter isn't good enough, because the > passed-in function can't modify bindings in the surrounding > larger function. > > (4) A thunk could be used today be creating a string (rather than > a pre-compiled function) and substituting in the thunk's string > (rather than the code to which it would compile) before execution, > though this would be ugly. > > (5) A thunk *might* be do-able with exec or eval if a function's > locals were still a dictionary. > notice that while closures with the current semantics of def cannot rebind, the internal mechanism for closures is perfectly capable of supporting rebinding. So thunks could be function objects. > (6) This has nothing whatsoever to do with for loops or generators, > except as a common use case. In particular, thunks may be used > to lock/unlock or unwind-protect some resource. Generators are > not the only place this is needed, but they have made the "what > do you mean, __del__ might never get called?" problem even worse, > and Ruby often solves these with a Ruby Block. > > (7) A __leave__ or __exit__ special method really turns into another > name for __del__. It might still be useful if it came with semantics > of "I explicitly don't care what order cycles are broken; call me as > soon as my object is garbage and who cares if it gets resurrected." > There have been recipes (from Tim?) for emulating this by using > a proxy to the resource, so that no __del__ cycles can form. > > -jJ > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/pedronis%40strakt.com From tim.peters at gmail.com Wed Mar 16 03:23:47 2005 From: tim.peters at gmail.com (Tim Peters) Date: Wed Mar 16 03:23:57 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> Message-ID: <1f7befae05031518235903a957@mail.gmail.com> [Alex Martelli] > I'm reasonably often using sum on lists of datetime.timedelta > instances, "durations", which ARE summable just like numbers even > though they aren't numbers. I believe everything else for which I've > used sum in production code DOES come under the general concept of > "numbers", in particular X+0 == X. Unfortunately, this equation > doesn't hold when X is a tiwmedelta, as X+0 raises an exception then. I count timedeltas "as numbers" too -- or at least I did when sum() was being designed, cuz I asked for the optional start= argument, and precisely in order to sum things like this (timedelta was indeed the driving example at the time). I can't say it bothers me to specify an appropriate identity element when 0 is inappropriate. If it switched to ignoring the start= argument unless the sequence was empty, that would be OK too, although I think sum(seq, start=0) same-as start + seq[0] + seq[1] + ... will always be easiest to explain, so all else being equal I prefer current behavior. The number of times I've passed a sequence to sum() with elements that were lists, tuples, or any other kind of object with a "concatenate" meaning for __add__ is zero so far. From pje at telecommunity.com Tue Mar 15 18:04:55 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Mar 16 05:30:20 2005 Subject: [Python-Dev] (no subject) In-Reply-To: <4236D6D7.2060601@iinet.net.au> References: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050315120343.0309fec0@mail.telecommunity.com> At 10:36 PM 3/15/05 +1000, Nick Coghlan wrote: >Phillip J. Eby wrote: >>I discussed this approach with Guido in private e-mail a few months back >>during discussion about an article I was writing for DDJ about >>decorators. We also discussed something very similar to 'update_meta()', >>but never settled on a name. Originally he wanted me to PEP the whole >>thing, but he wanted it to include optional type declaration info, so you >>can probably see why I haven't done anything on that yet. :) >>However, if we can define a __signature__ format that allows for type >>declaration, I imagine there'd be little problem with moving forward on it. > >But one of the reasons for providing 'update_meta' is so that additional >metadata (like __signature__) can later be added transparently. Yes, exactly. >Does deciding whether or not to supply the function really need to be >dependent on whether or not a format for __signature__ has been chosen? Um, no. Why would you think that? From jcarlson at uci.edu Wed Mar 16 09:39:44 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed Mar 16 09:42:05 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <42361811.5010707@canterbury.ac.nz> References: <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> Message-ID: <20050315185411.A496.JCARLSON@uci.edu> Greg Ewing wrote: > > Josiah Carlson wrote: > > > Since PEP 310 was already mentioned, can we just say that the discussion > > can be boiled down to different ways of spelling __enter__/__exit__ from > > PEP 310? > > It's not quite the same thing. PEP 310 suggests a mechanism > with a fixed control flow -- do something on entry, do the > code block, do something on exit. A general block-passing > mechanism would give complete control to the function as > to when and how to call the block. I would like to ask a question. Does Python want or need Ruby code blocks? I ask because that is what is being offered to cure what ails us. Don't get me wrong, I'm sure code blocks can solve quite a few problems (and is used as such in Ruby), but I'm not convinced that it is the solution for Python. Any manual on Ruby will invariably discuss code blocks as one of the most powerful features Ruby has to offer. Sounds great. Sounds like a great big sledgehammer that can be used to do a bunch of things...so what is currently being proposed as a use for them, and what can't they do (that would be really nice)? They are being offered, right now, as a setup and finalization mechanism. Essentially a way of allowing people to wrap their own blocks of code in custom try/finally code, or whatever their minds can think up. Ok, __enter__/__exit__ offers that. What else? If you were to pass your generator as a code block, then you could finalize a generator [1], and even raise exceptions in your code block, but it still wouldn't allow one to pass exceptions into a currently running generator (a long standing problem such that if we could, then we would get generator finalization for free [2]). What else? Let's dig back into the python-dev archives... http://mail.python.org/pipermail/python-dev/2003-February/032739.html Guido: > - synchronized-like, where the block should connect to its environment > > - property-like, where the block should introduce a new scope, and the > caller should be able to inspect or control that scope's namespace The synchronized-like thing is the custom try/finally, aka __enter__/__exit__ as specified in PEP 310. The property-like thing was perhaps to be an easier way to generate properties, which fairly quickly fell to the wayside in discussion, seemingly because people didn't see a need to add thunks for this. Later XML DOM parsing came into the discussion, and was quickly dismissed as being not possible due to the Python parser's limited lookahead. Someone else mentioned that thunks could be used to generate a switch statement, but no elaboration was done, and no PEP was actually written (switch has also had its own PEP, and even talk of a peephole optimization for certain if/elif/else blocks to be made into dictionary lookups...) So where has all this reading gotten me? To the point that I believe previous discussion had concluded that Ruby-style code blocks have little use in Python. *shrug* Greg: > However, it's possible that if generators were enhanced > with some means of allowing yield inside try-finally, > then generators plus PEP 310 would cover most use cases: > for-loops and generators when you want to loop, and > PEP 310 when you don't. > > So rather than coming up with new syntax at this point, > perhaps it would be better to concentrate on the problem > of yield inside try-finally. Even if the finally can't > be guaranteed to run under all conditions, I think it > would be useful if it could be arranged so that > > for x in somegenerator(): > ... > raise Blather > ... > > would caused any finallies that the generator was suspended > inside to be executed. Then the semantics would be the > same as if the for-loop-over-generator were implemented by > passing a thunk to a function that calls it repeatedly. PEP 288 using gen.throw() to trigger early finalization, with try/finally allowed. - Josiah [1] thunk limitations with generator exceptions def finalize(thunk, seq): #setup try: thunk(seq) finally: #finalize def foo(seq): with s=finalize(seq): for i in s: #do something with i in finalize's namespace... yield f(i) #raise an exception if desired for j in foo(seq): #do something with j #can't raise an exception in foo or finalize's executing code [2] PEP 288 goodies, with try/finally allowed (with code duplication, or refactoring, this can be done with try/except/else) def gen(): #setup try: #yields here finally: #finalization raise a = gen() for i in a: if some condition: #stop the generator and call cleanup code a.throw(StopIteration) #body From python at rcn.com Wed Mar 16 12:19:54 2005 From: python at rcn.com (Raymond Hettinger) Date: Wed Mar 16 12:24:21 2005 Subject: [Python-Dev] itertools.walk() Message-ID: <000401c52a1a$141899c0$fc33c797@oemcomputer> Some folks on comp.lang.python have been pushing for itertools to include a flatten() operation. Unless you guys have some thoughts on the subject, I'm inclined to accept the request. Rather than calling it flatten(), it would be called "walk" and provide a generalized capability to descend through nested iterables (similar to what os.walk does for directories). The one wrinkle is having a stoplist argument to specify types that should be considered atomic eventhough they might be iterable (strings for example). Raymond Hettinger From ncoghlan at iinet.net.au Wed Mar 16 12:43:39 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Mar 16 12:43:44 2005 Subject: [Python-Dev] (no subject) In-Reply-To: <5.1.1.6.0.20050315120343.0309fec0@mail.telecommunity.com> References: <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> <5.1.1.6.0.20050314101958.02116c60@mail.telecommunity.com> <5.1.1.6.0.20050315120343.0309fec0@mail.telecommunity.com> Message-ID: <42381BEB.2010504@iinet.net.au> Phillip J. Eby wrote: > At 10:36 PM 3/15/05 +1000, Nick Coghlan wrote: >> Does deciding whether or not to supply the function really need to be >> dependent on whether or not a format for __signature__ has been chosen? > > Um, no. Why would you think that? Pronoun confusion. I interpreted an 'it' in your message as referring to update_meta, but I now realise you only meant __signature__ :) Cheers, Nick -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From pedronis at strakt.com Wed Mar 16 13:03:03 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Wed Mar 16 13:02:03 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <20050315185411.A496.JCARLSON@uci.edu> References: <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> <20050315185411.A496.JCARLSON@uci.edu> Message-ID: <42382077.6010104@strakt.com> Josiah Carlson wrote: > Greg Ewing wrote: > >>Josiah Carlson wrote: >> >> >>>Since PEP 310 was already mentioned, can we just say that the discussion >>>can be boiled down to different ways of spelling __enter__/__exit__ from >>>PEP 310? >> >>It's not quite the same thing. PEP 310 suggests a mechanism >>with a fixed control flow -- do something on entry, do the >>code block, do something on exit. A general block-passing >>mechanism would give complete control to the function as >>to when and how to call the block. > > > I would like to ask a question. Does Python want or need Ruby code > blocks? I ask because that is what is being offered to cure what ails > us. Don't get me wrong, I'm sure code blocks can solve quite a few > problems (and is used as such in Ruby), but I'm not convinced that it is > the solution for Python. > > Any manual on Ruby will invariably discuss code blocks as one of the > most powerful features Ruby has to offer. Sounds great. Sounds like a > great big sledgehammer that can be used to do a bunch of things...so > what is currently being proposed as a use for them, and what can't they > do (that would be really nice)? > > They are being offered, right now, as a setup and finalization mechanism. > Essentially a way of allowing people to wrap their own blocks of code in > custom try/finally code, or whatever their minds can think up. Ok, > __enter__/__exit__ offers that. What else? > > If you were to pass your generator as a code block, then you could > finalize a generator [1], and even raise exceptions in your code block, > but it still wouldn't allow one to pass exceptions into a currently > running generator (a long standing problem such that if we could, then > we would get generator finalization for free [2]). > > > What else? Let's dig back into the python-dev archives... > http://mail.python.org/pipermail/python-dev/2003-February/032739.html > > Guido: > >>- synchronized-like, where the block should connect to its environment >> >>- property-like, where the block should introduce a new scope, and the >> caller should be able to inspect or control that scope's namespace > > > The synchronized-like thing is the custom try/finally, aka > __enter__/__exit__ as specified in PEP 310. > > The property-like thing was perhaps to be an easier way to generate > properties, which fairly quickly fell to the wayside in discussion, > seemingly because people didn't see a need to add thunks for this. > > Later XML DOM parsing came into the discussion, and was quickly > dismissed as being not possible due to the Python parser's limited > lookahead. > > Someone else mentioned that thunks could be used to generate a switch > statement, but no elaboration was done, and no PEP was actually written > (switch has also had its own PEP, and even talk of a peephole > optimization for certain if/elif/else blocks to be made into dictionary > lookups...) > > > > So where has all this reading gotten me? To the point that I believe > previous discussion had concluded that Ruby-style code blocks have > little use in Python. *shrug* > well, I think some people desire a more streamlined way of writing code like: def f(...) ... def g(...) ... x = h(...,f,g) [property, setting up callbacks etc are cases of this] were f,g etc definitions would appear inline and the whole has a statement flavor; because this is Python a form that does not involve a lot parantheses would be nice. Of course if the functions then are allowed to change the surrounding bindings this could be used for resource release issues etc. Notice that decorators basically support a special case of this. But yes, apart for the issue of rebinding (and if one wants non-local returns), this is stricly about sugar. From bob at redivi.com Wed Mar 16 13:22:51 2005 From: bob at redivi.com (Bob Ippolito) Date: Wed Mar 16 13:23:09 2005 Subject: [Python-Dev] itertools.walk() In-Reply-To: <000401c52a1a$141899c0$fc33c797@oemcomputer> References: <000401c52a1a$141899c0$fc33c797@oemcomputer> Message-ID: On Mar 16, 2005, at 6:19, Raymond Hettinger wrote: > Some folks on comp.lang.python have been pushing for itertools to > include a flatten() operation. Unless you guys have some thoughts on > the subject, I'm inclined to accept the request. > > Rather than calling it flatten(), it would be called "walk" and provide > a generalized capability to descend through nested iterables (similar > to > what os.walk does for directories). The one wrinkle is having a > stoplist argument to specify types that should be considered atomic > eventhough they might be iterable (strings for example). You could alternatively give them a way to supply their own "iter" function, like the code I demonstrate below: from itertools import chain def nostring_iter(obj): if isinstance(obj, basestring): raise TypeError return iter(obj) def uniqueiter_factory(iterfunc=nostring_iter): def uniqueiter(obj, uniques={}): iterable = iterfunc(obj) if id(obj) in uniques: raise TypeError uniques[id(obj)] = obj return iterable return uniqueiter # maybe there should be a bfswalk too? def walk(iterable, iterfunc=nostring_iter): iterables = iter((iterable,)) while True: for obj in iterables: try: iterable = iterfunc(obj) except TypeError: yield obj else: iterables = chain(iterable, iterables) break else: break >>> data = [('foo', 'bar'), 'baz', 5] >>> list(walk(data)) ['foo', 'bar', 'baz', 5] >>> list(walk(data, uniqueiter_factory(iter))) ['f', 'o', 'o', 'b', 'a', 'r', 'b', 'a', 'z', 5] >>> data.append(data) >>> list(walk(data, uniqueiter_factory())) ['foo', 'bar', 'baz', 5, [('foo', 'bar'), 'baz', 5, [...]]] -bob From ncoghlan at iinet.net.au Wed Mar 16 13:45:10 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Mar 16 13:45:16 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> Message-ID: <42382A56.6050003@iinet.net.au> Guido van Rossum wrote: >>2. How would the initial value that forms the basis of summation be built for >>non-empty sequences? > > > Here's you're way off. There's never any use of "+=", so never any > need to create a new object. The algorithm I had in mind was: > > - if empty, return 2nd arg > - if one item, return that > - if more than one item (A, B, C, ...) return (...((A + B) + C) + ...) There I go again, missing the obvious and thinking things are more complicated than they really are. . . > But I'm not so sure now. Thinking ahead to generic types, I'd like the > full signature to be: > > def sum(seq: sequence[T], initial: T = 0) -> T. > > and that's exactly what it is today. Conclusion: sum() is perfect after all! So the official verdict is "sum() is mainly intended for numbers, but can be used with other types by supplying a default argument"? I guess that leaves Alex's question of whether or not supplying a string of some description as the initial value can be legitimately translated to: if isinstance(initial, basestring): return initial + type(initial)().join(seq) rather than raising the current TypeError that suggests the programmer may want to rewrite their code. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Wed Mar 16 14:37:24 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Wed Mar 16 14:37:31 2005 Subject: [Python-Dev] itertools.walk() In-Reply-To: References: <000401c52a1a$141899c0$fc33c797@oemcomputer> Message-ID: <42383694.802@iinet.net.au> Bob Ippolito wrote: > On Mar 16, 2005, at 6:19, Raymond Hettinger wrote: > >> Some folks on comp.lang.python have been pushing for itertools to >> include a flatten() operation. Unless you guys have some thoughts on >> the subject, I'm inclined to accept the request. >> >> Rather than calling it flatten(), it would be called "walk" and provide >> a generalized capability to descend through nested iterables (similar to >> what os.walk does for directories). The one wrinkle is having a >> stoplist argument to specify types that should be considered atomic >> eventhough they might be iterable (strings for example). > > > You could alternatively give them a way to supply their own "iter" > function, like the code I demonstrate below: I think the extra flexibility ends up making the function harder to comprehend and use. Here's a version with a simple stoplist: from itertools import chain def walk(iterable, atomic_types=(basestring,)): itr = iter(iterable) while True: for item in itr: if isinstance(item, atomic_types): yield item continue try: subitr = iter(item) except TypeError: yield item else: itr = chain(walk(subitr), itr) break else: break This makes it easy to reclassify certain things like dictionaries or tuples as atomic elements. > # maybe there should be a bfswalk too? By putting the 'walk(subitr)' after the current itr when chaining? If Raymond does decide to go for the flexible approach rather than the simple one, then I'd vote for a full-featured approach like: def walk(iterable, depth_first=True, atomic_types=(basestring,), iter_factory=iter): itr = iter(iterable) while True: for item in itr: if isinstance(item, atomic_types): yield item continue try: subitr = iter_factory(item) except TypeError: yield item else: if depth_first: itr = chain(walk(subitr), itr) else: itr = chain(itr, walk(subitr)) break else: break Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From anthony at python.org Wed Mar 16 14:28:08 2005 From: anthony at python.org (Anthony Baxter) Date: Wed Mar 16 14:48:09 2005 Subject: [Python-Dev] BRANCH FREEZE for 2.4.1rc2 at 2005-03-18 0000 UTC Message-ID: <200503170028.08698.anthony@python.org> The release24-maint branch should be considered FROZEN as at 0000 UTC on 2005-03-18 - in other words, in about 11 hours time. Allegedly this is around 1900 on the 17th for the US East Coast. I'll post again once it's unfrozen. From here, we'll be aiming at a 2.4.1 final for the 29th - straight after PyCon. I'll post again when the branch is available for checkins. And I'll repeat my request for people to be ultra-conservative with checkins to the branch until 2.4.1 final is out. If in doubt, ping me first. Those contact details again: AIM: anthonyatekit Jabber: anthonyatekit@jabber.org IRC: #python-dev on Freenode. Thanks, Anthony -- Anthony Baxter It's never too late to have a happy childhood. From anthony at interlink.com.au Wed Mar 16 15:06:15 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 16 15:06:51 2005 Subject: [Python-Dev] BRANCH FREEZE for 2.4.1rc2 at 2005-03-**17** 0000 UTC In-Reply-To: <200503170028.08698.anthony@python.org> References: <200503170028.08698.anthony@python.org> Message-ID: <200503170106.16875.anthony@interlink.com.au> On Thursday 17 March 2005 00:28, Anthony Baxter wrote: > The release24-maint branch should be considered FROZEN as at 0000 UTC > on 2005-03-18 That should of course be 2005-03-17. -- Anthony Baxter It's never too late to have a happy childhood. From bob at redivi.com Wed Mar 16 15:17:39 2005 From: bob at redivi.com (Bob Ippolito) Date: Wed Mar 16 15:18:00 2005 Subject: [Python-Dev] itertools.walk() In-Reply-To: <42383694.802@iinet.net.au> References: <000401c52a1a$141899c0$fc33c797@oemcomputer> <42383694.802@iinet.net.au> Message-ID: <40d419fccfee93900159318263b990bc@redivi.com> On Mar 16, 2005, at 8:37 AM, Nick Coghlan wrote: > Bob Ippolito wrote: >> On Mar 16, 2005, at 6:19, Raymond Hettinger wrote: >>> Some folks on comp.lang.python have been pushing for itertools to >>> include a flatten() operation. Unless you guys have some thoughts on >>> the subject, I'm inclined to accept the request. >>> >>> Rather than calling it flatten(), it would be called "walk" and >>> provide >>> a generalized capability to descend through nested iterables >>> (similar to >>> what os.walk does for directories). The one wrinkle is having a >>> stoplist argument to specify types that should be considered atomic >>> eventhough they might be iterable (strings for example). >> You could alternatively give them a way to supply their own "iter" >> function, like the code I demonstrate below: > > I think the extra flexibility ends up making the function harder to > comprehend and use. Here's a version with a simple stoplist: ... > This makes it easy to reclassify certain things like dictionaries or > tuples as atomic elements. > > > # maybe there should be a bfswalk too? > > By putting the 'walk(subitr)' after the current itr when chaining? Yeah > If Raymond does decide to go for the flexible approach rather than the > simple one, then I'd vote for a full-featured approach like: I don't mind that at all. It's certainly convenient to have an easy stoplist. The problem with the way you have implemented it is that basestring will cause infinite recursion if you use the built-in iter, so if you provide your own atomic_types, you damn well better remember to add in basestring. > def walk(iterable, depth_first=True, atomic_types=(basestring,), > iter_factory=iter): > itr = iter(iterable) > while True: > for item in itr: > if isinstance(item, atomic_types): > yield item > continue > try: > subitr = iter_factory(item) > except TypeError: > yield item > else: > if depth_first: > itr = chain(walk(subitr), itr) > else: > itr = chain(itr, walk(subitr)) > break > else: > break I'm not sure why it's useful to explode the stack with all that recursion? Mine didn't do that. The control flow is nearly identical, but it looks more fragile (and you would get some really evil stack trace if iter_factory(foo) happened to raise something other than TypeError). -bob From michael.walter at gmail.com Wed Mar 16 15:38:44 2005 From: michael.walter at gmail.com (Michael Walter) Date: Wed Mar 16 15:38:49 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> Message-ID: <877e9a17050316063813cfa73e@mail.gmail.com> On Tue, 15 Mar 2005 07:47:20 -0800, Guido van Rossum wrote: > But I'm not so sure now. Thinking ahead to generic types, I'd like the > full signature to be: > > def sum(seq: sequence[T], initial: T = 0) -> T. Would this _syntax_ work with generic types: def sum(seq: sequence[T], initial: T = T()) -> T. Cheers, Michael From ark-mlist at att.net Wed Mar 16 17:04:37 2005 From: ark-mlist at att.net (Andrew Koenig) Date: Wed Mar 16 17:04:36 2005 Subject: [Python-Dev] Lambda & deferred evaluation (was: Adding any() andall()) In-Reply-To: <42323DFD.4040105@iinet.net.au> Message-ID: <006201c52a41$da9d2620$6402a8c0@arkdesktop> > >>Lambda will be more difficult. Eric Raymond adapted an anti-gun control > >>slogan and said "you can pry lambda out of my cold dead hands." A bunch > >>of folks will sorely miss the ability to create anonymous functions on > >>the fly. When lambda is used for deferred argument evaluation (a la PEP > >>312), the def syntax is a crummy substitute. > > Yeah, I'm with you here. As warty as lambda is, it just is so damn > > convenient some times. I've recently been using it as a companion to > > property(), providing concise definitions of read-only attributes. I'm with you on lambda: I've found a few places where losing it would be a major inconvenience. One example is in a library I wrote to implement the Snobol4 algorithm for recursive-descent pattern matching. Using that algorithm relies heavily on the ability to specify subexpressions for "deferred evaluation" -- their values must not be computed until they are actually needed. For example, here is a mini-specification for arithmetic expressions in Snobol4: id = any(letters) span(letters digits) primary = id | '(' *expr ')' factor = primary arbno(any("*/") primary) expr = factor arbno(any("+-") factor) This should be fairly straightforward once you realize that "arbno" is like a regular-expression *, blank is used for concatenation, and | (meaning "or") binds less tightly than blank. The whole definition depends on *expr, which is a request for deferred evaluation of expr. In other words, it refers to the value of expr when encountered during matching, rather than the (null) value it has when the assignment to primary is evaluated. Through appropriate use of overloading, I have been able to translate this code into the following Python: id = any(letters) + span(letters + digits) primary = id | '(' + defer(lambda: expr) + ')' factor = primary + arbno(any("*/") + primary) expr = factor + arbno(any("+-") + factor) I do not want to have to rewrite this code as: def defer_expr: return expr id = any(letters) + span(letters + digits) primary = id | '(' + defer(defer_expr) + ')' factor = primary + arbno(any("*/") + primary) expr = factor + arbno(any("+-") + factor) because I do not want to have to make up a new name, and because in practice, I've seen patterns in which there are many deferred evaluations, not just one as in this example. In other words, I want deferred evaluation to remain a conceptually inexpensive operation, and lambda is the only way of doing this. From gvanrossum at gmail.com Wed Mar 16 17:26:39 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed Mar 16 17:26:45 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <42382A56.6050003@iinet.net.au> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> <42382A56.6050003@iinet.net.au> Message-ID: > I guess that leaves Alex's question of whether or not supplying a string of some > description as the initial value can be legitimately translated to: > > if isinstance(initial, basestring): > return initial + type(initial)().join(seq) If you're trying to get people in the habit of writing sum(x, "") instead of "".join(x), I fear that they'll try sum(x, " ") instead of " ".join(x), and be sadly disappointed. Frankly, I've had enough of this exploration of alternative definitions for sum(), and think it is perfect as it is. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gvanrossum at gmail.com Wed Mar 16 17:28:22 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed Mar 16 17:28:27 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <877e9a17050316063813cfa73e@mail.gmail.com> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> <877e9a17050316063813cfa73e@mail.gmail.com> Message-ID: > > Thinking ahead to generic types, I'd like the full signature to be: > > > > def sum(seq: sequence[T], initial: T = 0) -> T. > > Would this _syntax_ work with generic types: > > def sum(seq: sequence[T], initial: T = T()) -> T. Maybe, but it would preclude union types; continuing with the (bad) example of strings, what should one choose for T when seq == ['a', u'b']? The general case is a sequence of objects of different types that are mutually addable. This can be made to work with the (hypothetical!!!!) type system by using unions, but you can't instantiate an instance of a union without being more specific. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jrw at pobox.com Wed Mar 16 18:37:21 2005 From: jrw at pobox.com (John Williams) Date: Wed Mar 16 18:37:23 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <877e9a17050316063813cfa73e@mail.gmail.com> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> <877e9a17050316063813cfa73e@mail.gmail.com> Message-ID: <42386ED1.2030408@pobox.com> Michael Walter wrote: > On Tue, 15 Mar 2005 07:47:20 -0800, Guido van Rossum > wrote: > >>But I'm not so sure now. Thinking ahead to generic types, I'd like the >>full signature to be: >> >> def sum(seq: sequence[T], initial: T = 0) -> T. > > > Would this _syntax_ work with generic types: > > def sum(seq: sequence[T], initial: T = T()) -> T. This doesn't make sense with existing semantics because default arguments are evaluated when the function is defined, but T() can't be evaluated until the function is called. I'm not sure there's a way around that problem without turning default arguments into a trap for the unwary. jw From jcarlson at uci.edu Wed Mar 16 20:25:52 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed Mar 16 20:43:21 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <42382077.6010104@strakt.com> References: <20050315185411.A496.JCARLSON@uci.edu> <42382077.6010104@strakt.com> Message-ID: <20050316101555.A49C.JCARLSON@uci.edu> Samuele Pedroni wrote: > Josiah Carlson wrote: > > Greg Ewing wrote: > > > >>Josiah Carlson wrote: > >> > >> > >>>Since PEP 310 was already mentioned, can we just say that the discussion > >>>can be boiled down to different ways of spelling __enter__/__exit__ from > >>>PEP 310? > >> > >>It's not quite the same thing. PEP 310 suggests a mechanism > >>with a fixed control flow -- do something on entry, do the > >>code block, do something on exit. A general block-passing > >>mechanism would give complete control to the function as > >>to when and how to call the block. > > > > > > I would like to ask a question. Does Python want or need Ruby code > > blocks? I ask because that is what is being offered to cure what ails > > us. Don't get me wrong, I'm sure code blocks can solve quite a few > > problems (and is used as such in Ruby), but I'm not convinced that it is > > the solution for Python. > > > > Any manual on Ruby will invariably discuss code blocks as one of the > > most powerful features Ruby has to offer. Sounds great. Sounds like a > > great big sledgehammer that can be used to do a bunch of things...so > > what is currently being proposed as a use for them, and what can't they > > do (that would be really nice)? [snip myself] > > The synchronized-like thing is the custom try/finally, aka > > __enter__/__exit__ as specified in PEP 310. > > > > The property-like thing was perhaps to be an easier way to generate > > properties, which fairly quickly fell to the wayside in discussion, > > seemingly because people didn't see a need to add thunks for this. [snip myself] Samuele: > well, I think some people desire a more streamlined way of writing code > like: > > def f(...) > ... > def g(...) > ... > x = h(...,f,g) > > [property, setting up callbacks etc are cases of this] I think properties are the most used case where this kind of thing would be nice. Though the only thing that I've ever had a gripe with properties is that I didn't like the trailing property() call - which is why I wrote a property helper decorator (a use can be seen in [1]). But my needs are small, so maybe this kind of thing isn't sufficient for those who write hundreds of properties. > were f,g etc definitions would appear inline and the whole has a > statement flavor; because this is Python a form that does not involve a > lot parantheses would be nice. Of course if the functions then are > allowed to change the surrounding bindings this could be used for > resource release issues etc. Resource release, right, already been discussed, and already known this is a use case. Now, people keep saying that using code blocks can make things like properties easier to construct. Ok, I'll bite, someone write properties using code blocks. I tried to, but I prefer original properties to what I wrote. Maybe someone who is a thunk advocate can write something better. Now's your chance, show the world how thunks make properties prettier and easier to understand. Feel free to use whatever syntax is your favorite (if it may be ambiguous to third party, document it). If someone can make a thunk that looks better than my example [1] below, then I will agree that helping properties is a legitimate use case (I have had serious doubts from the beginning), but if not, then code blocks as a solution to the 'property problem' should be taken off the table. > Notice that decorators basically support a special case of this. > > But yes, apart for the issue of rebinding (and if one wants non-local > returns), this is stricly about sugar. But the sugar isn't all that sweet. It solves the problem of resource acquisition and release (also solved by PEP 310), and may help with property-like things (helped by a decorator). Any other use cases for one of the most powerful features of Ruby, in Python? - Josiah [1] example use of my property helper decorator class foo(object): prop = property_maker("put documentation here") @prop #getter def a(self): return 1 @prop #setter def a(self, v): pass @prop #deleter def a(self): pass From greg at electricrain.com Wed Mar 16 23:26:11 2005 From: greg at electricrain.com (Gregory P. Smith) Date: Wed Mar 16 23:26:13 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: References: <200503092317.07909.anthony@interlink.com.au> <16942.62520.281370.469617@montanaro.dyndns.org> <91a5753c78295cb177214d74dbe8da40@redivi.com> <16945.61680.296812.366920@montanaro.dyndns.org> Message-ID: <20050316222611.GB7211@zot.electricrain.com> On Fri, Mar 11, 2005 at 06:47:11PM -0500, Bob Ippolito wrote: > > On Mar 11, 2005, at 2:26 PM, Skip Montanaro wrote: > > > > > Bob> try: > > Bob> set > > Bob> except NameError: > > Bob> from sets import Set as set > > > > Bob> You don't need the rest. > > > >Sure, but then pychecker bitches about a statement that appears to > >have no > >effect. ;-) > > Well then fix PyChecker to look for this pattern :) > > -bob or make it even uglier to hide from pychecker by writing that as: exec(""" try: set except NameError: from sets import Set as set """) From t-meyer at ihug.co.nz Wed Mar 16 23:32:35 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Wed Mar 16 23:32:48 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: Message-ID: [Bob Ippolito] >>>> try: >>>> set >>>> except NameError: >>>> from sets import Set as set >>>> >>>> You don't need the rest. [Skip Montanaro] >>> Sure, but then pychecker bitches about a statement that appears to >>> have no effect. ;-) [Bob Ippolito] >> Well then fix PyChecker to look for this pattern :) +1. [Gregory P. Smith] > or make it even uglier to hide from pychecker by writing that as: > > exec(""" > try: > set > except NameError: > from sets import Set as set > """) I presume that was somewhat tongue-in-cheek, but if it wasn't, please reconsider. Modulefinder isn't able to realise that set (or sets.Set) is needed with the latter (a problem of this very nature was just fixed with bsddb), which causes trouble for people later on. =Tony.Meyer From Jack.Jansen at cwi.nl Thu Mar 17 00:01:35 2005 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Thu Mar 17 00:01:39 2005 Subject: [Python-Dev] Problems with definition of _POSIX_C_SOURCE Message-ID: On a platform I won't mention here I'm running into problems compiling Python, because of it defining _POSIX_C_SOURCE. It turns out that on this platform the definition causes all sorts of declarations in sys/types.h to be skipped (presumably because they're not official Posix identifiers), which in turn causes platform-specific headers to fail. The comment in pyconfig.h suggests that defining _POSIX_C_SOURCE may enable certain features, but the actual system headers appear to work the other way around: it seems that defining this will disable features that are not strict Posix. Does anyone know what the real meaning of this define is? Because if it is the former then Python is right, but if it is the latter Python really has no business defining it: in general Python isn't 100% posix-compliant because it'll use all sorts of platform-dependent (and, thus, potentially non-posix-compliant) code... This problem is currently stopping Python 2.4.1 to compile on this platform, so if anyone can provide any insight that would be very helpful... -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From tim.peters at gmail.com Thu Mar 17 00:07:53 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 17 00:21:03 2005 Subject: [Python-Dev] Problems with definition of _POSIX_C_SOURCE In-Reply-To: References: Message-ID: <1f7befae0503161507319a0193@mail.gmail.com> [Jack Jansen] > On a platform I won't mention here I'm running into problems compiling > Python, because of it defining _POSIX_C_SOURCE. ... > Does anyone know what the real meaning of this define is? LOL. Here's the Official Story: http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_02.html Look near the top, under "The _POSIX_C_SOURCE Feature Test Macro". This will tell you: When an application includes a header described by IEEE Std 1003.1-2001, and when this feature test macro is defined to have the value 200112L: yadda yadda yadda yadda yadda yadda yadda yadda Then again, every journey of a million miles begins with 200112L small steps ... From greg at electricrain.com Thu Mar 17 00:24:10 2005 From: greg at electricrain.com (Gregory P. Smith) Date: Thu Mar 17 00:24:13 2005 Subject: [Python-Dev] rationale for the no-new-features approach In-Reply-To: References: Message-ID: <20050316232410.GC7211@zot.electricrain.com> > [Gregory P. Smith] > > or make it even uglier to hide from pychecker by writing that as: > > > > exec(""" > > try: > > set > > except NameError: > > from sets import Set as set > > """) > > I presume that was somewhat tongue-in-cheek, but if it wasn't, please > reconsider. Modulefinder isn't able to realise that set (or sets.Set) is > needed with the latter (a problem of this very nature was just fixed with > bsddb), which causes trouble for people later on. > > =Tony.Meyer hehe yes sorry, i left off the :P From sabbey at u.washington.edu Thu Mar 17 01:41:52 2005 From: sabbey at u.washington.edu (Brian Sabbey) Date: Thu Mar 17 01:41:56 2005 Subject: [Python-Dev] thunks (was: code blocks using 'for' loops and generators) In-Reply-To: References: Message-ID: Jim Jewett wrote: > It may be time to PEP (or re-PEP), if only to clarify what people are > actually asking for. I will PEPify this, unless someone does not think I am the correct person to do so. The PEP is probably a better place to try to address questions you raise, as well as give the rationale that Josiah Carlson was looking for. But, in short: > Brian Sabbey's example from message > http://mail.python.org/pipermail/python-dev/2005-March/052202.html > *seems* reasonably clear, but I don't see how it relates in any way to > "for" loops or generators, except as one (but not the only) use case. The original post in this thread was an idea about using 'for' loops and generators, but that idea has since been replaced with something else. > (1) Calls for "Ruby blocks" or "thunks" are basically calls for > placeholders in a function. These placeholders will be filled > with code from someplace else, but will execute in the function's > own local namespace. It wasn't my intention that the thunk would execute in the function's namespace ("function" here is to mean the function that takes the thunk as an argument). I was thinking that scope rules for the thunk would mimic the rules for control flow structures. -Brian From greg.ewing at canterbury.ac.nz Thu Mar 17 02:14:48 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu Mar 17 02:15:03 2005 Subject: [Python-Dev] code blocks using 'for' loops and generators In-Reply-To: <42382077.6010104@strakt.com> References: <20050314095844.A470.JCARLSON@uci.edu> <42361811.5010707@canterbury.ac.nz> <20050315185411.A496.JCARLSON@uci.edu> <42382077.6010104@strakt.com> Message-ID: <4238DA08.1030107@canterbury.ac.nz> Samuele Pedroni wrote: > well, I think some people desire a more streamlined way of writing code > like: > > def f(...) > ... > def g(...) > ... > x = h(...,f,g) Using the recently-proposed 'where' facility, this could be written x = h(..., f, g) where: def f(...): ... def g(...): ... > Of course if the functions then are > allowed to change the surrounding bindings this could be used for > resource release issues etc. Yes, rebinding in the surrounding scope is the one thing that style wouldn't give you -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 17 02:31:43 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu Mar 17 02:31:57 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <1f7befae05031518235903a957@mail.gmail.com> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <1f7befae05031518235903a957@mail.gmail.com> Message-ID: <4238DDFF.4080009@canterbury.ac.nz> Tim Peters wrote: > I can't say it bothers me to specify an appropriate identity element > when 0 is inappropriate. Maybe instead of a single sum() function, each summable type should have a sum() static method which uses an identity appropriate to that type. So to sum a list of integers you would use int.sum(x), to sum floats you would use float.sum(x), and to sum timedeltas you would use timedelta.sum(x). This would also neatly solve the problem of people trying to sum strings, because there would be no str.sum() method. Or alternatively, there could be a str.sum(x) which was equivalent to "".join(x). Although it might be better to call it str.concat() instead of str.sum(). -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 17 02:34:23 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu Mar 17 02:34:39 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <42386ED1.2030408@pobox.com> References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> <877e9a17050316063813cfa73e@mail.gmail.com> <42386ED1.2030408@pobox.com> Message-ID: <4238DE9F.9060006@canterbury.ac.nz> John Williams wrote: > Michael Walter wrote: > >> Would this _syntax_ work with generic types: >> >> def sum(seq: sequence[T], initial: T = T()) -> T. > > This doesn't make sense with existing semantics because default > arguments are evaluated when the function is defined, but T() can't be > evaluated until the function is called. Not to mention that if the seq is empty, there's no way of knowing what T to instantiate... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 17 02:47:28 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu Mar 17 02:47:45 2005 Subject: [Python-Dev] RE: code blocks using 'for' loops and generators In-Reply-To: References: Message-ID: <4238E1B0.8050209@canterbury.ac.nz> Jim Jewett wrote: > (2) A function as a parameter isn't good enough, because the > passed-in function can't see bindings in the surrounding larger > function. (It still sees the lexical scope it which it was defined.) That sounds confused, because the lexical scope it which it was defined is exactly what it *should* see. > (4) A thunk could be used today be creating a string (rather than > a pre-compiled function) and substituting in the thunk's string Again, you seem to be under a misapprehension about how code blocks should work. They should be lexically scoped, not dynamically scoped. > (7) A __leave__ or __exit__ special method really turns into another > name for __del__. Not really. A PEP-310-style __exit__ method is explicitly invoked at well-defined times, not left until the object is reclaimed. It doesn't suffer from any of the problems of __del__. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From michael.walter at gmail.com Thu Mar 17 02:48:01 2005 From: michael.walter at gmail.com (Michael Walter) Date: Thu Mar 17 02:48:05 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> <877e9a17050316063813cfa73e@mail.gmail.com> Message-ID: <877e9a1705031617485004fe38@mail.gmail.com> On Wed, 16 Mar 2005 08:28:22 -0800, Guido van Rossum wrote: > > > Thinking ahead to generic types, I'd like the full signature to be: > > > > > > def sum(seq: sequence[T], initial: T = 0) -> T. > > > > Would this _syntax_ work with generic types: > > > > def sum(seq: sequence[T], initial: T = T()) -> T. > > Maybe, but it would preclude union types; continuing with the (bad) > example of strings, what should one choose for T when seq == ['a', > u'b']? The general case is a sequence of objects of different types > that are mutually addable. This can be made to work with the > (hypothetical!!!!) type system by using unions, but you can't > instantiate an instance of a union without being more specific. Continuing that hypothetical thought, it would be perfectly acceptable to make require an argument for union types T. Maybe T() should only be valid for non-union types. Several questions like "when should T() be evaluated" [1], "how can we avoid ': T = T()' leading to a type error" and "how about optional parameters in front of ': T = T()'" just popped up in my mind. Michael [1] Thanks, John! From michael.walter at gmail.com Thu Mar 17 02:52:01 2005 From: michael.walter at gmail.com (Michael Walter) Date: Thu Mar 17 02:52:03 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: <877e9a1705031617513a89d257@mail.gmail.com> References: <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> <877e9a17050316063813cfa73e@mail.gmail.com> <42386ED1.2030408@pobox.com> <4238DE9F.9060006@canterbury.ac.nz> <877e9a1705031617513a89d257@mail.gmail.com> Message-ID: <877e9a1705031617521644f3be@mail.gmail.com> On Thu, 17 Mar 2005 14:34:23 +1300, Greg Ewing wrote: > Not to mention that if the seq is empty, there's no > way of knowing what T to instantiate... You just use the same T as inferred for seq : sequence[T] . Michael From martin at v.loewis.de Thu Mar 17 09:19:17 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Mar 17 09:19:20 2005 Subject: [Python-Dev] Problems with definition of _POSIX_C_SOURCE In-Reply-To: References: Message-ID: <42393D85.9090401@v.loewis.de> Jack Jansen wrote: > The comment in pyconfig.h suggests that defining _POSIX_C_SOURCE may > enable certain features, but the actual system headers appear to work > the other way around: it seems that defining this will disable features > that are not strict Posix. > Does anyone know what the real meaning of this define is? Because if it > is the former then Python is right, but if it is the latter Python > really has no business defining it As you can see from the formal definition that Tim gave you, both is right: the macro causes system headers to provide the functions that POSIX says they should provide, and remove functions that POSIX does not mention, except when enabled through other feature selection macros. > in general Python isn't 100% > posix-compliant because it'll use all sorts of platform-dependent (and, > thus, potentially non-posix-compliant) code... Python is 100% POSIX compliant. It also uses extensions to POSIX on platforms that provide them, but if these extensions are not available, it falls back to just not using them. So Python really uses "POSIX with extension". A careful operating system developer will understand that this is a useful programming model, and provide feature selection macros to enable features that go beyond POSIX. That's why you can see various feature selection macros at the beginning of configure.in. In case you wonder why Python defines this in the first place: some platforms really need the definition, or else they don't provide the proper header contents (i.e. they fall back to ISO C for some headers), most notably Tru64 and HP-UX. Other systems implement different versions of the same API, e.g. Solaris, and defining _POSIX_C_SOURCE makes these systems provide the POSIX version of the API. > This problem is currently stopping Python 2.4.1 to compile on this > platform, so if anyone can provide any insight that would be very > helpful... Just define _THIS_PLATFORM_SOURCE. If there is no such define, complain to the vendor of this platform, and ask Apple to provide such a macro. If this falls on deaf ears, go to the block "Some systems cannot stand _XOPEN_SOURCE being defined at all;" in configure.in and make another entry. Make sure that entry: - lists the precise reason for the entry (e.g. what structure/type/function gets hidden that shouldn't be hidden) - lists your name as the contact to ask for details - is specific to the particular release of this platform, so if future versions of this platform fix the bug, the work-around of disabling _XOPEN_SOURCE and _POSIX_C_SOURCE can be removed Regards, Martin From ncoghlan at iinet.net.au Thu Mar 17 15:50:53 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 17 15:50:58 2005 Subject: [Python-Dev] properties with decorators (was: code blocks using 'for' loops and generators) In-Reply-To: <20050316101555.A49C.JCARLSON@uci.edu> References: <20050315185411.A496.JCARLSON@uci.edu> <42382077.6010104@strakt.com> <20050316101555.A49C.JCARLSON@uci.edu> Message-ID: <4239994D.6080007@iinet.net.au> Josiah Carlson wrote: > Samuele Pedroni wrote: [snip] >>well, I think some people desire a more streamlined way of writing code >>like: >> >>def f(...) >>... >>def g(...) >>... >>x = h(...,f,g) >> >>[property, setting up callbacks etc are cases of this] > > > > I think properties are the most used case where this kind of thing would > be nice. Though the only thing that I've ever had a gripe with > properties is that I didn't like the trailing property() call - which is > why I wrote a property helper decorator (a use can be seen in [1]). But > my needs are small, so maybe this kind of thing isn't sufficient for > those who write hundreds of properties. [snip] I'm still trying to decide if the following is an elegant solution to defining properties, or a horrible abuse of function decorators: def as_property(func): return property(*func()) class Example(object): @as_property def x(): def get(self): print "Getting!" def set(self, val): print "Setting!" def delete(self): print "Deleting!" return get, set, delete Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Thu Mar 17 15:54:02 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 17 15:54:07 2005 Subject: [Python-Dev] Rationale for sum()'s design? In-Reply-To: References: <4234D941.9040605@canterbury.ac.nz> <8f0366d60265d201c50220df4d508b2f@xs4all.nl> <4236271F.5020407@canterbury.ac.nz> <1f7befae050314161678706275@mail.gmail.com> <4236D333.3070007@iinet.net.au> <42382A56.6050003@iinet.net.au> Message-ID: <42399A0A.20909@iinet.net.au> Guido van Rossum wrote: >>I guess that leaves Alex's question of whether or not supplying a string of some >>description as the initial value can be legitimately translated to: >> >> if isinstance(initial, basestring): >> return initial + type(initial)().join(seq) > > > If you're trying to get people in the habit of writing sum(x, "") > instead of "".join(x), I fear that they'll try sum(x, " ") instead of > " ".join(x), and be sadly disappointed. That works for me as a reason not to provide this feature. It's somewhat heartening when discussions like this turn out to show that the original design was right after all :) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Thu Mar 17 16:28:29 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 17 16:28:38 2005 Subject: [Python-Dev] itertools.walk() In-Reply-To: <40d419fccfee93900159318263b990bc@redivi.com> References: <000401c52a1a$141899c0$fc33c797@oemcomputer> <42383694.802@iinet.net.au> <40d419fccfee93900159318263b990bc@redivi.com> Message-ID: <4239A21D.80500@iinet.net.au> Bob Ippolito wrote: > I'm not sure why it's useful to explode the stack with all that > recursion? Mine didn't do that. The control flow is nearly identical, > but it looks more fragile (and you would get some really evil stack > trace if iter_factory(foo) happened to raise something other than > TypeError). It was a holdover from my first version which *was* recursive. When I switched to using your chaining style, I didn't think to get rid of the now unneeded recursion. So just drop the recursive calls to 'walk', and it should be back to your structure. For the 'infinite recursion on basestring' example, PJE at one point suggested a "if item is iterable" guard that immediately yielded the item. Allowing breadth first operation requires something a little different, though: from itertools import chain def walk(iterable, depth_first=True, atomic_types=(basestring,), iter_factory=iter): itr = iter(iterable) while True: for item in itr: if isinstance(item, atomic_types): yield item continue try: subitr = iter_factory(item) except TypeError: yield item continue # Block simple cycles (like characters) try: subitem = subitr.next() except StopIteration: continue if subitem is item: yield subitem continue if depth_first: itr = chain([subitem], subitr, itr) else: itr = chain(itr, [subitem], subitr) break else: break Py> seq [['123', '456'], 'abc', 'abc', 'abc', 'abc', ['xyz']] Py> list(walk(seq)) ['123', '456', 'abc', 'abc', 'abc', 'abc', 'xyz'] Py> list(walk(seq, depth_first=False)) ['abc', 'abc', 'abc', 'abc', '123', '456', 'xyz'] Py> list(walk(seq, atomic_types=())) ['1', '2', '3', '4', '5', '6', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'x', 'y', 'z'] Py> list(walk(seq, depth_first=False, atomic_types=())) ['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', '1', '2', '3', '4', '5', '6', 'x', 'y', 'z'] Raymond may simply decide to keep things simple instead, of course. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From gvanrossum at gmail.com Thu Mar 17 16:42:20 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Mar 17 16:42:23 2005 Subject: [Python-Dev] Python 2.4 won the "Jolt productivity award" last night Message-ID: Python 2.4 won the "Jolt productivity award" last night. That's the runner-up award; in our category, languages and development tools, the Jolt (the category winner) went to Eclipse 3.0; the other runners-up were IntelliJ and RealBasic (no comment :-). Like usually, open source projects got several awards; both Subversion and Hibernate (an open source Java persistency library) got Jolts. Maybe from now on we should add "award-winning" whenever we refer to Python 2.4. If someone can find out the website where the results are summarized (I'm sure there is one but I'm too lazy to Google). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gmccaughan at synaptics-uk.com Thu Mar 17 17:01:03 2005 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu Mar 17 17:01:36 2005 Subject: [Python-Dev] Python 2.4 won the "Jolt productivity award" last night In-Reply-To: References: Message-ID: <200503171601.03731.gmccaughan@synaptics-uk.com> On Thursday 2005-03-17 15:42, Guido van Rossum wrote: > Python 2.4 won the "Jolt productivity award" last night. That's the > runner-up award; in our category, languages and development tools, the > Jolt (the category winner) went to Eclipse 3.0; the other runners-up > were IntelliJ and RealBasic (no comment :-). > > Like usually, open source projects got several awards; both Subversion > and Hibernate (an open source Java persistency library) got Jolts. > > Maybe from now on we should add "award-winning" whenever we refer to > Python 2.4. If someone can find out the website where the results are > summarized (I'm sure there is one but I'm too lazy to Google). http://www.sdmagazine.com/jolts/ , but it's not been updated yet and therefore still has last year's winners on it. I haven't found anything with more up-to-date results. This year's finalists are listed at http://www.sdmagazine.com/jolts/15th_jolt_finalists.html . -- g From jcarlson at uci.edu Thu Mar 17 18:01:27 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Mar 17 18:03:10 2005 Subject: [Python-Dev] properties with decorators (was: code blocks using 'for' loops and generators) In-Reply-To: <4239994D.6080007@iinet.net.au> References: <20050316101555.A49C.JCARLSON@uci.edu> <4239994D.6080007@iinet.net.au> Message-ID: <20050317085132.A4A2.JCARLSON@uci.edu> Nick Coghlan wrote: > > Josiah Carlson wrote: > > Samuele Pedroni wrote: > [snip] > >>well, I think some people desire a more streamlined way of writing code > >>like: > >> > >>def f(...) > >>... > >>def g(...) > >>... > >>x = h(...,f,g) > >> > >>[property, setting up callbacks etc are cases of this] > > > > > > > > I think properties are the most used case where this kind of thing would > > be nice. Though the only thing that I've ever had a gripe with > > properties is that I didn't like the trailing property() call - which is > > why I wrote a property helper decorator (a use can be seen in [1]). But > > my needs are small, so maybe this kind of thing isn't sufficient for > > those who write hundreds of properties. > [snip] > > I'm still trying to decide if the following is an elegant solution to defining > properties, or a horrible abuse of function decorators: [snip example] The only issue is that you are left with a closure afterwards, no big deal, unless you've got hundreds of thousands of examples of this. I like your method anyways. - Josiah From exarkun at divmod.com Thu Mar 17 19:36:13 2005 From: exarkun at divmod.com (Jp Calderone) Date: Thu Mar 17 19:36:19 2005 Subject: [Python-Dev] properties with decorators (was: code blocks using 'for' loops and generators) In-Reply-To: <20050317085132.A4A2.JCARLSON@uci.edu> Message-ID: <20050317183613.13806.556069976.divmod.quotient.8145@ohm> On Thu, 17 Mar 2005 09:01:27 -0800, Josiah Carlson wrote: > > Nick Coghlan wrote: > > > > Josiah Carlson wrote: > > > > > > [snip] > > > > > > I think properties are the most used case where this kind of thing would > > > be nice. Though the only thing that I've ever had a gripe with > > > properties is that I didn't like the trailing property() call - which is > > > why I wrote a property helper decorator (a use can be seen in [1]). But > > > my needs are small, so maybe this kind of thing isn't sufficient for > > > those who write hundreds of properties. > > [snip] > > > > I'm still trying to decide if the following is an elegant solution to defining > > properties, or a horrible abuse of function decorators: > > [snip example] > > The only issue is that you are left with a closure afterwards, no big > deal, unless you've got hundreds of thousands of examples of this. I > like your method anyways. No closed over variables, actually. So no closure. > > - Josiah > Jp From jcarlson at uci.edu Thu Mar 17 20:15:55 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Mar 17 20:17:14 2005 Subject: [Python-Dev] properties with decorators (was: code blocks using In-Reply-To: <20050317183613.13806.556069976.divmod.quotient.8145@ohm> References: <20050317085132.A4A2.JCARLSON@uci.edu> <20050317183613.13806.556069976.divmod.quotient.8145@ohm> Message-ID: <20050317111355.A4A5.JCARLSON@uci.edu> Jp Calderone wrote: > > On Thu, 17 Mar 2005 09:01:27 -0800, Josiah Carlson wrote: > > > > Nick Coghlan wrote: > > > > > > Josiah Carlson wrote: > > > > > > > > [snip] > > > > > > > > I think properties are the most used case where this kind of thing would > > > > be nice. Though the only thing that I've ever had a gripe with > > > > properties is that I didn't like the trailing property() call - which is > > > > why I wrote a property helper decorator (a use can be seen in [1]). But > > > > my needs are small, so maybe this kind of thing isn't sufficient for > > > > those who write hundreds of properties. > > > [snip] > > > > > > I'm still trying to decide if the following is an elegant solution to defining > > > properties, or a horrible abuse of function decorators: > > > > [snip example] > > > > The only issue is that you are left with a closure afterwards, no big > > deal, unless you've got hundreds of thousands of examples of this. I > > like your method anyways. > > No closed over variables, actually. So no closure. My mistake (caused by a misunderstanding of when closures are not created, obviously). - Josiah From jhylton at gmail.com Thu Mar 17 21:29:48 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Thu Mar 17 21:29:52 2005 Subject: [Python-Dev] thread semantics for file objects Message-ID: Are the thread semantics for file objecst documented anywhere? I don't see anything in the library manual, which is where I expected to find it. It looks like read and write are atomic by virtue of fread and fwrite being atomic. I'm less sure what guarantees, if any, the other methods attempt to provide. For example, it looks like concurrent calls to writelines() will interleave entire lines, but not parts of lines. Concurrent calls to readlines() provide insane results, but I don't know if that's a bug or a feature. Specifically, if your file has a line that is longer than the internal buffer size SMALLCHUNK you're likely to get parts of that line chopped up into different lines in the resulting return values. If we can come up with intended semantics, I'd be willing to prepare a patch for the documentation. Jeremy From aahz at pythoncraft.com Thu Mar 17 22:25:44 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu Mar 17 22:25:46 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: Message-ID: <20050317212544.GB3483@panix.com> On Thu, Mar 17, 2005, Jeremy Hylton wrote: > > Are the thread semantics for file objecst documented anywhere? I > don't see anything in the library manual, which is where I expected to > find it. It looks like read and write are atomic by virtue of fread > and fwrite being atomic. Uncle Timmy will no doubt agree with me: the semantics don't matter. NEVER, NEVER access the same file object from multiple threads, unless you're using a lock. And even using a lock is stupid. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From jhylton at gmail.com Thu Mar 17 22:47:20 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Thu Mar 17 22:47:23 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <20050317212544.GB3483@panix.com> References: <20050317212544.GB3483@panix.com> Message-ID: On Thu, 17 Mar 2005 16:25:44 -0500, Aahz wrote: > On Thu, Mar 17, 2005, Jeremy Hylton wrote: > > > > Are the thread semantics for file objecst documented anywhere? I > > don't see anything in the library manual, which is where I expected to > > find it. It looks like read and write are atomic by virtue of fread > > and fwrite being atomic. > > Uncle Timmy will no doubt agree with me: the semantics don't matter. > NEVER, NEVER access the same file object from multiple threads, unless > you're using a lock. And even using a lock is stupid. I'm not looking for your permission or approval. I just want to know what semantics are intended. If the documentation wants to say that the semantics are undefined that okay, although I think we need to say more because some behavior has been provided by the implementation for a long time. Jeremy From pedronis at strakt.com Thu Mar 17 23:00:18 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Thu Mar 17 22:59:12 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: <20050317212544.GB3483@panix.com> Message-ID: <4239FDF2.60208@strakt.com> Jeremy Hylton wrote: > On Thu, 17 Mar 2005 16:25:44 -0500, Aahz wrote: > >>On Thu, Mar 17, 2005, Jeremy Hylton wrote: >> >>>Are the thread semantics for file objecst documented anywhere? I >>>don't see anything in the library manual, which is where I expected to >>>find it. It looks like read and write are atomic by virtue of fread >>>and fwrite being atomic. >> >>Uncle Timmy will no doubt agree with me: the semantics don't matter. >>NEVER, NEVER access the same file object from multiple threads, unless >>you're using a lock. And even using a lock is stupid. > > > I'm not looking for your permission or approval. I just want to know > what semantics are intended. If the documentation wants to say that > the semantics are undefined that okay, although I think we need to say > more because some behavior has been provided by the implementation for > a long time. > I think this is left unspecified for example by Java too. I would be surprised if Jython would offer the same characteristics in this respect as CPython. From martin at v.loewis.de Thu Mar 17 23:04:16 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Mar 17 23:04:19 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: <20050317212544.GB3483@panix.com> Message-ID: <4239FEE0.1090209@v.loewis.de> Jeremy Hylton wrote: >>>Are the thread semantics for file objecst documented anywhere? I >>>don't see anything in the library manual, which is where I expected to >>>find it. It looks like read and write are atomic by virtue of fread >>>and fwrite being atomic. >> >>Uncle Timmy will no doubt agree with me: the semantics don't matter. >>NEVER, NEVER access the same file object from multiple threads, unless >>you're using a lock. And even using a lock is stupid. > > > I'm not looking for your permission or approval. Literally, the answer to your question is "no". In fact, Python does not specify *any* interleaving semantics for threads whatsoever. The only statement to this respect is """ Not all built-in functions that may block waiting for I/O allow other threads to run. (The most popular ones (\function{time.sleep()}, \method{\var{file}.read()}, \function{select.select()}) work as expected.) """ Of course, this says it works as expected, without saying what actually is expected. > I just want to know what semantics are intended. But this is not what you've asked :-) Anyway, expected by whom? Aahz clearly expects that the semantics are unspecified, as he expects that nobody ever even attempts to read the same file from multiple threads. > If the documentation wants to say that > the semantics are undefined that okay, Formally, there is no need to say that something is undefined. Not defining anything is sufficient. So the semantics *is* undefined, whether the documentation "wants" to say that or not. > although I think we need to say > more because some behavior has been provided by the implementation for > a long time. That immediately rings the Jython bell, and perhaps also the PyPy bell. So if you want to say something, just go ahead. Before I make the documentation want to say that, I would like to make it say more basic things first (e.g. that stores to variables are atomic). Regards, Martin From tim.peters at gmail.com Thu Mar 17 23:13:05 2005 From: tim.peters at gmail.com (Tim Peters) Date: Thu Mar 17 23:14:12 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: Message-ID: <1f7befae050317141355cc6348@mail.gmail.com> [Jeremy Hylton] > Are the thread semantics for file objecst documented anywhere? No. At base level, they're inherited from the C stdio implementation. Since the C standard doesn't even mention threads, that's all platform-dependent. POSIX defines thread semantics for file I/O, but fat lot of good that does you on Windows, etc. > I don't see anything in the library manual, which is where I expected to > find it. It looks like read and write are atomic by virtue of fread > and fwrite being atomic. I wouldn't consider this as more than CPython implementation accidents in the cases it appears to apply. For example, in universal-newlines mode, are you sure f.read(n) always maps to exactly one fread() call? > I'm less sure what guarantees, if any, the other methods attempt to > provide. I don't believe they're _trying_ to provide anything specific. > For example, it looks like concurrent calls to writelines() will interleave entire > lines, but not parts of lines. Concurrent calls to readlines() provide insane > results, but I don't know if that's a bug or a feature. Specifically, if your file has a > line that is longer than the internal buffer size SMALLCHUNK you're likely to > get parts of that line chopped up into different lines in the resulting return values. And you're _still_ not thinking "implementation accidents" ? > If we can come up with intended semantics, I'd be willing to prepare a > patch for the documentation. I think Aahz was on target here: NEVER, NEVER access the same file object from multiple threads, unless you're using a lock. And here he went overboard: And even using a lock is stupid. ZODB's FileStorage is bristling with locks protecting multi-threaded access to file objects, therefore that can't be stupid. QED From jhylton at gmail.com Thu Mar 17 23:15:49 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Thu Mar 17 23:15:52 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <4239FEE0.1090209@v.loewis.de> References: <20050317212544.GB3483@panix.com> <4239FEE0.1090209@v.loewis.de> Message-ID: On Thu, 17 Mar 2005 23:04:16 +0100, "Martin v. L?wis" wrote: > Jeremy Hylton wrote: > >>>Are the thread semantics for file objecst documented anywhere? I > >>>don't see anything in the library manual, which is where I expected to > >>>find it. It looks like read and write are atomic by virtue of fread > >>>and fwrite being atomic. > >> > >>Uncle Timmy will no doubt agree with me: the semantics don't matter. > >>NEVER, NEVER access the same file object from multiple threads, unless > >>you're using a lock. And even using a lock is stupid. > > > > > > I'm not looking for your permission or approval. > > Literally, the answer to your question is "no". In fact, Python does not > specify *any* interleaving semantics for threads whatsoever. The only > statement to this respect is I'm surprised that it does not, for example, guarantee that reads and writes are atomic, since CPython relies on fread and fwrite which are atomic. Also, there are other operations that go to the trouble of calling flockfile(). What's the point if we don't provide any guarantees? <0.6 wink>. If it is not part of the specified behavior, then I suppose it's a quality of implementation issue. Either way it would be helpful if the Python documentation said something, e.g. you can rely on readline() being threadsafe or you can't but the current CPython implementation happens to be. readline() seemed like an interesting case because readlines() doesn't have the same implementation and the behavior is different. So, as another example, you could ask whether readlines() has a bug or not. Jeremy From jhylton at gmail.com Thu Mar 17 23:22:15 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Thu Mar 17 23:22:17 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <1f7befae050317141355cc6348@mail.gmail.com> References: <1f7befae050317141355cc6348@mail.gmail.com> Message-ID: On Thu, 17 Mar 2005 17:13:05 -0500, Tim Peters wrote: > [Jeremy Hylton] > > Are the thread semantics for file objecst documented anywhere? > > No. At base level, they're inherited from the C stdio implementation. > Since the C standard doesn't even mention threads, that's all > platform-dependent. POSIX defines thread semantics for file I/O, but > fat lot of good that does you on Windows, etc. Fair enough. I didn't consider Windows at all or other non-POSIX platforms. > > > I don't see anything in the library manual, which is where I expected to > > find it. It looks like read and write are atomic by virtue of fread > > and fwrite being atomic. > > I wouldn't consider this as more than CPython implementation accidents > in the cases it appears to apply. For example, in universal-newlines > mode, are you sure f.read(n) always maps to exactly one fread() call? Universal newline reads and get_line() both lock the stream if the platform supports it. So I expect that they are atomic on those platforms. But it certainly seems safe to conclude this is a quality of implementation issue. Otherwise, why bother with the flockfile() at all, right? Or is there some correctness issue I'm not seeing that requires the locking for some basic safety in the implementation. > And even using a lock is stupid. > > ZODB's FileStorage is bristling with locks protecting multi-threaded > access to file objects, therefore that can't be stupid. QED Using a lock seemed like a good idea there and still seems like a good idea now :-). jeremy From aahz at pythoncraft.com Thu Mar 17 23:27:13 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu Mar 17 23:27:16 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <1f7befae050317141355cc6348@mail.gmail.com> References: <1f7befae050317141355cc6348@mail.gmail.com> Message-ID: <20050317222713.GA6131@panix.com> On Thu, Mar 17, 2005, Tim Peters wrote: > > I think Aahz was on target here: > > NEVER, NEVER access the same file object from multiple threads, unless > you're using a lock. > > And here he went overboard: > > And even using a lock is stupid. > > ZODB's FileStorage is bristling with locks protecting multi-threaded > access to file objects, therefore that can't be stupid. QED Heh. And how much time have you spent debugging race conditions and such? That's the thrust of my point, same as we tell people to avoid locks and use Queue instead. I know that my statement isn't absolutely true in the sense that it's possible to make code work that accesses external objects across threads. (Which is why I didn't garnish that part with emphasis.) But it's still stupid, 95-99% of the time. Actually, I did skip over one other counter-example: stdout is usually safe across threads provided one builds up a single string. Still not something to rely on. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From martin at v.loewis.de Thu Mar 17 23:57:52 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Mar 17 23:57:55 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: <20050317212544.GB3483@panix.com> <4239FEE0.1090209@v.loewis.de> Message-ID: <423A0B70.4020705@v.loewis.de> Jeremy Hylton wrote: >>>>>Are the thread semantics for file objecst documented anywhere? >>Literally, the answer to your question is "no". > I'm surprised that it does not, for example, guarantee that reads and > writes are atomic, since CPython relies on fread and fwrite which are > atomic. Where is the connection? Why would anything that CPython requires from the C library have any effect on Python's documentation? The only effect on Python documentation is that anybody writes it. Nobody cares, so nobody writes documentation. Remember, you were asking what behaviour is *documented*, not what behaviour is guaranteed by the implementation (in a specific version of the implementation). > Also, there are other operations that go to the trouble of calling > flockfile(). What's the point if we don't provide any guarantees? Because nobody cares about guarantees in the documentation. Instead, people care about observable behaviour. So if you get a crash due to a race condition, you care, you report a bug, the Python developer agrees its a bug, and fixes it by adding synchronization. Nobody reported a bug to the Python documentation. > <0.6 wink>. If it is not part of the specified behavior, then I > suppose it's a quality of implementation issue. Either way it would > be helpful if the Python documentation said something, e.g. you can > rely on readline() being threadsafe or you can't but the current > CPython implementation happens to be. It would be helpful to whom? To you? I doubt this, as you will be the one who writes the documentation :-) > readline() seemed like an interesting case because readlines() doesn't > have the same implementation and the behavior is different. So, as > another example, you could ask whether readlines() has a bug or not. Nobody knows. It depends on the Python developer who reviews the bug report. Most likely, he considers it tricky and leaves it open for somebody else. If his name is Martin, he will find that this is not a bug (because it does not cause a crash, and does not contradict with the documentation), and he will reclassify it as a wishlist item. If his name is Tim, and if he has a good day, he will fix it, and add a comment on floating point numbers. Regards, Martin From tim.peters at gmail.com Fri Mar 18 00:14:46 2005 From: tim.peters at gmail.com (Tim Peters) Date: Fri Mar 18 00:14:48 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: <1f7befae050317141355cc6348@mail.gmail.com> Message-ID: <1f7befae050317151439ed8dc0@mail.gmail.com> [Jeremy Hylton] ... > Universal newline reads and get_line() both lock the stream if the > platform supports it. So I expect that they are atomic on those > platforms. Well, certainly not get_line(). That locks and unlocks the stream _inside_ an enclosing for-loop. Looks quite possible for different threads to read different parts of "the same line" if multiple threads are trying to do get_line() simultaneously. It releases the GIL inside the for-loop too, so other threads _can_ sneak in. We put a lot of work into speeding those getc()-in-a-loop functions. There was undocumented agreement at the time that they "should be" thread-safe in this sense: provided the platform C stdio wasn't thread-braindead, then if you had N threads all simultaneously reading a file object containing B bytes, while nobody wrote to that file object, then the total number of bytes seen by all N threads would sum to B at the time they all saw EOF. This was a much stronger guarantee than Perl provided at the time (and, for all I know, still provides), and we (at least I) wrote little test programs at the time demonstrating that the total number of bytes Perl saw in this case was unpredictable, while Python's did sum to B. Of course Perl didn't document any of this either, and it Pythonland was clearly specific to the horrid tricks in CPython's fileobject.c. > But it certainly seems safe to conclude this is a quality of > implementation issue. Or a sheer pigheadness-of-implementor issue . > Otherwise, why bother with the flockfile() at all, right? Or is there some > correctness issue I'm not seeing that requires the locking for some basic > safety in the implementation. There are correctness issues, but we still ignore them; locking relieves, but doesn't solve, them. For example, C doesn't (and POSIX doesn't either!) define what happens if you mix reads with writes on a file opened for update unless a file-positioning operation (like seek) intervenes, and that's pretty easy for threads to run afoul of. Python does nothing to stop you from trying, and behavior if you do is truly all over the map across boxes. IIRC, one of the multi-threaded test programs I mentioned above provoked ugly death in the bowels of MS's I/O libraries when I threw an undisciplined writer thread into the mix too. This was reported to MS, and their response was "so don't that -- it's undefined". Locking the stream at least cuts down the chance of that happening, although that's not the primary reason for it. Heck, we still have a years-open critical bug against segfaults when one thread tries to close a file object while another threading is reading from it, right? >>> And even using a lock is stupid. >> ZODB's FileStorage is bristling with locks protecting multi-threaded >> access to file objects, therefore that can't be stupid. QED > Using a lock seemed like a good idea there and still seems like a good > idea now :-). Damn straight, and we're certain it has nothing to do with those large runs of NUL bytes that sometime overwrite peoples' critical data for no reason at all . From bac at OCF.Berkeley.EDU Fri Mar 18 03:21:33 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Mar 18 03:21:37 2005 Subject: [Python-Dev] python-dev Summary for 2005-03-01 through 2005-03-15 [draft] Message-ID: <423A3B2D.8030302@ocf.berkeley.edu> Amazingly on time thanks to the quarter being over. You can't see me jumping up and down in joy over that fact, but I am while trying not to hit the ceiling as I do it (for those of you who have never met me, I'm 6'6" tall, so jumping in a room is not always the smartest thing for me, especially when ceiling fans are involved). Since I will be on a plane most of tomorrow heading to DC for PyCon I won't get to this any sooner than Saturday while I am at the sprints. Might send it out Saturday or Sunday during a lull in the sprint, so please get corrections and additions in by then. -------------------------------------- ===================== Summary Announcements ===================== ----------------------------- Second to last summary for me ----------------------------- Just a reminder, after this Summary there is only one more left for me to write. After that Tim Lesher, Tony Meyer, and Steven Bethard will be taking over. ----------------- See you at PyCon! ----------------- PyCon_ is practically upon us! If you are going to be there, great! Please feel free to say hello if you run into me (will be at the sprints and the conference Wednesday and Thursday; skipping Friday to see a friend). Always happy to stop-and-chat. .. _PyCon: http://www.pycon.org/ ------------------------ 2.4.1 should be out soon ------------------------ Python 2.4.1c1 is out. Very shortly c2 will be released. Assuming no major issues come up, 2.4 final will be out. But in order to make sure no issues come up, we need the code to be tested! Please get the code and run the regression tests. If you are on a UNIX system it is as easy as running ``make test`` (``make testall`` is even better). The tests can also be run on non-UNIX systems; see http://docs.python.org/lib/regrtest.html on how. ========= Summaries ========= ---------------------- 2.4 should be out soon ---------------------- Python 2.4.1c1 was releaseed, but enough bugs were found and subsequently fixed that c2 release will occur before 2.4 final comes out. Contributing threads: - `2.4.1c1 March 10th, 2.4.1 March 17th <>`__ - `Failing tests: marshal, warnings <>`__ - `BRANCH FREEZE for 2.4.1rc1, 0000 UTC, 2005-03-10 <>`__ - `branch release24-maint is unfrozen, 2.4.1rc2? <>`__ - `os.access and Unicode <>`__ - `RELEASED Python 2.4.1, release candidate 1 <>`__ - `distutils fix for building Zope against Python 2.4.1c1 <>`__ - `Python2.4.1c1 and win32com <>`__ - `Open issues for 2.4.1 <>`__ ------------------------------------------- Getting state of all threads in interpreter ------------------------------------------- Florent Guillaume wrote some code for Zope that returned the current state of all threads in the interpreter, regardless of whether they were hung or not. Tim Peters suggested someone write up some code so that this could be made available in Python itself. Contributing threads: - `Useful thread project for 2.5? <>`__ --------------------------------- No new features in micro releases --------------------------------- A bug in os.access() not allowing Unicode strings triggered the discussion of whether it was a bugfix to repair the issue or a new feature. In the end it was decided it was a bugfix. But the point was specified that micro releases should never have any new feature, no matter how small. Contributing threads: - `[Python-checkins] python/dist/src/Modules ossaudiodev.c, 1.35, 1.36 <>`__ - `No new features <>`__ - `os.access and Unicode <>`__ - `rationale for the no-new-features approach <>`__ ------------------------------------- Python wins Jolt "Productivity Award" ------------------------------------- Python was runner-up in the `15th annual Jolt Awards`_ in the category of "Languages and Development Environments", being given the "Productivity Award". Python is now award-winning. =) .. _15th annual Jolt Awards: http://www.sdmagazine.com/jolts/15th_jolt_finalists.html Contributing threads: - `FWD: SD MAgazine.com - Jolt Awards Winners <>`__ - `Python 2.4 won the "Jolt productivity award" last night <>`__ ------------------------------ New built-ins: any() and all() ------------------------------ Python 2.5 gains two new built-ins: any(), which returns True if the iterable passed to it contains any true items, and all(), which returns True if all the items in the iterable passed to it are true. Contributing threads: - `Adding any() and all() <>`__ -------------------------------- Abbreviating list comprehensions -------------------------------- The idea of allowing list comprehensions when the item being appended to the new list is passed directly in was proposed: ``[x in seq if f(x)`` would be equivalent to ``[x for x in seq if f(x)]``. The debate on this one is still going, but my gut says it won't be accepted; TOOWTDI and all. Contributing threads: - `Adding any() and all() <>`__ - `comprehension abbreviation <>`__ ------------------------- sum() semantics discussed ------------------------- Guido's blog entry on `the fate of reduce() in Python 3000`_ (which reiterated Guido's plan to cut map(), reduce(), filter() and lambdas (what about zip()?) caused a huge discussion on whether sum() worked the best way possible. As it stands now, sum() only accepts a sequence of numbers and its optional second argument works as an initial value to build off of. The suggestion was put forth of making the second argument more of a default argument if the passed-in sequence was empty. Otherwise the second argument would be ignored. But further discussion solidified the idea that sum() works quite well as it is and thus won't be changed in Python 3000. .. _the fate of reduce() in Python 3000: http://www.artima.com/weblogs/viewpost.jsp?thread=98196 Contributing threads: - `sum() <>`__ - `Rationale for sum()'s design? <>`__ =============== Skipped Threads =============== + Re: python-dev Summary for 2005-01-16 through 2005-01-31 + Documentation for __new__ + Migrating to subversion + Decimal & returning NotImplemented (or not) + itemgetter/attrgetter extension + LinkedHashSet/LinkedHashMap equivalents + Re: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.49.2.3, 1.49.2.4 idlever.py, 1.22.2.1, 1.22.2.2 + code blocks using 'for' loops and generators + can we stop pretending _PyTyple_Lookup is internal? + (no subject) Thank you Michael Chermside for that very descriptive subject =) From abo at minkirri.apana.org.au Fri Mar 18 04:30:27 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Fri Mar 18 04:30:38 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. Message-ID: <1111116627.3762.4.camel@schizo> G'day, the recent thread about thread semantics for file objects reminded me I had a draft pep for extending file objects to support non-blocking mode. This is handy for handling files in async applications (the non-threaded way of doing things concurrently). Its pretty rough, but if I fuss over it any more I'll never get it out... -- Donovan Baarda http://minkirri.apana.org.au/~abo/ -------------- next part -------------- PEP: XXX Title: Make builtin file objects support non-blocking mode Version: $Revision: 1.0 $ Last-Modified: $Date: 2005/03/18 11:34:00 $ Author: Donovan Baarda Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 06-Jan-2005 Python-Version: 3.5 Post-History: 06-Jan-2005 Abstract ======== This PEP suggests a way that the existing builtin file type could be extended to better support non-blocking read and write modes required for asynchronous applications using things like select and popen2. Rationale ========= Many Python library methods and classes like select.select(), os.popen2(), and subprocess.Popen() return and/or operate on builtin file objects. However even simple applications of these methods and classes require the files to be in non-blocking mode. Currently the built in file type does not support non-blocking mode very well. Setting a file into non-blocking mode and reading or writing to it can only be done reliably by operating on the file.fileno() file descriptor. This requires using the fnctl and os module file descriptor manipulation methods. Details ======= The documentation of file.read() warns; "Also note that when in non-blocking mode, less data than what was requested may be returned, even if no size parameter was given". An empty string is returned to indicate an EOF condition. It is possible that file.read() in non-blocking mode will not produce any data before EOF is reached. Currently there is no documented way to identify the difference between reaching EOF and an empty non-blocking read. The documented behaviour of file.write() in non-blocking mode is undefined. When writing to a file in non-blocking mode, it is possible that not all of the data gets written. Currently there is no documented way of handling or indicating a partial write. The file.read() and file.write() methods are implemented using the underlying C read() and write() fuctions. As a side effect of this, they have the following undocumented behaviour when operating on non-blocking files; A file.write() that fails to write all the provided data immediately will write part of the data, then raise IOError with an errno of EAGAIN. There is no indication how much of the data was successfully written. A file.read() that fails to read all the requested data immediately will return the partial data that was read. A file.read() that fails to read any data immediately will raise IOError with an errno of EAGAIN. Proposed Changes ======================== What is required is to add a setblocking() method that simplifies setting non-blocking mode, and extending/documenting read() and write() so they can be reliably used in non-blocking mode. file.setblocking(flag) Extension -------------------------------- This method implements the socket.setblocking() method for file objects. if flag is 0, the file is set to non-blocking, else to blocking mode. file.read([size]) Changes -------------------------- The read method's current behaviour needs to be documented, so its actual behaviour can be used to differentiate between an empty non-blocking read, and EOF. This means recording that IOError(EAGAIN) is raised for an empty non-blocking read. file.write(str) Changes -------------------- The write method needs to have a useful behaviour for partial non-blocking writes defined, implemented, and documented. This includes returning how many bytes of "str" are successfully written, and raising IOError(EAGAIN) for an unsuccessful write (one that failed to write anything). Impact of Changes ================= As these changes are primarily extensions, they should not have much impact on any existing code. The file.read() changes are only documenting current behaviour. This could have no impact on any existing code. The file.write() change makes this method return an int instead of returning nothing (None). The only code this could affect would be something relying on file.write() returning None. I suspect there is no code that would do this. The file.setblocking() change adds a new method. The only existing code this could affect is code that checks for the presense/absense of a setblocking method on a file. There may be code out there that does this to differentiate between a file and a socket. As there are much better ways to do this, I suspect that there would be no code that does this. Examples ======== For example, the following simple code using popen2 will "hang" if the huge_in string is larger than the os buffering can read/write in one hit. import os child_in, child_out = os.popen2("/usr/bin/cat") child_in.write(huge_in) huge_out = child_out.read() The only safe way to read and write to the popen2 files and avoid blocking, without special knowledge of the io behaviour of the executed command, is to use non-blocking mode. To set a file object "f" into non-blocking mode requires manipulating the file's file descriptor using the Python library fnctl module as follows; import os,fnctl # get the file descriptor fd = f.fileno() # get the file's current flag settings fl = fcntl.fcntl(fd, fcntl.F_GETFL) # update the file's flags to put the file into non-blocking mode. fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK) Once a file is in non-blocking mode, the file object's read() and write() methods cannot reliably be used. Instead you must use the os.read() and os.write() methods on the fileno() of the file; import os str = os.read(f.fileno(), count) count = os.write(f.fileno(), str) Implementation =============== Right now, this functionality can be implemented using an extended file class import os,fnctl class File(file): def setblocking(self,flag): " set/clear blocking mode" # get the file descriptor fd = f.fileno() # get the file's current flag settings fl = fcntl.fcntl(fd, fcntl.F_GETFL) if flag: # clear non-blocking mode from flags fl = fl & ~os.O_NONBLOCK else: # set non-blocking mode from flags fl = fl | os.O_NONBLOCK # update the file's flags fcntl.fcntl(fd, fcntl.F_SETFL, fl) def write(self,str): try: return os.write(self.fileno(),str) except OSError,inst: raise IOError(inst.errno,inst.strerror,inst.filename) A real implementation should be done by modifying the C implementations of the built-in file type. Resources ========= .. [1] Posix write() manual page. (man 3 write) .. [2] Poxix read() manual page. (man 3 read) References ========== Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From jhylton at gmail.com Fri Mar 18 04:56:00 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Mar 18 04:56:02 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <423A0B70.4020705@v.loewis.de> References: <20050317212544.GB3483@panix.com> <4239FEE0.1090209@v.loewis.de> <423A0B70.4020705@v.loewis.de> Message-ID: On Thu, 17 Mar 2005 23:57:52 +0100, "Martin v. L?wis" wrote: > Remember, you were asking what behaviour is *documented*, not what > behaviour is guaranteed by the implementation (in a specific version > of the implementation). Martin, I think you're trying to find more finesse in my question than I ever intended. I intended to ask -- hey, what are the semantics we intend in this case? since the documentation doesn't say, we could improve them by capturing the intended semantics. > > Also, there are other operations that go to the trouble of calling > > flockfile(). What's the point if we don't provide any guarantees? > > Because nobody cares about guarantees in the documentation. Instead, > people care about observable behaviour. So if you get a crash due to a > race condition, you care, you report a bug, the Python developer agrees > its a bug, and fixes it by adding synchronization. As Tim later reported this wasn't to address a crash, but to appease a pig headed developer :-). I'm surprised by your claim that whether something is a bug depends on the person who reviews it. In practice, this may be the case, but I've always been under the impression that there was rough consensus about what constituted a bug and what a feature. I'd certainly say its a goal to strive for. It sounds like the weakest intended behavior we have is the one Tim reported: "provided the platform C stdio wasn't thread-braindead, then if you had N threads all simultaneously reading a file object containing B bytes, while nobody wrote to that file object, then the total number of bytes seen by all N threads would sum to B at the time they all saw EOF." It seems to me like a good idea to document this intended behavior somewhere. Jeremy From andrewm at object-craft.com.au Fri Mar 18 06:11:53 2005 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri Mar 18 06:11:44 2005 Subject: [Python-Dev] Faster Set.discard() method? Message-ID: <20050318051153.025043C74C@coffee.object-craft.com.au> To avoid the exception in the discard method, it could be implemented as: def discard(self, element): """Remove an element from a set if it is a member. If the element is not a member, do nothing. """ try: self._data.pop(element, None) except TypeError: transform = getattr(element, "__as_temporarily_immutable__", None) if transform is None: raise # re-raise the TypeError exception we caught del self._data[transform()] Currently, it's implemented as the much clearer: try: self.remove(element) except KeyError: pass But the dict.pop method is about 12 times faster. Is this worth doing? -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From t-meyer at ihug.co.nz Fri Mar 18 06:28:20 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Fri Mar 18 06:28:28 2005 Subject: [Python-Dev] Faster Set.discard() method? In-Reply-To: Message-ID: > To avoid the exception in the discard method, it could be > implemented as: > > def discard(self, element): > """Remove an element from a set if it is a member. > > If the element is not a member, do nothing. > """ > try: > self._data.pop(element, None) > except TypeError: > transform = getattr(element, > "__as_temporarily_immutable__", None) > if transform is None: > raise # re-raise the TypeError exception we caught > del self._data[transform()] [...] > But the dict.pop method is about 12 times faster. Is this worth doing? The 2.4 builtin set's discard function looks like it does roughly the same as the 2.3 sets.Set. Have you tried comparing a C version of your version with the 2.4 set to see if there are speedups there, too? IMO keeping the sets.Set version as clean and readable as possible is nice, since the reason this exists is for other implementations (Jython, PyPy, ...) and documentation, right? OTOH, speeding up the CPython implementation is nice and it's read by many fewer people. =Tony.Meyer From andrewm at object-craft.com.au Fri Mar 18 06:44:23 2005 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri Mar 18 06:44:14 2005 Subject: [Python-Dev] Faster Set.discard() method? In-Reply-To: References: Message-ID: <20050318054423.8E01F3C74C@coffee.object-craft.com.au> >> But the dict.pop method is about 12 times faster. Is this worth doing? > >The 2.4 builtin set's discard function looks like it does roughly the same >as the 2.3 sets.Set. Have you tried comparing a C version of your version >with the 2.4 set to see if there are speedups there, too? Ah. I had forgotten it was builtin - I'd found the python implementation and concluded the C implementation didn't make it into 2.4 for some reason... 8-) Yes, the builtin set.discard() method is already faster than dict.pop(). >IMO keeping the sets.Set version as clean and readable as possible is nice, >since the reason this exists is for other implementations (Jython, PyPy, >...) and documentation, right? OTOH, speeding up the CPython implementation >is nice and it's read by many fewer people. No, you're right - making sets.Set less readable than it already is would be a step backwards. On the other hand, Jython and PyPy are already in trouble - the builtin set() is not entirely compatible with sets.Set. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From t-meyer at ihug.co.nz Fri Mar 18 07:04:06 2005 From: t-meyer at ihug.co.nz (Tony Meyer) Date: Fri Mar 18 07:04:11 2005 Subject: [Python-Dev] Faster Set.discard() method? In-Reply-To: Message-ID: >>> But the dict.pop method is about 12 times faster. Is this >>> worth doing? >> >> The 2.4 builtin set's discard function looks like it does >> roughly the same as the 2.3 sets.Set. Have you tried comparing >> a C version of your version with the 2.4 set to see if there are >> speedups there, too? > > Ah. I had forgotten it was builtin - I'd found the python > implementation and concluded the C implementation didn't make > it into 2.4 for some reason... 8-) > > Yes, the builtin set.discard() method is already faster than > dict.pop(). The C implementation has this code: """ if (PyDict_DelItem(so->data, item) == -1) { if (!PyErr_ExceptionMatches(PyExc_KeyError)) return NULL; PyErr_Clear(); } """ Which is more-or-less the same as the sets.Set version, right? What I was wondering was whether changing that C to a C version of your dict.pop() version would also result in speedups. Are Exceptions really that slow, even at the C level? =Tony.Meyer From andrewm at object-craft.com.au Fri Mar 18 07:19:44 2005 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri Mar 18 07:19:31 2005 Subject: [Python-Dev] Faster Set.discard() method? In-Reply-To: References: Message-ID: <20050318061944.97E0B3C74C@coffee.object-craft.com.au> >The C implementation has this code: > >""" > if (PyDict_DelItem(so->data, item) == -1) { > if (!PyErr_ExceptionMatches(PyExc_KeyError)) > return NULL; > PyErr_Clear(); > } >""" > >Which is more-or-less the same as the sets.Set version, right? What I was >wondering was whether changing that C to a C version of your dict.pop() >version would also result in speedups. Are Exceptions really that slow, >even at the C level? No, exceptions are fast at the C level - all they do is set a flag. The expense of exceptions is saving a restoring python frames, I think, which doesn't happen in this case. So the current implementation is ideal for C code - clear and fast. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From anthony at python.org Fri Mar 18 07:22:01 2005 From: anthony at python.org (Anthony Baxter) Date: Fri Mar 18 07:22:34 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 2 Message-ID: <200503181722.13891.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.1 (release candidate 2). Python 2.4.1 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of the bugs squished in this release. Assuming no major problems crop up, a final release of Python 2.4.1 will be out around the 29th of March - straight after PyCon. For more information on Python 2.4.1, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.4.1 Highlights of this new release include: - Bug fixes. According to the release notes, several dozen bugs have been fixed, including a fix for the SimpleXMLRPCServer security issue (PSF-2005-001). - A handful other bugs discovered in the first release candidate have been fixed in this version. Highlights of the previous major Python release (2.4) are available from the Python 2.4 page, at http://www.python.org/2.4/highlights.html Enjoy the new release, Anthony Anthony Baxter anthony@python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050318/92043986/attachment.pgp From martin at v.loewis.de Fri Mar 18 07:57:25 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 18 07:57:29 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: <20050317212544.GB3483@panix.com> <4239FEE0.1090209@v.loewis.de> <423A0B70.4020705@v.loewis.de> Message-ID: <423A7BD5.3090806@v.loewis.de> Jeremy Hylton wrote: > It sounds like the weakest intended behavior we have is the one Tim > reported: "provided the platform C stdio wasn't thread-braindead, > then if you had N threads all simultaneously reading a file object > containing B bytes, while nobody wrote to that file object, then the > total number of bytes seen by all N threads would sum > to B at the time they all saw EOF." It seems to me like a good idea > to document this intended behavior somewhere. The guarantee that "we" want to make is certainly stronger: if the threads all read from the same file, each will get a series of "chunks". The guarantee is that it is possible to combine the chunks in a way to get the original contents of the file (i.e. not only the sum of the bytes is correct, but also the contents). However, I see little value adding this specific guarantee to the documentation when so many other aspects of thread interleaving are unspecified. For example, if a thread reads a dictionary simultaneous to a write in another thread, and the read and the write deal with different keys, there is a guarantee that they won't affect each other. If they operate on the same key, the read either gets the old value, or the new value, but not both. And so on. Writing down all these properties does little good, IMO. This includes your proposed property of file reads: anybody reading your statement will think "of course it works this way - why even mention it". Regards, Martin From simon.brunning at gmail.com Fri Mar 18 10:07:19 2005 From: simon.brunning at gmail.com (Simon Brunning) Date: Fri Mar 18 10:07:22 2005 Subject: [Python-Dev] Python 2.4 won the "Jolt productivity award" last night In-Reply-To: <200503171601.03731.gmccaughan@synaptics-uk.com> References: <200503171601.03731.gmccaughan@synaptics-uk.com> Message-ID: <8c7f10c6050318010749e8fb6c@mail.gmail.com> On Thu, 17 Mar 2005 16:01:03 +0000, Gareth McCaughan wrote: > http://www.sdmagazine.com/jolts/ , > > but it's not been updated yet and therefore still has last year's > winners on it. I haven't found anything with more up-to-date > results. The 2005 winners are listed here: http://www.sdmagazine.com/pressroom/jolt_winners_2005.pdf -- Cheers, Simon B, simon@brunningonline.net, http://www.brunningonline.net/simon/blog/ From ncoghlan at iinet.net.au Fri Mar 18 11:57:59 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Fri Mar 18 11:58:04 2005 Subject: [Python-Dev] python-dev Summary for 2005-03-01 through 2005-03-15 [draft] In-Reply-To: <423A3B2D.8030302@ocf.berkeley.edu> References: <423A3B2D.8030302@ocf.berkeley.edu> Message-ID: <423AB437.9040301@iinet.net.au> > ------------------------- > sum() semantics discussed > ------------------------- > Guido's blog entry on `the fate of reduce() in Python 3000`_ (which > reiterated Guido's plan to cut map(), reduce(), filter() and lambdas > (what about zip()?) caused a huge discussion on whether sum() worked the > best way possible. As it stands now, sum() only accepts a sequence of > numbers and its optional second argument works as an initial value to > build off of. That last sentence isn't quite true. With an appropriate second argument, sum can be used to sum any sequence (even one containing strings): Py> class additive_identity(object): ... def __add__(self, other): ... return other ... Py> sum(["a"] * 5, additive_identity()) 'aaaaa' This is fairly abusive of sum, though :) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From p.f.moore at gmail.com Fri Mar 18 14:16:19 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri Mar 18 14:16:21 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <423A7BD5.3090806@v.loewis.de> References: <20050317212544.GB3483@panix.com> <4239FEE0.1090209@v.loewis.de> <423A0B70.4020705@v.loewis.de> <423A7BD5.3090806@v.loewis.de> Message-ID: <79990c6b0503180516d0dabec@mail.gmail.com> On Fri, 18 Mar 2005 07:57:25 +0100, "Martin v. L?wis" wrote: > The guarantee that "we" want to make is certainly stronger: if the > threads all read from the same file, each will get a series of "chunks". > The guarantee is that it is possible to combine the chunks in a way to > get the original contents of the file (i.e. not only the sum of the > bytes is correct, but also the contents). That would be a useful property to be able to rely on, certainly. (Although in practical terms, probably a lot less than people would *like* to see guaranteed :-)) > However, I see little value adding this specific guarantee to the > documentation when so many other aspects of thread interleaving > are unspecified. I'm not sure I agree. It's an improvement in the situation, so why not add it? It may even encourage others, when thinking about threading issues, to consider whether the documentation should guarantee anything - and if so, to add that guarantee. Over time, the documentation gets better at describing thread-related behaviour - and correspondingly, people get (somewhat) more confident that where the documentation doesn't guarantee things, it's because there is a good reason. > For example, if a thread reads a dictionary simultaneous to a write > in another thread, and the read and the write deal with different > keys, there is a guarantee that they won't affect each other. If they > operate on the same key, the read either gets the old value, or the > new value, but not both. If this is a genuine guarantee, then let's document it! I asked about precisely this issue on python-list a long while ago, and no-one could provide me with a confident answer (I couldn't be sure myself, my head explodes when I try to understand thread-related code). The only confident answer I got was "you're safe if you use a lock", but taking that position to extremes results in massive levels of unnecessary serialisation. > Writing down all these properties does little good, IMO. Not a huge amount of good, certainly. But no harm, and a little bit of direct good, and also some indirect good in terms of making it clear that the issue has been thought about. I suppose what I am saying that there is a practical difference between "undefined" and "unknown", even if there isn't a theoretical one... Of course, there's an implied requirement here to confirm any documented guarantees in Jython, and IronPython, and PyPy, and... But given that none of these (yet) implement the full Python 2.4 language definition, as far as I am aware, it's probably not sensible to get too hung up on this fact (although confirming that a guarantee doesn't cause major implementation difficulties would be reasonable). Paul. From p.f.moore at gmail.com Fri Mar 18 14:19:55 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Fri Mar 18 14:19:57 2005 Subject: [Python-Dev] python-dev Summary for 2005-03-01 through 2005-03-15 [draft] In-Reply-To: <423A3B2D.8030302@ocf.berkeley.edu> References: <423A3B2D.8030302@ocf.berkeley.edu> Message-ID: <79990c6b05031805193bd793d6@mail.gmail.com> On Thu, 17 Mar 2005 18:21:33 -0800, Brett C. wrote: > ------------------------ > 2.4.1 should be out soon > ------------------------ > Python 2.4.1c1 is out. Very shortly c2 will be released. Assuming no major > issues come up, 2.4 final will be out. You probably mean something like "2.4.1 final will be out shortly afterwards" (I don't recall if a date has been confirmed). Paul From barry at python.org Fri Mar 18 15:08:20 2005 From: barry at python.org (Barry Warsaw) Date: Fri Mar 18 15:08:38 2005 Subject: [Python-Dev] Faster Set.discard() method? In-Reply-To: <20050318061944.97E0B3C74C@coffee.object-craft.com.au> References: <20050318061944.97E0B3C74C@coffee.object-craft.com.au> Message-ID: <1111154900.20407.10.camel@presto.wooz.org> On Fri, 2005-03-18 at 01:19, Andrew McNamara wrote: > No, exceptions are fast at the C level - all they do is set a flag. The > expense of exceptions is saving a restoring python frames, I think, > which doesn't happen in this case. So the current implementation is > ideal for C code - clear and fast. The other advantage for raising and catching exceptions entirely in C is that the (class) exceptions are never instantiated. Once you cross the C-Python barrier you have to pay for that instantiation. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050318/3dd8d91f/attachment.pgp From simon.brunning at gmail.com Fri Mar 18 15:11:07 2005 From: simon.brunning at gmail.com (Simon Brunning) Date: Fri Mar 18 15:11:10 2005 Subject: [Python-Dev] Python 2.4 won the "Jolt productivity award" last night In-Reply-To: <8c7f10c6050318010749e8fb6c@mail.gmail.com> References: <200503171601.03731.gmccaughan@synaptics-uk.com> <8c7f10c6050318010749e8fb6c@mail.gmail.com> Message-ID: <8c7f10c6050318061173885a6b@mail.gmail.com> On Fri, 18 Mar 2005 09:07:19 +0000, Simon Brunning wrote: > The 2005 winners are listed here: > http://www.sdmagazine.com/pressroom/jolt_winners_2005.pdf Oh, and while I'm breaking cover on python-dev; congratulations to the lot of you for this. You all richly deserve it. -- Cheers, Simon B, simon@brunningonline.net, http://www.brunningonline.net/simon/blog/ From Brenckmann.Sourceforge at gmx.de Fri Mar 18 15:14:47 2005 From: Brenckmann.Sourceforge at gmx.de (Dirk Brenckmann) Date: Fri Mar 18 15:14:49 2005 Subject: [Python-Dev] __metaclass__ problem Message-ID: <1376.1111155287@www21.gmx.net> Hi there, first of all I'd like to introduce myself, because I'm new to this list. If I did wrong to post here, please be patient... The reason for my posting is my previous work with __metaclass__ and advice.py, which is nice to use. While working with __metaclass__ I found situations, where I could not explain the resulting output/behaviour of type.__new__ (super(...,...).__new__) using the available documentation. This is why I've been posting bug no. 1164631 - super(...).__new__( ... ) behaves "unexpected". After having checked the C code of 'typeobject.c', I think I might explain the (undocumented) behaviour now (I've been adding a comment to bug no. 1164631). It is - in short - the following: Lines 1602 to 1626 of 'typeobject.c' decide the "winning" metaclass within any class hierarchy. The effekt I noticed - and still consider to be a bug - is introduced in lines 1611 to 1614: The metaclass of the current class will always be the type of class which is at most subclassed from the metatype previously named as "winner". In consequence a programmer only is in control of the "metaclass" of his class, if he decides it to be a subtype of all former metaclasses he used in his class hierarchy, or if he uses the same metaclass as the superclass does. The reasons, why I consider this a bug is: 1. This feature is undocumented. If there is a documentation of it, i might not have found it - or maybe it was not detailed enough to make a programmer (like me: just starting with metaclasses) understand that point. In either cases it would be great to complete the documentation. 2. The class code using __metaclass__, produced by a programmer sets clear directives for the way the resulting Product (the class) has to be and/or to behave. If instead a product even might behave in a completely other way, because it just intends to, this either is a M$ Product ;-) or a bug. 3. If the intention is fixed by a specification, the bug is not the products/programs behaviour then. Instead the bug is a missing Exception, which should inform the programmer of a programming error which violates the specifications. E.g.: TypeError( "__metaclass__ is not of the expected type." ) (or whatever...) 4. Apart the points 1 to 3 and without knowledge of the discussions you probably had while developing this __metaclass__ thing, I'd call it a pitty, if using supertypes of metaclasses as metaclass in a subtype of a class is prohibited. This would make the whole thing somehow incomplete, because it would result in the need of a programmer to know, all metaclasses that have been used by all classes down the mro to 'object'. ...ok - these are my thoughts to what I called the "__metaclass problem" in the subject of this mail. Of course, I'm not into python as long as you are, so: 1. I might have tried something completely stupid 2. I did not take into account the discussions you already might have had about that. Above all I'm not a native english speaker... ;-) ... please don't flame :-) Any feedback (or bugfixes? :-) welcome. Dirk Brenckmann -- "Happy ProMail" bis 24. März: http://www.gmx.net/de/go/promail Zum 6. Geburtstag gibt's GMX ProMail jetzt 66 Tage kostenlos! From jhylton at gmail.com Fri Mar 18 16:17:40 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Mar 18 16:17:46 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <423A7BD5.3090806@v.loewis.de> References: <20050317212544.GB3483@panix.com> <4239FEE0.1090209@v.loewis.de> <423A0B70.4020705@v.loewis.de> <423A7BD5.3090806@v.loewis.de> Message-ID: On Fri, 18 Mar 2005 07:57:25 +0100, "Martin v. L?wis" wrote: > Writing down all these properties does little good, IMO. This includes > your proposed property of file reads: anybody reading your statement > will think "of course it works this way - why even mention it". The thingsa that are so obvious they don't need to be written down are often the most interesting things to write down. In fact, you started the thread by saying there were no guarantees whatsoever and chiding me for asking if there were any. But it seems there are some intended semantics that are strong than what you would find in C or Perl. Hence, I don't think they would be obvious to anyone who comes to Python from one of those languages. I agree that the semantics of multi-threaded Python programs is an enormous domain and we're discussing a tiny corner of it. I agree that it would be quite challenging to get better documentation or specifications here. But I also think that every little bit helps. Jeremy From sxanth at cs.teiath.gr Sat Mar 19 03:21:45 2005 From: sxanth at cs.teiath.gr (stelios xanthakis) Date: Fri Mar 18 17:12:43 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: References: <20050317212544.GB3483@panix.com> Message-ID: <423B8CB9.7050300@cs.teiath.gr> Jeremy Hylton wrote: >On Thu, 17 Mar 2005 16:25:44 -0500, Aahz wrote: > > >>On Thu, Mar 17, 2005, Jeremy Hylton wrote: >> >> >>>Are the thread semantics for file objecst documented anywhere? I >>>don't see anything in the library manual, which is where I expected to >>>find it. It looks like read and write are atomic by virtue of fread >>>and fwrite being atomic. >>> >>> >>Uncle Timmy will no doubt agree with me: the semantics don't matter. >>NEVER, NEVER access the same file object from multiple threads, unless >>you're using a lock. And even using a lock is stupid. >> >> > >I just want to know >what semantics are intended. If the documentation wants to say that >the semantics are undefined that okay, although I think we need to say >more because some behavior has been provided by the implementation for >a long time. > > I think that when two threads write to the same fd without syncronization, the result is not deterministic anyway. In the case they are reading from the same fd, even worse! (and therefore the input cannot be useful to any serious algorithm) Python (libc in fact) just guarantees that there will be no crashes and corruption of data if the read/write functions are reentered. But ensuring that readline/writeline/etc is atomic would probably be a waste of resources protect the input/output for a case where the data is as good as random noise anyway. So I think aahz is right. Stelios From skip at pobox.com Fri Mar 18 18:06:53 2005 From: skip at pobox.com (Skip Montanaro) Date: Fri Mar 18 18:06:56 2005 Subject: [Python-Dev] Example workaround classes for using Unicode with csv module... Message-ID: <16955.2733.160226.850562@montanaro.dyndns.org> I added UnicodeReader and UnicodeWriter example classes to the csv module docs just now. They mention problems with ASCII NUL characters (which I vaguely remember - NUL-terminated strings are used internally, right?). Do NULs still present a problem? I saw nothing in the log messages that mentioned "ascii" or "nul" so I presume it is. Here's what I added. Let me know if you think it needs any corrections, especially if there's a better way to word "as long as you avoid encodings like utf-16 that use NULs". Can that just be "as long as you avoid multi-byte encodings other than utf-8"? I'd like to have something like this in the docs to demonstrate a reasonable workaround for the current no-Unicode code without casting it in stone by adding it to csv.py. -------------------------------------------------------------------------- The \module{csv} module doesn't directly support reading and writing Unicode, but it is 8-bit clean save for some problems with \ASCII{} NUL characters, so you can write classes that handle the encoding and decoding for you as long as you avoid encodings like utf-16 that use NULs. \begin{verbatim} import csv class UnicodeReader: def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds): self.reader = csv.reader(f, dialect=dialect, **kwds) self.encoding = encoding def next(self): row = self.reader.next() return [unicode(s, self.encoding) for s in row] def __iter__(self): return self class UnicodeWriter: def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds): self.writer = csv.writer(f, dialect=dialect, **kwds) self.encoding = encoding def writerow(self, row): self.writer.writerow([s.encode("utf-8") for s in row]) def writerows(self, rows): for row in rows: self.writerow(row) \end{verbatim} They should work just like the \class{csv.reader} and \class{csv.writer} classes but add an \var{encoding} parameter. -------------------------------------------------------------------------- Thx, Skip From martin at v.loewis.de Fri Mar 18 19:41:35 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 18 19:41:39 2005 Subject: [Python-Dev] thread semantics for file objects In-Reply-To: <423B8CB9.7050300@cs.teiath.gr> References: <20050317212544.GB3483@panix.com> <423B8CB9.7050300@cs.teiath.gr> Message-ID: <423B20DF.70500@v.loewis.de> stelios xanthakis wrote: > I think that when two threads write to the same fd without > syncronization, the result is not > deterministic anyway. In the case they are reading from the same fd, > even worse! (and therefore > the input cannot be useful to any serious algorithm) Yes, but we are not talking about the same fd. Instead, we talk about the same FILE*. A thread-safe libc guarantees (AFAIK) that the data passed to fwrite are appended as a whole. This, in turn, means that the data passed to Python's file.write are also appended as a whole. I'm pretty sure this property also holds on Windows. Regards, Martin From walter at livinglogic.de Fri Mar 18 20:10:39 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri Mar 18 20:10:44 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Doc/lib libcsv.tex, 1.18, 1.19 In-Reply-To: References: Message-ID: <423B27AF.7080206@livinglogic.de> montanaro@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Doc/lib > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22325 > > Modified Files: > libcsv.tex > Log Message: > add UnicodeReader and UnicodeWriter example classes > > [....] > +The \module{csv} module doesn't directly support reading and writing > +Unicode, but it is 8-bit clean save for some problems with \ASCII{} NUL > +characters, so you can write classes that handle the encoding and decoding > +for you as long as you avoid encodings like utf-16 that use NULs. The problem is more the fact that UTF-16 would require a stateful codec. For the UnicodeWriter IMHO the best solution would be to make the csv module unicode transparent (i.e. it simply calls the write method of the underlying stream). Then it should be possible to stack the streams like this: import csv, codecs w = cvs.writer(codecs.getwriter("utf-16")(open("foo.csv", "wb")) w.writerow([u"foo", u"bar")] If csv was implemented in Python this would be trivial. Bye, Walter D?rwald From rodsenra at gpr.com.br Fri Mar 18 21:56:17 2005 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Fri Mar 18 21:56:03 2005 Subject: [Python-Dev] Change 'env var BROWSER override' semantics in webbrowser.py Message-ID: <423B4071.90105@gpr.com.br> Hi, I propose a small change in webbrowse.py module. At the present time: """ Under Unix, if the environment variable BROWSER exists, it is interpreted to override the platform default list of browsers,... """ (extract from Python-2.4/Doc/html/lib/module-webbrowser.html) I propose the following change: ''' --- webbrowser.py 2005-03-18 17:43:58.736854784 -0300 +++ webbrowser.py.new 2005-03-18 17:29:57.016815680 -0300 @@ -355,7 +355,7 @@ if "BROWSER" in os.environ: # It's the user's responsibility to register handlers for any unknown # browser referenced by this value, before calling open(). - _tryorder = os.environ["BROWSER"].split(os.pathsep) + _tryorder = os.environ["BROWSER"].split(os.pathsep) + _tryorder for cmd in _tryorder: if not cmd.lower() in _browsers: ''' This changes the semantics a bit, but in a positive way: - When the environment variable BROWSER is messed up, there is no reason not to try the other detected browser alternatives - If the BROWSER variable is Ok, than it is respected If nobody stand against it, or would like to propose an alternative (optimized) implementation, I can submit a patch to sf to alter both code and documentation. I'd appreciate to know if you consider this a bug or a new feature ? I consider this a bug: """ The webbrowser module provides a very high-level interface to allow displaying Web-based documents to users. The controller objects are easy to use and are platform-independent. Under most circumstances, simply calling the open() function from this module will do the right thing. """ (extract from Python-2.4/Doc/html/lib/module-webbrowser.html) Ignoring valid *already* detected browsers, due to a broken environment variable does not sound like _do the right thing_. cheers, Senra -- Rodrigo Senra MSc Computer Engineer rodsenra@gpr.com.br GPr Sistemas Ltda http://www.gpr.com.br From martin at v.loewis.de Fri Mar 18 23:34:40 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 18 23:34:42 2005 Subject: [Python-Dev] Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <423B4071.90105@gpr.com.br> References: <423B4071.90105@gpr.com.br> Message-ID: <423B5780.1040401@v.loewis.de> Rodrigo Dias Arruda Senra wrote: > I propose a small change in webbrowse.py module. I think I'm generally in favour of such a change. However: - please don't post patches to python-dev, unless you *want* them to be ignored. Typically, nobody will pick up patches from python-dev and apply them, except for rare cases (e.g. urgent bug fixes of serious breakages); post to SF instead. - please accompany your change with a documentation patch. It may be that the exact search procedure for browsers is not specified yet; take that chance and specify it so clearly that the code without your patch would actually conflict with the documentation, so that readers of the new code can verify that this is indeed the documentation-mandated behaviour. - consider integration of test cases. As this involves startup code and environment variables, I would be willing to waive the test case requirement here as unimplementable. However, do consider writing test cases. Regards, Martin From martin at v.loewis.de Fri Mar 18 23:37:14 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Mar 18 23:37:17 2005 Subject: [Python-Dev] RELEASED Python 2.4.1, release candidate 1 In-Reply-To: <20050313095206.F26EA1E4006@bag.python.org> References: <20050313095206.F26EA1E4006@bag.python.org> Message-ID: <423B581A.4030408@v.loewis.de> Vincent Wehren wrote: > To check what I mentioned on comp.lang.python earlier, I ran the installer > again (with 2.4.1 still intact), selected the "Change Python 2.4.1c1" radio > button, clicked the "Finish" Button, clicked the "Advanced" button, clicked > the "Cancel" button, and clicked "Yes" to the question "Are you sure you > want to cancel the Python 2.4.1c1 installation". > This crashed msiexec.exe. I was able to reproduce this on Windows XP > Professional, Service Pack 2. I could reproduce it, but I doubt I can do anything about it. When it asked whether I want to report this to MS, I did - recently, I learned what the magic procedure behind this analysis is, and that there is a good chance that this report really ends up at the developer of installer responsible for the code that crashed. I'm pretty certain it is none of my code which causes the crash (the same rule applies as with Python: it may raise an exception, bring up an error message, and so on - but it should never crash). Regards, Martin From reinhold-birkenfeld-nospam at wolke7.net Fri Mar 18 23:44:30 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Fri Mar 18 23:42:27 2005 Subject: [Python-Dev] Re: Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <423B5780.1040401@v.loewis.de> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Rodrigo Dias Arruda Senra wrote: >> I propose a small change in webbrowse.py module. > > I think I'm generally in favour of such a change. However: > > - please don't post patches to python-dev, unless you *want* > them to be ignored. Typically, nobody will pick up patches > from python-dev and apply them, except for rare cases (e.g. > urgent bug fixes of serious breakages); post to SF instead. > - please accompany your change with a documentation patch. > It may be that the exact search procedure for browsers is > not specified yet; take that chance and specify it so > clearly that the code without your patch would actually > conflict with the documentation, so that readers of > the new code can verify that this is indeed the > documentation-mandated behaviour. > - consider integration of test cases. As this involves > startup code and environment variables, I would be willing > to waive the test case requirement here as unimplementable. > However, do consider writing test cases. Additionally, there are several patches on SF that pertain to webbrowser.py; perhaps you can review some of them... Reinhold -- Mail address is perfectly valid! From ncoghlan at iinet.net.au Sat Mar 19 00:55:04 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 19 00:55:09 2005 Subject: [Python-Dev] __metaclass__ problem In-Reply-To: <1376.1111155287@www21.gmx.net> References: <1376.1111155287@www21.gmx.net> Message-ID: <423B6A58.7000700@iinet.net.au> Dirk Brenckmann wrote: > In consequence a programmer only is in control of the "metaclass" of his > class, if he decides it to be a subtype of all former metaclasses he used in > his class hierarchy, or if he uses the same metaclass as the superclass > does. The behaviour is intentional, but you are correct that it is not fully documented in the official documentation [1]. Some of the 'wrinkles' described in Guido's original Python 2.2 essay [2] may need to be transferred to the docs. For instance: """For new-style metaclasses, there is a constraint that the chosen metaclass is equal to, or a subclass of, each of the metaclasses of the bases. Consider a class C with two base classes, B1 and B2. Let's say M = C.__class__, M1 = B1.__class__, M2 = B2.__class__. Then we require issubclass(M, M1) and issubclass(M, M2). (This is because a method of B1 should be able to call a meta-method defined in M1 on self.__class__, even when self is an instance of a subclass of B1.)""" If you are not getting an exception when breaking this rule, my guess would be that your metaclasses are not inheriting from 'type', or else are not invoking type's __new__ method. The logic to trigger the exception lives in type's __new__ method - if that doesn't get invoked, you won't get the exception. (Note that this also addresses your final objection: if you genuinely don't like the behaviour imposed by using 'type' as the metaclass, then don't use 'type' as the metaclass!) Cheers, Nick. [1] http://www.python.org/dev/doc/devel/ref/metaclasses.html [2] http://www.python.org/2.2/descrintro.html#metaclasses -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From fdrake at acm.org Sat Mar 19 00:55:09 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Sat Mar 19 00:55:33 2005 Subject: [Python-Dev] Re: Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> Message-ID: <200503181855.09229.fdrake@acm.org> On Friday 18 March 2005 17:44, Reinhold Birkenfeld wrote: > Additionally, there are several patches on SF that pertain to > webbrowser.py; perhaps you can review some of them... Given the time I haven't been able to devote to the webbrowser module, a consolidated set of reviews would be very helpful. Patch reviews should be written in the tracker, as always, and a summary of all the webbrowser-related patches in a single email to python-dev. -Fred -- Fred L. Drake, Jr. From ncoghlan at iinet.net.au Sat Mar 19 01:11:41 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 19 01:11:46 2005 Subject: [Python-Dev] __metaclass__ problem In-Reply-To: <423B6A58.7000700@iinet.net.au> References: <1376.1111155287@www21.gmx.net> <423B6A58.7000700@iinet.net.au> Message-ID: <423B6E3D.9080801@iinet.net.au> Nick Coghlan wrote: > If you are not getting an exception when breaking this rule, my guess > would be that your metaclasses are not inheriting from 'type', or else > are not invoking type's __new__ method. The logic to trigger the > exception lives in type's __new__ method - if that doesn't get invoked, > you won't get the exception. OK, I actually read the bug report - I think the 'invalid metaclass' exception should also be getting thrown in the case described there. Py> class Meta1(type): pass ... Py> class Meta2(Meta1): pass ... Py> class MetaA(type): pass ... Py> class C1(object): ... __metaclass__ = Meta1 ... Py> class C2(C1): ... __metaclass__ = MetaA ... Traceback (most recent call last): File "", line 1, in ? TypeError: Error when calling the metaclass bases metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases Py> class C2(C1): ... __metaclass__ = Meta2 ... Py> class C3(C2): ... __metaclass__ = Meta1 ... Py> type(C3) Py> 'Meta1' is NOT a subclass of 'Meta2', yet the exception is not thrown. Instead, the explicitly requested metaclass has been silently replaced with a subclass. I think the OP is justified in calling that 'suprising'. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From pje at telecommunity.com Sat Mar 19 01:33:22 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Mar 19 01:29:58 2005 Subject: [Python-Dev] __metaclass__ problem In-Reply-To: <423B6E3D.9080801@iinet.net.au> References: <423B6A58.7000700@iinet.net.au> <1376.1111155287@www21.gmx.net> <423B6A58.7000700@iinet.net.au> Message-ID: <5.1.1.6.0.20050318193139.03e84680@mail.telecommunity.com> At 10:11 AM 3/19/05 +1000, Nick Coghlan wrote: >Nick Coghlan wrote: >>If you are not getting an exception when breaking this rule, my guess >>would be that your metaclasses are not inheriting from 'type', or else >>are not invoking type's __new__ method. The logic to trigger the >>exception lives in type's __new__ method - if that doesn't get invoked, >>you won't get the exception. > >OK, I actually read the bug report - I think the 'invalid metaclass' >exception should also be getting thrown in the case described there. > >Py> class Meta1(type): pass >... >Py> class Meta2(Meta1): pass >... >Py> class MetaA(type): pass >... >Py> class C1(object): >... __metaclass__ = Meta1 >... >Py> class C2(C1): >... __metaclass__ = Meta2 >... >Py> class C3(C2): >... __metaclass__ = Meta1 >... >Py> type(C3) > >Py> > >'Meta1' is NOT a subclass of 'Meta2', yet the exception is not thrown. >Instead, the explicitly requested metaclass has been silently replaced >with a subclass. I think the OP is justified in calling that 'suprising'. This is precisely the documented (in Guido's essay) behavior. That is, type.__new__ uses the "most derived" of the explicit metaclass and the __class__ attributes of the bases. From gward at python.net Sat Mar 19 02:19:40 2005 From: gward at python.net (Greg Ward) Date: Sat Mar 19 02:19:43 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <1111116627.3762.4.camel@schizo> References: <1111116627.3762.4.camel@schizo> Message-ID: <20050319011940.GA5632@cthulhu.gerg.ca> On 18 March 2005, Donovan Baarda said: > Rationale > ========= > > Many Python library methods and classes like select.select(), os.popen2(), > and subprocess.Popen() return and/or operate on builtin file objects. > However even simple applications of these methods and classes require the > files to be in non-blocking mode. > > Currently the built in file type does not support non-blocking mode very > well. Setting a file into non-blocking mode and reading or writing to it > can only be done reliably by operating on the file.fileno() file descriptor. > This requires using the fnctl and os module file descriptor manipulation > methods. Is having to use fcntl and os really so awful? At least it requires the programmer to prove he knows what he's doing putting this file into non-blocking mode, and that he really wants to do it. ;-) > Details > ======= > > The documentation of file.read() warns; "Also note that when in non-blocking > mode, less data than what was requested may be returned, even if no size > parameter was given". An empty string is returned to indicate an EOF > condition. It is possible that file.read() in non-blocking mode will not > produce any data before EOF is reached. Currently there is no documented > way to identify the difference between reaching EOF and an empty > non-blocking read. > > The documented behaviour of file.write() in non-blocking mode is undefined. > When writing to a file in non-blocking mode, it is possible that not all of > the data gets written. Currently there is no documented way of handling or > indicating a partial write. That's more interesting and a better motivation for this PEP. > file.read([size]) Changes > -------------------------- > > The read method's current behaviour needs to be documented, so its actual > behaviour can be used to differentiate between an empty non-blocking read, > and EOF. This means recording that IOError(EAGAIN) is raised for an empty > non-blocking read. > > > file.write(str) Changes > -------------------- > > The write method needs to have a useful behaviour for partial non-blocking > writes defined, implemented, and documented. This includes returning how > many bytes of "str" are successfully written, and raising IOError(EAGAIN) > for an unsuccessful write (one that failed to write anything). Proposing semantic changes to file.read() and write() is bound to raise hackles. One idea for soothing such objections: only make these changes active when setblocking(False) is in effect. I.e., a setblocking(True) file (the default, right?) behaves as you described above, warts and all. (So old code that uses fcntl() continues to "work" as before.) But files that have had setblocking(False) called could gain these new semantics that you propose. Greg -- Greg Ward http://www.gerg.ca/ I just forgot my whole philosophy of life!!! From ncoghlan at iinet.net.au Sat Mar 19 02:27:40 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sat Mar 19 02:27:45 2005 Subject: [Python-Dev] __metaclass__ problem In-Reply-To: <5.1.1.6.0.20050318193139.03e84680@mail.telecommunity.com> References: <423B6A58.7000700@iinet.net.au> <1376.1111155287@www21.gmx.net> <423B6A58.7000700@iinet.net.au> <5.1.1.6.0.20050318193139.03e84680@mail.telecommunity.com> Message-ID: <423B800C.3010909@iinet.net.au> Phillip J. Eby wrote: > At 10:11 AM 3/19/05 +1000, Nick Coghlan wrote: >> 'Meta1' is NOT a subclass of 'Meta2', yet the exception is not thrown. >> Instead, the explicitly requested metaclass has been silently replaced >> with a subclass. I think the OP is justified in calling that 'suprising'. > > > This is precisely the documented (in Guido's essay) behavior. That is, > type.__new__ uses the "most derived" of the explicit metaclass and the > __class__ attributes of the bases. Hmm, you're right. From Guido's essay [1]: """However, if one of the base metaclasses satisfies the constraint (including the explicitly given __metaclass__, if any), the first base metaclass found satisfying the constraint will be used as the metaclass.""" I missed this when re-reading it earlier. Unfortunately, that means an explicitly set __metaclass__ may be ignored, if a base class has a subclass of the explicitly supplied type as its metaclass (the exact problem the OP was objecting to). IOW, I was extremely surprised to learn that "('__metaclass__' in vars(C)) and (type(C) is C.__metaclass__)" is NOT an invariant Python supports for new-style classes (it breaks for C3 in my example). I have no objection to the standard rule when __metaclass__ is not given, or if it is __metaclass__ that satisifies the constraint. It's only when __metaclass__ *is* given, but doesn't meet the constraint, that I would expect an exception rather than for Python to choose to use one of the base metaclasses instead. Anyway, assuming Guido is happy with the status quo, it just means the above text needs to be included when the __metaclass__ documentation is updated. Cheers, Nick. [1] http://www.python.org/2.2/descrintro.html#metaclasses -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From foom at fuhm.net Sat Mar 19 02:41:25 2005 From: foom at fuhm.net (James Y Knight) Date: Sat Mar 19 02:41:31 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <20050319011940.GA5632@cthulhu.gerg.ca> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> Message-ID: On Mar 18, 2005, at 8:19 PM, Greg Ward wrote: > Is having to use fcntl and os really so awful? At least it requires > the programmer to prove he knows what he's doing putting this file > into non-blocking mode, and that he really wants to do it. ;-) I'd tend to agree. :) Moreover, I don't think fread/fwrite are guaranteed to work as you would expect with non-blocking file descriptors. So, providing a setblocking() call to files would require calling read/write instead of fread/fwrite in all the file methods, at least when in non-blocking mode. I don't think that's a good idea. James From kbk at shore.net Sat Mar 19 04:58:17 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Mar 19 04:58:32 2005 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200503190358.j2J3wHbm020728@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 286 open ( +7) / 2801 closed ( +4) / 3087 total (+11) Bugs : 870 open (+19) / 4867 closed (+14) / 5737 total (+33) RFE : 175 open ( +2) / 150 closed ( +0) / 325 total ( +2) New / Reopened Patches ______________________ inspect.py fix for bug #1143895 (2005-03-09) CLOSED http://python.org/sf/1159931 reopened by arigo inspect.py fix for bug #1143895 (2005-03-09) CLOSED http://python.org/sf/1159931 opened by Simon Percivall use ReleaseItanium configuration for zlib IA64 build (2005-03-09) http://python.org/sf/1160164 opened by Trent Mick skip winsound for Windows/IA64 build (2005-03-09) http://python.org/sf/1160169 opened by Trent Mick Add method to function objects to simplify decoration (2005-03-12) http://python.org/sf/1161819 opened by Nick Coghlan python-config (2005-03-12) http://python.org/sf/1161914 opened by Andre Jonsson don't add -OPT:Olimit=0 for icc (2005-03-12) http://python.org/sf/1162023 opened by Michael Hoffman Heap class for heapq module (2005-03-13) http://python.org/sf/1162363 opened by Stefan Behnel EditorWindow's title with non-ASCII chars. (2005-03-14) http://python.org/sf/1162825 opened by SUZUKI Hisao _POSIX_SEMAPHORES checked incorrectly (2005-03-14) CLOSED http://python.org/sf/1163249 opened by Bob Ippolito the quotes page on the Web site links to something broken (2005-03-14) http://python.org/sf/1163314 opened by Shannon -jj Behrens small sanity checks for user-defined mros (2005-03-15) http://python.org/sf/1163731 opened by Michael Hudson Using size_t where appropriate (2005-03-18) http://python.org/sf/1166195 opened by Martin v. L?wis Patches Closed ______________ inspect.py fix for bug #1143895 (2005-03-09) http://python.org/sf/1159931 closed by jlgijsbers inspect.py fix for bug #1143895 (2005-03-09) http://python.org/sf/1159931 closed by jlgijsbers Syntax for iterator slicing, concatenation and repetition (2005-01-24) http://python.org/sf/1108272 closed by ncoghlan fix for a deadlock in the logging module (2005-03-07) http://python.org/sf/1158052 closed by vsajip _POSIX_SEMAPHORES checked incorrectly (2005-03-15) http://python.org/sf/1163249 closed by anthonybaxter New / Reopened Bugs ___________________ Setup file needs entries for collections, itertools, strop (2005-03-09) CLOSED http://python.org/sf/1160187 opened by Niraj Bajpai urllib2 post error when using httpproxy (2005-03-10) http://python.org/sf/1160328 opened by small tiger digit-only tag values are mishandled in Tkinter (2005-03-09) http://python.org/sf/1160383 opened by Ilya Sandler Autoconf failure on FreeBSD 5.3, and AC_INIT set incorrectly (2005-03-10) CLOSED http://python.org/sf/1160534 opened by Richard Brooksby Can't build Zope on Windows w/ 2.4.1c1 (2005-03-10) CLOSED http://python.org/sf/1160802 opened by Tim Peters Neverending warnings from asyncore (2005-03-11) http://python.org/sf/1161031 opened by Tony Meyer Install problem 2.4.1rc1 on Win98 (2005-03-11) CLOSED http://python.org/sf/1161187 opened by Spider Incorrect return value in gc docs (2005-03-11) CLOSED http://python.org/sf/1161476 opened by Aravind Minor error in section 3.2 (2005-03-11) http://python.org/sf/1161595 opened by Jeremy Barbay configure incorrectly adds -OPT:Olimit=0 for icc (2005-03-12) http://python.org/sf/1162001 opened by Michael Hoffman inspect.getmembers() breaks sometimes (2005-03-12) http://python.org/sf/1162154 opened by Doug Quale subprocess.Popen with copious output hangs (2005-03-13) http://python.org/sf/1162428 opened by neuhauser Parsing failures in parsedate_tz (2005-03-13) http://python.org/sf/1162477 opened by TH Unable to Install Python 2.4.1rc1 on windows XP SP2 (2005-03-13) http://python.org/sf/1162677 opened by Sergio Correia typesseq-mutable lacks note on combined key/cmp usage (2005-03-14) http://python.org/sf/1162912 opened by Stefan Behnel IDNA StreamReader broken (2005-03-14) http://python.org/sf/1163178 opened by Walter D?rwald Syntax error on large file with MBCS encoding (2005-03-14) http://python.org/sf/1163244 opened by Tim N. van der Leeuw "special" decimals aren't hashable (2005-03-14) CLOSED http://python.org/sf/1163325 opened by Marien Zwart correct/clarify documentation for super (2005-03-14) http://python.org/sf/1163367 opened by Steven Bethard uncaught AttributeError deep in urllib (2005-03-14) http://python.org/sf/1163401 opened by K Lars Lohn Sub threads execute in restricted mode (2005-03-15) http://python.org/sf/1163563 opened by anothermax subprocess pipe fails under Emacs (2005-03-15) http://python.org/sf/1163759 opened by Anders J. Munch super(...).__new__( ... ) behaves "unexpected" (2005-03-16) http://python.org/sf/1164631 opened by Dirk Brenckmann UserDict is not iterable (2005-03-16) CLOSED http://python.org/sf/1164726 opened by Kent Johnson Tkdnd.py crashes due to read-only attributes (2005-03-16) http://python.org/sf/1164742 opened by tynods xmlrpclib.DateTime.decode() should stringify argument (2005-03-16) http://python.org/sf/1164912 opened by Allan Saddi logging.basicConfig creates phantom handler (2005-03-16) CLOSED http://python.org/sf/1164953 opened by logistix Property access with decorator makes interpreter crash (2005-03-17) http://python.org/sf/1165306 opened by Remy Blank KeyboardInterrupt causes segmentation fault with threads (2005-03-17) http://python.org/sf/1165761 opened by Jeff Stearns SSL_pending is not used (2005-03-18) http://python.org/sf/1166206 opened by hahahhah missing sequence tests - pull from deque (2005-03-18) http://python.org/sf/1166274 opened by Jim Jewett No os.spawn*p* on Windows (2005-03-19) http://python.org/sf/1166378 opened by Chris Palmer Bugs Closed ___________ inspect.getsource() breakage in 2.4 (2005-02-18) http://python.org/sf/1143895 closed by jlgijsbers Setup file needs entries for collections, itertools, strop (2005-03-09) http://python.org/sf/1160187 closed by rhettinger unixccompiler.py should deal with env in linker (2005-03-07) http://python.org/sf/1158005 closed by jackjansen Autoconf failure on FreeBSD 5.3, and AC_INIT set incorrectly (2005-03-10) http://python.org/sf/1160534 closed by loewis Can't build Zope on Windows w/ 2.4.1c1 (2005-03-10) http://python.org/sf/1160802 closed by tim_one test_socket fails when not connected (2003-03-19) http://python.org/sf/706450 closed by bcannon Install problem 2.4.1rc1 on Win98 (2005-03-11) http://python.org/sf/1161187 closed by loewis Incorrect return value in gc docs (2005-03-11) http://python.org/sf/1161476 closed by iamfenris [2.4 regression] seeking in codecs.reader broken (2005-03-03) http://python.org/sf/1156259 closed by doerwalter configure and gmake fail in openbsd 3.5 i386 (2004-06-24) http://python.org/sf/978632 closed by loewis "special" decimals aren't hashable (2005-03-14) http://python.org/sf/1163325 closed by rhettinger Warnings in Python.h with gcc 4.0.0 (2005-01-20) http://python.org/sf/1105699 closed by loewis UserDict is not iterable (2005-03-16) http://python.org/sf/1164726 closed by rhettinger logging.basicConfig creates phantom handler (2005-03-17) http://python.org/sf/1164953 closed by vsajip New / Reopened RFE __________________ expm1 and log1p (2005-03-14) http://python.org/sf/1163139 opened by Keith Briggs ConfigParser alternative key-value delimitier (2005-03-17) http://python.org/sf/1165404 opened by Sergey Dorofeev From aahz at pythoncraft.com Sat Mar 19 15:51:27 2005 From: aahz at pythoncraft.com (Aahz) Date: Sat Mar 19 15:51:29 2005 Subject: [Python-Dev] Python 2.4 won the "Jolt productivity award" last night In-Reply-To: <8c7f10c6050318010749e8fb6c@mail.gmail.com> References: <200503171601.03731.gmccaughan@synaptics-uk.com> <8c7f10c6050318010749e8fb6c@mail.gmail.com> Message-ID: <20050319145126.GA6322@panix.com> On Fri, Mar 18, 2005, Simon Brunning wrote: > On Thu, 17 Mar 2005 16:01:03 +0000, Gareth McCaughan > wrote: >> >> http://www.sdmagazine.com/jolts/ , >> >> but it's not been updated yet and therefore still has last year's >> winners on it. I haven't found anything with more up-to-date >> results. > > The 2005 winners are listed here: > http://www.sdmagazine.com/pressroom/jolt_winners_2005.pdf I've put this up on the website, but http://www.sdmagazine.com/pressroom/ claims that press releases are only up for two months, so we'll need a permanent URL later. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From bac at OCF.Berkeley.EDU Sat Mar 19 16:12:02 2005 From: bac at OCF.Berkeley.EDU (Brett Cannon) Date: Sat Mar 19 16:12:09 2005 Subject: [Python-Dev] python-dev Summary for 2005-03-01 through 2005-03-15 [draft] In-Reply-To: <423AB437.9040301@iinet.net.au> References: <423A3B2D.8030302@ocf.berkeley.edu> <423AB437.9040301@iinet.net.au> Message-ID: [Nick Coghlan] >> ------------------------- >> sum() semantics discussed >> ------------------------- >> Guido's blog entry on `the fate of reduce() in Python 3000`_ (which >> reiterated Guido's plan to cut map(), reduce(), filter() and lambdas (what >> about zip()?) caused a huge discussion on whether sum() worked the best way >> possible. As it stands now, sum() only accepts a sequence of numbers and >> its optional second argument works as an initial value to build off of. > > That last sentence isn't quite true. With an appropriate second argument, sum > can be used to sum any sequence (even one containing strings): > > Py> class additive_identity(object): > ... def __add__(self, other): > ... return other > ... > Py> sum(["a"] * 5, additive_identity()) > 'aaaaa' > That is slightly evil, and here is a simpler example of evilness: sum([[1],[2],[3]], []) Had to look at the source to understand how it was working. =) First thing learned while at the sprints! -Brett From bac at OCF.Berkeley.EDU Sat Mar 19 16:13:44 2005 From: bac at OCF.Berkeley.EDU (Brett Cannon) Date: Sat Mar 19 16:13:53 2005 Subject: [Python-Dev] python-dev Summary for 2005-03-01 through 2005-03-15 [draft] In-Reply-To: <79990c6b05031805193bd793d6@mail.gmail.com> References: <423A3B2D.8030302@ocf.berkeley.edu> <79990c6b05031805193bd793d6@mail.gmail.com> Message-ID: [Paul Moore] > On Thu, 17 Mar 2005 18:21:33 -0800, Brett C. wrote: >> ------------------------ >> 2.4.1 should be out soon >> ------------------------ >> Python 2.4.1c1 is out. Very shortly c2 will be released. Assuming no major >> issues come up, 2.4 final will be out. > > You probably mean something like "2.4.1 final will be out shortly > afterwards" (I don't recall if a date has been confirmed). > Yes I did. =) Fixed. -Brett From kbk at shore.net Sat Mar 19 18:49:20 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Mar 19 18:49:49 2005 Subject: [Python-Dev] python-dev Summary for 2005-03-01 through 2005-03-15 [draft] In-Reply-To: <423AB437.9040301@iinet.net.au> (Nick Coghlan's message of "Fri, 18 Mar 2005 20:57:59 +1000") References: <423A3B2D.8030302@ocf.berkeley.edu> <423AB437.9040301@iinet.net.au> Message-ID: <87hdj7plun.fsf@hydra.bayview.thirdcreek.com> Nick Coghlan writes: > That last sentence isn't quite true. With an appropriate second > argument, sum can be used to sum any sequence (even one containing > strings): > > Py> class additive_identity(object): > ... def __add__(self, other): > ... return other > ... ==> setup > Py> sum(["a"] * 5, additive_identity()) > 'aaaaa' > > This is fairly abusive of sum, though :) Python 2.5a0 (#85, Jan 27 2005, 17:41:04) [GCC 2.95.3 20010125 (prerelease, propolice)] on openbsd3 Type "copyright", "credits" or "license()" for more information. IDLE 1.2a0 Py> t = timeit.Timer("sum(('a','bcd', 'e'), ai())", setup) Py> t.repeat() [3.3861918449401855, 3.2027261257171631, 3.1891348361968994] Py> t2 = timeit.Timer("''.join(('a','bcd', 'e'))") Py> t2.repeat() [1.1249339580535889, 1.1143810749053955, 1.0990779399871826] Py> t3 = timeit.Timer("'a' + 'bcd' + 'e'") Py> t3.repeat() [0.10233211517333984, 0.099857091903686523, 0.10514688491821289] -- KBK From jafo at tummy.com Sat Mar 19 19:50:20 2005 From: jafo at tummy.com (Sean Reifschneider) Date: Sat Mar 19 21:50:03 2005 Subject: [Python-Dev] bdist_deb checkin comments Message-ID: <20050319185020.GB13774@tummy.com> I'm hoping to get the bdist_deb patch committed this week. This is SF patch 1054967: https://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 Does anyone have any feedback on this before I do so? I'm imagining committing it into the Python CVS, but as my first non-RPM commit I'd like some feedback if that would be the wrong thing to do. Thanks, Sean -- Language is the most important .. uh.. I think you know what I'm trying to say. -- Steve Martin Sean Reifschneider, Member of Technical Staff tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin From phd at phd.pp.ru Sat Mar 19 21:28:37 2005 From: phd at phd.pp.ru (Oleg Broytmann) Date: Sat Mar 19 21:50:05 2005 Subject: [Python-Dev] Re: Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <200503181855.09229.fdrake@acm.org> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> Message-ID: <20050319202836.GA6273@phd.pp.ru> On Fri, Mar 18, 2005 at 06:55:09PM -0500, Fred L. Drake, Jr. wrote: > On Friday 18 March 2005 17:44, Reinhold Birkenfeld wrote: > > Additionally, there are several patches on SF that pertain to > > webbrowser.py; perhaps you can review some of them... > > Given the time I haven't been able to devote to the webbrowser module, a > consolidated set of reviews would be very helpful. > > Patch reviews should be written in the tracker, as always, and a summary of > all the webbrowser-related patches in a single email to python-dev. I am in game, as one of those patches is mine. I've started to review patches... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From kbk at shore.net Sun Mar 20 00:20:44 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Sun Mar 20 00:20:58 2005 Subject: [Python-Dev] bdist_deb checkin comments In-Reply-To: <20050319185020.GB13774@tummy.com> (Sean Reifschneider's message of "Sat, 19 Mar 2005 11:50:20 -0700") References: <20050319185020.GB13774@tummy.com> Message-ID: <87acozp6ib.fsf@hydra.bayview.thirdcreek.com> Sean Reifschneider writes: > Does anyone have any feedback on this before I do so? I made a few comments on the Tracker. -- KBK From ncoghlan at iinet.net.au Sun Mar 20 02:47:44 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sun Mar 20 02:47:49 2005 Subject: [Python-Dev] identity operands (was python-dev Summary for 2005-03-01 through 2005-03-15 [draft]) In-Reply-To: <87hdj7plun.fsf@hydra.bayview.thirdcreek.com> References: <423A3B2D.8030302@ocf.berkeley.edu> <423AB437.9040301@iinet.net.au> <87hdj7plun.fsf@hydra.bayview.thirdcreek.com> Message-ID: <423CD640.9000102@iinet.net.au> Kurt B. Kaiser wrote: > Nick Coghlan writes: >>This is fairly abusive of sum, though :) > [snip Kurt's timings] Even avoiding the object instantiation doesn't help much: Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. Py> setup = """\ ... class additive_identity(object): ... def __add__(self, other): ... return other ... ... ai = additive_identity() ... """ Py> t = timeit.Timer("sum(('a', 'bcd', 'e'), ai)", setup) Py> t.repeat() [1.7930380266773089, 1.7397206526577538, 1.7193376076378759] Py> t = timeit.Timer("''.join(('a', 'bcd', 'e'))") Py> t.repeat() [0.58006451058344055, 0.58431742467269032, 0.5788117914319173] Py> t = timeit.Timer("'a' + 'bcd' + 'e'") Py> t.repeat() [0.29404383490157215, 0.29554694930084224, 0.29612594514117063] (I almost forgot to repeat your other tests to account for the differences in machine speed!) So, using "".join is roughly three times as fast as abusing sum :) I'm still intrigued by the concept of providing an object or objects (e.g. in operator) that work as an identity operand for all cases where it makes sense: Commutative operations (always return 'other'): __add__(self, other) __radd__(self, other) __mul__(self, other) __rmul__(self, other) __xor__(self, other) __rxor__(self, other) __and__(self, other) __rand__(self, other) __or__(self, other) __ror__(self, other) Non-commutative operations with identity on the right (return 'other') __rsub__(self, other) __rdiv__(self, other) __rtruediv__(self, other) __rfloordiv__(self, other) __rmod__(self, other) __rdivmod__(self, other) __rpow__(self, other) __rlshift__(self, other) __rrshift__(self, other) Other non-commutative operations with a sensible identity: __sub__(self, other): return -other Non-commutative operations without a sensible identity: __div__(self, other) __truediv__(self, other) __floordiv__(self, other) __mod__(self, other) __divmod__(self, other) __pow__(self, other[, modulo]) __lshift__(self, other) __rshift__(self, other) If it was done, it would probably be best to do this as at least two objects - one for which bool(additive_identity) evaluates to False, and the other for which bool(multiplicative_identity) evaluates to True. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From jrodrigog at gmail.com Sun Mar 20 03:25:24 2005 From: jrodrigog at gmail.com (Juan Carlos Rodrigo Garcia) Date: Sun Mar 20 03:25:25 2005 Subject: [Python-Dev] Python 2.4 | 7.3 The for statement Message-ID: <5dcb081c05031918257536d79@mail.gmail.com> Hi My name is Juan Carlos Rodrigo, and I love Python. It is the most impressive and usefull language that I have ever seen. I am studing, at http://www.uoc.edu, an Information Technology Postgraduate. And I have programmed some REXX applications in my past Jobs (So I love python, no more ENDS). This is my Graduate final project http://ip.xpyro.com made in mod_python. I love mod_python too. Please don't crash my AMD making queries. :| --------8<--------8<--------8<--------8<--------8<--------8<-------- Python 2.4 | 7.3 The for statement: ----------------------------------- for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" suite] New for statement: ------------------ for_stmt ::= "for" target_list "in" expression_list [ "and" expression ] ":" suite ["else" ":" suite] ** If the expression evaluates to False before entering the for, jump else. ** If the expression is evaluated to False after the first iteration, break. So ?What we can do with this new for?, and ?It is going to avoid line exceed?: "My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed." [1] It is easier if we see it beforehand: ------------------------------------- leave = False alist = [1,2,3,3,4,5,6,7,8,9] for item in alist and not leave: if item is 1: leave = True Avoiding code exceed: --------------------- a = 1 b = 2 c = 3 alist = [1,2,3,4,5,6,7,8,9] for item in alist and a < 2 and b < 3 and c < 4: if item == 3: a += 1 if item == 2: b += 1 if item == 1: c += 1 print "%d %d %d" % (a,b,c) # three lines off (25% less on breaks) Other features and the else: ---------------------------- alist = [1,2,3] enter = False if list[0] == 4: enter = True for item in alist and enter: print item else: print "No account" The real problem: ----------------- "The exercise to translate an arbitrary flow diagram more or less mechanically into a jump-less one, however, is not to be recommended." [1] Ok, it's not recommended, at large, but Python should make it possible, and then the people will choose. [1] Go To Statement Considered Harmful Edsger W. Dijkstra http://www.acm.org/classics/oct95/ PD: Your work is impressive, thanks. -- Juan Carlos Rodrigo Garcia. jrodrigog@gmail.com From ncoghlan at iinet.net.au Sun Mar 20 03:46:32 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sun Mar 20 03:46:38 2005 Subject: [Python-Dev] Python 2.4 | 7.3 The for statement In-Reply-To: <5dcb081c05031918257536d79@mail.gmail.com> References: <5dcb081c05031918257536d79@mail.gmail.com> Message-ID: <423CE408.9050001@iinet.net.au> Juan Carlos Rodrigo Garcia wrote: > It is easier if we see it beforehand: > ------------------------------------- > > leave = False > alist = [1,2,3,3,4,5,6,7,8,9] > for item in alist and not leave: > if item is 1: leave = True Interesting idea, but not really needed given the existence of the break statement: for item in alist: if item is 1: break Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Sun Mar 20 04:29:43 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sun Mar 20 04:29:47 2005 Subject: [Python-Dev] Python 2.4 | 7.3 The for statement In-Reply-To: <1111287740.23373.0.camel@workspot.xpyro.com> References: <5dcb081c05031918257536d79@mail.gmail.com> <423CE408.9050001@iinet.net.au> <1111287740.23373.0.camel@workspot.xpyro.com> Message-ID: <423CEE27.7050202@iinet.net.au> Juan Carlos Rodrigo wrote: >>Interesting idea, but not really needed given the existence of the break statement: > > > Goto = break > I'm not interested. All non-sequential control structures are merely constrained ways of using goto (the underlying machine code will devolve into conditional and unconditional branches and jumps - i.e. goto's). 'break' is a highly constrained form of goto and a fundamental part of structured programming (as is 'continue') - each is limited to a local effect on the loop that contains them. This is a far cry from the ability to jump to an arbitrary label that gives goto its bad reputation. I used to share your sentiment regarding break and continue - experience (especially Python experience) has convinced me otherwise. Python embraces the concept of breaking out of a loop to the point that it even has an 'else' clause on loop structures that is executed only if the loop is exited naturally rather than via a break statement. Regardless of whether you personally choose to use break and continue, the existence of those statements is the main reason that the addition of a flag condition to for loops is highly unlikely. If you want to terminate a for loop before the iterable is exhausted, the recommended solution is to use a break statement. To illustrate the point about all control structures being gotos, even a simple do-nothing for loop results in a JUMP_ABSOLUTE (goto!) in the generated bytecode: Py> def f(): ... for item in range(10): ... pass ... Py> import dis Py> dis.dis(f) 2 0 SETUP_LOOP 20 (to 23) 3 LOAD_GLOBAL 0 (range) 6 LOAD_CONST 1 (10) 9 CALL_FUNCTION 1 12 GET_ITER >> 13 FOR_ITER 6 (to 22) 16 STORE_FAST 0 (item) 3 19 JUMP_ABSOLUTE 13 >> 22 POP_BLOCK >> 23 LOAD_CONST 0 (None) 26 RETURN_VALUE Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From aahz at pythoncraft.com Sun Mar 20 05:32:44 2005 From: aahz at pythoncraft.com (Aahz) Date: Sun Mar 20 05:32:47 2005 Subject: [Python-Dev] Python 2.4 | 7.3 The for statement In-Reply-To: <5dcb081c05031918257536d79@mail.gmail.com> References: <5dcb081c05031918257536d79@mail.gmail.com> Message-ID: <20050320043244.GB15570@panix.com> On Sun, Mar 20, 2005, Juan Carlos Rodrigo Garcia wrote: > > Hi My name is Juan Carlos Rodrigo, and I love Python. It is the most > impressive and usefull language that I have ever seen. Glad to hear that! However, your post is off-topic for python-dev; you'll have a better discussion posting to comp.lang.python. Thanks. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From ncoghlan at iinet.net.au Sun Mar 20 10:27:06 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Sun Mar 20 10:27:13 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches Message-ID: <423D41EA.6080009@iinet.net.au> What's the current situation with providing fixes for AST branch problems? There's a simple one in parsenumber in Python/ast.c relating to int/long unification and octal literals. The fix is just a direct copy of the relevant code from parsenumber in the old compile.c. Fixing it means the only failures left in test_grammar are those relating to generator expressions. I've put a patch on SF (1166879) and assigned it to Jeremy with group AST. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From rodsenra at gpr.com.br Sun Mar 20 15:28:02 2005 From: rodsenra at gpr.com.br (rodsenra@gpr.com.br) Date: Sun Mar 20 15:31:17 2005 Subject: [Python-Dev] Patch review: all webbrowser.py related patches up to 2005-03-20 Message-ID: <1301.200.158.47.2.1111328882.squirrel@200.158.47.2> 1144816 webbrowser.Netscape.open bug fix ----------------------------------------------------------------- 1077979 Simple webbrowser fix for netscape -remote ----------------------------------------------------------------- 1144816 and 1077979 are the the same patch, as documented in a comment for 1144816 by Oleg Broytmann. The wrong behaviour was reported to happen in Mozilla (unspecified version), Netscape 7.2 and Mozilla-firefox (unspecified version). I could not reproduce the problem neither with Mozilla 1.7.2 nor with Firefox 1.0.1. Nevertheless, applying the patch does not break current functionality and might fix bugs in older browsers. I recommend applying 1077979, and closing 1144816. 954628 Replacement for webbrowser.py which has problems. ----------------------------------------------------------------- There is no patch attached, but the whole file was uploaded. Twice! There is already a comment by Oleg Broytmann telling the person to re-submit it in as a proper patch. There are multiple changes in it, perhaps multiples patches. Each addresing a single change. Some of the proposed changes are declared untested. Some changes are controversial, like using a new environment variable PYTHONBROWSER prior to checking BROWSER. There are no references to this being discussed in python-dev. IMO this adds unecessary complexity, if the user is not satisfied with the BROWSER settings is easier and better to reconfigured it than to create a new variable. I recommend closing 954628. 728278 Multiple webbrowser.py bug fixes / improvements ------------------------------------------------------------------ Against the [http://python.org/patches/ Patch Submission Guidelines] this entry has uploaded both the whole file and .tgz, but no patch file. The large numbers of changes make it difficult to review. All the changes were documented to the top of the file, I don't believe they belong there. At least that is not complaint to the first rule in http://www.python.org/patches/style.html There are several OS X specific corrections that I'm unable to reproduce/validate, since I have no access to that platform. The modules was renamed to wingwebbrowser.py and the test case was built using this name. I believe this is inadequate, the original module name should be kept. There are three categories of changes: bug fixes, internal changes and interface changes. IMO it would be better to break this patch in at least three, matching these types of changes. Bug fixes would have a higher priority, less chance to break backward compatibility and more chance to be incorporated earlier. It should be applied before 1077979, otherwise that fix would be reversed. In spite of the formatting comments above, there are *valuable* changes in the submitted file. This is a nice candidate to be tackled in a Python Bug Day, since it functionality involves a broad range of platforms and user browsers preferences. Moreover, thourough test cases are hard to be cooked. I recommend keeping it open until somebody reshaped it properly and then applying it. 754022 Enhanced webbrowser.py -------------------------------------------------- This patch is properly formatted. I would change 'return 1' for 'return True', since this targets Py 2.4.x or above. It address some of the issues already addressed in entry 728278, since it proposes less changes and it is properly formatted I recommend applying it before 728278. 1166780 Fix _tryorder in webbrowser.py ----------------------------------------------------------- This is my own patch. At the present time (Py2.4) documentation says: """ Under Unix, if the environment variable BROWSER exists, it is interpreted to override the platform default list of browsers,... """ (extract from Python-2.4/Doc/html/lib/module-webbrowser.html) But, when the environment variable BROWSER is messed up, there is no reason not to try the other detected browser alternatives. In this patch, if the BROWSER variable is Ok, than it is respected. Otherwise, the previously detected browsers are tryied out as if BROWSER variable never existed. This does not break backward compatibility and adds more chance for webbrowser.open() to succeed. This patch was made against CVS head 2005-03-20, and aims to 2.5, but can safely be apllied to any 2.4.x bug fix release. ---------------------------------- best regards, Rod Senra From rodsenra at gpr.com.br Sun Mar 20 15:31:27 2005 From: rodsenra at gpr.com.br (rodsenra@gpr.com.br) Date: Sun Mar 20 15:34:41 2005 Subject: [Python-Dev] Re: Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <200503181855.09229.fdrake@acm.org> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> Message-ID: <1309.200.158.47.2.1111329087.squirrel@200.158.47.2> > On Friday 18 March 2005 17:44, Reinhold Birkenfeld wrote: > > Additionally, there are several patches on SF that pertain to > > webbrowser.py; perhaps you can review some of them... By all means. Done! > Given the time I haven't been able to devote to the webbrowser module, a > consolidated set of reviews would be very helpful. > > Patch reviews should be written in the tracker, as always, and a summary > of all the webbrowser-related patches in a single email to python-dev. Thank you. That was valuable information. best regards, Rod Senra From rodsenra at gpr.com.br Sun Mar 20 15:40:27 2005 From: rodsenra at gpr.com.br (rodsenra@gpr.com.br) Date: Sun Mar 20 15:43:41 2005 Subject: [Python-Dev] Re: Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <20050319202836.GA6273@phd.pp.ru> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> Message-ID: <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> > I am in game, as one of those patches is mine. I've started to review > patches... Hi Oleg, Perhaps you could focus in 728278. It addresses some of the issues you have addressed in 754022, but it is not properly formatted. If you could merge into your patch the result of "set(728278)-set(754022)", it would be great. These two patches have the biggest number of changes, and most significant ones. Naturally they are also the most conflicting. If these two are merged, then I believe *all* webbrowser.py related patches could be addressed and closed quickly. best regards, Rod Senra From rodsenra at gpr.com.br Sun Mar 20 15:59:29 2005 From: rodsenra at gpr.com.br (rodsenra@gpr.com.br) Date: Sun Mar 20 16:02:41 2005 Subject: [Python-Dev] Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <423B5780.1040401@v.loewis.de> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> Message-ID: <1424.200.158.47.2.1111330769.squirrel@200.158.47.2> > Rodrigo Dias Arruda Senra wrote: >> I propose a small change in webbrowse.py module. > > I think I'm generally in favour of such a change. Thanks. I'm glad to know! > - please don't post patches to python-dev, Sorry. I'm aware of that. For clarification's sake, I was not _posting_ a patch to the mailing list. My intention was to *discuss* the patch, and since it was so small, I found easier to copy it than to explain it. Since nobody opposed to the idea, I already have submitted a patch to the tracker (1166780 Fix _tryorder in webbrowser.py). > - please accompany your change with a documentation patch. Indeed. Already available in sf. > I would be willing to waive the test case requirement here as > unimplementable. That is a relief in this particular case . Thank you for all your advice. best regards, Rod Senra From bac at ocf.berkeley.edu Sun Mar 20 16:08:09 2005 From: bac at ocf.berkeley.edu (Brett C.) Date: Sun Mar 20 16:08:15 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches In-Reply-To: <423D41EA.6080009@iinet.net.au> References: <423D41EA.6080009@iinet.net.au> Message-ID: <423D91D9.1080009@ocf.berkeley.edu> Nick Coghlan wrote: > What's the current situation with providing fixes for AST branch problems? > Make sure "AST" is used in the subject line; e.g., "[AST]" at the beginning. Unfortunately the AST group is only available for patches; not listed for bug reports (don't know why; can this be fixed?). Other than that, just assign it to me since I will most likely be doing AST work in the near future. -Brett From tim.peters at gmail.com Sun Mar 20 16:53:00 2005 From: tim.peters at gmail.com (Tim Peters) Date: Sun Mar 20 16:53:02 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches In-Reply-To: <423D91D9.1080009@ocf.berkeley.edu> References: <423D41EA.6080009@iinet.net.au> <423D91D9.1080009@ocf.berkeley.edu> Message-ID: <1f7befae050320075358638b27@mail.gmail.com> [Brett C.] > Make sure "AST" is used in the subject line; e.g., "[AST]" at the beginning. > Unfortunately the AST group is only available for patches; not listed for bug > reports (don't know why; can this be fixed?). Your wish is my command: there's an AST group in Python's bug tracker now. FYI, each tracker has a distinct set of metadata choices, and nothing shows up in any of 'em by magic. > Other than that, just assign it to me since I will most likely be doing AST > work in the near future. Unfortunately, auto-assign keys off Category instead of Group metadata. From phd at mail2.phd.pp.ru Sun Mar 20 17:22:33 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Sun Mar 20 17:22:41 2005 Subject: [Python-Dev] Re: Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> Message-ID: <20050320162233.GC28606@phd.pp.ru> On Sun, Mar 20, 2005 at 11:40:27AM -0300, rodsenra@gpr.com.br wrote: > Perhaps you could focus in 728278. It addresses some of the issues you > have addressed in 754022, but it is not properly formatted. I started to look at it yesterday, but it's so big and does so many things at once... it requires a lot of time... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ark-mlist at att.net Sun Mar 20 18:31:20 2005 From: ark-mlist at att.net (Andrew Koenig) Date: Sun Mar 20 18:31:09 2005 Subject: [Python-Dev] Python 2.4 | 7.3 The for statement In-Reply-To: <5dcb081c05031918257536d79@mail.gmail.com> Message-ID: <009601c52d72$a1b82a10$6402a8c0@arkdesktop> > for_stmt ::= "for" target_list "in" expression_list > [ "and" expression ] ":" > suite ["else" ":" suite] It can't work. The expression_list could be just a variable, as could the expression, in which case you get "for" target_list "in" variable "and" variable ":" and, of course variable "and" variable is already an expression. So it's ambiguous. From firemoth at gmail.com Sun Mar 20 21:09:47 2005 From: firemoth at gmail.com (Timothy Fitz) Date: Sun Mar 20 21:09:49 2005 Subject: [Python-Dev] identity operands (was python-dev Summary for 2005-03-01 through 2005-03-15 [draft]) In-Reply-To: <423CD640.9000102@iinet.net.au> References: <423A3B2D.8030302@ocf.berkeley.edu> <423AB437.9040301@iinet.net.au> <87hdj7plun.fsf@hydra.bayview.thirdcreek.com> <423CD640.9000102@iinet.net.au> Message-ID: <972ec5bd05032012097df01d5b@mail.gmail.com> [Nick Coghlan] > So, using "".join is roughly three times as fast as abusing sum :) True in the case where you're concatenating three strings, but what about 100 strings? (Full source attached) Py> timeit.Timer("sum(strings, ai)", setup).repeat() [33.553668413164239, 32.861660909417253, 33.092705357803851] Py> timeit.Timer("''.join(strings)", setup).repeat() [5.4385152302876492, 5.3633637794747671, 5.3587657090496066] Py> timeit.Timer(" + ".join('"' + s + '"' for s in strings), "").repeat() [17.726616371633828, 17.785511845779279, 18.179861127601413] So at 100 strings, the difference is over 5x, and I assume you'll see the relative distance increase as you increase the number of strings. Timothy Fitz -------------- next part -------------- import timeit from random import choice from random import randrange from string import uppercase setup = """ class additive_identity(object): def __add__(self, other): return other ai = additive_identity() from random import choice from random import randrange from string import uppercase strings = ["".join(choice(uppercase) for i in range(randrange(10))) for i in range(100)]""" strings = ["".join(choice(uppercase) for i in range(randrange(10))) for i in range(100)] print "SUM:", timeit.Timer("sum(strings, ai)", setup).repeat() print "JOIN:", timeit.Timer("''.join(strings)", setup).repeat() print "ADD:", timeit.Timer(" + ".join('"' + s + '"' for s in strings), "").repeat() From olsongt at verizon.net Sun Mar 20 21:35:53 2005 From: olsongt at verizon.net (Grant Olson) Date: Sun Mar 20 21:38:51 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches In-Reply-To: <423D91D9.1080009@ocf.berkeley.edu> Message-ID: <0IDO004EW40OEPH0@vms040.mailsrvcs.net> > Make sure "AST" is used in the subject line; e.g., "[AST]" at > the beginning. > Unfortunately the AST group is only available for patches; > not listed for bug reports (don't know why; can this be fixed?). > > Other than that, just assign it to me since I will most > likely be doing AST work in the near future. > Brett, I sent a few outstanding ones your way. I hate to complain again, I know everyone involved are volunteers, etc, etc; but it'd be really nice if someone could take a look at '[ 742621 ] ast-branch: msvc project sync'. The patch is almost two years old now, has been updated for VC7.1 and still hasn't been checked in. Without this, any Windows developers who check out the ast-branch will experience a broken build. -Grant From bac at ocf.berkeley.edu Sun Mar 20 22:15:02 2005 From: bac at ocf.berkeley.edu (Brett C.) Date: Sun Mar 20 22:15:07 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches In-Reply-To: <0IDO004EW40OEPH0@vms040.mailsrvcs.net> References: <0IDO004EW40OEPH0@vms040.mailsrvcs.net> Message-ID: <423DE7D6.8060108@ocf.berkeley.edu> Grant Olson wrote: > > > >>Make sure "AST" is used in the subject line; e.g., "[AST]" at >>the beginning. >>Unfortunately the AST group is only available for patches; >>not listed for bug reports (don't know why; can this be fixed?). >> >>Other than that, just assign it to me since I will most >>likely be doing AST work in the near future. >> > > > > Brett, > > I sent a few outstanding ones your way. I hate to complain again, I know > everyone involved are volunteers, etc, etc; but it'd be really nice if > someone could take a look at '[ 742621 ] ast-branch: msvc project sync'. > The patch is almost two years old now, has been updated for VC7.1 and still > hasn't been checked in. Without this, any Windows developers who check out > the ast-branch will experience a broken build. > The problem is that I don't have access to a Windows machine so there is no way to verify the patch myself. I will try to get someone here to verify the patch, but I am not comfortable doing a blind commit. If one other person could verify that the patch is good I will apply it. -Brett From andrewm at object-craft.com.au Mon Mar 21 00:28:02 2005 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Mon Mar 21 00:27:53 2005 Subject: [Python-Dev] Re: [Csv] Example workaround classes for using Unicode with csv module... In-Reply-To: <16955.2733.160226.850562@montanaro.dyndns.org> References: <16955.2733.160226.850562@montanaro.dyndns.org> Message-ID: <20050320232802.6C7333C091@coffee.object-craft.com.au> >I added UnicodeReader and UnicodeWriter example classes to the csv module >docs just now. They mention problems with ASCII NUL characters (which I >vaguely remember - NUL-terminated strings are used internally, right?). Do >NULs still present a problem? I saw nothing in the log messages that >mentioned "ascii" or "nul" so I presume it is. That's right - it still uses null terminated strings internally, and the various special characters (quotechar, escapechar, etc) use null to mean "not specified". Fixing this would cause much upheaval. >Here's what I added. Let me know if you think it needs any corrections, >especially if there's a better way to word "as long as you avoid encodings >like utf-16 that use NULs". Can that just be "as long as you avoid >multi-byte encodings other than utf-8"? I think only utf-8 provides the guarantees needed for this to work - specifically, multi-byte characters need to have the high bit set (otherwise a delimiter or other special character appearing within a multi-byte character would upset the parsing), while at the same time having single byte characters for the characters with special meaning to the parser: note also that none of the special characters (quotechar, delimiter, escapechar, etc) can be a multi-byte sequence. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From abo at minkirri.apana.org.au Mon Mar 21 02:59:12 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Mon Mar 21 02:59:20 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> Message-ID: <1111370352.3763.51.camel@schizo> On Fri, 2005-03-18 at 20:41 -0500, James Y Knight wrote: > On Mar 18, 2005, at 8:19 PM, Greg Ward wrote: > > Is having to use fcntl and os really so awful? At least it requires > > the programmer to prove he knows what he's doing putting this file > > into non-blocking mode, and that he really wants to do it. ;-) Consider the following. This is pretty much the only way you can use popen2 reliably without knowing specific behaviours of the executed command; import os,fnctl,select def process_data(cmd,data): child_in, child_out = os.popen2(cmd) child_in = child_in.fileno() # / flags = fcntl.fcntl(child_in, fcntl.F_GETFL) # |1) fcntl.fcntl(child_in, fcntl.F_SETFL, flags | os.O_NONBLOCK) # \ child_out = child_out.fileno() # / flags = fcntl.fcntl(child_out, fcntl.F_GETFL) # |2) fcntl.fcntl(child_out, fcntl.F_SETFL, flags | os.O_NONBLOCK)# \ ans = "" li = [child_out] lo = [child_in] while li or lo: i,o,e = select.select(li,lo,[]) # 3 if i: buf = os.read(child_out,2048) # 4 if buf: ans += buf else: li=[] if o: if data: count=os.write(child_in,data[:2048]) # 4 data = data[count:] else: lo=[] return ans For 1) and 2), note that popen2 returns file objects, but as they cannot be reliably used as file objects, we ignore them and grab their fileno(). Why does popen2 return file objects if they cannot reliably be used? The flags get/set using fnctl is arcane stuff for what is pretty much essential operations after a popen2. These could be replaced by; child_in.setblocking(False) child_out.setblocking(False) For 3), select() can operate on file objects directly. However, since you cannot reliably read/write file objects in non-blocking mode, we use the fileno's. Why can select operate with file objects if file objects cannot be reliably read/written? For 4), we are using the os.read/write methods to operate on the fileno's. Under the proposed PEP we could use the file objects read/write methods instead. I guess the thing that annoys me the most is the asymmetry of popen2 and select using file objects, but needing to use the os.read/write and fileno()'s for reading and writing. > I'd tend to agree. :) Moreover, I don't think fread/fwrite are > guaranteed to work as you would expect with non-blocking file > descriptors. So, providing a setblocking() call to files would require > calling read/write instead of fread/fwrite in all the file methods, at > least when in non-blocking mode. I don't think that's a good idea. Hmm.. I assumed file.read() and file.write() were implemented using read/write from their observed behaviour. The documentation of fread/fwrite doesn't mention the behaviour in non-blocking mode at all. The observed behaviour suggests that fread/fwrite are implemented using read/write and hence get the same behaviour. The documentation implies that the behaviour in non-blocking mode will reflect the behaviour of read/write, with EAGAIN errors reported via ferror() indicating empty non-blocking reads/writes. If the behaviour of fread/fwrite is indeed indeterminate under non-blocking mode, then yes, file objects in non-blocking mode would have to use read/write instead of fread/fwrite. However, I don't think this is required. I know this PEP is kinda insignificant and minor. It doesn't save much, but it doesn't change much, and makes things a bit cleaner. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From greg.ewing at canterbury.ac.nz Mon Mar 21 06:32:36 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon Mar 21 06:32:54 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <20050319011940.GA5632@cthulhu.gerg.ca> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> Message-ID: <423E5C74.9040008@canterbury.ac.nz> > On 18 March 2005, Donovan Baarda said: >>Many Python library methods and classes like select.select(), os.popen2(), >>and subprocess.Popen() return and/or operate on builtin file objects. >>However even simple applications of these methods and classes require the >>files to be in non-blocking mode. I don't agree with that. There's no need to use non-blocking I/O when using select(), and in fact things are less confusing if you don't. >>The read method's current behaviour needs to be documented, so its actual >>behaviour can be used to differentiate between an empty non-blocking read, >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty >>non-blocking read. Isn't that unix-specific? The file object is supposed to provide a more or less platform-independent interface, I thought. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From nicksjacobson at yahoo.com Mon Mar 21 08:10:43 2005 From: nicksjacobson at yahoo.com (Nicholas Jacobson) Date: Mon Mar 21 08:10:46 2005 Subject: [Python-Dev] docstring before function declaration Message-ID: <20050321071043.98485.qmail@web53907.mail.yahoo.com> IIRC, Guido once mentioned that he regretted not setting function docstrings to come before the function declaration line, instead of after. i.e. """This describes class Bar.""" class Bar: ... Or with a decorator: """This describes class Bar.""" @classmethod class Bar: ... Versus the current method: class Bar: """This describes class Bar.""" def foo: ... where the docstring looks like it refers to foo, not Bar. I think that commenting the function before its declaration, at the same tabbed point, increases the code's readability. What do you think about making this change at some point (e.g. Python 3000)? Or if not, then having the option to toggle to this layout? Thanks, --Nick Jacobson __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From anthony at interlink.com.au Mon Mar 21 08:59:18 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Mar 21 08:59:38 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: <20050321071043.98485.qmail@web53907.mail.yahoo.com> References: <20050321071043.98485.qmail@web53907.mail.yahoo.com> Message-ID: <200503211859.20859.anthony@interlink.com.au> On Monday 21 March 2005 18:10, Nicholas Jacobson wrote: > IIRC, Guido once mentioned that he regretted not > setting function docstrings to come before the > function declaration line, instead of after. How do you distinguish between a docstring at the top of a module that's immediately followed by a function? Is it the module docstring or the function docstring? -- Anthony Baxter It's never too late to have a happy childhood. From abo at minkirri.apana.org.au Mon Mar 21 09:04:02 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Mon Mar 21 09:04:17 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <423E5C74.9040008@canterbury.ac.nz> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> Message-ID: <1111392242.3763.69.camel@schizo> On Mon, 2005-03-21 at 17:32 +1200, Greg Ewing wrote: > > On 18 March 2005, Donovan Baarda said: > > >>Many Python library methods and classes like select.select(), os.popen2(), > >>and subprocess.Popen() return and/or operate on builtin file objects. > >>However even simple applications of these methods and classes require the > >>files to be in non-blocking mode. > > I don't agree with that. There's no need to use non-blocking > I/O when using select(), and in fact things are less confusing > if you don't. You would think that... and the fact that select, popen2 etc all use file objects encourage you to think that. However, this is a trap that can catch you out badly. Check the attached python scripts that demonstrate the problem. Because staller.py outputs and flushes a fragment of data smaller than selector.py uses for its reads, the select statement is triggered, but the corresponding read blocks. A similar thing can happen with writes... if the child process consumes a fragment smaller than the write buffer of the selector process, then the select can trigger and the corresponding write can block because there is not enough space in the file buffer. The only ways to ensure that a select process does not block like this, without using non-blocking mode, are; 1) use a buffer size of 1 in the select process. 2) understand the child process's read/write behaviour and adjust the selector process accordingly... ie by making the buffer sizes just right for the child process, Note that it all interacts with the file objects buffer sizes too... making for some extremely hard to debug intermittent behaviour. > >>The read method's current behaviour needs to be documented, so its actual > >>behaviour can be used to differentiate between an empty non-blocking read, > >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty > >>non-blocking read. > > Isn't that unix-specific? The file object is supposed to > provide a more or less platform-independent interface, I > thought. I think the fread/fwrite and read/write behaviour is posix standard and possibly C standard stuff... so it _should_ be the same on other platforms. -- Donovan Baarda http://minkirri.apana.org.au/~abo/ -------------- next part -------------- A non-text attachment was scrubbed... Name: selector.py Type: application/x-python Size: 756 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050321/b70dd852/selector.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: staller.py Type: application/x-python Size: 126 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050321/b70dd852/staller.bin From astrand at lysator.liu.se Mon Mar 21 09:45:57 2005 From: astrand at lysator.liu.se (Peter Astrand) Date: Mon Mar 21 09:46:10 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <1111392242.3763.69.camel@schizo> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> <1111392242.3763.69.camel@schizo> Message-ID: On Mon, 21 Mar 2005, Donovan Baarda wrote: > > I don't agree with that. There's no need to use non-blocking > > I/O when using select(), and in fact things are less confusing > > if you don't. > > You would think that... and the fact that select, popen2 etc all use > file objects encourage you to think that. However, this is a trap that > can catch you out badly. Check the attached python scripts that > demonstrate the problem. This is no "trap". When select() indicates that you can write or read, it means that you can write or read at least one byte. The .read() and .write() file methods, however, always writes and reads *everything*. These works, basically, just like fread()/fwrite(). > The only ways to ensure that a select process does not block like this, > without using non-blocking mode, are; > > 1) use a buffer size of 1 in the select process. > > 2) understand the child process's read/write behaviour and adjust the > selector process accordingly... ie by making the buffer sizes just right > for the child process, 3) Use os.read / os.write. > > >>The read method's current behaviour needs to be documented, so its actual > > >>behaviour can be used to differentiate between an empty non-blocking read, > > >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty > > >>non-blocking read. > > > > Isn't that unix-specific? The file object is supposed to > > provide a more or less platform-independent interface, I > > thought. > > I think the fread/fwrite and read/write behaviour is posix standard and > possibly C standard stuff... so it _should_ be the same on other > platforms. Sorry if I've misunderstood your point, but fread()/fwrite() does not return EAGAIN. /Peter ?strand From nicksjacobson at yahoo.com Mon Mar 21 10:08:59 2005 From: nicksjacobson at yahoo.com (Nicholas Jacobson) Date: Mon Mar 21 10:09:03 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: 6667 Message-ID: <20050321090900.69341.qmail@web53904.mail.yahoo.com> Rule #1: If the docstring is the first line of a module, it's the module's docstring. Rule #2: If the docstring comes right before a class/function, it's that class/function's docstring. > How do you distinguish between a docstring at the > top of a module > that's immediately followed by a function? Is it > the module docstring > or the function docstring? It's both. The docstring would be assigned to both the module and the function. This is a *good* thing when there is a module with only one function in it. i.e. there should only be one docstring for both, and this saves repetition of that docstring. If a programmer wanted a docstring for the function but not the module, a blank first line would do the trick. A docstring for the module but not the function? Put a blank line between the module's docstring and the function. --Nick Jacobson __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From abo at minkirri.apana.org.au Mon Mar 21 10:39:50 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Mon Mar 21 10:40:01 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> <1111392242.3763.69.camel@schizo> Message-ID: <002d01c52df9$ed4ed3c0$24ed0ccb@apana.org.au> G'day, From: "Peter Astrand" > On Mon, 21 Mar 2005, Donovan Baarda wrote: [...] > This is no "trap". When select() indicates that you can write or read, it > means that you can write or read at least one byte. The .read() and > .write() file methods, however, always writes and reads *everything*. > These works, basically, just like fread()/fwrite(). yep, which is why you can only use them reliably in a select loop if you read/write one byte at a time. > > The only ways to ensure that a select process does not block like this, > > without using non-blocking mode, are; > > > > 1) use a buffer size of 1 in the select process. > > > > 2) understand the child process's read/write behaviour and adjust the > > selector process accordingly... ie by making the buffer sizes just right > > for the child process, > > 3) Use os.read / os.write. [...] but os.read / os.write will block too. Try it... replace the file read/writes in selector.py. They will only do partial reads if the file is put into non-blocking mode. > > I think the fread/fwrite and read/write behaviour is posix standard and > > possibly C standard stuff... so it _should_ be the same on other > > platforms. > > Sorry if I've misunderstood your point, but fread()/fwrite() does not > return EAGAIN. no, fread()/fwrite() will return 0 if nothing was read/written, and ferror() will return EAGAIN to indicated that it was a "would block" condition.... at least I think it does... the man page simply says ferror() returns a non-zero value. Looking at the python implementation of file.read(), for an empty fread() where ferror() is non zero, it only raises IOError if errno is not EAGAIN or EWOULDBLOCK. It blindly clearerr()'s for any other partial read. The implementation of file.write() raises IOError whenever there is an incomplete write. So it looks, as I pointed out in the draft PEP, that the current file.read() supports non-blocking mode, but file.write() doesn't... a bit asymmetric :-) ---------------------------------------------------------------- Donovan Baarda http://minkirri.apana.org.au/~abo/ ---------------------------------------------------------------- From abo at minkirri.apana.org.au Mon Mar 21 11:06:51 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Mon Mar 21 11:07:08 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blockingmode. References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> Message-ID: <004301c52dfd$b3eff330$24ed0ccb@apana.org.au> G'day, From: "Greg Ward" > On 18 March 2005, Donovan Baarda said: [...] > > Currently the built in file type does not support non-blocking mode very > > well. Setting a file into non-blocking mode and reading or writing to it > > can only be done reliably by operating on the file.fileno() file descriptor. > > This requires using the fnctl and os module file descriptor manipulation > > methods. > > Is having to use fcntl and os really so awful? At least it requires > the programmer to prove he knows what he's doing putting this file > into non-blocking mode, and that he really wants to do it. ;-) It's not that bad I guess... but then I'm proposing a very minor change to fix it. The bit that annoys me is popen2() and select() give this false sense of "File Object compatability", when in reality you can't use them reliably with file objects. It is also kind of disturbing that file.read() actually does work in non-blocking mode, but file.write() doesn't. The source for file.read() shows a fair bit of effort towards making it work for non-blocking mode... why not do the same for file.write()? > > Details > > ======= > > > > The documentation of file.read() warns; "Also note that when in non-blocking > > mode, less data than what was requested may be returned, even if no size > > parameter was given". An empty string is returned to indicate an EOF > > condition. It is possible that file.read() in non-blocking mode will not > > produce any data before EOF is reached. Currently there is no documented > > way to identify the difference between reaching EOF and an empty > > non-blocking read. > > > > The documented behaviour of file.write() in non-blocking mode is undefined. > > When writing to a file in non-blocking mode, it is possible that not all of > > the data gets written. Currently there is no documented way of handling or > > indicating a partial write. > > That's more interesting and a better motivation for this PEP. The other solution to this of course is to simply say "file.read() and file.write() don't work in non-blocking mode", but that would be a step backwards for the current file.read(). > > file.read([size]) Changes > > -------------------------- > > > > The read method's current behaviour needs to be documented, so its actual > > behaviour can be used to differentiate between an empty non-blocking read, > > and EOF. This means recording that IOError(EAGAIN) is raised for an empty > > non-blocking read. > > > > > > file.write(str) Changes > > -------------------- > > > > The write method needs to have a useful behaviour for partial non-blocking > > writes defined, implemented, and documented. This includes returning how > > many bytes of "str" are successfully written, and raising IOError(EAGAIN) > > for an unsuccessful write (one that failed to write anything). > > Proposing semantic changes to file.read() and write() is bound to > raise hackles. One idea for soothing such objections: only make these > changes active when setblocking(False) is in effect. I.e., a > setblocking(True) file (the default, right?) behaves as you described > above, warts and all. (So old code that uses fcntl() continues to > "work" as before.) But files that have had setblocking(False) called > could gain these new semantics that you propose. There is nothing in this proposal that would break or change the behaviour of any existing code, unless it was relying on file.write() returning None. or checking that file objects don't have a "setblocking" method. Note that the change for file.read() is simply to document the current behaviour... not to actually change it. The change for file.write() is a little more dramatic, but I really can't imagine anyone relying on file.write() returning None. A compromise would be to have file.write() return None in blocking mode, and a count in non-blocking mode... but I still can't believe people will rely on it returning None :-) It would be more useful to always return a count, so that methods using them could handle both modes easily. Note that I did consider some more dramatic changes that would have made them even easier to use. Things like raising an exception for EOF instead of EAGAIN would actually make a lot of things easier to code... but it would be too big a change. ---------------------------------------------------------------- Donovan Baarda http://minkirri.apana.org.au/~abo/ ---------------------------------------------------------------- From astrand at lysator.liu.se Mon Mar 21 11:42:47 2005 From: astrand at lysator.liu.se (Peter Astrand) Date: Mon Mar 21 11:43:00 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <002d01c52df9$ed4ed3c0$24ed0ccb@apana.org.au> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> <1111392242.3763.69.camel@schizo> <002d01c52df9$ed4ed3c0$24ed0ccb@apana.org.au> Message-ID: On Mon, 21 Mar 2005, Donovan Baarda wrote: > > > The only ways to ensure that a select process does not block like this, > > > without using non-blocking mode, are; > > 3) Use os.read / os.write. > [...] > > but os.read / os.write will block too. No. >Try it... replace the file > read/writes in selector.py. They will only do partial reads if the file is > put into non-blocking mode. I've just tried it; I replaced: data = o.read(BUFF_SIZE) with: data = os.read(o.fileno(), BUFF_SIZE) Works for me without any hangs. Another example is the subprocess module, which does not use non-blocking mode in any way. (If you are using pipes, however, you shouldn't write more than PIPE_BUF bytes in each write.) > > > I think the fread/fwrite and read/write behaviour is posix standard and > > > possibly C standard stuff... so it _should_ be the same on other > > > platforms. > > > > Sorry if I've misunderstood your point, but fread()/fwrite() does not > > return EAGAIN. > > no, fread()/fwrite() will return 0 if nothing was read/written, and ferror() > will return EAGAIN to indicated that it was a "would block" condition.... at > least I think it does... the man page simply says ferror() returns a > non-zero value. fread() should loop internally on EAGAIN, in blocking mode. /Peter ?strand From abo at minkirri.apana.org.au Mon Mar 21 13:31:43 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Mon Mar 21 13:31:59 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> <1111392242.3763.69.camel@schizo> <002d01c52df9$ed4ed3c0$24ed0ccb@apana.org.au> Message-ID: <1111408304.29604.4.camel@minkirri.apana.org.au> On Mon, 2005-03-21 at 11:42 +0100, Peter Astrand wrote: > On Mon, 21 Mar 2005, Donovan Baarda wrote: > > > > > The only ways to ensure that a select process does not block like this, > > > > without using non-blocking mode, are; > > > > 3) Use os.read / os.write. > > [...] > > > > but os.read / os.write will block too. > > No. [...] Hmmm... you are right... that changes things. Blocking vs non-blocking becomes kinda moot if read/write will do partial writes in blocking mode. > fread() should loop internally on EAGAIN, in blocking mode. Yeah, I was talking about non-blocking mode... -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From p.f.moore at gmail.com Mon Mar 21 14:05:39 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Mon Mar 21 14:05:42 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <423E5C74.9040008@canterbury.ac.nz> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> Message-ID: <79990c6b050321050545fbe859@mail.gmail.com> On Mon, 21 Mar 2005 17:32:36 +1200, Greg Ewing wrote: > > On 18 March 2005, Donovan Baarda said: > >>The read method's current behaviour needs to be documented, so its actual > >>behaviour can be used to differentiate between an empty non-blocking read, > >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty > >>non-blocking read. > > Isn't that unix-specific? The file object is supposed to > provide a more or less platform-independent interface, I > thought. The whole thing is, I believe, highly Unix-specific. I say this because I am essentially a Windows programmer, and the proposal means almost nothing to me :-) More seriously, non-blocking IO and select-type readability checks are VERY different on Windows, and so I would expect the implementation of this chance to be completely different as well. The C standard says nothing about non-blocking IO. While POSIX might, that doesn't apply to Windows. Oh, and in case it's not obvious, I'm -1 on something "Unix-only" here. Python file objects are supposed to be cross-platform, in general. Paul. PS Donovan's sample code seems to be process-related - if so, isn't that what the new subprocess module was supposed to resolve (process-related communication in a platform-independent way)? If the only use case is with subprocesses, then is this change needed at all? From anthony at interlink.com.au Mon Mar 21 14:13:42 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Mar 21 14:13:56 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: <20050321090900.69341.qmail@web53904.mail.yahoo.com> References: <20050321090900.69341.qmail@web53904.mail.yahoo.com> Message-ID: <200503220013.42973.anthony@interlink.com.au> On Monday 21 March 2005 20:08, Nicholas Jacobson wrote: > > How do you distinguish between a docstring at the > > top of a module > > that's immediately followed by a function? Is it > > the module docstring > > or the function docstring? > > It's both. The docstring would be assigned to both > the module and the function. This is a *good* thing > when there is a module with only one function in it. > i.e. there should only be one docstring for both, and > this saves repetition of that docstring. > > If a programmer wanted a docstring for the function > but not the module, a blank first line would do the > trick. A docstring for the module but not the > function? Put a blank line between the module's > docstring and the function. Yuk. This is magic taken to a ridiculous level. Note that "blank lines" currently have no meaning in Python, and adding a meaning to them is not my idea of a good thing. -- Anthony Baxter It's never too late to have a happy childhood. From bac at ocf.berkeley.edu Mon Mar 21 15:23:45 2005 From: bac at ocf.berkeley.edu (Brett C.) Date: Mon Mar 21 15:23:53 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: <20050321071043.98485.qmail@web53907.mail.yahoo.com> References: <20050321071043.98485.qmail@web53907.mail.yahoo.com> Message-ID: <423ED8F1.6080805@ocf.berkeley.edu> Nicholas Jacobson wrote: > IIRC, Guido once mentioned that he regretted not > setting function docstrings to come before the > function declaration line, instead of after. > He did, but I don't know how strong that regret is. > i.e. > > """This describes class Bar.""" > class Bar: > ... > > Or with a decorator: > > """This describes class Bar.""" > @classmethod > class Bar: > ... > > Versus the current method: > > class Bar: > """This describes class Bar.""" > def foo: > ... > I am going to be -42 on this one. I personally love having the docstring below the definition line. So much so, in fact, that in personal C code I use the same style for documentation. I find it easier to browse the source since where a definition starts is much cleaner (yes, syntax highlighting and searching for ``\s*def `` works as well, but I am thinking when you are just scrolling). Beyond that I can't really rationalize it beyond just aesthetics at the moment. But I definitely prefer the current style. -Brett From eric.nieuwland at xs4all.nl Mon Mar 21 15:50:14 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Mon Mar 21 15:50:17 2005 Subject: Fwd: [Python-Dev] docstring before function declaration Message-ID: <4f84a8c30c63aa17406851c691af55c9@xs4all.nl> Nicholas Jacobson wrote: > IIRC, Guido once mentioned that he regretted not > setting function docstrings to come before the > function declaration line, instead of after. > > [ examples deleted ] > > I think that commenting the function before its > declaration, at the same tabbed point, increases the > code's readability. > > What do you think about making this change at some > point (e.g. Python 3000)? Or if not, then having the > option to toggle to this layout? Please don't. All class attributes are declared within the class. Pulling out the docstring would make it an exception. The current situation is very clear and easy to explain to newbies. If you feel the current position of the docstring may be the first attribute's docstring, insert a newline at the proper positions to separate the two. It works for me. --eric From ncoghlan at iinet.net.au Mon Mar 21 15:50:23 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Mon Mar 21 15:50:37 2005 Subject: [Python-Dev] [AST] A somewhat less trivial patch than the last one. . . Message-ID: <423EDF2F.3000004@iinet.net.au> I've put a first cut at generator expressions for the AST branch on Sourceforge. It's enough to get test_grammar to pass, and tinkering at the interactive prompt appears to work. The patch also fixes a problem with displaying interim results for functions entered at the interactive prompt (I noticed it when trying to get the AST branch genexp bytecode to roughly match that produced by Python 2.4). I didn't check if classes suffer from the same problem, though. Finally, the patch fixes asdl_seq_new to avoid allocating large chunks of memory when passed a size of zero. (This one bit me when trying to get generator expressions to work as arguments - a function call may end up with zero length asdl sequences for either the positional or the keyword arguments). Patch number is 1167628. Cheers, Nick. P.S. Could the powers-that-be add me to the developer list on Sourceforge? I'm interested in better access to the SF trackers, rather than CVS access, though. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From theller at python.net Mon Mar 21 16:08:57 2005 From: theller at python.net (Thomas Heller) Date: Mon Mar 21 16:09:05 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils/tests test_dist.py, 1.1, 1.2 In-Reply-To: (fdrake@users.sourceforge.net's message of "Sun, 20 Mar 2005 14:19:49 -0800") References: Message-ID: <8y4hniie.fsf@python.net> fdrake@users.sourceforge.net writes: > PEP 314 implementation (client side): I'm not sure where I should post this, but shouldn't there be a way to specify the encoding of the metadata? There are people (not me, fortunately), with umlauts in their names, for example. Thomas From jhylton at gmail.com Mon Mar 21 16:21:21 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Mon Mar 21 16:21:23 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils/tests test_dist.py, 1.1, 1.2 In-Reply-To: <8y4hniie.fsf@python.net> References: <8y4hniie.fsf@python.net> Message-ID: On Mon, 21 Mar 2005 16:08:57 +0100, Thomas Heller wrote: > fdrake@users.sourceforge.net writes: > > > PEP 314 implementation (client side): > > I'm not sure where I should post this, but shouldn't there be a way to > specify the encoding of the metadata? There are people (not me, > fortunately), with umlauts in their names, for example. UTF-8 as the default would accommodate most uses, but it does seem essential to have some way of specifying an encoding. Jeremy From fdrake at acm.org Mon Mar 21 17:37:49 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon Mar 21 17:38:03 2005 Subject: [Python-Dev] distutils/PyPI package metadata fields In-Reply-To: <8y4hniie.fsf@python.net> References: <8y4hniie.fsf@python.net> Message-ID: <200503211137.49160.fdrake@acm.org> On Monday 21 March 2005 10:08, Thomas Heller wrote: > I'm not sure where I should post this, but shouldn't there be a way to > specify the encoding of the metadata? There are people (not me, > fortunately), with umlauts in their names, for example. Agreed. I think there are a number of additional metadata fields which would be valuable, but which don't exist. Given that PEP 314 is close to being completely implemented, that's probably not the place to add them. I think a new PEP should be written to describe the new fields, and call that Metadata-Version 1.2. Some sort of Metadata-Encoding field should be included. There should also be a field for the version of Python that's required. -Fred -- Fred L. Drake, Jr. From bac at ocf.berkeley.edu Mon Mar 21 17:53:04 2005 From: bac at ocf.berkeley.edu (Brett C.) Date: Mon Mar 21 17:53:12 2005 Subject: [Python-Dev] [AST] question about marshal_write_*() fxns Message-ID: <423EFBF0.20807@ocf.berkeley.edu> For those of you who don't know, I am sprinting on the AST branch here on PyCon. Specifically, I am fleshing out Python/compile.txt so that it can act as a good intro to new users and as a design doc. But one of things I am not sure of is what the marshal_write_*() functions in Python/Python-ast.c are used for. I assume they output to the marshal format, but there is mention of a byte stream format and so I thought it might be that as well. Anyone know which one it is? -Brett From nas at arctrix.com Mon Mar 21 18:50:49 2005 From: nas at arctrix.com (Neil Schemenauer) Date: Mon Mar 21 18:50:53 2005 Subject: [Python-Dev] [AST] question about marshal_write_*() fxns In-Reply-To: <423EFBF0.20807@ocf.berkeley.edu> References: <423EFBF0.20807@ocf.berkeley.edu> Message-ID: <20050321175048.GA16041@mems-exchange.org> On Mon, Mar 21, 2005 at 11:53:04AM -0500, Brett C. wrote: > But one of things I am not sure of is what the marshal_write_*() functions > in Python/Python-ast.c are used for. I assume they output to the marshal > format, but there is mention of a byte stream format and so I thought it > might be that as well. I believe the idea is that they can be used to move an AST back and forth between Python "space" (e.g. you could build an AST using Python code and then marshal it and send it to the C compiler). Neil From Scott.Daniels at Acm.Org Mon Mar 21 19:10:31 2005 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Mon Mar 21 19:10:53 2005 Subject: [Python-Dev] Re: docstring before function declaration In-Reply-To: <423ED8F1.6080805@ocf.berkeley.edu> References: <20050321071043.98485.qmail@web53907.mail.yahoo.com> <423ED8F1.6080805@ocf.berkeley.edu> Message-ID: Brett C. wrote: > I am going to be -42 on this one. I personally love having the > docstring below the definition line.... I can't really rationalize > it beyond just aesthetics at the moment.... I completely agree that the current form is better. It reduces the temptation to use boilerplate docstrings. No comment is better than an uninformative comment. If you don't want to spend the time to write a comment, step back and let me read the code itself. If the docstring is below the declaration, you needn't repeat the declaration in the comment (and people are less tempted to do so). Documentation and code should come from a human mind, and should communicate to another human mind. Attempting to automate the task of documentation creates hard-to-read code, interrupted by large meaningless comments which, often as not, are copied from a template and incompletely editted to be appropriate to the given function. Sorry about the rant, but this is another of my hot buttons. The single most disappointing thing I encountered on one project in a large corporation was an operating system requirements document that was being developped. The group had, at one point, a twenty-two page document with no real content. Really, the twenty two pages included an introduction, conclusion, table of contents, appendix, and index. It just didn't have anything but section headings. It was a thrilling triumph of form over function; a real Suahuab aesthetic, to coin a term. --Scott David Daniels Scott.Daniels@Acm.Org From bac at ocf.berkeley.edu Mon Mar 21 20:17:48 2005 From: bac at ocf.berkeley.edu (Brett C.) Date: Mon Mar 21 20:17:58 2005 Subject: [Python-Dev] [AST] question about marshal_write_*() fxns In-Reply-To: <20050321175048.GA16041@mems-exchange.org> References: <423EFBF0.20807@ocf.berkeley.edu> <20050321175048.GA16041@mems-exchange.org> Message-ID: <423F1DDC.3020604@ocf.berkeley.edu> Neil Schemenauer wrote: > On Mon, Mar 21, 2005 at 11:53:04AM -0500, Brett C. wrote: > >>But one of things I am not sure of is what the marshal_write_*() functions >>in Python/Python-ast.c are used for. I assume they output to the marshal >>format, but there is mention of a byte stream format and so I thought it >>might be that as well. > > > I believe the idea is that they can be used to move an AST back and > forth between Python "space" (e.g. you could build an AST using > Python code and then marshal it and send it to the C compiler). > Jeremy emailed me and said the same thing. Name should be changed, though, to prevent this confusion. Jeremy thought maybe the name should be changed. Here are some ideas for a different name (thinking in terms of what the name would be changed to for the prefix of the function names): - byte_encode - linear_form - zephyr_encoding - flat_form - flat_prefix - prefix_form - scheme_form =) For those that don't know and it wasn't obvious from the names, the format is a prefix-based, linear form. -Brett From bac at ocf.berkeley.edu Mon Mar 21 20:33:36 2005 From: bac at ocf.berkeley.edu (Brett C.) Date: Mon Mar 21 20:33:46 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches In-Reply-To: <0IDO004EW40OEPH0@vms040.mailsrvcs.net> References: <0IDO004EW40OEPH0@vms040.mailsrvcs.net> Message-ID: <423F2190.1050209@ocf.berkeley.edu> Grant Olson wrote: > > > >>Make sure "AST" is used in the subject line; e.g., "[AST]" at >>the beginning. >>Unfortunately the AST group is only available for patches; >>not listed for bug reports (don't know why; can this be fixed?). >> >>Other than that, just assign it to me since I will most >>likely be doing AST work in the near future. >> > > > > Brett, > > I sent a few outstanding ones your way. I hate to complain again, I know > everyone involved are volunteers, etc, etc; but it'd be really nice if > someone could take a look at '[ 742621 ] ast-branch: msvc project sync'. > The patch is almost two years old now, has been updated for VC7.1 and still > hasn't been checked in. Without this, any Windows developers who check out > the ast-branch will experience a broken build. > OK, thanks to John Ehresman here at PyCon sprint I got logistix's patch applied. Beyond a warning that a warning that decode_unicode() is never called and the parser module failing to compile under Windows everything should be fine for compiling the AST branch. Passing the test suite, though, is another question. =) -Brett From facundobatista at gmail.com Mon Mar 21 21:09:21 2005 From: facundobatista at gmail.com (Facundo Batista) Date: Mon Mar 21 21:09:25 2005 Subject: [Python-Dev] Deprecating 2.2 old bugs Message-ID: Going on with the old bugs checking, here are the results for 2.2. When I'll finish this will be put in an informational PEP. When I verified the bug, I filled two fields: - Summary: the same subject as in SF - Group: the bug's group at verifying time. - Bug #: the bug number - Verified: is the date when I checked the bug. - Action: is what I did then. If the bug survived the verification, the next two fields are applicable (if not, I put a dash, the idea is to keep this info easily parseable): - Final: is the action took by someone who eliminated the bug from that category (closed, moved to Py2.4, etc). - By: is the someone who did the final action. So, here's the info... Summary: docs need to discuss // and __future__.division Group: 2.2 Bug #: 449093 Verified: 25-Nov-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. Final: Group changed to 2.4. By: facundobatista Summary: new int overflow handling needs docs Group: 2.2 Bug #: 454446 Verified: 25-Nov-2004 Action: Closed: Won't fix. Behaviour changed. Final: - By: - Summary: urllib2 proxyhandle won't work. Group: 2.2 Bug #: 487471 Verified: 25-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. For a comment is not clear to me if it's a bug or not. Final: Closed: Won't fix. By: facundobatista Summary: BasHTTPServer IE Mac 5.1 size problem Group: 2.2 Bug #: 812340 Verified: 08-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: Section 11.20: xmlrpclib Details Group: 2.2 Bug #: 791067 Verified: 11-Nov-2004 Action: Deprecation alerted. Doc changed a lot, don't know enough about the subject to be clear to me if the problem is solved or not. Final: Closed: Won't fix. By: facundobatista Summary: socket.inet_aton on 64bit platform Group: 2.2 Bug #: 767150 Verified: 11-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: Error using Tkinter embeded in C++ Group: 2.2 Bug #: 699068 Verified: 11-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. For a comment is not clear to me if the problem actually existed. Final: Closed: Won't fix. By: facundobatista Summary: raw_input defers alarm signal Group: 2.2 Bug #: 685846 Verified: 11-Nov-2004 Action: Changed to Group Py2.3, there's a patch: #706406 Final: - By: - Summary: repr.repr not always safe Group: 2.2 Bug #: 666958 Verified: 15-Nov-2004 Action: Changed to Group Py2.3: it seems a bug to me. Final: - By: - Summary: win32 os.path.normpath not correct for leading slash cases Group: 2.2 Bug #: 665336 Verified: 16-Nov-2004 Action: Changed to Group Py2.4: it seems a bug to me. Final: - By: - Summary: memory leaks when importing posix module Group: 2.2 Bug #: 613222 Verified: 21-Mar-2005 Action: Changed to Group Py2.3, considering original post. Final: - By: - Summary: xml.sax second time file loading problem Group: 2.2 Bug #: 606692 Verified: 15-Nov-2004 Action: Deprecation alerted. Final: Closed: Won't fix. By: facundobatista Summary: urllib ftp broken if Win32 proxy Group: 2.2 Bug #: 532007 Verified: 15-Nov-2004 Action: Deprecation alerted. Final: Closed: Won't fix. By: facundobatista Summary: Bugs in rfc822.parseaddr() Group: 2.2 Bug #: 531205 Verified: 24-Nov-2004 Action: Changed to Group Py2.4: it seems a bug to me. Final: Closed. Invalid. By: loewis Summary: shelve update fails on "large"; entry Group: 2.2 Bug #: 523425 Verified: 25-Nov-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. Final: Closed: Fixed (the original submitter posted that is fixed in later version) By: facundobatista Summary: exception cannot be new-style class Group: 2.2 Bug #: 518846 Verified: 24-Nov-2004 Action: Changed to Group Py2.3: it seems a bug to me. Final: - By: - Summary: Missing docs for module imputil Group: 2.2 Bug #: 515751 Verified: 25-Nov-2004 Action: Changed to Group Py2.4: it seems a bug to me. Final: - By: - Regards, . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://pyar.decode.com.ar/ From greg.ewing at canterbury.ac.nz Tue Mar 22 01:39:29 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 22 01:39:46 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: <20050321090900.69341.qmail@web53904.mail.yahoo.com> References: <20050321090900.69341.qmail@web53904.mail.yahoo.com> Message-ID: <423F6941.5070801@canterbury.ac.nz> Nicholas Jacobson wrote: > If a programmer wanted a docstring for the function > but not the module, a blank first line would do the > trick. A docstring for the module but not the > function? Put a blank line between the module's > docstring and the function. -1 on all this making of blank lines significant. Currently I can leave some space before/after a docstring without breaking anything. This can help readability, especially for class docstrings. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 22 01:49:23 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 22 01:49:39 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <1111370352.3763.51.camel@schizo> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <1111370352.3763.51.camel@schizo> Message-ID: <423F6B93.7020904@canterbury.ac.nz> Donovan Baarda wrote: > Consider the following. This is pretty much the only way you can use > popen2 reliably without knowing specific behaviours of the executed > command; > > ... > fcntl.fcntl(child_in, fcntl.F_SETFL, flags | os.O_NONBLOCK) # \ > ... # / > fcntl.fcntl(child_out, fcntl.F_SETFL, flags | os.O_NONBLOCK)# \ I still don't believe you need to make these non-blocking. When select() returns a fd for reading/writing, it's telling you that the next os.read/os.write call on it will not block. Making the fd non-blocking as well is unnecessary and perhaps even undesirable. > For 1) and 2), note that popen2 returns file objects, but as they cannot > be reliably used as file objects, we ignore them and grab their > fileno(). Why does popen2 return file objects if they cannot reliably be > used? I would go along with giving file objects alternative read/write methods which behave more like os.read/os.write, maybe called something like readsome() and writesome(). That would eliminate the need to extract and manipulate the fds, and might make it possible to do some of this stuff in a more platform-independent way. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Tue Mar 22 01:55:55 2005 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue Mar 22 01:56:11 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <1111392242.3763.69.camel@schizo> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> <1111392242.3763.69.camel@schizo> Message-ID: <423F6D1B.5020509@canterbury.ac.nz> Donovan Baarda wrote: > On Mon, 2005-03-21 at 17:32 +1200, Greg Ewing wrote: >> >>I don't agree with that. There's no need to use non-blocking >>I/O when using select(), and in fact things are less confusing >>if you don't. > > Because staller.py outputs and flushes a fragment of data smaller than > selector.py uses for its reads, the select statement is triggered, but > the corresponding read blocks. Your selector.py is using file object read/write methods, not os.read and os.write. I fully agree that you can't reliably use stdio-style I/O in conjunction with select(). But as long as you use os-level I/O, there shouldn't be any need to make anything non-blocking. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+ From abo at minkirri.apana.org.au Tue Mar 22 02:08:46 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue Mar 22 02:08:55 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <423F6B93.7020904@canterbury.ac.nz> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <1111370352.3763.51.camel@schizo> <423F6B93.7020904@canterbury.ac.nz> Message-ID: <1111453727.3786.40.camel@schizo> On Tue, 2005-03-22 at 12:49 +1200, Greg Ewing wrote: > Donovan Baarda wrote: > > > Consider the following. This is pretty much the only way you can use > > popen2 reliably without knowing specific behaviours of the executed > > command; > > > > ... > > fcntl.fcntl(child_in, fcntl.F_SETFL, flags | os.O_NONBLOCK) # \ > > ... # / > > fcntl.fcntl(child_out, fcntl.F_SETFL, flags | os.O_NONBLOCK)# \ > > I still don't believe you need to make these non-blocking. > When select() returns a fd for reading/writing, it's telling > you that the next os.read/os.write call on it will not block. > Making the fd non-blocking as well is unnecessary and perhaps > even undesirable. Yeah... For some reason I had it in my head that os.read/os.write would not do partial/incomplete reads/writes unless the file was in non-blocking mode. > > For 1) and 2), note that popen2 returns file objects, but as they cannot > > be reliably used as file objects, we ignore them and grab their > > fileno(). Why does popen2 return file objects if they cannot reliably be > > used? > > I would go along with giving file objects alternative read/write > methods which behave more like os.read/os.write, maybe called > something like readsome() and writesome(). That would eliminate > the need to extract and manipulate the fds, and might make it > possible to do some of this stuff in a more platform-independent > way. The fact that partial reads/writes are possible without non-blocking mode changes things a fair bit. Also, the lack of fnctl support in Windows needs to be taken into account too. I still think the support for partial reads in non-blocking mode on file.read() is inconsistent with the absence of partial write support in file.write(). I think this PEP still has some merit for cleaning up this inconsistency, but otherwise doesn't gain much... just adding a return count to file.write() and clearing up the documented behaviour is enough to do this. The lack of support on win32 for non-blocking mode, combined with the reduced need for it, makes adding a "setblocking" method undesirable. I don't know what the best thing to do now is... I guess the readsome/writesome is probably best, but given that os.read/os.write is not that bad, perhaps it's best to just forget I even suggested this PEP :-) -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From abo at minkirri.apana.org.au Tue Mar 22 02:17:20 2005 From: abo at minkirri.apana.org.au (Donovan Baarda) Date: Tue Mar 22 02:17:28 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <1111408304.29604.4.camel@minkirri.apana.org.au> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <423E5C74.9040008@canterbury.ac.nz> <1111392242.3763.69.camel@schizo> <002d01c52df9$ed4ed3c0$24ed0ccb@apana.org.au> <1111408304.29604.4.camel@minkirri.apana.org.au> Message-ID: <1111454240.3786.47.camel@schizo> On Mon, 2005-03-21 at 23:31 +1100, Donovan Baarda wrote: > On Mon, 2005-03-21 at 11:42 +0100, Peter Astrand wrote: > > On Mon, 21 Mar 2005, Donovan Baarda wrote: > > > > > > > The only ways to ensure that a select process does not block like this, > > > > > without using non-blocking mode, are; > > > > > > 3) Use os.read / os.write. > > > [...] > > > > > > but os.read / os.write will block too. > > > > No. > [...] > > Hmmm... you are right... that changes things. Blocking vs non-blocking > becomes kinda moot if read/write will do partial writes in blocking > mode. > > > fread() should loop internally on EAGAIN, in blocking mode. > > Yeah, I was talking about non-blocking mode... Actually, in blocking mode you never get EAGAIN.... read() only gets EAGAIN on an empty non-blocking read(). In non-blocking mode, EAGAIN is considered an error by fread(), so it will return a partial read. The python implementation of file.read() will return this partial read, and clear the EAGAIN error, or raise IOError if it was an empty read (to differentiate between an empty read and EOF). -- Donovan Baarda http://minkirri.apana.org.au/~abo/ From gward at python.net Tue Mar 22 03:39:38 2005 From: gward at python.net (Greg Ward) Date: Tue Mar 22 03:39:44 2005 Subject: [Python-Dev] Patch review: all webbrowser.py related patches up to 2005-03-20 In-Reply-To: <1301.200.158.47.2.1111328882.squirrel@200.158.47.2> References: <1301.200.158.47.2.1111328882.squirrel@200.158.47.2> Message-ID: <20050322023938.GA12329@cthulhu.gerg.ca> On 20 March 2005, rodsenra@gpr.com.br said: > 1166780 Fix _tryorder in webbrowser.py > ----------------------------------------------------------- > > This is my own patch. > > At the present time (Py2.4) documentation says: > """ > Under Unix, if the environment variable BROWSER exists, > it is interpreted to override the platform default list of > browsers,... > """ > (extract from Python-2.4/Doc/html/lib/module-webbrowser.html) > > But, when the environment variable BROWSER is messed up, > there is no reason not to try the other detected browser alternatives. I like it. Short, simple, and obvious. Can you think of a way to unit-test it? I took a brief look at the code, and it wasn't obvious to me. Might require factoring out a lazy initializer, and even then it might not work. And that would certainly call for more unit tests! Still, it's worth a shot. Greg -- Greg Ward http://www.gerg.ca/ Predestination was doomed from the start. From ncoghlan at iinet.net.au Tue Mar 22 04:10:50 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 22 04:10:56 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches In-Reply-To: <423F2190.1050209@ocf.berkeley.edu> References: <0IDO004EW40OEPH0@vms040.mailsrvcs.net> <423F2190.1050209@ocf.berkeley.edu> Message-ID: <423F8CBA.4060103@iinet.net.au> Brett C. wrote: > OK, thanks to John Ehresman here at PyCon sprint I got logistix's patch > applied. Beyond a warning that a warning that decode_unicode() is never > called and the parser module failing to compile under Windows everything > should be fine for compiling the AST branch. Under Linux (Suse 9.1), the parser module compiles with a couple of warnings about implicit declaration of PyParser_SimpleParseString, but the .so fails to load due to a missing symbol PyNode_Compile. > Passing the test suite, though, is another question. =) Heh. I managed to get test_grammar to pass 100% last night, which is at least a start! And I must say that the new system is rather nice to work with - it took me a while to work out what was going on, but the multi-pass 'build AST', 'build symbol table', 'compile AST' is much cleaner than it is with the lists-of-lists arrangement on the trunk (which is the entire point of the AST-branch, I guess. . .) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From ncoghlan at iinet.net.au Tue Mar 22 05:12:02 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 22 05:14:06 2005 Subject: [Python-Dev] [AST] question about marshal_write_*() fxns In-Reply-To: <423EFBF0.20807@ocf.berkeley.edu> References: <423EFBF0.20807@ocf.berkeley.edu> Message-ID: <423F9B12.9000007@iinet.net.au> Brett C. wrote: > For those of you who don't know, I am sprinting on the AST branch here > on PyCon. Ah, so that's why it's quiet this week :) > But one of things I am not sure of is what the marshal_write_*() > functions in Python/Python-ast.c are used for. I assume they output to > the marshal format, but there is mention of a byte stream format and so > I thought it might be that as well. Given Neil & Jeremy's answers, perhaps "linear_ast_*" or "bytestream_ast_*" would work as a new prefix? Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From tzot at mediconsa.com Sun Mar 20 14:46:01 2005 From: tzot at mediconsa.com (Christos Georgiou) Date: Tue Mar 22 10:15:11 2005 Subject: [Python-Dev] Re: Python 2.4 | 7.3 The for statement References: <5dcb081c05031918257536d79@mail.gmail.com> Message-ID: >It is easier if we see it beforehand: >------------------------------------- > >leave = False >alist = [1,2,3,3,4,5,6,7,8,9] >for item in alist and not leave: > if item is 1: leave = True Apart from other objections, this is valid Python now, and failing with 'TypeError: iteration over non-sequence'. self.introduce: This is not my first Python-dev message, but I never have introduced myself so far. So: I began my career as a programmer in 1990 working with C and Unify Accell/SQL, and continued with various RDBMS systems (Ingres, Oracle) creating commercial tailor-made applications (CRM and ERP, but before these terms were coined AFAIK). However, since 1986 (during school) I helped older friends (students) with C programs on Unix systems (met Unix before IBM compatible machines). And I won't mention Sinclair ZX Spectrum and Sinclair QL use as a kid :) Along programming, I worked as a sysadm for my companies' systems (and as a support sysadm for clients' systems, including SysV/68k, Ultrix, AIX, HP-UX, SCO, Irix, and various Linux distros since 2.4 kernel), so I have a thorough shell scripting background. I never liked Perl when I found out about it, so I continued my awk usage. I also had a very good knowledge of Access 97 at the time (loved the data engine, hated the program itself). Python I met in 2000 as a script to analyse LaBrea tarpit logs, and reading the code instantly clicked for me (it was very close to the language I always intended to write myself RSN :). It was love at first code viewing. I have since become a Python advocate. Personal computer-related interests include image processing (and cataloguing), database design, game solving, "interconnected" generic information storage and retrieval (attempts in which will be superseded soon --hopefully-- by Chandler... :) My current employment as part of the customer support team for SGI systems in EMEA territory leaves little time for Python (attempts at social life worsens things too), and often enough this lack of time makes my posts to comp.lang.python somewhat sloppy... Though I won't ever be a called a *bot, I surely hope I can contribute a little. Thanks for reading! From eric.nieuwland at xs4all.nl Tue Mar 22 10:16:21 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Tue Mar 22 10:16:57 2005 Subject: [Python-Dev] Draft PEP to make file objects support non-blocking mode. In-Reply-To: <1111453727.3786.40.camel@schizo> References: <1111116627.3762.4.camel@schizo> <20050319011940.GA5632@cthulhu.gerg.ca> <1111370352.3763.51.camel@schizo> <423F6B93.7020904@canterbury.ac.nz> <1111453727.3786.40.camel@schizo> Message-ID: Donovan Baarda wrote: > The fact that partial reads/writes are possible without non-blocking > mode changes things a fair bit. Also, the lack of fnctl support in > Windows needs to be taken into account too. > > ... [ snip ] ... > > The lack of support on win32 for non-blocking mode, combined with the > reduced need for it, makes adding a "setblocking" method undesirable. > > I don't know what the best thing to do now is... I guess the > readsome/writesome is probably best, but given that os.read/os.write is > not that bad, perhaps it's best to just forget I even suggested this > PEP :-) My EUR 0.01 is that we should aim at a higher level of abstraction. I really don't care Windows, Unix, Linux, WhateverOS provide me with a specific low level service. I care about the conceptual thing I'm trying to establish. Any abstraction that provides a means to express a solution more closely to that conceptual level is a winner. --eric From phd at mail2.phd.pp.ru Tue Mar 22 11:28:42 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Tue Mar 22 11:28:50 2005 Subject: [Python-Dev] Re: Change 'env var BROWSER override' semantics in webbrowser.py In-Reply-To: <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> Message-ID: <20050322102842.GA25949@phd.pp.ru> On Sun, Mar 20, 2005 at 11:40:27AM -0300, rodsenra@gpr.com.br wrote: > Perhaps you could focus in 728278. It addresses some of the issues you > have addressed in 754022, but it is not properly formatted. If you could > merge into your patch the result of "set(728278)-set(754022)", it would > be great. > > These two patches have the biggest number of changes, and most significant > ones. Naturally they are also the most conflicting. > > If these two are merged, then I believe *all* webbrowser.py related > patches could be addressed and closed quickly. I am working on them. I am going to consolidate these patches along with 954628 and 1166780 into one big patch. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ncoghlan at iinet.net.au Tue Mar 22 14:22:10 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Mar 22 14:33:08 2005 Subject: [Python-Dev] [AST] A somewhat less trivial patch than the last one. . . In-Reply-To: <423EDF2F.3000004@iinet.net.au> References: <423EDF2F.3000004@iinet.net.au> Message-ID: <42401C02.3050009@iinet.net.au> Nick Coghlan wrote: > I've put a first cut at generator expressions for the AST branch on > Sourceforge. It's enough to get test_grammar to pass, and tinkering at > the interactive prompt appears to work. I'm going to be out of the country until late next week, so if the folks sprinting on the AST branch at PyCon want to incorporate this (or parts of it), please go ahead. I'll be interested in seeing the results of the sprint when I get back. With any luck, generator expressions and decorators will both be available in a clean checkout :) Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From bac at ocf.berkeley.edu Tue Mar 22 15:24:26 2005 From: bac at ocf.berkeley.edu (Brett C.) Date: Tue Mar 22 15:24:34 2005 Subject: [Python-Dev] [AST] Procedure for AST Branch patches In-Reply-To: <423F8CBA.4060103@iinet.net.au> References: <0IDO004EW40OEPH0@vms040.mailsrvcs.net> <423F2190.1050209@ocf.berkeley.edu> <423F8CBA.4060103@iinet.net.au> Message-ID: <42402A9A.9060302@ocf.berkeley.edu> Nick Coghlan wrote: > Brett C. wrote: > >> OK, thanks to John Ehresman here at PyCon sprint I got logistix's >> patch applied. Beyond a warning that a warning that decode_unicode() >> is never called and the parser module failing to compile under Windows >> everything should be fine for compiling the AST branch. > > > Under Linux (Suse 9.1), the parser module compiles with a couple of > warnings about implicit declaration of PyParser_SimpleParseString, but > the .so fails to load due to a missing symbol PyNode_Compile. > It's a known issue. I am just ignoring the failed compile. >> Passing the test suite, though, is another question. =) > > > Heh. I managed to get test_grammar to pass 100% last night, which is at > least a start! > Yeah, I am hoping to look at your generators patch before you get back in the country. > And I must say that the new system is rather nice to work with - it took > me a while to work out what was going on, but the multi-pass 'build > AST', 'build symbol table', 'compile AST' is much cleaner than it is > with the lists-of-lists arrangement on the trunk (which is the entire > point of the AST-branch, I guess. . .) > How the system works will be *much* clearer with the new version of Python/compile.txt, assuming everyone has not been lying to me about the clarity. -Brett From nicksjacobson at yahoo.com Tue Mar 22 19:42:52 2005 From: nicksjacobson at yahoo.com (Nicholas Jacobson) Date: Tue Mar 22 19:42:56 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: 6667 Message-ID: <20050322184252.64079.qmail@web53904.mail.yahoo.com> Oops, you're right. What I should have said is to use an empty docstring as follows: """""" """Function docstring.""" def foo: ... or: """Module docstring.""" """""" def foo: ... So the first docstring is the module docstring, and the next is the first class/function. And again, if there's only one, it will belong to both. --- Anthony Baxter wrote: > On Monday 21 March 2005 20:08, Nicholas Jacobson > wrote: > > > How do you distinguish between a docstring at > the > > > top of a module > > > that's immediately followed by a function? Is > it > > > the module docstring > > > or the function docstring? > > > > It's both. The docstring would be assigned to > both > > the module and the function. This is a *good* > thing > > when there is a module with only one function in > it. > > i.e. there should only be one docstring for both, > and > > this saves repetition of that docstring. > > > > If a programmer wanted a docstring for the > function > > but not the module, a blank first line would do > the > > trick. A docstring for the module but not the > > function? Put a blank line between the module's > > docstring and the function. > > Yuk. This is magic taken to a ridiculous level. Note > that > "blank lines" currently have no meaning in Python, > and adding > a meaning to them is not my idea of a good thing. > > -- > Anthony Baxter > It's never too late to have a happy childhood. > __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From walter at livinglogic.de Tue Mar 22 21:32:17 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Mar 22 21:32:26 2005 Subject: [Python-Dev] New PyPI broken package editing Message-ID: <424080D1.2030001@livinglogic.de> I've uploaded a new package to the new PyPI. Editing this new packages gives me a unicode error. The URL is http://www.python.org/pypi?:action=submit_form&name=ll-ansistyle&version=0.6.1 The error I get is the following: --- Error... There's been a problem with your request exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 92: ordinal not in range(128) ---- I've used the distutils from current CVS and have author=u"Walter D?rwald" in my setup.py Bye, Walter D?rwald From martin at v.loewis.de Tue Mar 22 22:01:23 2005 From: martin at v.loewis.de (martin@v.loewis.de) Date: Tue Mar 22 22:01:26 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <424080D1.2030001@livinglogic.de> References: <424080D1.2030001@livinglogic.de> Message-ID: <1111525283.424087a3921d7@www.domainfactory-webmail.de> Zitat von Walter D?rwald : > I've uploaded a new package to the new PyPI. Editing this > new packages gives me a unicode error. The URL is > > http://www.python.org/pypi?:action=submit_form&name=ll-ansistyle&version=0.6.1 I see that the package is online now, so I assume that it now worked? > I've used the distutils from current CVS and have > author=u"Walter D?rwald" > in my setup.py This isn't supposed to work yet. Are you using the register command on this? Can you tell where it decides to encode as Latin-1? PyPI will reject anything that is not UTF-8. As for the uploads: you'll have noticed that it put the sdist files into packages/2.5; this is not supposed to happen. If you delete the files, and reupload them with the current CVS, the files should go into /packages/source. Regards, Martin From jimjjewett at gmail.com Tue Mar 22 22:09:29 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue Mar 22 22:09:31 2005 Subject: [Python-Dev] Re: python/dist/src/Lib/distutils/command upload.py, 1.3, 1.4 In-Reply-To: References: Message-ID: On Tue, 22 Mar 2005 12:32:46 -0800, loewis@users.sourceforge.net wrote: > Don't set the Python version for sdist uploads. Why not? I realize that version is more important for pre-compiled extension modules, but it applies even to a pure python package. Source code that uses @decoration won't work in python 1.5 Defaulting to the version used to create the package isn't perfect, and may sometimes be too conservative -- but as a default, is it really worse than nothing? -jJ From walter at livinglogic.de Tue Mar 22 22:40:50 2005 From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Mar 22 22:40:54 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <1111525283.424087a3921d7@www.domainfactory-webmail.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> Message-ID: <1160.84.56.112.149.1111527650.squirrel@isar.livinglogic.de> > Zitat von Walter D?rwald : > >> I've uploaded a new package to the new PyPI. Editing this >> new packages gives me a unicode error. The URL is >> >> http://www.python.org/pypi?:action=submit_form&name=ll-ansistyle&version=0.6.1 > > I see that the package is online now, so I assume that > it now worked? Uploading worked, but editing the package afterwards fails. >> I've used the distutils from current CVS and have >> author=u"Walter D?rwald" >> in my setup.py > > This isn't supposed to work yet. Are you using the > register command on this? Can you tell where it decides > to encode as Latin-1? PyPI will reject anything that is > not UTF-8. You might be right. I've tried with author=u"..." and author=u"...".encode("utf-8"). The second version might have been the one that worked. > As for the uploads: you'll have noticed that it put the > sdist files into packages/2.5; this is not supposed to > happen. If you delete the files, and reupload them with > the current CVS, the files should go into /packages/source. OK, I'll try again tomorrow morning. Bye, Walter D?rwald From facundobatista at gmail.com Tue Mar 22 23:21:49 2005 From: facundobatista at gmail.com (Facundo Batista) Date: Tue Mar 22 23:21:51 2005 Subject: [Python-Dev] Deprecating 2.2.1 old bugs Message-ID: Going on with the old bugs checking, here are the results for 2.2.1. When I'll finish, this will be put in an informational PEP. When I verified the bug, I filled two fields: - Summary: the same subject as in SF - Group: the bug's group at verifying time. - Bug #: the bug number - Verified: is the date when I checked the bug. - Action: is what I did then. If the bug survived the verification, the next two fields are applicable (if not, I put a dash, the idea is to keep this info easily parseable): - Final: is the action took by someone who eliminated the bug from that category (closed, moved to Py2.4, etc). - By: is the someone who did the final action. So: Summary: segfault when running smtplib example Group: 2.2.1 Bug #: 1003195 Verified: 26-Dic-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: sgmllib incorrectly handles entities in attributes Group: 2.2.1 Bug #: 779388 Verified: 26-Dic-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: urlparse goes wrong with IP:port without scheme Group: 2.2.1 Bug #: 754016 Verified: 26-Dic-2004 Action: Changed to Group Py2.3: it seems a bug to me. Final: - By: - Summary: Failed assert in stringobject.c Group: 2.2.1 Bug #: 737947 Verified: 26-Dic-2004 Action: Closed: Fixed. The problem is solved, verified in Python 2.3.4. Final: - By: - Summary: A large block of commands after an "if" cannot be Group: 2.2.1 Bug #: 711268 Verified: 27-Dic-2004 Action: Changed to Group Py2.4: it seems a bug to me. Final: Re-classified as a Feature Request By: rhettinger Summary: urllib.url2pathname, pathname2url doc strings inconsistent Group: 2.2.1 Bug #: 649974 Verified: 26-Dic-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. Final: Changed the group to Python 2.4 By: mike_j_brown Summary: nturl2path.url2pathname() mishandles /// Group: 2.2.1 Bug #: 649961 Verified: 27-Dic-2004 Action: Deprecation alerted. Not sure if there's a bug, maybe in the documentation. Final: Closed: Won't fix. By: mike_j_brown Summary: print to unicode stream should __unicode Group: 2.2.1 Bug #: 637094 Verified: 01-Dec-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. Final: Re-classified as a Feature Request By: lemburg Summary: Unix popen does not return exit status Group: 2.2.1 Bug #: 608635 Verified: 26-Dic-2004 Action: Deprecation alerted. Don't know enough about the subject to be able to reproduce the issue. Final: Closed: Won't fix. By: facundobatista 608033: I can not try it, don't have that context. Summary: Implied __init__.py not copied Group: 2.2.1 Bug #: 608033 Verified: 01-Dic-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: pydoc -g dumps core on Solaris 2.8 Group: 2.2.1 Bug #: 602627 Verified: 30-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: makesetup fails: long Setup.local lines Group: 2.2.1 Bug #: 591287 Verified: 30-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: os.tmpfile() can fail on win32 Group: 2.2.1 Bug #: 591104 Verified: 30-Nov-2004 Action: Deprecation alerted. Because a comment is not clear to me if the problem actually existed. Final: Closed: Won't fix. By: facundobatista Summary: pthread_exit missing in thread_pthread.h Group: 2.2.1 Bug #: 579116 Verified: 30-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Summary: cgi.py broken with xmlrpclib on Python 2 Group: 2.2.1 Bug #: 569316 Verified: 30-Nov-2004 Action: Deprecation alerted. Don't know enough about the subject to be able to reproduce the issue. Final: Closed: Won't fix. By: facundobatista Summary: Popen exectuion blocking w/threads Group: 2.2.1 Bug #: 566037 Verified: 29-Nov-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. Works in different context. Final: Closed: Won't fix. By: facundobatista Summary: fpectl build on solaris: -lsunmath Group: 2.2.1 Bug #: 530163 Verified: 29-Nov-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. I can not try it, don't have that context. Final: Closed: Won't fix. By: facundobatista Regards, . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://pyar.decode.com.ar/ From fdrake at acm.org Wed Mar 23 00:00:50 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Mar 23 00:01:07 2005 Subject: [Python-Dev] Re: python/dist/src/Lib/distutils/command upload.py, 1.3, 1.4 In-Reply-To: References: Message-ID: <200503221800.50452.fdrake@acm.org> On Tuesday 22 March 2005 16:09, Jim Jewett wrote: > Why not? I realize that version is more important for pre-compiled > extension modules, but it applies even to a pure python package. > > Source code that uses @decoration won't work in python 1.5 This is distinct from the version of Python used to build a distribution. I've noted this as a metadata field that is needed, and plan to draft a PEP to include this and a few others. What I don't want is to conflate fields that should be separate, and end up with crufty data in PyPI. It's better to know that some data is missing than to be wrong about it. > Defaulting to the version used to create the package isn't perfect, > and may sometimes be too conservative -- but as a default, is it > really worse than nothing? Yes, because we can't tell the difference later. For now, you can include comments in the long description or on a project web page. A proper field will be added for this in the future (hopefully not too distant). -Fred -- Fred L. Drake, Jr. From jimjjewett at gmail.com Wed Mar 23 00:29:05 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed Mar 23 00:29:07 2005 Subject: [Python-Dev] Re: python/dist/src/Lib/distutils/command upload.py, 1.3, 1.4 In-Reply-To: <200503221800.50452.fdrake@acm.org> References: <200503221800.50452.fdrake@acm.org> Message-ID: On Tue, 22 Mar 2005 18:00:50 -0500, "Fred L. Drake, Jr." wrote: > On Tuesday 22 March 2005 16:09, Jim Jewett wrote: > > Why not? I realize that version is more important for pre-compiled > > extension modules, but it applies even to a pure python package. > > Source code that uses @decoration won't work in python 1.5 > This is distinct from the version of Python used to build a distribution. In theory, yes. My suspicion is that if people are using the defaults, then they are probably also using a python version that they have tested with -- and perhaps the only one that they have tested with. > > Defaulting to the version used to create the package isn't perfect, > > and may sometimes be too conservative -- but as a default, is it > > really worse than nothing? > Yes, because we can't tell the difference later. For now, you can include > comments in the long description or on a project web page. A proper field > will be added for this in the future (hopefully not too distant). How about changing it to tack on a "(guess)" instead of just deleting it? Or does that change break too many other things/cause too much ugliness for the timeframe it will be used in? -jJ From fdrake at acm.org Wed Mar 23 00:39:19 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Mar 23 00:39:48 2005 Subject: [Python-Dev] Re: python/dist/src/Lib/distutils/command upload.py, 1.3, 1.4 In-Reply-To: References: <200503221800.50452.fdrake@acm.org> Message-ID: <200503221839.19082.fdrake@acm.org> On Tuesday 22 March 2005 18:29, Jim Jewett wrote: > > This is distinct from the version of Python used to build a > > distribution. > > In theory, yes. > > My suspicion is that if people are using the defaults, then they are > probably also using a python version that they have tested with -- and > perhaps the only one that they have tested with. I think this varies substantially. I routinely test cross-version code with several versions of Python. > How about changing it to tack on a "(guess)" instead of just deleting it? But it's not a guess for non-source distributions. > Or does that change break too many other things/cause too much > ugliness for the timeframe it will be used in? Too much ugliness. Remember, this field is attached to the uploaded file, not the release as a whole. Many packages are going to see uploads for two or more versions of Python (PyXML for example). Getting the metadata wrong is just too evil. Now that more people have seen the code for PyPI (and understand more of it), it'll be easier to implement new fields once they've been carefully defined. -Fred -- Fred L. Drake, Jr. From martin at v.loewis.de Wed Mar 23 00:54:54 2005 From: martin at v.loewis.de (martin@v.loewis.de) Date: Wed Mar 23 00:55:05 2005 Subject: [Python-Dev] Re: python/dist/src/Lib/distutils/command upload.py, 1.3, 1.4 In-Reply-To: References: <200503221800.50452.fdrake@acm.org> Message-ID: <1111535694.4240b04e10f14@www.domainfactory-webmail.de> Zitat von Jim Jewett : > How about changing it to tack on a "(guess)" instead of just deleting it? I think it would be confusing to users. Currently, the only version you would get into the database is 2.5, as earlier versions don't have the upload command. So we would have all sdist releases under packages/2.5, which would be too confusing, IMO. > Or does that change break too many other things/cause too much > ugliness for the timeframe it will be used in? It's also used to build the download link under python.org/packages. At the moment, changing the upload command would have no effect, since PyPI always puts sdist packages under packages/source, and clears the Python version if there is one. As Fred says, if a package has a minimum Python version, it should specify this - either in a new metadata field, or through an entry into Requires; PEP 314 somewhat suggests that you can do Depends: sys (>=2.3) Regards, Martin From jcarlson at uci.edu Wed Mar 23 03:28:35 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed Mar 23 03:29:35 2005 Subject: [Python-Dev] @decoration of classes Message-ID: <20050322182041.DB5D.JCARLSON@uci.edu> Good day, I just noticed that decoration of classes was not included with the @decoration syntax that made it into Python 2.4. While I understand that class decoration was not a part of PEP 318, I remember people were interested in decorating classes for all sorts of reasons, among them as a prefix notation for documenting (which seems to nearly satisfy Nicholas Jacobson) as well as a partial metaclass replacement. Is the fact that it didn't make it into 2.4 due to a pronouncement that I missed/forgot, lack of time, or was it merely forgotten? Thanks, - Josiah From gvanrossum at gmail.com Wed Mar 23 04:20:17 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Wed Mar 23 04:20:20 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050322182041.DB5D.JCARLSON@uci.edu> References: <20050322182041.DB5D.JCARLSON@uci.edu> Message-ID: > I just noticed that decoration of classes was not included with the > @decoration syntax that made it into Python 2.4. While I understand > that class decoration was not a part of PEP 318, I remember people were > interested in decorating classes for all sorts of reasons, among them as > a prefix notation for documenting (which seems to nearly satisfy > Nicholas Jacobson) as well as a partial metaclass replacement. > > Is the fact that it didn't make it into 2.4 due to a pronouncement that > I missed/forgot, lack of time, or was it merely forgotten? I don't recall whether I pronounced or not, but it is my opinion that this isn't addressing nearly as big an issue for classes as it is for functions/methods; in particular, the main problem of having all the special handling *after* the body of the function doesn't occur for classes, where you can put all the special handling at the top of the body, perhaps aided by a crafty metaclass. It would take a lot of convincing before I would think that class @decorators are better than metaclasses. In any case the fact that it wasn't in the PEP was plenty of reason not to add it to 2.4. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From anthony at interlink.com.au Wed Mar 23 04:35:34 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Wed Mar 23 04:36:08 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: References: <20050322182041.DB5D.JCARLSON@uci.edu> Message-ID: <200503231435.35807.anthony@interlink.com.au> On Wednesday 23 March 2005 14:20, Guido van Rossum wrote: > It would take a lot of convincing before I would think that class > @decorators are better than metaclasses. > > In any case the fact that it wasn't in the PEP was plenty of reason > not to add it to 2.4. Minor clarification - it _was_ in the PEP, but when I did the big rewrite to match the state of the art, the class decoration stuff was ripped out. If it's thought worthwhile, someone could update the PEP to include Guido's message. -- Anthony Baxter It's never too late to have a happy childhood. From walter at livinglogic.de Wed Mar 23 12:07:58 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Mar 23 12:08:02 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <1111525283.424087a3921d7@www.domainfactory-webmail.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> Message-ID: <42414E0E.60808@livinglogic.de> martin@v.loewis.de wrote: > Zitat von Walter D?rwald : > >>I've uploaded a new package to the new PyPI. Editing this >>new packages gives me a unicode error. The URL is >> >>http://www.python.org/pypi?:action=submit_form&name=ll-ansistyle&version=0.6.1 > > I see that the package is online now, so I assume that > it now worked? OK, I've deleted the files and the packages. Running "setup.py register" with author=u"Walter D?rwald" in setup.py gives me: --- running register Using PyPI login from /home/walter/.pypirc Server response (500): Internal Server Error --- Using author=u"Walter D?rwald".encode("utf-8") in setup.py works. I'm not sure if this is the right approach. The encoding I specify in setup.py should be independent of the encoding used between distutils and PyPI to communicate on the wire. I.e. the author (and maintainer) argument should always be unicode. When str is passed, this is treated as any other str in a unicode context, it is decoded using the default encoding. This would fix another problem: It would make it nearly impossible to send a request to PyPI with the wrong encoding, because any encoding problems are sorted out completely on the client side. > [...] > As for the uploads: you'll have noticed that it put the > sdist files into packages/2.5; this is not supposed to > happen. If you delete the files, and reupload them with > the current CVS, the files should go into /packages/source. OK, I've re-uploaded the packages. BTW, uploading the packages a second time leads to the following problem: --- running upload Submitting dist/ll-ansistyle-0.6.1.tar.bz2 to http://www.python.org/pypi Upload failed (500): There's been a problem with your request Submitting dist/ll-ansistyle-0.6.1.tar.gz to http://www.python.org/pypi Upload failed (500): There's been a problem with your request --- Is there a way to display the HTTP response by PyPI? Editing the package is still broken. The link "edit" on the page http://www.python.org/pypi/ll-ansistyle/0.6.1 gives: --- Error... There's been a problem with your request exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 92: ordinal not in range(128) --- Bye, Walter D?rwald From stan at phidani.be Wed Mar 23 12:13:56 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 23 12:14:01 2005 Subject: [Python-Dev] bug in pythondotnet implementation. Maybe related to a bug in cpython implementation...help!!!! Message-ID: <42414F74.9000109@phidani.be> hello, I would welcome any help regarding: -how can I get/give more info on what's happening? -how to solve that stuff? thanks a lot in advance. here is the problem: I have a python (actually pythondotnet) process freezing on windows, like that: Thread Start Address: >Symbol Name: Line Number: PC: >mscoree!_CorExeMain() + 0x0 --- 7917D08C > >Thread Stack: > >ntdll ! KiFastSystemCallRet() + 0x >KERNEL32 ! WaitForSingleObject() + 0x12 >python24 ! PySys_WriteStderr() + 0x14d >python24 ! PyTuple_Type() + 0x0 How can I know who's calling PyTuple_Type()??? see below for full description of my problem. am running Simpy (python simulation framework) within pythondotnet, and, even though this process is single-thread, it hangs misteriously, in an unpredictable way... Python console cease to respond to Ctrl-C events... Here is the current Thread status: Thread Start Address: Symbol Name: Line Number: PC: mscoree!_CorExeMain() + 0x0 --- 7917D08C Thread Stack: ntdll ! KiFastSystemCallRet() + 0x KERNEL32 ! WaitForSingleObject() + 0x12 python24 ! PySys_WriteStderr() + 0x14d python24 ! PyTuple_Type() + 0x0 as the entry point in the hanging thread (higher on stack) is PyTuple_Type() and as PyTuple is defined in C# (src/runtime/PyTuple.cs), I suspect this might be the cause of my problem. [PythonNet-1.0-beta4]> grep -nr "PyTuple_Type" . Binary file ./DLLs/_socket.pyd matches Binary file ./python24.dll matches [PythonNet-1.0-beta4]> However, I would like to be able to go higher in the stack, to see what caused this deadlock. Any proposed strategy to guess what happened, or to track down the problem? thanks a lot, Stan. From p.f.moore at gmail.com Wed Mar 23 13:27:13 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Wed Mar 23 13:27:15 2005 Subject: [Python-Dev] bug in pythondotnet implementation. Maybe related to a bug in cpython implementation...help!!!! In-Reply-To: <42414F74.9000109@phidani.be> References: <42414F74.9000109@phidani.be> Message-ID: <79990c6b050323042725a86d14@mail.gmail.com> On Wed, 23 Mar 2005 12:13:56 +0100, Stan Pinte wrote: > I would welcome any help regarding: > > -how can I get/give more info on what's happening? > -how to solve that stuff? > > thanks a lot in advance. > > here is the problem: > > I have a python (actually pythondotnet) process freezing on windows, > like that: Hi, This is off-topic for python-dev, which is for discussion of the development OF Python, not for discussion of programs written IN Python. For this problem, I'd suggest that you ask either on comp.lang.python, or probably more appropriately on one of the python.NET lists (I know there are some, but I'm afraid I can't recall the details). Thanks, Paul. From phd at mail2.phd.pp.ru Wed Mar 23 13:40:20 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed Mar 23 13:40:22 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 Message-ID: <20050323124020.GA9693@phd.pp.ru> Hello! While I'm working on webbrowser... Why do all graphical browsers are called with their stdout/stderr redirected to /dev/null? Do we really need to hide problems from the user? Browsers are usually silent beasts - they interact with the user using windows, not stdio. (Text-mode browsers, naturally, use stdout... for their windows). I'd like to remove all those redirects. Any opinion? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From rodsenra at gpr.com.br Wed Mar 23 14:25:12 2005 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Wed Mar 23 14:25:19 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323124020.GA9693@phd.pp.ru> References: <20050323124020.GA9693@phd.pp.ru> Message-ID: <20050323102512.0000605e@Fenix> On Wed, 23 Mar 2005 15:40:20 +0300 Oleg Broytmann wrote: > Hello! > > While I'm working on webbrowser... Great. > Why do all graphical browsers are called with their stdout/stderr redirected > to /dev/null? Under some linux distros (I'm positive for some Mdk releases), Mozilla is compiled dumping a lot of info to stdout/stderr. Since one of the goals of webbrowser is to give the end-user a stress-free experience, there goes the mentioned nullification . In a development environment, a developer should not find difficulty to reverse that if needed. > I'd like to remove all those redirects. Any opinion? -1 for me. best regards, Rod Senra From phd at mail2.phd.pp.ru Wed Mar 23 15:27:44 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed Mar 23 15:27:46 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323102512.0000605e@Fenix> References: <20050323124020.GA9693@phd.pp.ru> <20050323102512.0000605e@Fenix> Message-ID: <20050323142744.GB12947@phd.pp.ru> On Wed, Mar 23, 2005 at 10:25:12AM -0300, Rodrigo Dias Arruda Senra wrote: > > Why do all graphical browsers are called with their stdout/stderr redirected > > to /dev/null? > > Under some linux distros (I'm positive for some Mdk releases), Mozilla is > compiled dumping a lot of info to stdout/stderr. Since one of the goals of > webbrowser is to give the end-user a stress-free experience, there goes the > mentioned nullification . I see the point. Still I don't know what is worse and more stressful - to hide errors or to show errors. MandrakeZilla spits too much to stdout/err? That's certainly a problem. Should we "fix" it and hide from the user? I don't think so. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From rodsenra at gpr.com.br Wed Mar 23 15:59:24 2005 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Wed Mar 23 15:59:31 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323142744.GB12947@phd.pp.ru> References: <20050323124020.GA9693@phd.pp.ru> <20050323102512.0000605e@Fenix> <20050323142744.GB12947@phd.pp.ru> Message-ID: <20050323115924.00003889@Fenix> [Rod Senra]: > > Under some linux distros (I'm positive for some Mdk releases), Mozilla is > > compiled dumping a lot of info to stdout/stderr. Since one of the goals of > > webbrowser is to give the end-user a stress-free experience, there goes the > > mentioned nullification . [Oleg Broytmann]: > > I see the point. Still I don't know what is worse and more stressful > to hide errors or to show errors. > MandrakeZilla spits too much to stdout/err? That's certainly a > problem. Should we "fix" it and hide from the user? I don't think so. That is undoubtly a good argument. In general, if the end user could fix or report a problem based on a stdout/stderror message, I couln't agree more on keeping them flowing. However, there are two other issues: 1) If a *graphical* application dumps messages to the console, that might be disruptive to other console applications. IMVHO, a log file should be used instead. (strong argument) 2) If a dummy user sees a warning or info message in stdout/stdin that is not necessarily critical, it might interpret it wrongly as a error message and generate a false bug report. (weak argument) In the case of webbrowser.py, since detection process might face a diverse plethora of browsers (even unknown if defined by environment variables), we cannot predict if 1) or 2) will ever happen. Therefore, my -1 vote in my previous reply. But I do see your point . Has this same issue been dealt in another stdlib module ? best regards, Rod Senra From hermantootrot at hotmail.com Wed Mar 23 16:05:27 2005 From: hermantootrot at hotmail.com (Herman Toothrot) Date: Wed Mar 23 16:05:31 2005 Subject: [Python-Dev] Ye don't be needin' these! Message-ID: Avast! Why be there builtins divmod and pow, when operators **, /, and % should be good enough for ya? It runs counter to TOOWTDI, I be thinking. Arr. H. Toothrot _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today - it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/ From phd at mail2.phd.pp.ru Wed Mar 23 16:59:46 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed Mar 23 16:59:52 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323115924.00003889@Fenix> References: <20050323124020.GA9693@phd.pp.ru> <20050323102512.0000605e@Fenix> <20050323142744.GB12947@phd.pp.ru> <20050323115924.00003889@Fenix> Message-ID: <20050323155946.GB16781@phd.pp.ru> On Wed, Mar 23, 2005 at 11:59:24AM -0300, Rodrigo Dias Arruda Senra wrote: > Has this same issue been dealt in another stdlib module ? pydoc.py: rc = os.system('netscape -remote "openURL(%s)" &' % url) if rc: os.system('netscape "%s" &' % url) PS. This, of course, should must be fixed - pydoc must use webbrowser.py! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at mail2.phd.pp.ru Wed Mar 23 17:00:51 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed Mar 23 17:00:55 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323155946.GB16781@phd.pp.ru> References: <20050323124020.GA9693@phd.pp.ru> <20050323102512.0000605e@Fenix> <20050323142744.GB12947@phd.pp.ru> <20050323115924.00003889@Fenix> <20050323155946.GB16781@phd.pp.ru> Message-ID: <20050323160051.GC16781@phd.pp.ru> Oops... > PS. This, of course, should must be fixed - pydoc must use webbrowser.py! ^^^^^^ delete (-: Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From aahz at pythoncraft.com Wed Mar 23 17:20:00 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed Mar 23 17:20:03 2005 Subject: [Python-Dev] Ye don't be needin' these! In-Reply-To: References: Message-ID: <20050323162000.GA23623@panix.com> On Wed, Mar 23, 2005, Herman Toothrot wrote: > > Avast! Why be there builtins divmod and pow, when operators **, /, and % > should be good enough for ya? It runs counter to TOOWTDI, I be thinking. > Arr. This is off-topic for python-dev. Please post to comp.lang.python instead. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From rkern at ucsd.edu Wed Mar 23 17:04:07 2005 From: rkern at ucsd.edu (Robert Kern) Date: Wed Mar 23 17:27:15 2005 Subject: [Python-Dev] Re: Ye don't be needin' these! In-Reply-To: References: Message-ID: Herman Toothrot wrote: > Avast! Why be there builtins divmod and pow, when operators **, /, and > % should be good enough for ya? It runs counter to TOOWTDI, I be > thinking. Arr. Well, divmod(x, y) does both / and % in one shot, which can be very useful. pow(x, y[, z]) has an optional third argument ((x**y) % z), which is necessary for really large numbers like the ones you play with in cryptography. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From python-dev at zesty.ca Wed Mar 23 17:33:53 2005 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Wed Mar 23 17:33:56 2005 Subject: [Python-Dev] Shorthand for lambda Message-ID: Hey folks, I'm sitting over here in the AppleScript talk and Jacob is explaining a module called 'appscript' that interfaces to the Apple Events system. What caught my eye was this example: from appscript import * ab = app('Address Book') people = ab.people.filter(its.emails != []) That last line asks the Address Book to select only entries with e-mail addresses. The weird 'its' object comes from the appscript module -- asking for its properties and using operators causes it to set up thunks for you. It dawned on me that you could use this idea to make the whole filter/lambda experience vastly more pleasant. I whipped up a quick implementation: >>> from placeholder import _ >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89] >>> filter(_ < 30, numbers) [5, 9, 1, 24] >>> map(_ + 10, numbers) [15, 19, 66, 44, 11, 34, 47, 99] >>> Look ma, no lambdas! I bet someone has already done this before, right? -- ?!ng From jhylton at gmail.com Wed Mar 23 17:44:10 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Wed Mar 23 17:44:12 2005 Subject: [Python-Dev] Shorthand for lambda In-Reply-To: References: Message-ID: For filter and map, list comprehensions and generator expressions are the answer. >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89] >>> [x for x in numbers if x < 30] [5, 9, 1, 24] >>> (x for x in numbers if x < 30) >>> list(_) [5, 9, 1, 24] Jeremy On Wed, 23 Mar 2005 10:33:53 -0600 (CST), Ka-Ping Yee wrote: > Hey folks, > > I'm sitting over here in the AppleScript talk and Jacob is explaining a > module called 'appscript' that interfaces to the Apple Events system. > > What caught my eye was this example: > > from appscript import * > ab = app('Address Book') > people = ab.people.filter(its.emails != []) > > That last line asks the Address Book to select only entries with > e-mail addresses. The weird 'its' object comes from the appscript > module -- asking for its properties and using operators causes it > to set up thunks for you. > > It dawned on me that you could use this idea to make the whole > filter/lambda experience vastly more pleasant. I whipped up a quick > implementation: > > >>> from placeholder import _ > >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89] > >>> filter(_ < 30, numbers) > [5, 9, 1, 24] > >>> map(_ + 10, numbers) > [15, 19, 66, 44, 11, 34, 47, 99] > >>> > > Look ma, no lambdas! > > I bet someone has already done this before, right? > > -- ?!ng > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From p.f.moore at gmail.com Wed Mar 23 17:48:42 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Wed Mar 23 17:48:44 2005 Subject: [Python-Dev] Shorthand for lambda In-Reply-To: References: Message-ID: <79990c6b05032308486868c30a@mail.gmail.com> On Wed, 23 Mar 2005 10:33:53 -0600 (CST), Ka-Ping Yee wrote: > It dawned on me that you could use this idea to make the whole > filter/lambda experience vastly more pleasant. I whipped up a quick > implementation: > > >>> from placeholder import _ > >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89] > >>> filter(_ < 30, numbers) > [5, 9, 1, 24] > >>> map(_ + 10, numbers) > [15, 19, 66, 44, 11, 34, 47, 99] > >>> > > Look ma, no lambdas! > > I bet someone has already done this before, right? I thought about it, but couldn't convince myself that it would work properly in all cases. I was thinking in terms of operator overloading of everything possible - how did you do it? Paul. From tjreedy at udel.edu Wed Mar 23 17:34:09 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Mar 23 17:55:23 2005 Subject: [Python-Dev] Re: Ye don't be needin' these! References: Message-ID: "Herman Toothrot" wrote in message news:BAY23-F5C463CC802BE7A7513435BB4F0@phx.gbl... > Avast! Why be there builtins divmod and pow, when operators **, /, and % > should be good enough for ya? It runs counter to TOOWTDI, I be thinking. Questions like this should be asked on comp.lang.python or the python mailing list. I'll answer if I see it there. Terry J. Reedy From jcarlson at uci.edu Wed Mar 23 18:48:08 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed Mar 23 18:49:07 2005 Subject: [Python-Dev] Shorthand for lambda In-Reply-To: <79990c6b05032308486868c30a@mail.gmail.com> References: <79990c6b05032308486868c30a@mail.gmail.com> Message-ID: <20050323093710.DB60.JCARLSON@uci.edu> Paul Moore wrote: > > On Wed, 23 Mar 2005 10:33:53 -0600 (CST), Ka-Ping Yee > wrote: > > It dawned on me that you could use this idea to make the whole > > filter/lambda experience vastly more pleasant. I whipped up a quick > > implementation: > > > > >>> from placeholder import _ > > >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89] > > >>> filter(_ < 30, numbers) > > [5, 9, 1, 24] > > >>> map(_ + 10, numbers) > > [15, 19, 66, 44, 11, 34, 47, 99] > > >>> > > > > Look ma, no lambdas! > > > > I bet someone has already done this before, right? > > I thought about it, but couldn't convince myself that it would work > properly in all cases. I was thinking in terms of operator overloading > of everything possible - how did you do it? PyTables allows something very similar for "in-kernel" searches of data, but only on a single constraint. I would imagine that Ka-Ping did it by only allowing a single operation per item. - Josiah From python-dev at zesty.ca Wed Mar 23 19:11:55 2005 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Wed Mar 23 19:12:01 2005 Subject: [Python-Dev] Shorthand for lambda In-Reply-To: <20050323093710.DB60.JCARLSON@uci.edu> References: <79990c6b05032308486868c30a@mail.gmail.com> <20050323093710.DB60.JCARLSON@uci.edu> Message-ID: On Wed, 23 Mar 2005, Josiah Carlson wrote: > Paul Moore wrote: > > I thought about it, but couldn't convince myself that it would work > > properly in all cases. I was thinking in terms of operator overloading > > of everything possible - how did you do it? > > PyTables allows something very similar for "in-kernel" searches of data, > but only on a single constraint. I would imagine that Ka-Ping did it by > only allowing a single operation per item. You can do more than one operation, but the usage is still quite limited. The item placeholder must be the first operand for it to work. >>> numbers = [3, 8, 4, 1, 2] >>> filter(_ < 5, numbers) [3, 4, 1, 2] >>> map(_ * 5 + 7, numbers) [10, 15, 11, 8, 9] I tried implementing __len__, but that doesn't work because Python enforces a type restriction. >>> words = 'lovely spam and eggs'.split() >>> filter(len(_) == 4, words) Traceback (most recent call last): File "", line 1, in ? TypeError: __len__() should return an int __getitem__ and __getattr__ mostly work. However, in order to call a method on the placeholder you have to add an underscore to distinguish it from retrieving an attribute. >>> filter(_.endswith_('s'), words) ['eggs'] You can check out http://zesty.ca/python for the gory details. As Jeremy wrote, the proper way to do map and filter is to use a list comprehension, so these are bad examples. The original motivation was to provide a way to write lambda expressions for cases where you aren't doing map or filter. For that, it works, but only in limited cases. I realize this isn't that practical. It's mainly for your amusement -- yet another in a long tradition of hacks that use operator overloading to hijack the Python parser. (Also a long tradition of me doing silly things in public.) -- ?!ng From reinhold-birkenfeld-nospam at wolke7.net Wed Mar 23 19:02:47 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed Mar 23 19:12:23 2005 Subject: [Python-Dev] Re: Shorthand for lambda In-Reply-To: References: Message-ID: Ka-Ping Yee wrote: > It dawned on me that you could use this idea to make the whole > filter/lambda experience vastly more pleasant. I whipped up a quick > implementation: > > >>> from placeholder import _ > >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89] > >>> filter(_ < 30, numbers) > [5, 9, 1, 24] > >>> map(_ + 10, numbers) > [15, 19, 66, 44, 11, 34, 47, 99] > >>> > > Look ma, no lambdas! What does you implementation do for this: >>> somevar = False >>> filter(_ and False, numbers) Reinhold From python-dev at zesty.ca Wed Mar 23 19:16:17 2005 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Wed Mar 23 19:16:29 2005 Subject: [Python-Dev] Re: Shorthand for lambda In-Reply-To: References: Message-ID: On Wed, 23 Mar 2005, Reinhold Birkenfeld wrote: > What does you implementation do for this: > > >>> somevar = False > >>> filter(_ and False, numbers) It fails. (For the same reason that __len__ doesn't work -- Python insists that __nonzero__ must return an int.) Though i must say i have no idea what you are trying to do here. If you filter on False, you'll always get an empty list. -- ?!ng From phd at mail2.phd.pp.ru Wed Mar 23 19:29:30 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed Mar 23 19:29:36 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050322102842.GA25949@phd.pp.ru> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> Message-ID: <20050323182930.GB1084@phd.pp.ru> On Tue, Mar 22, 2005 at 01:28:42PM +0300, Oleg Broytmann wrote: > On Sun, Mar 20, 2005 at 11:40:27AM -0300, rodsenra@gpr.com.br wrote: > > Perhaps you could focus in 728278. It addresses some of the issues you > > have addressed in 754022, but it is not properly formatted. If you could > > merge into your patch the result of "set(728278)-set(754022)", it would > > be great. > > > > These two patches have the biggest number of changes, and most significant > > ones. Naturally they are also the most conflicting. > > > > If these two are merged, then I believe *all* webbrowser.py related > > patches could be addressed and closed quickly. > > I am working on them. I am going to consolidate these patches along > with 954628 and 1166780 into one big patch. Well, I've consolidated patches 728278, 954628, 1166780 into 754022. Some parts of those patches were applied, some rejected, many things changed. Suggested resolutions: http://python.org/sf/728278 Close with resolution "partially applied, partially rejected". http://python.org/sf/754022 Review and apply! ;) http://python.org/sf/1166780 Close with resolution "applied". (Though it was not applied in exactly that form...) http://python.org/sf/1077979 Close with resolution "applied long ago". http://python.org/sf/1144816 Close with resolution "duplicate of 1077979". I tested the consolidated patch on Linux with Mozilla/links/elinks browsers, and on w32 with default-browser and with Mozilla. I also added elinks support - currently it is very similar to links, but I am going to extend its remote capabilities. (Yes, that small text-mode broswer supports remoting, windows and tabs! Who'd think?!.) Also I'm going to add "new-tab" support similar to "new-window" for Mozilla/Firefox and elinks. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From bob at redivi.com Wed Mar 23 19:51:18 2005 From: bob at redivi.com (Bob Ippolito) Date: Wed Mar 23 19:51:25 2005 Subject: [Python-Dev] Re: Shorthand for lambda In-Reply-To: References: Message-ID: <6db1c5569e2a29f3608855889b7bd529@redivi.com> On Mar 23, 2005, at 1:16 PM, Ka-Ping Yee wrote: > On Wed, 23 Mar 2005, Reinhold Birkenfeld wrote: >> What does you implementation do for this: >> >>>>> somevar = False >>>>> filter(_ and False, numbers) > > It fails. (For the same reason that __len__ doesn't work -- > Python insists that __nonzero__ must return an int.) Though > i must say i have no idea what you are trying to do here. > If you filter on False, you'll always get an empty list. Similarly, appscript provides function versions of operators named such as AND and OR. I suppose there could be a length one as well (in AppleScript terminology, it would be called count), or you could technically denote it as __len__(), but that's quite ugly. I had implemented something quite similar to this a long time ago, but considered it "evil" and never used it for anything: http://tinyurl.com/6ft4h -bob From tlesher at gmail.com Wed Mar 23 19:59:29 2005 From: tlesher at gmail.com (Tim Lesher) Date: Wed Mar 23 19:59:34 2005 Subject: [Python-Dev] Re: Ye don't be needin' these! In-Reply-To: References: Message-ID: <9613db60050323105973d50748@mail.gmail.com> On Wed, 23 Mar 2005 11:34:09 -0500, Terry Reedy wrote: > > "Herman Toothrot" wrote in message > news:BAY23-F5C463CC802BE7A7513435BB4F0@phx.gbl... > > Avast! Why be there builtins divmod and pow, when operators **, /, and % > > should be good enough for ya? It runs counter to TOOWTDI, I be thinking. > > Questions like this should be asked on comp.lang.python or the python > mailing list. I'll answer if I see it there. I have to wonder if this wasn't a tongue-in-cheek message sent from a just-created hotmail account, by an existing python-dev participant who's embroiled in the current "do we even need functionals anymore" discussion... -- Tim Lesher From reinhold-birkenfeld-nospam at wolke7.net Wed Mar 23 20:02:32 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Wed Mar 23 20:10:02 2005 Subject: [Python-Dev] Re: Shorthand for lambda In-Reply-To: References: Message-ID: Ka-Ping Yee wrote: > On Wed, 23 Mar 2005, Reinhold Birkenfeld wrote: >> What does you implementation do for this: >> >> >>> somevar = False >> >>> filter(_ and False, numbers) > > It fails. (For the same reason that __len__ doesn't work -- > Python insists that __nonzero__ must return an int.) Though > i must say i have no idea what you are trying to do here. > If you filter on False, you'll always get an empty list. I know; I just wanted to show that this approach can be very misleading as and/or can't be overloaded. Reinhold -- Mail address is perfectly valid! From rodsenra at gpr.com.br Wed Mar 23 20:29:36 2005 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Wed Mar 23 20:29:42 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050323182930.GB1084@phd.pp.ru> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> Message-ID: <20050323162936.00004052@Fenix> On Wed, 23 Mar 2005 21:29:30 +0300 Oleg Broytmann wrote: > Suggested resolutions: > http://python.org/sf/754022 > Review and apply! ;) Reviewed. Thank you Oleg, fine integration job. I added a +1 comment to the tracker and copied your remaining obs to 754022 history. So a commiter-dev just have to mind about 754022. > I also added elinks support - currently it is very similar to links, > but I am going to extend its remote capabilities. (Yes, that small > text-mode broswer supports remoting, windows and tabs! Who'd think?!.) > Also I'm going to add "new-tab" support similar to "new-window" for > Mozilla/Firefox and elinks. Excellent. cheers, Rod Senra From tdelaney at avaya.com Wed Mar 23 21:43:48 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Wed Mar 23 21:45:03 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE02520454@au3010avexu1.global.avaya.com> Rodrigo Dias Arruda Senra wrote: > However, there are two other issues: > 1) If a *graphical* application dumps messages to the console, > that might be disruptive to other console applications. > IMVHO, a log file should be used instead. (strong argument) Perhaps instead webbrowser.py should redirect to a (preferably specified) log file. The detail is then still available, but hidden from normal usage. Tim Delaney From jcarlson at uci.edu Wed Mar 23 22:26:44 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed Mar 23 22:28:14 2005 Subject: [Python-Dev] Shorthand for lambda In-Reply-To: References: <20050323093710.DB60.JCARLSON@uci.edu> Message-ID: <20050323132347.DB66.JCARLSON@uci.edu> Ka-Ping Yee wrote: > > On Wed, 23 Mar 2005, Josiah Carlson wrote: > > Paul Moore wrote: > > > I thought about it, but couldn't convince myself that it would work > > > properly in all cases. I was thinking in terms of operator overloading > > > of everything possible - how did you do it? > > > > PyTables allows something very similar for "in-kernel" searches of data, > > but only on a single constraint. I would imagine that Ka-Ping did it by > > only allowing a single operation per item. > > You can do more than one operation, but the usage is still quite limited. > The item placeholder must be the first operand for it to work. > > >>> numbers = [3, 8, 4, 1, 2] > >>> filter(_ < 5, numbers) > [3, 4, 1, 2] > >>> map(_ * 5 + 7, numbers) > [10, 15, 11, 8, 9] Your implementation "works" by not crashing, but that last map should certainly return [22, 47, 27, 12, 17] rather than what it does. It may be extensible to actually work the way I expect such operations to work. - Josiah From florian.proff.schulze at gmx.net Wed Mar 23 22:58:51 2005 From: florian.proff.schulze at gmx.net (Florian Schulze) Date: Wed Mar 23 22:59:16 2005 Subject: [Python-Dev] Re: Re: Ye don't be needin' these! References: <9613db60050323105973d50748@mail.gmail.com> Message-ID: On Wed, 23 Mar 2005 13:59:29 -0500, Tim Lesher wrote: > On Wed, 23 Mar 2005 11:34:09 -0500, Terry Reedy wrote: >> >> "Herman Toothrot" wrote in message >> news:BAY23-F5C463CC802BE7A7513435BB4F0@phx.gbl... >> > Avast! Why be there builtins divmod and pow, when operators **, /, >> and % >> > should be good enough for ya? It runs counter to TOOWTDI, I be >> thinking. >> >> Questions like this should be asked on comp.lang.python or the python >> mailing list. I'll answer if I see it there. > > I have to wonder if this wasn't a tongue-in-cheek message sent from a > just-created hotmail account, by an existing python-dev participant > who's embroiled in the current "do we even need functionals anymore" > discussion... > BTW, Herman Toothrot is from Monkey Island. Regards, Florian Schulze From tlesher at gmail.com Wed Mar 23 23:06:40 2005 From: tlesher at gmail.com (Tim Lesher) Date: Wed Mar 23 23:06:43 2005 Subject: [Python-Dev] Re: Re: Ye don't be needin' these! In-Reply-To: References: <9613db60050323105973d50748@mail.gmail.com> Message-ID: <9613db60050323140666738021@mail.gmail.com> On Wed, 23 Mar 2005 22:58:51 +0100, Florian Schulze wrote: > BTW, Herman Toothrot is from Monkey Island. Right. That's what leads me to believe 1) it's not a serious post, and 2) it's from someone who's old enough to know better. -- Tim Lesher From fdrake at acm.org Thu Mar 24 00:39:41 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu Mar 24 00:39:48 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323124020.GA9693@phd.pp.ru> References: <20050323124020.GA9693@phd.pp.ru> Message-ID: <200503231839.42006.fdrake@acm.org> On Wednesday 23 March 2005 07:40, Oleg Broytmann wrote: > While I'm working on webbrowser... Why do all graphical browsers are > called with their stdout/stderr redirected to /dev/null? Do we really > need to hide problems from the user? Browsers are usually silent beasts > - they interact with the user using windows, not stdio. I don't remember why I did that; feel free to remove it. If it's actually useful, then 1) it should turn up before 2.5 final anyway, 2) we really want to know why, even if it just turns into a code comment, and 3) it should probably be controllable via the API, if its useful at all. -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Thu Mar 24 00:45:11 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu Mar 24 00:45:15 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323102512.0000605e@Fenix> References: <20050323124020.GA9693@phd.pp.ru> <20050323102512.0000605e@Fenix> Message-ID: <200503231845.11694.fdrake@acm.org> On Wednesday 23 March 2005 08:25, Rodrigo Dias Arruda Senra wrote: > Under some linux distros (I'm positive for some Mdk releases), Mozilla is > compiled dumping a lot of info to stdout/stderr. Since one of the goals of > webbrowser is to give the end-user a stress-free experience, there goes > the mentioned nullification . This sounds familliar. This was certainly true when Mozilla was young and I actually wrote the webbrowser module. (Or was that, when Grail was young? I don't even remember if there was a Mozilla for the first version!) > In a development environment, a developer should not find difficulty to > reverse that if needed. Right. I think if the API provides a control for this and some mention is made in the documentation, that would be good. -Fred -- Fred L. Drake, Jr. From gward at python.net Thu Mar 24 01:53:52 2005 From: gward at python.net (Greg Ward) Date: Thu Mar 24 01:53:55 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <20050323124020.GA9693@phd.pp.ru> References: <20050323124020.GA9693@phd.pp.ru> Message-ID: <20050324005352.GA3624@cthulhu.gerg.ca> On 23 March 2005, Oleg Broytmann said: > I'd like to remove all those redirects. Any opinion? +0.5. The beauty of Python is that it generally provides thin wrappers: when writing a convenient wrapper, it's OK to expose the underlying beast, warts and all. (I had a minor epiphany about this recently when digging into ossaudiodev again: turns out that certain ioctls are implemented subtly differently in OSS and in ALSA's OSS emulation layer. ossaudiodev, as it turns out, faithfully mirrors this inconsistency to Python programmers. The inconsistency is not ossaudiodev's fault, so it's not ossaudiodev's problem to fix it. Python programmers should have access to everything that C programmers have access to, only with less typing. If that means they have to worry about Mozilla dumping lots of text to stdout, or ALSA implementing certain ioctls differently than OSS, so be it.) (But, oh yeah: +1 to Fred's suggestion of making redirection controllable. Something like this: default should be no redirection, programmer should be allowed to specify what to do with stdout/stderr of GUI browsers.) Greg -- Greg Ward http://www.gerg.ca/ If at first you don't succeed, give up--no use making a damn fool of yourself. From jcarlson at uci.edu Thu Mar 24 02:21:39 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Mar 24 02:23:27 2005 Subject: [Python-Dev] Re: Re: Ye don't be needin' these! In-Reply-To: <9613db60050323140666738021@mail.gmail.com> References: <9613db60050323140666738021@mail.gmail.com> Message-ID: <20050323172040.DB69.JCARLSON@uci.edu> Tim Lesher wrote: > > On Wed, 23 Mar 2005 22:58:51 +0100, Florian Schulze > wrote: > > BTW, Herman Toothrot is from Monkey Island. > > Right. That's what leads me to believe 1) it's not a serious post, > and 2) it's from someone who's old enough to know better. I thought the pirate talk plus scurvy reference was sufficient for that (I didn't have the opportunity to play Monkey Island when I was a child). - Josiah From bo at thorsen-consulting.dk Mon Mar 21 10:54:49 2005 From: bo at thorsen-consulting.dk (Bo Thorsen) Date: Thu Mar 24 03:40:51 2005 Subject: [Python-Dev] C API for the bool type? Message-ID: <200503211054.49142.bo@thorsen-consulting.dk> Hi people, If this is not the correct place to post this problem, I apologize. In that case, please be gentle and point me to a better mailing list. I'm coding a text editor in Qt that uses Python for macros. The problem I have is that want to use the bool type introduced in 2.3, but I can't see how to do this. On http://docs.python.org/api/arg-parsing.html the format units are described, but no bool is there. I guess there are two possibilities. Either the documentation is not updated with the new format unit, or it doesn't exist. If it doesn't exist, I guess I should use the int format unit and call the http://docs.python.org/api/boolObjects.html functions to see if this is actually a bool or not? I hope you can help me with this question. Please CC me with answers, since I am not a member of this list. Thanks, Bo Thorsen. -- Thorsen Consulting ApS - Qt programming services http://www.thorsen-consulting.dk From nicksjacobson at yahoo.com Mon Mar 21 19:58:30 2005 From: nicksjacobson at yahoo.com (Nicholas Jacobson) Date: Thu Mar 24 03:40:53 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: 6667 Message-ID: <20050321185830.28881.qmail@web53908.mail.yahoo.com> Oops, you're right. What I should have said is to use a blank docstring as follows: """""" """Function docstring.""" def foo: ... or: """Module docstring.""" """""" def foo: ... --- Anthony Baxter wrote: > On Monday 21 March 2005 20:08, Nicholas Jacobson > wrote: > > > How do you distinguish between a docstring at > the > > > top of a module > > > that's immediately followed by a function? Is > it > > > the module docstring > > > or the function docstring? > > > > It's both. The docstring would be assigned to > both > > the module and the function. This is a *good* > thing > > when there is a module with only one function in > it. > > i.e. there should only be one docstring for both, > and > > this saves repetition of that docstring. > > > > If a programmer wanted a docstring for the > function > > but not the module, a blank first line would do > the > > trick. A docstring for the module but not the > > function? Put a blank line between the module's > > docstring and the function. > > Yuk. This is magic taken to a ridiculous level. Note > that > "blank lines" currently have no meaning in Python, > and adding > a meaning to them is not my idea of a good thing. > > -- > Anthony Baxter > It's never too late to have a happy childhood. > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From jafo at tummy.com Wed Mar 23 22:21:50 2005 From: jafo at tummy.com (Sean Reifschneider) Date: Thu Mar 24 03:40:56 2005 Subject: [Python-Dev] bdist_deb checkin comments In-Reply-To: <87acozp6ib.fsf@hydra.bayview.thirdcreek.com> References: <20050319185020.GB13774@tummy.com> <87acozp6ib.fsf@hydra.bayview.thirdcreek.com> Message-ID: <20050323212150.GA32523@tummy.com> On Sat, Mar 19, 2005 at 06:20:44PM -0500, Kurt B. Kaiser wrote: >Sean Reifschneider writes: >> Does anyone have any feedback on this before I do so? > >I made a few comments on the Tracker. Thanks a lot, they look great. I'll try to get the submitter to follow up on it. Sean -- Sure I like country music, and I like violins. But right now I need a telecaster through a vibrolux turned up to ten. Sean Reifschneider, Member of Technical Staff tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin From anthony at interlink.com.au Thu Mar 24 03:46:44 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Mar 24 03:47:05 2005 Subject: [Python-Dev] docstring before function declaration In-Reply-To: <20050321185830.28881.qmail@web53908.mail.yahoo.com> References: <20050321185830.28881.qmail@web53908.mail.yahoo.com> Message-ID: <200503241346.46108.anthony@interlink.com.au> On Tuesday 22 March 2005 05:58, Nicholas Jacobson wrote: > Oops, you're right. > > What I should have said is to use a blank docstring as > follows: It's still unclear to me what a file containing a single docstring followed by a def() line means. And this ambiguity doesn't seem to be solvable, so I'm a solid -1 on this change. (In addition, I should note that I tried editing a moderate sized file to put the docstrings before the defs - to my eyes, it made the file more cluttered and much less pleasing to the eye) -- Anthony Baxter It's never too late to have a happy childhood. From tjreedy at udel.edu Thu Mar 24 06:02:39 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Thu Mar 24 06:02:48 2005 Subject: [Python-Dev] Re: C API for the bool type? References: <200503211054.49142.bo@thorsen-consulting.dk> Message-ID: "Bo Thorsen" wrote in message news:200503211054.49142.bo@thorsen-consulting.dk... > If this is not the correct place to post this problem, I apologize. In > that case, please be gentle and point me to a better mailing list. The general Python mailing list (pyrhon-list ?) also at python.org. Or comp.lang.python (the two are gated to each other). Or gmane.comp.python.general. Or google groups. From phd at mail2.phd.pp.ru Thu Mar 24 13:25:03 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu Mar 24 13:26:05 2005 Subject: [Python-Dev] webbrowser.py: browser >/dev/null 2>&1 In-Reply-To: <200503231839.42006.fdrake@acm.org> References: <20050323124020.GA9693@phd.pp.ru> <200503231839.42006.fdrake@acm.org> Message-ID: <20050324122503.GB26943@phd.pp.ru> On Wed, Mar 23, 2005 at 06:39:41PM -0500, Fred L. Drake, Jr. wrote: > On Wednesday 23 March 2005 07:40, Oleg Broytmann wrote: > > While I'm working on webbrowser... Why do all graphical browsers are > > called with their stdout/stderr redirected to /dev/null? Do we really > > need to hide problems from the user? Browsers are usually silent beasts > > - they interact with the user using windows, not stdio. > > I don't remember why I did that I found one reason. For browsers that support remote commands the module runs two command - one to try to run the remote command, and another to run the browser in non-remote mode if the first command fails (there is no remote browser). If the remote command fails it reports the problem on stderr, but that's not an error from the module point of view, so the module hides it. But for the second command - browser + URL in non-remote mode - the module should not hide errors. I'll fix that. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From phd at mail2.phd.pp.ru Thu Mar 24 14:30:35 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu Mar 24 14:30:39 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050323182930.GB1084@phd.pp.ru> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> Message-ID: <20050324133035.GB29041@phd.pp.ru> On Wed, Mar 23, 2005 at 09:29:30PM +0300, Oleg Broytmann wrote: > I also added elinks support - currently it is very similar to links, > but I am going to extend its remote capabilities. (Yes, that small > text-mode broswer supports remoting, windows and tabs! Who'd think?!.) > Also I'm going to add "new-tab" support similar to "new-window" for > Mozilla/Firefox and elinks. I've reworked the patch once more. I moved some common functionality into the UnixBrowser class and added two new features - Elinks launcher class (elinks supports remote commands in a manner very similar to Mozilla) and new-tab functionality for browsers that support tabbed browsing (Mozilla and elinks); a user can now run "webbrowser -t URL" to open the URL a new tab. All classes in the module are now new-style classes (except for the Error class). All .open() methods accept "new" parameter: 0 - open the url in the same window, 1 - open in a new window (0 and 1 codes works as before), and new code 2 - open in a new tab if possible. These are probably the last features I wanted to add to the module. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From hyeshik at gmail.com Thu Mar 24 15:01:14 2005 From: hyeshik at gmail.com (Hye-Shik Chang) Date: Thu Mar 24 15:01:17 2005 Subject: [Python-Dev] Shorthand for lambda In-Reply-To: References: Message-ID: <4f0b69dc0503240601458098b6@mail.gmail.com> On Wed, 23 Mar 2005 10:33:53 -0600 (CST), Ka-Ping Yee wrote: > Hey folks, > > >>> from placeholder import _ > >>> numbers = [5, 9, 56, 34, 1, 24, 37, 89] > >>> filter(_ < 30, numbers) > [5, 9, 1, 24] > >>> map(_ + 10, numbers) > [15, 19, 66, 44, 11, 34, 47, 99] > >>> > > Look ma, no lambdas! > > I bet someone has already done this before, right? > I tried it once before: ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/local-distfiles/perky/anonfunc-1.0.tar.gz >>> from anonfunc import * >>> f = (arg0 * arg1 + 3 * arg2) ** 2 >>> f(5, 6, 7) 2601 >>> f = kw1 and kw2 >>> f(kw1=False, kw2=True) True >>> f(kw2=False, kw1=True) False But there were some serious drawbacks to use it instead of lambda. * (a,b,c) is impossible * unable to use it with list comprehensions and generator expressions * can't call functions and use its return value * and many builtin functions as you described (such as len(), sorted()) Hye-Shik From rodsenra at gpr.com.br Thu Mar 24 15:11:11 2005 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Thu Mar 24 15:11:16 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050324133035.GB29041@phd.pp.ru> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> Message-ID: <20050324111111.16812549@localhost.localdomain> On Thu, 24 Mar 2005 16:30:35 +0300 Oleg Broytmann wrote: > I've reworked the patch once more. I moved some common functionality > into the UnixBrowser class and added two new features ... I do not want to abuse your generosity. However, if you could make the necessary changes to the documentation, since it will be easier to you than anyone else right know, I'll gladly check the consistency between code and docs. tchau, Senra -- Rodrigo Senra rsenra _at_ acm _dot_ org http://www.ic.unicamp.br/~921234 From phd at mail2.phd.pp.ru Thu Mar 24 15:13:30 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu Mar 24 15:13:33 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050324111111.16812549@localhost.localdomain> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> Message-ID: <20050324141330.GF30864@phd.pp.ru> On Thu, Mar 24, 2005 at 11:11:11AM -0300, Rodrigo Dias Arruda Senra wrote: > However, if you could make the necessary changes to the documentation, I'll try, but certainly not in TeX format... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From rodsenra at gpr.com.br Thu Mar 24 15:36:41 2005 From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra) Date: Thu Mar 24 15:36:47 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050324141330.GF30864@phd.pp.ru> References: <423B4071.90105@gpr.com.br> <423B5780.1040401@v.loewis.de> <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> <20050324141330.GF30864@phd.pp.ru> Message-ID: <20050324113641.297a4a18@localhost.localdomain> On Thu, 24 Mar 2005 17:13:30 +0300 Oleg Broytmann wrote: > On Thu, Mar 24, 2005 at 11:11:11AM -0300, Rodrigo Dias Arruda Senra wrote: > > However, if you could make the necessary changes to the documentation, > > I'll try, but certainly not in TeX format... Edit libwebbrowser.tex as you see fit, then send it to me and I'll TeXify it back to you. tchau, Rod -- Rodrigo Senra MSc Computer Engineer rodsenra@gpr.com.br GPr Sistemas Ltda http://www.gpr.com.br From fdrake at acm.org Thu Mar 24 20:00:19 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu Mar 24 20:00:25 2005 Subject: [Python-Dev] C API for the bool type? In-Reply-To: <200503211054.49142.bo@thorsen-consulting.dk> References: <200503211054.49142.bo@thorsen-consulting.dk> Message-ID: <200503241400.19600.fdrake@acm.org> On Monday 21 March 2005 04:54, Bo Thorsen wrote: > If this is not the correct place to post this problem, I apologize. In > that case, please be gentle and point me to a better mailing list. You should be able to get answers to this kind of question on comp.lang.python (aka python-list at python.org). But I'll give you this one for free. :-) > I'm coding a text editor in Qt that uses Python for macros. The problem I > have is that want to use the bool type introduced in 2.3, but I can't see > how to do this. On http://docs.python.org/api/arg-parsing.html the format > units are described, but no bool is there. There is no format character for boolean, and you don't actually need one. Recall that in Python, all objects have some interpretation as a truth value. The easiest way to check this from C code is to pass the object to PyObject_IsTrue() or PyObject_Not() (depending on the sense of the test you want, and how you want your code to read). The argument can be retrieved in your PyArg_ParseTuple*() call using the "O" format character. > If it doesn't exist, I guess I should use the int format unit and call the > http://docs.python.org/api/boolObjects.html functions to see if this is > actually a bool or not? In most cases, you really don't care if the object is actually a bool. The recipe above will also work in older versions of Python. That lets you use bools for the reason they were really introduced: to enhance readability. -Fred -- Fred L. Drake, Jr. From jimjjewett at gmail.com Thu Mar 24 22:11:11 2005 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu Mar 24 22:11:13 2005 Subject: [Python-Dev] @decoration of classes Message-ID: Josiah Carlson: > I just noticed that decoration of classes was not included with the > @decoration syntax that made it into Python 2.4. ... > Is the fact that it didn't make it into 2.4 due to a pronouncement Yes, but it wasn't a permanent decision, and I don't think he used the magic word "pronounce". Adding class decoration later is not difficult. Retracting class decoration later would have been impossible. Therefore Guido decided for version 2.4, but left the question open for the unspecified future. Changing it would take arguments stronger than symmetry or "why not?", so it probably won't change for 2.5, but if you have a compelling use case ... (Mine have all been using an object as a callable -- basically replacing a function. The itch hasn't been strong enough to bother me.) -jJ From jcarlson at uci.edu Thu Mar 24 23:55:49 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Mar 24 23:57:12 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: References: Message-ID: <20050324142035.DB6C.JCARLSON@uci.edu> Jim Jewett wrote: > > Josiah Carlson: > > > I just noticed that decoration of classes was not included with the > > @decoration syntax that made it into Python 2.4. ... > > > Is the fact that it didn't make it into 2.4 due to a pronouncement > > Yes, but it wasn't a permanent decision, and I don't think > he used the magic word "pronounce". > > Adding class decoration later is not difficult. > > Retracting class decoration later would have been impossible. > > Therefore Guido decided for version 2.4, but left the question > open for the unspecified future. Changing it would take > arguments stronger than symmetry or "why not?", so it > probably won't change for 2.5, but if you have a compelling > use case ... > > (Mine have all been using an object as a callable -- basically > replacing a function. The itch hasn't been strong enough to > bother me.) I think of it like this; I learned restructured text because I wanted to learn a markup for writing things that didn't encumber the writing. At a certain point, it became necessary for me to learn TeX. Does that mean that reST has no use to me now that I know TeX? Of course not, reST still has its place, and when I write documentation, I write it in reST. When I begin to write academic papers, I write it in reST, then later translate it to TeX, because that is the language of typeset academic writings. I could write it first in TeX, but the notation can get cumbersome, and reST is still valid (whether or not it survives the decade; I still have the source, a python interpreter, and an emulator). While I have not used it often, I have done the equivalent of decorating classes; it is as natural (though perhaps not quite as useful initially) as decorating functions, and as simple as writing in reST. I also happen to know a bit about using metaclasses, though tend to shy away from it, because it feels cumbersome. Can you do everything with TeX that you can with reST? Of course. Does it make reST any less valid or desireable to use? Of course not. Can you use a metaclass to do anything you could do with class decorators? Probably (I don't know the corner cases that would make it not so). Does that mean that class decoration is any less valid or desireable to use? Of course not. Does that mean that it should have a syntax? That is the question. I personally choose class decoration over metaclasses (when possible) because I feel it is easier to use. However, as my use case has not been compelling enough in the past, I is certainly not compelling enough now. If someone cares more about this, who has more influence and wants to continue the conversation; feel free, but I'll still use Python tomorrow without class decoration syntax. - Josiah P.S. You know, it's kind of like putting sprinkles on a banana split. Some people like the sprinkles, some people don't. Some people complain when they are there, some people can't eat it without the sprinkles. But you know what? You've got the banana split already, enjoy it. If you get the sprinkles, so be it, but hassling the waitress for sprinkles is a pretty rude thing to do to a woman in an ice cream parlor. From oliphant at ee.byu.edu Sat Mar 26 00:13:43 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Sat Mar 26 00:13:52 2005 Subject: [Python-Dev] Using descriptors to dynamically attach methods written in Python to C-defined (new-style) types Message-ID: <42449B27.90500@ee.byu.edu> In updating Numeric to take advantage of the new features in Python, I've come across the need to attach a Python-written function as a method to a C-builtin. I don't want to inherit, I just want to extend the methods of a builtin type using a Python function. I was thinking of updating the new type objects dictionary with a new entry that is a descriptor object. It seems that the descriptor mechanism makes this a relatively straightforward thing. My question is, can I use the already-available Descriptor objects to do this, or will I need to define another Descriptor object. (Perhaps a PythonMethod descriptor object to complement the Method Descriptor). Any hints will be helpful. -Travis Oliphant From bob at redivi.com Sat Mar 26 01:39:46 2005 From: bob at redivi.com (Bob Ippolito) Date: Sat Mar 26 01:39:47 2005 Subject: [Python-Dev] Using descriptors to dynamically attach methods written in Python to C-defined (new-style) types In-Reply-To: <42449B27.90500@ee.byu.edu> References: <42449B27.90500@ee.byu.edu> Message-ID: <59eab01a072d2a9177cdb5b6171fcd5e@redivi.com> On Mar 25, 2005, at 6:13 PM, Travis Oliphant wrote: > In updating Numeric to take advantage of the new features in Python, > I've come across the need > to attach a Python-written function as a method to a C-builtin. I > don't want to inherit, I just want to extend the methods of a builtin > type using a Python function. I was thinking of updating the new > type objects dictionary with a new entry that is a descriptor object. You probably do want to inherit at least once from Python, see below. > It seems that the descriptor mechanism makes this a relatively > straightforward thing. My question is, can I use the > already-available Descriptor objects to do this, or will I need to > define another Descriptor object. (Perhaps a PythonMethod descriptor > object to complement the Method Descriptor). > Any hints will be helpful. If the type object does have a mutable dictionary (not a read-only dictproxy), you're pretty much already done. Functions implement the descriptor protocol, presumably even in the way you want them to. Meaning, if you just stick a function into a (new style) class dict, when you fetch it from an instance you'll end up with a bound method. When you fetch it from the class, you end up with an unbound method. There's no noticeable difference between if you did this, or if the class had the function when it was defined. Functions (aka methods) defined in a class body are *nothing special*. However, by default, extension types don't have a dict and the easiest way to get one is just to subclass it from Python. So, what you can do is simply not make the C types part of the public API, use subclasses of them as the API for the classes that you need/want to be extensible in this manner. This is the reason you can't do "object.foo = 1" but you CAN do it to any Python subclass of object (unless you explicitly disallow it by way of metaclass or slots). -bob From khuranavivek_in at yahoo.com Sat Mar 26 04:34:05 2005 From: khuranavivek_in at yahoo.com (vivek khurana) Date: Sat Mar 26 04:34:09 2005 Subject: [Python-Dev] tree data structure and python Message-ID: <20050326033406.60712.qmail@web40614.mail.yahoo.com> Hi! all i am a new member on this list. I have to implement tree data structure using python. How it can be done in python. Is there an existing data structure which can be used as tree? I have searched archives and manuals but no luck. Regards VK Hug the REALITY ;-) Disclaimer The facts expressed here belong to everybody, the opinions to me. The distinction is yours to draw... __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From tjreedy at udel.edu Sat Mar 26 06:12:44 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Mar 26 06:13:15 2005 Subject: [Python-Dev] Re: tree data structure and python References: <20050326033406.60712.qmail@web40614.mail.yahoo.com> Message-ID: "vivek khurana" wrote in message news:20050326033406.60712.qmail@web40614.mail.yahoo.com... > i am a new member on this list. I have to implement > tree data structure using python. How it can be done > in python. Is there an existing data structure which > can be used as tree? I have searched archives and > manuals but no luck. As the name hints, this list is for developing Python. Usage questions such as the above should be directed to the general python mailing list or comp.lang.python, each of which are gated to each other. TJR From skip at pobox.com Sat Mar 26 13:16:53 2005 From: skip at pobox.com (Skip Montanaro) Date: Sat Mar 26 13:16:57 2005 Subject: [Python-Dev] tree data structure and python In-Reply-To: <20050326033406.60712.qmail@web40614.mail.yahoo.com> References: <20050326033406.60712.qmail@web40614.mail.yahoo.com> Message-ID: <16965.21173.304305.220605@montanaro.dyndns.org> vivek> I have to implement tree data structure using python. How it can vivek> be done in python. Wrong list. This is about development *of* Python, not development *with* Python. Try python-list@python.org (or its sister Usenet newsgroup, comp.lang.python) instead. -- Skip Montanaro skip@pobox.com From gvanrossum at gmail.com Sat Mar 26 18:00:47 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Sat Mar 26 18:00:50 2005 Subject: [Python-Dev] FYI: news items about Burton Report on P-languages In-Reply-To: <1111793792.158987.60d394253ade5ce0.5c449b4e@persist.google.com> References: <1111793792.158987.60d394253ade5ce0.5c449b4e@persist.google.com> Message-ID: ---------- Forwarded message ---------- From: Google Alerts Date: Fri, 25 Mar 2005 15:36:32 -0800 (PST) Subject: Google Alert - python -spamalot -monty To: gvanrossum@gmail.com Google Alert for: python -spamalot -monty Report: P-Languages Better For Enterprise InternetNews.com - USA PHP (define), Perl (define), and Python (define) (the P-Languages) have really come into their own over the last four or five years because of their ability to ... See all stories on this topic Perl, Python and PHP get the nod for Enterprise. HTML FIX IT.COM - Grand Rapids,MI,USA Perl Python and PHP are all Open Source programming languages and there is a ton of free software written in all of those languages available online. ... Report: P-Languages Better For Enterprise (InternetNews) ACM Queue - New York,NY,USA PHP, Perl and Python are mission-critical ready and more effective than C++, Java, and C# in some scripting scenarios, according to Burton Group. ... ________________________________ This once a day Google Alert is brought to you by Google. Remove this alert. Create another alert. Manage your alerts. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric.nieuwland at xs4all.nl Sat Mar 26 18:30:54 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Sat Mar 26 18:31:09 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050324142035.DB6C.JCARLSON@uci.edu> References: <20050324142035.DB6C.JCARLSON@uci.edu> Message-ID: Given the ideas so far, would it possible to: def meta(cls): ... @meta class X(...): ... ?? --eric From jcarlson at uci.edu Sat Mar 26 21:36:08 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat Mar 26 21:38:36 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: References: <20050324142035.DB6C.JCARLSON@uci.edu> Message-ID: <20050326122905.DB7D.JCARLSON@uci.edu> Eric Nieuwland wrote: > > Given the ideas so far, would it possible to: > > def meta(cls): > ... > > @meta > class X(...): > ... It is not implemented in Python 2.4. From what I understand, making it happen in Python 2.5 would not be terribly difficult. The question is about a "compelling use case". Is there a use where this syntax is significantly better, easier, etc., than an equivalent metaclass? Would people use the above syntax if it were available? What would you use the above syntax to do? - Josiah From eric.nieuwland at xs4all.nl Sat Mar 26 22:49:33 2005 From: eric.nieuwland at xs4all.nl (Eric Nieuwland) Date: Sat Mar 26 22:49:41 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050326122905.DB7D.JCARLSON@uci.edu> References: <20050324142035.DB6C.JCARLSON@uci.edu> <20050326122905.DB7D.JCARLSON@uci.edu> Message-ID: <4138a63e5b65e7289054b0519991aa0e@xs4all.nl> On 26 mrt 2005, at 21:36, Josiah Carlson wrote: > Eric Nieuwland wrote: >> Given the ideas so far, would it possible to: >> >> def meta(cls): >> ... >> >> @meta >> class X(...): >> ... > > It is not implemented in Python 2.4. From what I understand, making it > happen in Python 2.5 would not be terribly difficult. The question is > about a "compelling use case". Is there a use where this syntax is > significantly better, easier, etc., than an equivalent metaclass? > Would > people use the above syntax if it were available? > > What would you use the above syntax to do? Well, I can imagine using @meta(MyMetaClass) class MyClass(...): ... instead of class MyClass(...): __metaclass__ = MyMetaClass ... Somehow, it seems more aesthetic to me. --eric From exarkun at divmod.com Sat Mar 26 23:16:49 2005 From: exarkun at divmod.com (Jp Calderone) Date: Sat Mar 26 23:16:58 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <4138a63e5b65e7289054b0519991aa0e@xs4all.nl> Message-ID: <20050326221649.13806.1670758735.divmod.quotient.25896@ohm> On Sat, 26 Mar 2005 22:49:33 +0100, Eric Nieuwland wrote: >On 26 mrt 2005, at 21:36, Josiah Carlson wrote: > > Eric Nieuwland wrote: > >> Given the ideas so far, would it possible to: > >> > >> def meta(cls): > >> ... > >> > >> @meta > >> class X(...): > >> ... > > > > It is not implemented in Python 2.4. From what I understand, making it > > happen in Python 2.5 would not be terribly difficult. The question is > > about a "compelling use case". Is there a use where this syntax is > > significantly better, easier, etc., than an equivalent metaclass? > > Would > > people use the above syntax if it were available? > > > > What would you use the above syntax to do? > > Well, I can imagine using > > @meta(MyMetaClass) > class MyClass(...): > ... > > instead of > > class MyClass(...): > __metaclass__ = MyMetaClass > ... > > Somehow, it seems more aesthetic to me. This doesn't quite work the same, though. The former creates a new instance of ClassType, then (presumably) rips it apart and passes the pieces to MyMetaClass. The latter just passes the pieces to MyMetaClass unassembled. I can imagine cases where the class creation would fail during the first step of the former process, so I don't think this is actually a use-case for class decorators. Jp From mdehoon at ims.u-tokyo.ac.jp Sun Mar 27 10:16:28 2005 From: mdehoon at ims.u-tokyo.ac.jp (Michiel Jan Laurens de Hoon) Date: Sun Mar 27 10:11:47 2005 Subject: [Python-Dev] A bunch of patch reviews Message-ID: <42466BDC.3090209@ims.u-tokyo.ac.jp> Below are a set of five patch reviews. I don't have any patch to push for at this point, so these patch reviews are just for you to read and enjoy. Thanks everybody for developing and maintaining Python. I wouldn't know what to do without it. --Michiel. Patch [ 853890 ] Optional keyword unicode args not handled correctly The skipitem function in Python/getargs.c misses code for several argument formats. This patch solves one of them ('u' and 'u#'), patch 985713 solves another one ('w'). There are several more that need to be solved. I've asked the patch contributor to write a complete patch, including all missing formats. This patch is rather straightforward and solves a serious problem, so I'd recommend accepting it once it is complete. Patch [ 1167316 ] a fix for doctest crash when it is ran on itself doctest.py in Lib fails when it is run on itself. The error is due to missing "import doctest" statements and similar small problems in the doctest docstrings. Patch seems to work correctly. Patch [ 1166780 ] Fix _tryorder in webbrowser.py In webbrowser.py, a user can override the default list of web browsers to be opened by setting the BROWSER variable. Currently, if BROWSER is set, the default list is not used at all. The patch contributor notes that if BROWSER is set incorrectly, this may result in no browser being opened at all. While this is true, seeing a different browser open instead of the one specified in BROWSER may lead to confusion, and may lead to future bug reports saying "webbrowser.py ignores the BROWSER variable" if a user sets BROWSER incorrectly. So my suggestion is to at least print a warning message if the browser specified in BROWSER cannot be used, and then proceed by opening one of the default browsers. Patch [ 764221 ] add set/getattr interface to tkFont.Font objects This patch has already made it into Python2.4, so I think this patch can be closed. Patch [ 1163314 ] the quotes page on the Web site links to something broken The patch writer is correct that the link is broken, but it's not a Python bug. I've suggested him to write to the web master. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon From martin at v.loewis.de Sun Mar 27 17:21:38 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Mar 27 18:21:39 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <42414E0E.60808@livinglogic.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> Message-ID: <4246CF82.2060500@v.loewis.de> Walter D?rwald wrote: > I'm not sure if this is the right approach. I think the approach is right, but the implementation is wrong. > The encoding I specify in > setup.py should be independent of the encoding used between distutils > and PyPI to communicate on the wire. I.e. the author (and maintainer) > argument should always be unicode. "should" is a correct description. It should allow Unicode strings, which it then should encode to UTF-8 during transmission. The matter of fact is that the register command as released in 2.4 (and 2.4.1) doesn't. > When str is passed, this is treated > as any other str in a unicode context, it is decoded using the default > encoding. This would fix another problem: It would make it nearly > impossible to send a request to PyPI with the wrong encoding, because > any encoding problems are sorted out completely on the client side. distutils should *not* assume that byte strings are in the default encoding. It is fair to assume they are in ASCII; if the administrator has changed the default encoding, then this cannot possibly affect all the setup.py files out there. Also, it is a fact that the deployed versions of the register command just send byte strings in setup.py as-is, without trying to do any kind of recoding. In any case, PyPI now requires that the form submission uses UTF-8, and refuses anything else. So it *is* impossible to send, say, Latin-1; whether the client makes that happen by properly encoding Unicode strings or whether they are in setup.py in the first place does not matter. > Is there a way to display the HTTP response by PyPI? Yes, please invoke upload with --show-response. > Editing the package is still broken. The link "edit" on the page > http://www.python.org/pypi/ll-ansistyle/0.6.1 gives: > --- > Error... > > There's been a problem with your request > > exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in > position 92: ordinal not in range(128) I see. I'll investigate. Martin From bac at OCF.Berkeley.EDU Mon Mar 28 02:31:13 2005 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Mar 28 02:31:22 2005 Subject: [Python-Dev] Deprecating 2.2 old bugs In-Reply-To: References: Message-ID: <42475051.1000901@ocf.berkeley.edu> Facundo Batista wrote: > Going on with the old bugs checking, here are the results for 2.2. > When I'll finish this will be put in an informational PEP. > I just want to publicly thank Facundo for doing this. I remember when I went through one weekend and did a ton of old bug reports; it ain't exactly fun. And he even spent his PyCon sprint time on this some more. So thanks from me for putting the time in on this. -Brett From andymac at bullseye.apana.org.au Mon Mar 28 13:41:09 2005 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Mon Mar 28 13:42:48 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python thread_pthread.h, 2.53, 2.53.4.1 In-Reply-To: References: Message-ID: <4247ED55.4060708@bullseye.apana.org.au> anthonybaxter@users.sourceforge.net wrote: >Update of /cvsroot/python/python/dist/src/Python >In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25872 > >Modified Files: > Tag: release24-maint > thread_pthread.h >Log Message: >Patch #1163249 - Correctly handle _POSIX_SEMAPHORES == -1 to mean no >support for posix semaphores. > > >Index: thread_pthread.h >=================================================================== >RCS file: /cvsroot/python/python/dist/src/Python/thread_pthread.h,v >retrieving revision 2.53 >retrieving revision 2.53.4.1 >diff -u -d -r2.53 -r2.53.4.1 >--- thread_pthread.h 7 Jul 2004 17:44:12 -0000 2.53 >+++ thread_pthread.h 16 Mar 2005 04:13:29 -0000 2.53.4.1 >@@ -16,9 +16,13 @@ > family of functions must indicate this by defining > _POSIX_SEMAPHORES. */ > #ifdef _POSIX_SEMAPHORES >+#if _POSIX_SEMAPHORES == -1 >+#define HAVE_BROKEN_POSIX_SEMAPHORES >+#else > #include > #include > #endif >+#endif > > #if !defined(pthread_attr_default) > # define pthread_attr_default ((pthread_attr_t *)NULL) > >_______________________________________________ >Python-checkins mailing list >Python-checkins@python.org >http://mail.python.org/mailman/listinfo/python-checkins > > This change has broken the build on FreeBSD 4.x for me: gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes - I. -I./Include -DPy_BUILD_CORE -o Python/thread.o Python/thread.c In file included from Python/thread.c:101: Python/thread_pthread.h:19: syntax error *** Error code 1 Backing it out allows the build to proceed & there are no unexpected test failures. Compiler is gcc 2.95.4. Regards, Andrew. ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From martin at v.loewis.de Mon Mar 28 14:42:39 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Mar 28 14:42:43 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python thread_pthread.h, 2.53, 2.53.4.1 In-Reply-To: <4247ED55.4060708@bullseye.apana.org.au> References: <4247ED55.4060708@bullseye.apana.org.au> Message-ID: <4247FBBF.4010608@v.loewis.de> Andrew MacIntyre wrote: > This change has broken the build on FreeBSD 4.x for me: > > gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall > -Wstrict-prototypes - > I. -I./Include -DPy_BUILD_CORE -o Python/thread.o Python/thread.c > In file included from Python/thread.c:101: > Python/thread_pthread.h:19: syntax error > *** Error code 1 This should be fixed now, please try again and report whether it works. It would be really nice if you could try to analyse such problems deeper in the future. In this case, it would have helped if you had reported that _POSIX_SEMAPHORES is defined as #define _POSIX_SEMAPHORES so that the #if line expands to #if == -1 The standard solution in this case is to write #if (_POSIX_SEMAPHORES+0) == -1 so that it expands to a binary add if _POSIX_SEMAPHORES really is a number (as it should be, according to POSIX); if some system incorrectly defines _POSIX_SEMAPHORES to empty, you get #if (+0) == -1 which still should compile. Regards, Martin From python at rcn.com Mon Mar 28 15:12:55 2005 From: python at rcn.com (Raymond Hettinger) Date: Mon Mar 28 15:17:34 2005 Subject: [Python-Dev] sum() In-Reply-To: <1f7befae0503121730568b118f@mail.gmail.com> Message-ID: <000401c53397$dc7a2d80$6ebb9d8d@oemcomputer> [Tim] > For contrast, here's one that doesn't use frexp(), and is probably > faster because of that; internally, len(sums) probably won't exceed 5 > in real life (but could get as large as 2K for pathological inputs, > spanning fp's full dynamic range): > > def summer4(iterable): > sums = [0.0] > for x in iterable: > sums.append(x) > for i in xrange(len(sums)-2, -1, -1): > y = sums[i] > if abs(x) < abs(y): > x, y = y, x > hi = x+y > lo = y - (hi - x) > if lo: > sums[i+1] = lo > else: > del sums[i+1] > x = hi > sums[0] = x > return sum(reversed(sums), 0.0) Here's a rewrite with the partial sums ordered by increasing magnitude. A cursor is used to write-out new, non-zero terms. That makes the list growth or shrinkage occur at the end of the list and it avoids reversed indexing. Those two changes made the code easier for me to follow. Leaving off the 0.0 makes the routine generic. It works equally well with int, long, Decimal, float, and complex. def summer5(iterable, start=0): "Binary or Decimal floating point summation accurate to full precision." partials = [] # sorted, non-overlapping partial sums for x in iterable: i = 0 # cursor for writing-out new partials sums for y in partials: if abs(x) < abs(y): x, y = y, x hi = x + y lo = y - (hi - x) if lo: partials[i] = lo i += 1 x = hi partials[i:] = [x] return sum(partials, start) The loop invariants for the list of partial sums are: assert partials == sorted(partials, key=abs) assert nonoverlapping(partials) where a rough check for overlaps is: def nonoverlapping(seqn, offset=100): """Verify that sequence elements have no overlapping bits. Set offset to -Emin to handle the full range of floats. """ cumv = 0L for elem in seqn: m, exp = frexp(abs(elem)) v = int(m * 2 ** 53) * 2L ** (exp + offset) if cumv & v: return False cumv |= v return True Raymond From mcherm at mcherm.com Mon Mar 28 19:25:18 2005 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Mar 28 19:25:21 2005 Subject: [Python-Dev] @decoration of classes Message-ID: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> Josiah Carlson writes: [... stuff about reST and TeX ...] > While I have not used it often, I have done the equivalent of decorating > classes; it is as natural (though perhaps not quite as useful initially) > as decorating functions, [... stuff about ice cream and sprinkles ...] Hmm... the only bit that I found particularly interesting there was the bit where you mention that you've used class decorators (without the syntax) before. What did you use them for? After all, the current state of things is "don't bother implementing class decorators because there is no compelling use case". If you've got some sample use cases, what were they? For my own part, I observe the following. Because a function decorator acts after the function object is created, there are limits to what it can do to the function. It can add some markup (eg: set properties or doc strings). It can "hook" either before or after the function is called. Or it can "veto" the function call and do something else instead. In the case of function calls, these are pretty much the interesting things you would want to do. Similarly, because a class decorator acts after the class is created there are limits on what it can do. It can modify the class object (replacing methods and such). It can add markup. It can replace the class with another (perhaps a proxy or some such). But most of these are things which are more easily done by a metaclass... and all of them *can* be done by metaclasses. The only noticable advantage that I see to class decorators over metaclasses is that there's a more straightforward way to combine them. And I'm not sure that combining metaclasses (or class decorators) is something I want to encourage. So I'm inclined to use different tools for modifying functions and modifying classes because the ways you want to modify them are different, and decorators are "tuned" to what people normally want to do with functions (like simple wrapping) while metaclasses are "tuned" to what people normally want to do with classes (like support for inheritance. But a few *good* use cases would change my mind. -- Michael Chermside From jcarlson at uci.edu Mon Mar 28 21:42:51 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon Mar 28 21:44:05 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> References: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> Message-ID: <20050328095535.DB80.JCARLSON@uci.edu> Michael Chermside wrote: > > Josiah Carlson writes: > > [... stuff about reST and TeX ...] > > While I have not used it often, I have done the equivalent of decorating > > classes; it is as natural (though perhaps not quite as useful initially) > > as decorating functions, > [... stuff about ice cream and sprinkles ...] > > Hmm... the only bit that I found particularly interesting there was the bit > where you mention that you've used class decorators (without the syntax) > before. > > What did you use them for? After all, the current state of things is "don't > bother implementing class decorators because there is no compelling use > case". If you've got some sample use cases, what were they? 99% of my use cases have been of the form of decorating all of the methods of a class at once (optionally excluding __init__ and __new__) using @sync or binding constants [1]. > For my own part, I observe the following. Because a function decorator > acts after the function object is created, there are limits to what it > can do to the function. It can add some markup (eg: set properties or > doc strings). It can "hook" either before or after the function is > called. Or it can "veto" the function call and do something else > instead. In the case of function calls, these are pretty much the > interesting things you would want to do. Unless one is willing to rewrite the bytecode; in which case things like Raymond's binding constants decorator or even the inline decorator (that I found and then lost) are possible and useful. > Similarly, because a class decorator acts after the class is created > there are limits on what it can do. It can modify the class object > (replacing methods and such). It can add markup. It can replace the > class with another (perhaps a proxy or some such). But most of these > are things which are more easily done by a metaclass... and all of > them *can* be done by metaclasses. The only noticable advantage that > I see to class decorators over metaclasses is that there's a more > straightforward way to combine them. And I'm not sure that combining > metaclasses (or class decorators) is something I want to encourage. Indeed, though people do combine metaclasses (or at least attempt to do so), often in strange, wonderful, and not so wonderful ways. I actually learned metaclasses because someone asked a question about combining them, and I wanted to understand the question (if not answer it). > So I'm inclined to use different tools for modifying functions and > modifying classes because the ways you want to modify them are > different, and decorators are "tuned" to what people normally want > to do with functions (like simple wrapping) while metaclasses are > "tuned" to what people normally want to do with classes (like support > for inheritance. While one can call what is being done with decorators "simple wrapping", I believe it goes a bit beyond that. With properly defined @accepts and @returns decorators, certainly one can perform runtime validation of call/return, but with a proper validation mechanism, one can do compile-time type inference and call type validation/verification (PyChecker with types, not just number of arguments). This kind of thing would give us the (desired by some) optional typing information for passed and returned arguments. Of course none of the above is a new idea, but I'm pointing it out so that we remember that there already exists a mechanism for doing this without further syntax changes to the language (someone once offered "def f(int:foo=3)" or some such, and this along with may other things are possible with decorators). As a side-note, I personally think of function/method decoration as a kind of subclassing of a function (as I have mentioned here before[2]); and if it were treated as such (with an attribute that references the previous function/method), one wouldn't need to copy __doc__, __name__, etc., attributes onto certain decorated functions. As for what most people use metaclasses for, I don't know, I try not to use metaclasses if possible (I haven't had a _compelling_ need so far), and don't use class decoration very often either. > But a few *good* use cases would change my mind. As I have said from the beginning, I don't believe any of my use cases are compelling, as I don't believe that sprinkles on a banana split are compelling. I also don't believe that someone is going to come forward with a compelling use case when compared against function/method decorators (like PyObjC wrapping, @accepts, @returns, @dispatch, @memoize, @synchronize, @classmethod, @staticmethod, ...), as I don't believe that even metaclasses are as compelling as function/method decoration. With that said; if it is there, people will use it. Someone will even post on their blog about how it is the best thing ever, or even how it ruined the language. So be it. In any case, I've spent more time writing emails about this particular topic than I will ever see returned by using class decoration syntax myself (over some equivalent method), so this is probably my last email on the subject. - Josiah [1] - Raymond's recipie for binding constants at compile time. A user also provides a metaclass version, but I prefer the class decoration. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940 [2] - [Python-Dev] decorator support http://mail.python.org/pipermail/python-dev/2004-September/048631.html From Scott.Daniels at Acm.Org Mon Mar 28 22:52:21 2005 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Mon Mar 28 22:53:19 2005 Subject: [Python-Dev] Re: @decoration of classes In-Reply-To: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> References: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> Message-ID: Michael Chermside wrote: > Josiah Carlson writes: >>...While I have not used it often, I have done the equivalent of decorating >>classes; it is as natural (though perhaps not quite as useful initially) >>as decorating functions, > > But a few *good* use cases would change my mind. Until this weekend, I really had no idea what a good use case for class decorators would look like. However, I stumbled upon something interesting. I was refactoring Kirby Urner's "hypertoons" code this weekend and hit upon an interesting use for decorators. On reflection, this might well be a candidate for class decorators. The unusual thing is that it does nothing to the decorated function; it simply stores it in a data structure. The "converter" decorator builder can be used for data values, including classes, but there is an advantage to having them at the top of the definition. Using such decorators looks like: @converter('inch', 'foot') def someconversion(... @converter('foot', 'yard') def anotherconversion(... @converter('yard', 'furlong') def yetanotherconversion(... Classes can be put into the data structures with: converter('flour', 'bread')(BakingClass) _But_ (at least for the app I was fiddling with) decorating at the top of declaration helps show the purpose of the class. Have a look at: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/393010 and see what you think. -- Scott David Daniels Scott.Daniels@Acm.Org From rsenra at acm.org Mon Mar 28 23:31:29 2005 From: rsenra at acm.org (Rodrigo Dias Arruda Senra) Date: Mon Mar 28 23:28:13 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050328114205.1508614b@localhost.localdomain> References: <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> <20050324141330.GF30864@phd.pp.ru> <20050324113641.297a4a18@localhost.localdomain> <20050328142735.GA3087@phd.pp.ru> <20050328114205.1508614b@localhost.localdomain> Message-ID: <20050328183129.75d6c8c8@Goku> | > On Thu, Mar 24, 2005 at 11:36:41AM -0300, Rodrigo Dias Arruda Senra wrote: | > > Edit libwebbrowser.tex as you see fit, then send it to me | > > and I'll TeXify it back to you. | > | > Uploaded to http://python.org/sf/754022 . I am not a native English | > speaker, and this is the first time I've edited a TeX file. Please | > correct my grammar, spelling, TeX, whatever... Outstanding work Oleg. I read it through and wouldn't change a bit. I have revised: libwebbrowser.tex.patch and webbrowser.py.patch. They are Ok regarding grammar, TeX and ... whatever . I recommend to apply both files. However, I would withdraw the third file -- webbrowser wrapper script, since the same functionality can be accomplished with: python -m webbrowser http://www.python.org best regards, Senra -- ,_ | ) Rodrigo Senra |(______ ----------------------------------------------- _( (|__|] GPr Sistemas http://www.gpr.com.br _ | (|___|] IC - Unicamp http://www.ic.unicamp.br/~921234 ___ (|__|] L___(|_|] ----------------------------------------------- From jack at performancedrivers.com Tue Mar 29 02:22:23 2005 From: jack at performancedrivers.com (Jack Diederich) Date: Tue Mar 29 02:22:27 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> References: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> Message-ID: <20050329002223.GH12663@performancedrivers.com> On Mon, Mar 28, 2005 at 09:25:18AM -0800, Michael Chermside wrote: > Josiah Carlson writes: > > [... stuff about reST and TeX ...] > > While I have not used it often, I have done the equivalent of decorating > > classes; it is as natural (though perhaps not quite as useful initially) > > as decorating functions, > [... stuff about ice cream and sprinkles ...] > > Hmm... the only bit that I found particularly interesting there was the bit > where you mention that you've used class decorators (without the syntax) > before. > > What did you use them for? After all, the current state of things is "don't > bother implementing class decorators because there is no compelling use > case". If you've got some sample use cases, what were they? > > For my own part, I observe the following. Because a function decorator > acts after the function object is created, there are limits to what it > can do to the function. It can add some markup (eg: set properties or > doc strings). It can "hook" either before or after the function is > called. Or it can "veto" the function call and do something else > instead. In the case of function calls, these are pretty much the > interesting things you would want to do. > > Similarly, because a class decorator acts after the class is created > there are limits on what it can do. It can modify the class object > (replacing methods and such). It can add markup. It can replace the > class with another (perhaps a proxy or some such). But most of these > are things which are more easily done by a metaclass... and all of > them *can* be done by metaclasses. The only noticable advantage that > I see to class decorators over metaclasses is that there's a more > straightforward way to combine them. And I'm not sure that combining > metaclasses (or class decorators) is something I want to encourage. > > So I'm inclined to use different tools for modifying functions and > modifying classes because the ways you want to modify them are > different, and decorators are "tuned" to what people normally want > to do with functions (like simple wrapping) while metaclasses are > "tuned" to what people normally want to do with classes (like support > for inheritance. > Metaclasses are a muddle because they can do everything a class can do and more, since metclasses are to classes as classes are to objects. >From the bottom up: objects: get magic methods from their class can manipulate non-magic methods and attributes on a per-instance basis classes: get magic methods from their metaclass can create magic methods and attributes used by objects. Plus anything objects can do metaclasses: can create magic methods of a class can define class methods and attributes Plus anything a class can do. Metaclasses are a muddle because we (the royal we) are used to doing most things at the class level. Defining class methods in a metaclass like this probably isn't your regular style >>> class MetaK(type): ... def foo(cls): pass ... >>> class K: ... __metaclass__ = MetaK ... def bar(cls): pass ... bar = classmethod(bar) ... >>> K.foo > >>> K.bar > I'm happy using classmethod instead of writing a metaclass everytime I need a classmethod. I think everyone is class-centric, or we could define static methods using class MetaK(type): def foo(): pass foo = metaclassmethod(foo) # class method of a type is a static method? Since I've never wanted to set a type's __repr__ I use metaclasses as a handy place to do mundane class-level manipulations. I don't actually need to do type level manipulations. So that's why I like class decorators, it lets me push type level manipulations (manipulating a class from its type's __init__ or __new__) down to the class level where my brain normally hangs out. I just want to change an object's behavior and I'd welcome a chance to do it by manipulating the class and not the class's class (which is the object's class's class, yikes!) -jackdied ps, I tried to raise a simliar point at PyCon during Alex Martelli's Q&A but got flustered and screwed it all up. From jack at performancedrivers.com Tue Mar 29 02:55:50 2005 From: jack at performancedrivers.com (Jack Diederich) Date: Tue Mar 29 02:55:54 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050326122905.DB7D.JCARLSON@uci.edu> References: <20050324142035.DB6C.JCARLSON@uci.edu> <20050326122905.DB7D.JCARLSON@uci.edu> Message-ID: <20050329005550.GI12663@performancedrivers.com> On Sat, Mar 26, 2005 at 12:36:08PM -0800, Josiah Carlson wrote: > > Eric Nieuwland wrote: > > > > Given the ideas so far, would it possible to: > > > > def meta(cls): > > ... > > > > @meta > > class X(...): > > ... > > It is not implemented in Python 2.4. From what I understand, making it > happen in Python 2.5 would not be terribly difficult. The question is > about a "compelling use case". Is there a use where this syntax is > significantly better, easier, etc., than an equivalent metaclass? Would > people use the above syntax if it were available? > For compelling, I think the code smell put off by the "no conflict" metaclass generator recipe (which also appeared in Alex Martelli's PyCon talk) is fairly compelling from a duck typing point of view. # would you rather class K: __metaclass__ = no_conflict(MetaA, MetaB) # or @decoA @decoB class K: pass Unless you actually want a class two inherit magic methods from two different types you don't need two metaclasses. You just need the class manipulations that are done in two different metaclasses. I get around this[1] by defining a function that calls things that manipulate classes, the metaclass's init will make the 'register' function static if it is defined in the __dict__ and then call it with (name, cls). If I called that method 'decorate' instead and spelled it @decorate I'd be a happy camper. -jackdied [1] "Register" metatype, define the register method to screw around with your class definition or leave it out to let your parent class do its thing class Register(type): def __init__(cls, name, bases, dict): if ('register' in dict): setattr(cls, 'register', staticmethod(dict['register'])) cls.register(name, cls) I call it Register because largely I just use it to check the __dict__ for special methods and put classes in one or more global buckets. I have cron jobs that operate on the different buckets. From pje at telecommunity.com Tue Mar 29 04:44:59 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Mar 29 04:41:13 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050329005550.GI12663@performancedrivers.com> References: <20050326122905.DB7D.JCARLSON@uci.edu> <20050324142035.DB6C.JCARLSON@uci.edu> <20050326122905.DB7D.JCARLSON@uci.edu> Message-ID: <5.1.1.6.0.20050328214020.02e98e20@mail.telecommunity.com> At 07:55 PM 3/28/05 -0500, Jack Diederich wrote: >For compelling, I think the code smell put off by the "no conflict" metaclass >generator recipe (which also appeared in Alex Martelli's PyCon talk) is fairly >compelling from a duck typing point of view. > ># would you rather >class K: > __metaclass__ = no_conflict(MetaA, MetaB) ># or >@decoA >@decoB >class K: pass Actually, it's possible today with: class K: decoB() decoA() As long as decoA() and decoB() use the "class advisor" mechanism I built for Zope and PyProtocols. That mechanism basically sticks a custom __metaclass__ into the enclosing class, and implements a simple protocol for chaining subsequently-defined class advisors. See 'protocols.advice' in PyProtocols or 'zope.interface.advice' in Zope or Twisted for details. Anyway, I'm not certain that moving these functions up to decorator status will really do anything useful; you can already put them near the top of the class definition, such that they're relatively prominent. From steve at holdenweb.com Tue Mar 29 08:28:19 2005 From: steve at holdenweb.com (Steve Holden) Date: Tue Mar 29 08:28:58 2005 Subject: [Python-Dev] Re: comprehension abbreviation (was: Adding any() and all()) In-Reply-To: References: Message-ID: Jim Jewett wrote: > Gareth McCaughan wrote: > >>Some bit of my brain is convinced that [x in stuff if condition] >>is the Right Syntax and keeps making me type it even though >>I know it doesn't work. > > > (and I agree with Gareth) > > > On Monday 2005-03-14 12:42, Eric Nieuwland wrote: > >>The full syntax is: >>[ f(x) for x in seq if pred(x) ] >>being allowed to write 'x' instead of 'identity(x)' is already a >>shortcut, just as dropping the conditional part. > > > I think this is the heart of the disagreement. > > Mentally, I'm not collecting some function of x (which happens > to be identity). I am filtering an existing set. Being able to > collect f(x) instead is just a useful but hackish shortcut. > Have it your own way, but if you happen to need a list of transformed elements of a filtered list (and that isn't an uncommon requirement) then the idea of selecting the set members and then transforming the copies as a separate step seems a little ... unnecessary. Having to write [x for x in seq] to produce a copy of a list doesn't seem that outrageous to me, and I don't find the predicate-less case of your proposal that convincing: [x in seq] seems somehow too terse. [...] regards Steve -- Steve Holden +1 703 861 4237 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/ From phd at mail2.phd.pp.ru Tue Mar 29 11:14:54 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Tue Mar 29 11:14:58 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050328183129.75d6c8c8@Goku> References: <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> <20050324141330.GF30864@phd.pp.ru> <20050324113641.297a4a18@localhost.localdomain> <20050328142735.GA3087@phd.pp.ru> <20050328114205.1508614b@localhost.localdomain> <20050328183129.75d6c8c8@Goku> Message-ID: <20050329091454.GB30894@phd.pp.ru> Hello! On Mon, Mar 28, 2005 at 06:31:29PM -0300, Rodrigo Dias Arruda Senra wrote: > Outstanding work Oleg. I read it through and wouldn't change a bit. > I have revised: libwebbrowser.tex.patch and webbrowser.py.patch. > They are Ok regarding grammar, TeX and ... whatever . > I recommend to apply both files. Thank you very much! > However, I would withdraw the third file -- webbrowser wrapper script, since the same > functionality can be accomplished with: > > python -m webbrowser http://www.python.org Oh, that's an idea! I haven't thought about it because I didn't switched to python 2.4 yet - commercial programs that I'm developing are based on 2.3. I copied the "webbrowser script" idea from pydoc/pydoc.py. Fred, now it's your turn. Please review http://python.org/sf/754022 and decide what to do with all that. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From anthony at interlink.com.au Tue Mar 29 14:43:42 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Mar 29 15:50:24 2005 Subject: [Python-Dev] BRANCH FREEZE for 2.4.1 final, 2005-03-30 00:00 UTC Message-ID: <200503292243.43435.anthony@interlink.com.au> The release24-maint branch is FROZEN from 00:00 UTC, 2005-03-30 (in about 11 hours from now). As usual, unless you're in the set (Anthony, MvL, Fred), please hold off on checkins to the branch until I send a message unfreezing it. Once we've had the appropriate brown-paper-bag time delay, the branch will be unfrozen for 2.4.2 in about 6 months time. -- Anthony Baxter It's never too late to have a happy childhood. From andymac at bullseye.apana.org.au Tue Mar 29 15:35:46 2005 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Tue Mar 29 16:00:01 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python thread_pthread.h, 2.53, 2.53.4.1 In-Reply-To: <4247FBBF.4010608@v.loewis.de> References: <4247ED55.4060708@bullseye.apana.org.au> <4247FBBF.4010608@v.loewis.de> Message-ID: <424959B2.2070404@bullseye.apana.org.au> Martin v. L?wis wrote: > Andrew MacIntyre wrote: > >> This change has broken the build on FreeBSD 4.x for me: >> >> gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall >> -Wstrict-prototypes - >> I. -I./Include -DPy_BUILD_CORE -o Python/thread.o Python/thread.c >> In file included from Python/thread.c:101: >> Python/thread_pthread.h:19: syntax error >> *** Error code 1 > > > This should be fixed now, please try again and report whether it > works. It does. > It would be really nice if you could try to analyse such problems > deeper in the future. I would have if I'd had the time; sorry that I didn't state that. All I really intended was to alert Anthony to a problem on a widely used but non-primary target platform, so that he could make a decision about whether to worry about it or not for the 2.4.1 release. I had intended (but again didn't state) to follow this up at the earliest opportunity (which probably would have been this evening). > In this case, it would have helped if you > had reported that _POSIX_SEMAPHORES is defined as > > #define _POSIX_SEMAPHORES > > so that the #if line expands to > > #if == -1 > > The standard solution in this case is to write > > #if (_POSIX_SEMAPHORES+0) == -1 First time I can recall seeing that sort of situation - which I guess shows my lack of serious cross-platform C programming experience. Thanks for the fix and taking the time to explain. Regards, Andrew. ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From reinhold-birkenfeld-nospam at wolke7.net Tue Mar 29 18:46:09 2005 From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld) Date: Tue Mar 29 18:48:41 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050328183129.75d6c8c8@Goku> References: <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> <20050324141330.GF30864@phd.pp.ru> <20050324113641.297a4a18@localhost.localdomain> <20050328142735.GA3087@phd.pp.ru> <20050328114205.1508614b@localhost.localdomain> <20050328183129.75d6c8c8@Goku> Message-ID: Rodrigo Dias Arruda Senra wrote: > | > On Thu, Mar 24, 2005 at 11:36:41AM -0300, Rodrigo Dias Arruda Senra wrote: > | > > Edit libwebbrowser.tex as you see fit, then send it to me > | > > and I'll TeXify it back to you. > | > > | > Uploaded to http://python.org/sf/754022 . I am not a native English > | > speaker, and this is the first time I've edited a TeX file. Please > | > correct my grammar, spelling, TeX, whatever... > > Outstanding work Oleg. I read it through and wouldn't change a bit. > I have revised: libwebbrowser.tex.patch and webbrowser.py.patch. > They are Ok regarding grammar, TeX and ... whatever . > I recommend to apply both files. FWIW, same judgement from me. Reinhold -- Mail address is perfectly valid! From phd at mail2.phd.pp.ru Tue Mar 29 18:57:26 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Tue Mar 29 18:57:29 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: References: <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> <20050324141330.GF30864@phd.pp.ru> <20050324113641.297a4a18@localhost.localdomain> <20050328142735.GA3087@phd.pp.ru> <20050328114205.1508614b@localhost.localdomain> <20050328183129.75d6c8c8@Goku> Message-ID: <20050329165726.GB16219@phd.pp.ru> On Tue, Mar 29, 2005 at 06:46:09PM +0200, Reinhold Birkenfeld wrote: > Rodrigo Dias Arruda Senra wrote: > > Outstanding work Oleg. I read it through and wouldn't change a bit. > > I have revised: libwebbrowser.tex.patch and webbrowser.py.patch. > > They are Ok regarding grammar, TeX and ... whatever . > > I recommend to apply both files. > > FWIW, same judgement from me. Thank you! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From jcarlson at uci.edu Tue Mar 29 19:21:12 2005 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue Mar 29 19:22:45 2005 Subject: [Python-Dev] Re: comprehension abbreviation (was: Adding any() and all()) In-Reply-To: References: Message-ID: <20050329091542.DB89.JCARLSON@uci.edu> Steve Holden wrote: [...] > Having to write > > [x for x in seq] > > to produce a copy of a list doesn't seem that outrageous to me, and I > don't find the predicate-less case of your proposal that convincing: > > [x in seq] > > seems somehow too terse. And is already valid Python syntax; producing a list of a boolean (if x is bound), a TypeError (if seq is a dictionary, x is bound, and x isn't hashable), or a NameError (if x is not bound). If I recall, changing the meaning of valid Python syntax is to be frowned upon, and the suggestion should be tossed out the window strictly because of that reason. As for "for x" or its equivalent, being too much additional overhead to type in list comprehensions, I think maybe we are getting too picky for our own good. - Josiah From gvanrossum at gmail.com Tue Mar 29 19:39:58 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Tue Mar 29 19:40:04 2005 Subject: [Python-Dev] Re: comprehension abbreviation (was: Adding any() and all()) In-Reply-To: <20050329091542.DB89.JCARLSON@uci.edu> References: <20050329091542.DB89.JCARLSON@uci.edu> Message-ID: I thought I had been clear already, but since this thread keeps going maybe I need to reiterate that there's zero chance that this syntax proposal (or anything like it) will be accepted. You all are of course free to continue to discuss it, but as I've explained before it just isn't worth it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amacbeth at gmail.com Mon Mar 28 03:55:28 2005 From: amacbeth at gmail.com (Adam MacBeth) Date: Tue Mar 29 22:34:43 2005 Subject: [Python-Dev] using SCons to build Python Message-ID: <4ef057f605032717559e1cb6a@mail.gmail.com> Has anyone ever considered using SCons to build Python? SCons is a great build tool written in Python that provides some Autoconf-like functionality (http://www.scons.org). It seems like this type of self-hosting would be a great testament to the power of Python as well as helping to reinforce the strength of SCons as a next generation build tool. Thanks, Adam From phd at phd.pp.ru Mon Mar 28 16:27:35 2005 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue Mar 29 22:34:45 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050324113641.297a4a18@localhost.localdomain> References: <200503181855.09229.fdrake@acm.org> <20050319202836.GA6273@phd.pp.ru> <1323.200.158.47.2.1111329627.squirrel@200.158.47.2> <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> <20050324141330.GF30864@phd.pp.ru> <20050324113641.297a4a18@localhost.localdomain> Message-ID: <20050328142735.GA3087@phd.pp.ru> Hi! On Thu, Mar 24, 2005 at 11:36:41AM -0300, Rodrigo Dias Arruda Senra wrote: > Edit libwebbrowser.tex as you see fit, then send it to me > and I'll TeXify it back to you. Uploaded to http://python.org/sf/754022 . I am not a native English speaker, and this is the first time I've edited a TeX file. Please correct my grammar, spelling, TeX, whatever... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From oliphant at ee.byu.edu Tue Mar 29 23:04:23 2005 From: oliphant at ee.byu.edu (Travis Oliphant) Date: Tue Mar 29 23:04:28 2005 Subject: [Python-Dev] 64-bit sequence and buffer protocol Message-ID: <4249C2D7.4060601@ee.byu.edu> I'm posting to this list to again generate open discussion on the problem in current Python that an int is used in both the Python sequence protocol and the Python buffer protocol. The problem is that a C-int is typically only 4 bytes long while there are many applications (mmap for example), that would like to access sequences much larger than can be addressed with 32 bits. There are two aspects to this problem: 1) Some 64-bit systems still define an C-int as 4 bytes long (so even in-memory sequence objects could not be addressed using the sequence protocol). 2) Even 32-bit systems have occasion to sequence a more abstract object (perhaps it is not all in memory) which requires more than 32 bits to address. These are the solutions I've seen: 1) Convert all C-ints to Py_LONG_LONG in the sequence and buffer protocols. 2) Add new C-API's that mirror the current ones which use Py_LONG_LONG instead of the current int. 3) Change Python to use the mapping protocol first (even for slicing) when both the mapping and sequence protocols are defined. 4) Tell writers of such large objects to not use the sequence and/or buffer protocols and instead use the mapping protocol and a different "bytes" object (that currently they would have to implement themselves and ignore the buffer protocol C-API). What is the opinion of people on this list about how to fix the problem. I believe Martin was looking at the problem and had told Perry Greenfield he was "fixing it." Apparently at the recent PyCon, Perry and he talked and Martin said the problem is harder than he had initially thought. It would be good to document what some of this problems are so that the community can assist in fixing this problem. -Travis O. From walter at livinglogic.de Tue Mar 29 23:38:34 2005 From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Mar 29 23:38:37 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <4246CF82.2060500@v.loewis.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> <4246CF82.2060500@v.loewis.de> Message-ID: <1751.84.56.104.245.1112132314.squirrel@isar.livinglogic.de> Martin v. L?wis sagte: > Walter D?rwald wrote: >> I'm not sure if this is the right approach. > > I think the approach is right, but the implementation is wrong. > >> The encoding I specify in >> setup.py should be independent of the encoding used between distutils and PyPI to communicate on the wire. I.e. the author >> (and maintainer) argument should always be unicode. > > "should" is a correct description. It should allow Unicode strings, which it then should encode to UTF-8 during transmission. > The matter of fact is that the register command as released in 2.4 (and 2.4.1) doesn't. OK, that's the problem. >> When str is passed, this is treated >> as any other str in a unicode context, it is decoded using the default encoding. This would fix another problem: It would >> make it nearly impossible to send a request to PyPI with the wrong encoding, because any encoding problems are sorted out >> completely on the client side. > > distutils should *not* assume that byte strings are in the default encoding. It is fair to assume they are in ASCII; They should be the same. If not, the installation is broken (or at least scripts that rely on this break anywhere else). > if the > administrator has changed the default encoding, then this cannot possibly affect all the setup.py files out there. Also, it > is a fact that the > deployed versions of the register command just send byte strings > in setup.py as-is, without trying to do any kind of recoding. >> In any case, PyPI now requires that the form submission uses UTF-8, and refuses anything else. So it *is* impossible to send, > say, > Latin-1; whether the client makes that happen by properly encoding Unicode strings or whether they are in setup.py in the > first place does not matter. So can I have one setup.py for both Python 2.4 and Python 2.5 that does the correct thing when creating a Windows installer for Python 2.4 (I've used Unicode strings for that until now) and using the upload command with Python CVS (which seems to require a byte string now)? I'd like to avoid having to use version checks in setup.py. > [...] Bye, Walter D?rwald From walter at livinglogic.de Tue Mar 29 23:41:16 2005 From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=) Date: Tue Mar 29 23:41:20 2005 Subject: [Python-Dev] Pickling instances of nested classes Message-ID: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> Currently instances of nested classes can't be pickled. For old style classes unpickling fails to find the class: >>> import cPickle >>> class Foo: ... class Bar: ... pass ... >>> cPickle.loads(cPickle.dumps(Foo.Bar())) Traceback (most recent call last): File "", line 1, in ? AttributeError: 'module' object has no attribute 'Bar' For new style classes, pickling itself fails: >>> class Foo(object): ... class Bar(object): ... pass ... >>> cPickle.loads(cPickle.dumps(Foo.Bar())) Traceback (most recent call last): File "", line 1, in ? cPickle.PicklingError: Can't pickle : attribute lookup __main__.Bar failed I think this should be fixed (see below for use cases). There's an old bug report open for this (http://www.python.org/sf/633930). Classes would need access to their fully qualified name (starting from the module level) (perhaps in a new class attribute __fullname__?) and the pickle machinery would have to use this name when storing class names instead of __name__. The second part should be easy to implement. The first part is harder. It can't be done by changing meta classes, because classes are created "inside out", i.e. in: class Foo: class Bar: class Baz: pass when Baz class is created, the Bar class hasn't been assigned a name yet. Another problem is that it should only be done for classes *defined* inside other classes, i.e. in class Foo: pass class Bar: Baz = Foo Foo should remain unchanged. For this I guess the parser would have to be changed to somehow keep a stack of currently "open" class definitions, so the __fullname__ attribute (or dict entry) can be set once a new class statement is encountered. Of course the remaining interesting question is if this change is worth it and if there are use cases for it. Possible use cases require that: 1) There's a collection of name objects in the scope of a class. 2) These objects are subclasses of other (nested or global) classes. 3) There are instances of those classes. There are two use cases that immediately come to mind: XML and ORMs. For XML: 1) Those classes are the element types and the nested classes are the attributes. 2) Being able to define those attributes as separate classes makes it possible to implement custom functionality (e.g. for validation or for handling certain attribute types like URLs, colors etc.) and 3) Those classes get instantiated when an XML tree is created or parsed. A framework that does this (and my main motivation for writing this :)) is XIST (see http://www.livinglogic.de/Python/xist/). For the ORM case: Each top level class defines a table and the nested classes are the fields, i.e. something like this: class person(Table): class firstname(Varchar): "The person's first name" null = False class lastname(Varchar): "The person's last name" null = False class password(Varchar): "Login password" def validate(self, value): if not any(c.islower() for c in value) and \ not any(c.isupper() for c in value) and \ not any(c.isdigit() for c in value): raise InvalidField("password requires a mix of upper and lower" "case letters and digits") Instances of these classes are the records read from the database. A framework that does something similar to this (although AFAIK fields are not nested classes is SQLObject (http://sqlobject.org/) So is this change wanted? useful? implementable with reasonable effort? Or just not worth it? Bye, Walter D?rwald From aahz at pythoncraft.com Wed Mar 30 00:15:21 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed Mar 30 00:15:24 2005 Subject: [Python-Dev] using SCons to build Python In-Reply-To: <4ef057f605032717559e1cb6a@mail.gmail.com> References: <4ef057f605032717559e1cb6a@mail.gmail.com> Message-ID: <20050329221521.GC22273@panix.com> On Sun, Mar 27, 2005, Adam MacBeth wrote: > > Has anyone ever considered using SCons to build Python? SCons is a > great build tool written in Python that provides some Autoconf-like > functionality (http://www.scons.org). It seems like this type of > self-hosting would be a great testament to the power of Python as well > as helping to reinforce the strength of SCons as a next generation > build tool. Unfortunately, this leads to a bootstrapping problem. :-( It might be worthwhile investigating moving parts of CPython's build process to SCons, but core Python needs to stick with make. If you want to push this idea forward, you'll probably need to create a proof-of-concept. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From bob at redivi.com Wed Mar 30 00:21:33 2005 From: bob at redivi.com (Bob Ippolito) Date: Wed Mar 30 00:21:41 2005 Subject: [Python-Dev] using SCons to build Python In-Reply-To: <20050329221521.GC22273@panix.com> References: <4ef057f605032717559e1cb6a@mail.gmail.com> <20050329221521.GC22273@panix.com> Message-ID: <28ef143fdb7117f2e97cf6b563e4d8bf@redivi.com> On Mar 29, 2005, at 5:15 PM, Aahz wrote: > On Sun, Mar 27, 2005, Adam MacBeth wrote: >> >> Has anyone ever considered using SCons to build Python? SCons is a >> great build tool written in Python that provides some Autoconf-like >> functionality (http://www.scons.org). It seems like this type of >> self-hosting would be a great testament to the power of Python as well >> as helping to reinforce the strength of SCons as a next generation >> build tool. > > Unfortunately, this leads to a bootstrapping problem. :-( It might be > worthwhile investigating moving parts of CPython's build process to > SCons, but core Python needs to stick with make. If you want to push > this idea forward, you'll probably need to create a proof-of-concept. I think it would also be important for SCons to gain more traction amongst the Python community before it filters into the core. All of the Python extensions I use, and the ones I write myself, are built using distutils. I'm not really fond of distutils, but last I looked at SCons it didn't seem like a big enough win for mostly-Python-projects.. The fact that it uses 1.5.2-isms, where distutils has (in theory) moved beyond that, makes it even less of a sell to me. -bob From martin at v.loewis.de Wed Mar 30 00:25:43 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 30 00:25:46 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <1751.84.56.104.245.1112132314.squirrel@isar.livinglogic.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> <4246CF82.2060500@v.loewis.de> <1751.84.56.104.245.1112132314.squirrel@isar.livinglogic.de> Message-ID: <4249D5E7.2020509@v.loewis.de> Walter D?rwald wrote: > So can I have one setup.py for both Python 2.4 and Python 2.5 that does the correct thing when creating a Windows installer for > Python 2.4 (I've used Unicode strings for that until now) and using the upload command with Python CVS (which seems to require a > byte string now)? I'd like to avoid having to use version checks in setup.py. Well, the upload command doesn't look at the metadata. It is the register command which does, and it indeed requires utf-8 at the moment. This can be fixed, of course, but not for already-released versions. Regards, Martin From mwh at python.net Wed Mar 30 00:29:17 2005 From: mwh at python.net (Michael Hudson) Date: Wed Mar 30 00:29:19 2005 Subject: [Python-Dev] using SCons to build Python In-Reply-To: <4ef057f605032717559e1cb6a@mail.gmail.com> (Adam MacBeth's message of "Sun, 27 Mar 2005 17:55:28 -0800") References: <4ef057f605032717559e1cb6a@mail.gmail.com> Message-ID: <2mfyyef5mq.fsf@starship.python.net> Adam MacBeth writes: > Has anyone ever considered using SCons to build Python? Well, er, there's an obvious problem here somewhere... > SCons is a great build tool written in Python that provides some > Autoconf-like functionality (http://www.scons.org). It seems like > this type of self-hosting would be a great testament to the power of > Python as well as helping to reinforce the strength of SCons as a > next generation build tool. As an active Python developer, I'd like some less fluffy benefits than that. Cheers, mwh -- 34. The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From martin at v.loewis.de Wed Mar 30 00:40:21 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 30 00:40:24 2005 Subject: [Python-Dev] 64-bit sequence and buffer protocol In-Reply-To: <4249C2D7.4060601@ee.byu.edu> References: <4249C2D7.4060601@ee.byu.edu> Message-ID: <4249D955.2050900@v.loewis.de> Travis Oliphant wrote: > What is the opinion of people on this list about how to fix the > problem. I believe Martin was looking at the problem and had told > Perry Greenfield he was "fixing it." Apparently at the recent PyCon, > Perry and he talked and Martin said the problem is harder than he had > initially thought. It would be good to document what some of this > problems are so that the community can assist in fixing this problem. I have put a patch on http://sourceforge.net/tracker/index.php?func=detail&aid=1166195&group_id=5470&atid=305470 which solves this problem (eventually); this is the pre-PyCon version; I'll update it to the post-PyCon version later this month. I'll also write a PEP with the proposed changes. > 1) Convert all C-ints to Py_LONG_LONG in the sequence and buffer protocols. This would be bad, since it would cause an overhead on 32-bit systems. Instead, I propose to change all C ints holding indexes and sizes to Py_ssize_t. > 2) Add new C-API's that mirror the current ones which use Py_LONG_LONG > instead of the current int. I'll propose a type flag, where each type can indicate whether it expects indexes and sizes as int or as Py_ssize_t. However, there are more issues. In particular, PyArg_ParseTuple needs to change to expect a different index type for selected "i" arguments; it also needs to change to possibly store a different type into the length of a "s#" argument. This altogether doesn't support types that exceed 2**31 elements even on an 32-bit system (or 2**63 elements on a 64-bit system); authors of such types would have to follow the advise > 4) Tell writers of such large objects to not use the sequence and/or >buffer protocols and instead use the mapping protocol and a different >" bytes" object (that currently they would have to implement themselves >and ignore the buffer protocol C-API). Regards, Martin From martin at v.loewis.de Wed Mar 30 00:43:52 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 30 00:43:55 2005 Subject: [Python-Dev] Pickling instances of nested classes In-Reply-To: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> References: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> Message-ID: <4249DA28.7040904@v.loewis.de> Walter D?rwald wrote: > So is this change wanted? useful? implementable with reasonable effort? Or > just not worth it? I think it is just not worth it. This means I won't attempt to implement it. I think I defined originally the __module__ attribute for classes to support better pickling (and defined it to be a string to avoid cycles); we considered the nested classes case at the time and concluded "oh well, don't do that then". Regards, Martin From pedronis at strakt.com Wed Mar 30 01:47:16 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Wed Mar 30 01:45:49 2005 Subject: [Python-Dev] Pickling instances of nested classes In-Reply-To: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> References: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> Message-ID: <4249E904.30808@strakt.com> Walter D?rwald wrote: > For XML: 1) Those classes are the element types and the nested classes > are the attributes. 2) Being able to define those attributes as separate > classes makes it possible to implement custom functionality (e.g. for > validation or for handling certain attribute types like URLs, colors etc.) > and 3) Those classes get instantiated when an XML tree is created or parsed. > A framework that does this (and my main motivation for writing this :)) is > XIST (see http://www.livinglogic.de/Python/xist/). > > For the ORM case: Each top level class defines a table and the nested > classes are the fields, i.e. something like this: > > class person(Table): > class firstname(Varchar): > "The person's first name" > null = False > class lastname(Varchar): > "The person's last name" > null = False > class password(Varchar): > "Login password" > def validate(self, value): > if not any(c.islower() for c in value) and \ > not any(c.isupper() for c in value) and \ > not any(c.isdigit() for c in value): > raise InvalidField("password requires a mix of upper and lower" > "case letters and digits") > > Instances of these classes are the records read from the database. A framework > that does something similar to this (although AFAIK fields are not nested > classes is SQLObject (http://sqlobject.org/) > > > So is this change wanted? useful? implementable with reasonable effort? Or > just not worth it? > notice that in this cases often metaclasses are involved or could easely be, so if pickling would honor __reduce__ or __reduce_ex__ on metaclasses (which right now it doesn't treating their instances as normal classes) one could roll her own solution without the burden for the language of implementing pickling of nested classes in general, so I think that would make more sense, to add support to honor __reduce__/__reduce_ex__ for metaclasses. From barry at python.org Wed Mar 30 01:49:02 2005 From: barry at python.org (Barry Warsaw) Date: Wed Mar 30 01:49:05 2005 Subject: [Python-Dev] using SCons to build Python In-Reply-To: <4ef057f605032717559e1cb6a@mail.gmail.com> References: <4ef057f605032717559e1cb6a@mail.gmail.com> Message-ID: <1112140142.17497.369.camel@geddy.wooz.org> On Sun, 2005-03-27 at 20:55, Adam MacBeth wrote: > Has anyone ever considered using SCons to build Python? SCons is a > great build tool written in Python that provides some Autoconf-like > functionality (http://www.scons.org). It seems like this type of > self-hosting would be a great testament to the power of Python as well > as helping to reinforce the strength of SCons as a next generation > build tool. SCons is cool and all -- I use it as a build tool for some of our products -- but it has a lot of quirks and takes a lot of getting used to. The autoconf-based build that we have now works well enough (for *nix) so IMO, switching to SCons is effort that would be better spent elsewhere. I don't see lots of upside to switching. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/python-dev/attachments/20050329/6c3b282a/attachment-0001.pgp From tjreedy at udel.edu Wed Mar 30 03:41:16 2005 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Mar 30 03:42:01 2005 Subject: [Python-Dev] Re: comprehension abbreviation (was: Adding any() andall()) References: Message-ID: "Steve Holden" wrote in message news:d2asg9$3tf$1@sea.gmane.org... > Having to write > > [x for x in seq] > > to produce a copy of a list doesn't seem that outrageous to me, Except for (currently) leaving the last value of sequence bound to 'x' after making the copy, how is the above different from list(seq)? TJR From kbk at shore.net Wed Mar 30 04:08:55 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Mar 30 04:09:25 2005 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> Patch / Bug Summary ___________________ Patches : 297 open (+11) / 2812 closed (+11) / 3109 total (+22) Bugs : 871 open ( +1) / 4900 closed (+33) / 5771 total (+34) RFE : 175 open ( +0) / 150 closed ( +0) / 325 total ( +0) New / Reopened Patches ______________________ Decimal interaction with __rop__ (2005-03-19) http://python.org/sf/1166602 opened by Facundo Batista Fix _tryorder in webbrowser.py (2005-03-20) http://python.org/sf/1166780 opened by Rodrigo Dias Arruda Senra Fix handling of large octal literals (2005-03-20) CLOSED http://python.org/sf/1166879 opened by Nick Coghlan locale.getdefaultencoding: LANGUAGE not correctly parsed (2005-03-20) http://python.org/sf/1166938 opened by Matthias Klose locale.getdefaultencoding: precedence of LANGUAGE / LANG (2005-03-20) http://python.org/sf/1166948 opened by Matthias Klose locale: 'utf' is not a known encoding (2005-03-20) http://python.org/sf/1166957 opened by Matthias Klose asdl_c.py fix for long lines (2005-03-20) CLOSED http://python.org/sf/1167117 opened by John Ehresman Tar file symbolic link bug fix (2005-03-20) CLOSED http://python.org/sf/1167128 opened by Ellers a fix for doctest crash when it is ran on itself (2005-03-20) CLOSED http://python.org/sf/1167316 opened by Ilya Sandler [AST] Generator expressions (2005-03-22) http://python.org/sf/1167628 opened by Nick Coghlan ast for decorators (2005-03-21) http://python.org/sf/1167709 opened by John Ehresman distutils.dir_utils.mkpath to support unicode (2005-03-21) http://python.org/sf/1167716 opened by John M. Camara Add current dir when running try_run test program (2005-03-22) http://python.org/sf/1168055 opened by Michiel de Hoon tarfile.py: set sizes of non-regular files to zero. (2005-03-22) http://python.org/sf/1168594 opened by Lars Gust?bel Handle ungzipped man pages in rpm builds correctly (2005-03-23) http://python.org/sf/1169193 opened by Robin Green [ast branch] unicode literal fixes (2005-03-25) http://python.org/sf/1170272 opened by John Ehresman Method for cell objects to access contents (2005-03-24) http://python.org/sf/1170323 opened by paul cannon Newline in error output of py_compile.compile (2005-03-26) http://python.org/sf/1171150 opened by paul cannon bug fix for islice() in docs (2005-03-27) CLOSED http://python.org/sf/1171417 opened by Pernici Mario Patch for [ 1170331 ] Error in base64.b32decode (2005-03-27) http://python.org/sf/1171487 opened by logistix Fix compile/link when using Darwin 8 (2005-03-28) CLOSED http://python.org/sf/1171735 opened by Bob Ippolito Fix compile/link when using Darwin 8 (2005-03-28) CLOSED http://python.org/sf/1171767 opened by Bob Ippolito long long support for array module (2005-03-29) http://python.org/sf/1172711 opened by Oren Tirosh Patches Closed ______________ [AST] Fix handling of large octal literals (2005-03-20) http://python.org/sf/1166879 closed by bcannon asdl_c.py fix for long lines (2005-03-20) http://python.org/sf/1167117 closed by bcannon Tar file symbolic link bug fix (2005-03-20) http://python.org/sf/1167128 closed by loewis ast-branch: msvc project sync (2003-05-23) http://python.org/sf/742621 closed by bcannon ast-branch: hacks so asdl_c.py generates compilable code (2005-01-14) http://python.org/sf/1102710 closed by bcannon a fix for doctest crash when it is ran on itself (2005-03-21) http://python.org/sf/1167316 closed by tim_one urllib2 .getheaders attribute error (2005-02-06) http://python.org/sf/1117588 closed by fdrake the quotes page on the Web site links to something broken (2005-03-14) http://python.org/sf/1163314 closed by jjinux add set/getattr interface to tkFont.Font objects (2003-07-01) http://python.org/sf/764221 closed by loewis bug fix for islice() in docs (2005-03-27) http://python.org/sf/1171417 closed by rhettinger Fix compile/link when using Darwin 8 (2005-03-28) http://python.org/sf/1171735 closed by etrepum Fix compile/link when using Darwin 8 (2005-03-28) http://python.org/sf/1171767 closed by etrepum New / Reopened Bugs ___________________ IterableUserDict not in docs (2005-03-19) http://python.org/sf/1166582 opened by Kent Johnson The readline module can cause python to segfault (2005-03-19) http://python.org/sf/1166660 opened by Yariv Ido [ast branch] fatal error when compiling test_bool.py (2005-03-19) CLOSED http://python.org/sf/1166704 opened by John Ehresman [ast branch] fatal error when compiling test_bool.py (2005-03-20) http://python.org/sf/1166714 opened by John Ehresman Fails assertion in winsig.c under VC 8.0 (2005-03-21) http://python.org/sf/1167262 opened by Timothy Fitz Error "exec"ing python code (2005-03-21) http://python.org/sf/1167300 opened by Stefan Seefeld xmlrpclib invalid url (2005-03-21) CLOSED http://python.org/sf/1167329 opened by Mark Doukidis Python 2.4 causes BitTorrent 3.4.2 failure (2005-03-21) http://python.org/sf/1167397 opened by Andrew P. Lentvorski, Jr. Argument genexp corner case (2005-03-21) http://python.org/sf/1167751 opened by John Ehresman Line ending documentation is misleading (2005-03-21) http://python.org/sf/1167922 opened by Paul Moore threading.Thread.join() cannot be interrupted by a Ctrl-C (2005-03-21) http://python.org/sf/1167930 opened by Nicholas Veeser Python 2.5a0 Tutorial errors and observations (2005-03-21) http://python.org/sf/1168135 opened by Michael R Bax Possible windows+python bug (2005-03-22) http://python.org/sf/1168427 opened by holo9 frame.f_exc_type,value,traceback (2005-03-22) http://python.org/sf/1168746 opened by Armin Rigo ftplib.py string index out of range (2005-03-23) http://python.org/sf/1168983 opened by David Carroll PySys_WriteStderr() -> WaitForSingleObject() hangs system (2005-03-23) http://python.org/sf/1169108 opened by stan pinte improvement of ossaudiodev module doc. (2005-03-23) CLOSED http://python.org/sf/1169212 opened by hiower Install fail code 2932 after fail to copy python_icon.exe (2005-03-24) http://python.org/sf/1169633 opened by Wagner modules missing from list of Carbon modules (2005-03-23) http://python.org/sf/1169679 opened by Jesse Weinstein HTTPResponse.getheaders() returns lowercased header names (2005-03-24) http://python.org/sf/1170065 opened by yain BaseCookie does not call value_decode (2005-03-25) http://python.org/sf/1170279 opened by Ryan Lovett zipfile UnicodeDecodeError (2005-03-25) http://python.org/sf/1170311 opened by adam davis Error in base64.b32decode (2005-03-25) http://python.org/sf/1170331 opened by toidinamai doc speaks of extensions option while it's *called* ext_modu (2005-03-25) http://python.org/sf/1170422 opened by J?rgen A. Erhard why should URL be required for all packages (2005-03-25) http://python.org/sf/1170424 opened by J?rgen A. Erhard gc module docu (2005-03-25) CLOSED http://python.org/sf/1170460 opened by Martin Renold weakref.proxy incorrect behaviour (2005-03-26) http://python.org/sf/1170766 opened by Alexander Kozlovsky Thread.join() fails to release Lock on KeyboardInterrupt (2005-03-26) http://python.org/sf/1171023 reopened by phansen Thread.join() fails to release Lock on KeyboardInterrupt (2005-03-26) http://python.org/sf/1171023 opened by Peter Hansen Error in code example in main tutorial, section 9.3.4 (2005-03-28) CLOSED http://python.org/sf/1171819 opened by Niek error in documentation for splitting empty string (2005-03-28) http://python.org/sf/1171994 opened by Wai Yip Tung BaseCookie does not call value_decode (2005-03-28) http://python.org/sf/1172011 opened by Ryan Lovett "cmp" should be "key" in sort doc (2005-03-29) CLOSED http://python.org/sf/1172554 opened by Keith Briggs "cmp" should be "key" in sort doc (2005-03-29) http://python.org/sf/1172581 opened by Keith Briggs dumbdbm hoses index on load failure (2005-03-29) http://python.org/sf/1172763 opened by currivan doctest.script_from_examples() result sometimes un-exec-able (2005-03-29) http://python.org/sf/1172785 opened by Jonathan E. Guyer Bugs Closed ___________ subprocess.Popen with copious output hangs (2005-03-13) http://python.org/sf/1162428 closed by astrand subprocess pipe fails under Emacs (2005-03-15) http://python.org/sf/1163759 closed by astrand KeyboardInterrupt causes segmentation fault with threads (2005-03-18) http://python.org/sf/1165761 closed by anthonybaxter super(...).__new__( ... ) behaves "unexpected" (2005-03-16) http://python.org/sf/1164631 closed by brenck Optik OptionParse important undocumented option (2005-01-10) http://python.org/sf/1099324 closed by gward optparse needs reference documentation (2004-07-19) http://python.org/sf/993601 closed by gward optparse docs missing section on Extending (2004-12-16) http://python.org/sf/1086675 closed by gward [ast branch] fatal error when compiling test_bool.py (2005-03-19) http://python.org/sf/1166704 closed by bcannon After fork in subthread, signals are blocked (2003-12-03) http://python.org/sf/853411 closed by mwh xmlrpclib invalid url (2005-03-21) http://python.org/sf/1167329 closed by montanaro BasHTTPServer IE Mac 5.1 size problem (2003-09-25) http://python.org/sf/812340 closed by facundobatista Section 11.20: xmlrpclib Details (2003-08-19) http://python.org/sf/791067 closed by facundobatista socket.inet_aton on 64bit platform (2003-07-07) http://python.org/sf/767150 closed by facundobatista xml.sax second time file loading problem (2002-09-09) http://python.org/sf/606692 closed by facundobatista Error using Tkinter embeded in C++ (2003-03-06) http://python.org/sf/699068 closed by facundobatista urllib ftp broken if Win32 proxy (2002-03-19) http://python.org/sf/532007 closed by facundobatista missing sequence tests - pull from deque (2005-03-18) http://python.org/sf/1166274 closed by doerwalter Unable to Install Python 2.4.1rc1 on windows XP SP2 (2005-03-14) http://python.org/sf/1162677 closed by loewis segfault when running smtplib example (2004-08-04) http://python.org/sf/1003195 closed by facundobatista sgmllib incorrectly handles entities in attributes (2003-07-29) http://python.org/sf/779388 closed by facundobatista Unix popen does not return exit status (2002-09-12) http://python.org/sf/608635 closed by facundobatista Implied __init__.py not copied (2002-09-11) http://python.org/sf/608033 closed by facundobatista pydoc -g dumps core on Solaris 2.8 (2002-08-30) http://python.org/sf/602627 closed by facundobatista makesetup fails: long Setup.local lines (2002-08-05) http://python.org/sf/591287 closed by facundobatista os.tmpfile() can fail on win32 (2002-08-05) http://python.org/sf/591104 closed by facundobatista pthread_exit missing in thread_pthread.h (2002-07-09) http://python.org/sf/579116 closed by facundobatista cgi.py broken with xmlrpclib on Python 2 (2002-06-15) http://python.org/sf/569316 closed by facundobatista Popen exectuion blocking w/threads (2002-06-07) http://python.org/sf/566037 closed by facundobatista fpectl build on solaris: -lsunmath (2002-03-15) http://python.org/sf/530163 closed by facundobatista small change in ossaudiodev module doc. (2005-03-23) http://python.org/sf/1169212 closed by gward Solaris and Python/pykota (2005-03-04) http://python.org/sf/1156441 closed by loewis gc module docu (2005-03-25) http://python.org/sf/1170460 closed by loewis Thread.join() fails to release Lock on KeyboardInterrupt (2005-03-26) http://python.org/sf/1171023 closed by bcannon Error in code example in main tutorial, section 9.3.4 (2005-03-28) http://python.org/sf/1171819 closed by loewis "cmp" should be "key" in sort doc (2005-03-29) http://python.org/sf/1172554 closed by rhettinger From aleaxit at yahoo.com Wed Mar 30 08:45:10 2005 From: aleaxit at yahoo.com (Alex Martelli) Date: Wed Mar 30 08:45:14 2005 Subject: [Python-Dev] Re: comprehension abbreviation (was: Adding any() andall()) In-Reply-To: References: Message-ID: On Mar 29, 2005, at 17:41, Terry Reedy wrote: ... > "Steve Holden" wrote in message > news:d2asg9$3tf$1@sea.gmane.org... >> Having to write >> >> [x for x in seq] >> >> to produce a copy of a list doesn't seem that outrageous to me, > > Except for (currently) leaving the last value of sequence bound to 'x' > after making the copy, how is the above different from list(seq)? Well, it's less concise, and over an order of magnitude slower: Nimue:~/pypy alex$ python2.4 -mtimeit -s'seq=range(1000)' '[x for x in seq]' 1000 loops, best of 3: 312 usec per loop Nimue:~/pypy alex$ python2.4 -mtimeit -s'seq=range(1000)' 'list(seq)' 10000 loops, best of 3: 24.3 usec per loop I fail to see any advantages to compensate for these minuses. Alex From wl at flexis.de Wed Mar 30 10:02:14 2005 From: wl at flexis.de (Wolfgang Langner) Date: Wed Mar 30 10:12:43 2005 Subject: [Python-Dev] Re: [maintenance doc updates] In-Reply-To: <200503300511.j2U5BkQx007073__8899.66696636311$1112159978$gmane$org@localhost.localdomain> References: <200503300511.j2U5BkQx007073__8899.66696636311$1112159978$gmane$org@localhost.localdomain> Message-ID: Hello, Fred L. Drake wrote: > The maintenance version of the documentation has been updated: > > http://www.python.org/dev/doc/maint24/ Very good. The download links under http://docs.python.org/download.html doesn't work any more. (packages not there) But 2.4.1 is not yet released, so I will only remind this not criticise. bye by Wolfgang From anthony at python.org Wed Mar 30 11:36:54 2005 From: anthony at python.org (Anthony Baxter) Date: Wed Mar 30 11:37:47 2005 Subject: [Python-Dev] RELEASED Python 2.4.1 (final) Message-ID: <200503301937.10660.anthony@python.org> On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.4.1 (final). Python 2.4.1 is a bug-fix release. See the release notes at the website (also available as Misc/NEWS in the source distribution) for details of the bugs squished in this release. Python 2.4.1 should be a completely painless upgrade from Python 2.4 - no new features have been added. For more information on Python 2.4.1, including download links for various platforms, release notes, and known issues, please see: http://www.python.org/2.4.1/ Highlights of this new release include: - Bug fixes. According to the release notes, several dozen bugs have been fixed, including a fix for the SimpleXMLRPCServer security issue (PSF-2005-001). Highlights of the previous major Python release (2.4) are available from the Python 2.4 page, at http://www.python.org/2.4/highlights.html Enjoy the new release, Anthony Anthony Baxter anthony@python.org Python Release Manager (on behalf of the entire python-dev team) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050330/439b00db/attachment.pgp From python at rcn.com Wed Mar 30 11:52:56 2005 From: python at rcn.com (Raymond Hettinger) Date: Wed Mar 30 11:57:39 2005 Subject: [Python-Dev] RELEASED Python 2.4.1 (final) In-Reply-To: <200503301937.10660.anthony@python.org> Message-ID: <000901c5350e$4058c1a0$2606a044@oemcomputer> > I'm happy to announce the release of Python 2.4.1 (final). Woohoo! Raymond From phd at mail2.phd.pp.ru Wed Mar 30 13:14:49 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Wed Mar 30 13:14:52 2005 Subject: [Python-Dev] Re: webbrowser.py In-Reply-To: <20050329091454.GB30894@phd.pp.ru> References: <20050322102842.GA25949@phd.pp.ru> <20050323182930.GB1084@phd.pp.ru> <20050324133035.GB29041@phd.pp.ru> <20050324111111.16812549@localhost.localdomain> <20050324141330.GF30864@phd.pp.ru> <20050324113641.297a4a18@localhost.localdomain> <20050328142735.GA3087@phd.pp.ru> <20050328114205.1508614b@localhost.localdomain> <20050328183129.75d6c8c8@Goku> <20050329091454.GB30894@phd.pp.ru> Message-ID: <20050330111449.GB10860@phd.pp.ru> Hello! Jim Jewett have sent me information about remote features of Opera, so I downloaded Opera for Linux and w32 and tested them. On Linux Opera support remote functionality almost identical to Mozilla. I added a new Opera controller. On w32 remote commands are ignored, so GenericBrowser still is used. The updated patches were uploaded to http://python.org/sf/754022 . This, I hope, is the last touch. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From mwh at python.net Wed Mar 30 13:28:16 2005 From: mwh at python.net (Michael Hudson) Date: Wed Mar 30 13:28:20 2005 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules readline.c, 2.82, 2.83 In-Reply-To: (mwh@users.sourceforge.net's message of "Wed, 30 Mar 2005 03:22:05 -0800") References: Message-ID: <2my8c5e5kf.fsf@starship.python.net> mwh@users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Modules > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10274 > > Modified Files: > readline.c > Log Message: > Fixes for > > [ 1166660 ] The readline module can cause python to segfault > > It seems to me that the code I'm rewriting here attempted to call any > user-supplied hook functions using the thread state of the thread that > called the hook-setting function, as opposed to that of the thread > that is currently executing. This doesn't work, in general. > > Fix this by using the PyGILState API (It wouldn't be that hard to > define a dummy version of said API when #ifndef WITH_THREAD, would > it?). > > Also, check the conversion to integer of the return value of a hook > function for errors (this problem was mentioned in the ipython bug > report linked to in the above bug). Oh, and I forgot to say: this should go into realease24-maint when that opens again. Cheers, mwh -- You're going to have to remember that I still think of Twisted as a big multiplayer game, and all this HTTP stuff is just kind of a grotty way to display room descriptions. -- Glyph Lefkowitz From walter at livinglogic.de Wed Mar 30 13:43:50 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Mar 30 13:43:54 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <4249D5E7.2020509@v.loewis.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> <4246CF82.2060500@v.loewis.de> <1751.84.56.104.245.1112132314.squirrel@isar.livinglogic.de> <4249D5E7.2020509@v.loewis.de> Message-ID: <424A90F6.50602@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> So can I have one setup.py for both Python 2.4 and Python 2.5 that >> does the correct thing when creating a Windows installer for >> Python 2.4 (I've used Unicode strings for that until now) and using >> the upload command with Python CVS (which seems to require a >> byte string now)? I'd like to avoid having to use version checks in >> setup.py. > > > Well, the upload command doesn't look at the metadata. It is the > register command which does, OK. > and it indeed requires utf-8 at the > moment. This can be fixed, of course, The register command in 2.4 (and current CVS) simply does a value = str(value) in post_to_server() so the encoded bytes sent depend on the default encoding. Would it be sufficient to change this to value = unicode(value).encode("utf-8") Another solution might be to include the encoding in the Content-type header of the request. IMHO the best solution would be to do both: Always use UTF-8 as the encoding and include this in the Content-type header in the request. PyPI should honor this encoding when it finds it and should fall back to whatever it used before if it doesn't. > but not for already-released > versions. True, but I can live with that as long as I can use the same setup.py for bdist_windist and register under Python 2.4 and upload under Python CVS. Bye, Walter D?rwald From skip at pobox.com Wed Mar 30 22:22:04 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed Mar 30 22:19:25 2005 Subject: [Python-Dev] Re: RELEASED Python 2.4.1 (final) In-Reply-To: References: <200503301937.10660.anthony@python.org> Message-ID: <16971.2668.863568.804610@montanaro.dyndns.org> Terry> The page http://www.python.org/download/ needs to be added to the Terry> list of things updated with a new release. Terry, I'll let others take care of that list (I don't know where it's kept). In the meantime, I updated the download page to reference 2.4.1. Skip From martin at v.loewis.de Wed Mar 30 22:37:09 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 30 22:37:15 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <424A90F6.50602@livinglogic.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> <4246CF82.2060500@v.loewis.de> <1751.84.56.104.245.1112132314.squirrel@isar.livinglogic.de> <4249D5E7.2020509@v.loewis.de> <424A90F6.50602@livinglogic.de> Message-ID: <424B0DF5.409@v.loewis.de> Walter D?rwald wrote: > The register command in 2.4 (and current CVS) simply does a > value = str(value) > in post_to_server() so the encoded bytes sent depend on the > default encoding. Would it be sufficient to change this to > value = unicode(value).encode("utf-8") Indeed. I think this can go into 2.4.2. > Another solution might be to include the encoding in the Content-type > header of the request. IMHO the best solution would be to do both: > Always use UTF-8 as the encoding and include this in the Content-type > header in the request. PyPI should honor this encoding when it finds > it and should fall back to whatever it used before if it doesn't. Yeah, well :-) Content-type in form upload is a mess, as you certainly know. It should be honored, but commonly isn't. This, in turn, causes browsers to ignore it. PyPI uses the CGI module. It currently decodes anything that doesn't have a filename attribute to UTF-8, causing rejection of anything that doesn't send UTF-8. This could be fixed/extended, but I think that would be best done in the CGI module, for consumption by any application that uses form upload. For example, doing cgi.FieldStorage(..., encoding="UTF-8") should cause a) decoding of every field that has an encoding= in its content type b) decoding of every field that is not a file to UTF-8. It is a file if it I) has a filename, or II) cannot be decoded to the target decoding For backwards compatibility, a) can only be enabled if the CGI application explicitly tells what encoding it expects. I'd like to state "contributions are welcome", although others may think differently. Regards, Martin From martin at v.loewis.de Wed Mar 30 22:56:13 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Mar 30 22:56:16 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <42414E0E.60808@livinglogic.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> Message-ID: <424B126D.9070602@v.loewis.de> Walter D?rwald wrote: > There's been a problem with your request > > exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in > position 92: ordinal not in range(128) That should be fixed now, please try again. Please report further errors you find to sf.net/projects/pypi. Suggestions/RFEs could go to the PyPI tracker, or here to python-dev. Regards, Martin From walter at livinglogic.de Wed Mar 30 23:38:41 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed Mar 30 23:38:45 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <424B126D.9070602@v.loewis.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> <424B126D.9070602@v.loewis.de> Message-ID: <424B1C61.8070306@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> There's been a problem with your request >> >> exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in >> position 92: ordinal not in range(128) > > That should be fixed now, please try again. Works perfectly, thanks! > [...] Bye, Walter D?rwald From ncoghlan at iinet.net.au Thu Mar 31 02:05:51 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 31 02:06:02 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> References: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> Message-ID: <424B3EDF.5010303@iinet.net.au> Michael Chermside wrote: > So I'm inclined to use different tools for modifying functions and > modifying classes because the ways you want to modify them are > different, and decorators are "tuned" to what people normally want > to do with functions (like simple wrapping) while metaclasses are > "tuned" to what people normally want to do with classes (like support > for inheritance. The area where I can see an overlap is in those decorators which, rather than altering the behaviour of the function itself, serve to register it with an external framework of some description. At the moment, a factory function that is actually implemented as a function can be registered with such a system using an @decorator. If you use the __new__/__init__ methods of a class as your factory, however, you need to either post-decorate the class or create a metaclass that performs the registration. It would be nice if decorators that worked for any callable (like the registration example) could be easily used with classes as well as standard functions. (The adaptation PEP's adapter registry is an actual situation that comes to mind) # A decorator that does not alter its argument def register(callable): # Register the callable somewhere ... return callable # Decorated factory function @register def factory(): pass # Post-decorated class class factory: pass register(factory) # Metaclass class RegisteredType(type): def __init__(self, name, bases, dict): super(self, RegisteredType).__init__(self, name, bases, dict) register(self) class factory: __metaclass__ = RegisteredType # Class decorator (not currently possible) @register class factory: pass PJE's example of moving the decoration near the top of the class definition without allowing class decoration contains an important caveat: it requires that the decorators be written to support doing that. Allowing class decoration means any appropriate decorators can be used, unmodified, to affect classes as well as functions. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From tdelaney at avaya.com Thu Mar 31 02:19:31 2005 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Mar 31 02:19:13 2005 Subject: [Python-Dev] @decoration of classes Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE02520488@au3010avexu1.global.avaya.com> Nick Coghlan wrote: > # A decorator that does not alter its argument > def register(callable): > # Register the callable somewhere > ... > return callable > > # Decorated factory function > @register > def factory(): > pass > > # Post-decorated class > class factory: > pass > register(factory) > > # Metaclass > class RegisteredType(type): > def __init__(self, name, bases, dict): > super(self, RegisteredType).__init__(self, name, bases, dict) > register(self) > > class factory: > __metaclass__ = RegisteredType > > # Class decorator (not currently possible) > @register > class factory: > pass class factory: @register def __call__(self): pass Just as an additional data point - obviously not applicable in all cases. Tim Delaney From ncoghlan at iinet.net.au Thu Mar 31 03:31:54 2005 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Mar 31 03:32:03 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE02520488@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE02520488@au3010avexu1.global.avaya.com> Message-ID: <424B530A.4060108@iinet.net.au> Delaney, Timothy C (Timothy) wrote: > class factory: > > @register > def __call__(self): > pass > > Just as an additional data point - obviously not applicable in all > cases. Yep, and it's obviously possible to do that now with just function decorators. Getting my head around what that actually *does* is a little trickier, though (as near as I can tell, it will register a standalone function named __call__, with nothing linking it to the containing class). What a class decorator allows is the registration of the whole process of invoking the metaclass and hence __new__ and __init__ as a single unit. Anyway, I don't particularly feel the lack of class decorators, but I thought I should mention this (perhaps small) category of non-transforming decorators as a situation where class decoration would make more sense to me than using a metaclass. Cheers, Nick. -- Nick Coghlan | ncoghlan@email.com | Brisbane, Australia --------------------------------------------------------------- http://boredomandlaziness.skystorm.net From pje at telecommunity.com Thu Mar 31 04:21:41 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Mar 31 04:17:49 2005 Subject: [Python-Dev] @decoration of classes In-Reply-To: <424B3EDF.5010303@iinet.net.au> References: <20050328092518.pkl2t03opmkg8sc0@mcherm.com> <20050328092518.pkl2t03opmkg8sc0@mcherm.com> Message-ID: <5.1.1.6.0.20050330211849.02e7a710@mail.telecommunity.com> At 10:05 AM 3/31/05 +1000, Nick Coghlan wrote: >PJE's example of moving the decoration near the top of the class >definition without allowing class decoration contains an important caveat: >it requires that the decorators be written to support doing that. Allowing >class decoration means any appropriate decorators can be used, unmodified, >to affect classes as well as functions. Yeah, but that can be trivially worked around using a 'decorate' function, e.g.: from protocols.advice import addClassAdvisor def decorate(*decorators): decorators = list(decorators)[::-1] def callback(cls): for dec in decorators: cls = cls(dec) return cls addClassAdvisor(callback) class SomeClass: decorate(dec1,dec2,...) From anthony at interlink.com.au Thu Mar 31 05:20:14 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Thu Mar 31 05:20:29 2005 Subject: [Python-Dev] BRANCH UNFROZEN Message-ID: <200503311320.15080.anthony@interlink.com.au> Ok, 2.4.1 seems good, so the release24-maint branch is unfrozen for bugfixes. We'll aim for a 2.4.2 around August/September, assuming no critical bugs come up before then... -- Anthony Baxter It's never too late to have a happy childhood. From walter at livinglogic.de Thu Mar 31 16:35:37 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Mar 31 16:35:43 2005 Subject: [Python-Dev] New PyPI broken package editing In-Reply-To: <424B0DF5.409@v.loewis.de> References: <424080D1.2030001@livinglogic.de> <1111525283.424087a3921d7@www.domainfactory-webmail.de> <42414E0E.60808@livinglogic.de> <4246CF82.2060500@v.loewis.de> <1751.84.56.104.245.1112132314.squirrel@isar.livinglogic.de> <4249D5E7.2020509@v.loewis.de> <424A90F6.50602@livinglogic.de> <424B0DF5.409@v.loewis.de> Message-ID: <424C0AB9.7060700@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> The register command in 2.4 (and current CVS) simply does a >> value = str(value) >> in post_to_server() so the encoded bytes sent depend on the >> default encoding. Would it be sufficient to change this to >> value = unicode(value).encode("utf-8") > > Indeed. I think this can go into 2.4.2. OK, I've checked this into HEAD and release24-maint (including the change to the Content-Type header). >> Another solution might be to include the encoding in the Content-type >> header of the request. IMHO the best solution would be to do both: >> Always use UTF-8 as the encoding and include this in the Content-type >> header in the request. PyPI should honor this encoding when it finds >> it and should fall back to whatever it used before if it doesn't. > > Yeah, well :-) Content-type in form upload is a mess, as you certainly > know. It should be honored, but commonly isn't. This, in turn, causes > browsers to ignore it. Fortunately we have both ends under control (except for old Python versions). > PyPI uses the CGI module. It currently decodes anything that doesn't > have a filename attribute to UTF-8, causing rejection of anything > that doesn't send UTF-8. This could be fixed/extended, but I think that > would be best done in the CGI module, for consumption by any application > that uses form upload. For example, doing > > cgi.FieldStorage(..., encoding="UTF-8") > > should cause > > a) decoding of every field that has an encoding= in its content > type > b) decoding of every field that is not a file to UTF-8. It is a > file if it > I) has a filename, or > II) cannot be decoded to the target decoding > > For backwards compatibility, a) can only be enabled if the CGI > application explicitly tells what encoding it expects. > > I'd like to state "contributions are welcome", although others > may think differently. OK, I'll see, if I can give this a try. Bye, Walter D?rwald From walter at livinglogic.de Thu Mar 31 16:45:08 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Mar 31 16:45:20 2005 Subject: [Python-Dev] Pickling instances of nested classes In-Reply-To: <4249E904.30808@strakt.com> References: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> <4249E904.30808@strakt.com> Message-ID: <424C0CF4.6040607@livinglogic.de> Samuele Pedroni wrote: > Walter D?rwald wrote: > >> [User cases for pickling instances of nested classes] >> So is this change wanted? useful? implementable with reasonable >> effort? Or >> just not worth it? > > notice that in this cases often metaclasses are involved or could easely > be, so if pickling would honor __reduce__ or __reduce_ex__ on > metaclasses (which right now it doesn't treating their instances as > normal classes) one could roll her own solution without the burden for > the language of implementing pickling of nested classes in general, so I > think that would make more sense, to add support to honor > __reduce__/__reduce_ex__ for metaclasses. Sorry, I don't understand: In most cases it can be possible to work around the nested classes problem by implementing custom pickling functionality (getstate/setstate/reduce/reduce_ex). But it is probably impossible to implement this once and for all in a common base class, because there's no way to find the real name of the nested class (or any other handle that makes it possible to retrieve the class from the module on unpickling). And having the full name of the class available would certainly help in debugging. Bye, Walter D?rwald From walter at livinglogic.de Thu Mar 31 16:52:56 2005 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Mar 31 16:52:58 2005 Subject: [Python-Dev] Pickling instances of nested classes In-Reply-To: <4249DA28.7040904@v.loewis.de> References: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> <4249DA28.7040904@v.loewis.de> Message-ID: <424C0EC8.6040607@livinglogic.de> Martin v. L?wis wrote: > Walter D?rwald wrote: > >> So is this change wanted? useful? implementable with reasonable >> effort? Or >> just not worth it? > > I think it is just not worth it. This means I won't attempt to implement > it. I think I defined originally the __module__ attribute for classes to > support better pickling (and defined it to be a string to avoid cycles); > we considered the nested classes case at the time and concluded "oh > well, don't do that then". This sounds like this change would have a far greater chance of getting into Python if a patch existed (I guess, this is the case for all changes ;)). Bye, Walter D?rwald From jepler at unpythonic.net Thu Mar 31 17:37:27 2005 From: jepler at unpythonic.net (Jeff Epler) Date: Thu Mar 31 17:37:33 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> Message-ID: <20050331153724.GA5083@unpythonic.net> I get 500 Internal Server Error messages when I try to access the URLs in the recent patch summary. Is this happening to anybody else? Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050331/61ebff20/attachment.pgp From phd at mail2.phd.pp.ru Thu Mar 31 17:54:53 2005 From: phd at mail2.phd.pp.ru (Oleg Broytmann) Date: Thu Mar 31 17:54:54 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <20050331153724.GA5083@unpythonic.net> References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> <20050331153724.GA5083@unpythonic.net> Message-ID: <20050331155453.GA23301@phd.pp.ru> On Thu, Mar 31, 2005 at 09:37:27AM -0600, Jeff Epler wrote: > I get 500 Internal Server Error messages when I try to access the URLs > in the recent patch summary. > > Is this happening to anybody else? Just visited http://python.org/sf/754022 to test - no problems, normal redirect to the SF tracker. Can you show an URL that's not working? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From kbk at shore.net Thu Mar 31 18:37:40 2005 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Mar 31 18:38:03 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <20050331153724.GA5083@unpythonic.net> (Jeff Epler's message of "Thu, 31 Mar 2005 09:37:27 -0600") References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> <20050331153724.GA5083@unpythonic.net> Message-ID: <87ll8393fv.fsf@hydra.bayview.thirdcreek.com> Jeff Epler writes: > I get 500 Internal Server Error messages when I try to access the URLs > in the recent patch summary. Yes, it seems that the python.org/sf/#### special facility is having a problem. IDs over 1 100 000 or so don't work. I sent a message to webmaster@python.org since I don't know who wrote that code. -- KBK From skip at pobox.com Thu Mar 31 20:40:27 2005 From: skip at pobox.com (Skip Montanaro) Date: Thu Mar 31 20:40:32 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <20050331153724.GA5083@unpythonic.net> References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> <20050331153724.GA5083@unpythonic.net> Message-ID: <16972.17435.367963.519680@montanaro.dyndns.org> >>>>> "Jeff" == Jeff Epler writes: Jeff> I get 500 Internal Server Error messages when I try to access the Jeff> URLs in the recent patch summary. Jeff> Is this happening to anybody else? Yup. I don't have time to look into the problem, however... Here's a traceback: Traceback (most recent call last): File "/usr/local/apache/cgi-bin/sf", line 82, in ? log_type(report, tracker) File "/usr/local/apache/cgi-bin/sf", line 42, in log_type os.unlink(idsfile+".lck") OSError: [Errno 2] No such file or directory: '/tmp/sourceforge-ids.txt.lck' and here's the script. It looks like lock file creation is failing but log_type() isn't returning, so in the finally clause deletion fails. Skip -------------- next part -------------- A non-text attachment was scrubbed... Name: sf Type: application/octet-stream Size: 2360 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050331/c7bdd605/sf.obj From gvanrossum at gmail.com Thu Mar 31 21:54:39 2005 From: gvanrossum at gmail.com (Guido van Rossum) Date: Thu Mar 31 21:54:42 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <87ll8393fv.fsf@hydra.bayview.thirdcreek.com> References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> <20050331153724.GA5083@unpythonic.net> <87ll8393fv.fsf@hydra.bayview.thirdcreek.com> Message-ID: I forwarded this to MvL, who wrote the code; he'll look into it but probably not before Sunday. On Thu, 31 Mar 2005 11:37:40 -0500, Kurt B. Kaiser wrote: > Jeff Epler writes: > > > I get 500 Internal Server Error messages when I try to access the URLs > > in the recent patch summary. > > Yes, it seems that the python.org/sf/#### special facility is having a > problem. IDs over 1 100 000 or so don't work. I sent a message to > webmaster@python.org since I don't know who wrote that code. > > -- > KBK > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pedronis at strakt.com Thu Mar 31 21:57:19 2005 From: pedronis at strakt.com (Samuele Pedroni) Date: Thu Mar 31 21:55:49 2005 Subject: [Python-Dev] Pickling instances of nested classes In-Reply-To: <424C0CF4.6040607@livinglogic.de> References: <1752.84.56.104.245.1112132476.squirrel@isar.livinglogic.de> <4249E904.30808@strakt.com> <424C0CF4.6040607@livinglogic.de> Message-ID: <424C561F.5060409@strakt.com> Walter D?rwald wrote: > Samuele Pedroni wrote: > >> Walter D?rwald wrote: >> >>> [User cases for pickling instances of nested classes] >>> So is this change wanted? useful? implementable with reasonable >>> effort? Or >>> just not worth it? >> >> >> notice that in this cases often metaclasses are involved or could >> easely be, so if pickling would honor __reduce__ or __reduce_ex__ on >> metaclasses (which right now it doesn't treating their instances as >> normal classes) one could roll her own solution without the burden for >> the language of implementing pickling of nested classes in general, so >> I think that would make more sense, to add support to honor >> __reduce__/__reduce_ex__ for metaclasses. > > > Sorry, I don't understand: In most cases it can be possible to > work around the nested classes problem by implementing custom pickling > functionality (getstate/setstate/reduce/reduce_ex). But it is probably > impossible to implement this once and for all in a common base class, > because there's no way to find the real name of the nested class (or any > other handle that makes it possible to retrieve the class from the > module on unpickling). > > And having the full name of the class available would certainly help in > debugging. > that's probably the only plus point but the names would be confusing wrt modules vs. classes. My point was that enabling reduce hooks at the metaclass level has propably other interesting applications, is far less complicated than your proposal to implement, it does not further complicate the notion of what happens at class creation time, and indeed avoids the implementation costs (for all python impls) of your proposal and still allows fairly generic solutions to the problem at hand because the solution can be formulated at the metaclass level. If pickle.py is patched along these lines [*] (strawman impl, not much tested but test_pickle.py still passes, needs further work to support __reduce_ex__ and cPickle would need similar changes) then this example works: class HierarchMeta(type): """metaclass such that inner classes know their outer class, with pickling support""" def __new__(cls, name, bases, dic): sub = [x for x in dic.values() if isinstance(x,HierarchMeta)] newtype = type.__new__(cls, name, bases, dic) for x in sub: x._outer_ = newtype return newtype def __reduce__(cls): if hasattr(cls, '_outer_'): return getattr, (cls._outer_, cls.__name__) else: return cls.__name__ # uses the HierarchMeta metaclass class Elm: __metaclass__ = HierarchMeta def __init__(self, **stuff): self.__dict__.update(stuff) def __repr__(self): return "<%s %s>" % (self.__class__.__name__, self.__dict__) # example class X(Elm): class Y(Elm): pass class Z(Elm): pass import pickle x = X(a=1) y = X.Y(b=2) z = X.Z(c=3) xs = pickle.dumps(x) ys = pickle.dumps(y) zs = pickle.dumps(z) print pickle.loads(xs) print pickle.loads(ys) print pickle.loads(zs) pedronis$ python2.4 example.py [*]: --- pickle.py.orig Wed Mar 30 20:37:14 2005 +++ pickle.py Thu Mar 31 21:09:41 2005 @@ -298,12 +298,19 @@ issc = issubclass(t, TypeType) except TypeError: # t is not a class (old Boost; see SF #502085) issc = 0 + reduce = None if issc: - self.save_global(obj) - return + for x in t.__mro__: + if x is not object and '__reduce__' in x.__dict__: + reduce = x.__dict__['__reduce__'] + break + else: + self.save_global(obj) + return # Check copy_reg.dispatch_table - reduce = dispatch_table.get(t) + if not reduce: + reduce = dispatch_table.get(t) if reduce: rv = reduce(obj) else: From martin at v.loewis.de Thu Mar 31 22:01:09 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Mar 31 22:01:12 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <16972.17435.367963.519680@montanaro.dyndns.org> References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> <20050331153724.GA5083@unpythonic.net> <16972.17435.367963.519680@montanaro.dyndns.org> Message-ID: <424C5705.5040706@v.loewis.de> Skip Montanaro wrote: > Here's a traceback: > > Traceback (most recent call last): > File "/usr/local/apache/cgi-bin/sf", line 82, in ? > log_type(report, tracker) > File "/usr/local/apache/cgi-bin/sf", line 42, in log_type > os.unlink(idsfile+".lck") > OSError: [Errno 2] No such file or directory: '/tmp/sourceforge-ids.txt.lck' > > and here's the script. It looks like lock file creation is failing but > log_type() isn't returning, so in the finally clause deletion fails. Somebody took "world" write permission on /tmp away; I don't know whether this was intentional - I have restored them for a moment. The script actually has two errors: it is not os.sleep, but time.sleep (so it would only try one time, not ten times); plus (and I'm glad you did not catch that, either :-) the finally block is always executed, even if the file was not created, and even if the function is left through return. I've fixed the script, however, if somebody takes away o+w on /tmp again, it will mean that the caching silently stops. Regards, Martin From martin at v.loewis.de Thu Mar 31 22:05:25 2005 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu Mar 31 22:05:28 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> <20050331153724.GA5083@unpythonic.net> <87ll8393fv.fsf@hydra.bayview.thirdcreek.com> Message-ID: <424C5805.8070307@v.loewis.de> Guido van Rossum wrote: > I forwarded this to MvL, who wrote the code; he'll look into it but > probably not before Sunday. Actually, now that I saw that it is a permission problem, it turned out to be fixable quite easily. Regards, Martin From python at rcn.com Thu Mar 31 23:00:19 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu Mar 31 23:05:19 2005 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Lib/logging handlers.py, 1.19, 1.19.2.1 In-Reply-To: Message-ID: <003101c53634$a60b87e0$d2bc958d@oemcomputer> > Tag: release24-maint > handlers.py > Log Message: > Added optional encoding argument to File based handlers and improved error > handling for SysLogHandler Are you sure you want to backport an API change and new feature? Raymond From jepler at unpythonic.net Thu Mar 31 23:25:38 2005 From: jepler at unpythonic.net (Jeff Epler) Date: Thu Mar 31 23:25:42 2005 Subject: python.org/sf URLs aren't working? Re: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <20050331153724.GA5083@unpythonic.net> References: <200503300208.j2U28tNw012327@bayview.thirdcreek.com> <20050331153724.GA5083@unpythonic.net> Message-ID: <20050331212537.GD10036@unpythonic.net> It's working again for me now. thanks! Jeff -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20050331/8ada5814/attachment.pgp