From martin at v.loewis.de Sun Feb 1 14:52:47 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Feb 1 14:58:22 2004 Subject: [Python-Dev] Compiling python with the free .NET SDK In-Reply-To: <400C4321.2040302@garthy.com> References: <4009A105.30303@garthy.com> <400AF3D9.2000508@v.loewis.de> <400C4321.2040302@garthy.com> Message-ID: <401D590F.6090304@v.loewis.de> Garth wrote: > Thanks for the build logs. Very useful and I can now build the whole of > python with the free microsoft tools and GNU make. Very good! If you are ready to release it, feel free to submit a patch to SF. > It appears that you still have a dependency on 'largeint.lib' in > pythoncore. This is no longer required and the largeint.h was removed a > in the move to VS.NET. (I think) When I remove it by hand editing the > file python still builds. Can you update the project files in cvs. Thanks for pointing that out. I just fixed it. Regards, Martin From martin at v.loewis.de Sun Feb 1 15:00:46 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Feb 1 15:01:31 2004 Subject: [Python-Dev] Switching to VC.NET 2003 In-Reply-To: <192001c3d41d$c5a71f30$2c00a8c0@eden> References: <192001c3d41d$c5a71f30$2c00a8c0@eden> Message-ID: <401D5AEE.7010001@v.loewis.de> Mark Hammond wrote: > In summary, we need to ensure that an arbitrary DLL with symbol references > to python24.dll can be loaded by an arbitrary process. Ok. I have now fixed the installer to put pythonxy.dll (and msvcr71.dll) into System32, and enabled library reference counting for it. I should remember to record your explanation in the MSI generator. Regards, Martin From kbk at shore.net Mon Feb 2 00:50:30 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Mon Feb 2 00:50:41 2004 Subject: [Python-Dev] Weekly Python Patch/Bug Summary Message-ID: <87n082uj15.fsf@hydra.localdomain> Patch / Bug Summary ___________________ Patches : 246 open ( +5) / 2300 closed ( +1) / 2546 total ( +6) Bugs : 685 open ( +8) / 3876 closed ( +6) / 4561 total (+14) RFE : 124 open ( +0) / 117 closed ( +1) / 241 total ( +1) New / Reopened Patches ______________________ Erroneous error message in test_types (2004-01-26) http://python.org/sf/885008 opened by Gerrit Holl _socket fails to build with latest SGI compiler (2004-01-27) http://python.org/sf/885905 opened by Greg Couch Installation database patch (2002-05-29) http://python.org/sf/562100 reopened by akuchling Bluetooth support for FreeBSD systems. (2004-01-31) http://python.org/sf/888148 opened by Hye-Shik Chang popen2.popen2() 'cmd' list syntax docstring (2004-01-31) http://python.org/sf/888380 opened by Jeff Connelly "van rossum" misspelled (2004-02-02) http://python.org/sf/888573 opened by George Yoshida cygwinccompiler should set C++ compiler (2004-02-02) http://python.org/sf/888839 opened by Seo Sanghyeon Patches Closed ______________ Bluetooth support for socket module (2004-01-09) http://python.org/sf/874083 closed by loewis "van rossum" misspelled (2004-02-01) http://python.org/sf/888573 closed by loewis New / Reopened Bugs ___________________ SIGTERM blocked in processes forked by threads (2004-01-25) http://python.org/sf/884533 opened by Steve Muir floating point precission (2004-01-26) http://python.org/sf/884598 opened by Johan Burati Incorrect ZeroDivisionError message with __future__ division (2004-01-26) http://python.org/sf/884841 opened by Gerrit Holl Two lines in object.c seem useless. (2004-01-26) http://python.org/sf/885293 opened by Josiah Carlson WinPython 2.3.3 crashes using popen2 to spawn lots of child (2004-01-28) http://python.org/sf/886488 opened by Frank Terhaar-Yonkers Python 2.2.3 crashes using popen2 to spawn lots of children (2004-01-28) http://python.org/sf/886492 opened by Frank Terhaar-Yonkers wrong documentation for popen2 (2004-01-28) http://python.org/sf/886619 opened by Serge Winitzki dom.Node.nodeValue not populated (2004-01-29) http://python.org/sf/886690 opened by Raide 2.3.3 str & list still use __getslice__ / __setslice__ (2004-01-29) http://python.org/sf/887042 opened by Florian B?sch "-framework Python" for building modules is bad (2004-01-29) http://python.org/sf/887242 opened by Bob Ippolito asyncore.py poll* missing urgent data (2004-01-29) http://python.org/sf/887279 opened by Jim Jewett segfault if redirecting directory (2004-01-31) http://python.org/sf/887946 opened by Inyeol Lee cPickle doesn't like protocol keyword argument, pickle does (2004-02-01) http://python.org/sf/888594 opened by Irmen de Jong FAQ 2.3 slices not used (2004-02-01) http://python.org/sf/888829 opened by Wayne Folta POP3 lib timeout (2004-02-01) http://python.org/sf/888830 opened by Wayne Folta Bugs Closed ___________ SIGTERM blocked in processes forked by threads (2004-01-26) http://python.org/sf/884533 closed by jackjansen floating point precission (2004-01-26) http://python.org/sf/884598 closed by rhettinger irreproducable segfault on int(1e100) (2003-06-19) http://python.org/sf/757520 closed by gerrit optparser help formatting nit (2003-11-14) http://python.org/sf/842213 closed by fdrake Two lines in object.c seem useless. (2004-01-26) http://python.org/sf/885293 closed by bcannon dom.Node.nodeValue not populated (2004-01-29) http://python.org/sf/886690 closed by raide cPickle doesn't raise error, pickle does (UnpicklingError) (2003-06-09) http://python.org/sf/751279 closed by irmen New / Reopened RFE __________________ Machine integers (2004-01-29) http://python.org/sf/887237 opened by Fredrik Johansson RFE Closed __________ Enable command line exit in IDLE. (2004-01-22) http://python.org/sf/882171 closed by rhettinger From FBatista at uniFON.com.ar Mon Feb 2 10:15:13 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Mon Feb 2 10:17:19 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type Message-ID: Stephen Horne writes: #- Good point. A 'DecimalExact' or similar function could perhaps be #- provided to replace the simple conversion when people have really #- thought about it and do really want it. So, making it short: When you have som rounding, it's ok to have a second parameter: Decimal(1.1, 2) == Decimal('1.1') I proposed that without the second parameter, the conversion to be exact: Decimal(1.1) == Decimal('110000000000000008881784197001252...e-51') Michael Chermside proposed to have the second parameter mandatory: avoids the problem of a newbie forgetting it, and if you want an exact conversion you can always put a very large number (e.g.: 60). If the second parameter will be mandatory, I prefer a non sense number there, for example: 0. And Stephen Horne proposed to have an explicit method to do the exact conversion, kind of: Decimal.fromFloat(1.1) == Decimal('110000000000000008881784197001252...e-51') So, we need to choose one of three ways to obtain an exact conversion: 1. Without a second parameter. 2. With a second parameter "big enough" or a special value (0? -1?) 3. With another method. Name? Thank you all! . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040202/6687a5e6/attachment.html From FBatista at uniFON.com.ar Mon Feb 2 10:34:42 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Mon Feb 2 10:36:49 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type Message-ID: Michael Chermside wrote: #- I think that we'd do even better to ommit the second #- use. People who #- really want to convert floats exactly can easily write #- "Decimal(1.1, 60)". But #- hardly anyone wants to convert floats exactly, while lots of #- newbies would #- forget to include the second parameter. I'd say just make #- Decimal(someFloat) #- raise a TypeError with a helpful message about how you need #- that second #- parameter when using floats. I take this in consideration in another response. #- Really, that's all I came up with. This is great, and I'm #- looking forward to #- using it. I would, though, be interested in a couple more #- syntax-related #- details: #- (a) What's the syntax for changing the context? I'd #- think we'd want #- a "pushDecimalContext()" and "popDecimalContext()" sort of #- approach, since most #- well-behaved routines will want to restore their caller's context. You can save a copy of the context in a variable and then restore it. No stack. #- (b) How about querying to determine a thread's current #- context? I don't #- have any use cases, but it would seem peculiar not to provide it. You do it to save it in a variable. For example: >>> from Decimal import * >>> import copy >>> c = getcontext() >>> cc = copy.copy(c) >>> d = Decimal(5.065253542) >>> d Decimal( (0, (5, 0, 6, 5, 2, 5, 3, 5, 4, 2), -9) ) >>> d + 1 # it will be rounded down Decimal( (0, (6, 0, 6, 5, 2, 5, 3, 5, 4), -8L) ) >>> c.set_rounding('up') # returns the old rounding 'half_even' >>> d + 1 # it will be rounded up Decimal( (0, (6, 0, 6, 5, 2, 5, 3, 5, 5), -8L) ) >>> setcontext(cc) >>> d + 1 Decimal( (0, (6, 0, 6, 5, 2, 5, 3, 5, 4), -8L) ) One question: This kind of stuff, should be in the final documentation? And in the final PEP? #- (c) Given a Decimal object, is there a straightforward #- way to determine its #- coefficient and exponent? Methods named .precision() and #- .exponent() might do #- the trick. No. I think it's a good idea. . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040202/bbc39174/attachment.html From Andrew at MacKeith.net Mon Feb 2 07:36:27 2004 From: Andrew at MacKeith.net (Andrew MacKeith) Date: Mon Feb 2 11:09:05 2004 Subject: [Python-Dev] Proposed: change to posting rules for python-dev References: <1075498254.1075.243.camel@anthem> <401AE3C3.2040203@comcast.net> Message-ID: <401E444B.9090800@MacKeith.net> Chris and Ann Reedy wrote: > Barry Warsaw wrote: > > Barry - > > I recently subscribed to the list after reading from the web site for > several months. I wonder if the proposed policy will end up cutting off > postings from people who follow the list but aren't otherwise subscibed, > and if that's good or bad. > > Chris I'm in the same position; I recently subscribed after reading the list for a while, and finding a subject to which I could contribute. Andrew > >> There's been some talk off-list about changing the posting rules for >> python-dev, such that non-member post attempts are automatically >> rejected (i.e. bounced). This means you'd have to be a member to post >> to the list. I think that's not an inappropriate rule for python-dev. >> >> taking-the-pulse-of-the-list-ly y'rs, >> -Barry >> > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/mackeith%40acm.org > From barry at python.org Mon Feb 2 11:09:38 2004 From: barry at python.org (Barry Warsaw) Date: Mon Feb 2 11:09:43 2004 Subject: [Python-Dev] Re: Proposed: change to posting rules for python-dev In-Reply-To: References: <877jz90wx9.fsf@hydra.localdomain> <1075512752.1075.350.camel@anthem> Message-ID: <1075738177.21821.421.camel@anthem> Okay folks, thanks for the offer of help. We've got enough volunteers now, so we'll continue to allow non-members to post. -Barry From jim.jewett at eds.com Mon Feb 2 13:27:04 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Mon Feb 2 13:27:17 2004 Subject: [Python-Dev] thread-safe generator in standard library Message-ID: In comp.lang.python, Andrae Muys started a discussion about how to use generators in a thread-safe manner. Newsgroup Subjects: Suggested generator to add to threading module. Accessing a shared generator from multiple threads There were several suggestions (some buggy) about how to implement it, and some questions about whether it was worth doing. Aahz asked why you would want to do it. The most obvious use case is to generate unique keys (as lisp gensym). The standard library currently has several ways to create temporary filenames. Unfortunately, they aren't all threadsafe, they often enforce the "temporary" aspect, they eventually run into hashing collisions, there is no good way to include ordering information, etc. The fact that these are in the standard library suggests that it is a common use case. The fact that there are several different versions each with their own drawbacks suggests that the problem is hard enough to justify putting a good solution in the library. At that point, Aahz suggested moving the discussion here, so ... I am doing so. Specific questions: (1) Which library? The tempfile libraries are an immediate use, but they are not the only use. The problem only comes up with threads, but it isn't an integral part of threading. itertools might be appropriate, but I'm not sure people will look to "tools" when dealing with "threads". It is also a bit different from the other tools. It could be its own module, but that adds more overhead. It could be part of the queue module, since a queue interface provides one of the solutions. (Override _get() to use iter.next() while already inside the threadlock. But what should be done about "put" and "qsize"? Should the user be able to check "empty()" without consuming a value or getting a StopIteration?) (2) Are there requirements on the implementation beyond "it works"? Does it need to be fast? Use/ avoid certain extensions? (3) What API? Should it provide a default iterator, or require that one be passed in? If one is passed in, does can/should this module assume that it is the only direct consumer, or does it need to worry about someone else calling .next() on the original generator? -jJ From tim.one at comcast.net Mon Feb 2 13:57:23 2004 From: tim.one at comcast.net (Tim Peters) Date: Mon Feb 2 13:57:27 2004 Subject: [Python-Dev] thread-safe generator in standard library In-Reply-To: Message-ID: [Jewett, Jim J] > ... > The most obvious use case is to generate unique keys > (as lisp gensym). Just noting a practical hack that's often sufficient: import sys genunique = iter(xrange(sys.maxint)).next Then each call to genunique() delivers "the next" (short) integer, and it inherits thread safety from the global interpreter lock. A similar effect can be gotten via import itertools genunique = itertools.count().next and that also inherits thread safety from the GIL. A difference is that the xrange spelling stops when it reaches sys.maxint, but the .count spelling silently wraps around to -sys.maxint-1 then (undetected overflow at the C level). From pf_moore at yahoo.co.uk Mon Feb 2 14:06:14 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Mon Feb 2 14:06:30 2004 Subject: [Python-Dev] Re: Draft: PEP for imports References: <20040119214528.GA9767@panix.com> <200401252159.i0PLxGj18366@c-24-5-183-134.client.comcast.net> <20040130133253.GA6338@vicky.ecs.soton.ac.uk> <401AE1FD.7050809@comcast.net> Message-ID: <3c9t8fo9.fsf@yahoo.co.uk> Chris Reedy writes: > Thinking about the semantics of the above, including corner cases, I > find myself wanting to make the semantics of: > > in "" > > to be equivalent to: > > sys.path.insert(0, "") > > del sys.path[0] # assuming the import doesn't modify sys.path This seems to me to be another example of the sort of setup/teardown idiom that PEP 310 was designed for: with path_entry("): import whatever Here, we have class path_entry def __init(self, path): self.path = path def __enter__(self): sys.path.insert(0, path) def __exit__(self): del sys.path[0] ... and of course, path_entry might be a suitable candidate for some sort of standard library module of useful "with" idioms. Hmm, maybe it's time to look at PEP 310 again. Paul -- This signature intentionally left blank From FBatista at uniFON.com.ar Mon Feb 2 17:00:56 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Mon Feb 2 17:06:29 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type Message-ID: Michael Chermside wrote: #- Imagine the following senario. I write some code which uses #- Decimal, and which sets the context to get the desired rounding #- behavior. Somewhere in my code I call a routine written by #- somebody else... perhaps some library code... which also does #- some calculations using Decimal. This subject already has been discussed in c.l.p: Emile van Sebille: ...from which I infer that Context exists in part to limit/round calculated results. Even if it were possible for me, as a user of Decimal, to set Context appropriately to achieve these ends, what if I use a library that also changes Context? The integrity of my calculations may be impacted. Aahz: That's correct. There needs to be a social convention that libraries intended for use by other people *CANNOT* muck with Context, and if they do so for their own internal calculations, they must save and restore Context. You can probably find additional information about this in Cowlishaw. Emile van Sebille: Enter the context stack. Aahz: Well, sure. And it won't be hard to add given that Decimal will already need to track thread-specific Context. But modules changing Context will still need to explicitly push and pop Context because usually you will just want to replace the current Context. Tim Peters: They have another choice, because Guido had a brilliant idea: the arithmetic operations in Eric's implementation are methods *of* a context object (because Guido suggested that). So a maximally robust library doesn't *have* to change the thread context at all: it can create whatever private context object(s) it needs, and spell arithmetic as explicit method calls on its private context object(s), so that the default thread context object is neither consulted nor modified. This is very robust, and in small doses is quite bearable. I don't have any problem to implement a context stack. But I think that it's no clear to be useful, so I think it's better to go for the sure staff, and when Decimal gets heavy use, if everybody agrees to add a context stack, we'll go for it. What do you think? . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040202/f934dc5e/attachment.html From tim.one at comcast.net Mon Feb 2 17:26:20 2004 From: tim.one at comcast.net (Tim Peters) Date: Mon Feb 2 17:26:24 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type In-Reply-To: Message-ID: [Facundo Batista] > ... > I don't have any problem to implement a context stack. But I think > that it's no clear to be useful, so I think it's better to go for the > sure staff, and when Decimal gets heavy use, if everybody agrees to > add a context stack, we'll go for it. What do you think? . YAGNI (You Ain't Gonna Need It). Most apps will never change the context. Most of the rest will set it once at program start (typically just to increase precision), and never change it again. The same is true of the elaborate FPU control registers in the Pentium HW most of the world uses today, and the "numeric context" here is just a small generalization of that. This isn't to say that numeric context isn't useful (or the elaborate FPU control registers): it's extremely useful, but generally only to a handful of numeric experts writing library utilities with severe "works in all possible endcases" requirements. They'll have the responsibility to document the effects their code has on context (when does it signal inexact? overflow? etc), and because they will change context visible to the caller in the documented ways, they've got no use for a stack (they'll generally need to restore *parts* of the context to entry conditions, but change other parts in defined ways -- same as, e.g., the builtin addition function). From mcherm at mcherm.com Mon Feb 2 15:42:12 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Mon Feb 2 18:55:57 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type Message-ID: <1075754532.401eb6248bd42@mcherm.com> I wrote: > (a) What's the syntax for changing the context? I'd > think we'd want > a "pushDecimalContext()" and "popDecimalContext()" sort of > approach, since most > well-behaved routines will want to restore their caller's context. Facundo Batista replies: > You can save a copy of the context in a variable and then > restore it. No stack. Imagine the following senario. I write some code which uses Decimal, and which sets the context to get the desired rounding behavior. Somewhere in my code I call a routine written by somebody else... perhaps some library code... which also does some calculations using Decimal. It would be a major pain if every time I called that library code it changed the context. I'd have to keep setting the context back after each and every call to the library code. A well-behaved library will not require me to go through these contortions, so after it does its work it will set the context back to what it was when I called it. It seems to me that in almost all situations where one changes the context, one will also want to restore it later. If so, then it seems perverse to tell every user that they must capture and store the old context someplace and restore it later. Why not make the best practice be the *obvious* idiom by providing matched pushContext() and popContext() functions as the normal way of setting the context. Of course, one could choose to call pushContext() without a matching popContext(), or call popContext() repeatedly until the stack is empty (some exception is raised), but if the best behavior is the easy behavior then most people will use it. And if that didn't convince you, then go ahead... I think it's only a minor detail. -- Michael Chermside From tim.one at comcast.net Mon Feb 2 19:28:55 2004 From: tim.one at comcast.net (Tim Peters) Date: Mon Feb 2 19:29:00 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type In-Reply-To: <1075754532.401eb6248bd42@mcherm.com> Message-ID: [Michael Chermside] > Imagine the following senario. I write some code which uses > Decimal, and which sets the context to get the desired rounding > behavior. Somewhere in my code I call a routine written by > somebody else... perhaps some library code... which also does > some calculations using Decimal. > > It would be a major pain if every time I called that library code > it changed the context. I'd have to keep setting the context back > after each and every call to the library code. A well-behaved > library will not require me to go through these contortions, so > after it does its work it will set the context back to what it was > when I called it. A well-designed library will restore only *part* of the context. Some of the state in numeric context controls arithmetic behavior, but other parts are informational, telling you about exceptional conditions that occurred. The user can choose to let exceptions be raised, or to suppress them (via settings in numeric context). In the latter case, knowledge that an exceptional condition occurred is *recorded* in context. A well-designed library can't assume that the user wants exceptions to be raised, but must inform a user running in "non-stop" mode that exceptional conditions occurred. Context is the way to do that; context holds both input and output state. It's difficult to write high-quality numeric libraries. The facilities offered by numeric context are a great aid in doing so, but it's still too complex for simple gimmicks like "a stack" to help. The burden is on code that changes context to do so correctly from its caller's POV; if you call a routine that makes a mess of your context, that's an error in the routine you called. In the same vein, if you write a routine that requires a specific rounding mode, and don't arrange to force that rounding mode for the duration of your routine, your routine is in error. It's also in error if it doesn't restore the rounding mode in effect when it was entered. It's also probably in error if it *does* blindly restore the settings of the informational flags. From greg at cosc.canterbury.ac.nz Mon Feb 2 19:46:08 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Feb 2 19:46:30 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type In-Reply-To: Message-ID: <200402030046.i130k8R11903@oma.cosc.canterbury.ac.nz> Tim Peters : > A well-designed library will restore only *part* of the context. Some of > the state in numeric context controls arithmetic behavior, but other parts > are informational, telling you about exceptional conditions that > occurred. That suggests there shouldn't be a single context object, but two separate objects, one for modes and one for exception flags. The modes object could then be saved and restored without affecting the flags. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From tim.one at comcast.net Mon Feb 2 20:04:55 2004 From: tim.one at comcast.net (Tim Peters) Date: Mon Feb 2 20:05:00 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type In-Reply-To: <200402030046.i130k8R11903@oma.cosc.canterbury.ac.nz> Message-ID: [Greg Ewing] > That suggests there shouldn't be a single context object, > but two separate objects, one for modes and one for exception > flags. The modes object could then be saved and restored > without affecting the flags. Possibly. I'm more interested to see us adhere as closely as possible to the relevant standards at the start; there's plenty to be done without piling on speculative "convenience features": http://www2.hursley.ibm.com/decimal/ From bsder at allcaps.org Mon Feb 2 20:46:00 2004 From: bsder at allcaps.org (Andrew P. Lentvorski, Jr.) Date: Mon Feb 2 20:38:39 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type In-Reply-To: <1075754532.401eb6248bd42@mcherm.com> References: <1075754532.401eb6248bd42@mcherm.com> Message-ID: <20040202172808.R33933@mail.allcaps.org> On Mon, 2 Feb 2004, Michael Chermside wrote: > It would be a major pain if every time I called that library code > it changed the context. That would be badly behaved library code. One thing that might be an issue is that context stacks tend to be real headaches in threaded code. The context stack must unwind in the "pushed" order even though the "completion" order may be very different. Such a stack would make the ordering of calls to the Decimal module subject to strange errors. Another thing that might come up is that a context could be a microprocessor resource. Fully restoring such a resource can often be very expensive and may not even be possible. -a From anthony at interlink.com.au Tue Feb 3 08:20:41 2004 From: anthony at interlink.com.au (Anthony Baxter) Date: Tue Feb 3 08:21:03 2004 Subject: [Python-Dev] release23-maint is reopened for business. Message-ID: <20040203132041.8C84125B4BA@bonanza.off.ekorp.com> The release23-maint branch (for bugfixes on the 2.3 train of code) is now open for your bugfixing pleasure. My next (planned) release off that branch will be May-June, assuming Tim doesn't create^H^H^H^H^H "find" any more critical bugs before then. (Oh sure, he'll claim that the bugs were there already...) ta, Anthony From guido at python.org Tue Feb 3 10:00:37 2004 From: guido at python.org (Guido van Rossum) Date: Tue Feb 3 10:00:47 2004 Subject: [Python-Dev] OSCON Reminder: proposal deadline Februrary 9 Message-ID: <200402031500.i13F0ct19651@c-24-5-183-134.client.comcast.net> Just a quick reminder: you have another week to submit a proposal for the Python 12 Conference at OSCON: The O'Reilly Open Source Convention (OSCON) will be held July 26-30, 2004 at the Portland Marriott Downtown in Portland, OR. http://conferences.oreillynet.com/os2004/ Proposals Submission Information--Deadline: February 9, 2004 http://conferences.oreillynet.com/cs/os2004/create/e_sess --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one at comcast.net Tue Feb 3 10:47:35 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 3 10:47:37 2004 Subject: [Python-Dev] release23-maint is reopened for business. In-Reply-To: <20040203132041.8C84125B4BA@bonanza.off.ekorp.com> Message-ID: [Anthony Baxter] > The release23-maint branch (for bugfixes on the 2.3 train of > code) is now open for your bugfixing pleasure. Yay! > My next (planned) release off that branch will be May-June, > assuming Tim doesn't create^H^H^H^H^H "find" any more critical > bugs before then. > > (Oh sure, he'll claim that the bugs were there already...) Alas, Jim Fulton already did: in typeobject.c's reduce_2(), if the object doesn't have __getstate__ but does have __dict__, the AttributeError from failing to find __getstate__ isn't cleared, so ceval ends up with a pending exception set after a call that returns normally. That's Bad (an utterly mysterious AttributeError gets raised after some later opcode notices the pending exception). That bug is in all flavors of 2.3, and on the head -- alas, there's no sane workaround for something like this. From jordanhaven at hotmail.com Tue Feb 3 16:44:18 2004 From: jordanhaven at hotmail.com (Jordan Haven) Date: Tue Feb 3 16:51:29 2004 Subject: [Python-Dev] Proposed addition to ftplib Message-ID: Hey Python developers! First, since this is my first message to this group, let me introduce myself. My name is Jordan Haven, I'm 16, I live just outside of DC and have been programming for only a few years (~ 2 years). Python was the first language I started, and I have loved it since the beginning. Now, onto the real purpose of this email. Recently, while creating a CLI FTP program as a precursor to my GUI version (SF project: ftpy), I ran into a roadblock: I couldn't change permissions on the remote file. So I searched the comp.lang.python newsgroup, the python-list archives, these development archives, and countless ftplib tutorials and books and couldn't find anything on chmod'ing files through FTP with the standard ftplib module. So I went through ftplib.py (Python 2.3), and there was no function for chmod'ing files. I think (and hope you all will agree with me :)) that there needs to be a chmod funtcion included in the ftplib module. So I took the liberty to write one up (see attached). It works fine on my box (Windows 2000 with Python 2.3), and there aren't any obvious backwards compatibility problems that I can see, so the only thing left is to test crossplatform portability. However, looking at the code there doesn't seem to be any issues with that, but if some of you could check it out on various other platforms, I would be most appreciative. Hopefully it (or a modified version) will make it into the next release of Python. Well, thank you for your time, and for taking a look at the included function. I look forward to hearing back from you. -- Jordan Haven _________________________________________________________________ Find high-speed ‘net deals — comparison-shop your local providers here. https://broadband.msn.com -------------- next part -------------- #ftplib.py function chmod # #Author: Jordan Haven # #This function should be added to the FTP class in the ftplib.py module # #Usage: From within an FTP program, after a connection has been made. Like so: # # ftp_host = raw_input("Host Name: ") # ftp_port = input("Port Number: ") # ftp_user = raw_input("User Name: ") # ftp_pass = getpass.getpass("Password: ") # ftp_acct = raw_input("Account (Leave blank if none): ") # if ftp_user and ftp_pass == "": # ftp = FTP(ftp_host) # ftp.login() # print "Thank you! FTPy is now attempting to connect to %s on port %s"% (ftp_host, ftp_port) # ftp = FTP(ftp_host) # ftp.connect(ftp_host, ftp_port) # ftp.login(ftp_user, ftp_pass, ftp_acct) # welcome = ftp.getwelcome() # print welcome # def ftpmain(): # cmd = raw_input("%s@%s: " %(user, ftp_host)) # ... code for functions like pwd and cwd, etc. ... # # elif cmd == 'chmod': # file = raw_input("File to change permissions: ") # mode = raw_input("CHMOD value: ") # ftp.chmod(mode, file) # ftpmain() # # ... other code ... # #The above is an example from the FTP program I'm making and is obviously only an example #of the implementation. # #Something to note: 'mode' is a string, not an int so it can be passed as a single command #by the function. # #And now, the actual function: def chmod(self, mode, file): '''Change file permissions for a file on the remote server.''' return self.voidcmd('SITE CHMOD ' + mode + ' ' + file) From gerrit at nl.linux.org Tue Feb 3 17:12:47 2004 From: gerrit at nl.linux.org (Gerrit Holl) Date: Tue Feb 3 17:19:44 2004 Subject: [Python-Dev] Proposed addition to ftplib In-Reply-To: References: Message-ID: <20040203221247.GA7246@nl.linux.org> Jordan Haven wrote: > Hopefully it (or a modified version) will make it into the next release of > Python. There is a SF area for Python patches. Sending it there raises the chance that developers will have a look at it: http://sourceforge.net/tracker/?group_id=5470&atid=305470 yours, Gerrit. PS two years ago, I was also a 16-year-old programmer with 2 years of Python programming experience and Python as a first language and I loved it :-) This situation is still true except that I'm 18 and have 4 years of experience. I am really not a Python developer but just happen to read the Python development mailing list ;-) -- PrePEP: Builtin path type http://people.nl.linux.org/~gerrit/creaties/path/pep-xxxx.html Asperger's Syndrome - a personal approach: http://people.nl.linux.org/~gerrit/english/ From barry at python.org Tue Feb 3 17:37:25 2004 From: barry at python.org (Barry Warsaw) Date: Tue Feb 3 17:37:31 2004 Subject: [Python-Dev] Proposed addition to ftplib In-Reply-To: <20040203221247.GA7246@nl.linux.org> References: <20040203221247.GA7246@nl.linux.org> Message-ID: <1075847844.482.110.camel@anthem> On Tue, 2004-02-03 at 17:12, Gerrit Holl wrote: > two years ago, I was also a 16-year-old programmer with 2 years of > Python programming experience and Python as a first language and I loved > it :-) This situation is still true except that I'm 18 and have 4 years > of experience. I am really not a Python developer but just happen to > read the Python development mailing list ;-) Thank you both for making me feel so, um, mature. at-least-tim's-still-older-than-me-ly y'rs, -Barry From jepler at unpythonic.net Tue Feb 3 20:24:58 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Tue Feb 3 20:26:04 2004 Subject: [Python-Dev] Proposed addition to ftplib In-Reply-To: References: Message-ID: <20040204012457.GA387@unpythonic.net> Jordan, It's always good to see people contributing to Python. It's good to have you here. I have two suggestions about your submission. Well, three. Zeroeth, take the advice of another respondant and use the Sourceforge patch submission interface. First, when contributing patches to Python, it's most customary to use the Unix tools 'diff' and 'patch' to generate a listing of the changes you made. (If you're a Windows user, perhaps someone can be more helpful than the FAQ entry at http://www.python.org/dev/devfaq.html#how-to-make-a-patch) Here's the patch for your change: --- /usr/lib/python2.2/ftplib.py 2003-02-24 18:24:16.000000000 -0600 +++ ./ftplib.py 2004-02-03 19:03:07.000000000 -0600 @@ -469,6 +469,10 @@ cmd = cmd + (' ' + arg) self.retrlines(cmd, func) + def chmod(self, mode, file): + '''Change file permissions for a file on the remote server.''' + return self.voidcmd('SITE CHMOD ' + mode + ' ' + file) + def rename(self, fromname, toname): '''Rename a file.''' resp = self.sendcmd('RNFR ' + fromname) Second, and this may seem foolish because of how little I know about the FTP protocol, I wonder if it makes sense to make the SITE command a method, and have chmod use it. This patch uses that idea, and makes it even easier for you to use more SITE commands easily (well, if it works): >>> f = ftplib.FTP(...) >>> f.login(...) >>> print f.site('help') 214-The following SITE commands are recognized. UMASK GROUPS IDLE CHECKMETHOD CHMOD ALIAS CHECKSUM HELP CDPATH 214 Direct comments to root@localhost. >>> f.site('chmod', '644', '/etc/shadow') ftplib.error_perm: 550 /etc/shadow: Operation not permitted. --- /usr/lib/python2.2/ftplib.py 2003-02-24 18:24:16.000000000 -0600 +++ ./ftplib.py 2004-02-03 19:17:37.000000000 -0600 @@ -469,6 +469,15 @@ cmd = cmd + (' ' + arg) self.retrlines(cmd, func) + def site(self, command, *args): + '''Perform a SITE command with the given arguments.''' + args = " ".join(('SITE', command.upper()) + args) + return self.sendcmd(args) + + def chmod(self, mode, filename): + '''Change file permissions for a file on the remote server.''' + return self.site('chmod', mode, filename) + def rename(self, fromname, toname): '''Rename a file.''' resp = self.sendcmd('RNFR ' + fromname) Jeff From pedronis at bluewin.ch Wed Feb 4 08:49:33 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Wed Feb 4 08:45:39 2004 Subject: [Python-Dev] namespace for generator expressions In-Reply-To: <5.1.0.14.0.20040126172420.03425dd0@mail.telecommunity.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE011DAA80@au3010avexu1.glob al.avaya.com> Message-ID: <5.2.1.1.0.20040204142652.0284d1c0@pop.bluewin.ch> At 17:36 26.01.2004 -0500, Phillip J. Eby wrote: >(Of course, I also often forget that free variables don't bind early in >nested functions! It always seems strange to me that I have to write a >function that returns a function in order to just define a function with >bound variables. Indeed, it's hard to think of a time where I've *ever* >wanted late binding of variables in a nested function.) 0) It should be noted that late binding makes likely the most sense when rebinding is allowed (a feature I still don't know whether it ought to be added, but that's beside the point). At the moment what it allows is mutually referring or self-recursive nested definitions: def h(): def f() # f can refer to g def g() ... [ not that is totally unimagibable a semantics where: def f(): x = 1 def g(): return x x = 2 return g() def f(): def g(): return x x = 1 x = 2 return g() both return 1 ... ] 1) we suffer a bit more the late binding semantics because creating a new scope is not a lightweight operation in Python, consider this Common Lisp code: > (defun f () (do ((x 0 (+ x 1)) (lst () (append lst (list (lambda () x))))) ((= x 4) lst))) > (mapcar #'funcall (f)) (4 4 4 4) kind of like: def f(): x = 0 lst = [] while not (x == 4): lst.append(lambda : x) x += 1 return lst >>> [ el() for el in f() ] [4, 4, 4, 4] an inline 'let' makes it do the what's maybe expected: >(defun f () (do ((x 0 (+ x 1)) (lst () (append lst (list (let ((x x)) (lambda () x)))))) ((= x 4) lst))) > (mapcar #'funcall (f)) (0 1 2 3) In Scheme both 'do' and named-let directly introduce a fresh scope per iteration: (define (f) (do ((x 0 (+ x 1)) (lst () (append lst (list (lambda () x))))) ((= x 4) lst) )) > (map (lambda (func) (func)) (f)) (0 1 2 3) regards. From python-dev-list at cstork.org Wed Feb 4 14:49:37 2004 From: python-dev-list at cstork.org (Christian Stork) Date: Wed Feb 4 14:51:51 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation Message-ID: <20040204194937.GA28378@ics.uci.edu> Hi, I just ran across something that looks like a very unfortunate interface change in the pickle module: Python 2.2: dump(object, file[, bin]) Python 2.3: dump(object, file[, protocol[, bin]]) ...and indeed I have code, which uses pickle.dump(obj, f, True). Hmm, ok, so probably this means now (under python 2.3) I'm using protocol number 1 (==True)?! Tricky. :-) Maybe that warrants a comment in the docs? Thanks, Chris -- Chris Stork <> Support eff.org! <> http://www.ics.uci.edu/~cstork/ OpenPGP fingerprint: B08B 602C C806 C492 D069 021E 41F3 8C8D 50F9 CA2F From skip at pobox.com Wed Feb 4 15:15:06 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Feb 4 15:15:18 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation In-Reply-To: <20040204194937.GA28378@ics.uci.edu> References: <20040204194937.GA28378@ics.uci.edu> Message-ID: <16417.21194.503155.112201@montanaro.dyndns.org> Chris> I just ran across something that looks like a very unfortunate interface Chris> change in the pickle module: Chris> Python 2.2: dump(object, file[, bin]) Chris> Python 2.3: dump(object, file[, protocol[, bin]]) >From the docs: Changed in version 2.3: The protocol parameter was added. The bin parameter is deprecated and only provided for backwards compatibility. You should use the protocol parameter instead. I don't know if that's sufficient for your needs, but it does suggest that dump() calls which used the bin parameter should be inspected. Skip From guido at python.org Wed Feb 4 16:05:12 2004 From: guido at python.org (Guido van Rossum) Date: Wed Feb 4 16:17:23 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation In-Reply-To: Your message of "Wed, 04 Feb 2004 11:49:37 PST." <20040204194937.GA28378@ics.uci.edu> References: <20040204194937.GA28378@ics.uci.edu> Message-ID: <200402042105.i14L5Cj22838@c-24-5-183-134.client.comcast.net> > I just ran across something that looks like a very unfortunate interface > change in the pickle module: > > Python 2.2: dump(object, file[, bin]) > Python 2.3: dump(object, file[, protocol[, bin]]) > > ...and indeed I have code, which uses pickle.dump(obj, f, True). > > Hmm, ok, so probably this means now (under python 2.3) I'm using > protocol number 1 (==True)?! Tricky. :-) Maybe that warrants a comment > in the docs? Are you sure you aren't misreading the docs? The number of arguments is the same. It's just that what was a boolean in 2.2 is an int in 2.3. IOW, protocol 0 is the old "text" protocol, protocol 1 is the old "binary" protocol, and protocol 2 is a new protocol introduced in Python 2.3. And indeed, 0==False, 1==True. --Guido van Rossum (home page: http://www.python.org/~guido/) From python-dev-list at cstork.org Wed Feb 4 17:00:35 2004 From: python-dev-list at cstork.org (Christian Stork) Date: Wed Feb 4 17:32:40 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation In-Reply-To: <16417.21194.503155.112201@montanaro.dyndns.org> References: <20040204194937.GA28378@ics.uci.edu> <16417.21194.503155.112201@montanaro.dyndns.org> Message-ID: <20040204220035.GA29309@ics.uci.edu> On Wed, Feb 04, 2004 at 02:15:06PM -0600, Skip Montanaro wrote: > Chris> I just ran across something that looks like a very unfortunate interface > Chris> change in the pickle module: Actually, I should have reworded that after I realized that it's only a little problem with the docs. The interface change is not unfortunate (indeed it is fortunate as Guido stated ;-) but tricky. That's why I ask for a little explanation in the docs for the "type re-interpretation", ie from bool to int. > Chris> Python 2.2: dump(object, file[, bin]) > Chris> Python 2.3: dump(object, file[, protocol[, bin]]) > > From the docs: > > Changed in version 2.3: The protocol parameter was added. The bin > parameter is deprecated and only provided for backwards compatibility. > You should use the protocol parameter instead. This is what I've been referring to. (I should have quoted it.) > I don't know if that's sufficient for your needs, but it does suggest that > dump() calls which used the bin parameter should be inspected. Well, actually there doesn't seem to be a need for reinspection unless one wants to use the new protocol (ie protocol 2). That's my whole point. It's not clear from the method signature that there is no incompatble interface change... [Again looking thru the docs...] Actually, it's not so bad, 3.14.2 of the LibRef kind of explains it all, and... [looking at pickle.py] I see that bin and protocol args are perfectly checked for sensibility. :-) Quintessence: Maybe change the description of dump in the "3.14.3 Usage" to something saying "Changed in version 2.3: The protocol parameter was added _but_maintains backward_compatibility_for_positional_arguments_of_the_former_bin parameter_. The bin parameter is deprecated and..." This prevents posts like this ;-) and is a help to the lazy ones who don't want to reread the whole subsection. Anyway, enough noise, Chris -- Chris Stork <> Support eff.org! <> http://www.ics.uci.edu/~cstork/ OpenPGP fingerprint: B08B 602C C806 C492 D069 021E 41F3 8C8D 50F9 CA2F From greg at cosc.canterbury.ac.nz Wed Feb 4 21:04:15 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Wed Feb 4 21:04:41 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation In-Reply-To: <200402042105.i14L5Cj22838@c-24-5-183-134.client.comcast.net> Message-ID: <200402050204.i1524Fa01624@oma.cosc.canterbury.ac.nz> Guido: > The number of arguments is the same. It's just that what was a > boolean in 2.2 is an int in 2.3. IOW, protocol 0 is the old "text" > protocol, protocol 1 is the old "binary" protocol, and protocol 2 is > a new protocol introduced in Python 2.3. And indeed, 0==False, > 1==True. This probably takes care of most cases, but there's still a potential for breakage if a program is using a non-canonical value for "true". Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From skip at pobox.com Thu Feb 5 10:58:51 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Feb 5 10:58:59 2004 Subject: [Python-Dev] Charles Waldman? Message-ID: <16418.26683.372241.98483@montanaro.dyndns.org> Sorry for the OT post. Has anyone got a contact for Charles Waldman? He used to be at FermiLab, then at a small trading firm in Chicago, but he's left there as well. Thx, Skip From bkc at murkworks.com Thu Feb 5 13:21:15 2004 From: bkc at murkworks.com (Brad Clements) Date: Thu Feb 5 13:19:50 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation In-Reply-To: <200402042105.i14L5Cj22838@c-24-5-183-134.client.comcast.net> References: Your message of "Wed, 04 Feb 2004 11:49:37 PST." <20040204194937.GA28378@ics.uci.edu> Message-ID: <402242E2.3198.24C03C2C@localhost> > > I just ran across something that looks like a very unfortunate interface > > change in the pickle module: > > > > Python 2.2: dump(object, file[, bin]) > > Python 2.3: dump(object, file[, protocol[, bin]]) > > > Are you sure you aren't misreading the docs? I ran into this problem the other day and thought that pickle and cPickle were doing different things in 2.3 because I tried passing 4 parameters to cPickle dump() and got a different error than passing 4 parameters to pickle's dump() >>> pickle.dump(x, f, 1, 1) Traceback (most recent call last): File "", line 1, in ? File "D:\Python23\lib\pickle.py", line 1382, in dump Pickler(file, protocol, bin).dump(obj) File "D:\Python23\lib\pickle.py", line 199, in __init__ raise ValueError, "can't specify both 'protocol' and 'bin'" ValueError: can't specify both 'protocol' and 'bin' >>> cPickle.dump(x, f, 1, 1) Traceback (most recent call last): File "", line 1, in ? TypeError: function takes at most 3 arguments (4 given) >>> I was pretty tired at the time and just said "huh.. well, I'll leave the code unchanged" which was cPickle.dump(x, f, 1) The way I read the brackets in the docs: dump(object, file[, protocol[, bin]]) you must supply object and file. You may supply protocol. If you supply protocol, you may also supply bin. The nesting of bin within protocol's brackets make me read it that way. Should it instead be: dump(object, file[, protocol | bin]) ? -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements From fdrake at acm.org Thu Feb 5 13:26:41 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu Feb 5 13:26:58 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation In-Reply-To: <402242E2.3198.24C03C2C@localhost> References: <20040204194937.GA28378@ics.uci.edu> <402242E2.3198.24C03C2C@localhost> Message-ID: <200402051326.41604.fdrake@acm.org> On Thursday 05 February 2004 01:21 pm, Brad Clements wrote: > The way I read the brackets in the docs: You read them the way they should be read! > Should it instead be: > > dump(object, file[, protocol | bin]) Yes, something like that would be better, but we don't have any precedent for that in the existing docs (that I can think of off the top of my head). Please open a documentation bug report on this and assign it to me. Thanks! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From bkc at murkworks.com Thu Feb 5 14:13:15 2004 From: bkc at murkworks.com (Brad Clements) Date: Thu Feb 5 14:27:06 2004 Subject: [Python-Dev] pickle.dump(..) interface change ... documentation In-Reply-To: <200402051326.41604.fdrake@acm.org> References: <402242E2.3198.24C03C2C@localhost> Message-ID: <40224F12.27133.24EFD848@localhost> On 5 Feb 2004 at 13:26, Fred L. Drake, Jr. wrote: > Please open a documentation bug report on this and assign it to me. its 891249 Maybe dump( object, file[, protocol_or_bin]) would work.. -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements From Jack.Jansen at cwi.nl Thu Feb 5 15:44:07 2004 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Thu Feb 5 16:24:46 2004 Subject: [Python-Dev] math.sqrt(-1) - NaN, ValueError or whatever you want? Message-ID: <0B505F5E-581C-11D8-B2BD-000A27B19B96@cwi.nl> The answers to my math questions never seem to stick, so I'm afraid I'm gonna have to ask this question again (even though I think I did the same a couple of years ago, but google doesn't find it). It turns out that on MacOS math.sqrt(-1) returns "nan", as does math.log(-1), and I didn't try any other ones. This is true for all possible combinations of MacPython 2.3, 2.4a0 or apple-Python 2.3, MacPython-OS9 or MacPython-OSX, and Mac OS 9 or Mac OS X. When I tried the same on a linux box (Python 2.4a0) it raised ValueError: math domain error, which is what I would have expected. Why this difference? Is it something I should fix? How? Or document it? -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From ilya.sandler at na.teleatlas.com Thu Feb 5 16:22:13 2004 From: ilya.sandler at na.teleatlas.com (Ilya Sandler) Date: Thu Feb 5 16:33:30 2004 Subject: [Python-Dev] pdb enhancements Message-ID: Hello, I was wondering whether I should submit a formal patch for the following pdb enhancements: 1) right now if you do "pdb script" pdb will stop with a confusing > (1)?() and you need to execute "s" before you get to the actual source being debugged, this seems somewhat unexpected and inconvenient A better way would be to go straight to the 1st line of user's code 2) right now when the script exits, the debugger exits too, taking with it all the debugging settings (like breakpoints) this seems very inconvenient Basically the cost of a too early exit is high (as you lose your breakpoints) while the cost of not exiting debugger at all is minimal (you just need to press Ctrl-D to exit when you need it) So I would think that pdb should not exit on script's exit. (and, say, gdb does not exit when program exits...) Do these suggestions make sense? Is implementing them likely to break anything else? Thanks, Ilya PS. I am new on this list and can only hope that the questions are appropriate for python-dev From perky at i18n.org Thu Feb 5 16:44:41 2004 From: perky at i18n.org (Hye-Shik Chang) Date: Thu Feb 5 16:42:43 2004 Subject: [Python-Dev] math.sqrt(-1) - NaN, ValueError or whatever you want? In-Reply-To: <0B505F5E-581C-11D8-B2BD-000A27B19B96@cwi.nl> References: <0B505F5E-581C-11D8-B2BD-000A27B19B96@cwi.nl> Message-ID: <20040205214441.GA87834@i18n.org> On Thu, Feb 05, 2004 at 09:44:07PM +0100, Jack Jansen wrote: > The answers to my math questions never seem to stick, so I'm afraid I'm > gonna have to ask this question again (even though I think I did the > same a couple of years ago, but google doesn't find it). > > It turns out that on MacOS math.sqrt(-1) returns "nan", as does > math.log(-1), and I didn't try any other ones. > > This is true for all possible combinations of MacPython 2.3, 2.4a0 or > apple-Python 2.3, MacPython-OS9 or MacPython-OSX, and Mac OS 9 or Mac > OS X. > > When I tried the same on a linux box (Python 2.4a0) it raised > ValueError: math domain error, which is what I would have expected. > > Why this difference? Is it something I should fix? How? Or document it? SF patch 871657* may apply to MacOS X, too. * math.sqrt EDOM handling for FreeBSD: http://www.python.org/sf/871657 Hye-Shik From aahz at pythoncraft.com Thu Feb 5 16:52:25 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Feb 5 16:53:03 2004 Subject: [Python-Dev] PEP 327: Decimal Data Type In-Reply-To: References: Message-ID: <20040205215225.GB23016@panix.com> On Mon, Feb 02, 2004, Batista, Facundo wrote: > > So, we need to choose one of three ways to obtain an exact conversion: > > 1. Without a second parameter. > 2. With a second parameter "big enough" or a special value (0? -1?) > 3. With another method. Name? Decimal.fromFloat() -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From tim.one at comcast.net Thu Feb 5 17:19:39 2004 From: tim.one at comcast.net (Tim Peters) Date: Thu Feb 5 17:19:37 2004 Subject: [Python-Dev] math.sqrt(-1) - NaN, ValueError or whatever you want? In-Reply-To: <0B505F5E-581C-11D8-B2BD-000A27B19B96@cwi.nl> Message-ID: [Jack Jansen] > The answers to my math questions never seem to stick, so I'm afraid > I'm gonna have to ask this question again (even though I think I did > the same a couple of years ago, but google doesn't find it). > > It turns out that on MacOS math.sqrt(-1) returns "nan", as does > math.log(-1), and I didn't try any other ones. > > This is true for all possible combinations of MacPython 2.3, 2.4a0 or > apple-Python 2.3, MacPython-OS9 or MacPython-OSX, and Mac OS 9 or Mac > OS X. > > When I tried the same on a linux box (Python 2.4a0) it raised > ValueError: math domain error, which is what I would have expected. Take this to heart (it's from the current math module docs): Note: The math module consists mostly of thin wrappers around the platform C math library functions. Behavior in exceptional cases is loosely specified by the C standards, and Python inherits much of its math-function error-reporting behavior from the platform C implementation. As a result, the specific exceptions raised in error cases (and even whether some arguments are considered to be exceptional at all) are not defined in any useful cross-platform or cross-release way. For example, whether math.log(0) returns -Inf or raises ValueError or OverflowError isn't defined, and in cases where math.log(0) raises OverflowError, math.log(0L) may raise ValueError instead. > Why this difference? Platform C. > Is it something I should fix? Good luck . > How? You'll understand the "" above after you give up. > Or document it? I think the above already does. Else you're trying to document a maze of moving targets (including, on some platforms, the specific compiler options, link options, and environment variables the user indulges). From guido at python.org Fri Feb 6 00:04:28 2004 From: guido at python.org (Guido van Rossum) Date: Fri Feb 6 00:04:37 2004 Subject: [Python-Dev] pdb enhancements In-Reply-To: Your message of "Thu, 05 Feb 2004 13:22:13 PST." References: Message-ID: <200402060504.i1654SN25095@c-24-5-183-134.client.comcast.net> > I was wondering whether I should submit a formal patch for the > following pdb enhancements: > > 1) right now if you do "pdb script" > pdb will stop with a confusing > > (1)?() > > and you need to execute "s" before you get to the actual source being > debugged, this seems somewhat unexpected and inconvenient > > A better way would be to go straight to the 1st line of user's code If you can fix that, it would be great. Just make sure that the fix doesn't break anything else. > 2) right now when the script exits, the debugger exits too, taking with it > all the debugging settings (like breakpoints) this seems very inconvenient > > Basically the cost of a too early exit is high (as you lose your > breakpoints) while the cost of not exiting debugger at all is minimal (you > just need to press Ctrl-D to exit when you need it) > > So I would think that pdb should not exit on script's exit. > (and, say, gdb does not exit when program exits...) > > Do these suggestions make sense? Is implementing them likely to break > anything else? Ditto. I don't know what it would break, but you will hve to test this thoroughly. > Thanks, > > Ilya > > PS. I am new on this list and can only hope that the questions are > appropriate for python-dev PS. Use the SF patch manager for submitting patches. Read http://python.org/dev/. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one at comcast.net Fri Feb 6 01:11:53 2004 From: tim.one at comcast.net (Tim Peters) Date: Fri Feb 6 01:11:51 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects weakrefobject.c, 1.15, 1.16 In-Reply-To: Message-ID: > Modified Files: > weakrefobject.c > Log Message: ... > Index: weakrefobject.c > =================================================================== ... and it leaves part of PyWeakref_NewRef() looking like: if (result != NULL) Py_INCREF(result); else { /* Note: new_weakref() can trigger cyclic GC, so the weakref list on ob can be mutated. This means that the ref and proxy pointers we got back earlier may have been collected, so we need to compute these values again before we use them. */ result = new_weakref(ob, callback); if (result != NULL) { if (callback == NULL) { insert_head(result, list); } else { When we call new_weakref(), callback may or may not be NULL. Assume it's NULL. Then we don't get into this code unless the first call to get_basic_refs() said there was not already a canonical weakref-without-callback weakref for ob. Cyclic gc can execute arbitrary Python code, if __del__ methods or callbacks get executed as a result of refcounts falling to 0 during cyclic gc. That code could in turn *create* a canonical weakref-without-callback weakref for ob, even while we're in the middle of creating our own. Suppose it does, and result isn't NULL. Then it looks like insert_head(result, list) is going to insert a second weakref-without-callback weakref for ob. Wouldn't that violate an excruciating global invariant? For example, PyObject_ClearWeakRefs() assumes that callback isn't NULL when it gets beyond the first two weakref objects in a weakref list, and get_basic_refs() assumes it never has to look at more than the first two weakref objects in a weakref list to determine whether a canonical weakref-proxy object for ob exists. From neal at metaslash.com Fri Feb 6 21:26:52 2004 From: neal at metaslash.com (Neal Norwitz) Date: Fri Feb 6 21:26:55 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules collectionsmodule.c, 1.2, 1.3 In-Reply-To: References: Message-ID: <20040207022652.GP10085@epoch.metaslash.com> On Fri, Feb 06, 2004 at 11:04:59AM -0800, rhettinger@users.sourceforge.net wrote: > > + static PyObject * > + deque_extend(dequeobject *deque, PyObject *iterable) > + { > + PyObject *it, *item; > + > + it = PyObject_GetIter(iterable); > + if (it == NULL) > + return NULL; > + > + while ((item = PyIter_Next(it)) != NULL) { > + deque->rightindex++; > + deque->len++; > + if (deque->rightindex == BLOCKLEN) { > + block *b = newblock(deque->rightblock, NULL); > + if (b == NULL) > + return NULL; In deque_extend and deque_extendleft, doesn't there need to be a Py_DECREF(it); before returning when newblock fails? > + assert(deque->rightblock->rightlink == NULL); > + deque->rightblock->rightlink = b; > + deque->rightblock = b; > + deque->rightindex = 0; > + } > + Py_INCREF(item); > + deque->rightblock->data[deque->rightindex] = item; > + } > + Py_DECREF(it); > + if (PyErr_Occurred()) > + return NULL; > + Py_RETURN_NONE; > + } From guido at python.org Sat Feb 7 00:06:40 2004 From: guido at python.org (Guido van Rossum) Date: Sat Feb 7 00:06:53 2004 Subject: [Python-Dev] OSCON Reminder: proposal deadline Februrary 9 Message-ID: <200402070506.i1756eJ26916@c-24-5-183-134.client.comcast.net> Just a quick reminder: you still have this weekend to submit a proposal for the Python 12 Conference at OSCON: The O'Reilly Open Source Convention (OSCON) will be held July 26-30, 2004 at the Portland Marriott Downtown in Portland, OR. http://conferences.oreillynet.com/os2004/ Proposals Submission Information--Deadline: February 9, 2004 http://conferences.oreillynet.com/cs/os2004/create/e_sess --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Fri Feb 6 22:56:27 2004 From: bac at OCF.Berkeley.EDU (Brett) Date: Sat Feb 7 01:45:13 2004 Subject: [Python-Dev] pyconfig.h macro usage In-Reply-To: <16380.38393.450171.642972@montanaro.dyndns.org> References: <16380.38393.450171.642972@montanaro.dyndns.org> Message-ID: <402461EB.9030802@ocf.berkeley.edu> Skip Montanaro wrote: > After Zacks message about configure.in cruft, I zapped the SIZEOF_CHAR macro > and noticed it was never used in any .c files. So that got me to thinking > (dangerous, I know)... What about: > > for macro in `egrep '#undef [_A-Z]' pyconfig.h.in \ > | awk '{print $2}' \ > | sort` > do > echo -n "$macro: " > find . -name '*.[ch]' \ > | egrep -v pyconfig \ > | xargs egrep $macro \ > | wc -l > done \ > | sort -n -k 2,2 > > Turns out there are quite a few macros defined in pyconfig.h that are > unused: > > HAVE_DUP2 SIZEOF_FLOAT > HAVE_GETPID SIZEOF_UINTPTR_T > HAVE_LIBDL SIZEOF_WCHAR_T > HAVE_LIBDLD SYS_SELECT_WITH_SYS_TIME > HAVE_LIBIEEE TM_IN_SYS_TIME > HAVE_LIBRESOLV WITH_DL_DLD > HAVE_PTHREAD_INIT WITH_DYLD > HAVE_STDARG_H WITH_LIBINTL > HAVE_STRDUP _FILE_OFFSET_BITS > HAVE_STRPTIME _LARGEFILE_SOURCE > HAVE_ST_BLOCKS _MINIX > HAVE_SYS_SOCKET_H _NETBSD_SOURCE > HAVE_TERMIOS_H _OSF_SOURCE > HAVE_TIMEGM _POSIX_1_SOURCE > HAVE_TM_ZONE _POSIX_C_SOURCE > HAVE_TRUNCATE _POSIX_SOURCE > HAVE_UCS4_TCL _REENTRANT > RETSIGTYPE __BSD_VISIBLE > SIZEOF_DOUBLE __EXTENSIONS__ > > Any votes for getting rid of them? > I think HAVE_STRPTIME can be ditched. Sorry for the late response. This is what happens when I don't really get to read some emails in detail until I do the next Summary. -Brett From skip at pobox.com Sat Feb 7 09:01:27 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Feb 7 09:01:31 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test Message-ID: <16420.61367.299906.722031@montanaro.dyndns.org> (For those of you not on python-checkins...) I've removed a number of obsolete features as specified by PEP 11. In particular, this morning I removed support for --without-universal-newlines and transitional pthread variants dating from the 1997 time frame. SunOS 4 support is also gone. For full details see Misc/NEWS. I've been testing on Mac OS X, but am also testing on Solaris 8 and Mandrake 8.1. If you haven't cvs up'd and rebuilt recently (don't forget to make clean and execute ./configure on Unix-y platforms) please do so and run make test. Thx, Skip From skip at pobox.com Sat Feb 7 10:04:17 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Feb 7 10:04:21 2004 Subject: [Python-Dev] test_anydbm segfault on Mandrake 8.1 Message-ID: <16420.65137.391122.664891@montanaro.dyndns.org> I'm getting this on Mandrake 8.1: % ./python Lib/test/test_anydbm.py U?S? ??2 Segmentation fault Running under gdb I get this for the Python stack: (gdb) ppystack /home/skip/src/python/dist/src/Lib/bsddb/__init__.py (337): _openDBEnv /home/skip/src/python/dist/src/Lib/bsddb/__init__.py (281): hashopen /home/skip/src/python/dist/src/Lib/dbhash.py (15): open /home/skip/src/python/dist/src/Lib/anydbm.py (66): open Lib/test/test_anydbm.py (36): test_anydbm_creation /home/skip/src/python/dist/src/Lib/unittest.py (245): run /home/skip/src/python/dist/src/Lib/unittest.py (419): __call__ /home/skip/src/python/dist/src/Lib/unittest.py (419): __call__ /home/skip/src/python/dist/src/Lib/unittest.py (686): run /home/skip/src/python/dist/src/Lib/test/test_support.py (255): run_suite /home/skip/src/python/dist/src/Lib/test/test_support.py (278): run_unittest Lib/test/test_anydbm.py (88): test_main Lib/test/test_anydbm.py (4): ? and this is the beginning of the gdb stack trace: #0 0x00000000 in ?? () #1 0x402a0d0f in DBEnv_open (self=0x401a6680, args=0x4026494c) at /home/skip/src/python/dist/src/Modules/_bsddb.c:3227 #2 0x080fd98a in PyCFunction_Call (func=0x40264dec, arg=0x4026494c, kw=0x0) at Objects/methodobject.c:93 #3 0x080b3c3d in call_function (pp_stack=0xbfffda7c, oparg=2) at Python/ceval.c:3452 #4 0x080b2080 in eval_frame (f=0x814e024) at Python/ceval.c:2126 #5 0x080b599b in fast_function (func=0x4026048c, pp_stack=0xbfffdb8c, n=0, na=0, nk=0) at Python/ceval.c:3531 #6 0x080b3b29 in call_function (pp_stack=0xbfffdb8c, oparg=0) at Python/ceval.c:3471 #7 0x080b2080 in eval_frame (f=0x8196794) at Python/ceval.c:2126 #8 0x080b2d92 in PyEval_EvalCodeEx (co=0x4021d820, globals=0x402179bc, locals=0x0, args=0x81b8630, argcount=3, kws=0x81b863c, kwcount=0, defs=0x402603b8, defcount=8, closure=0x0) at Python/ceval.c:2676 #9 0x080b5911 in fast_function (func=0x402573e4, pp_stack=0xbfffdd2c, n=3, na=3, nk=0) at Python/ceval.c:3542 ldd tells me this about the _bsddb shared object: % ldd build/lib.linux-i686-2.4/_bsddb.so libdb-3.3.so => /lib/libdb-3.3.so (0x40023000) libgcc_s.so.1 => /usr/local/lib/libgcc_s.so.1 (0x400b3000) libc.so.6 => /lib/libc.so.6 (0x400bb000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) and rpm suggests I've got Berkeley DB 3.3.11 installed. Is that recent enough for the bsddb package (suggesting a bug in setup.py) or is something else going on? Skip From martin at v.loewis.de Sat Feb 7 12:46:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Feb 7 12:47:02 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <16420.61367.299906.722031@montanaro.dyndns.org> References: <16420.61367.299906.722031@montanaro.dyndns.org> Message-ID: <40252484.4030505@v.loewis.de> Skip Montanaro wrote: > I've removed a number of obsolete features as specified by PEP 11. Thanks for doing that. Any proposals as to what should systems should be ignored in Python 2.5? In particular, I'm still in search for the minimum AIX version that we should support. As for removing SunOS4 support: plat-sunos4 can then go, as well. Regards, Martin From skip at pobox.com Sat Feb 7 13:45:51 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Feb 7 13:45:57 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <40252484.4030505@v.loewis.de> References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> Message-ID: <16421.12895.868946.402819@montanaro.dyndns.org> Martin> Any proposals as to what should systems should be ignored in Martin> Python 2.5? None from me. Martin> As for removing SunOS4 support: plat-sunos4 can then go, as Martin> well. Good point. I'll cvs remove it during my next round of edits. While I have your attention, are you aware of any specific preprocessor macros keyed to linux1? I see the plat-linux1 directory which has to be removed, but don't see "linux1" in any of the source files. Skip From jeremy at alum.mit.edu Sat Feb 7 14:25:39 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Sat Feb 7 14:42:49 2004 Subject: [Python-Dev] building python with profiling support Message-ID: <1076181939.21762.20.camel@localhost.localdomain> I've used gprof to profile the interpreter in the past, but I'm stuck trying to it again on my current machine. I configured Python by saying CC="gcc -pg" ../configure That's the way I've built Python for profiling in the past and that's what the README says to do. The python binary that gets generated doesn't generate a gmon.out file. It appears that the problem is with the command used to generate the actual Python executable: c++ -pthread -Xlinker -export-dynamic -o python \ Modules/python.o \ libpython2.4.a -lpthread -ldl -lutil -lm The -pg flag has been lost. Can anyone suggest a recipe that will work? Or suggest how to modify the Makefile so that the -pg flag gets passed along. Jeremy From perky at i18n.org Sat Feb 7 14:54:10 2004 From: perky at i18n.org (Hye-Shik Chang) Date: Sat Feb 7 14:52:00 2004 Subject: [Python-Dev] building python with profiling support In-Reply-To: <1076181939.21762.20.camel@localhost.localdomain> References: <1076181939.21762.20.camel@localhost.localdomain> Message-ID: <20040207195410.GA66834@i18n.org> On Sat, Feb 07, 2004 at 02:25:39PM -0500, Jeremy Hylton wrote: > I've used gprof to profile the interpreter in the past, but I'm stuck > trying to it again on my current machine. I configured Python by saying > CC="gcc -pg" ../configure > > That's the way I've built Python for profiling in the past and that's > what the README says to do. The python binary that gets generated > doesn't generate a gmon.out file. [snip] > Can anyone suggest a recipe that will work? Or suggest how to modify > the Makefile so that the -pg flag gets passed along. > I turn profiling on in this way: % CC="cc -pg" LDFLAGS="-pg" ./configure Hye-Shik From skip at pobox.com Sat Feb 7 15:16:55 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Feb 7 15:16:59 2004 Subject: [Python-Dev] supporting Quixote PTL files in pydoc Message-ID: <16421.18359.8820.790161@montanaro.dyndns.org> I have this slight change to pydoc.py which allows it to grok Quixote's PTL-based modules: % cvs diff -u pydoc.py Index: pydoc.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/pydoc.py,v retrieving revision 1.90 diff -c -u -r1.90 pydoc.py cvs diff: conflicting specifications of output style --- pydoc.py 29 Jan 2004 06:37:49 -0000 1.90 +++ pydoc.py 7 Feb 2004 20:07:43 -0000 @@ -2125,10 +2125,15 @@ sys.path.insert(0, '.') try: - opts, args = getopt.getopt(sys.argv[1:], 'gk:p:w') + opts, args = getopt.getopt(sys.argv[1:], 'gk:p:wq') writing = 0 for opt, val in opts: + if opt == '-q': + # support Quixote's PTL + import quixote + quixote.enable_ptl() + continue if opt == '-g': gui() return I realize this may seem a bit like the tail wagging the dog, but PTL functions are so close syntactically and semantically to Python that it seems like a fairly modest change to allow them to be supported. I wonder if something like this could reasonably be slipped into the repository without much fuss? I have a change for pychecker which allows it to check PTL modules as well. For those unfamiliar with it, Quixote's PTL is described here: http://www.mems-exchange.org/software/quixote/doc/PTL.html Skip From martin at v.loewis.de Sat Feb 7 15:24:08 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Feb 7 15:25:14 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <16421.12895.868946.402819@montanaro.dyndns.org> References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> Message-ID: <40254968.4050205@v.loewis.de> Skip Montanaro wrote: > While I have your attention, are you aware of any specific preprocessor > macros keyed to linux1? I see the plat-linux1 directory which has to be > removed, but don't see "linux1" in any of the source files. Not that I know of. plat-linux? doesn't really fit with the other systems, as the major version of the kernel is much less relevant for applications on Linux than it is on other systems. On Linux, it matters much more whether you have libc 4, 5, or 6 (which, in principle, can be arbitrarily combined with Linux 0, 1, or 2). Regards, Martin From skip at pobox.com Sat Feb 7 16:57:42 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Feb 7 16:57:46 2004 Subject: [Python-Dev] PEP-0011 up-to-date Message-ID: <16421.24406.923068.368161@montanaro.dyndns.org> I believe all the 2.4 stuff for PEP-0011 is now complete. Are any edits to the pep itself required? Skip From pje at telecommunity.com Sat Feb 7 17:02:57 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Feb 7 16:59:05 2004 Subject: [Python-Dev] supporting Quixote PTL files in pydoc In-Reply-To: <16421.18359.8820.790161@montanaro.dyndns.org> Message-ID: <5.1.0.14.0.20040207170107.0317ed50@mail.telecommunity.com> At 02:16 PM 2/7/04 -0600, Skip Montanaro wrote: >I have this slight change to pydoc.py which allows it to grok Quixote's >PTL-based modules: > >% cvs diff -u pydoc.py >Index: pydoc.py >=================================================================== >RCS file: /cvsroot/python/python/dist/src/Lib/pydoc.py,v >retrieving revision 1.90 >diff -c -u -r1.90 pydoc.py >cvs diff: conflicting specifications of output style >--- pydoc.py 29 Jan 2004 06:37:49 -0000 1.90 >+++ pydoc.py 7 Feb 2004 20:07:43 -0000 >@@ -2125,10 +2125,15 @@ > sys.path.insert(0, '.') > > try: >- opts, args = getopt.getopt(sys.argv[1:], 'gk:p:w') >+ opts, args = getopt.getopt(sys.argv[1:], 'gk:p:wq') > writing = 0 > > for opt, val in opts: >+ if opt == '-q': >+ # support Quixote's PTL >+ import quixote >+ quixote.enable_ptl() >+ continue > if opt == '-g': > gui() > return Couldn't this be accomplished by simply placing an enable_ptl() call in the __init__.py of packages that contain .ptl files? From skip at pobox.com Sat Feb 7 18:35:08 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Feb 7 18:35:14 2004 Subject: [Python-Dev] supporting Quixote PTL files in pydoc In-Reply-To: <5.1.0.14.0.20040207170107.0317ed50@mail.telecommunity.com> References: <16421.18359.8820.790161@montanaro.dyndns.org> <5.1.0.14.0.20040207170107.0317ed50@mail.telecommunity.com> Message-ID: <16421.30252.495483.175580@montanaro.dyndns.org> ... >> + # support Quixote's PTL >> + import quixote >> + quixote.enable_ptl() ... Phillip> Couldn't this be accomplished by simply placing an enable_ptl() Phillip> call in the __init__.py of packages that contain .ptl files? In theory, I guess it could, but it would require that all .ptl files reside in packages. Unless I'm mistaken, that's not currently a requirement. Skip From skip at pobox.com Sat Feb 7 18:46:50 2004 From: skip at pobox.com (Skip Montanaro) Date: Sat Feb 7 18:46:55 2004 Subject: [Python-Dev] *grumble* wchar.h Message-ID: <16421.30954.952412.678861@montanaro.dyndns.org> I decided to tackle the ANSI C assumption next. If we assume an ANSI C compiler, then a number of checks for header files required by the standard can be deleted. Working from the list here: http://en.wikipedia.org/wiki/ANSI_C_standard_library I started zapping tests. Once finished, I tried building. Everything seemed to work fine except the _locale module wouldn't build because my Mac doesn't have wchar.h. So, is wchar.h part of the ANSI C standard or is Apple's GCC setup (Mac OS X 10.2.8, gcc 3.3) in error? Skip From martin at v.loewis.de Sat Feb 7 19:01:41 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Feb 7 19:01:45 2004 Subject: [Python-Dev] building python with profiling support In-Reply-To: <1076181939.21762.20.camel@localhost.localdomain> References: <1076181939.21762.20.camel@localhost.localdomain> Message-ID: <40257C65.4040705@v.loewis.de> Jeremy Hylton wrote: > I've used gprof to profile the interpreter in the past, but I'm stuck > trying to it again on my current machine. I configured Python by saying > CC="gcc -pg" ../configure [...] > Can anyone suggest a recipe that will work? Or suggest how to modify > the Makefile so that the -pg flag gets passed along. It looks like building --without-cxx should work. Martin From martin at v.loewis.de Sat Feb 7 19:19:29 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Feb 7 19:19:33 2004 Subject: [Python-Dev] PEP-0011 up-to-date In-Reply-To: <16421.24406.923068.368161@montanaro.dyndns.org> References: <16421.24406.923068.368161@montanaro.dyndns.org> Message-ID: <40258091.4070503@v.loewis.de> Skip Montanaro wrote: > I believe all the 2.4 stuff for PEP-0011 is now complete. Are any edits to > the pep itself required? No. As I said, we could look for more stuff that we want to get rid off, but we could do that at any time. On my candidate list, further simplifications to the threading appear desirable: - Is anybody still using cthreads? SGI threads? (does Irix have pthreads now?) LWP threads? Also, there is a bit of code talking about Monterey. Does that system actually exist? Regards, Martin From martin at v.loewis.de Sat Feb 7 19:24:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Feb 7 19:24:45 2004 Subject: [Python-Dev] *grumble* wchar.h In-Reply-To: <16421.30954.952412.678861@montanaro.dyndns.org> References: <16421.30954.952412.678861@montanaro.dyndns.org> Message-ID: <402581CC.8000106@v.loewis.de> Skip Montanaro wrote: > So, is wchar.h part of the ANSI C standard or is Apple's GCC setup (Mac OS X > 10.2.8, gcc 3.3) in error? wchar.h is not part of ISO C 89 (and thus not of ANSI C). It was added to ISO C in Addendum 1 (1994), and is part of ISO C 99. Regards, Martin From jim at zope.com Sat Feb 7 22:33:40 2004 From: jim at zope.com (Jim Fulton) Date: Sat Feb 7 22:38:05 2004 Subject: [Python-Dev] Tests failing on the head Message-ID: <4025AE14.8000209@zope.com> On Red Hat 9: [jim@enterprised src]$ uname -a Linux enterprised.zope.com 2.4.20-28.9 #1 Thu Dec 18 13:45:22 EST 2003 i686 i686 i386 GNU/Linux [jim@enterprised src]$ gcc -v Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2.2/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux Thread model: posix gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5) ./configure --with-pydebug ... make ... ./python -E -tt ./Lib/test/regrtest.py -l -uall ... 240 tests OK. 8 tests failed: test_curses test_linuxaudiodev test_longexp test_ossaudiodev test_socket_ssl test_tarfile test_timeout test_urllibnet 23 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bz2 test_cd test_cl test_codecmaps_cn test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_gl test_imgfile test_macfs test_macostools test_nis test_normalization test_pep277 test_plistlib test_scriptpackages test_sunaudiodev test_winreg test_winsound 1 skip unexpected on linux2: test_bz2 Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From skip at pobox.com Sun Feb 8 07:47:53 2004 From: skip at pobox.com (Skip Montanaro) Date: Sun Feb 8 07:47:57 2004 Subject: [Python-Dev] *grumble* wchar.h In-Reply-To: <402581CC.8000106@v.loewis.de> References: <16421.30954.952412.678861@montanaro.dyndns.org> <402581CC.8000106@v.loewis.de> Message-ID: <16422.12281.959231.415847@montanaro.dyndns.org> >> So, is wchar.h part of the ANSI C standard or is Apple's GCC setup >> (Mac OS X 10.2.8, gcc 3.3) in error? Martin> wchar.h is not part of ISO C 89 (and thus not of ANSI C). It was Martin> added to ISO C in Addendum 1 (1994), and is part of ISO C 99. Dang. Okay, I'll add that back in. Is there a more authoritative online source for this stuff than what I stumbled upon? Skip From tim.one at comcast.net Sun Feb 8 14:37:43 2004 From: tim.one at comcast.net (Tim Peters) Date: Sun Feb 8 14:37:35 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Doc/tools prechm.py, 1.15.18.2, 1.15.18.3 In-Reply-To: Message-ID: > Modified Files: > Tag: release23-maint > prechm.py > Log Message: > SF patch #880552: Fix typo in usage message(prechm.py) > (Contributed by George Yoshida.) ... > > Index: prechm.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Doc/tools/prechm.py,v > retrieving revision 1.15.18.2 > retrieving revision 1.15.18.3 > diff -C2 -d -r1.15.18.2 -r1.15.18.3 > *** prechm.py 23 Sep 2003 20:34:40 -0000 1.15.18.2 > --- prechm.py 8 Feb 2004 19:26:13 -0000 1.15.18.3 > *************** > *** 24,28 **** > > usage_mode = ''' > ! Usage: make_chm.py [-c] [-k] [-p] [-v 1.5[.x]] filename > -c: does not build filename.hhc (Table of Contents) > -k: does not build filename.hhk (Index) > --- 24,28 ---- > > usage_mode = ''' > ! Usage: pre_chm.py [-c] [-k] [-p] [-v 1.5[.x]] filename > -c: does not build filename.hhc (Table of Contents) > -k: does not build filename.hhk (Index) > *************** If the name of the file is prechm.py, then saying pre_chm.py in the usage message appears just as wrong as saying make_chm.py . From martin at v.loewis.de Sun Feb 8 16:12:00 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Feb 8 16:13:25 2004 Subject: [Python-Dev] *grumble* wchar.h In-Reply-To: <16422.12281.959231.415847@montanaro.dyndns.org> References: <16421.30954.952412.678861@montanaro.dyndns.org> <402581CC.8000106@v.loewis.de> <16422.12281.959231.415847@montanaro.dyndns.org> Message-ID: <4026A620.40303@v.loewis.de> Skip Montanaro wrote: > Dang. Okay, I'll add that back in. Is there a more authoritative online > source for this stuff than what I stumbled upon? You mean, except for the standard itself, and including historical aspects? http://wwwold.dkuug.dk/JTC1/SC22/WG14/www/newinc9x.htm gives the history. As this is a document of the standards working group, it is pretty accurate. A bit more elaboration is from Clive Feather, who is one of the authors of standard C: http://www.lysator.liu.se/c/na1.html It appears that this site lists a lot of useful material: http://www.lysator.liu.se/c/ Notice that, by ANSI procedures, many ISO standards also become ANSI standards - so C99 is an ANSI standard as well. Therefore, a claim "ANSI C has wchar.h is equally right *and* wrong". Regards, Martin From skip at pobox.com Sun Feb 8 16:18:59 2004 From: skip at pobox.com (Skip Montanaro) Date: Sun Feb 8 16:20:46 2004 Subject: [Python-Dev] *grumble* wchar.h In-Reply-To: <4026A620.40303@v.loewis.de> References: <16421.30954.952412.678861@montanaro.dyndns.org> <402581CC.8000106@v.loewis.de> <16422.12281.959231.415847@montanaro.dyndns.org> <4026A620.40303@v.loewis.de> Message-ID: <16422.42947.292397.2731@montanaro.dyndns.org> >> Is there a more authoritative online source for this stuff than what >> I stumbled upon? [ several pointers I need to bookmark elided ] Martin> Notice that, by ANSI procedures, many ISO standards also become Martin> ANSI standards - so C99 is an ANSI standard as well. Therefore, Martin> a claim "ANSI C has wchar.h is equally right *and* wrong". So, I take it Python currently insists on conformance with C89, correct? I'll get it one of these days. Skip From martin at v.loewis.de Sun Feb 8 16:27:21 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun Feb 8 16:28:33 2004 Subject: [Python-Dev] *grumble* wchar.h In-Reply-To: <16422.42947.292397.2731@montanaro.dyndns.org> References: <16421.30954.952412.678861@montanaro.dyndns.org> <402581CC.8000106@v.loewis.de> <16422.12281.959231.415847@montanaro.dyndns.org> <4026A620.40303@v.loewis.de> <16422.42947.292397.2731@montanaro.dyndns.org> Message-ID: <4026A9B9.8020703@v.loewis.de> Skip Montanaro wrote: > So, I take it Python currently insists on conformance with C89, correct? Correct: It is ok if it breaks if the system does not have all C89 features. Lack of some C99 features is ok. (Notice that this is slightly different from "insists on conformance with C89": A C99 implementation is not strictly conforming with C89, e.g. because the value of the __STDC__ macro is different. However, a C99 implementation will support all C89 features). Regards, Martin From jim at zope.com Sun Feb 8 21:05:04 2004 From: jim at zope.com (Jim Fulton) Date: Sun Feb 8 21:07:23 2004 Subject: [Python-Dev] Tests failling on release23-maint branch Message-ID: <4026EAD0.8080506@zope.com> On RedHat 9: [jim@enterprised src]$ uname -a Linux enterprised.zope.com 2.4.20-28.9 #1 Thu Dec 18 13:45:22 EST 2003 i686 i686 i386 GNU/Linux [jim@enterprised src]$ gcc -v Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2.2/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux Thread model: posix gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5) [jim@enterprised src]$ ./configure --with-pydebug --prefix=/usr/local/python/release23-maintdb ... [jim@enterprised src]$ make ... [jim@enterprised src]$ nice ./python -E -tt ./Lib/test/regrtest.py -l -uall ... 6 tests failed: test_curses test_linuxaudiodev test_socket_ssl test_tarfile test_timeout test_urllibnet 19 tests skipped: test_aepack test_al test_bsddb185 test_bz2 test_cd test_cl test_email_codecs test_gl test_imgfile test_macfs test_macostools test_nis test_normalization test_pep277 test_plistlib test_scriptpackages test_sunaudiodev test_winreg test_winsound 1 skip unexpected on linux2: test_bz2 -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org From mwh at python.net Mon Feb 9 07:06:18 2004 From: mwh at python.net (Michael Hudson) Date: Mon Feb 9 07:06:22 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <40254968.4050205@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sat, 07 Feb 2004 21:24:08 +0100") References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> <40254968.4050205@v.loewis.de> Message-ID: <2mllnccv9h.fsf@starship.python.net> "Martin v. L?wis" writes: > Skip Montanaro wrote: >> While I have your attention, are you aware of any specific preprocessor >> macros keyed to linux1? I see the plat-linux1 directory which has to be >> removed, but don't see "linux1" in any of the source files. > > Not that I know of. plat-linux? doesn't really fit with the other > systems, as the major version of the kernel is much less relevant > for applications on Linux than it is on other systems. On Linux, > it matters much more whether you have libc 4, 5, or 6 (which, in > principle, can be arbitrarily combined with Linux 0, 1, or 2). There were a small bundle of bug reports from a guy with libc5 a week or so back, so there's at least some interest in that. OTOH, I don't feel like putting all that much effort into fixing the problems he reported (most of them were failry shallow, I think). Cheers, mwh -- Just point your web browser at http://www.python.org/search/ and look for "program", "doesn't", "work", or "my". Whenever you find someone else whose program didn't work, don't do what they did. Repeat as needed. -- Tim Peters, on python-help, 16 Jun 1998 From mwh at python.net Mon Feb 9 07:07:12 2004 From: mwh at python.net (Michael Hudson) Date: Mon Feb 9 07:07:15 2004 Subject: [Python-Dev] building python with profiling support In-Reply-To: <1076181939.21762.20.camel@localhost.localdomain> (Jeremy Hylton's message of "Sat, 07 Feb 2004 14:25:39 -0500") References: <1076181939.21762.20.camel@localhost.localdomain> Message-ID: <2mhdy0cv7z.fsf@starship.python.net> Jeremy Hylton writes: > I've used gprof to profile the interpreter in the past, but I'm stuck > trying to it again on my current machine. I configured Python by saying > CC="gcc -pg" ../configure > > That's the way I've built Python for profiling in the past and that's > what the README says to do. The python binary that gets generated > doesn't generate a gmon.out file. > > It appears that the problem is with the command used to generate the > actual Python executable: > > c++ -pthread -Xlinker -export-dynamic -o python \ > Modules/python.o \ > libpython2.4.a -lpthread -ldl -lutil -lm > > The -pg flag has been lost. > > Can anyone suggest a recipe that will work? Or suggest how to modify > the Makefile so that the -pg flag gets passed along. I think when I last did this I pasted the link line into my terminal and added "-pg" by hand... Cheers, mwh -- ARTHUR: The ravenours bugblatter beast of Traal ... is it safe? FORD: Oh yes, it's perfectly safe ... it's just us who are in trouble. -- The Hitch-Hikers Guide to the Galaxy, Episode 6 From mwh at python.net Mon Feb 9 08:12:02 2004 From: mwh at python.net (Michael Hudson) Date: Mon Feb 9 08:12:07 2004 Subject: [Python-Dev] Tests failling on release23-maint branch In-Reply-To: <4026EAD0.8080506@zope.com> (Jim Fulton's message of "Sun, 08 Feb 2004 21:05:04 -0500") References: <4026EAD0.8080506@zope.com> Message-ID: <2moes8bdnh.fsf@starship.python.net> Jim Fulton writes: > On RedHat 9: > [...] > 6 tests failed: > test_curses test_linuxaudiodev test_socket_ssl test_tarfile > test_timeout test_urllibnet Hmm, on my redhat linux system I only get test_linuxaudiodev failing (and I've long since given up on that one). Cheers, mwh -- ARTHUR: Why should he want to know where his towel is? FORD: Everybody should know where his towel is. ARTHUR: I think your head's come undone. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From guido at python.org Mon Feb 9 10:14:16 2004 From: guido at python.org (Guido van Rossum) Date: Mon Feb 9 10:14:24 2004 Subject: [Python-Dev] OSCON Reminder: proposal deadline *** TODAY *** Message-ID: <200402091514.i19FEGI24753@c-24-5-183-134.client.comcast.net> This is the FINAL reminder: you still have all day today to submit a proposal for the Python 12 Conference at OSCON: The O'Reilly Open Source Convention (OSCON) will be held July 26-30, 2004 at the Portland Marriott Downtown in Portland, OR. http://conferences.oreillynet.com/os2004/ Proposals Submission Information--Deadline: February 9, 2004 http://conferences.oreillynet.com/cs/os2004/create/e_sess --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy at alum.mit.edu Mon Feb 9 11:18:13 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Mon Feb 9 11:18:59 2004 Subject: [Python-Dev] Tests failling on release23-maint branch In-Reply-To: <2moes8bdnh.fsf@starship.python.net> References: <4026EAD0.8080506@zope.com> <2moes8bdnh.fsf@starship.python.net> Message-ID: <1076343493.21762.28.camel@localhost.localdomain> On Mon, 2004-02-09 at 08:12, Michael Hudson wrote: > Jim Fulton writes: > > > On RedHat 9: > > > [...] > > 6 tests failed: > > test_curses test_linuxaudiodev test_socket_ssl test_tarfile > > test_timeout test_urllibnet > > Hmm, on my redhat linux system I only get test_linuxaudiodev failing > (and I've long since given up on that one). That's the only one that fails for me. The test_ossaudiodev test hangs every time and the bsddb tests fail with random BerkeleyDB errors sometimes. In other words, only the standard brokenness. Jeremy From pinard at iro.umontreal.ca Mon Feb 9 10:51:17 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 9 11:32:37 2004 Subject: [Python-Dev] Allowing non-ASCII identifiers In-Reply-To: <4005A1DA.207@v.loewis.de> References: <4005A1DA.207@v.loewis.de> Message-ID: <20040209155117.GA7467@titan.progiciels-bpi.ca> [Martin von L?wis] > I'd like to work on adding support for non-ASCII characters > in identifiers[...] Such a support would surely be extremely welcome to me, and to most of my co-workers. There is likely many teams around this planet that would appreciate it as well. Tell me if you think I may help somehow, despite my modest means (I'm over-loaded with duties already, but this is the story for most of us). > 1. At run-time, identifiers are represented as Unicode objects unless > they are pure ASCII. IOW, they are converted from the source encoding > to Unicode objects in the process of parsing. This is already the case, isn't it? > 2. As a consequence of 1), all places there identifiers appear need to > support Unicode objects (e.g. __dict__, __getattr__, etc) I do not much know the internals, yet I suspect one more thing to consider is whether Unicode strings looking like non-ASCII identifiers should be interned or not, the same as currently done for ASCII. > 3. Legal non-ASCII identifiers are what legal non-ASCII identifiers > are in Java, except that Python may use a different version of the > Unicode character database. Python would share the property that > future versions allow more characters in identifiers than older > versions. > If you are too lazy too look up the Java definition, here is a > rough overview: An identifier is "JavaLetter JavaLetterOrDigit*" > JavaLetter is a character of the classes Lu, Ll, Lt, Lm, or Lo, > or a currency symbol (for Python: excluding $), or a connecting > punctuation character (which is unfortunately underspecified - will > research the implementation). > JavaLetterOrDigit is a JavaLetter, or a digit, a numeric letter, > a combining mark, a non-spacing mark, or an ignorable control > character. Then, maybe we should be a tad conservative whenever there is some doubt, rather than sticking too closely to Java. It is easier to open a bit more later, than to close what was opened. For example, all currency symbols might be verboten to start with. Or maybe not. Connecting punctuation characters might be limited to the underline to start with, and may be also added into JavaLetterOrDigit. A sure thing is that underlines should be allowed embedded within non-ASCII identifiers. Is the unbreakable space a "connecting punctuation"? :-) Just for the amusement, I noticed that if file `francais.py' contains: -----------------------------------------------------------------------> # -*- coding: Latin-1 -*- ?l?ve = 3 print ?l?ve -----------------------------------------------------------------------< and file `francais' contains: -----------------------------------------------------------------------> import locale locale.setlocale(locale.LC_ALL, '') import francais -----------------------------------------------------------------------< then command `python francais', in my environment where `LANG' is set to `fr_CA.ISO-8859-1', does yield: ----------------------------------------------------------------------> 3 ----------------------------------------------------------------------< So, the Python compiler is sensitive to the active locale. Someone pointed out, a good while ago, that Latin-1 characters were accepted interactively because `readline' was setting the locale, but it seems that setting the locale ourselves allows for batch import as well. This is kind of an happy bug! May we count on it being supported in the interim? :-) :-) -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From pinard at iro.umontreal.ca Mon Feb 9 11:27:04 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 9 11:32:49 2004 Subject: [Python-Dev] Allowing non-ASCII identifiers In-Reply-To: <200401142016.i0EKGuV10980@c-24-5-183-134.client.comcast.net> References: <4005A1DA.207@v.loewis.de> <200401142016.i0EKGuV10980@c-24-5-183-134.client.comcast.net> Message-ID: <20040209162704.GB7467@titan.progiciels-bpi.ca> [Guido van Rossum] > Since [non-ASCII identifiers] could create a serious burden for code > portability, I'd like to see a serious section on motivation and > discussion on how to keep Unicode out of the standard library and out > of most 3rd party distributions. Without that I'm strongly -1. There are two matters here. One is the technical portability, the other is more related to human issues. The technical portability burden is fairly limited to the `coding:' magic, and not very different from it. As long as the coding is known to Python, various modules in a single application could use each their coding and consequently, each their lexical behaviour for identifiers. It should not be much of a problem in practice. For the standard library, say, it is a matter of ruling out `coding:' within developers. The human issues are another thing, and I would guess -- without being fully sure -- that this is where your reluctance stands. There is a fear that people would not only comment in German (I'm using German as an example, of course), or use German strings, but also choose identifiers based on German words instead of English words, making programs less easy to read by English people, or relieving a bit of this constant pressure all programmers on this planet have to learn English, pressure which is probably deemed useful towards this uniformity. Almost all "foreigners" already know that if they aim the international community while programming, English is likely the best way to communicate, and they consequently comment in English, and choose their identifiers for being meaningful to English readers. But, the same as not all software is meant to be free (whatever that word means), not all software is meant for international distribution. Some English readers might not really imagine, but it is a constant misery, having to mangle identifiers while documenting and thinking in languages other than English, merely because the Python notion of letter is limited to the English subset. Granted, keywords and standard library use English, this is Python, and this is not at stake here! However, there is a good part of code in local (or in-house) programs which is thought as our crafted code, and even the linguistic change is useful (to us) for segregating between what comes from the language and what comes from us. The idea is extremely appealing of being able to craft and polish our code (comments, strings, identifiers) to make it as nice as it could get, while thinking in our native, natural language. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From tjreedy at udel.edu Mon Feb 9 13:55:48 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Mon Feb 9 13:55:54 2004 Subject: [Python-Dev] Re: Allowing non-ASCII identifiers References: <4005A1DA.207@v.loewis.de><200401142016.i0EKGuV10980@c-24-5-183-134.client.comcast.net> <20040209162704.GB7467@titan.progiciels-bpi.ca> Message-ID: "François Pinard" wrote in message news:20040209162704.GB7467@titan.progiciels-bpi.ca... >There are two matters here. One is the technical portability, the other >is more related to human issues. I think there are two human issues: language and alphabet (character set) used to transcribe the language. >fear that people would not only comment in German (I'm using German >as an example, of course), or use German strings, but also choose >identifiers based on German words instead of English words, making >programs less easy to read by English people, This, of course, can be and is being done now as long as one drops euro accents (I know, eleve is probably ugly to you) or transliterates other alphabets to the English subset of latin chars. For instance, if I remember correctly, part of the Python scripting for Blade of Darkness was done by Spanish-speaking programmers using Spanish identifiers and comments. There have also been snippets posted on c.l.p with German and other Euro languange words. But I can potentially read, understand, and even edit such code. For another example, if I read for wa in konichi: ... konichi[wa] = I can recognize that the two occurences each of 'konichi' and 'wa' are the same, and that konichi is dict-like, even if I do not know their English meanings (assuming they are not gibberish). But if they were in Japanese chars, for instance, matching names to names would be *much* harder. Having at various times more or less learned, and partly forgotten, how to at least phonetically read words in Cyrillic, Greek, and Devanagari (Sanskrit) characters, I can appreciate that learning Latin chars must also be a chore to those who learned something else as children. But I also know that I do not necessarily wish to learn a dozen more sets, nor would I wish such on everyone else. Having said this against the proposal, I suspect that a Unicode-identifier version of Python, official or not, is inevitable, especially if, as and when Python spreads beyond the internationalized elite of each country. If so, I would like to see such at least accompanied by transliteration programs (preferably two-way) using the most accepted transliteration scheme for each alphabet. Terry J. Reedy From bac at OCF.Berkeley.EDU Mon Feb 9 14:46:32 2004 From: bac at OCF.Berkeley.EDU (Brett) Date: Mon Feb 9 14:46:46 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown Message-ID: <4027E398.6050704@ocf.berkeley.edu> In 2.3, if you have a thread running when the interpreter tears itself down you get an exception from what looks like the thread catching an exception from trying to access something that has been torn down which then raises its own exception from the fact that globals have already disappeared since it tries to access currentThread which is a global (all of this is in my last comment in the bug report). My question is whether this is worth fixing. Since it only occurs if you don't shut down your threads it seems like it is not really a bug but just unexpected behavior from not cleaning up after yourself. But, as I said in my bug report post, I think it is fixable. So is it even worth fixing in some non-pretty way? -Brett From martin at v.loewis.de Mon Feb 9 17:03:11 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Feb 9 17:03:30 2004 Subject: [Python-Dev] Allowing non-ASCII identifiers In-Reply-To: <20040209155117.GA7467@titan.progiciels-bpi.ca> References: <4005A1DA.207@v.loewis.de> <20040209155117.GA7467@titan.progiciels-bpi.ca> Message-ID: <4028039F.8040008@v.loewis.de> Fran?ois Pinard wrote: >>1. At run-time, identifiers are represented as Unicode objects unless >>they are pure ASCII. IOW, they are converted from the source encoding >>to Unicode objects in the process of parsing. > > > This is already the case, isn't it? Currently, all identifiers are byte strings, at run-time, representing ASCII characters. IOW, you currently won't observe Unicode strings as identifiers. >>2. As a consequence of 1), all places there identifiers appear need to >>support Unicode objects (e.g. __dict__, __getattr__, etc) > > > I do not much know the internals, yet I suspect one more thing to > consider is whether Unicode strings looking like non-ASCII identifiers > should be interned or not, the same as currently done for ASCII. Indeed; I had not thought about this. > # -*- coding: Latin-1 -*- > ?l?ve = 3 > print ?l?ve [...] > So, the Python compiler is sensitive to the active locale. Yes, that's a bug. It will use byte strings as identifiers (without running your example, I'd expect that dir() shows they are UTF-8) > This is kind of an happy bug! May we count on it being supported in the > interim? :-) :-) I would think so: this bug has been present for quite some time, and nobody complained :-) Martin From pinard at iro.umontreal.ca Mon Feb 9 17:05:06 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 9 17:10:34 2004 Subject: [Python-Dev] Re: Allowing non-ASCII identifiers In-Reply-To: References: <20040209162704.GB7467@titan.progiciels-bpi.ca> Message-ID: <20040209220506.GB14076@titan.progiciels-bpi.ca> [Terry Reedy] > I think there are two human issues: language and alphabet (character set) > used to transcribe the language. > [...] I can potentially read, understand, and even edit such code. > For another example, if I read > for wa in konichi: > ... > konichi[wa] = > I can recognize that the two occurences each of 'konichi' and 'wa' are > the same [...] if they were in Japanese chars, for instance, matching > names to names would be *much* harder. Hello, Terry, and other Python developers. Some people "pronounce" in their head, and recognise the music :-). For them, phonetic rendering is a help. Others would not have much trouble recognising recurring images, even if they do not associate sounds with them. Japanese or Korean people used to write to me once in a while, and I was quickly recognising their names written with their own script, without a real need to look at the accompanying phonetic transcription. You know, when I read English words, I do not pronounce them correctly in my head. English people do not see the difficulty for a foreigner to get the proper pronunciation (not even speaking of tonic accents, which I surely have all wrong most of the times!). English looks especially difficult to me, phonetically-wise. French is my own language, and I worked for a while on automatic phonetic transcriptions, and despite French looks more "regular" than English on that topic, I know for having done that automatic transcription is a difficult exercise. > But I also know that I do not necessarily wish to learn a dozen more > sets, nor would I wish such on everyone else. Nobody is really asking you to do so. As long as my co-workers and I could read each other, we are happy. We do not have the need that our code be understandable or easy everywhere else, this is not a problem. And if Japanese Pythoneers use Kanji or Hiragana, and are happy with them, I would surely not come and consider their pleasure is not worth, merely because I would need some adaptation to handle their code. The game here is to be comfortable in contexts where work is not meant to be widely disseminated. > I would like to see such [Unicode-identifier version of Python] at > least accompanied by transliteration programs (preferably two-way) > using the most accepted transliteration scheme for each alphabet. Such tools could be undoubtedly fun to have, but I suspect they would not really serve a strong need. When I read an English Python program, maybe that I would like a tool that will transliterate that English program into some phonetic alphabet so I can figure out how it should sound -- but this is a difficult and unnecessary challenge, deep down. I do not know Asian languages, yet people explained to me that the correspondence between pictogram and phonetics is far from being easy. The same pictogram could be used with different meanings and have different pronunciation. Even with the same meaning, the pronunciation could be very different, not only between regions, but also between countries (for example, Chinese and Japanese could figure out each other pictogram for many of them, but have absolutely no commonalty in pronunciation). The other way, from Hiragana to Kanji for Japanese, is also very difficult, because quite ambiguous as well. All this to say that best would be to let people be comfortable in their own language while using Python, without sketching difficult pre-requisites over the project, which do not buy us much in practice. In my previous works on internationalisation, one of the most recurrent problem I observed is people trying to solve others' problems, which they do not have themselves (they want to save the World![*]). A typical example would be that some people object to non-ASCII identifiers in Python, foreseeing the difficulties others might have to find proper editors for handling these sources. One can exhaust himself into such discussions. The proper attitude in such matters, in my opinion, is for one to concentrate on the technical problems related to Python implementation, yet without taking everything else on one's shoulders. If the job is well done enough, people will happily take care and adapt. -------------------- [*] (For trekkies:) They all have a little Wesley Crusher in them! :-) -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From martin at v.loewis.de Mon Feb 9 17:21:01 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Feb 9 17:21:28 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <2mllnccv9h.fsf@starship.python.net> References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> <40254968.4050205@v.loewis.de> <2mllnccv9h.fsf@starship.python.net> Message-ID: <402807CD.1060200@v.loewis.de> Michael Hudson wrote: > There were a small bundle of bug reports from a guy with libc5 a week > or so back, so there's at least some interest in that. OTOH, I don't > feel like putting all that much effort into fixing the problems he > reported (most of them were failry shallow, I think). So would you propose to "unsupport" Linux with libc 5? By PEP 11 rules, this would become "unsupported" in Python 2.4, with the possibility of revival if a volunteer steps forward. Regards, Martin P.S. I don't like to call this "We deprecate libc5", because we are in no position to decide the future of libc5 (we are not maintaining it in the first place). Instead, we just stop supporting it. From martin at v.loewis.de Mon Feb 9 17:26:21 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Feb 9 17:43:43 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: <4027E398.6050704@ocf.berkeley.edu> References: <4027E398.6050704@ocf.berkeley.edu> Message-ID: <4028090D.5050003@v.loewis.de> Brett wrote: > My question is whether this is worth fixing. Since it only occurs if > you don't shut down your threads it seems like it is not really a bug > but just unexpected behavior from not cleaning up after yourself. But, > as I said in my bug report post, I think it is fixable. So is it even > worth fixing in some non-pretty way? IMO, any bug is worth fixing. This being free software, contributors are free to chose what they work on. It would be much more useful if you would provide thread cancellation in the process :-) Regards, Martin From pinard at iro.umontreal.ca Mon Feb 9 17:43:59 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 9 17:54:10 2004 Subject: [Python-Dev] Allowing non-ASCII identifiers In-Reply-To: <4028039F.8040008@v.loewis.de> References: <4005A1DA.207@v.loewis.de> <20040209155117.GA7467@titan.progiciels-bpi.ca> <4028039F.8040008@v.loewis.de> Message-ID: <20040209224359.GA17199@titan.progiciels-bpi.ca> [Martin von L?wis] > Fran?ois Pinard wrote: > >>1. At run-time, identifiers are represented as Unicode objects unless > >>they are pure ASCII. IOW, they are converted from the source encoding > >>to Unicode objects in the process of parsing. > >This is already the case, isn't it? > Currently, all identifiers are byte strings, at run-time, representing > ASCII characters. IOW, you currently won't observe Unicode strings > as identifiers. Oops, sorry. I misread your sentence as limiting itself to identifiers. I thought having read that the effect of `coding:' was to convert the whole source to Unicode before the scanner pass. This is all from fuzzy memory. > >I do not much know the internals, yet I suspect one more thing to > >consider is whether Unicode strings looking like non-ASCII identifiers > >should be interned or not, the same as currently done for ASCII. > Indeed; I had not thought about this. This is only an optimisation consideration, which might be premature. On the other hand, speed considerations should not be serious enough to play against one willing to write national identifiers, in the long run. > ># -*- coding: Latin-1 -*- > >?l?ve = 3 > >print ?l?ve > [...] > >So, the Python compiler is sensitive to the active locale. > Yes, that's a bug. It will use byte strings as identifiers (without > running your example, I'd expect that dir() shows they are UTF-8) They seem to be Latin-1. Consider that characters could not be classified correctly in a Latin-1 environment, if they were UTF-8. > >This is kind of an happy bug! May we count on it being supported in the > >interim? :-) :-) > I would think so: this bug has been present for quite some time, > and nobody complained :-) Would Guido accept to pronounce on that? :-) As much as we starve to write our things in better French, we would not allow ourselves to write code that will likely break later. We work in a production environment. A Python command option to set the locale from environment variables, before compilation of `__main__' starts, but that might be too much effort in the wrong direction. Best and simplest would be that the `coding:' pragma really drives the character set used in a file. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From pinard at iro.umontreal.ca Mon Feb 9 19:17:07 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 9 19:21:25 2004 Subject: [Python-Dev] Allowing non-ASCII identifiers In-Reply-To: <20040209224359.GA17199@titan.progiciels-bpi.ca> References: <4005A1DA.207@v.loewis.de> <20040209155117.GA7467@titan.progiciels-bpi.ca> <4028039F.8040008@v.loewis.de> <20040209224359.GA17199@titan.progiciels-bpi.ca> Message-ID: <20040210001707.GA19316@titan.progiciels-bpi.ca> [Fran?ois Pinard] > [Martin von L?wis] > > Fran?ois Pinard wrote: > > >>1. At run-time, identifiers are represented as Unicode objects unless > > >>they are pure ASCII. IOW, they are converted from the source encoding > > >>to Unicode objects in the process of parsing. > > >This is already the case, isn't it? > > Currently, all identifiers are byte strings, at run-time, representing > > ASCII characters. IOW, you currently won't observe Unicode strings > > as identifiers. > Oops, sorry. I misread your sentence as limiting itself to identifiers. > I thought having read that the effect of `coding:' was to convert the > whole source to Unicode before the scanner pass. This is all from fuzzy > memory. Re-oops! Really, I ought to be tired for writing so ambiguously. Should have written something more like: Oops, sorry. I misread your sentence, and missed the fact that it was limiting itself to identifiers. I thought I once read that the effect of `coding:' ... [etc.] > > ># -*- coding: Latin-1 -*- > > >?l?ve = 3 > > >print ?l?ve > > [...] > > >This is kind of an happy bug! May we count on it being supported in the > > >interim? :-) :-) > > I would think so: this bug has been present for quite some time, > > and nobody complained :-) > Would Guido accept to pronounce on that? :-) I'm still ambiguous above... Tss, tss! Would Guido pronounce on the fact that the bug will _not_ be corrected, at least not until Python supports non-ASCII identifiers in more complete and correct ways? -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From bac at OCF.Berkeley.EDU Mon Feb 9 19:24:17 2004 From: bac at OCF.Berkeley.EDU (Brett) Date: Mon Feb 9 19:24:28 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: <4028090D.5050003@v.loewis.de> References: <4027E398.6050704@ocf.berkeley.edu> <4028090D.5050003@v.loewis.de> Message-ID: <402824B1.8040305@ocf.berkeley.edu> Martin v. L?wis wrote: > Brett wrote: > >> My question is whether this is worth fixing. Since it only occurs if >> you don't shut down your threads it seems like it is not really a bug >> but just unexpected behavior from not cleaning up after yourself. >> But, as I said in my bug report post, I think it is fixable. So is it >> even worth fixing in some non-pretty way? > > > IMO, any bug is worth fixing. This being free software, contributors are > free to chose what they work on. > So you are classifying this as a bug then. That was part of my question since it only happens if you don't handle your threads properly (or at least I think properly; I would assume it is common practice to close down your threads). > It would be much more useful if you would provide thread cancellation > in the process :-) > For some crazy reason having to delve into the world of thread libraries for those cases where there is not an equivalent pthread_cancel does not sound fun. =) -Brett From pje at telecommunity.com Mon Feb 9 19:39:47 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Feb 9 19:35:26 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: <402824B1.8040305@ocf.berkeley.edu> References: <4028090D.5050003@v.loewis.de> <4027E398.6050704@ocf.berkeley.edu> <4028090D.5050003@v.loewis.de> Message-ID: <5.1.0.14.0.20040209193747.026f8030@mail.telecommunity.com> At 04:24 PM 2/9/04 -0800, Brett wrote: >For some crazy reason having to delve into the world of thread libraries >for those cases where there is not an equivalent pthread_cancel does not >sound fun. =) Presumably, you could make those either raise NotImplementedError, or not export the cancellation function to the thread module. Since people writing backward-compatible code will want to check for the existence of such a function anyway, it should not be a significant drawback to provide it only on platforms that are easily supported. And, once you've written an example, others who have more familiarity with those other platforms might be more motivated to supply patches adding the feature to those as well. :) From raymond.hettinger at verizon.net Tue Feb 10 02:15:39 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Tue Feb 10 02:17:10 2004 Subject: [Python-Dev] Optimization of the Year Message-ID: <00c401c3efa5$b08d5300$2008a044@oemcomputer> Hey, hey! After looking at this on and off for months, I finally found a safe way to dramatically improve the speed of building lists: http://users.rcn.com/python/download/list.diff We've always known that every list.append() resulted in a realloc() call after computing an adaptive, oversized block length. The obvious solution was to save the block size and avoid calls to realloc() when the block was already large enough to hold a new element. Unfortunately, the list structure was wide open to manipulation by every piece of code written since Python's infancy. Almost no assumptions could be made about other code directly hacking the list structure elements. The safe solution turned out to be saving a copy of the ob_item pointer whenever a block of known size was allocated. That copy is then used a guard for the size check. That way, if the pointer is swapped out by another routine (not a myth, it happens), we assume our saved block size is invalid and just do a normal realloc. The results are wonderful: * list(it) speeds up by at least 40% when "it" is an iterable not defining __len__ (there is no change for iterables that do define __len__ because they pre-size in a single step). . list comprehensions show a similar speed-up . list.append(x) is about twice as fast AFAICT, this is a straight-win with no trade-offs along the way. The only offsetting cost is adding two fields to the list structure (Tim said that was bad but I don't remember why). Please take a look and see if there are any holes in the theory. While the patch passes the test suite, it is possible that some extension module reallocs() the list to a smaller size (leaving the pointer unchanged) and then changes its mind and starts growing the list again. Given the magnitude of the speed-up, I think that risk is warranted. The offending code would have to be by-passing the C API for lists and tinkering with the internals in atypical way (swapping one pointer for another and/or altering ob_size is typical; downward resizing the existing pointer is not). Happy-tonight-ly yours, Raymond Hettinger P.S. SF is down for maintenance. Will post the patch there tomorrow. While the change is simple, the patch is long because all the NRESIZE macros had to be replaced by a function call. From perky at i18n.org Tue Feb 10 06:13:10 2004 From: perky at i18n.org (Hye-Shik Chang) Date: Tue Feb 10 06:10:48 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <00c401c3efa5$b08d5300$2008a044@oemcomputer> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> Message-ID: <20040210111310.GA31007@i18n.org> On Tue, Feb 10, 2004 at 02:15:39AM -0500, Raymond Hettinger wrote: > Hey, hey! After looking at this on and off for months, I finally > found a safe way to dramatically improve the speed of building lists: > > http://users.rcn.com/python/download/list.diff > What a surprise! It accellated pystone speed from 17583 to 21022 on my machine!! FYI, regrtests of python itself and several 3rd party extension modules are passed also. Great work! :) > [snip] > Please take a look and see if there are any holes in the theory. Your explanation sounds reasonable to me. And the patch looks good. (a minor cosmetics: unnecessary two semicolons on line 1876 ;) > > While the patch passes the test suite, it is possible that some > extension module reallocs() the list to a smaller size (leaving > the pointer unchanged) and then changes its mind and starts growing > the list again. Given the magnitude of the speed-up, I think that > risk is warranted. Indeed. Cheers, Hye-Shik From mwh at python.net Tue Feb 10 06:12:15 2004 From: mwh at python.net (Michael Hudson) Date: Tue Feb 10 06:12:18 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <402807CD.1060200@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Mon, 09 Feb 2004 23:21:01 +0100") References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> <40254968.4050205@v.loewis.de> <2mllnccv9h.fsf@starship.python.net> <402807CD.1060200@v.loewis.de> Message-ID: <2msmhj9oj4.fsf@starship.python.net> "Martin v. L?wis" writes: > Michael Hudson wrote: >> There were a small bundle of bug reports from a guy with libc5 a week >> or so back, so there's at least some interest in that. OTOH, I don't >> feel like putting all that much effort into fixing the problems he >> reported (most of them were failry shallow, I think). > > So would you propose to "unsupport" Linux with libc 5? Yes, for what that's worth. I'm not aware of any code specifically in aid of libc5, though... > By PEP 11 rules, this would become "unsupported" in Python 2.4, with > the possibility of revival if a volunteer steps forward. ... so basically this amounts to saying "if you have a problem with python on a libc5 system you'd better have a patch too". Cheers, mwh -- Famous remarks are very seldom quoted correctly. -- Simeon Strunsky From guido at python.org Tue Feb 10 10:40:28 2004 From: guido at python.org (Guido van Rossum) Date: Tue Feb 10 10:40:34 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Your message of "Tue, 10 Feb 2004 02:15:39 EST." <00c401c3efa5$b08d5300$2008a044@oemcomputer> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> Message-ID: <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> > Hey, hey! After looking at this on and off for months, I finally > found a safe way to dramatically improve the speed of building lists: > > http://users.rcn.com/python/download/list.diff Very cool, but... > We've always known that every list.append() resulted in a realloc() > call after computing an adaptive, oversized block length. > > The obvious solution was to save the block size and avoid calls to > realloc() when the block was already large enough to hold a new > element. Unfortunately, the list structure was wide open to > manipulation by every piece of code written since Python's infancy. > Almost no assumptions could be made about other code directly > hacking the list structure elements. Disagree. The public API documentation has never permitted modifying ob_item and ob_size directly. It would be breaking the abstraction. > The safe solution turned out to be saving a copy of the ob_item pointer > whenever a block of known size was allocated. That copy is then > used a guard for the size check. That way, if the pointer is swapped > out by another routine (not a myth, it happens), we assume our saved > block size is invalid and just do a normal realloc. "It happens." Where? If in the core, that code should simply be fixed. If in 3rd party code, that code is simply wrong. If indeed such 3rd party code exists, and we expect we can't get it all fixed before 2.4 is released, the tracked_item hack can be used as a temporary measure to hunt down all those 3rd party extensions that break the abstraction. I propose to issue a warning when it is discovered that ob_item != tracked_item. Then in 2.5 we can remove the tracked_item feature. Keeping tracked_item permanently as part of the API would be a nightmare. How would you document it? "You can make ob_item point to another malloc()ed block, as long as you update ob_size to reflect the size of the block, and as long as you free() the old block, and oh, by the way, you may also use realloc(), but if realloc() just happens to extend or truncate the block without moving it, your code may die in unexpected ways. > The results are wonderful: > > * list(it) speeds up by at least 40% when "it" is an iterable > not defining __len__ (there is no change for iterables that > do define __len__ because they pre-size in a single step). > > . list comprehensions show a similar speed-up > > . list.append(x) is about twice as fast No doubt about it! (I think Tim's earlier attempt had about the same performance.) > AFAICT, this is a straight-win with no trade-offs along the way. > The only offsetting cost is adding two fields to the list > structure (Tim said that was bad but I don't remember why). obmalloc rounds small block sizes up to multiples of 8. A list object needs 3 32-bit words for the GC struct, then 1 for the refcount, 1 for the type pointer, 1 for the size, and 1 for ob_item. That's 28 bytes. So we can add one word for free -- but a second one bumps the allocated size from 32 to 40. > Please take a look and see if there are any holes in the theory. > > While the patch passes the test suite, it is possible that some > extension module reallocs() the list to a smaller size (leaving > the pointer unchanged) and then changes its mind and starts growing > the list again. Given the magnitude of the speed-up, I think that > risk is warranted. The offending code would have to be by-passing > the C API for lists and tinkering with the internals in atypical > way (swapping one pointer for another and/or altering ob_size is > typical; downward resizing the existing pointer is not). We can't be wishy-washy about this. Either we allow external code to tinker, or we don't. If, in fact, there is 3rd party code that tinkers in illegal ways, we should try to get it fixed, not work around it forever. > P.S. SF is down for maintenance. Will post the patch there tomorrow. > While the change is simple, the patch is long because all the NRESIZE > macros had to be replaced by a function call. I just got email from SF announcing their "premium" (for-fee) service. Sigh. How long before we will have to pay for everything except read-only one-day-out-of-day CVS access? --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Tue Feb 10 10:57:37 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Tue Feb 10 10:57:55 2004 Subject: [Python-Dev] Re: Optimization of the Year References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> Message-ID: "Raymond Hettinger" wrote in message news:00c401c3efa5$b08d5300$2008a044@oemcomputer... > risk is warranted. The offending code would have to be by-passing > the C API for lists and tinkering with the internals in atypical > way (swapping one pointer for another and/or altering ob_size is > typical; downward resizing the existing pointer is not). Hasn't or can't such bypass and tinkering be defined as 'unsupported, proceed at your own risk, and don't complain when internals change', just as when accessing certain internals visible from Python code? TJR From tim.one at comcast.net Tue Feb 10 11:04:02 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 10 11:04:07 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <00c401c3efa5$b08d5300$2008a044@oemcomputer> Message-ID: [Raymond] > Hey, hey! After looking at this on and off for months, I finally > found a safe way to dramatically improve the speed of building lists: > > http://users.rcn.com/python/download/list.diff Cool! > ... > The safe solution turned out to be saving a copy of the ob_item > pointer whenever a block of known size was allocated. That copy is > then used a guard for the size check. That way, if the pointer is > swapped out by another routine (not a myth, it happens), we assume > our saved block size is invalid and just do a normal realloc. Share Guido's confusion/outrage about this: who does this? Would rather shoot them than humor them. > The results are wonderful: They'll vary massively across platforms (because realloc() performance varies massively across platforms -- we already saw that in the flurry of timing reports following my proof-of-concept patch). > ... > AFAICT, this is a straight-win with no trade-offs along the way. > The only offsetting cost is adding two fields to the list > structure (Tim said that was bad but I don't remember why). Guido explained it: we can add one 4-byte field to the list object header on a 32-bit box for free now, but adding two grows every list object by 8 bytes. > Please take a look and see if there are any holes in the theory. I can't look closely now, but I'll (re)note one thing: the roundupsize() algorithm is so convoluted now because it has to arrange to return the same result over long stretches of contiguous inputs, in order that *most* realloc() calls end up being trivial now. But if we're only calling realloc() when realloc() is actually needed, a much simpler approach is desirable; e.g., extra_len = (requested_len >> 4) + 2; allocated_len = requested_len + extra_len; The "+2" is to ensure that extra_len > 0 even when requested_len < 16. This is still very mild over-allocation, on the order of 6%. I'd be comfortable knocking the shift of 4 down to 3, so long as PyList_New(size) continues not to overallocate at all. From kbk at shore.net Tue Feb 10 11:05:00 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Feb 10 11:05:06 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <00c401c3efa5$b08d5300$2008a044@oemcomputer> (Raymond Hettinger's message of "Tue, 10 Feb 2004 02:15:39 -0500") References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> Message-ID: <874qty3opf.fsf@hydra.localdomain> "Raymond Hettinger" writes: > AFAICT, this is a straight-win with no trade-offs along the way. > The only offsetting cost is adding two fields to the list > structure (Tim said that was bad but I don't remember why). He said: ============== That Python's builtin list calls realloc() on every append is pretty gross, but it's also a tradeoff, saving the bytes in the list header that would be required to remember how many allocated-but-unused slots exist. If you've got a million tiny lists (and some apps do), burning an additional 8MB for something your app doesn't really need isn't attractive (on a 32-bit box today, the list header is 16 bytes; pymalloc aligns to 8-byte boundaries, so the next possible size is 24 bytes). list's very small overallocation (in the ballpark of 5%) is also aimed at minimizing memory burden. Speed isn't everything, tradeoffs are hard, and something loses when something else is favored. (BTW, I never would have considered implementing a resizable list type *without* keeping a field to record the amount of overallocation, so this isn't the tradeoff I would have made .) ============== -- KBK From edcjones at erols.com Tue Feb 10 11:07:29 2004 From: edcjones at erols.com (Edward C. Jones) Date: Tue Feb 10 11:18:08 2004 Subject: [Python-Dev] Syntax for "except" Message-ID: <402901C1.4040405@erols.com> Why was Python syntax designed so except (TypeError, ValueError): is OK but except [TypeError, ValueError]: is forbidden. Should this be changed? Is immutability needed here? Where in Python is immutability really needed or really improves efficiency? From tim.one at comcast.net Tue Feb 10 11:26:55 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 10 11:27:02 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Message-ID: [Tim] > ... > Guido explained it: we can add one 4-byte field to the list object > header on a 32-bit box for free now, but adding two grows every list > object by 8 bytes. To be more precise, we can add one 4-byte field for free on 32-bit boxes under compilers where "long double" has 4-byte alignment. We can't add anything for free under compilers where "long double" has 8-byte alignment; this includes Microsoft's compiler, alas. (The gc header is forced to long double alignment via a union trick, and that makes the gc header consume 16 bytes (4 of them unused padding) under MSVC.) From aahz at pythoncraft.com Tue Feb 10 11:39:04 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Feb 10 11:39:46 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: References: Message-ID: <20040210163904.GA29632@panix.com> On Tue, Feb 10, 2004, Tim Peters wrote: > [Tim] >> ... >> Guido explained it: we can add one 4-byte field to the list object >> header on a 32-bit box for free now, but adding two grows every list >> object by 8 bytes. > > To be more precise, we can add one 4-byte field for free on 32-bit > boxes under compilers where "long double" has 4-byte alignment. We > can't add anything for free under compilers where "long double" has > 8-byte alignment; this includes Microsoft's compiler, alas. (The gc > header is forced to long double alignment via a union trick, and that > makes the gc header consume 16 bytes (4 of them unused padding) under > MSVC.) So in other words, we might as well go ahead and trade space for speed? If someone really does have a million small lists, they're probably mutating some of them constantly (or they'd use tuples). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From python at rcn.com Tue Feb 10 11:47:00 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Feb 10 11:48:32 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> Message-ID: <001a01c3eff5$81a36160$fd2bc797@oemcomputer> > If indeed such 3rd party code exists, and we expect we can't get it > all fixed before 2.4 is released, the tracked_item hack can be used as > a temporary measure to hunt down all those 3rd party extensions that > break the abstraction. I propose to issue a warning when it is > discovered that ob_item != tracked_item. Then in 2.5 we can remove > the tracked_item feature. That is a reasonable transition strategy, take the immediate gain and then phase out list->tracked_item. The fix for offending code is a simple rule: if you mess with list->ob_item, then invalidate the list->allocated field by setting it to -1. After adding the warning and commenting the invalidation rule, is it okay to check this one in? > (I think Tim's earlier attempt had about the same > performance.) Right. Though this one doesn't crash the testsuite and it doesn't recompute roundupsize() on every call ;-) > We can't be wishy-washy about this. Either we allow external code to > tinker, or we don't. If, in fact, there is 3rd party code that > tinkers in illegal ways, we should try to get it fixed, not work > around it forever. Now with a fast PyList_Append(), there is much less of a reason to want to hack through the abstraction. Still, I'm okay with allowing tinkering using the above invalidation rule. Raymond Hettinger From guido at python.org Tue Feb 10 11:51:09 2004 From: guido at python.org (Guido van Rossum) Date: Tue Feb 10 11:51:15 2004 Subject: [Python-Dev] Syntax for "except" In-Reply-To: Your message of "Tue, 10 Feb 2004 11:07:29 EST." <402901C1.4040405@erols.com> References: <402901C1.4040405@erols.com> Message-ID: <200402101651.i1AGp9I31355@c-24-5-183-134.client.comcast.net> > Why was Python syntax designed so > except (TypeError, ValueError): > is OK but > except [TypeError, ValueError]: > is forbidden. Should this be changed? Is immutability needed here? Where > in Python is immutability really needed or really improves efficiency? I never *intended* this to be seen as a container, but rather as a use of parentheses for grouping. If I were to revise the syntax, I would probably change it around completely to avoid the common mistake of writing except TypeError, ValueError: ... (but I don't have a proposal). --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Tue Feb 10 11:55:21 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Feb 10 11:58:09 2004 Subject: [Python-Dev] Decimal.py in sandbox In-Reply-To: References: Message-ID: <20040210165521.GA6960@panix.com> On Tue, Feb 10, 2004, Batista, Facundo wrote: > Kevin Jacobs wrote: > > #- I'd be happier with at least one round of review before > #- committing to CVS. > #- The code is fairly complex and an extra set of eyes will help keep > #- things focused. I've also volunteered to be that extra set > #- of eyes, and > #- plan a quick turn-around on any patches sent to me. > #- > #- However, I don't have CVS write permission either, even to > #- the sandbox. > > So, I sent to Kevin the first diffs about Decimal.py, and he was OK to them. > > They're mostly about cosmetics, but I wanted a few not fundamental changes > to get the circuit working and then start with heavier changes. > > To whom I shall send the diffs to get committed? Should I commit them to > the sandbox myself? Given that it's sandbox, go ahead and commit yourself. We can always revert. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From guido at python.org Tue Feb 10 11:58:07 2004 From: guido at python.org (Guido van Rossum) Date: Tue Feb 10 11:58:14 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Your message of "Tue, 10 Feb 2004 11:47:00 EST." <001a01c3eff5$81a36160$fd2bc797@oemcomputer> References: <001a01c3eff5$81a36160$fd2bc797@oemcomputer> Message-ID: <200402101658.i1AGw7231399@c-24-5-183-134.client.comcast.net> > > If indeed such 3rd party code exists, and we expect we can't get it > > all fixed before 2.4 is released, the tracked_item hack can be used as > > a temporary measure to hunt down all those 3rd party extensions that > > break the abstraction. I propose to issue a warning when it is > > discovered that ob_item != tracked_item. Then in 2.5 we can remove > > the tracked_item feature. > > That is a reasonable transition strategy, take the immediate gain > and then phase out list->tracked_item. The fix for offending code is > a simple rule: if you mess with list->ob_item, then invalidate the > list->allocated field by setting it to -1. Still disagree. What if we have an even better idea next year? Offending code should switch to using the documented list APIs. I think the 'allocated' member (or any other member for that matter) should not be part of the documented API. > Now with a fast PyList_Append(), there is much less of a reason > to want to hack through the abstraction. Still, I'm okay with > allowing tinkering using the above invalidation rule. I'm not. We need to make our abstractions tighter, not looser. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Tue Feb 10 12:00:33 2004 From: skip at pobox.com (Skip Montanaro) Date: Tue Feb 10 12:00:47 2004 Subject: [Python-Dev] Another big ANSI-fication change - another good chance for testing Message-ID: <16425.3633.34134.717030@montanaro.dyndns.org> I just checked in changes to a fair number of files to no longer test for the presence of header files required by ANSI C (C89). I ran the test suite successfully on Mac OS X (gcc 3.3) and Solaris 8 (gcc 2.95.3). On Mandrake 8.1 (gcc 3.0.1) I had some trouble with tests related to the bsddb package, but I was having those same problems before applying my changes. Once I can do a cvs up again I'll try cygwin as well. Obvious stuff I couldn't check: Windows, OS/2, RISCOS. (The handcrafted pyconfig.h files for all changed.) Please give it a whirl. Skip From pje at telecommunity.com Tue Feb 10 12:06:23 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Feb 10 12:02:03 2004 Subject: [Python-Dev] Syntax for "except" In-Reply-To: <200402101651.i1AGp9I31355@c-24-5-183-134.client.comcast.ne t> References: <402901C1.4040405@erols.com> Message-ID: <5.1.0.14.0.20040210120116.02601170@mail.telecommunity.com> At 08:51 AM 2/10/04 -0800, Guido van Rossum wrote: > > Why was Python syntax designed so > > except (TypeError, ValueError): > > is OK but > > except [TypeError, ValueError]: > > is forbidden. Should this be changed? Is immutability needed here? Where > > in Python is immutability really needed or really improves efficiency? > >I never *intended* this to be seen as a container, but rather as a use >of parentheses for grouping. But the current implementation *does* have a nice side effect of allowing error psuedoclasses to be created, e.g.: FooOrBarError = FooError, BarError try: ... except FooOrBarError: ... This can be useful for dealing with stuff like DBAPI errors, where you're writing code that may need to work with multiple DBAPI modules' errors simultaneously. Since FooOrBarError can be rebound to reflect the possible errors in play, it allows some dyanmism without having to write a bunch of Python code to introspect sys.exc_info(), while also making the code's intention clearer than if it had to do such introspection. From FBatista at uniFON.com.ar Tue Feb 10 11:51:20 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Feb 10 12:07:56 2004 Subject: [Python-Dev] Decimal.py in sandbox Message-ID: Kevin Jacobs wrote: #- I'd be happier with at least one round of review before #- committing to CVS. #- The code is fairly complex and an extra set of eyes will help keep #- things focused. I've also volunteered to be that extra set #- of eyes, and #- plan a quick turn-around on any patches sent to me. #- #- However, I don't have CVS write permission either, even to #- the sandbox. So, I sent to Kevin the first diffs about Decimal.py, and he was OK to them. They're mostly about cosmetics, but I wanted a few not fundamental changes to get the circuit working and then start with heavier changes. To whom I shall send the diffs to get committed? Should I commit them to the sandbox myself? Thanks for all! . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040210/f62f33e2/attachment.html From perky at i18n.org Tue Feb 10 12:06:50 2004 From: perky at i18n.org (Hye-Shik Chang) Date: Tue Feb 10 12:21:57 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> Message-ID: <20040210170650.GA35930@i18n.org> On Tue, Feb 10, 2004 at 11:04:02AM -0500, Tim Peters wrote: > [Raymond] > > Hey, hey! After looking at this on and off for months, I finally > > found a safe way to dramatically improve the speed of building lists: > > > > http://users.rcn.com/python/download/list.diff > > Cool! > I made an updated patch very slightly modified from Raymond's. Dropped use of track_* member variables and applied simpler extra size calculation suggested by Tim. http://people.freebsd.org/~perky/list-r2.diff.txt Hye-Shik From andersjm at dancontrol.dk Tue Feb 10 12:25:18 2004 From: andersjm at dancontrol.dk (Anders J. Munch) Date: Tue Feb 10 12:25:31 2004 Subject: [Python-Dev] Optimization of the Year References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> Message-ID: <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> From: "Guido van Rossum" > > If in 3rd party code, that code is simply wrong. > > If indeed such 3rd party code exists, and we expect we can't get it > all fixed before 2.4 is released, the tracked_item hack can be used as > a temporary measure to hunt down all those 3rd party extensions that > break the abstraction. I propose to issue a warning when it is > discovered that ob_item != tracked_item. Then in 2.5 we can remove > the tracked_item feature. There is a simpler way to make sure all clients that illicitly use ob_item are updated appropriately: Break the build by renaming. If ob_item is renamed to ob_items or ob_item2 or whatever, any code that uses the old name will cease to compile. - Anders From FBatista at uniFON.com.ar Tue Feb 10 12:12:17 2004 From: FBatista at uniFON.com.ar (Batista, Facundo) Date: Tue Feb 10 12:32:49 2004 Subject: [Python-Dev] Decimal.py in sandbox Message-ID: Aahz wrote: #- On Tue, Feb 10, 2004, Batista, Facundo wrote: #- > To whom I shall send the diffs to get committed? Should I #- commit them to #- > the sandbox myself? #- #- Given that it's sandbox, go ahead and commit yourself. We can always #- revert. ;-) With what user? I'm facundobatista in SF. Have I, as facundobatista, commit privileges in the python sandbox? What should I do to have them? Thank you! . Facundo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ADVERTENCIA La informaci?n contenida en este mensaje y cualquier archivo anexo al mismo, son para uso exclusivo del destinatario y pueden contener informaci?n confidencial o propietaria, cuya divulgaci?n es sancionada por la ley. Si Ud. No es uno de los destinatarios consignados o la persona responsable de hacer llegar este mensaje a los destinatarios consignados, no est? autorizado a divulgar, copiar, distribuir o retener informaci?n (o parte de ella) contenida en este mensaje. Por favor notif?quenos respondiendo al remitente, borre el mensaje original y borre las copias (impresas o grabadas en cualquier medio magn?tico) que pueda haber realizado del mismo. Todas las opiniones contenidas en este mail son propias del autor del mensaje y no necesariamente coinciden con las de Telef?nica Comunicaciones Personales S.A. o alguna empresa asociada. Los mensajes electr?nicos pueden ser alterados, motivo por el cual Telef?nica Comunicaciones Personales S.A. no aceptar? ninguna obligaci?n cualquiera sea el resultante de este mensaje. Muchas Gracias. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040210/3846f77e/attachment.html From perky at i18n.org Tue Feb 10 12:54:24 2004 From: perky at i18n.org (Hye-Shik Chang) Date: Tue Feb 10 12:52:00 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> Message-ID: <20040210175424.GA37056@i18n.org> On Tue, Feb 10, 2004 at 06:25:18PM +0100, Anders J. Munch wrote: > From: "Guido van Rossum" > > > > If in 3rd party code, that code is simply wrong. > > > > If indeed such 3rd party code exists, and we expect we can't get it > > all fixed before 2.4 is released, the tracked_item hack can be used as > > a temporary measure to hunt down all those 3rd party extensions that > > break the abstraction. I propose to issue a warning when it is > > discovered that ob_item != tracked_item. Then in 2.5 we can remove > > the tracked_item feature. > > There is a simpler way to make sure all clients that illicitly use > ob_item are updated appropriately: Break the build by renaming. If > ob_item is renamed to ob_items or ob_item2 or whatever, any code that > uses the old name will cease to compile. > Something like this? typedef struct { PyObject_VAR_HEAD #ifdef Py_BUILD_CORE PyObject **ob_item; #else PyObject **__ob_item; #endif }; Hye-Shik From aleaxit at yahoo.com Tue Feb 10 12:16:58 2004 From: aleaxit at yahoo.com (Alex Martelli) Date: Tue Feb 10 12:54:11 2004 Subject: [Python-Dev] Syntax for "except" In-Reply-To: <5.1.0.14.0.20040210120116.02601170@mail.telecommunity.com> References: <402901C1.4040405@erols.com> <5.1.0.14.0.20040210120116.02601170@mail.telecommunity.com> Message-ID: <200402101816.58304.aleaxit@yahoo.com> On Tuesday 10 February 2004 06:06 pm, Phillip J. Eby wrote: ... > But the current implementation *does* have a nice side effect of allowing > error psuedoclasses to be created, e.g.: "The secret of life if having nice side effects" -- a Peanuts quote, I believe, though I can't confirm that with Google. I _do_ entirely agree that using a tuple of possible exception classes is sometimes VERY useful indeed! Alex From guido at python.org Tue Feb 10 12:56:44 2004 From: guido at python.org (Guido van Rossum) Date: Tue Feb 10 12:56:51 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Your message of "Tue, 10 Feb 2004 18:25:18 +0100." <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> Message-ID: <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> > There is a simpler way to make sure all clients that illicitly use > ob_item are updated appropriately: Break the build by renaming. If > ob_item is renamed to ob_items or ob_item2 or whatever, any code that > uses the old name will cease to compile. Good idea. We'd also have to break binary compatibility (of the List API) forcibly otherwise we could be fooled by 3rd party extensions compiled for older Python versions -- usually we're careful to keep binary compatibility. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Tue Feb 10 13:23:58 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Feb 10 13:24:45 2004 Subject: [Python-Dev] Decimal.py in sandbox In-Reply-To: References: Message-ID: <20040210182358.GA6318@panix.com> On Tue, Feb 10, 2004, Batista, Facundo wrote: > > Aahz wrote: > > #- On Tue, Feb 10, 2004, Batista, Facundo wrote: > #- > To whom I shall send the diffs to get committed? Should I > #- commit them to > #- > the sandbox myself? > #- > #- Given that it's sandbox, go ahead and commit yourself. We can always > #- revert. ;-) > > With what user? I'm facundobatista in SF. > Have I, as facundobatista, commit privileges in the python sandbox? > What should I do to have them? Ah, I assumed you already had privs. You'll need to send e-mail to one of the project admins; I suggest Tim (assuming he doesn't respond to your post). -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death." --GvR From python at rcn.com Tue Feb 10 12:08:39 2004 From: python at rcn.com (Raymond Hettinger) Date: Tue Feb 10 13:42:00 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Message-ID: <002201c3eff8$87686700$fd2bc797@oemcomputer> > > The results are wonderful: > > They'll vary massively across platforms (because realloc() performance > varies massively across The irony is that if the improvement is big enough to make you happy, then it means that you have a crummy realloc() and have to live with it for a long time. Raymond From tim.one at comcast.net Tue Feb 10 13:52:11 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 10 13:52:15 2004 Subject: [Python-Dev] Decimal.py in sandbox In-Reply-To: <20040210182358.GA6318@panix.com> Message-ID: [Aahz] >> Given that it's sandbox, go ahead and commit yourself. We can >> always revert. ;-) [Batista, Facundo] > With what user? I'm facundobatista in SF. > Have I, as facundobatista, commit privileges in the python sandbox? > What should I do to have them? You have them now, although it make take SF a while to propagate the changes. Note that there's no way to grant *just* sandbox commit privileges -- you have commit privs across the entire project now. That's mostly just to warn you to be careful when doing a commit, to avoid checking in private experimental changes to Python. From martin at v.loewis.de Tue Feb 10 13:54:00 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Feb 10 13:57:03 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: <402824B1.8040305@ocf.berkeley.edu> References: <4027E398.6050704@ocf.berkeley.edu> <4028090D.5050003@v.loewis.de> <402824B1.8040305@ocf.berkeley.edu> Message-ID: <402928C8.5050207@v.loewis.de> Brett wrote: > So you are classifying this as a bug then. That was part of my question > since it only happens if you don't handle your threads properly (or at > least I think properly; I would assume it is common practice to close > down your threads). Certainly: in a properly working program, one would terminate all threads first before trying to exit the program. Failure to do so should cause an exception (or otherwise have an "obvious" meaning). However, that exception should not be a "double-fault"; instead, it should give a clear indication what the problem is. > For some crazy reason having to delve into the world of thread libraries > for those cases where there is not an equivalent pthread_cancel does not > sound fun. =) You could safely assume that there is an equivalent to pthread_cancel; not having thread cancellation on non-pthreads-non-win32 systems seems acceptable to me (especially since we don't have thread cancellation on any system at the moment). Unfortunately, pthread_cancel is insufficient for Python, as you also want stack-unwinding on the cancelled thread, so that it can decref all objects it holds. If you can make this work, we could specify that exiting the interpreter when some threads are still sleeping inside Python code causes those threads to be cancelled. Regards, Martin From tjreedy at udel.edu Tue Feb 10 13:58:27 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Tue Feb 10 13:58:31 2004 Subject: [Python-Dev] Re: Syntax for "except" References: <402901C1.4040405@erols.com> Message-ID: "Edward C. Jones" wrote in message news:402901C1.4040405@erols.com... > Why was Python syntax designed so > except (TypeError, ValueError): > is OK but > except [TypeError, ValueError]: > is forbidden. Because one way to do it is enough! Func[arg] is also forbidden for same reason. > Should this be changed? >From a user viewpoint, definitely NO. 0 benefit, + learning and memory cost > Is immutability needed here? Turn the question around: is mutability needed here? No, the sequence is not named (usually) and so there is no possibility of mutation and no use for it if it is. Terry J. Reedy From tjreedy at udel.edu Tue Feb 10 14:14:01 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Tue Feb 10 14:14:10 2004 Subject: [Python-Dev] Re: Optimization of the Year References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> Message-ID: "Guido van Rossum" wrote in message news:200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net... > Disagree. The public API documentation has never permitted modifying > ob_item and ob_size directly. It would be breaking the abstraction. That answers my previous question. > If in 3rd party code, that code is simply wrong. > > If indeed such 3rd party code exists, and we expect we can't get it > all fixed before 2.4 is released, the tracked_item hack can be used as > a temporary measure to hunt down all those 3rd party extensions that > break the abstraction. I propose to issue a warning when it is > discovered that ob_item != tracked_item. Then in 2.5 we can remove > the tracked_item feature. Why not publish warning/announcement now (clp, clpa, and site), put warning code in alpha/beta and possibly c1, and remove in c2 and 2.4 final? You regularly make changes at or visible from the Python level with less accomodation than this. (Examples: byte code changes; deletion of methods() and members() methods in favor of uniform dir() in 2.2). Terry J. Reedy From martin at v.loewis.de Tue Feb 10 14:10:29 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Feb 10 14:14:51 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <2msmhj9oj4.fsf@starship.python.net> References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> <40254968.4050205@v.loewis.de> <2mllnccv9h.fsf@starship.python.net> <402807CD.1060200@v.loewis.de> <2msmhj9oj4.fsf@starship.python.net> Message-ID: <40292CA5.3060701@v.loewis.de> Michael Hudson wrote: > Yes, for what that's worth. I'm not aware of any code specifically in > aid of libc5, though... No. Instead, we would actively have to add some... >>By PEP 11 rules, this would become "unsupported" in Python 2.4, with >>the possibility of revival if a volunteer steps forward. > > > ... so basically this amounts to saying "if you have a problem with > python on a libc5 system you'd better have a patch too". and we would change configure to detect libc5 systems, and refuse to build on such systems (with an exit statement in configure). Users still wanting to build on libc5 could hack around this and remove the exit statement, or they would volunteer to take over Python-on-libc5 maintainership. Regards, Martin From flaw at rhid.com Tue Feb 10 14:23:20 2004 From: flaw at rhid.com (James William Pye) Date: Tue Feb 10 14:23:25 2004 Subject: [Python-Dev] Dynamically overloadable malloc/realloc/free Message-ID: <20040210192320.GD476@void.ph.cox.net> Greetings, I am writing a Python procedural language implementation for PostgreSQL(embedding)(Use CVS -tOPYMEM; http://gborg.postgresql.org/project/postgrespy/), and I have need for free'ing all memory allocated by Python when a Postgres ERROR occurs as there is a potential of longjmp'ing out of the interpreter, which would leak memory all over the floor. Actually, most places within the plpy library have memory allocated by Python has a chance of leaking.(Yes, there is a way to work around this, but it is ugly, IMO. See the small paragraph on how this is normally handled by PLs at the end of this letter) Ideally, Python would allocate memory within a Postgres MemoryContext, so that when an ERROR occurs Python's memory can be freed up in the same way as other Postgres allocations. This can be achieved by making Python's low-level object [rd]e-allocators use three mutable, global function pointers. In my test implementation I added these FPs to Objects/obmalloc.c void *(*PyMemInternal_Reallocate)(void *, size_t) = realloc; void *(*PyMemInternal_Allocate)(size_t) = malloc; void (*PyMemInternal_Free)(void *) = free; And, of course, then replace direct calls to malloc, realloc, and free in obmalloc.c and pymem.h with their global fp counterparts, and extern the declarations in pymem.h as well(not necessary, but seemed appropriate). Overloading the base type's tp_alloc and tp_free does not seem to be a complete option as many builtin types specify tp_alloc and tp_free as good ol' PyObject_Malloc/Del/etc(at least stringobject.c, IIRC), or with GC_* functions, and that does not cover direct calls to PyObject_*alloc|free anyways. I hear that there are some linker hacks that may be able to emulate this, but portability is very desirable as there are some PostgreSQL developers working on native Windows support. The main problem that I can see with this request is that my use may be a special case, which few embedders would ever need to use. Another possible solution is a function in obmalloc.c that iterates through the arenas and frees them up, this seems like it would be more likely to be accepted, but the former solution is more desirable for my usage. Kaboom Of course, just freeing up all the memory--either resetting the memory context that Python memory is allocated under or free'ing up the arenas--leaves the Python library in an unusable state, even if done after Py_Finalize(yes, I've tested this, dangling globals and states(especially in obmalloc.c, IIUC)). To do this without restarting the process or re-dlinking(My chosen solution for my application), it would require Py_Finalize to completely reset libpython; that is, doing complete finalization(resetting libpython to its pre-Py_Initialize state). This seems a rather large request, and probably beyond anything that anyone is willing to do for the rarely used result(?)(I'm not too eager to jump on it, if it's even acceptable, but if nobody has a problem with me tackling it, I may look into it). plpy I plan to add support for reloading dll's in Postgres to make this work properly for my app(closing and opening the lib should reset the globals, no? I haven't tested this yet, but I'm fairly confident that it is at least a reasonable assumption.). I think reload on ERROR would be a useful feature for lib authors, so I plan submit a proposal to pgsql-hackers soon, depending on what I am able to work out here.. Jumping Normally, PL's use *sigjmp fun to clean up this kind of memory, but it is a serious pain for me in plpy. Trapping these potential jumps must be done within every function with Python memory allocations that makes a call that may ERROR out(There are lots of them, especially in my Postgres interface module(the "if" CVS repository). You can get my patch against 2.3.3 that implements those global FPs at rhid.com/pymfp.patch, it is pretty trivial. Doesn't touch the thread*.c or strdup.c as I didn't think it really applied to it so much, but perhaps they should be updated as well. Comments, Criticisms, Flames? Regards, James William Pye -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 479 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040210/8232dea5/attachment.bin From jeremy at alum.mit.edu Tue Feb 10 14:25:52 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue Feb 10 14:26:31 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: <402928C8.5050207@v.loewis.de> References: <4027E398.6050704@ocf.berkeley.edu> <4028090D.5050003@v.loewis.de> <402824B1.8040305@ocf.berkeley.edu> <402928C8.5050207@v.loewis.de> Message-ID: <1076441151.19776.17.camel@localhost.localdomain> On Tue, 2004-02-10 at 13:54, "Martin v. L?wis" wrote: > Brett wrote: > > So you are classifying this as a bug then. That was part of my question > > since it only happens if you don't handle your threads properly (or at > > least I think properly; I would assume it is common practice to close > > down your threads). > > Certainly: in a properly working program, one would terminate all > threads first before trying to exit the program. Failure to do so > should cause an exception (or otherwise have an "obvious" meaning). > However, that exception should not be a "double-fault"; instead, > it should give a clear indication what the problem is. What about daemon threads? They are running by intention when the program exits, otherwise they wouldn't be daemon threads. Jeremy From guido at python.org Tue Feb 10 14:28:07 2004 From: guido at python.org (Guido van Rossum) Date: Tue Feb 10 14:28:14 2004 Subject: [Python-Dev] Re: Optimization of the Year In-Reply-To: Your message of "Tue, 10 Feb 2004 14:14:01 EST." References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> Message-ID: <200402101928.i1AJS8p31916@c-24-5-183-134.client.comcast.net> > > If in 3rd party code, that code is simply wrong. > > > > If indeed such 3rd party code exists, and we expect we can't get it > > all fixed before 2.4 is released, the tracked_item hack can be used as > > a temporary measure to hunt down all those 3rd party extensions that > > break the abstraction. I propose to issue a warning when it is > > discovered that ob_item != tracked_item. Then in 2.5 we can remove > > the tracked_item feature. > > Why not publish warning/announcement now (clp, clpa, and site), put warning > code in alpha/beta and possibly c1, and remove in c2 and 2.4 final? You > regularly make changes at or visible from the Python level with less > accomodation than this. (Examples: byte code changes; deletion of > methods() and members() methods in favor of uniform dir() in 2.2). Let's hear from Raymond about the use case. --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at barrys-emacs.org Mon Feb 9 19:30:31 2004 From: barry at barrys-emacs.org (Barry Scott) Date: Tue Feb 10 14:52:20 2004 Subject: Fwd: Re: [Python-Dev] *grumble* wchar.h Message-ID: <6.0.1.1.2.20040210002933.033f5ca8@torment.chelsea.private> Opss forgot to mail the list. > Barry> This site http://www.comeaucomputing.com/iso/ has a link to where > Barry> you can but ISO C99 for $20. Also you should be able to buy from > Barry> www.iso.org or your countries standards body. > >Barry, > >Thanks for the link. I wasn't aware any C standards were available in >electronic form. That makes it a lot more palatable than the cost of the >printed docs. > >Skip From martin at v.loewis.de Tue Feb 10 15:24:58 2004 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Feb 10 16:27:08 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: <1076441151.19776.17.camel@localhost.localdomain> References: <4027E398.6050704@ocf.berkeley.edu> <4028090D.5050003@v.loewis.de> <402824B1.8040305@ocf.berkeley.edu> <402928C8.5050207@v.loewis.de> <1076441151.19776.17.camel@localhost.localdomain> Message-ID: <40293E1A.7000403@v.loewis.de> Jeremy Hylton wrote: > What about daemon threads? They are running by intention when the > program exits, otherwise they wouldn't be daemon threads. Right. Shutting down a process using daemon threads should not cause an exception. So this is clearly a bug, or it would be one if it was reproducable in an example involving daemon threads. Regards, Martin From tim.one at comcast.net Tue Feb 10 16:54:12 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 10 16:54:25 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptionsat interpreter teardown In-Reply-To: <40293E1A.7000403@v.loewis.de> Message-ID: [Martin v. L?wis] > Right. Shutting down a process using daemon threads should not cause > an exception. So this is clearly a bug, or it would be one if it was > reproducable in an example involving daemon threads. The Zope3 test suite left a bunch of daemon threads around, and had lots of shutdown problems in the original 2.3. I even provoked a shutdown segfault once by putting print stmts in the threads, trying to see what was going wrong in them. This is one reason I changed 2.3.3 to stop calling gc a second time in Py_Finalize. Other reasons are explained in a comment block there, and an earlier comment explains how you can still provoke shutdown exceptions easily via a different route, up to & including "Fatal Python error". A fundamental problem is that if just about anything gets reclaimed during finalization, __del__ methods or weakref callbacks can get invoked, and as soon as we're back running Python code, it's open season on the GIL, so any number of threads can try to do anything. Since the interpreter is in the middle of being torn down, bad things can happen. Armin Rigo posted a patch (on SF) moving toward using gc to tear things down in a kinder order. I never had time to look into that, but it sounded like a promising approach. From arigo at tunes.org Tue Feb 10 18:14:56 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Feb 10 18:17:41 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <40292CA5.3060701@v.loewis.de> References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> <40254968.4050205@v.loewis.de> <2mllnccv9h.fsf@starship.python.net> <402807CD.1060200@v.loewis.de> <2msmhj9oj4.fsf@starship.python.net> <40292CA5.3060701@v.loewis.de> Message-ID: <20040210231456.GA7177@vicky.ecs.soton.ac.uk> Hello Martin, On Tue, Feb 10, 2004 at 08:10:29PM +0100, "Martin v. L?wis" wrote: > and we would change configure to detect libc5 systems, and refuse > to build on such systems (with an exit statement in configure). > Users still wanting to build on libc5 could hack around this and > remove the exit statement, or they would volunteer to take over > Python-on-libc5 maintainership. This looks like an artificial additional burden for someone who is really trying to install a couple of things on an old machine (like I did recently) and who only cares about Python for setting up a couple of simple things. I can't think of anything in the source that would prevent Python from compiling and running smoothly on almost any version of libc -- maybe there is, but then issue a warning if you like but just let me compile the damn thing so that I can proceed to whatever else I was trying to compile Python for in the first place :-) (We both know pretty well how to compile Python, but replace "Python" with some "libstuff" you don't know and care about but which a program you want keep asking for, and remember all these long hours of fixing small details to get the damn dependencies compiling.) Armin From arigo at tunes.org Tue Feb 10 18:51:57 2004 From: arigo at tunes.org (Armin Rigo) Date: Tue Feb 10 18:54:51 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> Message-ID: <20040210235157.GA14890@vicky.ecs.soton.ac.uk> Hello Guido, On Tue, Feb 10, 2004 at 09:56:44AM -0800, Guido van Rossum wrote: > Good idea. We'd also have to break binary compatibility (of the List > API) forcibly otherwise we could be fooled by 3rd party extensions > compiled for older Python versions -- usually we're careful to keep > binary compatibility. Unfortunately, the existence of the PyList_GET_XXX macros means that perfectly legal extension modules would break if you do something like changing the order of the fields. Is it more important to try and keep binary compatibility for these 99%, or to force a reproducible segfault in the remaining 1%? Armin From greg at cosc.canterbury.ac.nz Tue Feb 10 19:19:32 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Tue Feb 10 19:19:50 2004 Subject: [Python-Dev] Syntax for "except" In-Reply-To: <402901C1.4040405@erols.com> Message-ID: <200402110019.i1B0JWo07361@oma.cosc.canterbury.ac.nz> "Edward C. Jones" : > Why was Python syntax designed so > except (TypeError, ValueError): > is OK but > except [TypeError, ValueError]: > is forbidden. My guess would be that exception matching needs to be fast, so special-casing for tuples is desirable, and allowing more general sequences as well would add so little utility that it's not considered worth the extra code. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From janssen at parc.com Tue Feb 10 19:44:41 2004 From: janssen at parc.com (Bill Janssen) Date: Tue Feb 10 19:45:03 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: Your message of "Mon, 09 Feb 2004 16:24:17 PST." <402824B1.8040305@ocf.berkeley.edu> Message-ID: <04Feb10.164449pst."58611"@synergy1.parc.xerox.com> > So you are classifying this as a bug then. That was part of my question > since it only happens if you don't handle your threads properly (or at > least I think properly; I would assume it is common practice to close > down your threads). Really? I would assume it's common practice to let the garbage collector shut down your threads. After all, people don't handle errors either. Most code doesn't even check for them. Bill From tim.one at comcast.net Tue Feb 10 20:34:41 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 10 20:34:45 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> Message-ID: [Anders J. Munch] >> There is a simpler way to make sure all clients that illicitly use >> ob_item are updated appropriately: Break the build by renaming. If >> ob_item is renamed to ob_items or ob_item2 or whatever, any code that >> uses the old name will cease to compile. [Guido] > Good idea. We'd also have to break binary compatibility (of the List > API) forcibly otherwise we could be fooled by 3rd party extensions > compiled for older Python versions -- usually we're careful to keep > binary compatibility. I think it's a bad idea, at least until Raymond produces evidence that overwriting ob_item in extensions isn't "just a myth". Not even Zope's C code does that (which is my poster child for pushing the line on Python's internals). Google doesn't turn up few enough hits on ob_item to study exhaustively, and there's no hint via doing so that any code indexed by Google has mucked with ob_item either. From tim.one at comcast.net Tue Feb 10 20:37:00 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 10 20:37:04 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <20040210235157.GA14890@vicky.ecs.soton.ac.uk> Message-ID: [Armin Rigo] > Unfortunately, the existence of the PyList_GET_XXX macros means > that perfectly legal extension modules would break if you do something > like changing the order of the fields. The new field can be added at the end of the current list header struct. > Is it more important to try and keep binary compatibility for these 99%, > or to force a reproducible segfault in the remaining 1%? That question didn't make sense . From tim.one at comcast.net Tue Feb 10 22:31:26 2004 From: tim.one at comcast.net (Tim Peters) Date: Tue Feb 10 22:31:35 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <20040210163904.GA29632@panix.com> Message-ID: [Aahz] > So in other words, we might as well go ahead and trade space for > speed? And simplicity, yes. The current scheme is delicate and obscure as well as slow and compact. Worse, it's grown hairier over the years as more platform-specific cases of extremely bad performance were reported, and an obscure scheme that grows more obscure over time is (IMO) a loser. > If someone really does have a million small lists, they're > probably mutating some of them constantly (or they'd use tuples). I don't know about that one -- "head arguments" in either direction are too easy to make up here. Since my employer mostly runs Linux, and adding a 4- byte list header field under gcc won't make any difference to total allocation, nobody's gonna hit me if it happens. Hell, I'd be happy to rip out the irritating union-with-long-double gc header trick that forces worse results under MSVC too (same thing: an increase in beauty without personal pain). From edcjones at erols.com Tue Feb 10 20:24:21 2004 From: edcjones at erols.com (Edward C. Jones) Date: Tue Feb 10 22:57:16 2004 Subject: [Python-Dev] Re: Syntax for "except" Message-ID: <40298445.2070602@erols.com> Here is a weak proposal for a generalized type of exception. Since I know nothing about either language design or implementation, my hope is to trigger some thinking by the experts. I would like to propose a class ExceptionList which is an exception containing a list, It can be used in two ways. Suppose MyError = ExceptionList( (ValueError, TypeError) ) "except MyError:" handles MyError, ValueError, and TypeError or any exception derived from them. This generalizes the current "except (ValueError, TypeError):". "raise MyError" raises an exception that can be handled by any except clause that handles MyError, ValueError, or TypeError. Here is an example for this: Suppose I invent the exception LengthError which is raised when a sequence (including sys.argv) is the wrong length. It is also raised if a function is passed the wrong number of arguments. I have a lot of code which raises and/or handles ValueError or TypeError in these situations. MyError can handle exceptions raised by the legacy code or vice versa. Perhaps MyError could be a class with base classes ValueError and TypeError. From greg at electricrain.com Wed Feb 11 00:43:21 2004 From: greg at electricrain.com (Gregory P. Smith) Date: Wed Feb 11 00:43:26 2004 Subject: [Python-Dev] Re: Are we collecting benchmark results across machines In-Reply-To: <3FF40C59.31756.38CC66EA@localhost> References: <200401010014.i010EGt06465@c-24-5-183-134.client.comcast.net> <3FF40C59.31756.38CC66EA@localhost> Message-ID: <20040211054321.GD6231@zot.electricrain.com> On Thu, Jan 01, 2004 at 12:59:08PM -0500, Brad Clements wrote: > > Python 2.3.2 (#49) pystone test on my Win2k SP4 machine. > > Note this is a 1.6 ghz Pentium M (Centrino) > > Pystone(1.1) time for 50000 passes = 1.3762 > This machine benchmarks at 36331.9 pystones/second > > Faster than Anthony's 3Ghz P4. > > I believe this processor has a very large on-board cache.. I'm not sure how large. It's a > Dell Lattitude D800. > > My point being, I think pystone performance is greatly effected by CPU cache, more so > than CPU clock speed. yes, the pentium M is intel's fastest CPU though they don't like to admit it lest people stop buying P4s. current models have a 1mb L2 cache and an well optimized p3-derived core. cpu cache size will have an effect on any benchmark. if one python implementation outperforms another because of a better cache foot print thats good thing. cache architecture will only become more important as time goes on. From python at rcn.com Wed Feb 11 01:31:50 2004 From: python at rcn.com (Raymond Hettinger) Date: Wed Feb 11 01:33:24 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <20040210170650.GA35930@i18n.org> Message-ID: <001601c3f068$bb76dca0$fd2bc797@oemcomputer> [Hye-Shik] > I made an updated patch very slightly modified from Raymond's. > Dropped use of track_* member variables and applied simpler extra > size calculation suggested by Tim. > > http://people.freebsd.org/~perky/list-r2.diff.txt We have a winner. It has most of the performance benefits with no space trade-off. It is not as automatic as the previous patch. The patch had to add code to list_sort() to trap the temporary alteration of ob_item. If similar tricks get used in extensions, they will need to add "list->allocated = -1;" to invalidate the previous allocation size. If there are no dissents, I will check this in. Raymond ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From martin at v.loewis.de Wed Feb 11 01:59:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Feb 11 02:00:01 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <20040210231456.GA7177@vicky.ecs.soton.ac.uk> References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> <40254968.4050205@v.loewis.de> <2mllnccv9h.fsf@starship.python.net> <402807CD.1060200@v.loewis.de> <2msmhj9oj4.fsf@starship.python.net> <40292CA5.3060701@v.loewis.de> <20040210231456.GA7177@vicky.ecs.soton.ac.uk> Message-ID: <4029D2E0.4040702@v.loewis.de> Armin Rigo wrote: > This looks like an artificial additional burden for someone who is really > trying to install a couple of things on an old machine (like I did recently) > and who only cares about Python for setting up a couple of simple things. I > can't think of anything in the source that would prevent Python from compiling > and running smoothly on almost any version of libc -- maybe there is, but then > issue a warning if you like but just let me compile the damn thing so that I > can proceed to whatever else I was trying to compile Python for in the first > place :-) So are you volunteering to deal with the libc5-related bug reports? Currently, it appears we have 881812, 880996; 463576 was closed as "won't fix". As you can see from these reports, Python won't even build on these systems. Regards, Martin From martin at v.loewis.de Wed Feb 11 02:02:04 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Feb 11 02:02:31 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <20040210235157.GA14890@vicky.ecs.soton.ac.uk> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> <20040210235157.GA14890@vicky.ecs.soton.ac.uk> Message-ID: <4029D36C.1090100@v.loewis.de> Armin Rigo wrote: > Unfortunately, the existence of the PyList_GET_XXX macros means that perfectly > legal extension modules would break if you do something like changing the > order of the fields. Is it more important to try and keep binary > compatibility for these 99%, or to force a reproducible segfault in the > remaining 1%? I think Python 2.4 might be binary-incompatible with 2.3, anyway. Regards, Martin From arigo at tunes.org Wed Feb 11 06:24:07 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed Feb 11 06:26:51 2004 Subject: [Python-Dev] zapped several unsupported bits - now would be a good time to test In-Reply-To: <4029D2E0.4040702@v.loewis.de> References: <16420.61367.299906.722031@montanaro.dyndns.org> <40252484.4030505@v.loewis.de> <16421.12895.868946.402819@montanaro.dyndns.org> <40254968.4050205@v.loewis.de> <2mllnccv9h.fsf@starship.python.net> <402807CD.1060200@v.loewis.de> <2msmhj9oj4.fsf@starship.python.net> <40292CA5.3060701@v.loewis.de> <20040210231456.GA7177@vicky.ecs.soton.ac.uk> <4029D2E0.4040702@v.loewis.de> Message-ID: <20040211112407.GA22496@vicky.ecs.soton.ac.uk> Hello Martin, On Wed, Feb 11, 2004 at 07:59:44AM +0100, "Martin v. L?wis" wrote: > So are you volunteering to deal with the libc5-related bug reports? > Currently, it appears we have 881812, 880996; 463576 was closed as > "won't fix". As you can see from these reports, Python won't even build > on these systems. Sorry, I was not aware of these. If it doesn't build then I agree that it should either complain at configure-time, or be fixed... Armin From arigo at tunes.org Wed Feb 11 06:30:00 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed Feb 11 06:32:41 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <4029D36C.1090100@v.loewis.de> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> <20040210235157.GA14890@vicky.ecs.soton.ac.uk> <4029D36C.1090100@v.loewis.de> Message-ID: <20040211113000.GC22496@vicky.ecs.soton.ac.uk> Hello Martin, On Wed, Feb 11, 2004 at 08:02:04AM +0100, "Martin v. L?wis" wrote: > I think Python 2.4 might be binary-incompatible with 2.3, anyway. My (limited) experience with it is that although 2.n is generally incompatible with 2.n-1, for some relatively large proportion of extensions it doesn't matter because they just don't use the parts of the API that changed. If you make the list API binary-incompatible then obviously the proportion will shrink dramatically. A bientot, Armin. From gerrit at nl.linux.org Wed Feb 11 07:14:15 2004 From: gerrit at nl.linux.org (Gerrit) Date: Wed Feb 11 07:14:43 2004 Subject: [Python-Dev] Re: Syntax for "except" In-Reply-To: <40298445.2070602@erols.com> References: <40298445.2070602@erols.com> Message-ID: <20040211121415.GA4121@nl.linux.org> Edward C. Jones wrote: > Here is a weak proposal for a generalized type of exception. Since I=20 > know nothing about either language design or implementation, my hope is= =20 > to trigger some thinking by the experts. I don't know anything about it either. > MyError =3D ExceptionList( (ValueError, TypeError) ) > "raise MyError" > raises an exception that can be handled by any except clause that=20 > handles MyError, ValueError, or TypeError. > Perhaps MyError could be a class with base classes ValueError and TypeE= rror. So where's the need? You can already do: class MyError(ValueError, TypeError): ... etc.? Gerrit. --=20 Weather: 6=C2=B0C Hail Wednesday 11 February 2004 11:55 Twenthe Netherla= nds --=20 PrePEP: Builtin path type http://people.nl.linux.org/~gerrit/creaties/path/pep-xxxx.html Asperger's Syndrome - a personal approach: http://people.nl.linux.org/~gerrit/english/ From mcherm at mcherm.com Wed Feb 11 07:19:45 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Feb 11 07:19:49 2004 Subject: [Python-Dev] Re: Syntax for "except" Message-ID: <1076501985.402a1de1846a1@mcherm.com> Edward Jones writes: > I would like to propose a class ExceptionList which is an exception > containing a list, It can be used in two ways. Suppose > > MyError = ExceptionList( (ValueError, TypeError) ) > > "except MyError:" > handles MyError, ValueError, and TypeError or any exception derived from > them. This generalizes the current "except (ValueError, TypeError):". I don't believe that this *generalizes* using a tuple in an except clause, I believe it *exactly replicates* that functionality. In other words, I don't see how MyError = ExceptionList( (ValueError, TypeError) ) try: someCode() except MyError: handleIt() is any better than this (which works today): MyError = (ValueError, TypeError) try: someCode() except MyError: handleIt() So I'd be a strong -1 on providing a new way of doing something which is already easy to do (hey... it's LESS typing with the current syntax). Edward Jones continues: > "raise MyError" > raises an exception that can be handled by any except clause that > handles MyError, ValueError, or TypeError. Well, that's easy to do in current Python also: >>> class CompoundError(TypeError, ValueError): ... pass ... >>> try: ... raise CompoundError ... except ValueError: ... print "it's a value error" ... it's a value error >>> try: ... raise CompoundError ... except TypeError: ... print "it's a type error" ... it's a type error So again, there's no need for your proposal. But even if these things WEREN'T already built into Python, I'd STILL be opposed to adding new syntax to allow them. Catching more than one exception in the same except clause (the first use) is fairly handy, but throwing an exception which masquerades as several different exception types is rather peculiar... I can't think of a realistic way in which I would want to use this. -- Michael Chermside From oren-py-l at hishome.net Wed Feb 11 04:01:25 2004 From: oren-py-l at hishome.net (Oren Tirosh) Date: Wed Feb 11 07:40:12 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <20040210170650.GA35930@i18n.org> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <20040210170650.GA35930@i18n.org> Message-ID: <20040211090125.GA93683@hishome.net> On Wed, Feb 11, 2004 at 02:06:50AM +0900, Hye-Shik Chang wrote: > On Tue, Feb 10, 2004 at 11:04:02AM -0500, Tim Peters wrote: > > [Raymond] > > > Hey, hey! After looking at this on and off for months, I finally > > > found a safe way to dramatically improve the speed of building lists: > > > > > > http://users.rcn.com/python/download/list.diff > > > > Cool! > > > > I made an updated patch very slightly modified from Raymond's. > Dropped use of track_* member variables and applied simpler extra > size calculation suggested by Tim. > > http://people.freebsd.org/~perky/list-r2.diff.txt It would be nice if this could be combined with the pop(0) speedup for the idiomatic use of lists as queues. How about this: If op->allocated is positive it indicates the amount of memory allocated to avoid unnecessary realloc() calls. If negative it indicates how far the list's ob_item pointer has been bumped by popping (i.e. the actual malloced block is ob_item+allocated). When it's negative any appends to the list will have to trust the performance of realloc, but this seems a reasonable tradeoff because it's saving O(N) behavior. The common case (append to a list whose pointer has not been bumped by popping) should not be slowed down at all - the test for negativity is only in the branch taken when a resize is required. Yes, it's a little convoluted, but it the impact of this change seems to be restricted in its scope: assignments to ob_item need to make sure that allocated is also updated (just like your patch) and calls to PyMem need the real allocated pointer: ob_item + (allocated<0 ? allocated : 0) Oren From barry at python.org Wed Feb 11 07:53:44 2004 From: barry at python.org (Barry Warsaw) Date: Wed Feb 11 07:53:53 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: <04Feb10.164449pst."58611"@synergy1.parc.xerox.com> References: <04Feb10.164449pst."58611"@synergy1.parc.xerox.com> Message-ID: <1076504023.922.752.camel@anthem> On Tue, 2004-02-10 at 19:44, Bill Janssen wrote: > > So you are classifying this as a bug then. That was part of my question > > since it only happens if you don't handle your threads properly (or at > > least I think properly; I would assume it is common practice to close > > down your threads). > > Really? I would assume it's common practice to let the garbage > collector shut down your threads. After all, people don't handle > errors either. Most code doesn't even check for them. I don't trust the garbage collector to have any sane shutdown semantics, especially regarding threads. In my own code, I always make the best effort to shut threads (daemon or otherwise) down explicitly. -Barry From guido at python.org Wed Feb 11 10:24:08 2004 From: guido at python.org (Guido van Rossum) Date: Wed Feb 11 10:24:13 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Your message of "Wed, 11 Feb 2004 01:31:50 EST." <001601c3f068$bb76dca0$fd2bc797@oemcomputer> References: <001601c3f068$bb76dca0$fd2bc797@oemcomputer> Message-ID: <200402111524.i1BFO8S01192@c-24-5-183-134.client.comcast.net> > > http://people.freebsd.org/~perky/list-r2.diff.txt > > We have a winner. It has most of the performance benefits with no > space trade-off. (Except on Windows, if I understand Tim's comments.) > It is not as automatic as the previous patch. > The patch had to add code to list_sort() to trap the temporary > alteration of ob_item. If similar tricks get used in extensions, > they will need to add "list->allocated = -1;" to invalidate the > previous allocation size. > > If there are no dissents, I will check this in. I disagree with making the hack of setting list->allocated = -1 an official part of the list object API. The fact that list.sort() does this doesn't mean that other extensions should do this! Since you have ignored repeated requests for real-life examples of code that messes with ob_item outside the core, let's keep the 'allocated' field an internal detail of the list implementation. Then I can ignore the other posts about binary (in)compatibility too. :-) (Otherwise, you would have to write docs before checking it in.) --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Feb 11 10:26:29 2004 From: guido at python.org (Guido van Rossum) Date: Wed Feb 11 10:26:34 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: Your message of "Wed, 11 Feb 2004 07:53:44 EST." <1076504023.922.752.camel@anthem> References: <04Feb10.164449pst."58611"@synergy1.parc.xerox.com> <1076504023.922.752.camel@anthem> Message-ID: <200402111526.i1BFQT601232@c-24-5-183-134.client.comcast.net> > I don't trust the garbage collector to have any sane shutdown semantics, > especially regarding threads. In my own code, I always make the best > effort to shut threads (daemon or otherwise) down explicitly. Shutting down daemon threads is an oxymoron. --Guido van Rossum (home page: http://www.python.org/~guido/) From arigo at tunes.org Wed Feb 11 11:27:21 2004 From: arigo at tunes.org (Armin Rigo) Date: Wed Feb 11 11:30:21 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <20040210170650.GA35930@i18n.org> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <20040210170650.GA35930@i18n.org> Message-ID: <20040211162721.GA32421@vicky.ecs.soton.ac.uk> Hello, (Some of my messages seem to get lost and never make it through to the mailing list, don't understand why...) There are a few details and minor bugs (like missing memory checks) that I don't want to discuss in detail on python-dev. Here are the fixes, open to further eyeballing: http://arigo.tunes.org/list-r3.diff But there is something else that the patch does wrong: it never reclaims memory from the list, unlike the NRESIZE trick which used to realloc with a smaller size occasionally. This is very probably bad. We need to discuss heuristics for that as well. I guess the following could do: shrinking the list only when the ratio ob_size/allocated is below 3/4. Actually the over-allocation heuristic could also be given a few more thoughts; it is fine for the list.append() case, but we should check what its results are for other common patterns. For example one could argue that a concatenation "a+b" of two lists of approximately the same length should not necessarily over-allocate 1/16 of the resulting size. The following modified rule could take care of that: list_resize would allocate max(17/16 of current size, requested size) instead of 17/16 of the requested size, as it does now. Also, the list_sort trickery with empty_ob_item, which I know I am partially to blame for, seems to leak memory under certain conditions (but that's not related to the present patch so I'll postpone this investigation). Armin From python at rcn.com Wed Feb 11 11:35:12 2004 From: python at rcn.com (Raymond Hettinger) Date: Wed Feb 11 11:36:39 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <200402111524.i1BFO8S01192@c-24-5-183-134.client.comcast.net> Message-ID: <002301c3f0bd$069ddcc0$36bb2c81@oemcomputer> [GvR] > I disagree with making the hack of setting list->allocated = -1 an > official part of the list object API. Okay. > Since you have ignored repeated requests for real-life examples of > code that messes with ob_item outside the core, Haven't found any. Initially, I thought there were because the guard was getting tripped. Now, it looks like Hye-Shik's NULL check is sufficient. > let's keep the > 'allocated' field an internal detail of the list implementation. Yes. > Then > I can ignore the other posts about binary (in)compatibility too. :-) Right. [Armin] > BUT there is something that the patch does wrong: it never reclaims memory > from the list, unlike the NRESIZE trick which used to realloc with a > smaller > size occasionally. Yes. I had already caught that and sent in a change to Hye-Shik to make sure downward resizing can happen. Also, I'll incorporate the minor fixups you sent in. Any further adventures should probably be done as a separate patch (don't change too many things at once). Raymond From martin at v.loewis.de Wed Feb 11 11:56:52 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Feb 11 11:57:12 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <20040211113000.GC22496@vicky.ecs.soton.ac.uk> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> <20040210235157.GA14890@vicky.ecs.soton.ac.uk> <4029D36C.1090100@v.loewis.de> <20040211113000.GC22496@vicky.ecs.soton.ac.uk> Message-ID: <402A5ED4.2020406@v.loewis.de> Armin Rigo wrote: > If you make the list API binary-incompatible then obviously the proportion > will shrink dramatically. I still plan to introduce size_t/ssize_t as the data type for element counts in sequences. This will cause severe ABI incompatibilities on 64-bit systems (while causing no change on 32-bit systems). Regards, Martin From guido at python.org Wed Feb 11 12:12:42 2004 From: guido at python.org (Guido van Rossum) Date: Wed Feb 11 12:12:48 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Your message of "Wed, 11 Feb 2004 17:56:52 +0100." <402A5ED4.2020406@v.loewis.de> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> <20040210235157.GA14890@vicky.ecs.soton.ac.uk> <4029D36C.1090100@v.loewis.de> <20040211113000.GC22496@vicky.ecs.soton.ac.uk> <402A5ED4.2020406@v.loewis.de> Message-ID: <200402111712.i1BHCgM01454@c-24-5-183-134.client.comcast.net> > I still plan to introduce size_t/ssize_t as the data type for element > counts in sequences. This will cause severe ABI incompatibilities on > 64-bit systems (while causing no change on 32-bit systems). I'm all for this. I imagine there are many subtleties though -- I know I consciously chose 'int' for this purpose in Python 0, and the assumption that it fits in an int is everywhere. I would trust a decent compiler to find most cases, but not all. I alsoam not sure there are no incompatibilities on 32-bit systems -- the interpretation of certain bit patterns will be different. Also, occasionally (size_t)(-1) may be used to indicate an error (e.g. PySequence_Size()). --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Wed Feb 11 12:19:15 2004 From: janssen at parc.com (Bill Janssen) Date: Wed Feb 11 12:19:41 2004 Subject: [Python-Dev] bug 754449 and threads raising exceptions at interpreter teardown In-Reply-To: Your message of "Wed, 11 Feb 2004 04:53:44 PST." <1076504023.922.752.camel@anthem> Message-ID: <04Feb11.091919pst."58611"@synergy1.parc.xerox.com> > I don't trust the garbage collector to have any sane shutdown semantics, > especially regarding threads. In my own code, I always make the best > effort to shut threads (daemon or otherwise) down explicitly. Sure, you and me both. But I doubt it's common practice. The garbage collector should make a real effort to do something sensible. Bill From martin at v.loewis.de Wed Feb 11 12:52:16 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Feb 11 12:52:49 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <200402111712.i1BHCgM01454@c-24-5-183-134.client.comcast.net> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> <20040210235157.GA14890@vicky.ecs.soton.ac.uk> <4029D36C.1090100@v.loewis.de> <20040211113000.GC22496@vicky.ecs.soton.ac.uk> <402A5ED4.2020406@v.loewis.de> <200402111712.i1BHCgM01454@c-24-5-183-134.client.comcast.net> Message-ID: <402A6BD0.4070003@v.loewis.de> Guido van Rossum wrote: > I'm all for this. I imagine there are many subtleties though -- I > know I consciously chose 'int' for this purpose in Python 0, and the > assumption that it fits in an int is everywhere. I would trust a > decent compiler to find most cases, but not all. The MS Win64 compiler (Itanium) is pretty good at finding size_t/int conversions, as size_t is wider even than long on the system. I have patches sitting on my disk to change all function signatures, but not the structure layouts. They are (unfortunately) still incomplete, and it is doubtful whether I can complete them before PyCon. > Also, occasionally (size_t)(-1) may be used to indicate an error > (e.g. PySequence_Size()). This is the tricky part that I want to defer in the first round of patches: I believe ob_size should be a signed type, so it should be ssize_t where available. The tricky part is finding an appropriate type on systems where ssize_t is not available. (size_t)(-1) is not a problem per se; this is well-defined on all ISO C compilers. However, some code may expect that the error marker is smaller than any other value, which would not be the case for (size_t)(-1). Regards, Martin From guido at python.org Wed Feb 11 12:58:26 2004 From: guido at python.org (Guido van Rossum) Date: Wed Feb 11 12:58:31 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: Your message of "Wed, 11 Feb 2004 18:52:16 +0100." <402A6BD0.4070003@v.loewis.de> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> <20040210235157.GA14890@vicky.ecs.soton.ac.uk> <4029D36C.1090100@v.loewis.de> <20040211113000.GC22496@vicky.ecs.soton.ac.uk> <402A5ED4.2020406@v.loewis.de> <200402111712.i1BHCgM01454@c-24-5-183-134.client.comcast.net> <402A6BD0.4070003@v.loewis.de> Message-ID: <200402111758.i1BHwQw01542@c-24-5-183-134.client.comcast.net> > Guido van Rossum wrote: > > I'm all for this. I imagine there are many subtleties though -- I > > know I consciously chose 'int' for this purpose in Python 0, and the > > assumption that it fits in an int is everywhere. I would trust a > > decent compiler to find most cases, but not all. [MvL] > The MS Win64 compiler (Itanium) is pretty good at finding size_t/int > conversions, as size_t is wider even than long on the system. I have > patches sitting on my disk to change all function signatures, but > not the structure layouts. They are (unfortunately) still incomplete, > and it is doubtful whether I can complete them before PyCon. Maybe it can be a sprint topic? > > Also, occasionally (size_t)(-1) may be used to indicate an error > > (e.g. PySequence_Size()). > > This is the tricky part that I want to defer in the first round > of patches: I believe ob_size should be a signed type, so it should > be ssize_t where available. The tricky part is finding an appropriate > type on systems where ssize_t is not available. Why not simply use Py_intptr_t then? > (size_t)(-1) is not a problem per se; this is well-defined on all > ISO C compilers. However, some code may expect that the error marker > is smaller than any other value, which would not be the case for > (size_t)(-1). Right, that's what I meant. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Feb 11 13:12:21 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed Feb 11 13:12:56 2004 Subject: [Python-Dev] Optimization of the Year In-Reply-To: <200402111758.i1BHwQw01542@c-24-5-183-134.client.comcast.net> References: <00c401c3efa5$b08d5300$2008a044@oemcomputer> <200402101540.i1AFeSR31177@c-24-5-183-134.client.comcast.net> <003b01c3effa$db25dce0$f901a8c0@hirsevej.dk> <200402101756.i1AHuiH31607@c-24-5-183-134.client.comcast.net> <20040210235157.GA14890@vicky.ecs.soton.ac.uk> <4029D36C.1090100@v.loewis.de> <20040211113000.GC22496@vicky.ecs.soton.ac.uk> <402A5ED4.2020406@v.loewis.de> <200402111712.i1BHCgM01454@c-24-5-183-134.client.comcast.net> <402A6BD0.4070003@v.loewis.de> <200402111758.i1BHwQw01542@c-24-5-183-134.client.comcast.net> Message-ID: <402A7085.60508@v.loewis.de> Guido van Rossum wrote: > They are (unfortunately) still incomplete, >>and it is doubtful whether I can complete them before PyCon. > > > Maybe it can be a sprint topic? Sure! I can release what I have at the time. The risky part is that these are *many* changes, with the danger of introducing subtle bugs, so a review of the changes would be definitely helpful. >>This is the tricky part that I want to defer in the first round >>of patches: I believe ob_size should be a signed type, so it should >>be ssize_t where available. The tricky part is finding an appropriate >>type on systems where ssize_t is not available. > > > Why not simply use Py_intptr_t then? Conceptually, it is a different thing, because sizeof(void*) and sizeof(size_t) may differ on some platform. For practical purposes, they propably won't differ on any platform supported by Python. Regards, Martin From DavidA at ActiveState.com Wed Feb 11 15:35:42 2004 From: DavidA at ActiveState.com (David Ascher) Date: Wed Feb 11 15:39:29 2004 Subject: [Python-Dev] Nokia Message-ID: <402A921E.2030705@ActiveState.com> I'm at ETech 2004, where the CTO of Nokia talked about how they are _currently_ using Python to develop applications that run on their "66-series" platform (which includes the n-gage IIRC). Nokia shipped 100 million units last year. Interesting, no? --david PS: Yes, I'll be asking him to consider becoming a PSF sponsor member. From paul at prescod.net Thu Feb 12 04:37:15 2004 From: paul at prescod.net (Paul Prescod) Date: Thu Feb 12 07:44:43 2004 Subject: [Python-Dev] Re: [PSF-Members] Nokia In-Reply-To: <402A921E.2030705@ActiveState.com> References: <402A921E.2030705@ActiveState.com> Message-ID: <402B494B.2010001@prescod.net> David Ascher wrote: > I'm at ETech 2004, where the CTO of Nokia talked about how they are > _currently_ using Python to develop applications that run on their Can you give an example? Is there any hint that the phones might ship with the Python interpreter installed? > "66-series" platform (which includes the n-gage IIRC). Proably you mean "Series 60". Yes, it does include the N-Gage and the Symbian camera phones. http://www.forum.nokia.com/main/0,6566,010_40,00.html Series 60 phones are not just made by Nokia. Look at the bottom of that page. Series 60 is basically the main competitor to Microsoft's SmartPhone platform. (Series 60 is a layer on top of Symbian) > Nokia shipped 100 million units last year. Not in North America. ;) > Interesting, no? Yes, that's' great news! I knew something was confused when I first heard that they were porting Perl[1] to the phones. I was thinking: "Why Perl? Python already runs on Symbian." But then it was corrected.[2] On the other hand, Symbian uses Perl for their Symbian app SDK build process, so I thought that they were just more comfortable with it.[3] Paul Prescod [1] http://www.theregister.co.uk/content/64/34943.html [2] http://www.theregister.co.uk/content/64/35040.html [3] http://www.newlc.com/article.php3?id_article=128 From niemeyer at conectiva.com Thu Feb 12 09:11:35 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Feb 12 09:18:48 2004 Subject: [Python-Dev] Small test Message-ID: <20040212141134.GA10828@burma.localdomain> Hello folks, can you please test the following piece of code with your compilers and check if they're issuing any warnings? Thanks! #include int main(int argc, char *argv[]) { char c = 1; if (!(c & ~255)) printf("Thanks!\n"); return 0; } -- Gustavo Niemeyer http://niemeyer.net From tim at multitalents.net Thu Feb 12 11:58:58 2004 From: tim at multitalents.net (Tim Rice) Date: Thu Feb 12 11:59:24 2004 Subject: [Python-Dev] Small test In-Reply-To: <20040212141134.GA10828@burma.localdomain> References: <20040212141134.GA10828@burma.localdomain> Message-ID: On Thu, 12 Feb 2004, Gustavo Niemeyer wrote: > Hello folks, > > can you please test the following piece of code with your compilers > and check if they're issuing any warnings? Thanks! > > #include > int main(int argc, char *argv[]) > { > char c = 1; > if (!(c & ~255)) printf("Thanks!\n"); > return 0; > } > OK UnixWare 7.1.3 Optimizing C Compilation System (CCS) 4.1 10/31/02 (OU7.1.3bl11.1 OK UnixWare 2.03 Optimizing C Compilation System (CCS) 3.0 02/14/95 (e10) OK SCO OpenServer 3 Native cc and gcc 2.7.2.3 OK SCO OpenServer 5.0.4 SCO UNIX Development System Release 5.1.0Ha 25Apr97 OK UnixWare 7.1.1 Optimizing C Compilation System (CCS) 3.2 09/28/99 (santamariasbl 5.1.rs1ubl7) OK Caldera OpenLinux 3.1.1 gcc 2.95.2 OK SCO OpenServer 5.0.7 SCO UNIX Development System Release 5.2.0A 22Jul03 gcc 2.95.3 OK Solaris 7 gcc 2.95.2 -- Tim Rice Multitalents tim@multitalents.net From tismer at stackless.com Thu Feb 12 13:17:41 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Feb 12 13:14:28 2004 Subject: [Python-Dev] Sprint on Stackless Python! In-Reply-To: References: <40291EEB.7010201@stackless.com> <402A47CC.1060606@stackless.com> <402A5DD1.1070903@stackless.com> Message-ID: <402BC345.6010500@stackless.com> *********************************************** *** *** *** S t a c k l e s s S p r i n t *** *** *** *********************************************** Dear Stackless Friends and Python developers, we are planning a sprint on developing and learning Stackless Python. It will be located in Berlin, happening probably somewhere in March 2004. Duration is not settled, maybe little more than a weekend, maybe more, this depends on what you like. Contents? Well, there is a lot possible. Current topics which come into mind are * autoscheduling * scheduling object * brain storming! * Zope Wiki and Documentation * making channels really stackless * find the Bajo bug if I still can't? * demo applications * more regression tests * real zope apps? * minimalist Stackless Python with no hardware dependency * assembly-free Stackless with setjmp/longjmp * spreading the internals between developers but this is completely open for discussion. Level? Well, it will be a bit simpler than the pypy-sprints, which I think are very difficult, but Stackless has its built-in difficulties by nature. Usability? This is by far, now and forever, the best possible way to learn about Stackless Python and to become a core developer. But no guarantee possible :-) Fun? Guaranteed. Please contact me. cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From pinard at iro.umontreal.ca Thu Feb 12 14:41:27 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Thu Feb 12 14:42:07 2004 Subject: [Python-Dev] bool does not want to be subclassed? Message-ID: <20040212194127.GA6522@alcyon.progiciels-bpi.ca> Hi, people. I just noticed this: >>> class booleen(bool): ... def __repr__(self): ... if self: ... return "Vrai" ... return "Faux" ... Traceback (most recent call last): File "", line 1, in ? TypeError: type 'bool' is not an acceptable base type >>> and am a bit surprised that `bool' refuses to be sub-classed. Not that I cannot live without it, of course! But it might be useful being able to "cast" a `bool' into something for which I could control the representation, while keeping all other properties of `bool'. Is there a deep reason behind the forbidding? Or was it merely for possibly protecting users against themselves? :-) For me, this is a question, much more than an issue. Yet, if there is no deep reason, maybe the forbidding could be lifted? -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From aahz at pythoncraft.com Thu Feb 12 14:50:50 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Feb 12 14:51:31 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040212194127.GA6522@alcyon.progiciels-bpi.ca> References: <20040212194127.GA6522@alcyon.progiciels-bpi.ca> Message-ID: <20040212195050.GA27126@panix.com> On Thu, Feb 12, 2004, Fran?ois Pinard wrote: > > Hi, people. I just noticed this: > > >>> class booleen(bool): > ... def __repr__(self): > ... if self: > ... return "Vrai" > ... return "Faux" > ... > Traceback (most recent call last): > File "", line 1, in ? > TypeError: type 'bool' is not an acceptable base type > >>> Just to bolster Francois's case, I ran into this precise use case a week ago myself (where I wanted a bool but needed to change the output string). Didn't bother complaining about it, though. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach From bob at redivi.com Thu Feb 12 14:58:42 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 12 14:55:41 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040212194127.GA6522@alcyon.progiciels-bpi.ca> References: <20040212194127.GA6522@alcyon.progiciels-bpi.ca> Message-ID: On Feb 12, 2004, at 2:41 PM, Fran?ois Pinard wrote: > Hi, people. I just noticed this: > > >>>> class booleen(bool): > ... def __repr__(self): > ... if self: > ... return "Vrai" > ... return "Faux" > ... > Traceback (most recent call last): > File "", line 1, in ? > TypeError: type 'bool' is not an acceptable base type >>>> > > > and am a bit surprised that `bool' refuses to be sub-classed. Not that > I cannot live without it, of course! But it might be useful being > able to "cast" a `bool' into something for which I could control the > representation, while keeping all other properties of `bool'. > > Is there a deep reason behind the forbidding? Or was it merely for > possibly protecting users against themselves? :-) For me, this is a > question, much more than an issue. Yet, if there is no deep reason, > maybe the forbidding could be lifted? Probably because bool is supposed to only have two values, True or False.. not True, False, Yes, No, Vrai, Faux. You can subclass int and implement __nonzero__ just like bool does (in fact, bool does little more than that). Is there a good reason for wanting to do this? Do you want to "translate" all of Python to French? This is certainly not the way to go about it! You're going to receive True and False from other functions/keywords whether you like it or not. I would bet that "not Vrai" would be "False", for example. -bob From bob at redivi.com Thu Feb 12 15:02:38 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 12 14:59:32 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040212195050.GA27126@panix.com> References: <20040212194127.GA6522@alcyon.progiciels-bpi.ca> <20040212195050.GA27126@panix.com> Message-ID: <6886E60D-5D96-11D8-B1D7-000A95686CD8@redivi.com> On Feb 12, 2004, at 2:50 PM, Aahz wrote: > On Thu, Feb 12, 2004, Fran?ois Pinard wrote: >> >> Hi, people. I just noticed this: >> >>>>> class booleen(bool): >> ... def __repr__(self): >> ... if self: >> ... return "Vrai" >> ... return "Faux" >> ... >> Traceback (most recent call last): >> File "", line 1, in ? >> TypeError: type 'bool' is not an acceptable base type >>>>> > > Just to bolster Francois's case, I ran into this precise use case a > week > ago myself (where I wanted a bool but needed to change the output > string). Didn't bother complaining about it, though. ;-) Shouldn't you be "adapting" the objects for display with some kind of value transformer.. instead of trying to change the objects themselves in the "model"? You could do this with something like PyProtocols, for example. You really have to do something like this in most cases anyway, because __repr__ gets coerced to str if you use the repr() function. -bob From gerrit at nl.linux.org Thu Feb 12 15:13:18 2004 From: gerrit at nl.linux.org (Gerrit) Date: Thu Feb 12 15:13:35 2004 Subject: [Python-Dev] Small test In-Reply-To: <20040212141134.GA10828@burma.localdomain> References: <20040212141134.GA10828@burma.localdomain> Message-ID: <20040212201318.GA7219@nl.linux.org> Gustavo Niemeyer wrote: > can you please test the following piece of code with your compilers > and check if they're issuing any warnings? Thanks! >=20 > #include > int main(int argc, char *argv[]) > { > char c =3D 1; > if (!(c & ~255)) printf("Thanks!\n"); > return 0; > } No warning at Linux 2.4.22 gcc 3.3.2 glibc 2.3.2. Gerrit. --=20 Weather in Twenthe, Netherlands 12/02 19:25 UTC: 5.0=C2=B0C wind 2.2 m/s SW (57 m above NAP) --=20 Asperger's Syndrome - a personal approach: http://people.nl.linux.org/~gerrit/english/ From mon at abaqus.com Thu Feb 12 15:19:02 2004 From: mon at abaqus.com (Nick Monyatovsky) Date: Thu Feb 12 15:19:10 2004 Subject: [Python-Dev] Small test In-Reply-To: Tim Rice "Re: [Python-Dev] Small test" (Feb 12, 8:58) References: <20040212141134.GA10828@burma.localdomain> Message-ID: <1040212151902.ZM1698@reliant.hks.com> >------------------- Message from Tim Rice -------------------- > On Thu, 12 Feb 2004, Gustavo Niemeyer wrote: > > > Hello folks, > > > > can you please test the following piece of code with your compilers > > and check if they're issuing any warnings? Thanks! > > > > #include > > int main(int argc, char *argv[]) > > { > > char c = 1; > > if (!(c & ~255)) printf("Thanks!\n"); > > return 0; > > } > > > > OK UnixWare 7.1.3 > Optimizing C Compilation System (CCS) 4.1 10/31/02 (OU7.1.3bl11.1 > OK UnixWare 2.03 > Optimizing C Compilation System (CCS) 3.0 02/14/95 (e10) > OK SCO OpenServer 3 > Native cc and gcc 2.7.2.3 > OK SCO OpenServer 5.0.4 > SCO UNIX Development System Release 5.1.0Ha 25Apr97 > OK UnixWare 7.1.1 > Optimizing C Compilation System (CCS) 3.2 09/28/99 (santamariasbl 5.1.rs1ubl7) > OK Caldera OpenLinux 3.1.1 > gcc 2.95.2 > OK SCO OpenServer 5.0.7 > SCO UNIX Development System Release 5.2.0A 22Jul03 > gcc 2.95.3 > OK Solaris 7 > gcc 2.95.2 > > -- > Tim Rice Multitalents > tim@multitalents.net >-------------------------------------------------------- Not a squeak. I tried both 32-bit and 64-bit compilations where applicable. OK SGI IRIX 6.5 (32-bit/64-bit): MIPSpro Compilers: Version 7.3.1.3m OK Linux x86: 32-bit Intel C/C++ Version 8.0 (icpc) OK Linux x86: 32-bit Intel C/C++ Version 7.1 (icpc) OK Linux ia64: 64-bit Intel C/C++ Version 7.1 (ecpc) OK Windows: Microsoft 32-bit Visual C++ 6.0 (Version 12.00.8804) OK Windows: Microsoft 32-bit .Net C++ Compiler (Version 13.10.3077) OK SUN Solaris 8 (32-bit/64-bit): Sun ONE Studio 8 Compiler (CC) OK IBM AIX 5.1 (32-bit/64-bit): IBM VisualAge 5.0 (xlC) OK HP-UX/PA-RISC 11.00 (32/64-bit): aC++/C Compiler A.03.25 (aCC) OK HP-UX/Itanium 11.22 (64-bit): aC++/C Compiler A.05.41 (aCC) OK COMPAQ Tru64 UNIX V5.1 (64-bit): Compaq C/C++ Compiler V6.3-011 (cxx) -- Nick Monyatovsky -- mon@abaqus.com From bac at OCF.Berkeley.EDU Thu Feb 12 15:32:06 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Feb 12 15:32:23 2004 Subject: [Python-Dev] Small test In-Reply-To: <20040212141134.GA10828@burma.localdomain> References: <20040212141134.GA10828@burma.localdomain> Message-ID: <402BE2C6.6060501@ocf.berkeley.edu> Gustavo Niemeyer wrote: > Hello folks, > > can you please test the following piece of code with your compilers > and check if they're issuing any warnings? Thanks! > > #include > int main(int argc, char *argv[]) > { > char c = 1; > if (!(c & ~255)) printf("Thanks!\n"); > return 0; > } > On OS X 10.3.2 (Panther) which uses "gcc version 3.3 20030304 (Apple Computer, Inc. build 1495)" I compile with no warnings and it spits out "Thanks!". -Brett From astrand at lysator.liu.se Thu Feb 12 15:41:06 2004 From: astrand at lysator.liu.se (=?iso-8859-1?Q?Peter_=C5strand?=) Date: Thu Feb 12 15:41:13 2004 Subject: [Python-Dev] symtable module & globals Message-ID: I've found the symtable module, which is quite nice. Unfortunately, I cannot find any documentation, and the module don't work as I expect: If the source looks like this: foo = 1 def fie(): print foo Calling is_global() for the "foo" name returns False (using the fie functions SymbolTable). With a method, is_global() returns true, though: foo = 1 class C: def fie(self): print foo Have I misunderstood something, or is this a bug? -- /Peter ?strand From jeremy at alum.mit.edu Thu Feb 12 15:54:33 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Thu Feb 12 15:55:24 2004 Subject: [Python-Dev] symtable module & globals In-Reply-To: References: Message-ID: <1076619272.30191.75.camel@localhost.localdomain> On Thu, 2004-02-12 at 15:41, Peter ?strand wrote: > I've found the symtable module, which is quite nice. Unfortunately, I > cannot find any documentation, and the module don't work as I expect: You're very generous in complimenting an undocumented module that doesn't work as you expect . I think both things you report are bugs. Can you file a bug report and assign it to me? Jeremy From pinard at iro.umontreal.ca Thu Feb 12 16:01:32 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Thu Feb 12 16:02:14 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: References: <20040212194127.GA6522@alcyon.progiciels-bpi.ca> Message-ID: <20040212210132.GA7482@alcyon.progiciels-bpi.ca> [Bob Ippolito] > On Feb 12, 2004, at 2:41 PM, Fran?ois Pinard wrote: > >[...] and am a bit surprised that `bool' refuses to be sub-classed. > >Is there a deep reason behind the forbidding? Yet, [if not], maybe > >the forbidding could be lifted? > Probably because bool is supposed to only have two values, True or > False.. not True, False, Yes, No, Vrai, Faux. I do not understand the argument. `int' is supposed to only have int values (0, 1, -1, 2, -2, etc.), and we may subclass it nevertheless. Are you quoting "Yes", "No", "Vrai" and "Faux" as other names for True and False, or for other intermediate values between True and False? Once again, I do not understand the argument, nor the relation with the restriction of `bool' not being subclass-able. > You can subclass int and implement __nonzero__ just like bool does (in > fact, bool does little more than that). There is undoubtedly a lot of things I could do. I was just curious about the deep reasons why Python does not allow this particular thing. > Is there a good reason for wanting to do this? Do you want to > "translate" all of Python to French? This is certainly not the way > to go about it! "all of Python to French"? As you go with hyperbole! :-) No. This is for one rather big Python program I did not even write, but which I may soon have to maintain for a while. There is a lot of cruft in that program (in my opinion) coming from all the handling of subtle nuances between truth and falsity, which merely show in the printouts. I am merely pondering the hypothesis that if I could "attach" types to booleans as they get computed, it could simplify the program and ease maintenance somehow. The simplest way to tag a `bool' might be to subclass it. The type could drive how the `bool' gets printed, but also hints the algorithm about where/why it was produced. OK, we could argue a lot about if I should proceed like this or otherwise. This is not much important in practice, and not the issue here. Normally, Python lets me do choose my ways without interfering, and this is no problem in practice because its reasons are almost always very clear to me. But here, Python tells me "Don't do that!", and I wonder why it chooses to interfere in this case. > You're going to receive True and False from other functions/keywords > whether you like it or not. I would bet that "not Vrai" would be > "False", for example. "Whether I like it or not". When my girlfriend use that expression, not often hopefully, I usually read her as irritated :-). I do not feel aggressed by Python, and there is no fight in here. We are strictly speaking about overriding the __repr__ (and maybe __str__), and also overriding the type with a sub-class in case this useful. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From tim.one at comcast.net Thu Feb 12 16:09:28 2004 From: tim.one at comcast.net (Tim Peters) Date: Thu Feb 12 16:11:29 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040212210132.GA7482@alcyon.progiciels-bpi.ca> Message-ID: See http://mail.python.org/pipermail/python-dev/2002-March/020822.html [Guido] I thought about this last night, and realized that you shouldn't be allowed to subclass bool at all! A subclass would only be useful when it has instances, but the mere existance of an instance of a subclass of bool would break the invariant that True and False are the only instances of bool! (An instance of a subclass of C is also an instance of C.) I think it's important not to provide a backdoor to create additional bool instances, so I think bool should not be subclassable. From ajole-1 at gci.net Thu Feb 12 16:22:52 2004 From: ajole-1 at gci.net (Patrick Stinson) Date: Thu Feb 12 17:07:21 2004 Subject: [Python-Dev] NULL tstate Message-ID: <200402121222.52293.ajole-1@gci.net> I am using sip to generate python bindings for my c++ library. sip is also used to generate python bindings for the Qt Gui toolkit. The PyEval_*Thread and PyThreadState_* seem to end up with a NULL tstate parameter at different points of execution when I start instantiating more complicated sip-wrapped widget classes along with my own multi-threaded library. At one point, I moved the thread creation code into python (by causing it to simply call a method in my library to execute the thread), and it helped a bit. I'm obviously not looking for help with sip, but it would be nice to get a general overview of what conditions cause a NULL tstate to be passed to these functions. I've posted a few messages on the sip mailing list about this, but the problem seems a little too biased toward the scope of this list for any of them to answer. Cheers! From pinard at iro.umontreal.ca Thu Feb 12 17:11:13 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Thu Feb 12 17:11:48 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: References: <20040212210132.GA7482@alcyon.progiciels-bpi.ca> Message-ID: <20040212221113.GA8017@alcyon.progiciels-bpi.ca> [Tim Peters] > See > http://mail.python.org/pipermail/python-dev/2002-March/020822.html Thanks for the reference! > [Guido] > the mere existance of an instance of a subclass of bool would > break the invariant that True and False are the only instances of > bool! (An instance of a subclass of C is also an instance of C.) This is a convincing argument, and enough a reason. Thanks! Now, I know why Python refuses! :-) -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From tdelaney at avaya.com Thu Feb 12 18:02:23 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Feb 12 18:02:30 2004 Subject: [Python-Dev] bool does not want to be subclassed? Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE013085D1@au3010avexu1.global.avaya.com> > From: Bob Ippolito > > Probably because bool is supposed to only have two values, True or > False.. As referred to in Tim's reposting of Guido's mind, the important distinction is that bool is only ever supposed to have two *instances* (True and False). The difference between instance and value in this case is important - int can only have values of type integer, but it can have many instances with the same value. Tim Delaney From niemeyer at conectiva.com Thu Feb 12 18:43:40 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Thu Feb 12 18:50:38 2004 Subject: [Python-Dev] Small test In-Reply-To: <1040212151902.ZM1698@reliant.hks.com> References: <20040212141134.GA10828@burma.localdomain> <1040212151902.ZM1698@reliant.hks.com> Message-ID: <20040212234340.GA2098@burma.localdomain> > > > can you please test the following piece of code with your compilers > > > and check if they're issuing any warnings? Thanks! [...] > Not a squeak. I tried both 32-bit and 64-bit compilations where applicable. [...] Many thanks for everyone who contributed! We're now aware that this solution will really fix the compiler warnings being issued in sre. -- Gustavo Niemeyer http://niemeyer.net From Joshua.Marshall at mathworks.com Thu Feb 12 17:29:31 2004 From: Joshua.Marshall at mathworks.com (Joshua Marshall) Date: Thu Feb 12 19:33:27 2004 Subject: [Python-Dev] bool does not want to be subclassed? Message-ID: <7224B63940F10F40A48AC423597ADE57012DC805@MESSAGE-AH.ad.mathworks.com> > [Guido] > the mere existance of an instance of a subclass of bool would > break the invariant that True and False are the only instances of > bool! (An instance of a subclass of C is also an instance of C.) > [Fran?ois] > This is a convincing argument, and enough a reason. Thanks! Now, I know why Python refuses! :-) I don't think I'm convinced; the same argument could be used for integers (if it doesn't make sense to create a sort of boolean which isn't in the set { true, false }, then it doesn't make sense to create a sort of integer which isn't in the set { ..., -2, -1, 0, 1, 2, ... }). And maybe it doesn't, but this isn't the only reason for subclassing. Another reason for subclassing is to create items which can act like existing objects, but which have some additional behavior. From tdelaney at avaya.com Thu Feb 12 19:50:52 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Feb 12 19:50:59 2004 Subject: [Python-Dev] bool does not want to be subclassed? Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE0130865D@au3010avexu1.global.avaya.com> > From: Joshua Marshall > > I don't think I'm convinced; the same argument could be used > for integers (if it doesn't make sense to create a sort of > boolean which isn't in the set { true, false }, then it > doesn't make sense to create a sort of integer which isn't in > the set { ..., -2, -1, 0, 1, 2, ... }). And maybe it > doesn't, but this isn't the only reason for subclassing. > Another reason for subclassing is to create items which can > act like existing objects, but which have some additional behavior. So should we be able to subclass NoneType so as to break the invariant that there is one and only one None instance? Tim Delaney From aahz at pythoncraft.com Thu Feb 12 20:56:49 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Feb 12 20:56:58 2004 Subject: [Python-Dev] NULL tstate In-Reply-To: <200402121222.52293.ajole-1@gci.net> References: <200402121222.52293.ajole-1@gci.net> Message-ID: <20040213015649.GA20707@panix.com> On Thu, Feb 12, 2004, Patrick Stinson wrote: > > I am using sip to generate python bindings for my c++ library. sip > is also used to generate python bindings for the Qt Gui toolkit. The > PyEval_*Thread and PyThreadState_* seem to end up with a NULL tstate > parameter at different points of execution when I start instantiating > more complicated sip-wrapped widget classes along with my own > multi-threaded library. At one point, I moved the thread creation > code into python (by causing it to simply call a method in my library > to execute the thread), and it helped a bit. > > I'm obviously not looking for help with sip, but it would be nice > to get a general overview of what conditions cause a NULL tstate to > be passed to these functions. I've posted a few messages on the sip > mailing list about this, but the problem seems a little too biased > toward the scope of this list for any of them to answer. Actually, this is probably more appropriate for comp.lang.python; that's where usage questions belong, even usage questions about the API internals. If you're sure there's a bug, file a report on SF (with as minimal a testcase as possible), then report it here. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach From kbk at shore.net Thu Feb 12 23:34:06 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Feb 12 23:34:25 2004 Subject: [Python-Dev] Weekly Python Patch/Bug Summary In-Reply-To: <87n082uj15.fsf@hydra.localdomain> (Kurt B. Kaiser's message of "Mon, 02 Feb 2004 00:50:30 -0500") References: <87n082uj15.fsf@hydra.localdomain> Message-ID: <871xozsim9.fsf@hydra.localdomain> Patch / Bug Summary ___________________ Patches : 246 open ( +0) / 2310 closed (+10) / 2556 total (+10) Bugs : 705 open (+20) / 3887 closed (+11) / 4592 total (+31) RFE : 127 open ( +3) / 118 closed ( +1) / 245 total ( +4) New / Reopened Patches ______________________ making the version of SSL configurable when creating sockets (2004-02-03) http://python.org/sf/889813 opened by adam goucher Make `commands' module work on Windows (2004-02-03) http://python.org/sf/889949 opened by Miki Tebeka DragonFly BSD support (2004-02-03) http://python.org/sf/890203 opened by Jon Parise example for urllib2 has SyntaxError (2004-02-08) http://python.org/sf/892821 opened by George Yoshida tp_dealloc, threads and garbage collection (2004-02-09) http://python.org/sf/893566 opened by Greg Chapman SimpleXMLRPCServer.py optional allow_none argument (2004-02-09) http://python.org/sf/893642 opened by Li Zou Updated .spec file. (2004-02-08) http://python.org/sf/892673 opened by Sean Reifschneider hmac.HMAC.copy() speedup (2004-02-11) http://python.org/sf/895445 opened by Trevor Perrin pdb enhancements and bug fixes (2004-02-12) http://python.org/sf/896011 opened by Ilya Sandler Patches Closed ______________ Bluetooth support for FreeBSD systems. (2004-01-31) http://python.org/sf/888148 closed by perky input() vs __future__ (2004-01-14) http://python.org/sf/876178 closed by perky Japanese Unicode Codecs (2003-01-12) http://python.org/sf/666484 closed by perky regrtest.py -T for code coverage (2003-12-31) http://python.org/sf/868499 closed by bwarsaw >100k alloc wasted on startup (2004-01-12) http://python.org/sf/875689 closed by rhettinger Fix typo in usage message(prechm.py) (2004-01-20) http://python.org/sf/880552 closed by rhettinger dynamic execution profiling vs opcode prediction (2004-01-25) http://python.org/sf/884022 closed by rhettinger example for urllib2 has SyntaxError (2004-02-08) http://python.org/sf/892821 closed by rhettinger Erroneous error message in test_types (2004-01-26) http://python.org/sf/885008 closed by doerwalter easiest 1% improvement in pystone ever (2004-01-13) http://python.org/sf/876198 closed by mwh New / Reopened Bugs ___________________ Odd warning behaviors (2004-02-03) http://python.org/sf/890010 opened by Tim Peters colorsys: bad URL (2004-02-03) http://python.org/sf/890081 opened by David M. Cooke Line 0 SyntaxWarning with duplicate global declarations (2004-02-02) http://python.org/sf/889500 opened by Jeff Connelly upack fails under certain conditions on Windows (2004-02-04) http://python.org/sf/890687 opened by Kevin upack produces incorrect result in certain circumstances (2004-02-04) http://python.org/sf/890705 opened by Kevin BSDDB set_location bug (2004-02-05) http://python.org/sf/890872 opened by Mike C. Fletcher pickle dump argument description vague (2004-02-05) http://python.org/sf/891249 opened by Brad Clements dict.get() evaluates second parameter even if key is present (2004-02-05) http://python.org/sf/891367 opened by Claude Belisle pydoc crashes on a class property (2004-02-06) http://python.org/sf/891637 opened by Matthias Klose commands module doesn't support background commands (2004-02-06) http://python.org/sf/891832 opened by Skip Montanaro PyBool_* not documented in C API (2004-02-02) http://python.org/sf/889154 opened by Fred L. Drake, Jr. Multiple close() for asyncore.dispatcher. (2004-02-07) http://python.org/sf/892492 opened by Alexey Klimkin asyncore.dispactcher: incorrect connect (2004-02-02) http://python.org/sf/889153 opened by Sankov Dmitry Alexandrovich typo in textwrap doc page (2004-02-08) http://python.org/sf/892854 opened by Drew Perttula problem with pickling newstyle class instances (2004-02-08) http://python.org/sf/892902 opened by Toon Verstraelen Race condition in popen2 (2004-02-08) http://python.org/sf/892939 opened by Ken McNeil bundlebuilder standalone app doesn't fully quit (2004-02-02) http://python.org/sf/889200 opened by Russell Owen curses getkey() crash in raw mode (2004-02-09) http://python.org/sf/893250 opened by J. David Lee skipitem() in getargs.c missing some types (2004-02-09) http://python.org/sf/893549 opened by Kirill A Kryloff Segmentation fault running XSV (2004-02-09) http://python.org/sf/893610 opened by Jean-Baptiste Combes configure argument --libdir is ignored (2004-02-06) http://python.org/sf/891930 opened by G?ran Uddeborg - very minor typo (2004-02-10) http://python.org/sf/894428 opened by Nikolay Diakov Segmentation fault running XSV (2004-02-09) http://python.org/sf/893610 opened by Jean-Baptiste Combes Win32 installer should add python's directory to system path (2004-02-10) http://python.org/sf/894599 opened by Viktor Ferenczi build_ssl can't find ActiveState Perl from unusual locations (2004-02-10) http://python.org/sf/894743 opened by Heikki Toivonen email library Parser can't parse some messages (2004-02-12) http://python.org/sf/895510 opened by Jan Ondrej - another minor typo (2004-02-12) http://python.org/sf/895560 opened by Nikolay Diakov os.listdir fails for pathprefix \?\d:... (2004-02-12) http://python.org/sf/895567 opened by Guenter Kruschina symtable.Symbol.is_global() is strange (2004-02-12) http://python.org/sf/896061 opened by Peter ?strand Some Carbon modules missing (2004-02-12) http://python.org/sf/896199 opened by Russell Owen Unicode problem in os.path.getsize ? (2004-02-12) http://python.org/sf/896236 opened by Ronald L. Rivest Bugs Closed ___________ upack fails under certain conditions on Windows (2004-02-04) http://python.org/sf/890687 closed by theller unpack produces incorrect result in certain circumstances (2004-02-04) http://python.org/sf/890705 closed by theller dict.get() evaluates second parameter even if key is present (2004-02-05) http://python.org/sf/891367 closed by rhettinger Multiple close() for asyncore.dispatcher. (2004-02-07) http://python.org/sf/892492 closed by rhettinger opcode prediction distorts dynamic execution profile stats (2004-01-25) http://python.org/sf/884020 closed by rhettinger typo in textwrap doc page (2004-02-08) http://python.org/sf/892854 closed by rhettinger Config option --without-pth not working? (2002-04-11) http://python.org/sf/542751 closed by blueronin colorsys: bad URL (2004-02-03) http://python.org/sf/890081 closed by bcannon - very minor typo (2004-02-10) http://python.org/sf/894428 closed by nnorwitz - another minor typo (2004-02-12) http://python.org/sf/895560 closed by rhettinger metaclasses and __unicode__ (2004-01-25) http://python.org/sf/884064 closed by mwh New / Reopened RFE __________________ win32 raw socket support (2004-02-03) http://python.org/sf/889544 opened by Jesko Huettenhain debuggable/profileable Python framework (2004-02-07) http://python.org/sf/892707 opened by Bob Ippolito Enumerate with step (2004-02-08) http://python.org/sf/892804 opened by Gerrit Holl Have a split corresponding with os.path.join (2004-02-11) http://python.org/sf/894936 opened by Nicolas Fleury RFE Closed __________ Enumerate with step (2004-02-08) http://python.org/sf/892804 closed by rhettinger From Scott.Daniels at Acm.Org Fri Feb 13 01:21:25 2004 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri Feb 13 01:21:54 2004 Subject: [Python-Dev] Bits module -- early working version Message-ID: <402C6CE5.4090004@Acm.Org> I've been working on a module to get at the bits of all numeric types (no, I haven't thought of how to solve the decimal data type that is coming). I've finally got the bits module to pass all of my own tests, so I'm looking for bug reports, design critiques, and general input. I eventually plan to release this under an MIT-style license. If you are interested in seeing read access to the bits of python's numbers, and want to see what I'm up to / critique someone else's work / volunteer to help extend, check out: http://members.dsl-only.net/~daniels/bits.html If you want to know what questions I have, here are my current set: 1) Do you know a case that fails? 2) Should the names be lsb, lsbit, or lsbitno (and similarly for msb*)? I've gone with lsb, but cases can be made for the others. 3) Is "extract" the right name, or should I us a name more like "bits"? 4) Should bit(v, N) somehow be subsumed in extract(v, l, hi) (or whatever extract should be called)? 5) Have I adequately explained what these functions do? 6) Do these functions work as-is (from source) for various machines? I know/believe Intel x86/pentia work; what about alpha, 68K, .... 7) Should bitcount simply raise and exception on negative input? 8) If you supply an unexpected argument type for the number, should I try to calla corresponding method? (__bit__ for bit, __msb__ for msb, ....) So I'd like feedback before actually releasing. -- -Scott David Daniels Scott.Daniels@Acm.Org From martin at v.loewis.de Fri Feb 13 02:47:19 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri Feb 13 02:47:38 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <7224B63940F10F40A48AC423597ADE57012DC805@MESSAGE-AH.ad.mathworks.com> References: <7224B63940F10F40A48AC423597ADE57012DC805@MESSAGE-AH.ad.mathworks.com> Message-ID: <402C8107.7020603@v.loewis.de> Joshua Marshall wrote: > I don't think I'm convinced; the same argument could be used for > integers (if it doesn't make sense to create a sort of boolean which > isn't in the set { true, false }, then it doesn't make sense to > create a sort of integer which isn't in the set { ..., -2, -1, 0, 1, > 2, ... }). And maybe it doesn't, but this isn't the only reason for > subclassing. Another reason for subclassing is to create items which > can act like existing objects, but which have some additional > behavior. And indeed, for int, it is possible to have subclasses which have new instances whose values are in the set {..., -2, -1, 0, 1, 2, ...}. Indeed, it is possible to have multiple instances of int *itself* whose value is, say, 1000: >>> 500+500 is 500+500 False The same is not true for bool: There are only two *instances* of the type, not just two equivalence classes of equal values: >>> (4>5) is (3>9) True So bool guarantees: a) there are only two distinct values, and b) there are only two different objects representing these values. It is property b) which prohibits subclassing. Regards, Martin From miki.tebeka at zoran.com Fri Feb 13 08:01:41 2004 From: miki.tebeka at zoran.com (Miki Tebeka) Date: Fri Feb 13 10:05:07 2004 Subject: [Python-Dev] Small test In-Reply-To: References: Message-ID: <20040213130141.GL2068@zoran.com> Hello Gustavo , > can you please test the following piece of code with your compilers > and check if they're issuing any warnings? Thanks! > > #include > int main(int argc, char *argv[]) > { > char c = 1; > if (!(c & ~255)) printf("Thanks!\n"); > return 0; > } No complains from gcc 3.3.1 on cygwin. Bye. -- ------------------------------------------------------------------------- Miki Tebeka http://www.cs.bgu.ac.il/~tebeka The only difference between children and adults is the price of the toys. From pinard at iro.umontreal.ca Fri Feb 13 12:01:39 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Fri Feb 13 12:02:18 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <7224B63940F10F40A48AC423597ADE57012DC805@MESSAGE-AH.ad.mathworks.com> References: <7224B63940F10F40A48AC423597ADE57012DC805@MESSAGE-AH.ad.mathworks.com> Message-ID: <20040213170139.GA14216@alcyon.progiciels-bpi.ca> [Joshua Marshall] > [...] if it doesn't make sense to create a sort of boolean which isn't > in the set { true, false }, then it doesn't make sense to create a > sort of integer which isn't in the set { ..., -2, -1, 0, 1, 2, ... }. Merely chatting, here. In that same big application for which I considered (for a short moment) sub-classing `bool' to "mark" booleans with their computation origin, we have a particular other kind-of-boolean type, heavily used in the application, which runs six values from full falsity to full truth. Python surely made it very elegant. This one, I would have never ever considered sub-typing from `bool' :-). For fun, we named the constructor `poul' and we speak of pouleans, as a friendly tease to the mathematician in the team who decided the details of this particular logic (Jacques Poulin). About interesting extensions to integers, and to all kind of numbers in fact, including wet ones :-), a good read is the John Conway book: "On games and numbers". However, it is not worth a Python module yet! :-) -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From tjreedy at udel.edu Fri Feb 13 17:03:39 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Fri Feb 13 17:03:45 2004 Subject: [Python-Dev] Re: bool does not want to be subclassed? References: <7224B63940F10F40A48AC423597ADE57012DC805@MESSAGE-AH.ad.mathworks.com> <20040213170139.GA14216@alcyon.progiciels-bpi.ca> Message-ID: "François Pinard" wrote in message news:20040213170139.GA14216@alcyon.progiciels-bpi.ca... About interesting extensions to integers, and to all kind of numbers in fact, including wet ones :-), a good read is the John Conway book: "On games and numbers". However, it is not worth a Python module yet! :-) rather, On Numbers and Games (ONAG. 1976) Berlekamp, Conway, & Guy; Winning Ways...(1982) has some similar material Terry J. Reedy From Joshua.Marshall at mathworks.com Fri Feb 13 18:41:04 2004 From: Joshua.Marshall at mathworks.com (Joshua Marshall) Date: Fri Feb 13 18:41:09 2004 Subject: [Python-Dev] bool does not want to be subclassed? Message-ID: <7224B63940F10F40A48AC423597ADE57012DC808@MESSAGE-AH.ad.mathworks.com> [Joshua Marshall] > I don't think I'm convinced; the same argument could be used for > integers (if it doesn't make sense to create a sort of boolean which > isn't in the set { true, false }, then it doesn't make sense to create > a sort of integer which isn't in the set { ..., -2, -1, 0, 1, 2, ... > }). And maybe it doesn't, but this isn't the only reason for > subclassing. Another reason for subclassing is to create items which > can act like existing objects, but which have some additional > behavior. [Martin] > And indeed, for int, it is possible to have subclasses which > have new instances whose values are in the set {..., -2, -1, > 0, 1, 2, ...}. Indeed, it is possible to have multiple > instances of int *itself* whose value is, say, 1000: > > >>> 500+500 is 500+500 > False > > The same is not true for bool: There are only two *instances* > of the type, not just two equivalence classes of equal values: > > >>> (4>5) is (3>9) > True > > So bool guarantees: a) there are only two distinct values, > and b) there are only two different objects representing > these values. It is property b) which prohibits subclassing. Ah I misread Guido's comment--he's talking about the actual objects, not what they represent. So a different question... Can it be relied upon that two expressions which both evaluate to False both return the same object? That is, is it incorrect for a Python interpreter not to do this? I find this in the Python Reference Manual: "for immutable types, operations that compute new values may actually return a reference to any existing object with the same type and value" (note the "may"). Are bool and NoneType the only types for which this is reliably the case? From tim.one at comcast.net Fri Feb 13 18:48:23 2004 From: tim.one at comcast.net (Tim Peters) Date: Fri Feb 13 18:48:26 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <7224B63940F10F40A48AC423597ADE57012DC808@MESSAGE-AH.ad.mathworks.com> Message-ID: [Joshua Marshall] > ... > So a different question... Can it be relied upon that two expressions > which both evaluate to False both return the same object? Yes. > That is, is it incorrect for a Python interpreter not to do this? Yes. > I find this in the Python Reference Manual: "for immutable types, > operations that compute new values may actually return a reference to > any existing object with the same type and value" (note the "may"). > Are bool and NoneType the only types for which this is reliably the case? Type objects are also singletons (e.g., type(True) returns the same object as type(False); ditto type("True") and type("False") and type("xyz"); etc). From guido at python.org Fri Feb 13 18:59:48 2004 From: guido at python.org (Guido van Rossum) Date: Fri Feb 13 18:59:54 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: Your message of "Fri, 13 Feb 2004 18:41:04 EST." <7224B63940F10F40A48AC423597ADE57012DC808@MESSAGE-AH.ad.mathworks.com> References: <7224B63940F10F40A48AC423597ADE57012DC808@MESSAGE-AH.ad.mathworks.com> Message-ID: <200402132359.i1DNxmk03998@c-24-5-134-209.client.comcast.net> > So a different question... Can it be relied upon that two expressions > which both evaluate to False both return the same object? That is, is > it incorrect for a Python interpreter not to do this? I find this in > the Python Reference Manual: "for immutable types, operations that > compute new values may actually return a reference to any existing > object with the same type and value" (note the "may"). Are bool and > NoneType the only types for which this is reliably the case? Among immutable built-in types, these plus Ellipsis are the only ones. The language doesn't guarantee such a thing for numbers other than bool, strings, unicode or tuples. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Feb 13 19:03:55 2004 From: guido at python.org (Guido van Rossum) Date: Fri Feb 13 19:04:02 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: Your message of "Fri, 13 Feb 2004 18:48:23 EST." References: Message-ID: <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> > Type objects are also singletons (e.g., type(True) returns the same object > as type(False); ditto type("True") and type("False") and type("xyz"); etc). But types are not immutable, so their semantics are different anyway. The point of the remark was to emphasize that immutable types have some leniency in their semantics regarding object identity. For mutable types the semantics regarding object identity are defined explicitly by each type or operation (some operations always return a new object, others always return the same one). --Guido van Rossum (home page: http://www.python.org/~guido/) From neal at metaslash.com Fri Feb 13 21:05:36 2004 From: neal at metaslash.com (Neal Norwitz) Date: Fri Feb 13 21:05:43 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects listobject.c, 2.177, 2.178 In-Reply-To: References: Message-ID: <20040214020536.GG7856@epoch.metaslash.com> On Fri, Feb 13, 2004 at 03:36:41AM -0800, rhettinger@users.sourceforge.net wrote: > *************** > *** 518,540 **** > else { /* Insert d items; recycle ihigh-ilow items */ > ! NRESIZE(item, PyObject *, a->ob_size + d); > ! if (item == NULL) { > if (recycle != NULL) > PyMem_DEL(recycle); > - PyErr_NoMemory(); > - return -1; > } > --- 517,535 ---- > else { /* Insert d items; recycle ihigh-ilow items */ > ! s = a->ob_size; > ! if (list_resize(a, s+d) == -1) { > if (recycle != NULL) > PyMem_DEL(recycle); > } Missing the return -1; when list_resize() fails? In the following 2 for loops, the original code modified self->ob_size. In the new version I don't notice ob_size being updated. Given the context, I'm not sure if the behaviour is different or not between versions. > *************** > *** 592,612 **** > for (i = 1; i < n; i++) { /* Start counting at 1, not 0 */ > for (j = 0; j < size; j++) { > PyObject *o = PyList_GET_ITEM(self, j); > Py_INCREF(o); > ! PyList_SET_ITEM(self, self->ob_size++, o); > } > } > --- 586,602 ---- > for (i = 1; i < n; i++) { /* Start counting at 1, not 0 */ > for (j = 0; j < size; j++) { > PyObject *o = PyList_GET_ITEM(self, j); > Py_INCREF(o); > ! PyList_SET_ITEM(self, p++, o); > } > } > *************** > *** 687,707 **** > /* populate the end of self with b's items */ > for (i = 0; i < blen; i++) { > PyObject *o = PySequence_Fast_GET_ITEM(b, i); > Py_INCREF(o); > ! PyList_SET_ITEM(self, self->ob_size++, o); > } > --- 676,689 ---- > /* populate the end of self with b's items */ > for (i = 0; i < blen; i++) { > PyObject *o = PySequence_Fast_GET_ITEM(b, i); > Py_INCREF(o); > ! PyList_SET_ITEM(self, i+selflen, o); > } From python at rcn.com Fri Feb 13 21:41:52 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Feb 13 21:43:26 2004 Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Objects listobject.c, 2.177, 2.178 In-Reply-To: <20040214020536.GG7856@epoch.metaslash.com> Message-ID: <001901c3f2a4$1abf2a40$8ab89d8d@oemcomputer> [Neal Norwitz] > Missing the return -1; when list_resize() fails? Yes. Will fix. > In the following 2 for loops, the original code modified > self->ob_size. In the new version I don't notice ob_size > being updated. Given the context, I'm not sure if the behaviour > is different or not between versions. self->ob_size is now updated by list_resize() so the loop no longer needs to do the update. This is a nice improvement clients of list_resize() do not need to manage ob_size; rather, the just request the size they want and the data structure keeps its integrity. Raymond From tim.one at comcast.net Fri Feb 13 23:55:15 2004 From: tim.one at comcast.net (Tim Peters) Date: Fri Feb 13 23:55:19 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> Message-ID: >> Type objects are also singletons (e.g., type(True) returns the same >> object as type(False); ditto type("True") and type("False") and >> type("xyz"); etc). [Guido] > But types are not immutable, so their semantics are different anyway. > The point of the remark was to emphasize that immutable types have > some leniency in their semantics regarding object identity. For > mutable types the semantics regarding object identity are defined > explicitly by each type or operation (some operations always return a > new object, others always return the same one). The latter is forbidden by the language reference manual, though. Here's Joshua's original quote, but with more context: Types affect almost all aspects of object behavior. Even the importance of object identity is affected in some sense: for immutable types, operations that compute new values may actually return a reference to any existing object with the same type and value, while for mutable objects this is not allowed. I never thought of type objects as being mutable -- they're compared ("==") by identity, are usable as dict keys, act like immutable objects according to the text above, etc. Do you really think of them as being mutable? If so, why ? E.g., after "a = 1; b = 1", a and b may or may not refer to the same object with the value one, depending on the implementation, but after "c = []; d = []", c and d are guaranteed to refer to two different, unique, newly created empty lists. (Note that "c = d = []" assigns the same object to both c and d.) If the text in the first quoted paragraph isn't correct, then this second paragraph doesn't explain anything about mutable objects in general by way of an example that happens to pick on empty-list literals, rather it's making a promise only about empty-list literals, but in a misleading way (reading for all the world as if it *were* trying to explain something about mutable objects in general by way of an example that happens to pick on empty-list literals). I expect the easiest way out is for you to decide that type objects are immutable after all -- even if there are obscure ways to mutate them! From martin at v.loewis.de Sat Feb 14 08:28:32 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Feb 14 08:28:34 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: References: Message-ID: <402E2280.5070404@v.loewis.de> Tim Peters wrote: > I expect the easiest way out is for you to decide that type objects are > immutable after all -- even if there are obscure ways to mutate them! It's not at all obscure: >>> class A(object):pass ... >>> type(A).__name__ 'type' >>> A.foo = 1 >>> A.foo 1 Look, Ma, I'm mutating a type! No, son, you are just modifying it. The properties you quote (comparison checks for identity, usable as a dictionary key) don't imply at all that an object is immutable. Regards, Martin From gerrit at nl.linux.org Sat Feb 14 11:14:45 2004 From: gerrit at nl.linux.org (Gerrit) Date: Sat Feb 14 11:15:33 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <402E2280.5070404@v.loewis.de> References: <402E2280.5070404@v.loewis.de> Message-ID: <20040214161445.GA21251@nl.linux.org> "Martin v. L=EF=BF=BDwis" wrote: > Tim Peters wrote: > >I expect the easiest way out is for you to decide that type objects ar= e > >immutable after all -- even if there are obscure ways to mutate them! >=20 > It's not at all obscure: >=20 > >>> class A(object):pass > ... > >>> type(A).__name__ > 'type' > >>> A.foo =3D 1 > >>> A.foo > 1 >=20 > Look, Ma, I'm mutating a type! > No, son, you are just modifying it. Isn't this just modifying A.__dict__, where __getattribute__ looks into to see whether something is defined? Or is that the same thing? Gerrit. --=20 Weather in Twenthe, Netherlands 14/02 15:25 UTC: 8.0=C2=B0C wind 2.2 m/s SW (57 m above NAP) --=20 Asperger's Syndrome - a personal approach: http://people.nl.linux.org/~gerrit/english/ From martin at v.loewis.de Sat Feb 14 11:22:34 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Feb 14 11:23:18 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040214161445.GA21251@nl.linux.org> References: <402E2280.5070404@v.loewis.de> <20040214161445.GA21251@nl.linux.org> Message-ID: <402E4B4A.2060506@v.loewis.de> Gerrit wrote: > Isn't this just modifying A.__dict__, where __getattribute__ looks into > to see whether something is defined? Or is that the same thing? Correct. However, to the user, this should be an implementation detail, and is nearly indistinguishable from >>> A.__name__="B" >>> A Regards, Martin From tjreedy at udel.edu Sat Feb 14 12:39:42 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Sat Feb 14 12:40:02 2004 Subject: [Python-Dev] Re: bool does not want to be subclassed? References: <402E2280.5070404@v.loewis.de> Message-ID: "Martin v. Löwis" wrote in message news:402E2280.5070404@v.loewis.de... > Tim Peters wrote: > > I expect the easiest way out is for you to decide that type objects are > > immutable after all -- even if there are obscure ways to mutate them! > > It's not at all obscure: > > >>> class A(object):pass > ... > >>> type(A).__name__ > 'type' > >>> A.foo = 1 > >>> A.foo > 1 > > Look, Ma, I'm mutating a type! > No, son, you are just modifying it. > > The properties you quote (comparison checks for identity, usable as a > dictionary key) don't imply at all that an object is immutable. Leaving aside mutability, there does seems to be something distinctively 'n-gleton'-like about types in that the identity of a type object is (a surrogate for) its value. For instance, after def f1(): pass def f2(): pass f1 and f2 will compare unequal, simply because '==' does not check functions for attribute equality, including code-object, identity. Nonetheless, f1 and f2 are interchangible in any calculation not dependent on identity. On the otherhand, after class c1(object): pass class c2(object): pass c1 and c2 *are* different and produce instances of different types when called. This sometimes surprises people, leading to misplaced clp bug posts, when the duplication is unintentional, as with slightly different imports resulting in two modules derived from the same disk file. Terry J. Reedy From niemeyer at conectiva.com Sun Feb 15 19:47:36 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Sun Feb 15 18:54:32 2004 Subject: [Python-Dev] Source encoding doubt Message-ID: <20040216004736.GA9654@burma.localdomain> Hello! I'm wondering what I misunderstood in PEP263. In the following example, shouldn't it work correctly, showing '?????', as far as my terminal accepts ISO-8859-1? #!/usr/bin/python # -*- coding: utf-8 -*- s1 = "\xc3\xa1\xc3\xa9\xc3\xad\xc3\xb3\xc3\xba" s2 = u"\xc3\xa1\xc3\xa9\xc3\xad\xc3\xb3\xc3\xba" print s1.encode('iso-8859-1') print s2.encode('iso-8859-1') -- Gustavo Niemeyer http://niemeyer.net From niemeyer at conectiva.com Sun Feb 15 20:11:48 2004 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Sun Feb 15 19:18:43 2004 Subject: [Python-Dev] Re: Source encoding doubt In-Reply-To: <20040216004736.GA9654@burma.localdomain> References: <20040216004736.GA9654@burma.localdomain> Message-ID: <20040216011148.GA10022@burma.localdomain> > I'm wondering what I misunderstood in PEP263. In the following example, > shouldn't it work correctly, showing '?????', as far as my terminal > accepts ISO-8859-1? > > #!/usr/bin/python > # -*- coding: utf-8 -*- > s1 = "\xc3\xa1\xc3\xa9\xc3\xad\xc3\xb3\xc3\xba" > s2 = u"\xc3\xa1\xc3\xa9\xc3\xad\xc3\xb3\xc3\xba" > print s1.encode('iso-8859-1') > print s2.encode('iso-8859-1') Argh.. ok, understood. I need some sleep. These strings are not unicode, but escapings of unicode strings. -- Gustavo Niemeyer http://niemeyer.net From greg at cosc.canterbury.ac.nz Sun Feb 15 20:37:04 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Sun Feb 15 20:37:28 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <402C8107.7020603@v.loewis.de> Message-ID: <200402160137.i1G1b4T05760@oma.cosc.canterbury.ac.nz> Martin: > So bool guarantees: ... b) > there are only two different objects representing these values. Why is this property so important? I can understand why it's useful for there to be only one None, since programs do 'x is None' a lot. But it's not as if you can get away with saying 'x is True' or 'x is False' all over the place. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From pinard at iro.umontreal.ca Sun Feb 15 22:19:53 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Sun Feb 15 22:23:30 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <200402160137.i1G1b4T05760@oma.cosc.canterbury.ac.nz> References: <402C8107.7020603@v.loewis.de> <200402160137.i1G1b4T05760@oma.cosc.canterbury.ac.nz> Message-ID: <20040216031953.GA10333@titan.progiciels-bpi.ca> [Greg Ewing] > Martin: > > So bool guarantees: ... b) > > there are only two different objects representing these values. > Why is this property so important? When I read the quote from Guido speaking about code invariants, I presumed that the internal code for `bool' uses things like: if SUCH is True: ... else: ... relying on the facts that 1) if SUCH is true, than SUCH _is_ True, and also, 2) that SUCH not being True means that it is necessarily False. These surely allow for a simple and efficient `bool'. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From jeremy at alum.mit.edu Sun Feb 15 22:55:11 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Sun Feb 15 22:58:31 2004 Subject: [Python-Dev] PyDict_Size() error return Message-ID: <1076903710.23662.26.camel@localhost.localdomain> PyDict_Size() sets an exception and returns 0 if it is passed NULL or a non-dict. Is this a bug? It seems like it should return -1, otherwise you'd have to distinguish between an empty dict and an error by calling PyErr_Occurred(). Jeremy From guido at python.org Mon Feb 16 01:00:49 2004 From: guido at python.org (Guido van Rossum) Date: Mon Feb 16 01:01:03 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: Your message of "Mon, 16 Feb 2004 14:37:04 +1300." <200402160137.i1G1b4T05760@oma.cosc.canterbury.ac.nz> References: <200402160137.i1G1b4T05760@oma.cosc.canterbury.ac.nz> Message-ID: <200402160600.i1G60oP24381@c-24-5-134-209.client.comcast.net> > > there are only two different objects representing these values. > > Why is this property so important? > > I can understand why it's useful for there to be only > one None, since programs do 'x is None' a lot. But it's > not as if you can get away with saying 'x is True' or > 'x is False' all over the place. Why is it important *not* to have this property? Maintaining it is trivial, and it helps the implementation a bit (it's actually *natural* to have only the two instances). --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Feb 16 01:06:41 2004 From: guido at python.org (Guido van Rossum) Date: Mon Feb 16 01:06:50 2004 Subject: [Python-Dev] PyDict_Size() error return In-Reply-To: Your message of "Sun, 15 Feb 2004 22:55:11 EST." <1076903710.23662.26.camel@localhost.localdomain> References: <1076903710.23662.26.camel@localhost.localdomain> Message-ID: <200402160606.i1G66fY24411@c-24-5-134-209.client.comcast.net> > PyDict_Size() sets an exception and returns 0 if it is passed NULL or a > non-dict. Is this a bug? It seems like it should return -1, otherwise > you'd have to distinguish between an empty dict and an error by calling > PyErr_Occurred(). Looks like an (old) bug to me. All other similar cases return NULL or -1, and some calls to PyDict_Size() actually check for < 0. (Many calls don't check for errors, but those probably "know" that they are dealing with a dict and maintain that invariant in other ways.) --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard at iro.umontreal.ca Mon Feb 16 08:37:25 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 16 09:21:13 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <200402160600.i1G60oP24381@c-24-5-134-209.client.comcast.net> References: <200402160137.i1G1b4T05760@oma.cosc.canterbury.ac.nz> <200402160600.i1G60oP24381@c-24-5-134-209.client.comcast.net> Message-ID: <20040216133725.GA3123@titan.progiciels-bpi.ca> [Guido van Rossum] > > > there are only two different objects representing these values. > > Why is this property so important? > > I can understand why it's useful for there to be only one None, > > since programs do 'x is None' a lot. But it's not as if you can get > > away with saying 'x is True' or 'x is False' all over the place. > Why is it important *not* to have this property? Maintaining it is > trivial, and it helps the implementation a bit (it's actually > *natural* to have only the two instances). It is not so _important_ *not* to have this property. However, if it did not have this property, this would allow for sub-classing `bool', something useful at times, as I tried to explain in previous messages. The usefulness of sub-classing `bool' would likely not be worth any significant slow-down of the implementation. Yet, if the slowdown was not to be significant, than it might be worth removing the restriction about `bool' not allowing sub-classing, so making it more like `int', and most other types. I was surprised at first to see that `bool' could not be subclassed, and so, I asked why; the explanations given on this list have been helpful. On the other hand, Python should _ideally_ try to limit the number of limitations which surprise users. That ideal is difficult, as Python has many complex corners when you dive a bit under the surface, and some of these complexities just peek through the surface, once in a while. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From pinard at iro.umontreal.ca Mon Feb 16 09:37:38 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 16 09:41:41 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: References: <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> Message-ID: <20040216143738.GA4445@titan.progiciels-bpi.ca> [Tim Peters] > >> Type objects are also singletons (e.g., type(True) returns the same > >> object as type(False); ditto type("True") and type("False") and > >> type("xyz"); etc). > [Guido] > > But types are not immutable, so their semantics are different anyway. > I never thought of type objects as being mutable [...] I'm not sure of the meaning of "immutability of a type". Looking around, I found out this other message from Guido (quoted in its entirety in case the context changes the wanted meaning of words): ----------------------------------------------------------------------> From: Guido van Rossum Date: Thu, 18 Oct 2001 21:00:28 GMT Subject: Re: hashval and Numpy To: python-list@python.org Newsgroups: comp.lang.python > Michael Hudson wrote: > > |>>> a = Numeric.array([1,2]) > |>>> hash(a) > | 56 > |>>> d = {a:1} # a will be in the 56%8-th slot (in 2.2, anyway) > |>>> a[0] = 2 > |>>> hash(a) > | 57 > |>>> d[a] # looks in 57%8-th slot - which is empty > | KeyError > |>>> b = Numeric.array([1,2]) > |>>> hash(b) > | 56 > |>>> d[b] # looks in 56%8-th slot - but what's there is not __eq__ to b! > | KeyError > > | Moral: don't use mutable objects as dictionary keys. Hm. The Numeric.array object should not have defined __hash__ or tp_hash. It's wrong if a mutable object defines a hash function, for exactly this reason. ----------------------------------------------------------------------< However, all types derive from object, and object has `__hash__'. Consequently, I would be tempted to think that under the new system, everything deriving from object is immutable by construction, unless tricks are used against it, like maybe, an intermediate class overriding `__hash__' with some function with the specific goal of raising an exception. Is that so? What means a "mutable type" then? From martin at v.loewis.de Mon Feb 16 10:30:25 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon Feb 16 10:31:34 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040216143738.GA4445@titan.progiciels-bpi.ca> References: <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> <20040216143738.GA4445@titan.progiciels-bpi.ca> Message-ID: <4030E211.1050606@v.loewis.de> > Hm. The Numeric.array object should not have defined __hash__ or > tp_hash. It's wrong if a mutable object defines a hash function, for > exactly this reason. I think Guido was somewhat too terse here. Mutable objects should not define hash functions that change depending if their state changes - a hash must be constant for the life of the object. > However, all types derive from object, and object has `__hash__'. > Consequently, I would be tempted to think that under the new system, > everything deriving from object is immutable by construction, unless > tricks are used against it You are also misinterpreting "defines a hash function". Any object is hashable, but, unless specifically defined, the hash function uses the identity of the object (in CPython, its address) to compute the hash; this won't change if the state changes. So you should read Guido's statement also as "it's wrong if a mutable object *specifically* defines a hash function". > What means a "mutable type" then? A mutable type is one whose instances have state that can change over the life of the object. Of these, there are two categories of objects: - values: objects whose identity is mostly irrelevant, and who compare equal if certain aspects of their state are equal. array.array is an example of a value type. - identity objects: objects whose identity matters in addition to their state; different objects will never compare equal. Identity objects should always be hashable, and their hash should be based on the identity. Mutability does not matter wrt. to hashing and dictionary keys. Values fall into two further categories: - immutable values: they should define a hash that takes those parts of the state into account that is also considered for comparison; equal objects should hash equal. - mutable values: an attempt to hash them should raise an exception, as one would require the following, contradicting, properties of a hash on such objects: - equal objects should hash equal (requires to take state into account) - the hash of an object should never change (requuires not to take the state into account). Dictionaries and array.array instances are mutable values, so they should have no hash. Types are identity objects; they can hash based on their identity despite being mutable. Regards, Martin From tim.one at comcast.net Mon Feb 16 12:31:32 2004 From: tim.one at comcast.net (Tim Peters) Date: Mon Feb 16 12:35:39 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <4030E211.1050606@v.loewis.de> Message-ID: [Fran?ois Pinard] >> However, all types derive from object, and object has `__hash__'. >> Consequently, I would be tempted to think that under the new system, >> everything deriving from object is immutable by construction, unless >> tricks are used against it. That object has __hash__ now is considered to be a bug: "New style classes and __hash__" http://www.python.org/sf/660098 [Martin v. L?wis] > You are also misinterpreting "defines a hash function". Any object > is hashable, That's not so; e.g., >>> hash([]) Traceback (most recent call last): File "", line 1, in ? TypeError: list objects are unhashable >>> or >>> class C: ... def __cmp__(a, b): return 0 >>> hash(C()) Traceback (most recent call last): File "", line 1, in ? TypeError: unhashable instance >>> That changing the last to use a new-style class: >>> class C(object): ... def __cmp__(a, b): return 0 ... >>> hash(C()) 6973584 >>> doesn't complain is one of the bugs in the report referenced above. The docs for __hash__ still spell out the *intent*: If a class does not define a __cmp__() method it should not define a __hash__() operation either; if it defines __cmp__() or __eq__() but not __hash__(), its instances will not be usable as dictionary keys. If a class defines mutable objects and implements a __cmp__() or __eq__() method, it should not implement __hash__(), since the dictionary implementation requires that a key's hash value is immutable (if the object's hash value changes, it will be in the wrong hash bucket) Old-style class instances work that way, and to the extent new-style class instances don't in 2.3, the new-style class implementation is in error. > but, unless specifically defined, the hash function uses > the identity of the object (in CPython, its address) to compute the > hash; this won't change if the state changes. > > So you should read Guido's statement also as "it's wrong if a mutable > object *specifically* defines a hash function". >> What means a "mutable type" then? It's gotten more confused. When types and classes were distinct beasts, types were immutable and classes were mutable, and instances of types or classes could be mutable or immutable (depending on the type or class they were instances of). The word "type" has lost its crispness in that respect. > A mutable type is one whose instances have state that can change over > the life of the object. I accept that, but it's not very useful on its own. Whether a type compares instances via value or via identity, and whether a type's instances can be used as dict keys, and whether a type's instances can be *sanely* used as dict keys, and whether a type's instances are hashable, can't be predicted just from knowing that the type's instances are mutable or immutable -- but people think they can be . From guido at python.org Mon Feb 16 12:55:18 2004 From: guido at python.org (Guido van Rossum) Date: Mon Feb 16 12:55:33 2004 Subject: [Python-Dev] PyCon DC Early bird rate EXTENDED until Feb 22, 2004 Message-ID: <200402161755.i1GHtJC25824@c-24-5-134-209.client.comcast.net> Due to overwhelming success, the early bird deadline for PyCon DC 2004 has been *extended* to February 22. The early bird registration fee is only $175 ($125 for students). Monday the 23rd it goes up to $250 (really). To register, visit: http://www.pycon.org/dc2004/register/ Speakers, you too!!! DC 2004 will be held March 24-26, 2004 in Washington, D.C. The keynote speaker is Mitch Kapor of the Open Source Applications Foundation (http://www.osafoundation.org/). Famous C++/Java/Python author/evangelist Bruce Eckel will give a closing keynote. There will be a four-day development sprint before the conference. For all PyCon information: http://www.pycon.org/ See you all there! --Guido van Rossum (home page: http://www.python.org/~guido/) From pinard at iro.umontreal.ca Mon Feb 16 14:58:54 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 16 15:06:45 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: References: <4030E211.1050606@v.loewis.de> Message-ID: <20040216195854.GB9702@titan.progiciels-bpi.ca> [Tim Peters] > The docs for __hash__ still spell out the *intent*: [...] > >> What means a "mutable type" then? > It's gotten more confused. [...] I wondered, earlier today, if we are not documenting this all upside down. (Just wondering, please: I hope nobody will get upset! :-) Python cannot really enforce immutability in the general case, despite it does enforce it for some built-in types. Immutability is a fuzzy concept which represents an *intent*, as you say, and a lot of effort is spent around it in the documentation, trying to remove the fuzziness. > If a class does not define a __cmp__() method it should not > define a __hash__() operation either; if it defines __cmp__() > or __eq__() but not __hash__(), its instances will not be usable > as dictionary keys. If a class defines mutable objects and > implements a __cmp__() or __eq__() method, it should not implement > __hash__(), since the dictionary implementation requires that a key's > hash value is immutable (if the object's hash value changes, it will > be in the wrong hash bucket) Maybe the documentation should also state that `__hash__()' ought to be constant for the full duration of the life of an object. (Is this true? Martin almost says this in his reply. Is there a meaning or reason for a varying hash value for a given object?) Moreover, this might yield an interesting optimisation if not done already: an instance might compute its hash only once, either at creation time, or probably better, lazily until first needed -- but I'm getting away of what I would like to say. Things might be clearer if Python was first and clearly documenting the required properties of `hash()', and then sorting types into hashable (those having such a function) and non-hashable (those not having it). An only then, introducing immutability and mutability as useful user concepts that could be implemented by making objects hashable or not. Then, the documentation could stick to "hashable" and "not hashable", which are very precisely defined, and carefully and systematically avoid "immutable" and "mutable", which are more fuzzy. I have the impression that the net result would be that things would be clearer: tuples are hashable, lists are not; dictionaries require hashable keys. Objects having `__cmp__' (or `__eq__') loose their default hashability. Etc. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From pinard at iro.umontreal.ca Mon Feb 16 15:15:16 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 16 15:20:34 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <4030E211.1050606@v.loewis.de> References: <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> <20040216143738.GA4445@titan.progiciels-bpi.ca> <4030E211.1050606@v.loewis.de> Message-ID: <20040216201516.GA10833@titan.progiciels-bpi.ca> [Martin von L?wis] > A mutable type is one whose instances have state that can change over > the life of the object. [...] Interesting presentation. The following detail is worth stressing: > equal objects should hash equal. This should be stressed as a formal requirement for a correctly behaved hashing function. The other formal requirement being that a hash function should always compute the same value for the lifespan of an object, and that value should be a non-negative integer (not a long one?). A _desirable_ property, but not a requirement, is that two unequal objects yield different hashes. So, instead of explaining how we should cleverly use `__hash__' to describe immutability, we might describe what `__hash__' ought to be, and then merely notice that hashable objects could be useful to represent immutable objects (whether user objects or builtin objects). -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From pje at telecommunity.com Mon Feb 16 15:38:10 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon Feb 16 15:35:34 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040216201516.GA10833@titan.progiciels-bpi.ca> References: <4030E211.1050606@v.loewis.de> <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> <20040216143738.GA4445@titan.progiciels-bpi.ca> <4030E211.1050606@v.loewis.de> Message-ID: <5.1.0.14.0.20040216152908.037a5ac0@mail.telecommunity.com> At 03:15 PM 2/16/04 -0500, Fran?ois Pinard wrote: >So, instead of explaining how we should cleverly use `__hash__' to >describe immutability, we might describe what `__hash__' ought to >be, and then merely notice that hashable objects could be useful to >represent immutable objects (whether user objects or builtin objects). Actually, it might be even better to start with equality. Hashing is only meaningful for objects that can be tested for equality. That is: * By default, objects are equal only to themselves. But you can change this by overriding __cmp__ or __eq__. * Hashing is a technique used to group objects according to possible equality. So, if you want to create a hashable object that can be "equal to" other objects, it must return the same hash value as the objects it compares equal to. * Hash values must remain constant over the lifetime of an object. Thus, if an object may compare equal to a different set of objects over its lifetime, it must *not* be hashable. * Therefore, if an object is to be hashable, any attributes of the object that are used in calculating the hash (or performing comparisons) may not be changed once the its __hash__ method has been called. Usually, this is implemented by making an object completely unchangeable once created (e.g. tuples), but some user classes may implement this by "freezing" an object's contents once it is hashed (e.g. kjbuckets.kjSet). From astrand at lysator.liu.se Mon Feb 16 16:15:27 2004 From: astrand at lysator.liu.se (Peter Astrand) Date: Mon Feb 16 16:15:35 2004 Subject: [Python-Dev] Re: symtable module & globals In-Reply-To: Message-ID: > I've found the symtable module, which is quite nice. Unfortunately, I > cannot find any documentation, and the module don't work as I expect: When I run my program, which uses the symtable module, on Python 2.3, I get: -:0: SyntaxWarning: import * only allowed at module level -:0: SyntaxWarning: import * only allowed at module level -:0: SyntaxWarning: import * only allowed at module level -:0: SyntaxWarning: import * only allowed at module level In a way this is not very surprising, since I call symtable with: symtable.symtable(source, "foo", "exec") This is a big problem for me, since I need to inspect symtables on a module level. Would it be possible to make it possible for symtable to operate in a module context? Or is it safe to ignore these warnings? -- /Peter ?strand From pinard at iro.umontreal.ca Mon Feb 16 16:09:16 2004 From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard) Date: Mon Feb 16 16:39:55 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <5.1.0.14.0.20040216152908.037a5ac0@mail.telecommunity.com> References: <4030E211.1050606@v.loewis.de> <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> <20040216143738.GA4445@titan.progiciels-bpi.ca> <4030E211.1050606@v.loewis.de> <5.1.0.14.0.20040216152908.037a5ac0@mail.telecommunity.com> Message-ID: <20040216210916.GA13377@titan.progiciels-bpi.ca> [Phillip J. Eby] > [...] it might be even better to start with equality. Hashing is only > meaningful for objects that can be tested for equality. [...] Clean and clear. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From greg at cosc.canterbury.ac.nz Mon Feb 16 18:38:29 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Feb 16 18:38:51 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <200402160600.i1G60oP24381@c-24-5-134-209.client.comcast.net> Message-ID: <200402162338.i1GNcTG01653@oma.cosc.canterbury.ac.nz> Guido: > Why is it important *not* to have this property? Probably not greatly important, it just seems to me that there ought to be a fairly strong reason for taking the somewhat drastic step of making something non-subclassable. > it helps the implementation a bit Only a tiny bit, surely? Anything which accepts a boolean has to be prepared to deal with any type of object anyway. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Mon Feb 16 19:14:43 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Feb 16 19:15:00 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <4030E211.1050606@v.loewis.de> Message-ID: <200402170014.i1H0Ehb02521@oma.cosc.canterbury.ac.nz> Martin: > Of these, there are two categories of objects: > - values: > - identity objects: > > Values fall into two further categories: > - immutable values: > - mutable values: All these rules can be boiled down to: * Objects which compare equal should hash equal. * The notion of equality should not change over the lifetime of the object. There's no need to mention mutability at all. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Mon Feb 16 19:20:49 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Feb 16 19:20:56 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <20040216195854.GB9702@titan.progiciels-bpi.ca> Message-ID: <200402170020.i1H0KnW02624@oma.cosc.canterbury.ac.nz> François Pinard: > tuples are hashable, lists are not; Actually that would be "tuples are hashable if all their elements are hashable". It's still very clear, though! I like this idea. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From guido at python.org Mon Feb 16 20:07:58 2004 From: guido at python.org (Guido van Rossum) Date: Mon Feb 16 20:08:06 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: Your message of "Tue, 17 Feb 2004 13:14:43 +1300." <200402170014.i1H0Ehb02521@oma.cosc.canterbury.ac.nz> References: <200402170014.i1H0Ehb02521@oma.cosc.canterbury.ac.nz> Message-ID: <200402170107.i1H17wi26620@c-24-5-134-209.client.comcast.net> > Martin: > > > Of these, there are two categories of objects: > > - values: > > - identity objects: > > > > Values fall into two further categories: > > - immutable values: > > - mutable values: [Greg] > All these rules can be boiled down to: > > * Objects which compare equal should hash equal. > * The notion of equality should not change over the > lifetime of the object. > > There's no need to mention mutability at all. Actually, think the very definition of a mutable object may be one whose equality can change over its lifetime. As you pointed out, we should talk about (im)mutable objects, not types, because (e.g.) tuples can contain lists. --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Feb 17 02:55:12 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Feb 17 02:55:49 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <5.1.0.14.0.20040216152908.037a5ac0@mail.telecommunity.com> References: <4030E211.1050606@v.loewis.de> <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> <20040216143738.GA4445@titan.progiciels-bpi.ca> <4030E211.1050606@v.loewis.de> <5.1.0.14.0.20040216152908.037a5ac0@mail.telecommunity.com> Message-ID: <4031C8E0.6030406@v.loewis.de> Phillip J. Eby wrote: > Actually, it might be even better to start with equality. Hashing is > only meaningful for objects that can be tested for equality. However, in Python, all objects can be tested for equality. So hashing is meaningful for all objects? It is not: it is only meaningful for objects which compare equal to the same other objects over their lifespan. Regards, Martin From martin at v.loewis.de Tue Feb 17 03:03:44 2004 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Feb 17 03:04:00 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <200402170014.i1H0Ehb02521@oma.cosc.canterbury.ac.nz> References: <200402170014.i1H0Ehb02521@oma.cosc.canterbury.ac.nz> Message-ID: <4031CAE0.8030106@v.loewis.de> Greg Ewing wrote: > All these rules can be boiled down to: > > * Objects which compare equal should hash equal. > * The notion of equality should not change over the > lifetime of the object. > > There's no need to mention mutability at all. I find "notion of equality" a fuzzy term: How do I determine whether it changes? The real requirement is that the hash of an object must not change. E.g. the the state of an object may change even if it contributes to the "notion of equality" if the state of all other objects changes simultaneously in the same way. Regards, Martin From pje at telecommunity.com Tue Feb 17 09:27:43 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue Feb 17 09:25:54 2004 Subject: [Python-Dev] bool does not want to be subclassed? In-Reply-To: <4031C8E0.6030406@v.loewis.de> References: <5.1.0.14.0.20040216152908.037a5ac0@mail.telecommunity.com> <4030E211.1050606@v.loewis.de> <200402140003.i1E03tV04024@c-24-5-134-209.client.comcast.net> <20040216143738.GA4445@titan.progiciels-bpi.ca> <4030E211.1050606@v.loewis.de> <5.1.0.14.0.20040216152908.037a5ac0@mail.telecommunity.com> Message-ID: <5.1.0.14.0.20040217091808.020b1060@mail.telecommunity.com> At 08:55 AM 2/17/04 +0100, Martin v. L?wis wrote: >Phillip J. Eby wrote: >>Actually, it might be even better to start with equality. Hashing is >>only meaningful for objects that can be tested for equality. > >However, in Python, all objects can be tested for equality. So hashing >is meaningful for all objects? > >It is not: it is only meaningful for objects which compare equal to the >same other objects over their lifespan. That's not valid logic; "X is only meaningful for Y" means that X implies Y, not the other way around. The fact that Y is a tautology doesn't imply that X is true, in fact if Y is a tautology then it only proves that "X implies Y" is true, because everything implies Y. However, I see your point that it's therefore silly to talk about things that imply a tautology. :) Anyway, if you read the rest of my post, you'd see that I explained the "lifespan" issue with a bit more precision than you have stated above. For example, I pointed out that it isn't necessary for an object to compare equal to the same other objects over its lifespan, only over the lifespan following its first __hash__ invocation. From tzot at sil-tec.gr Tue Feb 17 13:01:28 2004 From: tzot at sil-tec.gr (Christos Georgiou) Date: Tue Feb 17 14:07:35 2004 Subject: [Python-Dev] Small test Message-ID: <5jl430ho8h4qra0kibbcghv0euukc138ku@4ax.com> SGI Irix 6.5.20, MIPSPro Compilers 7.4 -n32 (default 32-bit mode) no complaints, prints Thanks! -64 (64-bit mode) no complaints, prints Thanks! -- Christos Georgiou, Customer Support Engineer Silicon Technologies S.A. 41-45 Kifissias Ave, Maroussi 151 23 Greece Tel +30 (21) 06152600 (ext. 605) Fax +30 (21) 06198140 "Dave always exaggerated." --HAL From garth at garthy.com Tue Feb 17 14:35:52 2004 From: garth at garthy.com (Garth) Date: Tue Feb 17 14:32:33 2004 Subject: [Python-Dev] Compiling python with the free .NET SDK Message-ID: <40326D18.20304@garthy.com> Hi , Attached is a tool to allow you to build python using the free .NET SDK and Platform SDK (and gnu tools from msys). It will generate the makefiles from the solution file. It's as easy as. 1. Copy all files into PCBuild 2 Run "sln_read.py makemake pcbuild.sln "Debug|Win32" or "sln_read.py makemake pcbuild.sln "Release|Win32" and the make files are generated 3. run make -f pcbuild.gnu.mak and python is built The readme included expands on this. There's still quite a bit todo but the tool works and I seem to be quite busy so here it is. It could be quite easilt modified to use gcc instead of cl or some other makefile thingy Enjoy Garth -------------- next part -------------- A non-text attachment was scrubbed... Name: vcproj2make-2004-02-17.zip Type: application/x-zip-compressed Size: 10706 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040217/b34c9235/vcproj2make-2004-02-17.bin From kbk at shore.net Wed Feb 18 13:28:30 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Wed Feb 18 13:28:43 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200402181828.i1IISUoa008933@hydra.localdomain> Patch / Bug Summary ___________________ Patches : 241 open ( -5) / 2315 closed ( +5) / 2556 total ( +0) Bugs : 711 open ( +6) / 3892 closed ( +5) / 4603 total (+11) RFE : 127 open ( +0) / 118 closed ( +0) / 245 total ( +0) Patches Closed ______________ urllib2 does not allow for absolute ftp paths (2003-10-03) http://python.org/sf/817379 closed by loewis tp_dealloc, threads and garbage collection (2004-02-09) http://python.org/sf/893566 closed by loewis urllib2 doesn't support non-anonymous ftp (2003-03-29) http://python.org/sf/711838 closed by loewis Updated .spec file. (2004-02-08) http://python.org/sf/892673 closed by loewis doctest handles comments incorrectly (2003-01-15) http://python.org/sf/668500 closed by tim_one New / Reopened Bugs ___________________ pyconfig.h is not placed in --includedir (2004-02-13) http://python.org/sf/896330 opened by Göran Uddeborg time.strftime crashes python (2004-02-15) http://python.org/sf/897625 opened by Tom Lynn test_strptime failures on FC2-test (2004-02-16) http://python.org/sf/897817 opened by Anthony Baxter db4 4.2 == :-( (test_anydbm and test_bsddb3) (2004-02-16) http://python.org/sf/897820 opened by Anthony Baxter Unknown color name on HP-UX (2004-02-16) http://python.org/sf/897872 opened by Richard Townsend pty.spawn() leaks file descriptors (2004-02-16) http://python.org/sf/897935 opened by James Henstridge strftime ignores date format on winxp (2004-02-17) http://python.org/sf/898253 opened by Matthew Sherborne symtable module crashes w/ coding declaration (2004-02-16) http://python.org/sf/898271 opened by Peter Åstrand Python 2.3 encoding parsing bug (2004-02-17) http://python.org/sf/898757 opened by Edward K. Ream test_threading (2003-08-19) http://python.org/sf/791542 reopened by bcannon 1==float('nan') (2004-02-17) http://python.org/sf/899109 opened by Arman Bostani Bugs Closed ___________ Inconsistent behavior of compressing the BZip2 format (2003-12-03) http://python.org/sf/853061 closed by niemeyer Unicode problem in os.path.getsize ? (2004-02-13) http://python.org/sf/896236 closed by loewis Core dump problems (2003-12-15) http://python.org/sf/860547 closed by rhettinger FAQ 2.3 slices not used (2004-02-02) http://python.org/sf/888829 closed by loewis test_threading (2003-08-19) http://python.org/sf/791542 closed by sleard1 Python 2.3 encoding parsing bug (2004-02-17) http://python.org/sf/898757 closed by loewis From jeremy at alum.mit.edu Wed Feb 18 14:54:53 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Wed Feb 18 14:58:33 2004 Subject: [Python-Dev] logging exceptions Message-ID: <1077134092.5317.65.camel@localhost.localdomain> Zope 2.7 is using the logging package internally as the implementation for its old zLOG API. We've run into one problem, probably caused by carelessness on our part. zLOG allows you to pass exc_info to be logged, but logging only allows you to specify a flag saying whether the current exception should be logged. (I recall that this was discussed at length, but not why the current solution was reached.) There are several call sites that depend on this feature to capture an exception, try to recover gracefully or log a traceback if recovery is impossible. So we depend on this feature, but can't get it out of logging very easily. We've come up with this as a work-around: prev = sys.exc_info() if prev != exc_info: try: raise exc_info[0], exc_info[1], exc_info[2] except: pass else: prev = None logging.error(..., exc_info=1) if prev is not None: try: raise prev[0], prev[1], prev[2] except: pass This seems like a pretty gross way to set and restore the current exception, but it would work. Would it be possible to add a feature to logging to allow us to specify the traceback directly? The only reason we need to set the current exception is this code in logging: def _log(self, level, msg, args, exc_info=None): # ... if exc_info: exc_info = sys.exc_info() record = self.makeRecord(self.name, level, fn, lno, msg, args, exc_info) self.handle(record) I suppose another option is to create a Logger subclass with a different _log() method. Jeremy From laorny at tiggy.com Tue Feb 17 16:51:21 2004 From: laorny at tiggy.com (laorny@tiggy.com) Date: Wed Feb 18 17:02:57 2004 Subject: [Python-Dev] Re: Your Valium refill is ready "Date Number"096 Message-ID: <881837212287511.21TOG@hotmail.com> New Super Viagra is here! (CIALIS) VALIUM & XANAX BACK IN STOCK! NO MEDICAL QUESTIONS ASKED EVER! SIMPLY PAY & RECEIVE! Private & Discrete on all levels of service. *36 hours of better sex *Longer lasting Sex *Potential for increase in penis size. More pleasure for the female by up to 4x's. *Plus we have Valium, Xanax, & MORE! *As Seen On TV We are a licensed distributor of the product and offer top quality at the lowest pirces! http://hotmail.com.ecliptic.sdxd3e.com/p6 This communication is privileged and contains confidential information intended only for the person(s) to whom it is addressed. Any unauthorized disclosure, copying, other distribution of this communication or taking any action on its contents is strictly prohibited. If you have received this message in error, please notify us immediately OR remove yourself from our list if there is no interest in regards to our services. http://hotmail.com.beset.remove_pharm3736.com/a.html typo optoacousticfraternalbeginner veal From miki.tebeka at zoran.com Wed Feb 18 08:07:13 2004 From: miki.tebeka at zoran.com (Miki Tebeka) Date: Wed Feb 18 17:03:01 2004 Subject: [Python-Dev] Enhancing `iter' Message-ID: <20040218130713.GN604@zoran.com> Hello All, Currently `iter' 2'nd parameter is used for comparison. I'd like to have the ability to give a function that will determine weather to stop or not. iter(range(10), lambda x: x > 7) will stop at 8. (There reverse logic might be uses as well: continue as long a `sentinal' returns True.) The only problem I can see is that someone might want to use a function object as a sentinal, then `iter' won't know if to call the function or to compare. However this seems very rare and can be solved by: def f(x, y): .... iter(something, lambda x: x == f) Any thoughts/objections? Bye. -- ------------------------------------------------------------------------- Miki Tebeka http://www.cs.bgu.ac.il/~tebeka The only difference between children and adults is the price of the toys. From hbl at st-andrews.ac.uk Wed Feb 18 12:51:41 2004 From: hbl at st-andrews.ac.uk (Hamish Lawson) Date: Wed Feb 18 17:03:04 2004 Subject: [Python-Dev] Suggestion: URL splitter that returns host, port, request URL Message-ID: <4033A62D.4050007@st-andrews.ac.uk> The HTTP clients provided by many libraries (httplib in the Standard Library, asynchttp, Twisted) do not deal with absolute URLs but instead with the three component parts of host, port, and request URL. However these are not the parts returned by either urlparse.urlsplit or urlparse.urlparse. I therefore suggest it might be useful to have in the Standard Library a function that splits an absolute URL into host, port, and request URL. So for the following URL: "http://www.python.org/cgi-bin/moinmoin/FrontPage?action=edit" the function would return ("www.python.org", 80, "/cgi-bin/moinmoin/FrontPage?action=edit") An optional parameter could specify which port to use as default. Hamish Lawson From bob at redivi.com Wed Feb 18 17:38:19 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Feb 18 17:35:40 2004 Subject: [Python-Dev] Enhancing `iter' In-Reply-To: <20040218130713.GN604@zoran.com> References: <20040218130713.GN604@zoran.com> Message-ID: <26939540-6263-11D8-8F69-000A95686CD8@redivi.com> On Feb 18, 2004, at 8:07 AM, Miki Tebeka wrote: > Currently `iter' 2'nd parameter is used for comparison. > I'd like to have the ability to give a function that will determine > weather > to stop or not. > > iter(range(10), lambda x: x > 7) > will stop at 8. > (There reverse logic might be uses as well: continue as long a > `sentinal' > returns True.) > > The only problem I can see is that someone might want to use a function > object as a sentinal, then `iter' won't know if to call the function > or to > compare. However this seems very rare and can be solved by: > def f(x, y): > .... > iter(something, lambda x: x == f) This is not necessary. >>> import itertools >>> list(itertools.takewhile(lambda x: x < 8, range(10))) [0, 1, 2, 3, 4, 5, 6, 7] (among the numerous other ways to say it with itertools .. or even generator comprehensions in Python 2.4) -bob From jepler at unpythonic.net Wed Feb 18 18:12:00 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Wed Feb 18 18:12:34 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <200402181828.i1IISUoa008933@hydra.localdomain> References: <200402181828.i1IISUoa008933@hydra.localdomain> Message-ID: <20040218231157.GD12835@unpythonic.net> [I apologize that I'm not adding this information to the bug report, but I'm not able to log into SF right now, the login page hangs while loading] > 1==float('nan') (2004-02-17) > http://python.org/sf/899109 opened by Arman Bostani I'm sure Tim can explain this better, but what happens is this: 1 is coerced to float for comparison. Then, the following C expression is evaluated (Objects/floatobject.c:float_compare): return (i < j) ? -1 : (i > j) ? 1 : 0; Because NaN is "unordered", 1 #include int main(void) { double a=1.0, b = atof("nan"); printf("a a) ? 1 : 0); return 0; } $ gcc -Wall compare.c && ./a.out nan 1.0 a Message-ID: <200402190020.i1J0KYC04180@oma.cosc.canterbury.ac.nz> Miki Tebeka : > Currently `iter' 2'nd parameter is used for comparison. > I'd like to have the ability to give a function that will determine weather > to stop or not. I think there should be a different function for this, rather than overloading it onto iter() by type-testing the second argument, which is rather un-Pythonic. BTW, I never knew iter() had a second argument. Having just looked up what it means, I think it was a mistake to put even the existing two functionalities into one] function -- especially given the comment in the docs about it doing "very different things depending on the presence of the second argument"! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From aahz at pythoncraft.com Wed Feb 18 21:11:33 2004 From: aahz at pythoncraft.com (Aahz) Date: Wed Feb 18 21:11:41 2004 Subject: [Python-Dev] Suggestion: URL splitter that returns host, port, request URL In-Reply-To: <4033A62D.4050007@st-andrews.ac.uk> References: <4033A62D.4050007@st-andrews.ac.uk> Message-ID: <20040219021133.GA10672@panix.com> On Wed, Feb 18, 2004, Hamish Lawson wrote: > > The HTTP clients provided by many libraries (httplib in the Standard > Library, asynchttp, Twisted) do not deal with absolute URLs but instead > with the three component parts of host, port, and request URL. However > these are not the parts returned by either urlparse.urlsplit or > urlparse.urlparse. I therefore suggest it might be useful to have in the > Standard Library a function that splits an absolute URL into host, port, > and request URL. So for the following URL: > > "http://www.python.org/cgi-bin/moinmoin/FrontPage?action=edit" > > the function would return > > ("www.python.org", 80, "/cgi-bin/moinmoin/FrontPage?action=edit") > > An optional parameter could specify which port to use as default. Sounds good; you'll need to post a patch on SF to make progress. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach From bob at redivi.com Wed Feb 18 21:26:33 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Feb 18 21:23:27 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) Message-ID: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Some time ago, mwh developed a patch that adds some syntactical sugar to def, which is equivalent to PEP 318 though it has a different and more flexible syntax... previous threads can be easily found here: http://www.google.com/search?q=+site:mail.python.org+%22meth-syntax- sugar%22 the latest version of mwh's patch is here: http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff Here's a quick overview: def foo(args) [sugary, expressions, list]: pass This is equivalent to: def foo(args): pass foo = list(expressions(sugary(foo))) This evaluation order is Guido approved, though at least one person wanted it to be the other way around One would use this in scenarios such as: class FooClass(object): def className(klass) [classmethod]: return klass.__name__ or, even more importantly (to me anyway): # we would change PyObjC to make this a built-in feature.. but, for completeness: import objc def signature(sig): def _signature(fn): return objc.selector(fn, signature=sig) return _signature class FooClass(NSObject): def returnsAnInteger(self) [signature('i@:')]: return 1 def returnsVoidTakesAnInteger_(self, anInteger) [signature('v@:i')]: pass With current syntax, PyObjC is extremely cumbersome: class FooClass(NSObject): def returnsAnInteger(self): return 1 returnsAnInteger = objc.selector(returnsAnInteger, signature='i@:') def returnsVoidTakesAnInteger_(self, anInteger): pass returnsVoidTakesAnInteger_ = objc.selector(returnsVoidTakesAnInteger_, signature='v@:i') # these are actually short examples, compare to something like: # textView_completions_forPartialWordRange_indexOfSelectedItem_ Why we need this: Without it, it's hard use PyObjC correctly. ObjC selector nomenclature is extremely verbose, and your fingers hurt after a while having to type each function name three times. The function __name__ is important, because the objc.selector type has to convert it to a selector:that:uses:colons:, or else the colon:using:name: would have to be specified manually. It makes classmethod/staticmethod/etc more palatable. What the patch doesn't do: lambda is not allowed in the "sugary expressions list" there's no *expansion and it won't take an actual list so if you want a prebaked list of transformations then you'll have to roll a callable that does it such as: def prebake(*expressions): def _prebake(fn): for transformation in expressions: fn = transformation(fn) return fn return fn This syntactical sugar for def is so unbelievably important to PyObjC (and likely other projects) that I am willing to distribute my own modified version of Python if it doesn't make 2.4 (though I would probably use Stackless as the base, but that's another plea for another day). The patch looks like it still applies to CVS HEAD, but there is a little fuzz on hunk 2. I have not tested it yet, but I have tested it against CVS HEAD of the 2.3.3 based Stackless. I'm willing to help however I can in order to get this into Python 2.4. -bob From neal at metaslash.com Wed Feb 18 21:24:41 2004 From: neal at metaslash.com (Neal Norwitz) Date: Wed Feb 18 21:24:51 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects listobject.c, 2.182, 2.183 In-Reply-To: References: Message-ID: <20040219022440.GW1444@epoch.metaslash.com> On Tue, Feb 17, 2004 at 03:36:19AM -0800, rhettinger@users.sourceforge.net wrote: > { > ! PyObject *v; > ! if (!PyArg_ParseTuple(args, "|i:pop", &i)) > return NULL; > --- 773,788 ---- > { > ! PyObject *v, *arg = NULL; > ! > ! if (!PyArg_UnpackTuple(args, "pop", 0, 1, &arg)) > return NULL; > + if (arg != NULL) { > + if (PyInt_Check(arg)) > + i = (int)(PyInt_AS_LONG((PyIntObject*) arg)); > + else { > + PyErr_SetString(PyExc_TypeError, "an integer is required"); > + return NULL; > + } > + } This is a change in behaviour. This used to work: >>> x = [1,2,3] >>> x.pop(1L) Traceback (most recent call last): File "", line 1, in ? TypeError: an integer is required From tim.one at comcast.net Wed Feb 18 21:35:18 2004 From: tim.one at comcast.net (Tim Peters) Date: Wed Feb 18 21:35:25 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <20040218231157.GD12835@unpythonic.net> Message-ID: [Jeff Epler] > [I apologize that I'm not adding this information to the bug report, > but I'm not able to log into SF right now, the login page hangs while > loading] > > 1==float('nan') (2004-02-17) > http://python.org/sf/899109 opened by Arman Bostani > > I'm sure Tim can explain this better, No, but I can give a simpler explanation: it's all accidents, and can (and does) vary across platforms. To start with, that float('nan') didn't raise an exception is an accident already. On Windows, e.g., it does raise an exception. > but what happens is this: 1 is coerced to float for comparison. Then, > the following C expression is evaluated > Objects/floatobject.c:float_compare): > return (i < j) ? -1 : (i > j) ? 1 : 0; Provided you get that far, yes. > Because NaN is "unordered", 1 and so python returns 0 from float_compare. That is one possible outcome . Here on 2.3.3, but on Windows: >>> inf = 1e500 >>> inf 1.#INF >>> nan = inf-inf >>> nan -1.#IND >>> cmp(1, nan) -1 >>> cmp(nan, 1) -1 >>> 1 == nan False >>> nan == nan False >>> nan < nan True >>> 1 < nan True >>> nan < 1 True >>> inf < nan True >>> nan < inf True >>> That's under MSVC 6. I know an earlier version of the MS compiler gave different results. BTW, non-accidental support for IEEE-754 oddballs (infs, NaNs, signed zeroes) is a wishlist item in PEP 42. Come next year, that standard will be 20 years old -- yet basically unusable despite near-universal HW support because languages just won't play along (in fairness, "playing along" in SW isn't easy, and nobody will pay for it -- how the HW manufacturers got roped into it remains something of a mystery to me, cuz it's not cheap or easy in HW either). From jepler at unpythonic.net Wed Feb 18 21:46:56 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Wed Feb 18 21:47:15 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: References: <20040218231157.GD12835@unpythonic.net> Message-ID: <20040219024656.GF11071@unpythonic.net> Tim, Thanks for reminding me that this all varies from machine to machine. I guess that what I was trying to say in my message is this: On my platform, I get the same result. Here's why. The term "unordered" came from me reading an x86 architecture reference and trying to use the same words the grown-ups use. It's too bad you can get infinity and nan other than by using float('os-specific mumbo-jumbo'), because if that weren't the case we could just force Python's syntax for floating-point literals on the argument to float(), never passing it to the platform atof() if it doesn't conform. That would, uh, completely solve all problems python programmers ever encounter with floats. Jeff From neal at metaslash.com Wed Feb 18 21:55:03 2004 From: neal at metaslash.com (Neal Norwitz) Date: Wed Feb 18 21:55:10 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/lib-tk Tkinter.py, 1.178, 1.179 In-Reply-To: References: Message-ID: <20040219025502.GX1444@epoch.metaslash.com> David, Thanks for adding a test for _tkinter.c. It's a really good start. Unfortunately, testPackageRequire() fails for me because I don't have Tclx. Is there any way to make the test more robust? I also have a question about the implementation. It looks like useTK is required to be an integer. Is that correct? I would have expected to be able to use any boolean value. Neal -- On Tue, Feb 17, 2004 at 09:59:55PM -0800, david_ascher@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Lib/lib-tk > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24617/dist/src/Lib/lib-tk > > Modified Files: > Tkinter.py > Log Message: > Implementation of patch 869468 > > Allow the user to create Tkinter.Tcl objects which are > just like Tkinter.Tk objects except that they do not > initialize Tk. This is useful in circumstances where the > script is being run on machines that do not have an X > server running -- in those cases, Tk initialization fails, > even if no window is ever created. > > Includes documentation change and tests. > > Tested on Linux, Solaris and Windows. > > Reviewed by Martin von Loewis. > > > Index: Tkinter.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/lib-tk/Tkinter.py,v > retrieving revision 1.178 > retrieving revision 1.179 > diff -C2 -d -r1.178 -r1.179 > *** Tkinter.py 12 Feb 2004 17:35:09 -0000 1.178 > --- Tkinter.py 18 Feb 2004 05:59:53 -0000 1.179 > *************** > *** 1547,1551 **** > of an appliation. It has an associated Tcl interpreter.""" > _w = '.' > ! def __init__(self, screenName=None, baseName=None, className='Tk'): > """Return a new Toplevel widget on screen SCREENNAME. A new Tcl interpreter will > be created. BASENAME will be used for the identification of the profile file (see > --- 1547,1551 ---- > of an appliation. It has an associated Tcl interpreter.""" > _w = '.' > ! def __init__(self, screenName=None, baseName=None, className='Tk', useTk=1): > """Return a new Toplevel widget on screen SCREENNAME. A new Tcl interpreter will > be created. BASENAME will be used for the identification of the profile file (see > *************** > *** 1553,1559 **** > It is constructed from sys.argv[0] without extensions if None is given. CLASSNAME > is the name of the widget class.""" > - global _default_root > self.master = None > self.children = {} > if baseName is None: > import sys, os > --- 1553,1562 ---- > It is constructed from sys.argv[0] without extensions if None is given. CLASSNAME > is the name of the widget class.""" > self.master = None > self.children = {} > + self._tkloaded = 0 > + # to avoid recursions in the getattr code in case of failure, we > + # ensure that self.tk is always _something_. > + self.tk = None > if baseName is None: > import sys, os > *************** > *** 1562,1567 **** > if ext not in ('.py', '.pyc', '.pyo'): > baseName = baseName + ext > ! self.tk = _tkinter.create(screenName, baseName, className) > ! self.tk.wantobjects(wantobjects) > if _MacOS and hasattr(_MacOS, 'SchedParams'): > # Disable event scanning except for Command-Period > --- 1565,1580 ---- > if ext not in ('.py', '.pyc', '.pyo'): > baseName = baseName + ext > ! interactive = 0 > ! self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk) > ! if useTk: > ! self._loadtk() > ! self.readprofile(baseName, className) > ! def loadtk(self): > ! if not self._tkloaded: > ! self.tk.loadtk() > ! self._loadtk() > ! def _loadtk(self): > ! self._tkloaded = 1 > ! global _default_root > if _MacOS and hasattr(_MacOS, 'SchedParams'): > # Disable event scanning except for Command-Period > *************** > *** 1588,1592 **** > self.tk.createcommand('tkerror', _tkerror) > self.tk.createcommand('exit', _exit) > - self.readprofile(baseName, className) > if _support_default_root and not _default_root: > _default_root = self > --- 1601,1604 ---- > *************** > *** 1630,1633 **** > --- 1642,1654 ---- > sys.last_traceback = tb > traceback.print_exception(exc, val, tb) > + def __getattr__(self, attr): > + "Delegate attribute access to the interpreter object" > + return getattr(self.tk, attr) > + def __hasattr__(self, attr): > + "Delegate attribute access to the interpreter object" > + return hasattr(self.tk, attr) > + def __delattr__(self, attr): > + "Delegate attribute access to the interpreter object" > + return delattr(self.tk, attr) > > # Ideally, the classes Pack, Place and Grid disappear, the > *************** > *** 1645,1648 **** > --- 1666,1673 ---- > # copied into the Pack, Place or Grid class. > > + > + def Tcl(screenName=None, baseName=None, className='Tk', useTk=0): > + return Tk(screenName, baseName, className, useTk) > + > class Pack: > """Geometry manager Pack. > > > _______________________________________________ > Python-checkins mailing list > Python-checkins@python.org > http://mail.python.org/mailman/listinfo/python-checkins From tim.one at comcast.net Wed Feb 18 22:28:51 2004 From: tim.one at comcast.net (Tim Peters) Date: Wed Feb 18 22:28:58 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <20040219024656.GF11071@unpythonic.net> Message-ID: [Jeff Epler] > Thanks for reminding me that this all varies from machine to machine. > I guess that what I was trying to say in my message is this: > > On my platform, I get the same result. Here's why. > > The term "unordered" came from me reading an x86 architecture > reference and trying to use the same words the grown-ups use. That manual got "unordered" from the IEEE-754 floating point standard, and it's a fine word to use. The problem is that the 754 standard (which also goes under a number of different names now) has no defined relationship to the C89 standard, so when talking about C code it doesn't matter at all what the 754 standard says: "unordered" just isn't a C89 concept. 754 does have a defined relationship to the newer C99 standard. > It's too bad you can get infinity and nan other than by using > float('os-specific mumbo-jumbo'), because if that weren't the case we > could just force Python's syntax for floating-point literals on the > argument to float(), never passing it to the platform atof() if it > doesn't conform. That would, uh, completely solve all problems python > programmers ever encounter with floats. It would be easier to just remove floats from Python for 2.4 . From DavidA at activestate.com Thu Feb 19 00:05:53 2004 From: DavidA at activestate.com (David Ascher) Date: Thu Feb 19 00:10:37 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/lib-tk Tkinter.py, 1.178, 1.179 In-Reply-To: <20040219025502.GX1444@epoch.metaslash.com> References: <20040219025502.GX1444@epoch.metaslash.com> Message-ID: <40344431.3000904@ActiveState.com> Neal Norwitz wrote: > David, > > Thanks for adding a test for _tkinter.c. It's a really good start. > Unfortunately, testPackageRequire() fails for me because I don't have > Tclx. Is there any way to make the test more robust? Will investigate. (Thanks for the other fixes btw). > I also have a question about the implementation. It looks like > useTK is required to be an integer. Is that correct? I would have > expected to be able to use any boolean value. Probably. Will fix. Thanks! --david From bac at OCF.Berkeley.EDU Thu Feb 19 01:06:08 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Feb 19 01:06:22 2004 Subject: [Python-Dev] python-dev Summary for 2004-01-01 through 2004-01-31 [rough draft] Message-ID: <40345250.90704@ocf.berkeley.edu> As usual, reply to this email with any corrections and such. Plan on getting this out this weekend. As for why this is late, read the Summary Announcements (basically illness and just really busy). And just so I can say I did everything I could, I will repeat myself here as I did in the Summary Announcements and mention I am looking for a summer job/internship. Any leads on any would be greatly appreciated. More details can be found in the Summary Announcements. ------------------- python-dev Summary for 2004-01-01 through 2004-01-31 ++++++++++++++++++++++++++++++++++++++++++++++++++++ This is a summary of traffic on the `python-dev mailing list`_ from January 1, 2004 through January 31, 2004. It is intended to inform the wider Python community of on-going developments on the list. To comment on anything mentioned here, just post to `comp.lang.python`_ (or email python-list@python.org which is a gateway to the newsgroup) with a subject line mentioning what you are discussing. All python-dev members are interested in seeing ideas discussed by the community, so don't hesitate to take a stance on something. And if all of this really interests you then get involved and join `python-dev`_! This is the thirty-third and -fourth summaries written by Brett Cannon (who is rather fed up with being sick recently). To contact me, please send email to brett at python.org ; I do not have the time to keep up on comp.lang.python and thus do not always catch follow-ups posted there. All summaries are archived at http://www.python.org/dev/summary/ . Please note that this summary is written using reStructuredText_ which can be found at http://docutils.sf.net/rst.html . Any unfamiliar punctuation is probably markup for reST_ (otherwise it is probably regular expression syntax or a typo =); you can safely ignore it, although I suggest learning reST; it's simple and is accepted for `PEP markup`_ and gives some perks for the HTML output. Also, because of the wonders of programs that like to reformat text, I cannot guarantee you will be able to run the text version of this summary through Docutils_ as-is unless it is from the original text file. .. _PEP Markup: http://www.python.org/peps/pep-0012.html The in-development version of the documentation for Python can be found at http://www.python.org/dev/doc/devel/ and should be used when looking up any documentation on something mentioned here. PEPs (Python Enhancement Proposals) are located at http://www.python.org/peps/ . To view files in the Python CVS online, go to http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . Reported bugs and suggested patches can be found at the SourceForge_ project page. The `Python Software Foundation`_ is the non-profit organization that holds the intellectual property for Python. It also tries to forward the development and use of Python. But the PSF cannot do this without donations. You can make a donation at http://python.org/psf/donations.html . Every penny helps so even a small donation (you can donate through PayPal or by check) helps. .. _python-dev: http://www.python.org/dev/ .. _SourceForge: http://sourceforge.net/tracker/?group_id=5470 .. _python-dev mailing list: http://mail.python.org/mailman/listinfo/python-dev .. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python .. _Docutils: http://docutils.sf.net/ .. _reST: .. _reStructuredText: http://docutils.sf.net/rst.html .. _Python Software Foundation: http://python.org/psf/ .. contents:: .. _last summary: http://www.python.org/dev/summary/2003-12-01_2003-12-31.html ===================== Summary Announcements ===================== Quick personal announcement/plea: I am looking for a summer job/internship. If you happen to have a position at your company or happen to know of one somewhere **please** let me know at brett@python.org . I obviously know Python rather well, but I do know other languages. I am quite willing to send you my resume and answer any questions you might have about experience, etc. I would truly appreciate any help anyone can provide me in finding something for the summer. I do have housing handled in the Los Angeles area and the Seattle/Bellevue area on top of the San Luis Obispo area, although I am willing to relocate for the summer. Sorry about the lateness of this summary. I realize this is going out when another one is already due. Unfortunately, on top of needing to secure a summer job/internship and trying to get that dealt with, I also got strep throat that was rather horrid. It was not fun to only be officially over pneumonia for a couple of weeks before I got sick again. Rather frustrating since I haven't been this ill so often since I was a little kid. And having to deal with school on top of being sick did not help matters. In actual Python-related news, PyCon's early-bird registration cut-off date has been extended to February 22nd! If you have not registered, go to http://pycon.org/dc2004/register/ and do so now! Last year was a blast and I expect no less from this year (I am actually spending my Spring Break at PyCon and I am quite happy to do so I enjoyed myself so much last year). Papers have been selected and it looks like we are going to have a great line-up of talks that cover the gamut of interests. In other words I highly recommend attending if you can. And even if you can't attend, if you are in the D.C. area, consider at least attending the first two days of the sprints which are the weekend preceding the conference (March 20-21). More info on the sprints can be found at http://www.python.org/cgi-bin/moinmoin/SprintPlan2004 . And now, on with the juicy stuff... ========= Summaries ========= --------------------- Benchmarking is a go! --------------------- After Guido committed the parrotbench code to CVS, various people began to run the benchmark and post their scores. If you are curious to see how your machine compares take a look at the contributing threads. Contributing threadds: - `Pie-thon benchmark code ready `__ - `Re: Are we collecting benchmark results across machines `__ ----------------------------------- Windows compilation moved to VC.NET ----------------------------------- Martin v. L??wis converted the project files and anything else needed so that Python can now be compiled using VC.NET . All of the previous files needed for compiling under VC 6 has been moved to PC/VC6 . Contributing threads: - `Switch to VC.NET 7.1 completed `__ -------------------------- Shutting Unicode errors up -------------------------- Enough people have asked for the ability to silence Unicode errors that Martin v. L??wis suggested adding a module attribute in sys to specify whether Unicode errors should be ignored or not. M.A. Lemburg, though, thought it was a bad idea and suggested instead that people use a custom codec that relaxed the error-handling. the thread seemed to end with no changes planned. Contributing threads: ` Relaxing Unicode error handling `__ --------------------------------------------------------------------------------------- new PEPs (sorry, nothing witty came to me about PEPs that was not excruciatingly corny) --------------------------------------------------------------------------------------- In all cases, ignore the Created date in terms of posting date; click on the Last Modified link if you care to see when the PEP was officially added. `PEP 324`_ proposes a new module named popen5. As suggested by the name, it hopes to come up with a unified module for process creation. To see Guido's initial response to the PEP and what he thinks is a "category killer" and why any new additions to the stdlib must be one, read http://mail.python.org/pipermail/python-dev/2004-January/041698.html . `PEP 325`_ proposes a 'close' method for generators so as to handle what normally would be handled in a try/finally block (such as closing files and such). `PEP 326`_ wants to add maximum and minimum singletons. `PEP 327`_ is the long-awaited decimal type PEP that has been in the process of being refined for quite a while. Development in the sandbox has been moving forward. `PEP 328`_ is the 'import' PEP in terms of multi-line imports and dealing with relative imports. .. _PEP 324: http://python.org/peps/pep-0324.html .. _PEP 325: http://python.org/peps/pep-0325.html .. _PEP 326: http://python.org/peps/pep-0326.html .. _PEP 327: http://python.org/peps/pep-0327.html .. _PEP 328: http://python.org/peps/pep-0328.html Contributing threads: - `PEP 324: popen5 - New POSIX process module `__ - `New version of installer `__ - `prePEP "Decimal data type" v0.2 `__ - `PEP 326 (quick location possibility) `__ - `PEP 327: Decimal Data Type `__ ------------------------------------- Making Python and HP-UX friends again ------------------------------------- Cameron Laird said he wanted to try to spear-head an attempt to get Python to compiled on HP-UX correctly. In the past threading support has been out of the question and getting Tkinter to go has been an issue as well along with curses. Cameron said he would like to get Python working for HP-UX as far back as version 10.20, although he said he would also be fine with the current 11.x versions. Most of it seems to have to do with requiring tweaks to Configure.in, Modules/Setup, and setup.py . As of this exact second no patches have been added to SourceForge_. You can get a version of Python 2.3.3 compiled for HP-UX at http://hpux.cs.utah.edu/hppd/hpux/Languages/python-2.3.3/ . This also led to a slight discussion of how slightly crufty Configure.in is. Any help bringing it up-to-date or cleaning up would be appreciated. Contributing threads: - `HP-UX clean-up `__ ---------------------------- msi installer for Python 2.4 ---------------------------- Going for the record of the person mentioned the largest number times in a single Summary who is not labeled a BDFL, Martin v. L??wis updated his msi installer for Python 2.4 . It can be found at http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.12421.msi along with info on the installer at http://www.dcl.hpi.uni-potsdam.de/home/loewis/msipackage.html . Contributing threads: - `New version of installer `__ - `Python MSI `__ ---------------------- New collections module ---------------------- Raymond Hettinger has created a new module called 'collections'. It primary purpose is to be home to "high-performance container datatypes". To start, it only has an implementation of deques. Contributing threads: - `collections module `__ - `collections module (correction) `__ - `Collections `__ ---------------- CJKCodecs in 2.4 ---------------- A perk of having Hye-Shik Chang an official developer of Python is getting his CJKCodecs integrated into Python 2.4 . Contributing threads: - `CJKCodecs integration into Python `__ - `Oodles of warnings in cjkcodecs `__ --------------- Statistics talk --------------- Raymond Hettinger asked for some suggestions on a statistics module he was working on. A whole bunch of stuff from names to algorithms to include came up. There is now a module in the sandbox being worked on to implement what was discussed. Contributing threads: - `Accumulation module `__ ------------------------------------ Installer problems for MacPython-OS9 ------------------------------------ Jack Jansen told python-dev how he has not been able to distribute a version of MacPython-OS9 for Python 2.3.3 because the free license for the installer expired. It was suggested that the PSF_ pay for a license. If you think this is a good idea and you are a PSF member consider bringing the topic up at the general meeting to be held at PyCon_. Contributing threads: - `No more releases of MacPython-OS9? `__ -------------------------------------- Anyone want to maintain MacPython-OS9? -------------------------------------- If anyone would like to take over maintenance of MacPython-OS9 so that it continues to exist beyond the 2.3 branch then contact Jack Jansen. Contributing threads: - `PEP 11 mistake? `__ -------------- Python in 2003 -------------- Go to http://python.org/topics/2003.html to read AM Kuchling's report on what happened to Python in 2003. Contributing threads: - `Python in 2003 summary `__ --------------------------------- Compiling using the free .NET SDK --------------------------------- If you wish to compile Python using the free .NET SDK then read the first email in the contributing thread thanks to someone named Garth. Contributing threads: - `Compiling python with the free .NET SDK `__ -------------------------- No dict.addlist() for you! -------------------------- The idea of adding a method to dict called 'addlist(k,v)' that would act like ``d.setdefault(k, []).append(v)``. It was eventually agreed upon that the perk of having it as a part of dict was not justified since the idiom it was implementing is simple enough so as to not really require adding the method. Contributing threads: - `dict.addlist() `__ -------------------------------------------------------- Want to be able to change Python's name at compile time? -------------------------------------------------------- Jack Jansen asked what people thought of adding the ability of being able to specify the name of Python through 'configure' and have the permeate throughout the code. The main reason for this is to allow people on OS X to be able to have other framework builds of Python that don't interfere with the installed version on OS X 10.3 . The thread ended with no definitive decision as to whether to move ahead or not. Contributing threads: - `Configure option to change Python's name? `__ --------------------------------- You think you are a Hotshot, huh? --------------------------------- Sparked by a question about where the development of the Hotshot profiler was, a new file was added to the scripts directory for making it easier to launch Hotshot for profiling. Contributing threads: - `Hotshot `__ ------------------------------------------------- Happy patch/bug statistics days are back again... ------------------------------------------------- Kurt Kaiser has taken over for Skip Montanaro (thanks to both for continuing and starting this weekly email, respectively) for sending out the "Weekly Python Patch/Bug Summary" email. Contributing threads: - `Weekly Python Patch/Bug Summary `__ --------------------------------------------------- Change in posting rules to python-dev (but barely!) --------------------------------------------------- The posting rules to python-dev have changed so that emails from non-subscribers must be ok'ed by moderators. But joining python-dev is still open to the public and requires no clearance from anyone so it is not a big hurdle to overcome to post to the list. Contributing threads: - `Proposed: change to posting rules for python-dev `__ ----------------------------------- GPL code in Python core is not okay ----------------------------------- Although the question was specific to python-mode.el, the general answer is that GPL code is not allowed into Python. The clash of the GPL with the PSF license since the latter is BSD-style. Contributing threads: - `Would GPL on python-mode.el be a problem? `__ From jcarlson at uci.edu Thu Feb 19 02:00:52 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Feb 19 02:04:24 2004 Subject: [Python-Dev] python-dev Summary for 2004-01-01 through 2004-01-31 [rough draft] In-Reply-To: <40345250.90704@ocf.berkeley.edu> References: <40345250.90704@ocf.berkeley.edu> Message-ID: <20040218225423.87B0.JCARLSON@uci.edu> > `PEP 326`_ wants to add maximum and minimum singletons. > .. _PEP 326: http://python.org/peps/pep-0326.html > - `PEP 326 (quick location possibility) > `__ I don't know if you want to mention the fact that it was rejected by Guido. I'm mostly done with the sample module and the last version of the PEP (as I said I would do in this post: http://mail.python.org/pipermail/python-dev/2004-January/042300.html ), but I've been very busy with a Randomized Algorithms course this quarter, and haven't had time (you would understand if you read the homework problems http://www.ics.uci.edu/~bagchi/ICS280/ ). If I get some time this weekend, I'll post the final PEP with a module that I suggest supporters use. Thanks, - Josiah From pf_moore at yahoo.co.uk Thu Feb 19 05:42:02 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Thu Feb 19 05:42:13 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: Bob Ippolito writes: > Some time ago, mwh developed a patch that adds some syntactical sugar > to def, which is equivalent to PEP 318 though it has a different and > more flexible syntax... [...] > I'm willing to help however I can in order to get this into Python 2.4. I like Michael's patch (or rather the functionality/syntax it offers), but I'd have to admit that it's for purely theoretical "that's nice" reasons - I don't have any use cases to add to yours. I'm not entirely sure why acceptance got stalled the way it did. While I don't recall wholehearted enthusiasm, nor do I recall any other proposal getting significant support. My impression was that it ended up left hanging "because there might be something better that we're missing". The fact that nothing else has been proposed since argues either that no-one is interested enough, or that there isn't likely to be a better option found. Either way, it's probably time to reopen the issue and decide one way or the other. As well as the question of whether anyone is interested, there was also a question as to whether it is useful enough to justify the language change. Your use cases may address these issues to an extent. Finally, there was an issue with the fact that the syntax didn't handle properties, but personally, I don't see that as a major issue - properties are just so different (combining 3 separate functions) that I wouldn't expect the same syntax to help in both cases. I'd like to see this resolved one way or another, and I'm also willing to help - but I'm not sure what needs doing next. Paul. -- This signature intentionally left blank From mwh at python.net Thu Feb 19 05:54:37 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 05:54:41 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <20040218231157.GD12835@unpythonic.net> (Jeff Epler's message of "Wed, 18 Feb 2004 17:12:00 -0600") References: <200402181828.i1IISUoa008933@hydra.localdomain> <20040218231157.GD12835@unpythonic.net> Message-ID: <2mr7wrxrte.fsf@starship.python.net> Jeff Epler writes: > [I apologize that I'm not adding this information to the bug report, but > I'm not able to log into SF right now, the login page hangs while loading] > >> 1==float('nan') (2004-02-17) >> http://python.org/sf/899109 opened by Arman Bostani > > I'm sure Tim can explain this better, but what happens is this: 1 is > coerced to float for comparison. Then, the following C expression is > evaluated (Objects/floatobject.c:float_compare): > return (i < j) ? -1 : (i > j) ? 1 : 0; Would it make (more) sense to implement rich comparisons for floats? I'm not enthusiastic about the patch that got pasted into the bug report. Cheers, mwh -- [1]For those of you who aren't aware "tossing" is a euphamism for, well, vigourously rubbing your love pole. You understand? Flogging the dolphin. Stretching the chicken's neck. Waving your magic wand. Basically, wanking. -- Just another Morfans SDA update From mwh at python.net Thu Feb 19 05:58:26 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 05:58:29 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> (Bob Ippolito's message of "Wed, 18 Feb 2004 21:26:33 -0500") References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: <2mn07fxrn1.fsf@starship.python.net> Bob Ippolito writes: > Some time ago, mwh developed a patch that adds some syntactical sugar > to def, which is equivalent to PEP 318 though it has a different and > more flexible syntax... > previous threads can be easily found here: > http://www.google.com/search?q=+site:mail.python.org+%22meth-syntax- > sugar%22 > the latest version of mwh's patch is here: > http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff > > Here's a quick overview: [snip] > What the patch doesn't do: > lambda is not allowed in the "sugary expressions list" Huh? For a bunch of versions of the patch, putting lambda in the "sugary expressions list" made Python trip an assert, but I believe I fixed that in the most recent version. Or do you mean something else? > there's no *expansion and it won't take an actual list so if > you want a prebaked list of transformations then you'll have > to roll a callable that does it such as: > > def prebake(*expressions): > def _prebake(fn): > for transformation in expressions: > fn = transformation(fn) > return fn > return fn I just don't understand what you mean here. You're contemplating def foo() [memoize, *list_of_callables]: ... Eugh! > This syntactical sugar for def is so unbelievably important to PyObjC > (and likely other projects) that I am willing to distribute my own > modified version of Python if it doesn't make 2.4 (though I would > probably use Stackless as the base, but that's another plea for > another day). > > The patch looks like it still applies to CVS HEAD, but there is a > little fuzz on hunk 2. I have not tested it yet, but I have tested it > against CVS HEAD of the 2.3.3 based Stackless. > > I'm willing to help however I can in order to get this into Python 2.4. PEP 306 is (intended to be) a checklist of what's needed. Cheers, mwh -- ARTHUR: Why are there three of you? LINTILLAS: Why is there only one of you? ARTHUR: Er... Could I have notice of that question? -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From mwh at python.net Thu Feb 19 06:01:04 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 06:01:13 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: (Paul Moore's message of "Thu, 19 Feb 2004 10:42:02 +0000") References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: <2misi3xrin.fsf@starship.python.net> Paul Moore writes: > Bob Ippolito writes: > >> Some time ago, mwh developed a patch that adds some syntactical sugar >> to def, which is equivalent to PEP 318 though it has a different and >> more flexible syntax... > > [...] > >> I'm willing to help however I can in order to get this into Python 2.4. > > I like Michael's patch (or rather the functionality/syntax it offers), > but I'd have to admit that it's for purely theoretical "that's nice" > reasons - I don't have any use cases to add to yours. > > I'm not entirely sure why acceptance got stalled the way it did. [...] > Finally, there was an issue with the fact that the syntax didn't > handle properties, I think this was roughly why discussion stalled -- there was a VAST thread on all kinds of syntactical extensions (some of which, in hindsight, look pretty silly), so much so that the initial proposal got lost in the noise. Please, let's not do this again :-) > but personally, I don't see that as a major issue - properties are > just so different (combining 3 separate functions) that I wouldn't > expect the same syntax to help in both cases. Yeah, my patch fails to deal with a case it wasn't ever intended to. Big deal. Cheers, mwh -- nonono, while we're making wild conjectures about the behavior of completely irrelevant tasks, we must not also make serious mistakes, or the data might suddenly become statistically valid. -- Erik Naggum, comp.lang.lisp From dave at boost-consulting.com Thu Feb 19 08:47:24 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Feb 19 08:49:33 2004 Subject: [Python-Dev] PEP 309 misuses the term "currying" Message-ID: The operation performed by the operator described here is actually called "Partial Application". A good definition of currying can be found in http://tinyurl.com/3d6w6 and http://tinyurl.com/ly29. Several languages have misused the term "curry" this way http://tinyurl.com/2p6gb, http://tinyurl.com/2onya (Python) http://tinyurl.com/36pus (JavaScript) http://tinyurl.com/3dj6j (Dylan) Currying transforms a function taking a single tuple argument into a function taking multiple arguments, and uncurrying reverses the process. >>> def curry(f): ... return lambda *args: f(args) ... >>> def add2(args): # add the first 2 elements of an argument tuple ... return args[0] + args[1] ... >>> curry(add2)(1,2) # make add2 accept multiple args 3 >>> def uncurry(f): ... return lambda x: f(*x) ... >>> uncurry(curry(add2))((1,2)) # reverse currying; pass a tuple 3 -- Dave Abrahams Boost Consulting www.boost-consulting.com From dave at boost-consulting.com Thu Feb 19 08:47:47 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Feb 19 08:50:21 2004 Subject: [Python-Dev] PEP 309 misuses the term "currying" Message-ID: The operation performed by the operator described here is actually called "Partial Application". A good definition of currying can be found in http://tinyurl.com/3d6w6 and http://tinyurl.com/ly29. Several languages have misused the term "curry" this way http://tinyurl.com/2p6gb, http://tinyurl.com/2onya (Python) http://tinyurl.com/36pus (JavaScript) http://tinyurl.com/3dj6j (Dylan) Currying transforms a function taking a single tuple argument into a function taking multiple arguments, and uncurrying reverses the process. >>> def curry(f): ... return lambda *args: f(args) ... >>> def add2(args): # add the first 2 elements of an argument tuple ... return args[0] + args[1] ... >>> curry(add2)(1,2) # make add2 accept multiple args 3 >>> def uncurry(f): ... return lambda x: f(*x) ... >>> uncurry(curry(add2))((1,2)) # reverse currying; pass a tuple 3 -- Dave Abrahams Boost Consulting www.boost-consulting.com From aahz at pythoncraft.com Thu Feb 19 09:04:16 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Feb 19 09:04:26 2004 Subject: [Python-Dev] PEP 309 misuses the term "currying" In-Reply-To: References: Message-ID: <20040219140416.GA23430@panix.com> On Thu, Feb 19, 2004, David Abrahams wrote: > > The operation performed by the operator described here is actually > called "Partial Application". A good definition of currying can be > found in http://tinyurl.com/3d6w6 and http://tinyurl.com/ly29. You probably should write directly to the PEP author. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach From dave at boost-consulting.com Thu Feb 19 10:07:34 2004 From: dave at boost-consulting.com (David Abrahams) Date: Thu Feb 19 10:09:13 2004 Subject: [Python-Dev] Re: PEP 309 misuses the term "currying" References: <20040219140416.GA23430@panix.com> Message-ID: Aahz writes: > On Thu, Feb 19, 2004, David Abrahams wrote: >> >> The operation performed by the operator described here is actually >> called "Partial Application". A good definition of currying can be >> found in http://tinyurl.com/3d6w6 and http://tinyurl.com/ly29. > > You probably should write directly to the PEP author. Done -- Dave Abrahams Boost Consulting www.boost-consulting.com From tismer at stackless.com Thu Feb 19 10:22:13 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Feb 19 10:18:47 2004 Subject: [Python-Dev] Nokia In-Reply-To: <402A921E.2030705@ActiveState.com> References: <402A921E.2030705@ActiveState.com> Message-ID: <4034D4A5.8010007@stackless.com> David Ascher wrote: > I'm at ETech 2004, where the CTO of Nokia talked about how they are > _currently_ using Python to develop applications that run on their > "66-series" platform (which includes the n-gage IIRC). > > Nokia shipped 100 million units last year. > > Interesting, no? > > --david > > PS: Yes, I'll be asking him to consider becoming a PSF sponsor member. I would like to ask them whether they would be interested in Stackless Python. It would make sense on a small device, bringing them light threads and program persistence. Does anybody know whom I should contact? thanks - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From jack at performancedrivers.com Thu Feb 19 10:43:11 2004 From: jack at performancedrivers.com (Jack Diederich) Date: Thu Feb 19 10:43:18 2004 Subject: [Python-Dev] Accumulation module In-Reply-To: <40062963.4070808@ActiveState.com> References: <20040113202158.GB9457@panix.com> <005801c3da78$01246940$e102a044@oemcomputer> <20040114210650.GA27649@panix.com> <40062963.4070808@ActiveState.com> Message-ID: <20040219154311.GB2429@performancedrivers.com> On Wed, Jan 14, 2004 at 09:47:15PM -0800, David Ascher wrote: > >>>>* What to call the module > >>>stats > >>There is already a stat module. Any chance of confusion? > >Make it statistics, then. > > > There is considerable prior art here worth considering: > > see http://www.nmr.mgh.harvard.edu/Neural_Systems_Group/gary/python.html *jumps into a dead thread* I wrote http://probstat.sourceforge.net/ which just does fast combination/permutation/cartesian products in C w/ a python binding. I'd be happy to throw a PSF license on it if you care for some or all of it. -jackdied From jcarlson at uci.edu Thu Feb 19 10:53:28 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Feb 19 10:56:43 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: <20040219070954.0A4B.JCARLSON@uci.edu> [Bob Ippolito] [snip] > This evaluation order is Guido approved, though at least one person > wanted it to be the other way around I can understand the desire to have it both ways. That is, while it makes sense to have functions applied in list-order, it also makes sense to have the resulting function visually in the same order. [snip some examples, a few from PyObjC] From what I understand (please correct me if I am wrong), the entirety of the PEP seeks to fix the situation where someone wants to alter a defined function, the most common case (though I've never used it) in making a methods class or static (or really instancemethod and function, if we check types in Python 2.3). Now, Bob brings this up in the context of PyObjC, and generating ':' separated naming that PyObjC needs. While I feel for him not wanting to type, and even being pained to type so much, I've never needed to do the sort of thing that the PEP or Bob offers, and don't know if it is a real syntactic gain. Certainly it makes convenient all of the examples that the PEP and Bob offer, but again, is the syntax and functionality necessarily clear? Both the PEP and Bob offer that the order of invocation of the 'translations' could be reversed, and still make sense. I think this is an example of the syntax not being clear. - Josiah From bob at redivi.com Thu Feb 19 11:00:01 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 19 10:56:54 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <2mn07fxrn1.fsf@starship.python.net> References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2mn07fxrn1.fsf@starship.python.net> Message-ID: On Feb 19, 2004, at 5:58 AM, Michael Hudson wrote: > Bob Ippolito writes: > >> Some time ago, mwh developed a patch that adds some syntactical sugar >> to def, which is equivalent to PEP 318 though it has a different and >> more flexible syntax... >> previous threads can be easily found here: >> http://www.google.com/search?q=+site:mail.python.org+%22meth-syntax- >> sugar%22 >> the latest version of mwh's patch is here: >> http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff >> >> Here's a quick overview: > [snip] >> What the patch doesn't do: >> lambda is not allowed in the "sugary expressions list" > > Huh? For a bunch of versions of the patch, putting lambda in the > "sugary expressions list" made Python trip an assert, but I believe I > fixed that in the most recent version. Or do you mean something else? Oh ok, it works, but lambda needs to go in parens. Makes sense. I tried it without parens first, thinking that the closing ] would be enough syntax for a single lambda to work, because it does for lists and function calls. >>> def foo(bar) [lambda a:a]: File "", line 1 def foo(bar) [lambda a:a]: ^ SyntaxError: invalid syntax >>> def foo(bar) [(lambda a:a)]: ... >> there's no *expansion and it won't take an actual list so if >> you want a prebaked list of transformations then you'll have >> to roll a callable that does it such as: >> >> def prebake(*expressions): >> def _prebake(fn): >> for transformation in expressions: >> fn = transformation(fn) >> return fn >> return fn > > I just don't understand what you mean here. > > You're contemplating > > def foo() [memoize, *list_of_callables]: > ... > > Eugh! I'm just saying.. not something I would use :) >> This syntactical sugar for def is so unbelievably important to PyObjC >> (and likely other projects) that I am willing to distribute my own >> modified version of Python if it doesn't make 2.4 (though I would >> probably use Stackless as the base, but that's another plea for >> another day). >> >> The patch looks like it still applies to CVS HEAD, but there is a >> little fuzz on hunk 2. I have not tested it yet, but I have tested it >> against CVS HEAD of the 2.3.3 based Stackless. >> >> I'm willing to help however I can in order to get this into Python >> 2.4. > > PEP 306 is (intended to be) a checklist of what's needed. Looks good. I see "Certain changes may require tweaks to the library module pycblr", but google doesn't even know what or where that is. I'll go through it and make a new CVS HEAD patch as well as test additions and doc changes this weekend. However, I'm not going to deal with Jython and I obviously can't do anything about pycblr :) -bob From bob at redivi.com Thu Feb 19 11:10:43 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 19 11:07:32 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> On Feb 19, 2004, at 5:42 AM, Paul Moore wrote: > Bob Ippolito writes: > >> Some time ago, mwh developed a patch that adds some syntactical sugar >> to def, which is equivalent to PEP 318 though it has a different and >> more flexible syntax... > > [...] > >> I'm willing to help however I can in order to get this into Python >> 2.4. > > I like Michael's patch (or rather the functionality/syntax it offers), > but I'd have to admit that it's for purely theoretical "that's nice" > reasons - I don't have any use cases to add to yours. > > I'm not entirely sure why acceptance got stalled the way it did. While > I don't recall wholehearted enthusiasm, nor do I recall any other > proposal getting significant support. My impression was that it ended > up left hanging "because there might be something better that we're > missing". The fact that nothing else has been proposed since argues > either that no-one is interested enough, or that there isn't likely to > be a better option found. Either way, it's probably time to reopen the > issue and decide one way or the other. > > As well as the question of whether anyone is interested, there was > also a question as to whether it is useful enough to justify the > language change. Your use cases may address these issues to an extent. > > Finally, there was an issue with the fact that the syntax didn't > handle properties, but personally, I don't see that as a major issue - > properties are just so different (combining 3 separate functions) that > I wouldn't expect the same syntax to help in both cases. > > I'd like to see this resolved one way or another, and I'm also willing > to help - but I'm not sure what needs doing next. Combining three separate functions is probably done best with a data descriptor base class, or using some naming convention and a metaclass anyway.. Also note that the patch also adds the same sugar to classes, so you could use it in lieu of metaclasses for some cases (like transforming some members that follow a naming convention into properties): >>> class Foo(object) [type]: ... pass ... >>> Foo -bob From bob at redivi.com Thu Feb 19 11:18:20 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 19 11:15:29 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <20040219070954.0A4B.JCARLSON@uci.edu> References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <20040219070954.0A4B.JCARLSON@uci.edu> Message-ID: <3BEF13B9-62F7-11D8-A0EB-000A95686CD8@redivi.com> On Feb 19, 2004, at 10:53 AM, Josiah Carlson wrote: > > [Bob Ippolito] > > [snip] >> This evaluation order is Guido approved, though at least one person >> wanted it to be the other way around > > I can understand the desire to have it both ways. That is, while it > makes sense to have functions applied in list-order, it also makes > sense > to have the resulting function visually in the same order. > > [snip some examples, a few from PyObjC] > > From what I understand (please correct me if I am wrong), the entirety > of the PEP seeks to fix the situation where someone wants to alter a > defined function, the most common case (though I've never used it) in > making a methods class or static (or really instancemethod and > function, > if we check types in Python 2.3). Yes, that's what it does, and that would be the common case. > Now, Bob brings this up in the context of PyObjC, and generating ':' > separated naming that PyObjC needs. While I feel for him not wanting > to > type, and even being pained to type so much, I've never needed to do > the > sort of thing that the PEP or Bob offers, and don't know if it is a > real > syntactic gain. Certainly it makes convenient all of the examples that > the PEP and Bob offer, but again, is the syntax and functionality > necessarily clear? It's not generating ':' separated naming that PyObjC needs. The examples I specified were annotating the functions with type information, so that they could be called from Objective C properly .. "returnsAnInteger", "returnsVoidTakesAnInteger". The metaclass for an ObjC-derived class makes sure all of the methods end up as selectors, but it can only provide "sensible defaults" for type information, and they are not always very sensible. > Both the PEP and Bob offer that the order of invocation of the > 'translations' could be reversed, and still make sense. I think this > is > an example of the syntax not being clear. The common case doesn't need a list, just one function, so it's not likely to confuse anyone more than iter(iterable, sentinel) does. >>> [raiseex(1), raiseex(2)] Traceback (most recent call last): File "", line 1, in ? File "", line 2, in raiseex ValueError: 1 >>> cmp(raiseex(1), raiseex(2)) Traceback (most recent call last): File "", line 1, in ? File "", line 2, in raiseex ValueError: 1 I think that left to right evaluation makes the most sense, because that's what Python always does unless parentheses are involved... and that's the iteration order for sequence types. I didn't really even consider the other way, until I saw that somebody else did.. and I'd be willing to guess that it was someone who has used a lot more lisp/scheme than I have ;) -bob From theller at python.net Thu Feb 19 11:12:37 2004 From: theller at python.net (Thomas Heller) Date: Thu Feb 19 11:36:15 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: (Bob Ippolito's message of "Thu, 19 Feb 2004 11:00:01 -0500") References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2mn07fxrn1.fsf@starship.python.net> Message-ID: Bob Ippolito writes: >> PEP 306 is (intended to be) a checklist of what's needed. > > Looks good. I see "Certain changes may require tweaks to the library > module pycblr", but google doesn't even know what or where that is. > I'll go through it and make a new CVS HEAD patch as well as test > additions and doc changes this weekend. However, I'm not going to > deal with Jython and I obviously can't do anything about pycblr :) > Seems to be a typo: the right spelling is pyclbr (Python class browser). Thomas From mwh at python.net Thu Feb 19 12:05:47 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 12:05:53 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: (Thomas Heller's message of "Thu, 19 Feb 2004 17:12:37 +0100") References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2mn07fxrn1.fsf@starship.python.net> Message-ID: <2m8yizxams.fsf@starship.python.net> Thomas Heller writes: > Bob Ippolito writes: > >>> PEP 306 is (intended to be) a checklist of what's needed. >> >> Looks good. I see "Certain changes may require tweaks to the library >> module pycblr", but google doesn't even know what or where that is. >> I'll go through it and make a new CVS HEAD patch as well as test >> additions and doc changes this weekend. However, I'm not going to >> deal with Jython and I obviously can't do anything about pycblr :) >> > > Seems to be a typo: the right spelling is pyclbr (Python class browser). Fixed! Cheers, mwh -- Guido (like us!) is a bit schizophrenic here: he wants to be a benevolent dictator, but also wants to treat people like grownups. This probably worked better before Python got a large American audience <0.9 wink>. -- Tim Peters, 10 Feb 2000 From mwh at python.net Thu Feb 19 12:06:29 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 12:06:33 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> (Bob Ippolito's message of "Thu, 19 Feb 2004 11:10:43 -0500") References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> Message-ID: <2m4qtnxalm.fsf@starship.python.net> Bob Ippolito writes: > Also note that the patch also adds the same sugar to classes, so you > could use it in lieu of metaclasses for some cases (like transforming > some members that follow a naming convention into properties): > > >>> class Foo(object) [type]: > ... pass > ... > >>> Foo > FWIW, I'm rather less convinced this is useful. Cheers, mwh -- I sense much distrust in you. Distrust leads to cynicism, cynicism leads to bitterness, bitterness leads to the Awareness Of True Reality which is referred to by those-who-lack-enlightenment as "paranoia". I approve. -- David P. Murphy, alt.sysadmin.recovery From tim.one at comcast.net Thu Feb 19 12:52:52 2004 From: tim.one at comcast.net (Tim Peters) Date: Thu Feb 19 12:52:54 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <2mr7wrxrte.fsf@starship.python.net> Message-ID: [Michael Hudson] > Would it make (more) sense to implement rich comparisons for floats? Not much. There are 32 distinct theoretical binary comparison operators under 754 semantics (16 for which subset of {<, =, >, unordered} you're interested in, and then each of those comes in two flavors depending on whether you want a signaling NaN comparand to, or not to, raise an exception), and the standard "recommends" implementing something like 26 of them. > I'm not enthusiastic about the patch that got pasted into the bug > report. That's dead on arrival -- apart from the dubious semantics it's trying to invent, it doesn't even work under MSVC 6 (which I've explained in the bug report). There's no reliable cross-platform C support for any of this stuff, therefore there's nothing Python can do about 754 oddballs without masses of platform-specific code (see fpectlmodule.c for a taste of that approach). From bob at redivi.com Thu Feb 19 12:24:06 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 19 12:53:59 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <2m4qtnxalm.fsf@starship.python.net> References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <2m4qtnxalm.fsf@starship.python.net> Message-ID: <6BD3AF5C-6300-11D8-A0EB-000A95686CD8@redivi.com> On Feb 19, 2004, at 12:06 PM, Michael Hudson wrote: > Bob Ippolito writes: > >> Also note that the patch also adds the same sugar to classes, so you >> could use it in lieu of metaclasses for some cases (like transforming >> some members that follow a naming convention into properties): >> >>>>> class Foo(object) [type]: >> ... pass >> ... >>>>> Foo >> > > FWIW, I'm rather less convinced this is useful. Me too, because you can do it with metaclasses... but then again, composing metaclasses is a pain in the butt. -bob From mwh at python.net Thu Feb 19 13:04:10 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 13:04:13 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: (Tim Peters's message of "Thu, 19 Feb 2004 12:52:52 -0500") References: Message-ID: <2mznbfvtd1.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] >> Would it make (more) sense to implement rich comparisons for floats? > > Not much. But a little bit? It might at least make the results closer to what the underlying C compiler does (modulo bugs in same, of course). > There are 32 distinct theoretical binary comparison operators under > 754 semantics (16 for which subset of {<, =, >, unordered} you're > interested in, and then each of those comes in two flavors depending > on whether you want a signaling NaN comparand to, or not to, raise > an exception), and the standard "recommends" implementing something > like 26 of them. Well, we don't want an SNaN to S, Python has no concept of unordered. That's down to 8. We don't care about the improper subsets of {<, =, >}. That's 6. Rich comparisons. >> I'm not enthusiastic about the patch that got pasted into the bug >> report. > > That's dead on arrival -- apart from the dubious semantics it's trying to > invent, it doesn't even work under MSVC 6 (which I've explained in the bug > report). There's no reliable cross-platform C support for any of this > stuff, therefore there's nothing Python can do about 754 oddballs without > masses of platform-specific code (see fpectlmodule.c for a taste of that > approach). I realise it's a game of "pick your surprise", but the case mentioned in the bug report is particularly, well, surprising. Cheers, mwh -- "Also, does the simple algorithm you used in Cyclops have a name?" "Not officially, but it answers to "hey, dumb-ass!" -- Neil Schemenauer and Tim Peters, 23 Feb 2001 From theller at python.net Thu Feb 19 13:36:35 2004 From: theller at python.net (Thomas Heller) Date: Thu Feb 19 13:36:40 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> (Bob Ippolito's message of "Wed, 18 Feb 2004 21:26:33 -0500") References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: Bob Ippolito writes: > Here's a quick overview: > > def foo(args) [sugary, expressions, list]: > pass > > This is equivalent to: > > def foo(args): > pass > foo = list(expressions(sugary(foo))) > > This evaluation order is Guido approved, though at least one person > wanted it to be the other way around > > One would use this in scenarios such as: > > class FooClass(object): > def className(klass) [classmethod]: > return klass.__name__ > > or, even more importantly (to me anyway): > > # we would change PyObjC to make this a built-in feature.. but, for > # completeness: > import objc > def signature(sig): > def _signature(fn): > return objc.selector(fn, signature=sig) > return _signature > > class FooClass(NSObject): > def returnsAnInteger(self) [signature('i@:')]: > return 1 > def returnsVoidTakesAnInteger_(self, anInteger) [signature('v@:i')]: > pass I was thinking of a similar usecase for ctypes, defining C callback functions. In ctypes, you would be able to write """ WndProc = WINFUNCTYPE(c_int, HWND, MSG, WPARAM, LPARAM) def my_wndproc(hwnd, msg, wParam, lParam) [WndProc]: .... """ instead of this """ WndProc = WINFUNCTYPE(c_int, HWND, MSG, WPARAM, LPARAM) def my_wndproc(hwnd, msg, wParam, lParam): .... my_wndproc = WndProc(my_wndproc) """ BTW (I have this patch not yet applied), is it possible to use more than one line, in this way? def function(arg1, arg2, arg3, arg4, arg5, arg6) [dothis, dothat, doanother]: All in all, I'm +1 on the patch. Thomas From bob at redivi.com Thu Feb 19 13:46:30 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 19 13:43:22 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: On Feb 19, 2004, at 1:36 PM, Thomas Heller wrote: > Bob Ippolito writes: > >> Here's a quick overview: >> >> def foo(args) [sugary, expressions, list]: >> pass >> >> This is equivalent to: >> >> def foo(args): >> pass >> foo = list(expressions(sugary(foo))) [examples showing how much nicer it makes type annotation in ctypes and PyObjC] > BTW (I have this patch not yet applied), is it possible to use more > than > one line, in this way? > > def function(arg1, arg2, arg3, arg4, arg5, arg6) > [dothis, dothat, doanother]: > > All in all, I'm +1 on the patch. No, that is not valid syntax with the current patch.. you would need: def function(arg1, arg2, arg3, arg4, arg5, arg6 ) [dothis, dothat, doanother]: or def function(arg1, arg2, arg3, arg4, arg5, arg6) [ dothis, dothat, doanother]: (or some variation, that makes it obvious to Python that the def is not complete). -bob From tim.one at comcast.net Thu Feb 19 13:52:49 2004 From: tim.one at comcast.net (Tim Peters) Date: Thu Feb 19 13:52:51 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <2mznbfvtd1.fsf@starship.python.net> Message-ID: [Michael Hudson] >>> Would it make (more) sense to implement rich comparisons for floats? [Tim] >> Not much. [Michael] > But a little bit? It might at least make the results closer to what > the underlying C compiler does (modulo bugs in same, of course). The intended behavior of NaNs is covered precisely by standards. Moving Python's nonsense behavior closer to the platform C's nonsense behavior doesn't scream "useful" to me. But I don't object to it either, if someone can't think of something better to do with their time . At least gcc users would get something much closer to what the standard intends (although their Python wouldn't be portable in these respects). OTOH, by pure accident, NaN==NaN *happens* to return the intended False today under Windows Python, and if that's changed, via rich comparison, to return True (which MSVC returns for the C expression NaN==NaN), then we've also managed to break currently-working code on Windows. 754 really isn't friendly to half-assed support. From pje at telecommunity.com Thu Feb 19 13:55:04 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 19 13:55:25 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <2m4qtnxalm.fsf@starship.python.net> References: <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> Message-ID: <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> At 05:06 PM 2/19/04 +0000, Michael Hudson wrote: >Bob Ippolito writes: > > > Also note that the patch also adds the same sugar to classes, so you > > could use it in lieu of metaclasses for some cases (like transforming > > some members that follow a naming convention into properties): > > > > >>> class Foo(object) [type]: > > ... pass > > ... > > >>> Foo > > > >FWIW, I'm rather less convinced this is useful. class Foo(object) [instancesProvide(IFoo)]: ... is actually somewhat more useful. Right now, the packages that do this sort of thing (Zope and PyProtocols) use a pseudometaclass in order to spell it more like this: class Foo(object): protocols.advise(instancesProvide=[IFoo]) Being able to spell it the other way would allow a less "magical" implementation for various types of class metadata. From mwh at python.net Thu Feb 19 14:17:44 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 14:17:47 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: (Tim Peters's message of "Thu, 19 Feb 2004 13:52:49 -0500") References: Message-ID: <2moerux4iv.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] >>>> Would it make (more) sense to implement rich comparisons for floats? > > [Tim] >>> Not much. > > [Michael] >> But a little bit? It might at least make the results closer to what >> the underlying C compiler does (modulo bugs in same, of course). > > The intended behavior of NaNs is covered precisely by standards. Moving > Python's nonsense behavior closer to the platform C's nonsense behavior > doesn't scream "useful" to me. As you say below, it helps with gcc (which more-or-less claims C99 support of late, so no great surprise). > But I don't object to it either, if someone can't think of something > better to do with their time . Heh. See the patch I just attached to the report :-) > At least gcc users would get something much closer to what the > standard intends (although their Python wouldn't be portable in > these respects). OTOH, by pure accident, NaN==NaN *happens* to > return the intended False today under Windows Python, and if that's > changed, via rich comparison, to return True (which MSVC returns for > the C expression NaN==NaN), then we've also managed to break > currently-working code on Windows. Is MSVC 7 any different here? I did try to poke around msdn.microsoft.com and found: This version of Visual C++ is not conformant with the C99 standard. which (a) was in a section about the library specifically and (b) doesn't answer the question. Oh, hang on: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndeepc/html/deep05282002.asp suggests (very much) that this *has* changed. But "someone" should check (I can't). (I certainly wasn't proposing this change for release23-maint). > 754 really isn't friendly to half-assed support. Indeed. But I think were the underlying compiler implements C99 semantics, it's unfriendly of Python to hide them. Cheers, mwh -- Ignoring the rules in the FAQ: 1" slice in spleen and prevention of immediate medical care. -- Mark C. Langston, asr From tim.one at comcast.net Thu Feb 19 14:33:37 2004 From: tim.one at comcast.net (Tim Peters) Date: Thu Feb 19 14:33:41 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: <2moerux4iv.fsf@starship.python.net> Message-ID: [Tim] >> At least gcc users would get something much closer to what the >> standard intends (although their Python wouldn't be portable in >> these respects). OTOH, by pure accident, NaN==NaN *happens* to >> return the intended False today under Windows Python, and if that's >> changed, via rich comparison, to return True (which MSVC returns for >> the C expression NaN==NaN), then we've also managed to break >> currently-working code on Windows. [Michael Hudson] > Is MSVC 7 any different here? I did try to poke around > msdn.microsoft.com and found: > > This version of Visual C++ is not conformant with the C99 > standard. > > which (a) was in a section about the library specifically and (b) > doesn't answer the question. Oh, hang on: > > http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndeepc/htm l/deep05282002.asp > > suggests (very much) that this *has* changed. But "someone" should > check (I can't). I don't have 7 running here either, but that article flat-out says NaN comparisons work correctly now. I believe it -- after all, it's published on a Microsoft site, and so is reviewed by 79 major governments for lawsuit fodder . > (I certainly wasn't proposing this change for release23-maint). We (Python) never promised anything about NaN behavior, so I don't really care if the "currently working" (by perverse accident) NaN==NaN result changes under MSVC 6. >> 754 really isn't friendly to half-assed support. > Indeed. But I think were the underlying compiler implements C99 > semantics, it's unfriendly of Python to hide them. That gets more involved; e.g., float operations should never raise exceptions then, not even division by 0 (although real 754 support includes ways to control whether exceptions get raised). I'm +1 on applying the patch for 2.4 regardless. From mwh at python.net Thu Feb 19 14:43:46 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 19 14:43:50 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary In-Reply-To: (Tim Peters's message of "Thu, 19 Feb 2004 14:33:37 -0500") References: Message-ID: <2mk72ix3bh.fsf@starship.python.net> "Tim Peters" writes: >> (I certainly wasn't proposing this change for release23-maint). > > We (Python) never promised anything about NaN behavior, so I don't really > care if the "currently working" (by perverse accident) NaN==NaN result > changes under MSVC 6. I do: I don't think we should change much behaviour short of segfaulting. But I'm at the conservative end of this spectrum[1]; if someone else wants to backport it, I won't howl. >>> 754 really isn't friendly to half-assed support. > >> Indeed. But I think were the underlying compiler implements C99 >> semantics, it's unfriendly of Python to hide them. > > That gets more involved; e.g., float operations should never raise > exceptions then, not even division by 0 (although real 754 support includes > ways to control whether exceptions get raised). I meant in the context of comparisons. > I'm +1 on applying the patch for 2.4 regardless. Done! Cheers, mwh [1] Possibly because I have more sympathy for the patch monkey than most, having worn that particular hat for a time :-) -- ARTHUR: But which is probably incapable of drinking the coffee. -- The Hitch-Hikers Guide to the Galaxy, Episode 6 From theller at python.net Thu Feb 19 14:45:13 2004 From: theller at python.net (Thomas Heller) Date: Thu Feb 19 14:45:26 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> (Phillip J. Eby's message of "Thu, 19 Feb 2004 13:55:04 -0500") References: <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 05:06 PM 2/19/04 +0000, Michael Hudson wrote: >>Bob Ippolito writes: >> >> > Also note that the patch also adds the same sugar to classes, so you >> > could use it in lieu of metaclasses for some cases (like transforming >> > some members that follow a naming convention into properties): >> > >> > >>> class Foo(object) [type]: >> > ... pass >> > ... >> > >>> Foo >> > >> >>FWIW, I'm rather less convinced this is useful. > > class Foo(object) [instancesProvide(IFoo)]: > ... > > is actually somewhat more useful. Right now, the packages that do > this sort of thing (Zope and PyProtocols) use a pseudometaclass in > order to spell it more like this: > > class Foo(object): > protocols.advise(instancesProvide=[IFoo]) > > Being able to spell it the other way would allow a less "magical" > implementation for various types of class metadata. But there is not really a dramatical advantage over this notation, with a suitable metaclass: class Foo(object): __metaclass__ = SomeMetaclass __provides__ = [IFoo] And my main question (top myself, maybe) with the change is: There should really be some killer features in 2.4 which allow to drop support for Python 2.2 and 2.3 in modules using this new syntax. None of the mentioned use cases apply to the standard library. Thomas From mal at egenix.com Thu Feb 19 16:18:37 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Feb 19 14:58:34 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS, 1.936, 1.937 In-Reply-To: References: Message-ID: <4035282D.50805@egenix.com> mwh@users.sourceforge.net wrote: > Update of /cvsroot/python/python/dist/src/Misc > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3489/Misc > > Modified Files: > NEWS > Log Message: > "Fix" (for certain configurations of the planets, including > recent gcc on Linux/x86) > > [ 899109 ] 1==float('nan') > > by implementing rich comparisons for floats. Have you benchmarked this change ? Rich comparisons sound a lot like overhead and the case you are trying to fix here isn't exactly a common one. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 19 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2004-01-23: Released mxODBC.Zope.DA 1.0.8 http://zope.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From bob at redivi.com Thu Feb 19 15:05:51 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 19 15:02:49 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: References: <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> Message-ID: <047EC9A6-6317-11D8-A0EB-000A95686CD8@redivi.com> On Feb 19, 2004, at 2:45 PM, Thomas Heller wrote: > "Phillip J. Eby" writes: > >> At 05:06 PM 2/19/04 +0000, Michael Hudson wrote: >>> Bob Ippolito writes: >>> >>>> Also note that the patch also adds the same sugar to classes, so you >>>> could use it in lieu of metaclasses for some cases (like >>>> transforming >>>> some members that follow a naming convention into properties): >>>> >>>>>>> class Foo(object) [type]: >>>> ... pass >>>> ... >>>>>>> Foo >>>> >>> >>> FWIW, I'm rather less convinced this is useful. >> >> class Foo(object) [instancesProvide(IFoo)]: >> ... >> >> is actually somewhat more useful. Right now, the packages that do >> this sort of thing (Zope and PyProtocols) use a pseudometaclass in >> order to spell it more like this: >> >> class Foo(object): >> protocols.advise(instancesProvide=[IFoo]) >> >> Being able to spell it the other way would allow a less "magical" >> implementation for various types of class metadata. > > But there is not really a dramatical advantage over this notation, with > a suitable metaclass: > > class Foo(object): > __metaclass__ = SomeMetaclass > __provides__ = [IFoo] > > And my main question (top myself, maybe) with the change is: > > There should really be some killer features in 2.4 which allow to > drop support for Python 2.2 and 2.3 in modules using this new syntax. > None of the mentioned use cases apply to the standard library. There needs to be an adapter implementation in the standard library at some point anyways (personal preference: PyProtocols). Using metaclasses is unintuitive and troublesome because of the single inheritance restriction. Try using mixins! :) PJE's workaround for this is in PyProtocols is to have the advise() function do FAR too much magic (stack introspection, I think) and implements a temporary __metaclass__ that ends up removing itself as soon as the class is created. I'm sure he would love to remove this gnarly code, and I'd love to not depend on it because it's fragile. For example, if you need a custom __metaclass__ and you accidentally put advise() before that declaration then your custom __metaclass__ gets ignored. -bob From pje at telecommunity.com Thu Feb 19 15:33:07 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 19 15:33:22 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: References: <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> Message-ID: <5.1.1.6.0.20040219152648.023695b0@telecommunity.com> At 08:45 PM 2/19/04 +0100, Thomas Heller wrote: >But there is not really a dramatical advantage over this notation, with >a suitable metaclass: > >class Foo(object): > __metaclass__ = SomeMetaclass > __provides__ = [IFoo] Actually, there *is* a dramatic advantage over that notation, since the idea is to allow the object to retain its *own* metaclass, not force it to use a different one. The "magic" syntax used by Zope and PyProtocols leaves the original class definition's metaclass completely unchanged, and the class decorator syntax could do the same. I'm not really strongly arguing for class decorator syntax, though, because the "magic" approach works well enough for me. I'm just pointing out that there *is* a reasonable use case for it. (Also, the "magic" approach has an additional disadvantage, in that if someone explicitly sets __metaclass__ *after* a magic function, it will silently prevent the magic from occurring. The class decorator syntax wouldn't have this problem.) While we're on the subject of PEP 318, I'd also like to mention that I find the: def foo() as bar: ... syntax much more readable than the: def foo() [bar]: ... syntax. But I'll take whatever I can get. :) From theller at python.net Thu Feb 19 15:39:47 2004 From: theller at python.net (Thomas Heller) Date: Thu Feb 19 15:39:49 2004 Subject: [Python-Dev] Re: Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <5.1.1.6.0.20040219152648.023695b0@telecommunity.com> (Phillip J. Eby's message of "Thu, 19 Feb 2004 15:33:07 -0500") References: <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <2B420438-62F6-11D8-A0EB-000A95686CD8@redivi.com> <5.1.1.6.0.20040219135135.02af3ec0@telecommunity.com> <5.1.1.6.0.20040219152648.023695b0@telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 08:45 PM 2/19/04 +0100, Thomas Heller wrote: > >>But there is not really a dramatical advantage over this notation, with >>a suitable metaclass: >> >>class Foo(object): >> __metaclass__ = SomeMetaclass >> __provides__ = [IFoo] > > Actually, there *is* a dramatic advantage over that notation, since > the idea is to allow the object to retain its *own* metaclass, not > force it to use a different one. > > The "magic" syntax used by Zope and PyProtocols leaves the original > class definition's metaclass completely unchanged, and the class > decorator syntax could do the same. I see, it makes sense now. Thomas From jim.jewett at eds.com Thu Feb 19 18:48:49 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Thu Feb 19 18:48:43 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) Message-ID: Bob Ippolito: > Also note that the patch also adds the same sugar to classes, Michael Hudson: > FWIW, I'm rather less convinced this is useful. On the other hand, it seems arbitrary to say that you can do this with one type of definition, but not another. Phillip J. Eby: > class Foo(object) [instancesProvide(IFoo)]: [is more useful for] > class Foo(object): > protocols.advise(instancesProvide=[IFoo]) I think this may be one reason this didn't happen before; wrapping is good, but it isn't entirely obvious how to write it or exactly what it should mean. Given that, it might be premature to use up the [] syntax. This might be a good use of macros ... but I have a feeling they won't appear quickly. -jJ From pje at telecommunity.com Thu Feb 19 19:19:34 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 19 19:19:52 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) In-Reply-To: Message-ID: <5.1.1.6.0.20040219191718.02c81ec0@telecommunity.com> At 06:48 PM 2/19/04 -0500, Jewett, Jim J wrote: >I think this may be one reason this didn't happen before; >wrapping is good, but it isn't entirely obvious how to >write it or exactly what it should mean. Given that, it >might be premature to use up the [] syntax. Well, I personally prefer the 'as' syntax for both class and function decorators. As for the semantics of class decorators, I would propose that they be identical to the semantics for function decorators. I don't think there's a point to adding more ways to spell '__metaclass__'. :) From vinay_sajip at red-dove.com Thu Feb 19 19:25:27 2004 From: vinay_sajip at red-dove.com (Vinay Sajip) Date: Thu Feb 19 19:41:34 2004 Subject: [Python-Dev] Re: logging exceptions References: <1077134092.5317.65.camel@localhost.localdomain> Message-ID: <000a01c3f748$0b49d580$652b6992@alpha> > Zope 2.7 is using the logging package internally as the implementation > for its old zLOG API. We've run into one problem, probably caused by > carelessness on our part. > > zLOG allows you to pass exc_info to be logged, but logging only allows > you to specify a flag saying whether the current exception should be > logged. (I recall that this was discussed at length, but not why the > current solution was reached.) There are several call sites that depend > on this feature to capture an exception, try to recover gracefully or > log a traceback if recovery is impossible. So we depend on this > feature, but can't get it out of logging very easily. > > We've come up with this as a work-around: ["Gross" workaround snipped] I propose to make the following change: def _log(self, level, msg, args, exc_info=None): # ... if exc_info: if type(exc_info) != types.TupleType: exc_info = sys.exc_info() record = self.makeRecord(self.name, level, fn, lno, msg, args, exc_info) self.handle(record) This should allow passing an exception tuple in the exc_info parameter - in which case, that's used instead of calling sys.exc_info(). It is a slight change to the semantics but I can't think of any (non-pathological) code which will break. Can anyone see any problems with this approach? Regards, Vinay From tdelaney at avaya.com Thu Feb 19 20:57:55 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Feb 19 20:58:08 2004 Subject: [Python-Dev] python-dev Summary for 2004-01-01 through 2004-01-31[rough draft] Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE01377F55@au3010avexu1.global.avaya.com> > From: Josiah Carlson > > I don't know if you want to mention the fact that it was rejected by > Guido. Actually, that's exactly the right thing to mention - it has been rejected after all. There's no problem with including a sample implementation with the final version of the PEP, and there's nothing stopping people using it, but it's important that people know it's been rejected and will not be in the standard library. Especially since the final version of the PEP is not yet up, and it hasn't been moved to the 'Rejected' section yet. Tim Delaney From jcarlson at uci.edu Thu Feb 19 21:32:49 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Feb 19 21:36:05 2004 Subject: [Python-Dev] python-dev Summary for 2004-01-01 through 2004-01-31[rough draft] In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE01377F55@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE01377F55@au3010avexu1.global.avaya.com> Message-ID: <20040219183006.0A54.JCARLSON@uci.edu> > > From: Josiah Carlson > > > > I don't know if you want to mention the fact that it was rejected by > > Guido. > > Actually, that's exactly the right thing to mention - it has been rejected after all. > > There's no problem with including a sample implementation with the final version of the PEP, and there's nothing stopping people using it, but it's important that people know it's been rejected and will not be in the standard library. Especially since the final version of the PEP is not yet up, and it hasn't been moved to the 'Rejected' section yet. I didn't expect there to be a problem with offering a standard implementation, for those who wish to use one. In the final version, there will be mention of the fact that it /has/ been rejected, including a reference to the post where Guido rejected it. That portion has already been done. I need to finish the module with a proper set of names for the objects. I'm still getting replies to a survey I did of all previous supporters. - Josiah From ncoghlan at iinet.net.au Thu Feb 19 21:48:10 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Feb 19 21:54:59 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) In-Reply-To: <5.1.1.6.0.20040219191718.02c81ec0@telecommunity.com> References: <5.1.1.6.0.20040219191718.02c81ec0@telecommunity.com> Message-ID: <4035756A.9070000@iinet.net.au> Phillip J. Eby wrote: > At 06:48 PM 2/19/04 -0500, Jewett, Jim J wrote: > >> I think this may be one reason this didn't happen before; >> wrapping is good, but it isn't entirely obvious how to >> write it or exactly what it should mean. Given that, it >> might be premature to use up the [] syntax. > > > Well, I personally prefer the 'as' syntax for both class and function > decorators. > > As for the semantics of class decorators, I would propose that they be > identical to the semantics for function decorators. I don't think > there's a point to adding more ways to spell '__metaclass__'. :) Do you mean: class decorated as decorator: pass is equivalent to: class decorated: pass decorated = decorator(decorated) ? And to get PJE's example to work would require a 'protocol decorator' method such as 'IFoo_decorator' which took a class as an argument, told PyProtocols about the class and the interface it supports, and then returned either the original class, or a suitably changed version? Regards, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268 From pje at telecommunity.com Thu Feb 19 22:33:32 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 19 22:28:51 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) In-Reply-To: <4035756A.9070000@iinet.net.au> References: <5.1.1.6.0.20040219191718.02c81ec0@telecommunity.com> <5.1.1.6.0.20040219191718.02c81ec0@telecommunity.com> Message-ID: <5.1.0.14.0.20040219222231.02a18970@mail.telecommunity.com> At 12:48 PM 2/20/04 +1000, Nick Coghlan wrote: >Phillip J. Eby wrote: > >>At 06:48 PM 2/19/04 -0500, Jewett, Jim J wrote: >> >>>I think this may be one reason this didn't happen before; >>>wrapping is good, but it isn't entirely obvious how to >>>write it or exactly what it should mean. Given that, it >>>might be premature to use up the [] syntax. >> >>Well, I personally prefer the 'as' syntax for both class and function >>decorators. >>As for the semantics of class decorators, I would propose that they be >>identical to the semantics for function decorators. I don't think >>there's a point to adding more ways to spell '__metaclass__'. :) > >Do you mean: > > class decorated as decorator: pass > >is equivalent to: > > class decorated: pass > decorated = decorator(decorated) > >? Yep. >And to get PJE's example to work would require a 'protocol decorator' >method such as 'IFoo_decorator' which took a class as an argument, told >PyProtocols about the class and the interface it supports, and then >returned either the original class, or a suitably changed version? Yes. In the specific cases of Zope and PyProtocols, such functions are called "class advisors". A class advisor takes a class as an argument, and returns either the original class (possibly modified in-place), or a suitable replacement. So, these "class advisors" in fact exist. The "magic" functions used by Zope and PyProtocols usually look something like: def magic(...): def advise(klass): ... return klass addClassAdvisor(advise) and are used like this: class Something: magic(...) The 'addClassAdvisor' does the magic stack and metaclass manipulation (which is a *very* ugly process, btw) to cause 'advise()' to get called on the resulting class, after calling the original metaclass to construct the class. If Python 2.4 included class decorators, then "un-magic" versions of these functions would look more like: def unmagic(...): def advise(klass): ... return klass return advise and you would use them like this: class Something as unmagic(...): pass All in all though, I'm sort of only +0 on class decorators, because the "magic" spelling is easier to read when there are complex declarations being done. The decorator syntax has the disadvantage of requiring line continuations or parens, and the result is a bit ugly. However, if class decorators are included, I believe they should follow the above semantics, rather than diverge from the function decorator semantics. Not just for consistency, but because metaclass hacks are much harder than they need to be. Many simple things that people would like to do with metaclasses can be more easily accomplished with a simple decorator. Decorators also don't have to automatically be reapplied to subclasses, while metaclasses do. From vinay_sajip at yahoo.co.uk Fri Feb 20 03:11:13 2004 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Fri Feb 20 03:08:33 2004 Subject: [Python-Dev] Fw: logging exceptions Message-ID: <003c01c3f789$1cb04480$652b6992@alpha> ----- Original Message ----- From: "Vinay Sajip" To: Cc: Sent: Friday, February 20, 2004 12:25 AM Subject: Re: logging exceptions > > Zope 2.7 is using the logging package internally as the implementation > > for its old zLOG API. We've run into one problem, probably caused by > > carelessness on our part. > > > > zLOG allows you to pass exc_info to be logged, but logging only allows > > you to specify a flag saying whether the current exception should be > > logged. (I recall that this was discussed at length, but not why the > > current solution was reached.) There are several call sites that depend > > on this feature to capture an exception, try to recover gracefully or > > log a traceback if recovery is impossible. So we depend on this > > feature, but can't get it out of logging very easily. > > > > We've come up with this as a work-around: > ["Gross" workaround snipped] > > I propose to make the following change: > def _log(self, level, msg, args, exc_info=None): > # ... > if exc_info: > if type(exc_info) != types.TupleType: > exc_info = sys.exc_info() > record = self.makeRecord(self.name, level, fn, lno, msg, args, > exc_info) > self.handle(record) > > This should allow passing an exception tuple in the exc_info parameter - in > which case, that's used instead of calling sys.exc_info(). It is a slight > change to the semantics but I can't think of any (non-pathological) code > which will break. Can anyone see any problems with this approach? > > Regards, > > Vinay > From mwh at python.net Fri Feb 20 06:15:11 2004 From: mwh at python.net (Michael Hudson) Date: Fri Feb 20 06:15:15 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Misc NEWS, 1.936, 1.937 In-Reply-To: <4035282D.50805@egenix.com> (M.'s message of "Thu, 19 Feb 2004 21:18:37 +0000") References: <4035282D.50805@egenix.com> Message-ID: <2mfzd6vw74.fsf@starship.python.net> "M.-A. Lemburg" writes: > mwh@users.sourceforge.net wrote: >> Update of /cvsroot/python/python/dist/src/Misc >> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3489/Misc >> Modified Files: >> NEWS Log Message: >> "Fix" (for certain configurations of the planets, including >> recent gcc on Linux/x86) >> [ 899109 ] 1==float('nan') >> by implementing rich comparisons for floats. > > Have you benchmarked this change ? No. Hadn't even occurred to me. A unscientific test suggests a slowdown of about 1 percent for '1.0 != 2.0' and a speed up of about the same for '1.0 == 1.0'. > Rich comparisons sound a lot like overhead and the case you > are trying to fix here isn't exactly a common one. Doesn't look like it makes much odds here. Cheers, mwh -- ... but I'd rather not reinvent the wheel if I don't have to. On the other hand, if the currently instantiated version of the wheel consists of a square rock covered with moss, I might as well just start fresh. -- Roy Smith, comp.lang.python From mal at egenix.com Fri Feb 20 06:15:35 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 20 06:15:40 2004 Subject: [Python-Dev] Float comparisons In-Reply-To: <4035282D.50805@egenix.com> References: <4035282D.50805@egenix.com> Message-ID: <4035EC57.3080609@egenix.com> [Reposting using a better subject line :-)] M.-A. Lemburg wrote: > mwh@users.sourceforge.net wrote: > >> Update of /cvsroot/python/python/dist/src/Misc >> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3489/Misc >> >> Modified Files: >> NEWS Log Message: >> "Fix" (for certain configurations of the planets, including >> recent gcc on Linux/x86) >> >> [ 899109 ] 1==float('nan') >> >> by implementing rich comparisons for floats. > > > Have you benchmarked this change ? > > Rich comparisons sound a lot like overhead and the case you > are trying to fix here isn't exactly a common one. > -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 20 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2004-01-23: Released mxODBC.Zope.DA 1.0.8 http://zope.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Fri Feb 20 06:27:23 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Feb 20 06:27:32 2004 Subject: [Python-Dev] Call for Papers: 1st International Workshop on Interpreted Languages Message-ID: <4035EF1B.2000609@egenix.com> Sebastian Bergmann, organizer of the above workshop, asked me to post this announcement to the relevant Python lists. http://www.sebastian-bergmann.de/InterpretedLanguages2004/ I will be giving a talk on how Python is developed representing the PSF. Would be nice if we could get some more talks on Python or Python related subjects into this workshop since it reaches out to a much larger public than the Python specific conferences such as PyCon, PythonUK or EuroPython. The workshop is part of the Net.ObjectDays, a conference on OO- and Internet-based technologies: http://www.netobjectdays.org/node04/en/index.html Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 20 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2004-01-23: Released mxODBC.Zope.DA 1.0.8 http://zope.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mwh at python.net Fri Feb 20 06:24:18 2004 From: mwh at python.net (Michael Hudson) Date: Fri Feb 20 06:45:06 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) In-Reply-To: (Jim J. Jewett's message of "Thu, 19 Feb 2004 18:48:49 -0500") References: Message-ID: <2mbrnuvvrx.fsf@starship.python.net> "Jewett, Jim J" writes: > Bob Ippolito: > >> Also note that the patch also adds the same sugar to classes, > > Michael Hudson: > >> FWIW, I'm rather less convinced this is useful. > > On the other hand, it seems arbitrary to say that you can do > this with one type of definition, but not another. Well, this is the argument that got me to add it to the patch; so this is the argument that I'm not so sure of. However, if Phillip et al. do have uses for class wrappers, then I'm happier with the idea. When I said "less convinced this is useful" that's actually precisely what I meant! > Phillip J. Eby: > >> class Foo(object) [instancesProvide(IFoo)]: > > [is more useful for] > >> class Foo(object): >> protocols.advise(instancesProvide=[IFoo]) > > > I think this may be one reason this didn't happen before; > wrapping is good, but it isn't entirely obvious how to > write it or exactly what it should mean. Given that, it > might be premature to use up the [] syntax. I'm not sure what you mean here. BTW, class C [decorator]: pass *HAS* to mean class C: pass C = decorator(C) and not be "another way to spell __metaclass__", because otherwise having more than one decorator becomes a nonsense. This applies less to the 'as' syntax (which I don't like as much for as-far-as-I-can-tell irrational reasons) so much, but that should still mean the same thing. > This might be a good use of macros ... but I have a feeling they > won't appear quickly. You don't say! Cheers, mwh -- The Internet is full. Go away. -- http://www.disobey.com/devilshat/ds011101.htm From gmccaughan at synaptics-uk.com Fri Feb 20 06:50:36 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Fri Feb 20 06:50:41 2004 Subject: [Python-Dev] Re: PEP 309 misuses the term "currying" Message-ID: <200402201150.36410.gmccaughan@synaptics-uk.com> [Dave, this may be the second copy of this message you receive; sorry about that.] Dave Abrahams wrote: > The operation performed by the operator described here is actually > called "Partial Application". A good definition of currying can be > found in http://tinyurl.com/3d6w6 and http://tinyurl.com/ly29. I disagree. Currying is turning a function that takes its arguments all at once as a tuple into a function that takes them one at a time by partial application. So if f(1,2,3)=64 and F is the curried version of f, then F(1)(2)(3)=64. The idea of currying arose in a context where functions are fundamentally single-argument; it's a way of converting between two different ways of building multiple-argument functions on top of a single-argument-only base. That's handy for mathematicians and theoretical computer scientists, who like to keep their underlying concepts as few and as simple as possible. But in Python, and in many other programming languages, functions with multiple arguments aren't built on a single-argument substrate, so there are *three* options for passing multiple arguments to a function. - Collect args into a tuple, pass that. - Just pass multiple arguments. - Pass arguments one at a time using partial application. And the middle one, the one that isn't generally considered by mathematicians doing lambda calculus, happens to be the *usual* case. How you apply the notion of "currying" to Python, then, depends on which pair of argument-passing conventions you choose to assimilate; in other words, on which of the two distinctions at work here you choose to emphasize. Currying is ... 1. "Turning a function that takes a single tuple argument into one that takes multiple arguments." 2. "Turning a function that takes it arguments all at once into one that takes them one at a time." According to #1, what PEP309 calls currying is no such thing. According to #2, what PEP309 calls currying *is* currying. It looks to me as if the second usage is the dominant one, in which case I think PEP309 is fine. -- g From beyer at imb-jena.de Fri Feb 20 07:07:15 2004 From: beyer at imb-jena.de (Andreas Beyer) Date: Fri Feb 20 08:07:40 2004 Subject: [Python-Dev] string.split Message-ID: <4035F873.4080602@imb-jena.de> The documentation of string.split() says: "... The returned list will then have one more item than the number of non-overlapping occurrences of the separator in the string. ..." The behaviour of split with Python 2.3.3 is: >>> '\tb'.split() ['b'] # Bug? >>> '\tb'.split('\t') ['', 'b'] >>> 'a\t\tb'.split() ['a', 'b'] >>> 'a\t\tb'.split('\t') ['a', '', 'b'] >>> I think there are different interpretations of what a separator is. That is not necessarily a bug, because without stripping a new-line at the end of the string would yield a non-intuitive result list. However, the difference between split with and without the 'sep' argument should be documented. Andreas From dave at boost-consulting.com Fri Feb 20 07:32:45 2004 From: dave at boost-consulting.com (David Abrahams) Date: Fri Feb 20 08:07:43 2004 Subject: [Python-Dev] Re: PEP 309 misuses the term "currying" In-Reply-To: <200402201105.01881.gmccaughan@synaptics-uk.com> (Gareth McCaughan's message of "Fri, 20 Feb 2004 11:05:01 +0000") References: <200402201105.01881.gmccaughan@synaptics-uk.com> Message-ID: Gareth McCaughan writes: > Dave Abrahams wrote: > >> The operation performed by the operator described here is actually >> called "Partial Application". A good definition of currying can be >> found in http://tinyurl.com/3d6w6 and http://tinyurl.com/ly29. > > I disagree. Currying is turning a function that takes its arguments > all at once as a tuple into a function that takes them one at a time > by partial application. > So if f(1,2,3)=64 and F is the curried version > of f, then F(1)(2)(3)=64. You've introduced some syntactic confusion. 1. The syntax you show above for the uncurried function f(1,2,3) means "pass all the arguments in a single tuple" in some languages -- the same ones that give "f x" the meaning "invoke f with argument x". In Python, the parens are part of the function call operator. It means "pass the arguments one at a time" (internals and implementation details aside). 2. Whether or not a language supports the syntax you show above for partial application can be seen as independent of the meaning of currying. > The idea of currying arose in a context where functions are > fundamentally single-argument; it's a way of converting between > two different ways of building multiple-argument functions > on top of a single-argument-only base. That's handy for > mathematicians and theoretical computer scientists, who > like to keep their underlying concepts as few and as simple > as possible. Sure. > But in Python, and in many other programming > languages, functions with multiple arguments aren't built on > a single-argument substrate Depends how you look at it, but I know what you mean. > so there are *three* options > for passing multiple arguments to a function. > - Collect args into a tuple, pass that. > - Just pass multiple arguments. > - Pass arguments one at a time using partial application. And the language doesn't support the third one by default. > And the middle one, the one that isn't generally considered > by mathematicians doing lambda calculus, happens to be the > *usual* case. How you apply the notion of "currying" to Python, > then, depends on which pair of argument-passing conventions > you choose to assimilate; in other words, on which of the two > distinctions at work here you choose to emphasize. Currying > is ... > > 1. "Turning a function that takes a single tuple argument into > one that takes multiple arguments." > > 2. "Turning a function that takes it arguments all at once into > one that takes them one at a time." > > According to #1, what PEP309 calls currying is no such thing. > According to #2, what PEP309 calls currying *is* currying. > > It looks to me as if the second usage is the dominant one, in > which case I think PEP309 is fine. I dont' see how you can make that claim. Curry is a one argument function that accepts a function and returns a new function; I hope we can agree on that. If this were currying according to your description, the result would always be callable with a single argument (yielding, perhaps, another function object). >>> from pep309 import curry >>> def add(a, b): ... return a+b ... >>> curry(add) >>> curry(add)(1) Traceback (most recent call last): File "", line 1, in ? File "curry.py", line 12, in __call__ return self.fn(*(self.args + args), **d) TypeError: add() takes exactly 2 arguments (1 given) >>> Even if we ignore the syntax issues I brought up earlier, this doesn't appear to resemble what you've been describing at all. If Python needs a function like the one being proposed in the PEP (I'm not convinced it does, since IIUC lambda covers that ability nicely), it should probably be called "bind" and should use something like the Boost interface, which doesn't tread on any existing meanings for the word and has been shown to be flexible, powerful, and useful. http://www.boost.org/libs/bind -- Dave Abrahams Boost Consulting www.boost-consulting.com From pf_moore at yahoo.co.uk Fri Feb 20 08:34:32 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Fri Feb 20 08:34:40 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) References: <5.1.1.6.0.20040219191718.02c81ec0@telecommunity.com> Message-ID: "Phillip J. Eby" writes: > At 06:48 PM 2/19/04 -0500, Jewett, Jim J wrote: > >>I think this may be one reason this didn't happen before; >>wrapping is good, but it isn't entirely obvious how to >>write it or exactly what it should mean. Given that, it >>might be premature to use up the [] syntax. > > Well, I personally prefer the 'as' syntax for both class and function > decorators. > > As for the semantics of class decorators, I would propose that they be > identical to the semantics for function decorators. I don't think > there's a point to adding more ways to spell '__metaclass__'. :) I think that one key point here is that the [...] syntax appealed to someone (MWH) enough for him to produce a patch. None of the variations, as far as I am yet aware, has resulted in a working patch. For me, "parcticality beats purity" may well apply here - given the choice between something that can be applied now, and something which provides the same (presumably...) semantics with a different syntax, but which doesn't yet exist, I'd say go for the former (or produce a competing patch - "put up or shut up", if you like, although that's stronger than I intend...) I don't see anyone arguing for different *semantics*. If I'm wrong, then it's getting lost in the discussions over syntax and usage. Paul. -- This signature intentionally left blank From sjoerd at acm.org Fri Feb 20 08:37:31 2004 From: sjoerd at acm.org (Sjoerd Mullender) Date: Fri Feb 20 08:37:52 2004 Subject: [Python-Dev] string.split In-Reply-To: <4035F873.4080602@imb-jena.de> References: <4035F873.4080602@imb-jena.de> Message-ID: <40360D9B.4090505@acm.org> Andreas Beyer wrote: > The documentation of string.split() says: > "... The returned list will then have one more item than the number of > non-overlapping occurrences of the separator in the string. ..." > > The behaviour of split with Python 2.3.3 is: > >>> '\tb'.split() > ['b'] # Bug? > >>> '\tb'.split('\t') > ['', 'b'] > >>> 'a\t\tb'.split() > ['a', 'b'] > >>> 'a\t\tb'.split('\t') > ['a', '', 'b'] > >>> > > I think there are different interpretations of what a separator is. That > is not necessarily a bug, because without stripping a new-line at the > end of the string would yield a non-intuitive result list. However, the > difference between split with and without the 'sep' argument should be > documented. This is intended behavior and is not going to be changed. However, I must agree that the documentation for the string method is somewhat lacking. The documentation of the split function in the string module is much clearer. Method: split([sep [,maxsplit]]) Return a list of the words in the string, using sep as the delimiter string. If maxsplit is given, at most maxsplit splits are done. If sep is not specified or None, any whitespace string is a separator. string.split function: split(s[, sep[, maxsplit]]) Return a list of the words of the string s. If the optional second argument sep is absent or None, the words are separated by arbitrary strings of whitespace characters (space, tab, newline, return, formfeed). If the second argument sep is present and not None, it specifies a string to be used as the word separator. The returned list will then have one more item than the number of non-overlapping occurrences of the separator in the string. The optional third argument maxsplit defaults to 0. If it is nonzero, at most maxsplit number of splits occur, and the remainder of the string is returned as the final element of the list (thus, the list will have at most maxsplit+1 elements). -- Sjoerd Mullender From gmccaughan at synaptics-uk.com Fri Feb 20 09:14:49 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Fri Feb 20 09:14:51 2004 Subject: [Python-Dev] Re: PEP 309 misuses the term "currying" In-Reply-To: References: <200402201105.01881.gmccaughan@synaptics-uk.com> Message-ID: <200402201414.49027.gmccaughan@synaptics-uk.com> On Friday 2004-02-20 12:32, David Abrahams wrote: [I said:] > > 1. "Turning a function that takes a single tuple argument into > > one that takes multiple arguments." > > > > 2. "Turning a function that takes it arguments all at once into > > one that takes them one at a time." > > > > According to #1, what PEP309 calls currying is no such thing. > > According to #2, what PEP309 calls currying *is* currying. > > > > It looks to me as if the second usage is the dominant one, in > > which case I think PEP309 is fine. > > I don't see how you can make that claim. Curry is a one argument > function that accepts a function and returns a new function; I hope > we can agree on that. If this were currying according to your > description, the result would always be callable with a single > argument (yielding, perhaps, another function object). Yow. I've just read PEP309, which (to my shame) I hadn't done when reading your original comment. I thought I could tell from your comments how PEP309 was using the term "currying". I was completely wrong; I misunderstood what you were protesting about, and I now agree: PEP309 misuses the term "curry". > Even if we ignore the syntax issues I brought up earlier, this doesn't > appear to resemble what you've been describing at all. No, you're right. Oops. -- g From jim.jewett at eds.com Fri Feb 20 13:39:17 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Fri Feb 20 13:39:24 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: Summary: With the current syntax, I think the cost is too high for the benefit. It might help to add a keyword, like def foo() wrapped_by [sugary, syntax, list]: Details: Phillip J. Eby: >> As for the semantics of class decorators, I would propose that they be >> identical to the semantics for function decorators. I don't think >> there's a point to adding more ways to spell '__metaclass__'. :) Paul Moore: > I don't see anyone arguing for different *semantics*. In that very paragraph are two alternatives for what it would mean on a class. One matches the function meaning for consistency. The other replaces __metaclass__, which someone thought would be more useful. Personally, I see the PEP as useful, but not useful enough to justify having to remember new syntax. Definately not useful enough to add new meaning to an unmarked list. Alternative syntax won't make it more useful, but might make it less costly to the rest of the language. When I follow a def with an immediate assignment, it is usually to something other than the original name. For instance, I may want to define a module-level function, even though I'm already within a function, or I might want to bind a second alias to the same function. (Come to think of it, "alias" is also a different but logical meaning, particularly for the "as" format.) The ugly solution is to use exec: # Note that this string is just a string, and the editor # won't help me. fn_str = (""" def %(name)s(): print "in fn %(name)s" globals()['%(name)s'] = %(name)s """) # Verified with 2.3, since multiple interpolation is subject # to too many typos. def makealpha(): for fn in 'abcde': exec(fn_str % dict(name='%s' % fn)) I would love a better solution - but this proposal is too limited to help. (= not enough benefit) That's fine, so long as it is also easy to ignore. (= low cost) A new definition type similar to class or def would be OK; I just wouldn't use it much. Changing the syntax of def is a bit more worrisome. With this patch, def foo():[sugary, syntax, list] and def foo() [sugary, syntax, list]: would have very different meanings. To be honest, when I first looked at Python, one of the selling points was that it strongly encouraged readable code. If I had come across the above distinction too soon (like, say, when I wondered how to define a function or class), ... I'm not sure I would have continued to look. def foo() wrapped_by [sugary, syntax, list]: would be far less problematic, because the keyword (1) serves notice that this is a special case (2) serves as a reminder of what the list does. (3) leaves open the possibility of future extensions with other keywords. -jJ From jeremy at alum.mit.edu Fri Feb 20 14:45:32 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Fri Feb 20 14:51:29 2004 Subject: [Python-Dev] Re: logging exceptions In-Reply-To: <000a01c3f748$0b49d580$652b6992@alpha> References: <1077134092.5317.65.camel@localhost.localdomain> <000a01c3f748$0b49d580$652b6992@alpha> Message-ID: <1077306331.2798.5.camel@localhost.localdomain> On Thu, 2004-02-19 at 19:25, Vinay Sajip wrote: > I propose to make the following change: > def _log(self, level, msg, args, exc_info=None): > # ... > if exc_info: > if type(exc_info) != types.TupleType: > exc_info = sys.exc_info() > record = self.makeRecord(self.name, level, fn, lno, msg, args, > exc_info) > self.handle(record) > > This should allow passing an exception tuple in the exc_info parameter - in > which case, that's used instead of calling sys.exc_info(). It is a slight > change to the semantics but I can't think of any (non-pathological) code > which will break. Can anyone see any problems with this approach? It will work for me. I suppose we could worry about someone have some iterable other than a tuple containing the exc_info, but I think that's even more obscure that my use case of wanting to pass the exc_info tuple. It doesn't make sense to store a heterogenous record in a list. Jeremy From python at rcn.com Sat Feb 21 01:46:52 2004 From: python at rcn.com (Raymond Hettinger) Date: Sat Feb 21 01:49:37 2004 Subject: [Python-Dev] string.split In-Reply-To: <40360D9B.4090505@acm.org> Message-ID: <001401c3f846$a0517d00$9500a044@oemcomputer> > Andreas Beyer wrote: > > I think there are different interpretations of what a separator is. That > > is not necessarily a bug, because without stripping a new-line at the > > end of the string would yield a non-intuitive result list. However, the > > difference between split with and without the 'sep' argument should be > > documented. [Sjoerd Mullender] > This is intended behavior and is not going to be changed. > > However, I must agree that the documentation for the string method is > somewhat lacking. The documentation of the split function in the string > module is much clearer. If you copy your note into SourceForge and assign it to me, you will substantially increase the odds that I will fix the docs in the next few days. Raymond Hettinger From sjoerd at acm.org Sat Feb 21 04:24:36 2004 From: sjoerd at acm.org (Sjoerd Mullender) Date: Sat Feb 21 04:24:50 2004 Subject: [Python-Dev] string.split In-Reply-To: <001401c3f846$a0517d00$9500a044@oemcomputer> References: <001401c3f846$a0517d00$9500a044@oemcomputer> Message-ID: <403723D4.5030700@acm.org> Raymond Hettinger wrote: >>Andreas Beyer wrote: >> >>>I think there are different interpretations of what a separator is. > > That > >>>is not necessarily a bug, because without stripping a new-line at > > the > >>>end of the string would yield a non-intuitive result list. However, > > the > >>>difference between split with and without the 'sep' argument should > > be > >>>documented. > > > [Sjoerd Mullender] > >>This is intended behavior and is not going to be changed. >> >>However, I must agree that the documentation for the string method is >>somewhat lacking. The documentation of the split function in the > > string > >>module is much clearer. > > > If you copy your note into SourceForge and assign it to me, you will > substantially increase the odds that I will fix the docs in the next few > days. Done. Bug # 901654. -- Sjoerd Mullender From scav at blueyonder.co.uk Sat Feb 21 12:39:15 2004 From: scav at blueyonder.co.uk (Peter Harris) Date: Sat Feb 21 12:35:22 2004 Subject: [Python-Dev] PEP 309, function currying Message-ID: <403797C3.3060405@blueyonder.co.uk> Hi I've updated PEP309 a little, since someone sent me a better implementation. I haven't mentioned in the latest version the recent objection by Dave Abrahams that: Several languages have misused the term "curry" this way mostly on the grounds that the main language using curry that way is English :) The first Google result I found that mentioned function currying was this: [ http://www.engr.uconn.edu/~jeffm/Papers/curry.html ] Function Currying in Scheme I'll quote from the abstract: Function currying is the process of partially, or incrementally, supplying arguments to a function. Curried functions are delayed functions expecting the remainder of the arguments to be supplied. Once all the arguments are supplied, the function evaluates normally. I haven't the time for an exhausive statistical and linguistic analysis. My vote goes for the prevalent definition of currying that is actually a useful functional programming abstraction, i.e. the one described above. Anyway, this is my last request for flames about the presentation and content of the PEP itself before I post a full draft of the proposed 'functional' module. Peter Harris From ark-mlist at att.net Sat Feb 21 14:15:54 2004 From: ark-mlist at att.net (Andrew Koenig) Date: Sat Feb 21 14:15:36 2004 Subject: [Python-Dev] PEP 309, function currying In-Reply-To: <403797C3.3060405@blueyonder.co.uk> Message-ID: <001501c3f8af$20a7d0e0$6402a8c0@arkdesktop> > I'll quote from the abstract: > Function currying is the process of partially, or incrementally, > supplying arguments to a function. Curried functions are delayed > functions expecting the remainder of the arguments to be supplied. > Once all the arguments are supplied, the function evaluates normally. But there's a perfectly good term for that behavior: partial application, and what that note calls curried functions are really closures. If you use currying to refer to what I've just called partial application, what term do you use to refer to currying? I'm with Dave on this one: If you have a function f that takes two arguments, then currying f should yield a function with one argument that returns a function that takes one argument. From ark-mlist at att.net Sat Feb 21 14:29:53 2004 From: ark-mlist at att.net (Andrew Koenig) Date: Sat Feb 21 14:29:40 2004 Subject: [Python-Dev] PEP 309, function currying In-Reply-To: <403797C3.3060405@blueyonder.co.uk> Message-ID: <001601c3f8b1$14ea5ff0$6402a8c0@arkdesktop> > The first Google result I found that mentioned function currying was this: > [ http://www.engr.uconn.edu/~jeffm/Papers/curry.html ] Function Currying > in Scheme I should have read the paper itself before commenting on it, because the paper uses the term correctly. > I'll quote from the abstract: > Function currying is the process of partially, or incrementally, > supplying arguments to a function. Curried functions are delayed > functions expecting the remainder of the arguments to be supplied. > Once all the arguments are supplied, the function evaluates normally. Unfortunately, the abstract *does not* use the term correctly :-) More accurately, the abstract uses the term twice: once correctly, once incorrectly. Here's what the abstract should say: Function currying is the process of transforming a function that expects all of its arguments at once into a delayed function (called a curried function) that expects the remainder of the arguments to be supplied. Once all the arguments are supplied, the function evaluates normally. This revised abstract is consistent with the usage of the term "curried functions" in section 1 of the paper. Moreover, it is also consistent with the way in which the function "curry" behaves in section 3. From pycon at python.org Sat Feb 21 16:10:47 2004 From: pycon at python.org (PyCon Chair) Date: Sat Feb 21 16:20:57 2004 Subject: [Python-Dev] PyCon talks and schedule posted Message-ID: <20040221211047.GA25618@panix.com> First of all, don't forget that early-bird reg ends midnight Sunday Feb 22. We've now got a list of talks and a planned schedule: http://www.pycon.org/dc2004/talks/ http://www.python.org/pycon/dc2004/schedule.html We've also added Guido van Rossum and Bruce Eckel as keynote speakers: http://www.pycon.org/dc2004/talks/keynote PyCon is a community-oriented conference targeting developers (both those using Python and those working on the Python project). It gives you opportunities to learn about significant advances in the Python development community, to participate in a programming sprint with some of the leading minds in the Open Source community, and to meet fellow developers from around the world. The organizers work to make the conference affordable and accessible to all. DC 2004 will be held March 24-26, 2004 in Washington, D.C. at the GWU Cafritz Conference Center. The keynote speakers are Mitch Kapor, Guido van Rossum, and Bruce Eckel. There will be a four-day development sprint before the conference (March 20-23). We're looking for volunteers to help run PyCon. If you're interested, subscribe to http://mail.python.org/mailman/listinfo/pycon-organizers Don't miss any PyCon announcements! Subscribe to http://mail.python.org/mailman/listinfo/pycon-announce You can discuss PyCon with other interested people by subscribing to http://mail.python.org/mailman/listinfo/pycon-interest The central resource for PyCon DC 2004 is http://www.pycon.org/dc2004/ -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From jcarlson at uci.edu Sat Feb 21 19:49:17 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat Feb 21 19:52:30 2004 Subject: [Python-Dev] PEP 326 Message-ID: <20040221163119.0E8C.JCARLSON@uci.edu> Hey python-dev-ers, As an update, I've just submitted my final changes to PEP 326, along with a link to my recommended implementation. The latest version of the PEP (until it gets updated in CVS) is available here: http://www.ics.uci.edu/~jcarlson/pep326/pep-0326.html The implementation will be available at the below link until I finish grad school: http://www.ics.uci.edu/~jcarlson/pep326/extremes.py As suggested by Andrew P. Lentvorski Jr. off-list, the names of the singletons are UniversalMaximum and UniversalMinimum, due to their self-documenting nature. If anyone doesn't understand what they are, just by reading their names, then jeebus help them. For those that want or need shorter names, uMax and uMin are equivalent aliases in extremes.py Unless something major happens, this is likely the last message python-dev will hear from me about PEP 326. Thank you for all the comments/questions/etc. during the PEP's lifetime. - Josiah From greg at cosc.canterbury.ac.nz Sun Feb 22 19:24:42 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Sun Feb 22 19:25:14 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: Message-ID: <200402230024.i1N0Ogp25517@oma.cosc.canterbury.ac.nz> "Jewett, Jim J" : > With the current syntax, I think the cost is too high > for the benefit. It might help to add a keyword, like > > def foo() wrapped_by [sugary, syntax, list]: Some keyword might help, but I don't like that particular choice, as it draws too much attention to the mechanism, which is not the important thing to be thinking about when reading the code. Consider def marmalade(self) wrapped_by [classmethod]: ... That's wrong -- marmalade isn't wrapped by a class method, it *is* a class method. I think it ought to be very readable without any keyword, provided the names of the wrapper functions are chosen appropriately. Can you honestly say that, as a Python newbie, if you had come across def marmalade(self) [classmethod]: ... you wouldn't have been able to guess the intent, if not the mechanism being used to achieve it? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Sun Feb 22 19:50:11 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Sun Feb 22 19:50:53 2004 Subject: [Python-Dev] PEP 309, function currying In-Reply-To: <001601c3f8b1$14ea5ff0$6402a8c0@arkdesktop> Message-ID: <200402230050.i1N0oBP25804@oma.cosc.canterbury.ac.nz> Andrew Koenig wrote: > I'm with Dave on this one: If you have a function f that takes two > arguments, then currying f should yield a function with one argument that > returns a function that takes one argument. and then he wrote: > > I'll quote from the abstract: > > Function currying is the process of partially, or incrementally, > > supplying arguments to a function. Curried functions are delayed > > functions expecting the remainder of the arguments to be supplied. > > Once all the arguments are supplied, the function evaluates normally. > > Here's what the abstract should say: > > Function currying is the process of transforming a function that expects all > of its arguments at once into a delayed function (called a curried function) > that expects the remainder of the arguments to be supplied. Once all the > arguments are supplied, the function evaluates normally. But isn't that just as wrong as the first version? It still seems to be describing partial application, not currying as you defined it above. Maybe it should say Function currying is the process of transforming a function that takes n arguments into a function that takes one argument and returns another function (itself also curried, if n > 2) that takes the remaining n-1 arguments. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From bac at OCF.Berkeley.EDU Mon Feb 23 02:09:32 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Feb 23 02:09:38 2004 Subject: [Python-Dev] Boundary checks on arguments to time.strftime() Message-ID: <4039A72C.2070904@ocf.berkeley.edu> Bug #897625 (http://python.org/sf/897625) deals with giving time.strftime() date information that is outside of proper bounds (e.g., using a negative number for the day of the week). This can lead to a crash since it seems some strftime implementations just index into an array for values for text and thus a negative value can lead to bad things happening. I would like to raise a ValueError if the argument is out of range. Problem is that this will break code. I could just force all negative values to all values outside the proper bounds to a reasonable value, but that seems to go against the path of least surprise. That is question 1. Question 2 is what to really check. This really is only a concern for month and day of the week since everything else is just a number and doesn't have some name representation. I could check all 9 values, though, or just these two. Question 3 is whether to extend this to time.asctime() . I have talked to Tim about this and his thoughts are to just deal with time.strftime() and leave everything else alone. That's fine with me, but there is the same possibility of having problems with asctime(). But then again, checking value for asctime() would potentially break even more code. So, thoughts? -Brett From python at rcn.com Mon Feb 23 02:36:36 2004 From: python at rcn.com (Raymond Hettinger) Date: Mon Feb 23 02:38:17 2004 Subject: [Python-Dev] Boundary checks on arguments to time.strftime() In-Reply-To: <4039A72C.2070904@ocf.berkeley.edu> Message-ID: <004a01c3f9df$c500e7e0$f72dc797@oemcomputer> [Brett] > Bug #897625 (http://python.org/sf/897625) deals with giving > time.strftime() date information that is outside of proper bounds (e.g., > using a negative number for the day of the week). This can lead to a > crash since it seems some strftime implementations just index into an > array for values for text and thus a negative value can lead to bad > things happening. > > I would like to raise a ValueError if the argument is out of range. > Problem is that this will break code. That is the right way to go. The docs have long specified the proper range of input values. If someone has created working code outside those bounds, raising an exception may be their only clue that their code is non-portable (best case) or simply buggy (worst case). > I could just force all negative > values to all values outside the proper bounds to a reasonable value, > but that seems to go against the path of least surprise. That is > question 1. -1. That is slower and more likely to mask genuine coding bugs. Also, it introduces new behavior that would need to be supported in perpetuity. > Question 2 is what to really check. This really is only a concern for > month and day of the week since everything else is just a number and > doesn't have some name representation. I could check all 9 values, > though, or just these two. The month and day of week are the most important since they have word name equivalents that are found by indexing into an array of pointers. But, if you're feeling bold, go ahead and check all of the ranges specified in the docs. That will make it more likely that programmer errors and non-portable wrapping tricks are detected early. > Question 3 is whether to extend this to time.asctime() . I have talked > to Tim about this and his thoughts are to just deal with time.strftime() > and leave everything else alone. That's fine with me, but there is the > same possibility of having problems with asctime(). But then again, > checking value for asctime() would potentially break even more code. I would leave time.asctime() alone. Its argument is typically one returned by another time function. So, it is less susceptible than strftime() where the arguments are constructed piecemeal. Raymond Hettinger From raymond.hettinger at verizon.net Mon Feb 23 04:49:52 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon Feb 23 04:51:31 2004 Subject: [Python-Dev] Optional separatorargument for file.writelines() and StringIO.writelines() Message-ID: <00ba01c3f9f2$62a22060$f72dc797@oemcomputer> Currently, writelines() does not add trailing line separators.. This is fine when working with readlines() but a PITA in other situations. If we added an optional separator argument, it would be easier to add newlines and we would gain some of the flexibility of str.join() at full C speed. Raymond Hettinger -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20040223/6f4fd224/attachment.html From mwh at python.net Mon Feb 23 05:57:27 2004 From: mwh at python.net (Michael Hudson) Date: Mon Feb 23 06:15:08 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <200402230024.i1N0Ogp25517@oma.cosc.canterbury.ac.nz> (Greg Ewing's message of "Mon, 23 Feb 2004 13:24:42 +1300 (NZDT)") References: <200402230024.i1N0Ogp25517@oma.cosc.canterbury.ac.nz> Message-ID: <2meksmukq0.fsf@starship.python.net> Greg Ewing writes: > "Jewett, Jim J" : > >> With the current syntax, I think the cost is too high >> for the benefit. It might help to add a keyword, like >> >> def foo() wrapped_by [sugary, syntax, list]: > > Some keyword might help, but I don't like that particular > choice, as it draws too much attention to the mechanism, > which is not the important thing to be thinking about > when reading the code. Consider > > def marmalade(self) wrapped_by [classmethod]: > ... > > That's wrong -- marmalade isn't wrapped by a class > method, it *is* a class method. > > I think it ought to be very readable without any keyword, > provided the names of the wrapper functions are chosen > appropriately. Can you honestly say that, as a Python > newbie, if you had come across > > def marmalade(self) [classmethod]: > ... > > you wouldn't have been able to guess the intent, if not > the mechanism being used to achieve it? This is what I've been thinking: there is likely *no* syntax that is going to make it totally obvious what's going on, so I don't think one should use this argument against any given syntax. As Greg says, the '[classmethod]' suggests that there will be something classmethod-y about marmalade. Cheers, mwh -- If I had wanted your website to make noise I would have licked my finger and rubbed it across the monitor. -- signature of "istartedi" on slashdot.org From aahz at pythoncraft.com Mon Feb 23 10:51:04 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Feb 23 10:51:13 2004 Subject: [Python-Dev] Boundary checks on arguments to time.strftime() In-Reply-To: <4039A72C.2070904@ocf.berkeley.edu> References: <4039A72C.2070904@ocf.berkeley.edu> Message-ID: <20040223155104.GA12325@panix.com> On Sun, Feb 22, 2004, Brett C. wrote: > > I would like to raise a ValueError if the argument is out of range. > Problem is that this will break code. +1 After all, code is already breaking. > I could just force all negative values to all values outside the > proper bounds to a reasonable value, but that seems to go against the > path of least surprise. That is question 1. -1 > Question 2 is what to really check. This really is only a concern for > month and day of the week since everything else is just a number and > doesn't have some name representation. I could check all 9 values, > though, or just these two. +0 > Question 3 is whether to extend this to time.asctime() . I have talked > to Tim about this and his thoughts are to just deal with time.strftime() > and leave everything else alone. That's fine with me, but there is the > same possibility of having problems with asctime(). But then again, > checking value for asctime() would potentially break even more code. +1 Behavior of similar functions should be as similar as possible. While I see Raymond's point about this being a less likely bug to hit, why not make the fix now while a pair of eyeballs that knows the code is already looking? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From guido at python.org Mon Feb 23 10:53:56 2004 From: guido at python.org (Guido van Rossum) Date: Mon Feb 23 10:54:03 2004 Subject: [Python-Dev] Boundary checks on arguments to time.strftime() In-Reply-To: Your message of "Mon, 23 Feb 2004 02:36:36 EST." <004a01c3f9df$c500e7e0$f72dc797@oemcomputer> References: <004a01c3f9df$c500e7e0$f72dc797@oemcomputer> Message-ID: <200402231553.i1NFruD03217@guido.python.org> > [Brett] > > Bug #897625 (http://python.org/sf/897625) deals with giving > > time.strftime() date information that is outside of proper bounds > (e.g., > > using a negative number for the day of the week). This can lead to a > > crash since it seems some strftime implementations just index into an > > array for values for text and thus a negative value can lead to bad > > things happening. > > > > I would like to raise a ValueError if the argument is out of range. > > Problem is that this will break code. > [RH] That is the right way to go. The docs have long specified the proper > range of input values. If someone has created working code outside > those bounds, raising an exception may be their only clue that their > code is non-portable (best case) or simply buggy (worst case). Agreed. How could an out-of-range causes crashes in the implementation *and* be a required feature? > > I could just force all negative > > values to all values outside the proper bounds to a reasonable value, > > but that seems to go against the path of least surprise. That is > > question 1. > > -1. That is slower and more likely to mask genuine coding bugs. Also, > it introduces new behavior that would need to be supported in > perpetuity. Agreed. > > Question 2 is what to really check. This really is only a concern for > > month and day of the week since everything else is just a number and > > doesn't have some name representation. I could check all 9 values, > > though, or just these two. > > The month and day of week are the most important since they have word > name equivalents that are found by indexing into an array of pointers. > But, if you're feeling bold, go ahead and check all of the ranges > specified in the docs. That will make it more likely that programmer > errors and non-portable wrapping tricks are detected early. To be consistent you should check all values. You could go oveboard and check for things like February 29, but I recommend against this. After all strftime() only does formatting. > > Question 3 is whether to extend this to time.asctime() . I have > > talked to Tim about this and his thoughts are to just deal with > > time.strftime() and leave everything else alone. That's fine with > > me, but there is the same possibility of having problems with > > asctime(). But then again, checking value for asctime() would > > potentially break even more code. Why? The question is, do we need to check to protect the implementation? Then I'd say we have no choice. Otherwise, the question is, should we let bad input cause bad output (the GIGO principle) or should we try to flag it? Which is better for the average application? Given that asctime() is likely used for printing messages, causing new exceptions in code that "worked" before is going to get complaints from users who deploy buggy programs and get embarrassed by the new exception. Has happened before. > I would leave time.asctime() alone. Its argument is typically one > returned by another time function. So, it is less susceptible than > strftime() where the arguments are constructed piecemeal. That I don't know. More likely, asctime() is simply not used as much, because its (fixed) output formats has few virtues except being an ancient Unix standard, and in today's internet world that's not enough. --Guido van Rossum (home page: http://www.python.org/~guido/) From ark-mlist at att.net Mon Feb 23 11:06:45 2004 From: ark-mlist at att.net (Andrew Koenig) Date: Mon Feb 23 11:06:46 2004 Subject: [Python-Dev] PEP 309, function currying In-Reply-To: <200402230050.i1N0oBP25804@oma.cosc.canterbury.ac.nz> Message-ID: <002601c3fa27$08ddd490$6402a8c0@arkdesktop> > > Here's what the abstract should say: > > > > Function currying is the process of transforming a function that expects > > all of its arguments at once into a delayed function (called a curried > > function) that expects the remainder of the arguments to be supplied. > > Once all the arguments are supplied, the function evaluates normally. > > But isn't that just as wrong as the first version? It still seems > to be describing partial application, not currying as you defined > it above. Maybe it should say > > Function currying is the process of transforming a function that > takes n arguments into a function that takes one argument and > returns another function (itself also curried, if n > 2) that > takes the remaining n-1 arguments. Actually, we're both wrong. I missed a level of indirection in trying to reuse words from the original abstract where possible, so I missed that the "delayed function" must itself return a function. You missed that the function returned takes only one argument, not n-1 arguments. Perhaps this way will work: Function currying is the process of transforming a function that expects all of its arguments at once into a delayed function (called a curried function), that expects only its first argument. If the first argument is the only argument, currying is the identity function. Otherwise, the delayed function, itself also curried, expects the first of the remaining arguments and yields either the result (if there is only one remaining argument) or a similarly delayed, curried function that deals with the second and subsequent remaining arguments. From tim.one at comcast.net Mon Feb 23 11:25:47 2004 From: tim.one at comcast.net (Tim Peters) Date: Mon Feb 23 11:38:51 2004 Subject: [Python-Dev] Boundary checks on arguments to time.strftime() In-Reply-To: <200402231553.i1NFruD03217@guido.python.org> Message-ID: [Guido] > ... > Agreed. How could an out-of-range causes crashes in the > implementation *and* be a required feature? The intersection of "causes crashes" and "required features" consists of platform C library bugs. The potential behavior changes go beyond that intersection, though: timemodule.c uses gettmarg() to build struct tm thingies. mktime() specifically allows struct tm fields to be out of range, but time.mktime() uses gettmarg(), so if the latter starts complaining, *some* existing uses of time.mktime() will start complaining too. We hit this in a small way when calender.timegm() was changed to use the datetime module: days = datetime.date(year, month, day).toordinal() - _EPOCH_ORD started complaining about senseless y,m,d triples then, and a user did file a bug report against that. The code was changed to the goofy: days = datetime.date(year, month, 1).toordinal() - _EPOCH_ORD + day - 1 instead so they could continue using senseless (wrt the specified year and month) days. "Legacy API"s are always touchy. We have one report of a crash now, due to using a negative tm_wday value in a tuple passed to strftime(). strftime() may want to gripe about that, but mktime() may not (the C std specifically requires that the mktime() implementation ignore the input value of tm_wday, so if any platform crashes on a negative tm_wday input to mktime(), that would be a platform C bug). From gmccaughan at synaptics-uk.com Mon Feb 23 11:58:23 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Mon Feb 23 11:58:27 2004 Subject: [Python-Dev] PEP 309, function currying In-Reply-To: <002601c3fa27$08ddd490$6402a8c0@arkdesktop> References: <002601c3fa27$08ddd490$6402a8c0@arkdesktop> Message-ID: <200402231658.23008.gmccaughan@synaptics-uk.com> On Monday 2004-02-23 16:06, Andrew Koenig wrote: > Function currying is the process of transforming a function that > expects all of its arguments at once into a delayed function > (called a curried function), that expects only its first argument. > If the first argument is the only argument, currying is the > identity function. Otherwise, the delayed function, itself also > curried, expects the first of the remaining arguments and yields > either the result (if there is only one remaining argument) or > a similarly delayed, curried function that deals with the second > and subsequent remaining arguments. I think it's better to define partical application and currying together, as follows. Partial application is a way of transforming a function by specifying values for some of its arguments. If f is a function of n arguments and 0 <= m <= n, then the partial application of f to y1,...,ym is a function f1 of n-m arguments such that f1(x(m+1),...,xn) = f(y1,...,ym, x(m+1),...,xn). Currying is a way of transforming a function so that instead of accepting all its arguments at once it accepts just the first, returning a function which accepts just the second, returning a function which accepts just the third, and so on. Formally, denote the partial application of f to y1,...,ym by f[y1,...,ym]; then currying is the function transformer C such that - when f is a function of no arguments, C(f) = f(); - when f is a function of at least one argument, C(f) is the 1-argument function taking y1 to C(f[y1]). Note that currying a no-argument function yields not a function but a constant; currying a 1-argument function yields the same function. It follows that C(f)(x1)(x2)...(xn) = f(x1,x2,...,xn). In a context where the definition of partial application isn't useful, this can be shortened to Currying is a way of transforming a function so that instead of accepting all its arguments at once it accepts just the first, returning a function which accepts just the second, returning a function which accepts just the third, and so on. Formally, given a function of at least one argument and a value y1, denote by f[y1] the function that takes (x2,...,xn) to f(y1,x2,...,xn). Then currying is the function transformer C such that - when f is a function of no arguments, C(f) = f(); - when f is a function of at least one argument, C(f) is the 1-argument function taking y1 to C(f[y1]). It follows that C(f)(x1)(x2)...(xn) = f(x1,x2,...,xn). Note that currying a no-argument function yields not a function but a constant; currying a 1-argument function yields the same function; currying any function with at least one argument yields a function that takes exactly one argument. I think it's elegant to start the definition at 0-argument functions, but obviously you can alternatively define currying only for functions of at least one argument, in which case you have the advantage that it always yields a function :-). -- g From skip at pobox.com Mon Feb 23 12:10:14 2004 From: skip at pobox.com (Skip Montanaro) Date: Mon Feb 23 12:10:26 2004 Subject: [Python-Dev] Re: [Python-checkins] python/nondist/sandbox/statistics statistics.py, NONE, 1.1 test_statistics.py, NONE, 1.1 In-Reply-To: References: Message-ID: <16442.13302.313737.325335@montanaro.dyndns.org> Raymond> The module focuses on everyday data reduction functions and Raymond> omits more sophisticated statistical tools. ... Raymond> __all__ = ['mean', 'stddev', 'product', 'nlargest', Raymond> 'nsmallest', 'select'] Raymond It's been years since I last looked at a statistics book or thought much about statistics, but I think the standard deviation was generally defined as the square root of the variance. Perhaps you could easily add the variance to your collection of functions by renaming "stddev" to "variance", remove the "** 0.5" from the return statements, then define stddev as def stddev(data, sample=True): return variance(data, sample) ** 0.5 Skip From jim.jewett at eds.com Mon Feb 23 12:01:30 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Mon Feb 23 12:19:29 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: From: Greg Ewing > "Jewett, Jim J" : >> With the current syntax, I think the cost is too high >> for the benefit. It might help to add a keyword, like >> def foo() wrapped_by [sugary, syntax, list]: > Some keyword might help, but I don't like that particular > choice, as it draws too much attention to the mechanism, I'm certainly not wedded to that specific word; how about "providing" "also_providing", "including", "extended_by", or "acting_as"? ("as" is better than nothing, but does suggest an alias more than a wrapper.) > I think it ought to be very readable without any keyword, > provided the names of the wrapper functions are chosen > appropriately. By someone with similar background. > Can you honestly say that, as a Python > newbie, if you had come across > def marmalade(self) [classmethod]: ... > you wouldn't have been able to guess the intent, if not > the mechanism being used to achieve it? The problem isn't that *this* function would be too mysterious; the problem is that it adds mystery to the *rest* of the language. I would have assumed that I have to make a (possibly implicit) similar declaration about every function, and that the declarations come from a (hopefully) small set, such as {public, private, static, synchronized, final, abstract} The fact that I would have to learn this set (and rules about which went together) would be a barrier to entry. The fact that I couldn't quickly find the exhaustive list (since no such declaration is actually required) would make me nervous. If I saw example code not using the [] declaration, I would assume that the defaults were *mostly* reasonable, but not always, and it would bother me that I couldn't find the information about when I needed to override it. My newbieness is still recent enough that this objection isn't theoretical; "how long before I can spend more time coding than searching through the manual" and "how long before surprises -- even in other people's code -- are at least immediately recognizable instead of deep bugs" were among the main reasons to choose python over Perl or Ruby. > def marmalade(self) [classmethod]: looks basic enough that I assume I have to understand it before I can write even a simple program - particularly since [] characters are pretty common in the rest of the language. The huge difference between (legal) definitions def marmalade(self): [classmethod] and def marmalade(self) [classmethod]: in a language that supposedly uses whitespace instead of syntax characters would make me assume that the language really does use all sorts of obsure syntax, but just hides it in ways that I can't easily recognize. def marmalade(self) unusual_modifiers[x,y,z]: at least tells me that this is an unusual case, and that [x, y, z] has its normal meaning, with any unexpected behavior attributable to the "unusual_modifiers" that calls/uses them. In this case, there would be only one possible unusual modifier, and it would be listed in the appropriate part of the documentation, and it would then be clear that I didn't need to worry about it for most of my own programs. (Note that "unusual_modifiers" doesn't even have to be self-documenting to get this benefit, so long as it is clearly optional and clearly not applicable to *all* definitions. "extended_format1" would already be useful.) I also suspect that this won't be the last extension to definition syntax. Adding a keyword to describe the extension means that someone who doesn't often use that particular extension can still skim the code without a manual handy. Adding new anonymous syntax is already a bit confusing, and would be unworkable by the fourth or fifth extension. -jJ From jf.pieronne at laposte.net Mon Feb 23 12:15:07 2004 From: jf.pieronne at laposte.net (=?ISO-8859-1?Q?Jean-Fran=E7ois_Pi=E9ronne?=) Date: Mon Feb 23 12:39:09 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support Message-ID: <403A351B.6060007@laposte.net> Hi all, I am one of the maintainer of Python on OpenVMS. Building from time to time Python 2.4 from CVS snapshot, I have just noticed that all the conditional compilation against WITH_UNIVERSAL_NEWLINES has been removed. This is a major problem on OpenVMS. VMS has a complex file system which is mostly record oriented and not stream oriented (even if it support stream oriented files, this is not the default). For example it support the following record formats: fixed-length, variable-length, variable with fixed-length control, 3 stream format, etc... The default is variable-length. So reading (opening) text file, for example during a import, in binary mode lead to incorrect results because there are no '\n' or any character (or combinaison of characters) at the end of the line. If the file is not open in binary mode the VMS CRTL automatically append a '\n' at the end of the record, so all work correctly. This not the case in binary mode, in this case the record is read as is, in fact you even get the control part of the record. So, is it possible to maintain this test, even as undocumented or is it acceptable to transform it into a specific VMS test. But I suspect that other mainframe file system will have the same problem. Thanks for any help. Regards, Jean-Fran?ois From ark-mlist at att.net Mon Feb 23 13:25:21 2004 From: ark-mlist at att.net (Andrew Koenig) Date: Mon Feb 23 13:25:48 2004 Subject: [Python-Dev] PEP 309, function currying In-Reply-To: <200402231658.23008.gmccaughan@synaptics-uk.com> Message-ID: <003201c3fa3a$658ad3b0$6402a8c0@arkdesktop> > I think it's better to define partical application and > currying together, as follows. I like it! Is there any chance of us all agreeing on this definition? From aahz at pythoncraft.com Mon Feb 23 13:35:52 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Feb 23 13:36:42 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <403A351B.6060007@laposte.net> References: <403A351B.6060007@laposte.net> Message-ID: <20040223183552.GA24056@panix.com> On Mon, Feb 23, 2004, Jean-Fran?ois Pi?ronne wrote: > > Building from time to time Python 2.4 from CVS snapshot, I have just > noticed that all the conditional compilation against > WITH_UNIVERSAL_NEWLINES has been removed. > > This is a major problem on OpenVMS. Please file a bug report on SF so that this doesn't get lost, then post the bug number here. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From scav at blueyonder.co.uk Mon Feb 23 18:45:00 2004 From: scav at blueyonder.co.uk (Peter Harris) Date: Mon Feb 23 18:41:04 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: References: Message-ID: <403A907C.1030403@blueyonder.co.uk> Greg Ewing wrote: > But isn't that just as wrong as the first version? It still seems > >to be describing partial application, not currying as you defined >it above. Maybe it should say > > Function currying is the process of transforming a function that > takes n arguments into a function that takes one argument and > returns another function (itself also curried, if n > 2) that > takes the remaining n-1 arguments. > > > Yes, that's currying alright, which PEP 309 does in no way describe. Hmm. OK, there is way too much imprecise thinking behind that PEP. Sorry. I did originally describe it as a closure in the first version, then someone corrected me about that too. I'll give the wording some more thought, but the problem I have is that to be useful, the class or function that implements it has to have a wieldy and clear name. Myself, I wouldn't like to use PartiallyApply(fn,*args,**kw). Quick unscientific name poll. Who likes... * curry() * closure() * partial() * partial_apply() * delayed() * other ? Thanks for your thoughts, folks. Peter Harris scav at blueyonder, co, uk From bob at redivi.com Mon Feb 23 19:10:02 2004 From: bob at redivi.com (Bob Ippolito) Date: Mon Feb 23 19:06:51 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: <403A907C.1030403@blueyonder.co.uk> References: <403A907C.1030403@blueyonder.co.uk> Message-ID: On Feb 23, 2004, at 6:45 PM, Peter Harris wrote: > Greg Ewing wrote: > >> But isn't that just as wrong as the first version? It still seems >> >> to be describing partial application, not currying as you defined >> it above. Maybe it should say >> >> Function currying is the process of transforming a function that >> takes n arguments into a function that takes one argument and >> returns another function (itself also curried, if n > 2) that >> takes the remaining n-1 arguments. >> >> > Yes, that's currying alright, which PEP 309 does in no way describe. > Hmm. > OK, there is way too much imprecise thinking behind that PEP. Sorry. > > I did originally describe it as a closure in the first version, then > someone corrected me about that too. I'll give the wording some more > thought, but the problem I have is that to be useful, the class or > function > that implements it has to have a wieldy and clear name. > > Myself, I wouldn't like to use PartiallyApply(fn,*args,**kw). > > Quick unscientific name poll. Who likes... > * curry() -1, if it does partial application and not currying > * closure() -1, this means something else (and more general) in Python > * partial() +1, reasonable to type, seems intuitive enough for someone familiar with the terminology > * partial_apply() +0, not english and too long :) should be partially_apply or partial_application if it's going to be that long winded > * delayed() -1, implies something else to me (makes me think of async code, like a twisted deferred) -bob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2357 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040223/14e68c8d/smime.bin From bac at OCF.Berkeley.EDU Mon Feb 23 19:09:52 2004 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Feb 23 19:09:59 2004 Subject: [Python-Dev] Boundary checks on arguments to time.strftime() In-Reply-To: <200402231553.i1NFruD03217@guido.python.org> References: <004a01c3f9df$c500e7e0$f72dc797@oemcomputer> <200402231553.i1NFruD03217@guido.python.org> Message-ID: <403A9650.80007@ocf.berkeley.edu> Guido van Rossum wrote: >>[Brett] >> >>>Bug #897625 (http://python.org/sf/897625) deals with giving >>>time.strftime() date information that is outside of proper bounds >> >>(e.g., >> >>>using a negative number for the day of the week). This can lead to a >>>crash since it seems some strftime implementations just index into an >>>array for values for text and thus a negative value can lead to bad >>>things happening. >>> >>>I would like to raise a ValueError if the argument is out of range. >>>Problem is that this will break code. >> > [RH] > That is the right way to go. The docs have long specified the proper > >>range of input values. If someone has created working code outside >>those bounds, raising an exception may be their only clue that their >>code is non-portable (best case) or simply buggy (worst case). > > > Agreed. How could an out-of-range causes crashes in the > implementation *and* be a required feature? > As Tim pointed out, time.mktime() is spec'ed to take funky values. But I was going to leave it out of this discussion and just leave it alone. OK, so question 1 is answered: raise the exception. > >>>I could just force all negative >>>values to all values outside the proper bounds to a reasonable value, >>>but that seems to go against the path of least surprise. That is >>>question 1. >> >>-1. That is slower and more likely to mask genuine coding bugs. Also, >>it introduces new behavior that would need to be supported in >>perpetuity. > > > Agreed. > I didn't like it either, but I thought I would throw it out as an option. > >>>Question 2 is what to really check. This really is only a concern for >>>month and day of the week since everything else is just a number and >>>doesn't have some name representation. I could check all 9 values, >>>though, or just these two. >> >>The month and day of week are the most important since they have word >>name equivalents that are found by indexing into an array of pointers. >>But, if you're feeling bold, go ahead and check all of the ranges >>specified in the docs. That will make it more likely that programmer >>errors and non-portable wrapping tricks are detected early. > > > To be consistent you should check all values. > OK. The only reason I brought it up was that people who don't have an explicit value for a field tend to set them to 0 (test_strftime actually does this). This means that code that doesn't set month, day of the month, and day of the year to at least 1 will break. I am not too upset over breaking code that doesn't set day of the year, but it is the 0 value month and day of month that I am a little worried about. But I would rather do a full-on check on all values. > You could go oveboard and check for things like February 29, but I > recommend against this. After all strftime() only does formatting. > No thanks. =) > >>>Question 3 is whether to extend this to time.asctime() . I have >>>talked to Tim about this and his thoughts are to just deal with >>>time.strftime() and leave everything else alone. That's fine with >>>me, but there is the same possibility of having problems with >>>asctime(). But then again, checking value for asctime() would >>>potentially break even more code. > > > Why? The question is, do we need to check to protect the > implementation? Then I'd say we have no choice. Otherwise, the > question is, should we let bad input cause bad output (the GIGO > principle) or should we try to flag it? Which is better for the > average application? Given that asctime() is likely used for printing > messages, causing new exceptions in code that "worked" before is going > to get complaints from users who deploy buggy programs and get > embarrassed by the new exception. Has happened before. > Well, if you look at the C99 spec (http://std.dkuug.dk/jtc1/sc22/open/n2794/ for those who care )for asctime(), the example uses straight indexes of strings so it could explode. But in asctime's case only the month value is of any worry. That I could just force to a valid value since it is the only thing that needs a check; shouldn't break very much code. But I am leaning toward your and Tim's view that this isn't really necessary and anyone using asctime() probably don't want to deal with fixing their code. > >>I would leave time.asctime() alone. Its argument is typically one >>returned by another time function. So, it is less susceptible than >>strftime() where the arguments are constructed piecemeal. > > > That I don't know. More likely, asctime() is simply not used as much, > because its (fixed) output formats has few virtues except being an > ancient Unix standard, and in today's internet world that's not enough. > Agreed. OK, so I will raise TypeError and check everything for time.strftime(). asctime() is the only iffy thing. I say either leave it be or force the month value to January if it is out of the proper range. Any opinions on one or the other? I am leaning towards the former personally. -Brett From greg at cosc.canterbury.ac.nz Mon Feb 23 19:40:16 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Feb 23 19:41:32 2004 Subject: [SPAM: 10.000] Re: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: Message-ID: <200402240040.i1O0eGs15921@oma.cosc.canterbury.ac.nz> "Jewett, Jim J" : > I would have assumed that I have to make a (possibly > implicit) similar declaration about every function Perhaps our brains work in different ways here, but I think I would be more likely to assume that, since the vast majority of Python function declarations I see have no such annotations, that it's something optional that's only rarely used. I take your point about having something to look up in the docs. In the case of the built-in decorators (classmethod, etc.) looking up the name of the decorator itself should provide an adequate starting point, if the docs are written properly. But that wouldn't help someone whose first introduction to decorators was by coming across a user-defined one. I still can't bring myself to like any of the keywords I've thought of or seen suggested. To serve the function of being up-lookable, it really needs to be a new keyword, specific to this use, rather than re-using a generic one like "as" or "is". But I can't think of one that doesn't put undue emphasis on the mechanism, which I don't think would be helpful to either newbies or experienced users. Newbies don't want to be swamped with getting their brains around all the subtleties of how it works, and experienced users don't want to have to think about it all the time. Moreover, the mechanism is so general, and can be used to achieve such a wide variety of effects, that the mechanism itself is really all they have in common. So I don't think its' going to be *possible* to find a single distinctive non-mechanism-related keyword that is truly apt for all its uses. By the way, there are already instances of syntax that people frequently find difficult to look up, such as the * and ** arguments. Maybe we could do with some kind of "explain" tool that you could feed a snippet of code to and it would provide manual references to all the syntactic features it contained? E.g. % pyexplain Welcome to the Python Syntax Explainer! What piece of code is puzzling you? >>> def marmalade(self, x) [classmethod]: This is a Function Definition (see Python Language Reference section 7.5). The items enclosed in [...] are Function Decorators (see Python Language Reference section ?.?). >>> Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From aahz at pythoncraft.com Mon Feb 23 19:49:37 2004 From: aahz at pythoncraft.com (Aahz) Date: Mon Feb 23 19:49:41 2004 Subject: [SPAM: 10.000] Re: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <200402240040.i1O0eGs15921@oma.cosc.canterbury.ac.nz> References: <200402240040.i1O0eGs15921@oma.cosc.canterbury.ac.nz> Message-ID: <20040224004937.GA8031@panix.com> On Tue, Feb 24, 2004, Greg Ewing wrote: > > To serve the function of being up-lookable, it really needs to be a > new keyword, specific to this use, rather than re-using a generic one > like "as" or "is". But I can't think of one that doesn't put undue > emphasis on the mechanism, which I don't think would be helpful to > either newbies or experienced users. Newbies don't want to be swamped > with getting their brains around all the subtleties of how it works, > and experienced users don't want to have to think about it all the > time. > > By the way, there are already instances of syntax that people > frequently find difficult to look up, such as the * and ** arguments. > Maybe we could do with some kind of "explain" tool that you could > feed a snippet of code to and it would provide manual references > to all the syntactic features it contained? What's really needed is a formal reference containing informal writing. IOW, someone looking up ``def`` ought to find the information zie needs, but more colloquially than is used in the Lang Reg. Making a start on that has been on my plate for many moons, but I've never gotten enough tuits.... -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From Paul.Moore at atosorigin.com Tue Feb 24 05:05:21 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Tue Feb 24 05:05:24 2004 Subject: [Python-Dev] Re: PEP 309, function currying Message-ID: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> From: Peter Harris > Yes, that's currying alright, which PEP 309 does in no way describe. > Hmm. > OK, there is way too much imprecise thinking behind that PEP. Sorry. > Quick unscientific name poll. Who likes... > * curry() > > * closure() > > * partial() > > * partial_apply() > > * delayed() > > * other ? I like curry(). The others don't bring the right concept to mind for me. I apologise if this offends the purists, but *all* of the recent attempts to "clarify" the difference between partial application and currying have simply confused me. If curry() gets screams of outrage, someone suggested bind() to follow the Boost C++ library's usage. That's tolerable, although I find it too generic a word. Paul From ncoghlan at iinet.net.au Tue Feb 24 05:18:03 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Tue Feb 24 05:18:11 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> References: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> Message-ID: <403B24DB.1030401@iinet.net.au> Moore, Paul wrote: > If curry() gets screams of outrage, someone suggested bind() to follow the > Boost C++ library's usage. That's tolerable, although I find it too generic > a word. I'm not really tracking all the details of this discussion, but thought I'd throw bind_args() into the ring as a possible name. I suspect bind() would have too many people looking for the nearest TCP/IP stack. . . Regards, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268 From jf.pieronne at laposte.net Tue Feb 24 05:26:11 2004 From: jf.pieronne at laposte.net (=?ISO-8859-1?Q?Jean-Fran=E7ois_Pi=E9ronne?=) Date: Tue Feb 24 05:26:15 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <20040223183552.GA24056@panix.com> References: <403A351B.6060007@laposte.net> <20040223183552.GA24056@panix.com> Message-ID: <403B26C3.5040005@laposte.net> Aahz wrote: > On Mon, Feb 23, 2004, Jean-Fran?ois Pi?ronne wrote: > >>Building from time to time Python 2.4 from CVS snapshot, I have just >>noticed that all the conditional compilation against >>WITH_UNIVERSAL_NEWLINES has been removed. >> >>This is a major problem on OpenVMS. > > > Please file a bug report on SF so that this doesn't get lost, then post > the bug number here. Done, request ID: 903339 Thanks, Jean-Fran?ois From fincher.8 at osu.edu Tue Feb 24 05:29:38 2004 From: fincher.8 at osu.edu (Jeremy Fincher) Date: Tue Feb 24 05:28:58 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: <403B24DB.1030401@iinet.net.au> References: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> <403B24DB.1030401@iinet.net.au> Message-ID: <59368FAE-66B4-11D8-AD8E-000A95AB67E4@osu.edu> On Feb 24, 2004, at 5:18 AM, Nick Coghlan wrote: > Moore, Paul wrote: >> If curry() gets screams of outrage, someone suggested bind() to >> follow the >> Boost C++ library's usage. That's tolerable, although I find it too >> generic >> a word. > > I suspect bind() would have too many people looking for the nearest > TCP/IP stack. . . In a "functional" module? Jeremy From gmccaughan at synaptics-uk.com Tue Feb 24 07:06:32 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Tue Feb 24 07:06:35 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> References: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> Message-ID: <200402241206.32175.gmccaughan@synaptics-uk.com> Paul Moore wrote: > I like curry(). The others don't bring the right concept to mind for me. > I apologise if this offends the purists, but *all* of the recent attempts > to "clarify" the difference between partial application and currying have > simply confused me. Partial application is specifying some of the arguments of a function, so you get a new function that takes fewer arguments. Currying is making a function take one argument at a time, doing partial application as it goes. Highbrow but enlightening example: Suppose F is a function that takes a programming language spec and a program in that language, and executes the program. (You could call it a "generic interpreter".) Then - Partial application turns F into a specialized interpreter, such as a Python interpreter. The partially applied function takes fewer parameters, because some have been fixed. - Currying turns F into an interpreter generator. You feed it a language spec and it spits out an interpreter. The curried function takes the same number of parameters, but you give it them in a different way. "Curry plus call equals partial-apply." > If curry() gets screams of outrage, someone suggested bind() to follow the > Boost C++ library's usage. That's tolerable, although I find it too generic > a word. I think I like that better than the other proposals currently on the table. * By the way, "purists" are more likely to be offended by being caricatured ("purists", "screams of outrage") than by the fact that someone has difficulty understanding what they say. But it's no big deal. -- g From tismer at stackless.com Tue Feb 24 09:38:20 2004 From: tismer at stackless.com (Christian Tismer) Date: Tue Feb 24 09:38:28 2004 Subject: [Python-Dev] cPickle incompatibility! Message-ID: <403B61DC.8000407@stackless.com> Hi developers, while hunting for problems with Stackless pickling, I found a bug in cPickle. Pickling of functions: pickle.py first tries a save_global. If that doesn't work, it tries a save_reduce. cPickle doesn't do that. This bug exists since Python 2.3. cvs -z9 diff -u -wb cPickle.c (in directory D:\cvsdown\python\dist\src\Modules\) Index: cPickle.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/cPickle.c,v retrieving revision 2.148 diff -u -w -b -r2.148 cPickle.c --- cPickle.c 12 Oct 2003 19:09:36 -0000 2.148 +++ cPickle.c 24 Feb 2004 14:28:34 -0000 @@ -2418,6 +2418,11 @@ case 'f': if (type == &PyFunction_Type) { res = save_global(self, args, NULL); + if (res && PyErr_ExceptionMatches(PickleError)) { + /* fall back to reduce */ + PyErr_Clear(); + break; + } goto finally; } break; May I check this one in? (or somebody else do it) ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From aahz at pythoncraft.com Tue Feb 24 09:51:23 2004 From: aahz at pythoncraft.com (Aahz) Date: Tue Feb 24 09:51:32 2004 Subject: [Python-Dev] cPickle incompatibility! In-Reply-To: <403B61DC.8000407@stackless.com> References: <403B61DC.8000407@stackless.com> Message-ID: <20040224145123.GB6991@panix.com> On Tue, Feb 24, 2004, Christian Tismer wrote: > > May I check this one in? (or somebody else do it) File a bug report, add the patch, and check it in if nobody responds in a reasonable timeframe. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From ark-mlist at att.net Tue Feb 24 10:08:32 2004 From: ark-mlist at att.net (Andrew Koenig) Date: Tue Feb 24 10:08:31 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> Message-ID: <003c01c3fae8$11a97220$6402a8c0@arkdesktop> > I like curry(). The others don't bring the right concept to mind for me. > I apologise if this offends the purists, but *all* of the recent attempts > to "clarify" the difference between partial application and currying have > simply confused me. Let me try an example, partly to be sure I understand it. Suppose have a function f with five arguments. Then I think you are proposing a function, which I shall call X for the moment, with the property that evaluating g = X(f, a, b) z = g(c, d, e) has the same effect as z = f(a, b, c, d, e) At least, that's how the code in the PEP seems to behave. If so, that's not currying. Currying would work like this: g = curry(f) # Note -- no other arguments g1 = g(a) g2 = g1(b) g3 = g2(c) g4 = g3(d) z = g4(e) with z having the same value as f(a, b, c, d, e) afterward. Note that curry takes only one argument, namely f. Instead of taking an argument of f directly, it returns a function that will accept f's first argument. It is that extra level of indirection that distinguishes currying from partial application. From gisle at ActiveState.com Tue Feb 24 10:31:59 2004 From: gisle at ActiveState.com (Gisle Aas) Date: Tue Feb 24 10:32:15 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> References: <16E1010E4581B049ABC51D4975CEDB8802C09CC9@UKDCX001.uk.int.atosorigin.com> Message-ID: "Moore, Paul" writes: > From: Peter Harris > > Yes, that's currying alright, which PEP 309 does in no way describe. > > Hmm. > > OK, there is way too much imprecise thinking behind that PEP. Sorry. > > > Quick unscientific name poll. Who likes... > > > * curry() > > > > * closure() > > > > * partial() > > > > * partial_apply() > > > > * delayed() > > > > * other ? > > I like curry(). The others don't bring the right concept to mind for me. > I apologise if this offends the purists, but *all* of the recent attempts > to "clarify" the difference between partial application and currying have > simply confused me. > > If curry() gets screams of outrage, someone suggested bind() to follow the > Boost C++ library's usage. That's tolerable, although I find it too generic > a word. Perl 6 plans to use f.assuming() for this. Search for "curry" in . The perl6 folks also use the term "curry" to mean "partial application". Regards, Gisle Aas, ActiveState From barry at python.org Tue Feb 24 10:50:03 2004 From: barry at python.org (Barry Warsaw) Date: Tue Feb 24 10:50:53 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> Message-ID: <1077637802.20340.36.camel@anthem.wooz.org> On Wed, 2004-02-18 at 21:26, Bob Ippolito wrote: > the latest version of mwh's patch is here: > http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff Does anybody else have problems applying this patch? I get: % patch < meth-syntax-sugar-3.diff patch: **** File Grammar is not a regular file -- can't patch Grammar/Grammar sure seems like a regular file to me though. Standard RH9 patch(1), v 2.5.4. Above patch snagged with wget. -Barry From sjoerd at acm.org Tue Feb 24 11:14:01 2004 From: sjoerd at acm.org (Sjoerd Mullender) Date: Tue Feb 24 11:16:20 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <1077637802.20340.36.camel@anthem.wooz.org> References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <1077637802.20340.36.camel@anthem.wooz.org> Message-ID: <403B7849.3050504@acm.org> Barry Warsaw wrote: > On Wed, 2004-02-18 at 21:26, Bob Ippolito wrote: > > >>the latest version of mwh's patch is here: >>http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff > > > Does anybody else have problems applying this patch? I get: > > % patch < meth-syntax-sugar-3.diff > patch: **** File Grammar is not a regular file -- can't patch > > Grammar/Grammar sure seems like a regular file to me though. Standard > RH9 patch(1), v 2.5.4. Above patch snagged with wget. Try patch -p0. Patch removes directory names by default. -- Sjoerd Mullender From barry at python.org Tue Feb 24 11:19:35 2004 From: barry at python.org (Barry Warsaw) Date: Tue Feb 24 11:19:50 2004 Subject: [Python-Dev] Plea for function/method syntax sugar (PEP 318, with a different syntax) In-Reply-To: <403B7849.3050504@acm.org> References: <08F6B684-6283-11D8-8F69-000A95686CD8@redivi.com> <1077637802.20340.36.camel@anthem.wooz.org> <403B7849.3050504@acm.org> Message-ID: <1077639574.20340.45.camel@anthem.wooz.org> On Tue, 2004-02-24 at 11:14, Sjoerd Mullender wrote: > Try patch -p0. Patch removes directory names by default. Blah. D'uh. Thanks. -B From edloper at gradient.cis.upenn.edu Tue Feb 24 12:05:07 2004 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Tue Feb 24 12:54:57 2004 Subject: [Python-Dev] Re: PEP 309, function currying In-Reply-To: References: Message-ID: <403B8443.6040401@gradient.cis.upenn.edu> Peter Harris wrote: >>* curry() >> >>* closure() >> >>* partial() >> >>* partial_apply() >> >>* delayed() >> >>* other ? bind() and bindargs() seem pretty reasonable to me. But if you don't like those, then how about fixargs() or fix_args()? That gets across the message that we're generating a new function that fixes the values of a subset of the arguments. Side note: I think that the following code will have unexpected behavior with the current implementations: >>> def f(x,y,z): return (x,y,z) >>> g = f.bind(y=3) >>> print g(1,2) TypeError: f() got multiple values for keyword argument 'y' (I would have expected this to return (1,3,2).) Of course, there's not really any way around it without using inspection, and even then you can't really inspect argument lists for builtins. But it might be worth noting in the docs the circumstances under which it's appropriate to bind w/ keyword args. -Edward From lists at hlabs.spb.ru Wed Feb 25 06:13:39 2004 From: lists at hlabs.spb.ru (Dmitry Vasiliev) Date: Wed Feb 25 03:11:50 2004 Subject: [Python-Dev] Optional separatorargument for file.writelines() and StringIO.writelines() In-Reply-To: <00ba01c3f9f2$62a22060$f72dc797@oemcomputer> References: <00ba01c3f9f2$62a22060$f72dc797@oemcomputer> Message-ID: <403C8363.8050009@hlabs.spb.ru> Raymond Hettinger wrote: > Currently, writelines() does not add trailing line separators.. > This is fine when working with readlines() but a PITA in other situations. > > If we added an optional separator argument, it would be easier to add > newlines and we would gain some of the flexibility of str.join() at full > C speed. Maybe not a separator but suffix, so newline will be added to last line too? -- Dmitry Vasiliev (dima at hlabs.spb.ru) http://hlabs.spb.ru From Jack.Jansen at cwi.nl Wed Feb 25 05:15:59 2004 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Wed Feb 25 05:15:17 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <403A351B.6060007@laposte.net> References: <403A351B.6060007@laposte.net> Message-ID: <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> On 23-feb-04, at 18:15, Jean-Fran?ois Pi?ronne wrote: > Hi all, > > I am one of the maintainer of Python on OpenVMS. > > Building from time to time Python 2.4 from CVS snapshot, I have just > noticed that all the conditional compilation against > WITH_UNIVERSAL_NEWLINES has been removed. > > This is a major problem on OpenVMS. Ah, I was afraid this was going to happen when I saw the checkin message about getting rid of the universal newlines conditional. The easiest solution (apart from putting WITH_UNIVERSAL_NEWLINES back in) is to just leave all the universal newlines machinery in place, but disable it. So, where the old situation was that there were ifdefs all over the place the new situation would be that all the code always goes through Py_UniversalNewlineFgets() and Py_UniversalNewlineFread(), but on systems with record-based I/O these just call fgets() and fread(). Then there's only one more ifdef to go: in open_the_file() don't set the mode to "rb" but just plain "r". I assume that sockets and other places where WITH_UNIVERSAL_NEWLINES may have been referred to aren't going to be a problem for you because those aren't applicable to VMS anyway, right? -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From aleaxit at yahoo.com Wed Feb 25 03:40:56 2004 From: aleaxit at yahoo.com (Alex Martelli) Date: Wed Feb 25 08:09:43 2004 Subject: [Python-Dev] Optional separatorargument for file.writelines() and StringIO.writelines() In-Reply-To: <403C8363.8050009@hlabs.spb.ru> References: <00ba01c3f9f2$62a22060$f72dc797@oemcomputer> <403C8363.8050009@hlabs.spb.ru> Message-ID: <542F9C2E-676E-11D8-A15D-000A95EFAE9E@yahoo.com> On 2004 Feb 25, at 12:13, Dmitry Vasiliev wrote: > Raymond Hettinger wrote: >> Currently, writelines() does not add trailing line separators.. >> This is fine when working with readlines() but a PITA in other >> situations. >> If we added an optional separator argument, it would be easier to add >> newlines and we would gain some of the flexibility of str.join() at >> full C speed. > > Maybe not a separator but suffix, so newline will be added to last > line too? Good point. And while a separator would be a slight nuisance to express otherwise, a "suffix" isn't -- it seems to me that f.writelines(x+'\n' for x in mylines) is a rather good way of expressing "suffix each line with a \n". I don't think this suffixing operation is so widely more important than other elaborations on items of mylines to make it worth specialcasing into a writelines argument [if anything, f.writelines(str(x) for x in mylines) would be the one elaboration that seems to me to be by far the most frequent -- still not worth specialcasing though, IMHO]. Alex From Paul.Moore at atosorigin.com Wed Feb 25 08:37:05 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Wed Feb 25 08:37:06 2004 Subject: [Python-Dev] Optional separatorargument for file.writelines()and StringIO.writelines() Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060DFC@UKDCX001.uk.int.atosorigin.com> From: Alex Martelli > On 2004 Feb 25, at 12:13, Dmitry Vasiliev wrote: >> Raymond Hettinger wrote: >>> Currently, writelines() does not add trailing line separators.. >>> This is fine when working with readlines() but a PITA in other >>> situations. >>> If we added an optional separator argument, it would be easier to add >>> newlines and we would gain some of the flexibility of str.join() at >>> full C speed. >> >> Maybe not a separator but suffix, so newline will be added to last >> line too? > > Good point. And while a separator would be a slight nuisance to > express otherwise, a "suffix" isn't -- it seems to me that > f.writelines(x+'\n' for x in mylines) is a rather good way of > expressing "suffix each line with a \n". I don't think this suffixing > operation is so widely more important than other elaborations on items > of mylines to make it worth specialcasing into a writelines argument > [if anything, f.writelines(str(x) for x in mylines) would be the one > elaboration that seems to me to be by far the most frequent -- still > not worth specialcasing though, IMHO]. Maybe there's a useful itertool lurking in here. I'm thinking of something that takes a list of iterables, and generates elements from each in turn, like a flattened izip: def interleave(*iterables): iterables = [iter(it) for it in iterables] while 1: # Let StopIteration fall through, but make sure that # each iterable generates the same number of results vals = [it.next() for it in iterables] for val in vals: yield val Then, we're talking about f.writelines(interleave(mylines, repeat('\n'))) [or if you want separators rather than suffixes, use f.writelines(islice(interleave(repeat('\n'), mylines), 2, None)) instead] Frankly, it's not an obvious win for the writelines case, but I was surprised that there was nothing like interleave in itertools, and that I couldn't find an easy way to write such a thing. Maybe a flatten itertool would be more generally useful - def flatten(iterable): for it in iterable: for val in it: yield val then interleave(*iterables) is flatten(izip(*iterables))... Thoughts? If not added as functions, would these be useful additions to the documentation? Paul. From python at rcn.com Wed Feb 25 09:03:18 2004 From: python at rcn.com (Raymond Hettinger) Date: Wed Feb 25 09:05:34 2004 Subject: [Python-Dev] Optional separatorargument forfile.writelines()and StringIO.writelines() In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060DFC@UKDCX001.uk.int.atosorigin.com> Message-ID: <001401c3fba8$1f9a76c0$7121a044@oemcomputer> [Paul Moore suggests itertools.interleave() or itertools.flatten()] I'll add these to the list of candidate itertools. A related contender is itertools.roundrobin(). Raymond From jf.pieronne at laposte.net Wed Feb 25 11:39:08 2004 From: jf.pieronne at laposte.net (=?ISO-8859-1?Q?Jean-Fran=E7ois_Pi=E9ronne?=) Date: Wed Feb 25 11:39:22 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> References: <403A351B.6060007@laposte.net> <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> Message-ID: <403CCFAC.1090809@laposte.net> Jack Jansen wrote: > > On 23-feb-04, at 18:15, Jean-Fran?ois Pi?ronne wrote: > >> Hi all, >> >> I am one of the maintainer of Python on OpenVMS. >> >> Building from time to time Python 2.4 from CVS snapshot, I have just >> noticed that all the conditional compilation against >> WITH_UNIVERSAL_NEWLINES has been removed. >> >> This is a major problem on OpenVMS. > > > Ah, I was afraid this was going to happen when I saw the checkin message > about getting rid of the universal newlines conditional. > > The easiest solution (apart from putting WITH_UNIVERSAL_NEWLINES back > in) is to just leave all the universal newlines machinery in place, but > disable it. So, where the old situation was that there were ifdefs all > over the place the new situation would be that all the code always goes > through Py_UniversalNewlineFgets() and Py_UniversalNewlineFread(), but > on systems with record-based I/O these just call fgets() and fread(). > Then there's only one more ifdef to go: in open_the_file() don't set the > mode to "rb" but just plain "r". > > I assume that sockets and other places where WITH_UNIVERSAL_NEWLINES may > have been referred to aren't going to be a problem for you because > those aren't applicable to VMS anyway, right? > -- > Jack Jansen, , http://www.cwi.nl/~jack > If I can't dance I don't want to be part of your revolution -- Emma Goldman > thanks for your help, Before I submit a patch, I have done the following update which seem to fix the problem, any comment are welcome. patch for include/fileobject.h @@ -54,3 +54,7 @@ */ +#ifdef __VMS +#define PY_STDIOTEXTMODE "" +#else #define PY_STDIOTEXTMODE "b" +#endif char *Py_UniversalNewlineFgets(char *, int, FILE*, PyObject *); patch for objects/fileobject.c @@ -119,3 +124,7 @@ f->f_buf = NULL; +#ifdef __VMS + f->f_univ_newline = 0; +#else f->f_univ_newline = (strchr(mode, 'U') != NULL); +#endif f->f_newlinetypes = NEWLINE_UNKNOWN; @@ -164,3 +173,11 @@ if (strcmp(mode, "U") == 0 || strcmp(mode, "rU") == 0) +#ifdef __VMS + /* Compatibility: specifying U in a Python without universal + ** newlines is allowed, and the file is opened as a normal text + ** file. + */ + mode = "r"; +#else mode = "rb"; +#endif #ifdef MS_WINDOWS patch for modules/bz2modules.c @@ -1293,3 +1293,7 @@ case 'U': +#ifdef __VMS + self->f_univ_newline = 0; +#else self->f_univ_newline = 1; +#endif break; Cheers, Jean-Fran?ois From rowen at cesmail.net Wed Feb 25 14:29:40 2004 From: rowen at cesmail.net (Russell E. Owen) Date: Wed Feb 25 14:30:03 2004 Subject: [Python-Dev] threading issue: preventing starting twice Message-ID: In the threading module, one must of course only start a thread once. I have some questions about handling this gracefully. First of all, the documentation only says it is an error to start a thread more than once. It does not say which exception is raised. On my system I find it is AssertionError. So...is that going to be true on all systems? Is the test based on "assert", in which case it will go away for optimized code (and then what happens)? Anyway, if it is always AssertionError, I'd like to get the docs updated to say that. Also...if possible, it'd be nice to have some way to ask if a thread has ever been started. More generally, it'd be nice to be able to determine what state a thread is in. I'm not sure if there are any states other than ready to run, running and finished. If that's it, perhaps one could add methods isReady and didRun (a getState method would probably be better if there are more states, but then one has to deal with magic constants). I realize it is a common Python paradigm to assume something will work (such as starting the thread) and catch the exception if it doesn't, but: - this is tricky if the exception is not documented - sometimes one simply wants to know, without actually starting the thread -- Russell From mcherm at mcherm.com Wed Feb 25 14:37:19 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Wed Feb 25 14:37:22 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <1077737839.403cf96f9e903@mcherm.com> Jim Jewett writes: [I'm not going to quote him... details at http://mail.python.org/pipermail/python-dev/2004-February/042685.html ] Jim, thank you for putting this so clearly. I want to second just about everything that you said. Since you've gone into it in great detail, I'll try the brief, bullet-points version: The "def name [modifiers]:" is unpythonic because: * It re-uses the "[]" for a completely new purpose, unrelated to lists or indexing. * It relies heavily on punctuation (the placement of the ":"). * It doesn't "read like english". * There's no good way for a newbie to look it up. All of these would be addressed by using a keyword. The only objection seems to be "we couldn't find a word we all liked". But that's hardly a good reason to go with pure punctuation. Me, personally, I like "as". It works great for wrappers like classmethod, and is a sufficiently generic word that it wouldn't offend my sense of grammer when used with other kinds of wrappers. But like Jim I'm not strongly in favor of any particular keyword, just against the use of pure punctuation. -- Michael Chermside From rowen at cesmail.net Wed Feb 25 14:38:26 2004 From: rowen at cesmail.net (Russell E. Owen) Date: Wed Feb 25 14:38:32 2004 Subject: [Python-Dev] Status of patch 764217, fix for tkFond.Font(name=)? Message-ID: I submitted patch 764217 awhile ago and it got some intial interest, but after applying the requested changes I've not heard anything for quite awhile. The summary of the patch is: kFont.Font(name=xxx) crashes if a font by the specified name already exists. This is a problem for several reasons, the main one being that it makes life really tough if you want to create a new tkFont.Font object for a given Tcl named font. Is there anything I can do to get this moving again? I don't want to be pushy, but I was certainly hoping to get it into Python 2.4. Regards, -- Russell From aahz at pythoncraft.com Wed Feb 25 14:48:37 2004 From: aahz at pythoncraft.com (Aahz) Date: Wed Feb 25 14:48:43 2004 Subject: [Python-Dev] threading issue: preventing starting twice In-Reply-To: References: Message-ID: <20040225194837.GA5220@panix.com> On Wed, Feb 25, 2004, Russell E. Owen wrote: > > First of all, the documentation only says it is an error to start a > thread more than once. It does not say which exception is raised. On my > system I find it is AssertionError. So...is that going to be true on all > systems? Is the test based on "assert", in which case it will go away > for optimized code (and then what happens)? > > Anyway, if it is always AssertionError, I'd like to get the docs updated > to say that. File a doc bug. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From rowen at cesmail.net Wed Feb 25 15:04:34 2004 From: rowen at cesmail.net (Russell E. Owen) Date: Wed Feb 25 15:04:55 2004 Subject: [Python-Dev] Re: threading issue: preventing starting twice References: <20040225194837.GA5220@panix.com> Message-ID: In article <20040225194837.GA5220@panix.com>, Aahz wrote: > On Wed, Feb 25, 2004, Russell E. Owen wrote: > > > > First of all, the documentation only says it is an error to start a > > thread more than once. It does not say which exception is raised. On my > > system I find it is AssertionError. So...is that going to be true on all > > systems? Is the test based on "assert", in which case it will go away > > for optimized code (and then what happens)? > > > > Anyway, if it is always AssertionError, I'd like to get the docs updated > > to say that. > > File a doc bug. Done. . -- Russell From scav at blueyonder.co.uk Wed Feb 25 16:00:23 2004 From: scav at blueyonder.co.uk (Peter Harris) Date: Wed Feb 25 15:57:46 2004 Subject: [Python-Dev] Re: Python-Dev Digest, Vol 7, Issue 45 In-Reply-To: References: Message-ID: <403D0CE7.9050603@blueyonder.co.uk> Edward Loper wrote: >Peter Harris wrote: > > >>>* curry() >>> >>>* closure() >>> >>>* partial() >>> >>>* partial_apply() >>> >>>* delayed() >>> >>>* other ? >>> >>> > >bind() and bindargs() seem pretty reasonable to me. But if you don't >like those, then how about fixargs() or fix_args()? That gets across >the message that we're generating a new function that fixes the values >of a subset of the arguments. > >Side note: I think that the following code will have unexpected behavior >with the current implementations: > > >>> def f(x,y,z): return (x,y,z) > >>> g = f.bind(y=3) > >>> print g(1,2) >TypeError: f() got multiple values for keyword argument 'y' > >(I would have expected this to return (1,3,2).) > >Of course, there's not really any way around it without using >inspection, and even then you can't really inspect argument lists for >builtins. But it might be worth noting in the docs the circumstances >under which it's appropriate to bind w/ keyword args. > >-Edward > > > Well, I'm coming to like partial(), because it's short enough, and relates to the term 'partial application'. I don't like verb-based names for this, because you are not doing an action so much as creating a thing. A noun would be splendid if an appropriate one could be found; failing that, an adjective will do, as it _implies_ a noun. So partial() evaluates as a 'partial' something, but bind() looks like it's binding something but not necessarily focusing attention on the fact that what we are making is an object. Peter Harris From skip at pobox.com Wed Feb 25 16:02:48 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Feb 25 16:05:06 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <1077737839.403cf96f9e903@mcherm.com> References: <1077737839.403cf96f9e903@mcherm.com> Message-ID: <16445.3448.583343.884960@montanaro.dyndns.org> Michael> The "def name [modifiers]:" is unpythonic because: Michael> * It re-uses the "[]" for a completely new purpose, unrelated Michael> to lists or indexing. Not necessarily. It names a list of functions to be applied to this function to get the final definition (I'll leave it to others to figure out the order of application :-). In particular, you could probably dream up some rather "interesting" list constructors which would allow the programmer (or even the user) to modify the behavior at function definition time: def is_enabled(m, fname, tracers=os.environ.get("TRACERS", "")): "ex: TRACERS=func1:trace,func2:count_lines ; export TRACERS" return tracers.find("%s:%s"%(fname,m.__name__)) != -1 modifiers = [trace, profile, count_lines] def func1(a,b,c) [m for m in modifiers if is_enabled(m, "func1")]: ... def func2(a,b,c) [m for m in modifiers if is_enabled(m, "func2")]: ... that it would necessarily be good to always use such complex constructs, but it might provide some as-yet-unanticipated power for debugging or other magic. If you are going to add [...] as a function modifier syntax, I would argue that it should allow the full power of list (or generator) comprehensions. Michael> * It relies heavily on punctuation (the placement of the Michael> ":"). The placement of the ":" hasn't changed. It's still at the end. A list constructor was just stuffed in between the argument list and the colon. Michael> * It doesn't "read like english". No doubt, but then not everything in Python does. Michael> * There's no good way for a newbie to look it up. How about in the language reference manual section on function definitions? I don't think every bit of the language needs to be tagged with a keyword so that people can more easily search the documentation. How do people find out about operator precedence? How do people learn about function names with leading underscores or leading and trailing double underscores? The language manages to succeed even though the most frequently used function name in Python is probably "__init__" and that the most common sorts of expressions are keyword-free. A newbie won't know off-the-bat what the "__"s mean and even though non-programmers will probably never have seen the "**" operator, they still manage to figure it out. Michael> All of these would be addressed by using a keyword. The only Michael> objection seems to be "we couldn't find a word we all Michael> liked". But that's hardly a good reason to go with pure Michael> punctuation. I think another keyword would be pretty superfluous. Function modifiers would be a fairly advanced feature anyway (I personally have yet to use classmethod or staticmethod), so the extra keyword would probably be unnecessary for most people reading and needing to truly understand a bit of code. If they are deemed not all that advanced, they should probably be discussed in the tutorial. Skip From bob at redivi.com Wed Feb 25 16:21:02 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Feb 25 16:17:51 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16445.3448.583343.884960@montanaro.dyndns.org> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> Message-ID: <834E575B-67D8-11D8-8DBD-000A95686CD8@redivi.com> On Feb 25, 2004, at 4:02 PM, Skip Montanaro wrote: > > Michael> The "def name [modifiers]:" is unpythonic because: > Michael> * It re-uses the "[]" for a completely new purpose, > unrelated > Michael> to lists or indexing. > > Not necessarily. It names a list of functions to be applied to this > function to get the final definition (I'll leave it to others to > figure out > the order of application :-). In particular, you could probably dream > up > some rather "interesting" list constructors which would allow the > programmer > (or even the user) to modify the behavior at function definition time: > > def is_enabled(m, fname, tracers=os.environ.get("TRACERS", "")): > "ex: TRACERS=func1:trace,func2:count_lines ; export TRACERS" > return tracers.find("%s:%s"%(fname,m.__name__)) != -1 > > modifiers = [trace, profile, count_lines] > > def func1(a,b,c) [m for m in modifiers if is_enabled(m, "func1")]: > ... > > def func2(a,b,c) [m for m in modifiers if is_enabled(m, "func2")]: > ... > > that it would necessarily be good to always use such > complex > constructs, but it might provide some as-yet-unanticipated power for > debugging or other magic. If you are going to add [...] as a function > modifier syntax, I would argue that it should allow the full power of > list > (or generator) comprehensions. I agree with you here, but the patch doesn't do that yet. Right now you would need to do something like: def fnseq(lst): def fnseq(fn): for obj in lst: fn = obj(fn) return fn return fnseq def func1(a,b,c) [fnseq([m for m in ...])]: -bob From jim.jewett at eds.com Wed Feb 25 16:43:14 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Wed Feb 25 16:43:56 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: Michael> The "def name [modifiers]:" is unpythonic because: Michael> * It re-uses the "[]" for a completely new purpose, Michael> unrelated to lists or indexing. Skip> Not necessarily. It names a list of functions to be applied Skip> to this function to get the final definition ... Skip> def func1(a,b,c) [m for m in modifiers if is_enabled(m, "func1")]: Skip> If you are going to add [...] as a function modifier syntax, Skip> I would argue that it should allow the full power of list Skip> (or generator) comprehensions. Agreed. And why limit it to lists - why not any expression that evaluates to a list? Or maybe any sequence? Which is one reason that a bare "[...]" doesn't seem sufficient. Michael> * It relies heavily on punctuation (the placement Michael> of the ":"). Skip> The placement of the ":" hasn't changed. It's still at Skip> the end. A list constructor was just stuffed in between Skip> the argument list and the colon. def f(arg) [x] : # wrap f in x def f(arg) : [x] # start f's def with a list of x f(arg) [x] # call f, then select the x'th return value and maybe: def f(arg) x : # wrap f in each element of x def f(arg) g(): # wrap f in each element returned by g() def f(arg) g()[x]: # wrap f in the x'th element returned by g() These are all well-defined for parsing, but I don't think the distinctions are obvious to a human who doesn't already know the syntax. Michael> * There's no good way for a newbie to look it up. Skip> How do people learn about function names with leading underscores Skip> or leading and trailing double underscores? They don't need to, in general. The specific functions that they do need (like __init__) are in the index. Though to be honest, this was a concern at first, when I thought that creating a class would require me to define a large number of methods. If I had been more deeply object-oriented, that might have put me off. Instead, I just started with functions instead of classes. Skip> even though non-programmers will probably never have seen the Skip> "**" operator, they still manage to figure it out. It took me several tries to get it working, though, and even then I was missing some points, so I had more confusing problems later. That isn't a desirable trait in new features. Michael> All of these would be addressed by using a keyword. The only Michael> objection seems to be "we couldn't find a word we all Michael> liked". But that's hardly a good reason to go with pure Michael> punctuation. Skip> Function modifiers would be a fairly advanced feature anyway And so they are a good candidate for: Explicit is better than implicit. Sparse is better than dense. Readability counts. In the face of ambiguity, refuse the temptation to guess. Namespaces are one honking great idea -- let's do more of those! -jJ From guido at python.org Wed Feb 25 16:40:49 2004 From: guido at python.org (Guido van Rossum) Date: Wed Feb 25 16:46:08 2004 Subject: [Python-Dev] threading issue: preventing starting twice In-Reply-To: Your message of "Wed, 25 Feb 2004 11:29:40 PST." References: Message-ID: <200402252140.i1PLeoT08847@guido.python.org> > In the threading module, one must of course only start a thread once. I > have some questions about handling this gracefully. > > First of all, the documentation only says it is an error to start a > thread more than once. It does not say which exception is raised. On my > system I find it is AssertionError. So...is that going to be true on all > systems? Is the test based on "assert", in which case it will go away > for optimized code (and then what happens)? Hey, it's open source! You can look for yourself. :-) Yes, the test is based on assert. > Anyway, if it is always AssertionError, I'd like to get the docs updated > to say that. Disagree. You shouldn't do that. What happens if you ignore the advice needn't be specified. > Also...if possible, it'd be nice to have some way to ask if a thread has > ever been started. More generally, it'd be nice to be able to determine > what state a thread is in. I'm not sure if there are any states other > than ready to run, running and finished. If that's it, perhaps one could > add methods isReady and didRun (a getState method would probably be > better if there are more states, but then one has to deal with magic > constants). There is isRunning(). Isn't that enough? I realize it doesn't distinguish between "not yet running" and "no longer running", but your application state should be enough to distinguish between the two, right? > I realize it is a common Python paradigm to assume something will work > (such as starting the thread) and catch the exception if it doesn't, but: > - this is tricky if the exception is not documented > - sometimes one simply wants to know, without actually starting the > thread What's your use case? Why can't you add application-specific state to track whatever you're interested in? --Guido van Rossum (home page: http://www.python.org/~guido/) From marktrussell at btopenworld.com Wed Feb 25 16:52:41 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Wed Feb 25 17:00:36 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16445.3448.583343.884960@montanaro.dyndns.org> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> Message-ID: <1077745960.1318.5.camel@localhost> On Wed, 2004-02-25 at 21:02, Skip Montanaro wrote: > I think another keyword would be pretty superfluous. Function modifiers > would be a fairly advanced feature anyway (I personally have yet to use > classmethod or staticmethod), so the extra keyword would probably be > unnecessary for most people reading and needing to truly understand a bit of > code. If they are deemed not all that advanced, they should probably be > discussed in the tutorial. Just a data point from a user: I use classmethod fairly frequently, and I would *love* to see some syntax to support it. As has been pointed out before, classmethod (and staticmethod) are a part of the interface of a function, and shouldn't be tacked on to the end of the body. Please let me put them where people (and hopefully pydoc and pychecker) can easily see them. I marginally prefer [] to "as", but I don't have strong feelings either way. Both seem quite readable and pythonic to me. Mark Russell From marktrussell at btopenworld.com Wed Feb 25 16:52:40 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Wed Feb 25 17:00:40 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16445.3448.583343.884960@montanaro.dyndns.org> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> Message-ID: <1077745960.1318.3.camel@localhost> On Wed, 2004-02-25 at 21:02, Skip Montanaro wrote: > I think another keyword would be pretty superfluous. Function modifiers > would be a fairly advanced feature anyway (I personally have yet to use > classmethod or staticmethod), so the extra keyword would probably be > unnecessary for most people reading and needing to truly understand a bit of > code. If they are deemed not all that advanced, they should probably be > discussed in the tutorial. Just a data point from a user: I use classmethod fairly frequently, and I would *love* to see some syntax to support it. As has been pointed out before, classmethod (and staticmethod) are a part of the interface of a function, and shouldn't be tacked on to the end of the body. Please let me put them where people (and hopefully pydoc and pychecker) can easily see them. I marginally prefer [] to "as", but I don't have strong feelings either way. Both seem quite readable and pythonic to me. Mark Russell From bob at redivi.com Wed Feb 25 17:08:33 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Feb 25 17:05:58 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <1077745960.1318.5.camel@localhost> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> Message-ID: <26C7C67C-67DF-11D8-8DBD-000A95686CD8@redivi.com> On Feb 25, 2004, at 4:52 PM, Mark Russell wrote: > On Wed, 2004-02-25 at 21:02, Skip Montanaro wrote: >> I think another keyword would be pretty superfluous. Function >> modifiers >> would be a fairly advanced feature anyway (I personally have yet to >> use >> classmethod or staticmethod), so the extra keyword would probably be >> unnecessary for most people reading and needing to truly understand a >> bit of >> code. If they are deemed not all that advanced, they should probably >> be >> discussed in the tutorial. > > Just a data point from a user: I use classmethod fairly frequently, and > I would *love* to see some syntax to support it. As has been pointed > out before, classmethod (and staticmethod) are a part of the interface > of a function, and shouldn't be tacked on to the end of the body. > Please let me put them where people (and hopefully pydoc and pychecker) > can easily see them. > > I marginally prefer [] to "as", but I don't have strong feelings either > way. Both seem quite readable and pythonic to me. I'm in the same camp, I /need/ something sooner than later to make certain kinds of code usable, I don't care if it's spelled [] or as. -bob From skip at pobox.com Wed Feb 25 17:30:54 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Feb 25 17:31:10 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: Message-ID: <16445.8734.718557.368423@montanaro.dyndns.org> Skip> even though non-programmers will probably never have seen the "**" Skip> operator, they still manage to figure it out. Jim> It took me several tries to get it working, though, and even then I Jim> was missing some points, so I had more confusing problems later. Jim> That isn't a desirable trait in new features. Sorry, I didn't mean func(a1, a2, *args, **kwds) I meant a = b ** 2 Skip> Function modifiers would be a fairly advanced feature anyway Jim> And so they are a good candidate for: ... [ zen bits clipped ] ... I don't follow your reasoning. If anything, features which are expected to be used by more advanced users should require less "COBOL" to be useful. I'm not arguing that the zen of Python shouldn't apply, just that advanced features (which will probably be used a lot less than more basic features) don't necessarily need quite the English-like structure supporting them in the language syntax. Skip From skip at pobox.com Wed Feb 25 17:37:26 2004 From: skip at pobox.com (Skip Montanaro) Date: Wed Feb 25 17:37:38 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <1077745960.1318.5.camel@localhost> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> Message-ID: <16445.9126.829946.723379@montanaro.dyndns.org> Mark> Just a data point from a user: I use classmethod fairly Mark> frequently, and I would *love* to see some syntax to support it. Mark> As has been pointed out before, classmethod (and staticmethod) are Mark> a part of the interface of a function, and shouldn't be tacked on Mark> to the end of the body. Please let me put them where people (and Mark> hopefully pydoc and pychecker) can easily see them. Mark> I marginally prefer [] to "as", but I don't have strong feelings Mark> either way. Both seem quite readable and pythonic to me. I'll add another data point. Quixote'd PTL uses almost this exact syntax to distinguish between PTL functions/methods which return HTML or plain text. The only difference is that it places the annotation before the argument list: def head [html] (title): ''' %(title)s ''' % locals() threecol_css() '' I haven't heard any Quixote users complain about the construct. (I don't know if PTL supports more than one annotation per function or not. I think the only choices are currently "html" or "text", which are obviously mutually exclusive.) Skip From Jack.Jansen at cwi.nl Wed Feb 25 17:49:35 2004 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Wed Feb 25 17:49:37 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <403CCFAC.1090809@laposte.net> References: <403A351B.6060007@laposte.net> <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> <403CCFAC.1090809@laposte.net> Message-ID: On 25 Feb 2004, at 17:39, Jean-Fran?ois Pi?ronne wrote: > > Before I submit a patch, I have done the following update which seem > to fix the problem, any comment are welcome. > > patch for include/fileobject.h > @@ -54,3 +54,7 @@ > */ > +#ifdef __VMS > +#define PY_STDIOTEXTMODE "" > +#else > #define PY_STDIOTEXTMODE "b" > +#endif [...] I wouldn't call it __VMS: if Python still runs on OS/390 or MVS or whatever they'll have the same problem. And I/O system where the underlying filesystem isn't bytestream-based is really what we're talking about, I think. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From marktrussell at btopenworld.com Wed Feb 25 18:33:40 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Wed Feb 25 18:33:45 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16445.9126.829946.723379@montanaro.dyndns.org> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> <16445.9126.829946.723379@montanaro.dyndns.org> Message-ID: <1077752020.1318.39.camel@localhost> On Wed, 2004-02-25 at 22:37, Skip Montanaro wrote: > I'll add another data point. Quixote'd PTL uses almost this exact syntax to > distinguish between PTL functions/methods which return HTML or plain text. > The only difference is that it places the annotation before the argument > list: > > def head [html] (title): I think the reason why this (and the proposed method annotation syntax) is readable is that it reads as a parenthetical note [i.e. something that is intended to convey a bit of extra information]. It would be interesting to show def foo(self) [classmethod]: to a relative python beginner and ask them the guess at the meaning. My hunch is that they would assume that "[classmethod]" modifies the meaning of the function defintion in some way. The curious could then look up classmethod in the manual, where they would presumably find a note about the [] syntax. Mark Russell From andrew-pythondev at puzzling.org Wed Feb 25 18:57:11 2004 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Wed Feb 25 18:57:17 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <1077752020.1318.39.camel@localhost> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> <16445.9126.829946.723379@montanaro.dyndns.org> <1077752020.1318.39.camel@localhost> Message-ID: <20040225235711.GB3669@frobozz> On Wed, Feb 25, 2004 at 11:33:40PM +0000, Mark Russell wrote: [...] > > It would be interesting to show > > def foo(self) [classmethod]: > > to a relative python beginner and ask them the guess at the meaning. My > hunch is that they would assume that "[classmethod]" modifies the > meaning of the function defintion in some way. The curious could then > look up classmethod in the manual, where they would presumably find a > note about the [] syntax. Exactly -- this is why a keyword isn't need, the decorator is the "keyword". Where Java people need to lookup "static" in their language manual, Python people will lookup "staticmethod" in the library reference. The documentation for staticmethod would presumably mention that it's a "function decorator", and link to the appropriate docs, and maybe even give an example. And regardless of "[]" vs. "as", I would expect that a confused user could read the language reference docs for function declarations (i.e. there's already a keyword for them to lookup: def!) to find out what the annotation syntax means. So, as far as I can see, "[]" vs. "as" will make no practical difference to the ease-of-documentation-lookup, so my preference is to go with the syntax that looks subjectively better to my eyes -- which is []. I don't mind "as" either, though. -Andrew. From edloper at gradient.cis.upenn.edu Wed Feb 25 19:11:06 2004 From: edloper at gradient.cis.upenn.edu (Edward Loper) Date: Wed Feb 25 19:09:48 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: Message-ID: <403D399A.8040903@gradient.cis.upenn.edu> Jim Jewett said: > def f(arg) [x] : # wrap f in x > def f(arg) : [x] # start f's def with a list of x > > [...] > > These are all well-defined for parsing, but I don't think the > distinctions are obvious to a human who doesn't already know > the syntax. This point has come up a couple times, and I don't think it's really a valid complaint, because... * You won't get confused when reading it: No one will ever write the second one (f's def consists of a single list), because it's not a useful function. Has anyone ever actually seen a function like this? So when you're reading the code, you're very unlikely to get confused. This is especially true because there will be an indented body under the f(arg), so the [x] can't possibly be part of the body. * If you write the wrong thing, it's easy to detect: If you accidentally put the [x] after the colon, then you'll get a parse error, since you'll *also* have a body. The current parse error might be a little cryptic (just a syntax error at the first line of the body), but if necessary the parser could be special- cased to give a more specific error there. Some of the other arguments against the new syntax seem reasonable to me, and I'm not yet decided whether I'm in favor of the syntax or not. But I just wanted to point out that I don't find this particular complaint against it very compelling. -Edward From paul at prescod.net Wed Feb 25 19:05:50 2004 From: paul at prescod.net (Paul Prescod) Date: Wed Feb 25 19:23:42 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225235711.GB3669@frobozz> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> <16445.9126.829946.723379@montanaro.dyndns.org> <1077752020.1318.39.camel@localhost> <20040225235711.GB3669@frobozz> Message-ID: <403D385E.90405@prescod.net> I hate to get into this loop where a hundred syntax options are discussed, most of them having been disgarded before, but why not use angle-brackets rather than square-brackets? def foo(self) : pass Paul Prescod From barry at python.org Wed Feb 25 21:18:36 2004 From: barry at python.org (Barry Warsaw) Date: Wed Feb 25 21:18:40 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403D385E.90405@prescod.net> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> <16445.9126.829946.723379@montanaro.dyndns.org> <1077752020.1318.39.camel@localhost> <20040225235711.GB3669@frobozz> <403D385E.90405@prescod.net> Message-ID: <1077761916.31165.29.camel@geddy.wooz.org> On Wed, 2004-02-25 at 19:05, Paul Prescod wrote: > I hate to get into this loop where a hundred syntax options are > discussed, most of them having been disgarded before, but why not use > angle-brackets rather than square-brackets? > > def foo(self) : > pass Of course, you meant << and >> right? :) -Barry From barry at python.org Wed Feb 25 21:23:31 2004 From: barry at python.org (Barry Warsaw) Date: Wed Feb 25 21:23:34 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225235711.GB3669@frobozz> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> <16445.9126.829946.723379@montanaro.dyndns.org> <1077752020.1318.39.camel@localhost> <20040225235711.GB3669@frobozz> Message-ID: <1077762210.31165.37.camel@geddy.wooz.org> On Wed, 2004-02-25 at 18:57, Andrew Bennetts wrote: > Exactly -- this is why a keyword isn't need, the decorator is the "keyword". Excellent point. The other thing about using square brackets syntactically is that they allow for multiple line constructs without resorting to backslashes. def foo(a, b, c) [firstdecorator, seconddecorator, thirddecorator]: vs. def foo(a, b, c) as firstdecorator, \ seconddecorator, \ thirddecorator: -Barry From jcarlson at uci.edu Wed Feb 25 21:56:25 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed Feb 25 21:59:37 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <1077745960.1318.3.camel@localhost> References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> Message-ID: <20040225180723.1F8E.JCARLSON@uci.edu> > Just a data point from a user: I use classmethod fairly frequently, and > I would *love* to see some syntax to support it. As has been pointed > out before, classmethod (and staticmethod) are a part of the interface > of a function, and shouldn't be tacked on to the end of the body. > Please let me put them where people (and hopefully pydoc and pychecker) > can easily see them. I personally don't see the point of using any syntax for wrapping a function like so: class a: def funct(self): pass a = classmethod(a) If we actually test this: >>> class a: ... def a(self): ... pass ... def b(self): ... pass ... def c(self): ... pass ... a = classmethod(a) ... b = staticmethod(b) ... >>> foo = a() >>> type(foo.a) >>> type(foo.b) >>> type(foo.c) It is entirely possible that I'm missing something, but according to Python 2.3.2, foo.a and foo.c are the same kind of thing, who am I to disagree? For another data point, I've never had to use staticmethod or classmethod in my code, and have never needed to use function decorators. In general, I am against the use of arbitrary syntax like the following; class foo: def a(self) [...]: #body def b(self) <...>: #as paul recently suggested #body Mostly because it leads to newbies asking what the hell it means. You only need to check the c.l.py newsgroup to discover 3-4 messages/week asking about the meaning of *args and **kwargs. If either of the above were allowed, new (and seasoned users for a while) would ask, "what does this thing mean, I've never seen anything like it before?" Honesly, a new (or used) keyword would make the most sense, perhaps "as", which has been offered before. At least then people could check the documentation index for "as" and get two listings, one for import statements, and one for decorator syntax. I think something like the following would make the most sense (it includes the opportunity for multi-line decorators as Barry suggests). class foo: def a(self) as [...]: #body Even though I think the above is the best syntax I've seen so far, that doesn't mean that I like it. Personally, I don't have any use for decorators currently, and could see new and old users of Python being stymied by examples of this in new code. Being able to reverse the application order of the decorators, and still have it make sense to a not-insignificant number of people, means that it could be confusing. I understand its application in PyObjC and in the use of staticmethod, but for the vast majority of users, I doubt that adding it would result in use. Certainly it seems like a chicken and egg problem (there are a few of those being discussed in c.l.py currently), but in all the code I've read "in the wild", I've never once seen anyone manually apply decorators to functions. I suggest a supporter of the PEP take a look through the standard library. If there exists a non-trivial number of examples of its use, maybe my opinion would change. On the other hand, if in the standard library, there exists some _very small number_ of examples of manual decorators, then perhaps this is application-specific. I have a feeling that including application-specific /syntax/ in Python is frowned upon. - Josiah From barry at python.org Wed Feb 25 22:39:42 2004 From: barry at python.org (Barry Warsaw) Date: Wed Feb 25 22:39:45 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225180723.1F8E.JCARLSON@uci.edu> References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: <1077766781.31165.60.camel@geddy.wooz.org> On Wed, 2004-02-25 at 21:56, Josiah Carlson wrote: > I personally don't see the point of using any syntax for wrapping a > function like so: > > class a: > def funct(self): > pass > a = classmethod(a) In a short example like this, you're right that it may not help much. But I've written a bunch of code that uses staticmethod or transformations of (some) methods in a class. When your method is three quarters of a page long, the separation between the function definition and the decoration is a readability burden. What I tend to do in that case is name the function with an underscore and use the 'public' name for th wrapped attribute. This is still less readable that it should be because that little "foo = wrapper(_foo)" line can easily get lost. I've rewritten some of my code to use both the [] and the 'as' syntax. Neither is ideal, primarily because the interesting information tends to get pushed way off to the right, where I think the eye is lazier because of the more ragged nature of "code right". Putting the decorator between the def and the method name actually doesn't look so bad to me, e.g.: def [classmethod] func(a, b, c): but I think with a longer decorator list, it might be problematic. I think that one element decorator lists will be the most common use and in that case, I like this syntax because it almost reads like English. E.g. "define a classmethod called func taking arguments a, b, and c". > Honesly, a new (or used) keyword would make the most sense, perhaps "as", > which has been offered before. At least then people could check the > documentation index for "as" and get two listings, one for import > statements, and one for decorator syntax. I agree with whoever said that people aren't going to look up 'as', they are going to look up 'def' when trying to understand what this stuff means. > Even though I think the above is the best syntax I've seen so far, that > doesn't mean that I like it. Personally, I don't have any use for > decorators currently, and could see new and old users of Python being > stymied by examples of this in new code. Actually stuff 'like this' makes for some much more elegant Python idioms. I think the features are just so new that it hasn't really made its way into much released production code. I think Python 2.4 is the right time frame for improving the utility of this very neat feature. -Barry From bob at redivi.com Wed Feb 25 22:51:24 2004 From: bob at redivi.com (Bob Ippolito) Date: Wed Feb 25 22:48:19 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225180723.1F8E.JCARLSON@uci.edu> References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: <0BE9732A-680F-11D8-8DBD-000A95686CD8@redivi.com> On Feb 25, 2004, at 9:56 PM, Josiah Carlson wrote: >> Just a data point from a user: I use classmethod fairly frequently, >> and >> I would *love* to see some syntax to support it. As has been pointed >> out before, classmethod (and staticmethod) are a part of the interface >> of a function, and shouldn't be tacked on to the end of the body. >> Please let me put them where people (and hopefully pydoc and >> pychecker) >> can easily see them. > > I personally don't see the point of using any syntax for wrapping a > function like so: > > class a: > def funct(self): > pass > a = classmethod(a) This just isn't valid Python code.. but anyways :) > If we actually test this: > >>>> class a: > ... def a(self): > ... pass > ... def b(self): > ... pass > ... def c(self): > ... pass > ... a = classmethod(a) > ... b = staticmethod(b) > ... >>>> foo = a() >>>> type(foo.a) > >>>> type(foo.b) > >>>> type(foo.c) > > > It is entirely possible that I'm missing something, but according to > Python 2.3.2, foo.a and foo.c are the same kind of thing, who am I to > disagree? What does the type(object) have to do with anything? I thought people have learned by now that it's not wise to use type for much of anything. You should instead opt for (at least) isinstance(..), getattr(...) or preferably a more robust adaptation solution such as Zope X3's components, Twisted's components, or PyProtocols (a subproject of PEAK). > For another data point, I've never had to use staticmethod or > classmethod in my code, and have never needed to use function > decorators. I never had to use Python either, but I think it's a great language that let's me work more effectively. There's nothing I can do in Python that I couldn't have done in another language, well, except enjoy writing code :) > In general, I am against the use of arbitrary syntax like the > following; > > class foo: > def a(self) [...]: > #body > def b(self) <...>: #as paul recently suggested > #body > > Mostly because it leads to newbies asking what the hell it means. You > only need to check the c.l.py newsgroup to discover 3-4 messages/week > asking about the meaning of *args and **kwargs. If either of the above > were allowed, new (and seasoned users for a while) would ask, "what > does > this thing mean, I've never seen anything like it before?" > > Honesly, a new (or used) keyword would make the most sense, perhaps > "as", > which has been offered before. At least then people could check the > documentation index for "as" and get two listings, one for import > statements, and one for decorator syntax. I think something like the > following would make the most sense (it includes the opportunity for > multi-line decorators as Barry suggests). > > class foo: > def a(self) as [...]: > #body > > > Even though I think the above is the best syntax I've seen so far, that > doesn't mean that I like it. Personally, I don't have any use for > decorators currently, and could see new and old users of Python being > stymied by examples of this in new code. Being able to reverse the > application order of the decorators, and still have it make sense to a > not-insignificant number of people, means that it could be confusing. I am fine with the "as" keyword. I don't really know how 'newbies' think, and an extra 4 characters for clarity doesn't bother me at all. Not having this syntax requires A WHOLE !@#$*( lot of verbosity and repetition, or drastic measures such as alternative compilers (Quixote) or really awful terrible introspection hacks (PEAK does stack introspection). I don't really care, as long as it makes usage much shorter than it is currently. I need this syntax, and I will maintain my own patched version of Python or alternate compiler if I have to (which is less effort than writing code that has to use existing grammar), but I would VERY much rather not. > I understand its application in PyObjC and in the use of staticmethod, > but for the vast majority of users, I doubt that adding it would > result in use. Certainly it seems like a chicken and egg problem > (there > are a few of those being discussed in c.l.py currently), but in all the > code I've read "in the wild", I've never once seen anyone manually > apply > decorators to functions. The vast majority of Python programmers either learned Python before it had decorators, hasn't read the documentation for it, or just aren't writing the kind of code that needs it. We have these great new style classes now, but it's hard to make any good use of them if it's not simple to create descriptors and tag things with metadata. > I suggest a supporter of the PEP take a look through the standard > library. If there exists a non-trivial number of examples of its use, > maybe my opinion would change. On the other hand, if in the standard > library, there exists some _very small number_ of examples of manual > decorators, then perhaps this is application-specific. I have a > feeling > that including application-specific /syntax/ in Python is frowned upon. This isn't at all fair, most (almost all?) of the standard library was written before Python had any decorators, by people who wanted backwards compatibility (for the most part). Additionally, the current grammar makes decorators so !@#$ inconvenient that nobody wants to use them unless they have to. Look no further than any PEAK or PyObjC project to see plenty of decorators. Especially PEAK. Off the top of my head, this new syntax would be extremely convenient for *at least* the following projects and software that uses them: ctypes Zope X3 Twisted PEAK / PyProtocols PyObjC (these two are guesses, I can't say this from experience) SQLObject Quixote PyObjC and ctypes need the syntax the most, because both projects attempt to make low-level libraries easier to extend and use by creating a domain specific languages within Python. Currently, this is only a partial success because defining functions and methods is actually harder than in ObjC or C respectively due to this severe limitation in Python's grammar. These projects could be extremely popular if they were nicer to use. As far as numbers go, I guarantee you that once this syntax exists, people (at least anyone who uses descriptors and doesn't need backwards compatibility) will start using it in droves. If not "vast majority users", at least framework/library writers... you know, the people writing the software that gets new users and business types excited about Python. As a bonus, some big abusers of lambda such as PEAK (~333 uses), Twisted (~268 uses), and the standard library (~295 uses) may end up using this syntax instead, because it covers some of the use cases where what you want is really some kind of "anonymous code block". -bob From jepler at unpythonic.net Wed Feb 25 23:03:15 2004 From: jepler at unpythonic.net (Jeff Epler) Date: Wed Feb 25 23:04:02 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403D385E.90405@prescod.net> References: <1077737839.403cf96f9e903@mcherm.com> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.5.camel@localhost> <16445.9126.829946.723379@montanaro.dyndns.org> <1077752020.1318.39.camel@localhost> <20040225235711.GB3669@frobozz> <403D385E.90405@prescod.net> Message-ID: <20040226040315.GB24013@unpythonic.net> On Wed, Feb 25, 2004 at 04:05:50PM -0800, Paul Prescod wrote: > I hate to get into this loop where a hundred syntax options are > discussed, most of them having been disgarded before, but why not use > angle-brackets rather than square-brackets? > > def foo(self) : I haven't yet used an editor that matched angle-brackets well, especially since they are usually binary operators, not grouping characters. Think "if a < b and c > d:"... Jeff From paul at prescod.net Thu Feb 26 00:07:48 2004 From: paul at prescod.net (Paul Prescod) Date: Thu Feb 26 00:13:43 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225180723.1F8E.JCARLSON@uci.edu> References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: <403D7F24.4010801@prescod.net> Josiah Carlson wrote: >... > > I suggest a supporter of the PEP take a look through the standard > library. If there exists a non-trivial number of examples of its use, > maybe my opinion would change. On the other hand, if in the standard > library, there exists some _very small number_ of examples of manual > decorators, then perhaps this is application-specific. I have a feeling > that including application-specific /syntax/ in Python is frowned upon. Here are the "applications" I've used that I think could benefit from the syntax: * Pyrex type declaration syntax (extends the Python syntax) * * PTL (extends the Python syntax and docstrings) * http://www.mems-exchange.org/software/quixote/doc/PTL.html * SPARK (abuses docstrings) * http://gnosis.cx/publish/programming/charming_python_b6.html * Grouch (abuses class docstrings) * http://www.mems-exchange.org/software/grouch/ * XPath callbacks (abuses docstrings) * http://mail.python.org/pipermail/xml-sig/2000-March/002008.html * Python COM (and XPCOM) servers (separate metadata from functions) * http://mail.python.org/pipermail/python-list/2001-December/075488.html * Zope ZPublisher * http://www.zope.org/Members/Amos/ExtendingZopeSlides.html#11 When a feature is emulated over and over that's a strong hint that the industry is asking for first-class support. Paul Prescod From shalabh at cafepy.com Thu Feb 26 01:58:54 2004 From: shalabh at cafepy.com (Shalabh Chaturvedi) Date: Thu Feb 26 01:58:58 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> Hello python-dev people: Just joined, intro below :) Barry Warsaw wrote: > I've rewritten some of my code to use both the [] and the 'as' syntax. > Neither is ideal, primarily because the interesting information tends to > get pushed way off to the right, where I think the eye is lazier because > of the more ragged nature of "code right". > > Putting the decorator between the def and the method name actually > doesn't look so bad to me, e.g.: > > def [classmethod] func(a, b, c): > > but I think with a longer decorator list, it might be problematic. I > think that one element decorator lists will be the most common use and > in that case, I like this syntax because it almost reads like English. I completely agree. This is the second-most elegant syntax, the most-elegant being: def classmethod func(a, b, c): Has this been completely rejected? The pep's argument is it becomes cluttered and obscure when the decorator is a function call: def synchronized(lock) classmethod func(a): Can we also require that decorators be identifiers only (no expressions allowed), forcing clean looking definitions? The above would be rewritten as: sync = synchronized(lock) def sync classmethod func(a): In fact sync would probably be reused throughout the class body. This keeps simple things simple, and complex things possible. In addition it encourages using few decorators, keeping the def line short. And did I mention no new keywords? The alternatives - using [], (), <> and/or expressions - only make it more cluttered, leaving the simple (and IMO most common) cases hard to read and write. They do provide power that is useful for specific domains, but unnecessary for general purpose programming. Which would be ok (since when is power a bad thing?) if it wasn't at the cost of readability. Cheers, Shalabh Chaturvedi PS: I am a software consultant and have been using Python for four years (which includes two years of full time development :-). Needless to say, it is my first choice for solving problems. Recently I have become more interested in development and promotion of Python. From fincher.8 at osu.edu Thu Feb 26 03:31:54 2004 From: fincher.8 at osu.edu (Jeremy Fincher) Date: Thu Feb 26 02:34:09 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> Message-ID: <200402260331.54818.fincher.8@osu.edu> On Thursday 26 February 2004 01:58 am, Shalabh Chaturvedi wrote: > > Putting the decorator between the def and the method name actually > > doesn't look so bad to me, e.g.: > > > > def [classmethod] func(a, b, c): > > > > but I think with a longer decorator list, it might be problematic. I > > think that one element decorator lists will be the most common use and > > in that case, I like this syntax because it almost reads like English. > > > I completely agree. This is the second-most elegant syntax, the > most-elegant being: > > def classmethod func(a, b, c): I also much prefer this syntax to any other suggested syntax. Perhaps it's because I'm used to C (or practically any other language with type declarations), which puts type declarations ("restraints") before the defined function rather than after. > Has this been completely rejected? The pep's argument is it becomes > cluttered and obscure when the decorator is a function call: > > def synchronized(lock) classmethod func(a): > > Can we also require that decorators be identifiers only (no expressions > allowed), forcing clean looking definitions? This would be fine with me. I would sacrifice the slight inconvenience of having to write give something a name in order to have what I believe to be significantly more readable code (much like I do with lambdas: I sacrifice the convenience of having an unnamed function on one line of code in order to maintain readability). > sync = synchronized(lock) > def sync classmethod func(a): This is much more readable, IMO, than: def func(a) [synchronized(lock), classmethod]: Jeremy From jcarlson at uci.edu Thu Feb 26 02:43:46 2004 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu Feb 26 02:46:48 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403D7F24.4010801@prescod.net> References: <20040225180723.1F8E.JCARLSON@uci.edu> <403D7F24.4010801@prescod.net> Message-ID: <20040225213942.1F95.JCARLSON@uci.edu> > Here are the "applications" I've used that I think could benefit from > the syntax: [snip...] You (and others) have convinced me. There exists a reasonably large number of use-cases, even if I don't have one. The real question is whether everyone can decide on a good syntax. I will make one request; almost anything but the below. def staticmethod decorator1 decorator2 decorator3 funct(arg): It makes finding the name of the function more work than it should be. - Josiah From pete at shinners.org Thu Feb 26 03:07:17 2004 From: pete at shinners.org (Pete Shinners) Date: Thu Feb 26 03:07:40 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225180723.1F8E.JCARLSON@uci.edu> References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: I'm not too concerned about the syntax. All current proposals weigh about equal to me. Although I don't believe "as" makes documentation lookup much easier. I think this feature will be another one of those positive changes that effects the entire way python is used. The kind of thing like iterators, string methods, etc. I've been playing with the idea for awhile and can throw out some extra use cases. Some of these may even be bad ideas apon further thought, but it's something to ponder. def main() [__main__]: """specify a function to be run if __name__=='__main__'. this would need some other python tricks to actually work.""" def cleanup() [sys.atexit]: """ensures this function is called when program quits.""" def handler(event) [self.button.slot('clicked')]: """automatically make this function a callback for some gui slot/signal mechanism.""" def food(arg) [pysco.optimize]: """specifically control functions that psyco takes over.""" def bard(arg, gar) [validate(int, str)]: """confirm arguments match a type. validate would have to be a function that returns a function. odd.""" def corn() [profiler.track]: """log specific runtime stats on this function.""" def sanitytest() [sys.debugfunc]: """function is ignored at runtime if python not in "-O" optimize mode (something like assert).""" def entrypoint(data) [__all__]: def hidden(data) [__hidden__]: """select functions for hiding or passing to 'import *'.""" I'm sure others will come to me as soon as I hit send. I don't know if I get a vote, but I'm +2 on PEP 318. From bob at redivi.com Thu Feb 26 03:16:01 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 03:12:43 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225213942.1F95.JCARLSON@uci.edu> References: <20040225180723.1F8E.JCARLSON@uci.edu> <403D7F24.4010801@prescod.net> <20040225213942.1F95.JCARLSON@uci.edu> Message-ID: <0390CFF4-6834-11D8-8DBD-000A95686CD8@redivi.com> On Feb 26, 2004, at 2:43 AM, Josiah Carlson wrote: >> Here are the "applications" I've used that I think could benefit from >> the syntax: > > [snip...] > > You (and others) have convinced me. There exists a reasonably large > number of use-cases, even if I don't have one. The real question is > whether everyone can decide on a good syntax. I will make one request; > almost anything but the below. Great :) > def staticmethod decorator1 decorator2 decorator3 funct(arg): > > It makes finding the name of the function more work than it should be. I prefer them after the argument list as well. -bob From Paul.Moore at atosorigin.com Thu Feb 26 04:20:39 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Thu Feb 26 04:20:46 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <16E1010E4581B049ABC51D4975CEDB8802C09CE3@UKDCX001.uk.int.atosorigin.com> From: Jewett, Jim J > Agreed. And why limit it to lists - why not any expression that > evaluates to a list? Or maybe any sequence? Which is one reason > that a bare "[...]" doesn't seem sufficient. Overgeneralisation without good use cases. Leave it as simple lists. Bob Ippolito just posted a pseq function which will do this for the (extremely rare) cases when it might be needed. > They don't need to, in general. The specific functions that they > do need (like __init__) are in the index. Though to be honest, > this was a concern at first, when I thought that creating a class would > require me to define a large number of methods. If I had been more > deeply object-oriented, that might have put me off. Instead, I just > started with functions instead of classes. Exactly. As a beginner, classes were more advanced than you needed. So the (relative) difficulty of finding the relevant details wasn't an issue - you just didn't use the feature for a while. By the time you needed classes, you were familiar enough with Python to know where to look. Heck, by the time most people will need the full power of the wrapping feature [1], they will probably know enough to understand the C code of the patch itself! It's pretty advanced usage, remember... [1] Except for extremely stylised cases like [classmethod], which will be for particular use cases - and you can look that one up under "classmethod"... Paul. From Paul.Moore at atosorigin.com Thu Feb 26 04:26:30 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Thu Feb 26 04:26:38 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <16E1010E4581B049ABC51D4975CEDB8802C09CE4@UKDCX001.uk.int.atosorigin.com> From: Andrew Bennetts > Exactly -- this is why a keyword isn't need, the decorator is the "keyword". > Where Java people need to lookup "static" in their language manual, Python > people will lookup "staticmethod" in the library reference. The > documentation for staticmethod would presumably mention that it's a "function > decorator", and link to the appropriate docs, and maybe even give an example. This is a very good point. Assuming the "as" keyword, any user who saw def f(a, b, c) as classmethod: .... and looked up "as" rather than "classmethod" has clearly been having a bad day. Who, as a new user, saw "import something as another" and looked up "as" rather than "import"? Go on - admit it. You're among friends :-) Paul. From marktrussell at btopenworld.com Thu Feb 26 05:18:20 2004 From: marktrussell at btopenworld.com (Mark Russell) Date: Thu Feb 26 05:18:26 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <200402260331.54818.fincher.8@osu.edu> References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> <200402260331.54818.fincher.8@osu.edu> Message-ID: <1077790693.785.6.camel@localhost> On Thu, 2004-02-26 at 08:31, Jeremy Fincher wrote: > > sync = synchronized(lock) > > def sync classmethod func(a): > > This is much more readable, IMO, than: > > def func(a) [synchronized(lock), classmethod]: Odd - I feel the exact opposite. The first form looks like "def blah blah blah" to me - there's nothing obvious to say *what* is being defined. By contrast the second form clearly defines "func" and gives some extra information about it. The name and parameters are more important than the additional information, so they ought to come first. Mark Russell From paul at prescod.net Thu Feb 26 05:31:51 2004 From: paul at prescod.net (Paul Prescod) Date: Thu Feb 26 05:35:27 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> Message-ID: <403DCB17.4050001@prescod.net> Shalabh Chaturvedi wrote: > ... > > sync = synchronized(lock) > def sync classmethod func(a): > > In fact sync would probably be reused throughout > the class body. In this case yes. But there are other usecases where it isn't as hot (pseudo Spark): grammar_rule = grammar("x := y z?") def grammar_rule func1(a): pass grammar_rule = grammar("y := y1 y2?") def grammar_rule func2(a): pass grammar_rule = grammar("z := z2? z1") def grammar_rule func3(a): pass But it isn't terrible. It just means you need to create meaningless temporaries. It also makes life a little harder for tools trying to read the code and figure out what is going on. This is an easy pattern to look for: def sync func(a)[synchronized(lock)]: For instance an IDE could make a tooltip wherever "func" is used to indicate that it is synchronized with a particular lock. That gets much harder if there are usually arbitrary amounts of Turing complete code between the function declaration and the decorator declaration. Another concern: It arguably HURTS readability if deciphering a function declaration requires scanning backwards looking for definitions like the one for "sync". I certainly see the strength of your proposal from a readability point of view. Paul Prescod From paul at prescod.net Thu Feb 26 05:47:12 2004 From: paul at prescod.net (Paul Prescod) Date: Thu Feb 26 05:50:21 2004 Subject: [Python-Dev] PEP 318: what C# does In-Reply-To: <1077790693.785.6.camel@localhost> References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> <200402260331.54818.fincher.8@osu.edu> <1077790693.785.6.camel@localhost> Message-ID: <403DCEB0.6090409@prescod.net> Here's what C# does: [System.Xml.Serialization.XmlRootAttribute("bookstore", Namespace="", IsNullable=false)] public class bookstoreType { .... [System.Xml.Serialization.XmlElementAttribute("book")] public bookType[] book; } One thing to note is that C# allows the wrapping of attributes, not just functions and classes. In Python this could be accomplished if we used delimiters that didn't look like lists (e.g. <> or [! !]). C# distinguishes between bareword declarations like "public" and "int" and metadata attachments like XML serialization information. I feel like there is a similar distinction lurking in the Python world. "staticmethod" says that the thing _is_ a static method. It changes the calling convention for that function. Whereas a "public" or "ZopePublish" attribute is pure metadata. It doesn't change the way you call the function, just who can call it. We could combine the C# idea with the bareword idea like so: def staticmethod x(): ... Statements cannot start with "<" so there is no ambiguity about the meaning of the character in this context. Paul Prescod From paul at prescod.net Thu Feb 26 05:50:42 2004 From: paul at prescod.net (Paul Prescod) Date: Thu Feb 26 05:55:09 2004 Subject: [Python-Dev] PEP 318: What Java does Message-ID: <403DCF82.4010409@prescod.net> @debug(devbuild=production,counter=1) public void testMethod() { } public @remote void ping() { } The Java way does make clear what is a language builtin ("public", "void") and what is extension metadata "@remote", "@debug". That may not be a bad thing... Paul Prescod From ncoghlan at iinet.net.au Thu Feb 26 06:25:59 2004 From: ncoghlan at iinet.net.au (Nick Coghlan) Date: Thu Feb 26 06:26:06 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09CE3@UKDCX001.uk.int.atosorigin.com> References: <16E1010E4581B049ABC51D4975CEDB8802C09CE3@UKDCX001.uk.int.atosorigin.com> Message-ID: <403DD7C7.5060509@iinet.net.au> Moore, Paul wrote: > [1] Except for extremely stylised cases like [classmethod], which will > be for particular use cases - and you can look that one up under > "classmethod"... I find this an interesting point - almost certainly, the introduction process for this syntax is going to be via one of the decorators in the standard library (classmethod, staticmethod, ?), which can be looked up directly. The write-ups for these would include a link to what a function decorator _is_, so that people can then learn that Python function decorators don't restrict you to whatever modifiers the language designers felt like providing (ala C, C++, Java etc). (Odd thought for the evening: def myFunc (arg) [private]: pass where 'private' wraps a function in some code that does some introspection to give you a _real_ private function, that refuses to be called from outside its defining context. Slow as hell, and the psuedo-privacy of name-mangling is generally enough, but just something that occured to me) Regards, Nick. -- Nick Coghlan | Brisbane, Australia Email: ncoghlan@email.com | Mobile: +61 409 573 268 From mwh at python.net Thu Feb 26 07:08:01 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 26 07:08:04 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: (Pete Shinners's message of "Thu, 26 Feb 2004 00:07:17 -0800") References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: <2m8yiqoxge.fsf@starship.python.net> Pete Shinners writes: > I'm not too concerned about the syntax. All current proposals weigh > about equal to me. Although I don't believe "as" makes documentation > lookup much easier. > > I think this feature will be another one of those positive changes > that effects the entire way python is used. The kind of thing like > iterators, string methods, etc. I've been playing with the idea for > awhile and can throw out some extra use cases. > > Some of these may even be bad ideas apon further thought, but it's > something to ponder. Actually, I think one of the downsides to the function decorator proposals, or at least a point that needs thought, is some of the loony things you can do with it :-) (and, like you say, the fact that this might well turn out to be a much used change). def pi_over_two() [apply]: return math.pi/2 Cheers, mwh -- A difference which makes no difference is no difference at all. -- William James (I think. Reference anyone?) From mwh at python.net Thu Feb 26 07:12:29 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 26 07:12:33 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> (Shalabh Chaturvedi's message of "Wed, 25 Feb 2004 22:58:54 -0800 (PST)") References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> Message-ID: <2m4qteox8y.fsf@starship.python.net> "Shalabh Chaturvedi" writes: > I completely agree. This is the second-most elegant syntax, the > most-elegant being: > > def classmethod func(a, b, c): > > Has this been completely rejected? I think it would be pretty hard to implement, parsing wise. Also, I think it risks being NAME soup. Punctuation isn't in-and-of- itself a bad thing! Cheers, mwh -- : exploding like a turd Never had that happen to me, I have to admit. They do that often in your world? -- Eric The Read & Dave Brown, asr From barry at python.org Thu Feb 26 08:20:43 2004 From: barry at python.org (Barry Warsaw) Date: Thu Feb 26 08:20:59 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225213942.1F95.JCARLSON@uci.edu> References: <20040225180723.1F8E.JCARLSON@uci.edu> <403D7F24.4010801@prescod.net> <20040225213942.1F95.JCARLSON@uci.edu> Message-ID: <1077801641.14349.46.camel@geddy.wooz.org> On Thu, 2004-02-26 at 02:43, Josiah Carlson wrote: > You (and others) have convinced me. There exists a reasonably large > number of use-cases, even if I don't have one. The real question is > whether everyone can decide on a good syntax. We won't, so if the PEP's accepted, the BDFL will do it for us. That's why we keep him around . -Barry From barry at python.org Thu Feb 26 08:23:00 2004 From: barry at python.org (Barry Warsaw) Date: Thu Feb 26 08:23:05 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <200402260331.54818.fincher.8@osu.edu> References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> <200402260331.54818.fincher.8@osu.edu> Message-ID: <1077801778.14349.49.camel@geddy.wooz.org> On Thu, 2004-02-26 at 03:31, Jeremy Fincher wrote: > efer this syntax to any other suggested syntax. Perhaps it's > because I'm used to C (or practically any other language with type > declarations), which puts type declarations ("restraints") before the defined > function rather than after. I think I still like the pre--function-name-bracket-bracketed-decorator syntax because of the English-like parsing combined with the visual (parenthetical) distinction. -Barry From barry at python.org Thu Feb 26 08:26:33 2004 From: barry at python.org (Barry Warsaw) Date: Thu Feb 26 08:26:39 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8802C09CE3@UKDCX001.uk.int.atosorigin.com> References: <16E1010E4581B049ABC51D4975CEDB8802C09CE3@UKDCX001.uk.int.atosorigin.com> Message-ID: <1077801992.14349.52.camel@geddy.wooz.org> On Thu, 2004-02-26 at 04:20, Moore, Paul wrote: > > Heck, by the time most people will need the full power of the wrapping > feature [1], they will probably know enough to understand the C code of > the patch itself! It's pretty advanced usage, remember... > > [1] Except for extremely stylised cases like [classmethod], which will > be for particular use cases - and you can look that one up under > "classmethod"... Nothing in PEP 318 is going to make the current way of wrapping illegal, so I would think that a beginning class might make life simpler by first teaching the post-method wrapping idiom. They might then explain how there's this neat (more advanced) decorator syntax for making things more succinct. -Barry From Paul.Moore at atosorigin.com Thu Feb 26 08:48:07 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Thu Feb 26 08:48:08 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060E05@UKDCX001.uk.int.atosorigin.com> From: Barry Warsaw > On Thu, 2004-02-26 at 02:43, Josiah Carlson wrote: > >> You (and others) have convinced me. There exists a reasonably large >> number of use-cases, even if I don't have one. The real question is >> whether everyone can decide on a good syntax. > > We won't, so if the PEP's accepted, the BDFL will do it for us. That's > why we keep him around . The PEP doesn't specify where discussion should take place, but the author hasn't participated here. At some point, he (or someone else) is going to have to put the PEP forward for acceptance. At the moment, the PEP appears fairly strongly in favour of "as" and against the use of brackets. But the author admits that he has not got the expertise to provide an implementation. So where do we go from here? Is anyone willing to modify MWH's patch to implement the "as" syntax, resulting in a PEP with associated implementation which can be submitted for approval? Should the PEP be changed to propose a syntax that matches the existing patch? I've copied the PEP author, in the hope that he can share his views. Paul. From mwh at python.net Thu Feb 26 09:16:33 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 26 09:16:36 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060E05@UKDCX001.uk.int.atosorigin.com> (Paul Moore's message of "Thu, 26 Feb 2004 13:48:07 -0000") References: <16E1010E4581B049ABC51D4975CEDB8803060E05@UKDCX001.uk.int.atosorigin.com> Message-ID: <2mznb6ncxq.fsf@starship.python.net> "Moore, Paul" writes: > So where do we go from here? Is anyone willing to modify MWH's patch to > implement the "as" syntax, resulting in a PEP with associated > implementation which can be submitted for approval? Should the PEP be > changed to propose a syntax that matches the existing patch? If Guido is in faviour of the 'as' variant, I can do it. There's more to it than that, though, as PEP 306 explains. Bob Ippolito said he would help with what remains. Cheers, mwh -- Very clever implementation techniques are required to implement this insanity correctly and usefully, not to mention that code written with this feature used and abused east and west is exceptionally exciting to debug. -- Erik Naggum on Algol-style "call-by-name" From skip at pobox.com Thu Feb 26 09:30:56 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Feb 26 09:31:07 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: <16446.800.614957.804144@montanaro.dyndns.org> Pete> I'm sure others will come to me as soon as I hit send. I don't Pete> know if I get a vote, but I'm +2 on PEP 318. I'm +1 on the idea though with the post-arglist list-y syntax: def funct(a1, a2) [d1, d2]: pass for the same reasons Barry elaborated yesterday. The list constructor allows you to split across lines and the decorators are in some sense secondary to the function name and arguments which deserve to be displayed more prominently. Using (...) provides too little visual distinction with the function's argument list. Using <...> is tough for syntax highlighting editors because < and > are already relational operators and shouldn't visually balance in that context. Using {...} implies use of a dictionary, which in turn implies unordered application of multiple decorators. Not using any sort of bracket forces the use of backslashes to continue the decorator list across lines. Using "as" with or without any other syntactic sugar just seems like keyword noise to me. It's not necessary for the parser to disambiguate things and doesn't add much to the comprehension of the construct. If Guido is determined to add some English there we might as well go whole hog: def funct(a1, a2) modified by [d1, d2, wink(0.5)]: pass Skip From aahz at pythoncraft.com Thu Feb 26 09:40:48 2004 From: aahz at pythoncraft.com (Aahz) Date: Thu Feb 26 09:41:06 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <1077801778.14349.49.camel@geddy.wooz.org> References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> <200402260331.54818.fincher.8@osu.edu> <1077801778.14349.49.camel@geddy.wooz.org> Message-ID: <20040226144048.GA18786@panix.com> On Thu, Feb 26, 2004, Barry Warsaw wrote: > On Thu, 2004-02-26 at 03:31, Jeremy Fincher wrote: >> >> efer this syntax to any other suggested syntax. Perhaps it's >> because I'm used to C (or practically any other language with type >> declarations), which puts type declarations ("restraints") before the >> defined function rather than after. > > I think I still like the > pre--function-name-bracket-bracketed-decorator syntax because of > the English-like parsing combined with the visual (parenthetical) > distinction. Eh? I can't parse that. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From tjreedy at udel.edu Thu Feb 26 10:45:15 2004 From: tjreedy at udel.edu (Terry Reedy) Date: Thu Feb 26 10:45:37 2004 Subject: [Python-Dev] Re: Re: new syntax for wrapping (PEP 318) References: <16E1010E4581B049ABC51D4975CEDB8802C09CE4@UKDCX001.uk.int.atosorigin.com> Message-ID: "Moore, Paul" wrote in message news:16E1010E4581B049ABC51D4975CEDB8802C09CE4@UKDCX001.uk.int.atosorigin.com... > This is a very good point. Assuming the "as" keyword, any user who saw > def f(a, b, c) as classmethod: > and looked up "as" rather than "classmethod" has clearly been having a bad day. >Who, as a new user, saw > "import something as another" > and looked up "as" rather than "import"? Go on - admit it. You're among friends :-) Doesn't matter. The ref manual index could/should gain an entry for 'as keyword' with subheadings (with appropriate links) for 'function definition' and 'import'. I was 70% surprised that there is no entry now (2.3.3 version). Terry J. Reedy From jim.jewett at eds.com Thu Feb 26 11:35:12 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Thu Feb 26 11:41:29 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: Pete Shinners: > Some of these may even be bad ideas apon further > thought, but it's something to ponder. Also, many of them don't work with the proposed syntax. Specifically, you're trying to provide a second binding for the function (such as running for __main__, or registering with the exit handler). For me, this is the most common use case, but ... PEP 318 doesn't handle it. (Which is one reason that I don't think this will be the *last* extension, which is one reason I don't want *bare* syntax.) Skip: > Quixote'd PTL uses almost this exact syntax to > distinguish between PTL functions/methods which > return HTML or plain text. The only difference > is that it places the annotation before the argument > list: > def head [html] (title): > I haven't heard any Quixote users complain about the > construct. But they do warn you that it doesn't quite work with standard python. Nobody comes across this syntax until they have gone out of their way to expect it. Even so, would it be better or worse for them to start finding code that says def head [classmethod] (title): I realize that the new syntax (patch version) would actually say something more like def head [html](title)[classmethod]: which goes a fair way toward exhausting special syntax. If they want to get more detailed (xml vs html, only do this if the user is authenticated, etc), then the new syntax starts to be useful to them as well. def head(title) decorators[html4, authenticated_user]: Even if the two cases are combined, I don't see it as useful enough to use up bare syntax. (And so I suggest yet another possible keyword, "decorators") Paul Prescod: > C# allows the wrapping of attributes, not just > functions and classes. > In Python this could be accomplished if we used > delimiters that didn't look like lists (e.g. <> or [! !]). Do this sort of thing often enough, and you have a keyword anyhow -- it just happens to look like line noise. > C# distinguishes between bareword declarations like > "public" and "int" and metadata attachments like > XML serialization information. ... > The Java way does make clear what is a language builtin > ("public", "void") and what is extension metadata > "@remote", "@debug". That may not be a bad thing... Unless we define a specific list (just classmethod and staticmethod?), then this might be more confusing than useful. in __builtins__? In the current standard library? in another module that might become standard someday? Skip Montanaro: > If anything, features which are expected to be used by > more advanced users should require less "COBOL" to be > useful. I'm not arguing that the zen of Python shouldn't > apply, just that advanced features (which will probably > be used a lot less than more basic features) don't > necessarily need quite the English-like structure supporting > them in the language syntax. Features which are used frequently need to be short. Features which are used rarely (like this one?) can be more verbose. I agree that if only advanced users will need it, then it can be a bit more complex to use. It should absolutely not be harder to recognize as irrelevant. If you use an almost meaningless keyword like "extended_syntax_form_a", then the only extra burden is on the advanced users who actually use it. (This is one reason that I'm not too concerned over *which* keyword gets used.) If you use bare syntax, then the burden is on everyone, and you have raised the barrier to entry for the rest of the language. Paul Moore: > Exactly. As a beginner, classes were more advanced than > you needed. So the (relative) difficulty of finding the > relevant details wasn't an issue - you just didn't use > the feature for a while. By the time you needed classes, > you were familiar enough with Python to know where to look. I have never *needed* classes, and have a bias towards functions. If *functions* had seemed awkward, I would have chosen another language instead. People with a bias towards objects - or wanting to learn about them - may well have different criteria. (In real life, most object systems are pretty wordy, so this particular concern may not bother OO folk as much. But the main use cases are for functions.) -jJ From Tony.Vignaux at mcs.vuw.ac.nz Wed Feb 25 14:23:45 2004 From: Tony.Vignaux at mcs.vuw.ac.nz (Tony Vignaux) Date: Thu Feb 26 12:17:28 2004 Subject: [Python-Dev] Accumulation module Message-ID: <403CF641.6010007@mcs.vuw.ac.nz> Raymond, You asked: > * Is there a way to compute the standard deviation without multiple > passes over the data (one to compute the mean and a second to sum the > squares of the differences from the mean? > > I do not understand the problem in getting stdev in a single pass. Perhaps I not understood your problem. If you have a series of values you merely have to calculate their sum and sum-of-squares and use the usual formula. Ignore this if this is dumb. -- Prof G A Vignaux Mathematical and Computing Sciences, Victoria University, PO Box 600, Work: +64 4 463 5276 Wellington, NZ Home: +64 4 934 7851 Tony.Vignaux.Remove_This_Bit@mcs.vuw.ac.nz Mobile: +64 21 89 7851 http://www.mcs.vuw.ac.nz/~vignaux From pje at telecommunity.com Thu Feb 26 12:14:13 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 26 12:19:18 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <0BE9732A-680F-11D8-8DBD-000A95686CD8@redivi.com> References: <20040225180723.1F8E.JCARLSON@uci.edu> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: <5.1.1.6.0.20040226121206.02d1aec0@telecommunity.com> At 10:51 PM 2/25/04 -0500, Bob Ippolito wrote: >Not having this syntax requires A WHOLE !@#$*( lot of verbosity and >repetition, or drastic measures such as alternative compilers (Quixote) or >really awful terrible introspection hacks (PEAK does stack introspection). Actually, that's only for class decoration. PEAK does method decoration the old fashioned way, i.e.: class Something: def aMethod(self): .... aMethod = binding.Make(aMethod) So, an accepted function decorator syntax will make this sort of thing clearer for PEAK as well. From bob at redivi.com Thu Feb 26 12:34:26 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 12:31:23 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: Message-ID: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> On Feb 26, 2004, at 11:35 AM, Jewett, Jim J wrote: > Pete Shinners: >> Some of these may even be bad ideas apon further >> thought, but it's something to ponder. > > Also, many of them don't work with the proposed > syntax. Specifically, you're trying to provide > a second binding for the function (such as running > for __main__, or registering with the exit handler). > > For me, this is the most common use case, but ... > PEP 318 doesn't handle it. (Which is one reason > that I don't think this will be the *last* extension, > which is one reason I don't want *bare* syntax.) Sorry, but you understate the capabilities of this new syntax and overestimate the need for future related syntax. Try the following decorators in the current Python interpreter, using the clumsy def foo(): ... foo = decorator(foo) pattern: def mainfunction(fn): # some more elaborate mechanism to make this happen when the module # is fully loaded is an exercise for the reader, but should be altogether possible # peak.util.imports would be a good inspiration for this if fn.__module__ == '__main__': fn() return fn import atexit def onexit(fn): atexit.register(fn) return fn > Skip: >> Quixote'd PTL uses almost this exact syntax to >> distinguish between PTL functions/methods which >> return HTML or plain text. The only difference >> is that it places the annotation before the argument >> list: > >> def head [html] (title): > >> I haven't heard any Quixote users complain about the >> construct. > > But they do warn you that it doesn't quite work with > standard python. Nobody comes across this syntax until > they have gone out of their way to expect it. Even so, > would it be better or worse for them to start finding > code that says > > def head [classmethod] (title): > > I realize that the new syntax (patch version) would > actually say something more like > > def head [html](title)[classmethod]: > > which goes a fair way toward exhausting special syntax. > If they want to get more detailed (xml vs html, only > do this if the user is authenticated, etc), then the > new syntax starts to be useful to them as well. > > def head(title) decorators[html4, authenticated_user]: > > Even if the two cases are combined, I don't see it > as useful enough to use up bare syntax. (And so I > suggest yet another possible keyword, "decorators") I don't want to argue these points specifically, however, in theory the proposed new syntax is enough to accommodate Quixote's domain specific language **without the need for a custom grammar**. Therefore, garbage like "def head[html](title)[classmethod]" just wouldn't happen. It would simply be a transition of "def head [html](title)" to whatever it would be called with the new syntax, whenever that is decided. > Paul Prescod: > >> C# allows the wrapping of attributes, not just >> functions and classes. > >> In Python this could be accomplished if we used >> delimiters that didn't look like lists (e.g. <> or [! !]). > > Do this sort of thing often enough, and you have a > keyword anyhow -- it just happens to look like line noise. > >> C# distinguishes between bareword declarations like >> "public" and "int" and metadata attachments like >> XML serialization information. > ... >> The Java way does make clear what is a language builtin >> ("public", "void") and what is extension metadata >> "@remote", "@debug". That may not be a bad thing... > > Unless we define a specific list (just classmethod and > staticmethod?), then this might be more confusing than > useful. in __builtins__? In the current standard library? > in another module that might become standard someday? > > Skip Montanaro: >> If anything, features which are expected to be used by >> more advanced users should require less "COBOL" to be >> useful. I'm not arguing that the zen of Python shouldn't >> apply, just that advanced features (which will probably >> be used a lot less than more basic features) don't >> necessarily need quite the English-like structure supporting >> them in the language syntax. > > Features which are used frequently need to be short. > Features which are used rarely (like this one?) can be > more verbose. > > I agree that if only advanced users will need it, then it > can be a bit more complex to use. It should absolutely > not be harder to recognize as irrelevant. > > If you use an almost meaningless keyword like > "extended_syntax_form_a", then the only extra burden is > on the advanced users who actually use it. (This is one > reason that I'm not too concerned over *which* keyword > gets used.) If you use bare syntax, then the burden is > on everyone, and you have raised the barrier to entry for > the rest of the language. The problem is that advanced users will need it /quite/ frequently :) Advanced users can already *do* all these things, but it's clumsy enough to make them want to use some other language or grammar to do it with (examples of this have been extensively covered). I don't think it's a burden on the rest of the language, I just think it's a burden on newbies who try and read advanced code. They'll probably be confused by what it does, anyway. > Paul Moore: > >> Exactly. As a beginner, classes were more advanced than >> you needed. So the (relative) difficulty of finding the >> relevant details wasn't an issue - you just didn't use >> the feature for a while. By the time you needed classes, >> you were familiar enough with Python to know where to look. > > I have never *needed* classes, and have a bias towards > functions. If *functions* had seemed awkward, I would have > chosen another language instead. People with a bias > towards objects - or wanting to learn about them - may > well have different criteria. (In real life, most > object systems are pretty wordy, so this particular concern > may not bother OO folk as much. But the main use cases are > for functions.) Classes are currently so much more preferable to functions because it's easier to make them carry around lots of beautiful wonderful metadata. When you use metadata (and metaclasses, and descriptors) properly, it means less boiler-plate code. This new syntax supports the addition of metadata, reduction of boiler-plate code, and makes creating descriptors painless. Of course, we can already do all these things right now with awfully clumsy and unreadable verbose syntax, but it's bad enough to where I would want to use another programming language or a preprocessor that could expand macros. Additionally, this new syntax will also allow us to remove scary stack introspection hacks that have actually *made it into production code*, because it's too inconvenient to use explicit metaclasses everywhere you want metadata on classes (f.ex., this class provides these interfaces, its instances provide these interfaces, etc.).. among other things. -bob From pje at telecommunity.com Thu Feb 26 12:33:43 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 26 12:34:37 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: Message-ID: <5.1.1.6.0.20040226122130.02d18180@telecommunity.com> At 11:35 AM 2/26/04 -0500, Jewett, Jim J wrote: >Pete Shinners: > > Some of these may even be bad ideas apon further > > thought, but it's something to ponder. > >Also, many of them don't work with the proposed >syntax. Specifically, you're trying to provide >a second binding for the function (such as running >for __main__, or registering with the exit handler). def __main__(func): if __name__=='__main__': func() This now works correctly as long as you define your main function as the last thing in your module. As for the exit handler thing, that works okay too. All that happens is that the function you decorate it as will be None. So after: def exitHandler() [atexit.register]: ... 'exitHandler' will be 'None', but the function as it was defined will be registered for running at system exit. So, yes, actually, these two ideas do work with the proposed syntax. >Features which are used frequently need to be short. >Features which are used rarely (like this one?) can be >more verbose. Actually, I think that if the feature exists, it will be used frequently and for a wide variety of things. It is probably the closest thing to having macros that Python will ever get. We don't even know how many other cool things people will come up with to do with them. >I agree that if only advanced users will need it, then it >can be a bit more complex to use. Advanced users will *create* decorators, but everybody will want to *use* them. >If you use bare syntax, then the burden is >on everyone, and you have raised the barrier to entry for >the rest of the language. I don't think that's true, even for the [] syntax case. Most decorator lists will be relatively short, and the items *in* the decorator list will almost always be a good place to look them up. It's only the reuse of non-decorators (like 'atexit.register') or undocumented custom decorators that's likely to be confusing in reading code. From bob at redivi.com Thu Feb 26 12:38:04 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 12:34:47 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <5.1.1.6.0.20040226121206.02d1aec0@telecommunity.com> References: <20040225180723.1F8E.JCARLSON@uci.edu> <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> <5.1.1.6.0.20040226121206.02d1aec0@telecommunity.com> Message-ID: <881CA714-6882-11D8-8DBD-000A95686CD8@redivi.com> On Feb 26, 2004, at 12:14 PM, Phillip J. Eby wrote: > At 10:51 PM 2/25/04 -0500, Bob Ippolito wrote: >> Not having this syntax requires A WHOLE !@#$*( lot of verbosity and >> repetition, or drastic measures such as alternative compilers >> (Quixote) or really awful terrible introspection hacks (PEAK does >> stack introspection). > > Actually, that's only for class decoration. PEAK does method > decoration the old fashioned way, i.e.: > > class Something: > def aMethod(self): > .... > > aMethod = binding.Make(aMethod) > > So, an accepted function decorator syntax will make this sort of thing > clearer for PEAK as well. Thanks for correcting me here, I was indeed referring to class decoration using stack introspection. The currently-being-debated patch includes this as well with the same syntax. -bob From tismer at stackless.com Thu Feb 26 12:47:38 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Feb 26 12:47:54 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> Message-ID: <403E313A.4070102@stackless.com> Bob Ippolito wrote: ... > Sorry, but you understate the capabilities of this new syntax and > overestimate the need for future related syntax. Try the following > decorators in the current Python interpreter, using the clumsy def > foo(): ... foo = decorator(foo) pattern: Ok, let me give a total different approach that doesn't use new syntax, just a little new semantics. No idea whether it is better, but at least it is easy to read. (Bob, I didn't know where to thow this in, so I used your message). We now have: class klass: def foo():... foo = decorator(foo) Now how about this class klass: foo = decorator def foo():... baz = decorator1, decorator2 def baz(): ... The simple idea is to change semantics that if a name already exists before a def, it is checked whether it is a decorator function or a tuple of these, and if so, they are called. call-me-dumb-but-don't-call-me-perlish - ly y'rs - chris :-)) -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From bob at redivi.com Thu Feb 26 12:56:56 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 12:53:38 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403E313A.4070102@stackless.com> References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> Message-ID: <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> On Feb 26, 2004, at 12:47 PM, Christian Tismer wrote: > Bob Ippolito wrote: > > ... > >> Sorry, but you understate the capabilities of this new syntax and >> overestimate the need for future related syntax. Try the following >> decorators in the current Python interpreter, using the clumsy def >> foo(): ... foo = decorator(foo) pattern: > > Ok, let me give a total different approach that doesn't use new > syntax, just a little new semantics. > No idea whether it is better, but at least it is easy to read. > (Bob, I didn't know where to thow this in, so I used your message). > > We now have: > > class klass: > def foo():... > foo = decorator(foo) > > Now how about this > > class klass: > foo = decorator > def foo():... > > baz = decorator1, decorator2 > def baz(): ... > > The simple idea is to change semantics that if a name already > exists before a def, it is checked whether it is a decorator > function or a tuple of these, and if so, they are called. > > call-me-dumb-but-don't-call-me-perlish - ly y'rs - chris :-)) That's too magical for me, and it means I have to spell out the name of the function (which may be quite long) twice.. which is better than three times, but not as good as once. -bob From tismer at stackless.com Thu Feb 26 13:00:02 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Feb 26 13:00:15 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> Message-ID: <403E3422.3010400@stackless.com> Bob Ippolito wrote: >> class klass: >> foo = decorator >> def foo():... >> >> baz = decorator1, decorator2 >> def baz(): ... >> >> The simple idea is to change semantics that if a name already >> exists before a def, it is checked whether it is a decorator >> function or a tuple of these, and if so, they are called. > That's too magical for me, and it means I have to spell out the name of > the function (which may be quite long) twice.. which is better than > three times, but not as good as once. Or even simpler, just push them as special constants on top of the def: class klass: classmethod def foo():... not too serious, but I hate new syntax... -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From bob at redivi.com Thu Feb 26 13:11:58 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 13:08:39 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403E3422.3010400@stackless.com> References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> <403E3422.3010400@stackless.com> Message-ID: <4430AC38-6887-11D8-8DBD-000A95686CD8@redivi.com> On Feb 26, 2004, at 1:00 PM, Christian Tismer wrote: > Bob Ippolito wrote: > >>> class klass: >>> foo = decorator >>> def foo():... >>> >>> baz = decorator1, decorator2 >>> def baz(): ... >>> >>> The simple idea is to change semantics that if a name already >>> exists before a def, it is checked whether it is a decorator >>> function or a tuple of these, and if so, they are called. > >> That's too magical for me, and it means I have to spell out the name >> of the function (which may be quite long) twice.. which is better >> than three times, but not as good as once. > > Or even simpler, just push them as special constants on top of the def: > > class klass: > classmethod > def foo():... > > not too serious, but I hate new syntax... Try writing large portions of code that are just begging for this new syntax! Try writing large portions of code that are just begging for this new syntax! Try writing large portions of code that are just begging for this new syntax! (three times, to emphasize the number of times that I have to type the name of each function when using a descriptor) (three times, to emphasize the number of times that I have to type the name of each function when using a descriptor) (three times, to emphasize the number of times that I have to type the name of each function when using a descriptor) The special pushed constants thing is just a little too weird for me as well :) -bob From gmccaughan at synaptics-uk.com Thu Feb 26 13:18:22 2004 From: gmccaughan at synaptics-uk.com (Gareth McCaughan) Date: Thu Feb 26 13:18:26 2004 Subject: [Python-Dev] Accumulation module In-Reply-To: <403CF641.6010007@mcs.vuw.ac.nz> References: <403CF641.6010007@mcs.vuw.ac.nz> Message-ID: <200402261818.22612.gmccaughan@synaptics-uk.com> On Wednesday 2004-02-25 19:23, Tony Vignaux wrote: > Raymond, > > You asked: > > * Is there a way to compute the standard deviation without multiple > > passes over the data (one to compute the mean and a second to sum the > > squares of the differences from the mean? > > I do not understand the problem in getting stdev in a single pass. > Perhaps I not understood your problem. If you have a series of values > you merely have to calculate their sum and sum-of-squares and use the > usual formula. The trouble with that is that it's numerically horrible. There are better ways, some of which were discussed in python-dev a few weeks ago. -- g From barry at python.org Thu Feb 26 13:33:05 2004 From: barry at python.org (Barry Warsaw) Date: Thu Feb 26 13:33:09 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <16E1010E4581B049ABC51D4975CEDB8803060E05@UKDCX001.uk.int.atosorigin.com> References: <16E1010E4581B049ABC51D4975CEDB8803060E05@UKDCX001.uk.int.atosorigin.com> Message-ID: <1077820384.15878.244.camel@geddy.wooz.org> On Thu, 2004-02-26 at 08:48, Moore, Paul wrote: > The PEP doesn't specify where discussion should take place, but the > author hasn't participated here. At some point, he (or someone else) is > going to have to put the PEP forward for acceptance. > > At the moment, the PEP appears fairly strongly in favour of "as" and > against the use of brackets. But the author admits that he has not > got the expertise to provide an implementation. At this point, I think the PEP should try to outline the various syntactic options in a way that the BDFL can look at them and make a decision. I think it's okay that the implementation picks one of those -- I don't think we necessarily need implementations of every syntax option right now. The existing one allows people to play with the semantics which should be good enough to decide whether we want new syntax to support this at all. -Barry From barry at python.org Thu Feb 26 13:34:31 2004 From: barry at python.org (Barry Warsaw) Date: Thu Feb 26 13:34:35 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040226144048.GA18786@panix.com> References: <62567.208.180.157.209.1077778734.squirrel@www.gameboard.org> <200402260331.54818.fincher.8@osu.edu> <1077801778.14349.49.camel@geddy.wooz.org> <20040226144048.GA18786@panix.com> Message-ID: <1077820471.15878.246.camel@geddy.wooz.org> On Thu, 2004-02-26 at 09:40, Aahz wrote: > On Thu, Feb 26, 2004, Barry Warsaw wrote: > > I think I still like the > > pre--function-name-bracket-bracketed-decorator syntax because of > > the English-like parsing combined with the visual (parenthetical) > > distinction. > > Eh? I can't parse that. ;-) :) def [classmethod] makeNewbie(a, b, c): pass -Barry From tismer at stackless.com Thu Feb 26 13:40:08 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Feb 26 13:41:39 2004 Subject: [Python-Dev] Accumulation module In-Reply-To: <403CF641.6010007@mcs.vuw.ac.nz> References: <403CF641.6010007@mcs.vuw.ac.nz> Message-ID: <403E3D88.2030106@stackless.com> Tony Vignaux wrote: > Raymond, > > You asked: > >> * Is there a way to compute the standard deviation without multiple >> passes over the data (one to compute the mean and a second to sum the >> squares of the differences from the mean? >> >> > > I do not understand the problem in getting stdev in a single pass. > Perhaps I not understood your problem. If you have a series of values > you merely have to calculate their sum and sum-of-squares and use the > usual formula. True, if you can use exact numbers. When calculating with floats (also named real, because they are everything else but real), you unfortunately calculate two things at the same time: Your intended calculation, plus some propagating error. The trick is to keep the error propagation as small as possible, while keeping the algorithm readable and effective. To my knowledge, always adding the smallest values first is best possible for minimizing cutoff/rounding, although it involes sorting. Using pairwise addition of values might be a good compromize, but I neither know state of the art nor did I follw the thread. > Ignore this if this is dumb. reals are dumb. ciao - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From kf7xm at netscape.net Thu Feb 26 13:31:44 2004 From: kf7xm at netscape.net (Vernon Cole) Date: Thu Feb 26 13:41:44 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines() and StringIO.writelines() In-Reply-To: <403C8363.8050009@hlabs.spb.ru> References: <00ba01c3f9f2$62a22060$f72dc797@oemcomputer> <403C8363.8050009@hlabs.spb.ru> Message-ID: This is a big YES! A function or method should do what is says. At present, writelines() should be named append(), because it does not actually write LINES. Us older folks often forget which environment we are working in. I am too used to typing "write" when I do not want a line terminator, and "writeln" when I do. (This agrees well with Python where, if I do NOT want an automatic end-of-line, I must put a trailing comma in the Print command.) But, suddenly, when I am sending text to a file, I must remember to hang "\n" on the end of everything. Confusing and non-user-friendly. Backslash-N is not exactly a worldwide way of stating the concept "go to a new line." Not everyone is a C programmer. This also ties in with the "UNIVERSAL_NEWLINE" thing. Only in *nix does a linefeed charactor equate to the end of a line in a text file. So, in every output line of every program I must pretend to be a C programmer and append a '\n' so that the runtime system can replace it with the actual End-of-Line used by my operating system. N.U.T.S.! Let me tell the file method that I want a UNIVERSAL_NEWLINE added to each line I send and let it be done in one step. Let it be: file.writelines(x for x in mylines, True) or even better: file.writeNewlines(True) file.writelines(x for x in mylines) Then in Python 3.0 file.writeNewlines can be defaulted True for all files opened in TEXT mode. Perhaps that will help the *nix crowd remember to actually OPEN binary files in binary mode. ------------ Vernon .............. Dmitry Vasiliev wrote: > Raymond Hettinger wrote: > >> Currently, writelines() does not add trailing line separators.. >> This is fine when working with readlines() but a PITA in other >> situations. >> >> If we added an optional separator argument, it would be easier to add >> newlines and we would gain some of the flexibility of str.join() at >> full C speed. > > > Maybe not a separator but suffix, so newline will be added to last line > too? > From theller at python.net Thu Feb 26 13:48:04 2004 From: theller at python.net (Thomas Heller) Date: Thu Feb 26 13:48:22 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403E3422.3010400@stackless.com> (Christian Tismer's message of "Thu, 26 Feb 2004 19:00:02 +0100") References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> <403E3422.3010400@stackless.com> Message-ID: Christian Tismer writes: > Bob Ippolito wrote: > >>> class klass: >>> foo = decorator >>> def foo():... >>> >>> baz = decorator1, decorator2 >>> def baz(): ... >>> >>> The simple idea is to change semantics that if a name already >>> exists before a def, it is checked whether it is a decorator >>> function or a tuple of these, and if so, they are called. > >> That's too magical for me, and it means I have to spell out the name >> of the function (which may be quite long) twice.. which is better >> than three times, but not as good as once. > > Or even simpler, just push them as special constants on top of the def: > > class klass: > classmethod > def foo():... > > not too serious, but I hate new syntax... The above code looks like as someone forgot to actually initialize the class variable (probably not too likely with the name 'classmethod', but with other names): class klass: classmethod = 42 def foo():... Thomas From mwh at python.net Thu Feb 26 13:51:25 2004 From: mwh at python.net (Michael Hudson) Date: Thu Feb 26 13:51:50 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403E3422.3010400@stackless.com> (Christian Tismer's message of "Thu, 26 Feb 2004 19:00:02 +0100") References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> <403E3422.3010400@stackless.com> Message-ID: <2mvfltoes2.fsf@starship.python.net> Christian Tismer writes: > Bob Ippolito wrote: > >> That's too magical for me, and it means I have to spell out the name >> of the function (which may be quite long) twice.. which is better >> than three times, but not as good as once. > > Or even simpler, just push them as special constants on top of the def: > > class klass: > classmethod > def foo():... > > not too serious, but I hate new syntax... Chris, stop this, or I'll start to think you're some kind of insane hacker type. Oh, hang on... Cheers, mwh :-) -- 3. Syntactic sugar causes cancer of the semicolon. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From tismer at stackless.com Thu Feb 26 13:54:43 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Feb 26 13:54:53 2004 Subject: [Python-Dev] Accumulation module In-Reply-To: <403E3D88.2030106@stackless.com> References: <403CF641.6010007@mcs.vuw.ac.nz> <403E3D88.2030106@stackless.com> Message-ID: <403E40F3.4070901@stackless.com> Christian Tismer wrote: > To my knowledge, always adding the smallest values first is > best possible for minimizing cutoff/rounding, although it > involes sorting. Oops, no, it is better to add those which are nearest! I should have read this earlier: http://www.owlnet.rice.edu/~caam420/lectures/fpsum.html ciao - chris p.s.: Note that products behave much better. Summation is really against the nature of float. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From d.holton at vanderbilt.edu Thu Feb 26 14:04:41 2004 From: d.holton at vanderbilt.edu (Doug Holton) Date: Thu Feb 26 14:04:50 2004 Subject: [Python-Dev] other uses for "as" [was Re: new syntax for wrapping (PEP 318)] In-Reply-To: References: Message-ID: <403E4349.6030001@vanderbilt.edu> > Skip> If you are going to add [...] as a function modifier syntax, > Skip> I would argue that it should allow the full power of list > Skip> (or generator) comprehensions. > > Agreed. And why limit it to lists - why not any expression that > evaluates to a list? Or maybe any sequence? Which is one reason > that a bare "[...]" doesn't seem sufficient. The "as" syntax is easier to understand and more readable, but you'd like to have the full power of lists as well (plus make the parsing easier). When there is just a single item though, the list syntax seems overkill. I can already tell if this feature is added it will likely stick with the list syntax instead of the "as" keyword because that is the easier solution to code. But here is an "as" solution that doesn't require extra coding for now. Have the parser check for a single item OR a list of items. You could add an "as" keyword but essentially ignore it in this case. This would mean though that this would not work: def myfunc(x, y) as synchronized, classmethod: ... You'd have to use the list syntax for multiple modifiers. If there is a decision to add the "as" keyword, here are some possible future expansions of its use. What it does is make the use of modifiers and type checking more readable. In general "X as Y" would mean --> Y(X) and "X as [Y, Z]" would mean --> Z(Y(X)) for example: X = temp_val as int means --> x = int(temp_val) thisobject = obj as MyType means --> thisobject = MyType(obj) Then there are Visual Basic -like params (let's not start a war about it): def func1(x as int, y as float): ... instead of: def func1(x, y): try: x2 = int(x) y2 = float(y) ... From tismer at stackless.com Thu Feb 26 14:05:48 2004 From: tismer at stackless.com (Christian Tismer) Date: Thu Feb 26 14:05:56 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> <403E3422.3010400@stackless.com> Message-ID: <403E438C.7040208@stackless.com> Thomas Heller wrote: > The above code looks like as someone forgot to actually initialize the > class variable (probably not too likely with the name 'classmethod', but > with other names): > > class klass: > classmethod = 42 > def foo():... Yes I know. I just wanted to encourage people to think of simpler alternatives but to spoil the language for tiny improvements. I didn't just want to say "this is ugly", but to think of ways to express "the following is meant to be a... method". Onw thing I could live with if I must is "class method", since it is similar to other languages. reminds-me-of-ternary-operators -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From jim.jewett at eds.com Thu Feb 26 14:12:17 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Thu Feb 26 14:12:31 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) Message-ID: Jim Jewett: >> Also, many of them don't work with the proposed >> syntax. Specifically, you're trying to provide >> a second binding for the function (such as running >> for __main__, or registering with the exit handler). >> For me, this is the most common use case, but ... >> PEP 318 doesn't handle it. (Which is one reason >> that I don't think this will be the *last* extension, >> which is one reason I don't want *bare* syntax.) Bob Ippolito: > Sorry, but you understate the capabilities of this new > syntax and overestimate the need for future related syntax. Not really; I guess I should have left in the full original example. def foo[__main__]: will not work. Yes, you can first define a decorator function specific to this particular function you're about to define, just as you can currently rebind the functions's name to a transform of the function. Both are ugly. Jim Jewett: >> I realize that the new syntax (patch version) would >> actually say something more like [for Quixote] >> def head [html](title)[classmethod]: Bob Ippolito: > in theory the proposed new syntax is enough to accommodate > Quixote's domain specific language **without the need for a > custom grammar**. Agreed. If it weren't for backwards compatibility, they could just migrate to the new version entirely. My point isn't that both are needed; my point is that using up unnamed syntax can come back to bite you later. (In fairness, this is also a slight knock against "as", which could just as easily mean "alias" - which I find a more common use case. But "as" is OK if the proponents really want something short.) Jim Jewett: >> Features which are used frequently need to be short. >> Features which are used rarely (like this one?) can be >> more verbose. >> I agree that if only advanced users will need it, then it >> can be a bit more complex to use. It should absolutely >> not be harder to recognize as irrelevant. Bob Ippolito: > The problem is that advanced users will need it /quite/ > frequently :) Fair enough; then it is reasonable to use an obscure abbreviation, so long as it isn't obscure enough to be mistaken for possibly normal usage. Use "X" as the magic keyword if you want. (Though even "as" is better.) > I don't think it's a burden on the rest of the language, > I just think it's a burden on newbies who try and read > advanced code. They'll probably be confused by what it > does, anyway. I haven't seen much in the way of difficulty-graded indices of sample code. A common introduction to a new language is to have a specific problem, and to hear that package X solves it. If package HyperMetaConfusing is the package that solves the problem, then it is the first package new users will look at. Jim Jewett: >> If *functions* had seemed awkward, I would have >> chosen another language instead. Bob Ippolito: > Classes are currently so much more preferable to > functions because it's easier to make them carry > around lots of beautiful wonderful metadata. For me, the advantage is that they save state. I understand that others use them very differently; my main point is that adding syntax to something I consider crucial (functions) is more of a problem (for me) than adding it to something that I don't consider crucial. If this transform applied only to class methods, it would not have put newbie-me off, because those aren't as basic. (I don't suggest this artificial distinction; I only point out that it affects the "cost" of the change.) I am willing to bet that most programmers (and non- programmers starting with python as a first language) will believe that they need to learn functions. So function syntax should be as transparent as possible. -jJ From theller at python.net Thu Feb 26 14:27:12 2004 From: theller at python.net (Thomas Heller) Date: Thu Feb 26 14:27:29 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403E438C.7040208@stackless.com> (Christian Tismer's message of "Thu, 26 Feb 2004 20:05:48 +0100") References: <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> <403E3422.3010400@stackless.com> <403E438C.7040208@stackless.com> Message-ID: Christian Tismer writes: > Thomas Heller wrote: > >> The above code looks like as someone forgot to actually initialize the >> class variable (probably not too likely with the name 'classmethod', but >> with other names): >> class klass: >> classmethod = 42 >> def foo():... > > Yes I know. I just wanted to encourage people to think > of simpler alternatives but to spoil the language > for tiny improvements. > I didn't just want to say "this is ugly", but to think of > ways to express "the following is meant to be a... method". > > Onw thing I could live with if I must is "class method", > since it is similar to other languages. > > reminds-me-of-ternary-operators -- chris For me, the advantage of the current def foo(cls): ... foo = classmethod(foo) is that it *exactly* describes what's going on. The difficulty, I would guess, to people coming from other languages, is that they are more used to (how can I explain it) a mental model where a compiler parses some syntax and generates code from that. In python, it *is* simply executable code, and it is executed. I'm +1 on this proposed syntax def foo(cls) [classmethod]: since is it easy to parse to my own eye. And I think it is much easier to learn than list comprehensions have been when they were introduced. I'm +2 on the cool stuff that would be possible with it. I'm +0 (maybe even -0) because it moves Python further away from 'executable pseudocode'. Thomas From oussoren at cistron.nl Thu Feb 26 14:28:42 2004 From: oussoren at cistron.nl (Ronald Oussoren) Date: Thu Feb 26 14:28:58 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <20040225180723.1F8E.JCARLSON@uci.edu> References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: On 26-feb-04, at 3:56, Josiah Carlson wrote: >> Just a data point from a user: I use classmethod fairly frequently, >> and >> I would *love* to see some syntax to support it. As has been pointed >> out before, classmethod (and staticmethod) are a part of the interface >> of a function, and shouldn't be tacked on to the end of the body. >> Please let me put them where people (and hopefully pydoc and >> pychecker) >> can easily see them. > > I personally don't see the point of using any syntax for wrapping a > function like so: > > class a: > def funct(self): > pass > a = classmethod(a) Not that I have anything usefull to add, but ... I find the proposed syntax easier to understand than the current alternative, even for short functions. In "def foo(self) [classmethod]: pass" the [classmethod] part feels like an anotation/declaration, while "foo = classmethod(foo)" has a more magical feel. I'd expect at least as much newbe questions about the current idiom than about the alternatives. Ronald -- X|support bv http://www.xsupport.nl/ T: +31 610271479 F: +31 204416173 From bob at redivi.com Thu Feb 26 14:34:56 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 14:31:40 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) In-Reply-To: References: Message-ID: On Feb 26, 2004, at 2:12 PM, Jewett, Jim J wrote: > Jim Jewett: >>> Also, many of them don't work with the proposed >>> syntax. Specifically, you're trying to provide >>> a second binding for the function (such as running >>> for __main__, or registering with the exit handler). > >>> For me, this is the most common use case, but ... >>> PEP 318 doesn't handle it. (Which is one reason >>> that I don't think this will be the *last* extension, >>> which is one reason I don't want *bare* syntax.) > > Bob Ippolito: >> Sorry, but you understate the capabilities of this new >> syntax and overestimate the need for future related syntax. > > Not really; I guess I should have left in the full original > example. > > def foo[__main__]: > > will not work. > > Yes, you can first define a decorator function specific to > this particular function you're about to define, just as you > can currently rebind the functions's name to a transform of > the function. Both are ugly. If the decorator were deemed useful enough, it would be part of the standard library of Python 2.5 (or whatever). I don't see the reasoning in your argument: * This new syntax lets us make common idioms easier * But we also need to add a new function to the standard library in order to make that particular idiom truly easier * So the new syntax sucks Nothing is stopping you from sticking with Python 2.3 for the rest of your life ;) > Jim Jewett: >>> I realize that the new syntax (patch version) would >>> actually say something more like [for Quixote] > >>> def head [html](title)[classmethod]: > > Bob Ippolito: >> in theory the proposed new syntax is enough to accommodate >> Quixote's domain specific language **without the need for a >> custom grammar**. > > Agreed. If it weren't for backwards compatibility, they could > just migrate to the new version entirely. My point isn't that > both are needed; my point is that using up unnamed syntax can > come back to bite you later. (In fairness, this is also a > slight knock against "as", which could just as easily mean > "alias" - which I find a more common use case. But "as" is > OK if the proponents really want something short.) Well migrating to the new syntax entirely will eventually take us from N compilers for domain specific python-like languages, to ~1 compiler because Python will be better at domain specific languages. I think this is reason enough, even without the staticmethod/classmethod usages. > Jim Jewett: >>> Features which are used frequently need to be short. >>> Features which are used rarely (like this one?) can be >>> more verbose. > >>> I agree that if only advanced users will need it, then it >>> can be a bit more complex to use. It should absolutely >>> not be harder to recognize as irrelevant. > > Bob Ippolito: >> The problem is that advanced users will need it /quite/ >> frequently :) > > Fair enough; then it is reasonable to use an obscure > abbreviation, so long as it isn't obscure enough to > be mistaken for possibly normal usage. Use "X" as the > magic keyword if you want. (Though even "as" is better.) > >> I don't think it's a burden on the rest of the language, >> I just think it's a burden on newbies who try and read >> advanced code. They'll probably be confused by what it >> does, anyway. > > I haven't seen much in the way of difficulty-graded > indices of sample code. A common introduction to a new > language is to have a specific problem, and to hear that > package X solves it. If package HyperMetaConfusing is > the package that solves the problem, then it is the first > package new users will look at. Chances are, HyperMetaConfusing will use decorators in its own source code, but not require them from the API. > Jim Jewett: >>> If *functions* had seemed awkward, I would have >>> chosen another language instead. > > Bob Ippolito: >> Classes are currently so much more preferable to >> functions because it's easier to make them carry >> around lots of beautiful wonderful metadata. > > For me, the advantage is that they save state. I prefer to use instances to save state, not classes :) > I understand that others use them very differently; > my main point is that adding syntax to something I > consider crucial (functions) is more of a problem > (for me) than adding it to something that I don't > consider crucial. > > If this transform applied only to class methods, it > would not have put newbie-me off, because those aren't > as basic. (I don't suggest this artificial distinction; > I only point out that it affects the "cost" of the > change.) > > I am willing to bet that most programmers (and non- > programmers starting with python as a first language) > will believe that they need to learn functions. So > function syntax should be as transparent as possible. It doesn't have any bearing on any existing code whatesoever (except potential for cleanup of the nasty stuff that uses decorators now), and you really just don't have to learn it for a while unless it becomes so popular that everyone uses it for everything (doubtful). Then again, I'm not trying to teach anyone python, nor am I trying to learn python basics. If you want a reduced Python syntax to teach with, give them Python 1.5, and save the good stuff for people who need to get stuff done ;) -bob From pje at telecommunity.com Thu Feb 26 14:40:08 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 26 14:40:15 2004 Subject: [Python-Dev] other uses for "as" [was Re: new syntax for wrapping (PEP 318)] In-Reply-To: <403E4349.6030001@vanderbilt.edu> References: Message-ID: <5.1.1.6.0.20040226143606.024b6730@telecommunity.com> At 01:04 PM 2/26/04 -0600, Doug Holton wrote: > > Skip> If you are going to add [...] as a function modifier syntax, > > Skip> I would argue that it should allow the full power of list > > Skip> (or generator) comprehensions. > > > > Agreed. And why limit it to lists - why not any expression that > > evaluates to a list? Or maybe any sequence? Which is one reason > > that a bare "[...]" doesn't seem sufficient. > > >The "as" syntax is easier to understand and more readable, but you'd like >to have the full power of lists as well (plus make the parsing >easier). When there is just a single item though, the list syntax seems >overkill. > >I can already tell if this feature is added it will likely stick with the >list syntax instead of the "as" keyword because that is the easier >solution to code. > >But here is an "as" solution that doesn't require extra coding for now. >Have the parser check for a single item OR a list of items. You could add >an "as" keyword but essentially ignore it in this case. This would mean >though that this would not work: >def myfunc(x, y) as synchronized, classmethod: > ... >You'd have to use the list syntax for multiple modifiers. > > >If there is a decision to add the "as" keyword, here are some possible >future expansions of its use. What it does is make the use of modifiers >and type checking more readable. > >In general "X as Y" would mean --> Y(X) >and "X as [Y, Z]" would mean --> Z(Y(X)) > >for example: >X = temp_val as int > means --> x = int(temp_val) > >thisobject = obj as MyType > means --> thisobject = MyType(obj) Hm, that sounds more like syntax for PEP 246 (adaptation). >Then there are Visual Basic -like params (let's not start a war about it): > >def func1(x as int, y as float): > ... This also seems like syntax for adaptation. But I don't see this latter use as an argument for "as", since: def func1(x [int], y [float]): would also be sensible syntax. But, the real monkeywrench here is that: def func1(x as int, y as float) as staticmethod: now looks like it *returns* a staticmethod, which is wrong. So, thanks to your argument, I'm now leaning a little more towards using [] rather than "as", because "as" looks like syntax that should be reserved for adaptation at a future time. :) From pje at telecommunity.com Thu Feb 26 14:50:24 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 26 14:50:27 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) In-Reply-To: Message-ID: <5.1.1.6.0.20040226144041.024ba2e0@telecommunity.com> At 02:12 PM 2/26/04 -0500, Jewett, Jim J wrote: >Jim Jewett: > >> Also, many of them don't work with the proposed > >> syntax. Specifically, you're trying to provide > >> a second binding for the function (such as running > >> for __main__, or registering with the exit handler). > > >> For me, this is the most common use case, but ... > >> PEP 318 doesn't handle it. (Which is one reason > >> that I don't think this will be the *last* extension, > >> which is one reason I don't want *bare* syntax.) > >Bob Ippolito: > > Sorry, but you understate the capabilities of this new > > syntax and overestimate the need for future related syntax. > >Not really; I guess I should have left in the full original >example. > > def foo[__main__]: > >will not work. You mean: def foo()[__main__]: and it will indeed work. Just try it: Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. IDLE 0.8 -- press F1 for help >>> def __main__(f): if f.func_globals['__name__']=='__main__': f() return f >>> def foo(): print "testing" >>> foo = __main__(foo) testing >>> foo __main__ could be defined in another module and imported. It's not specific to a particular main function. It simply runs the function if it's defined in the main program module. And in either case, it returns the original function, so that if the module is not "main", then the function is still available for importing. Thus, the 'foo' function would still be the 'foo' function, but it would also be run if it was the main module. The only remaining downside I see is that foo() could not call itself recursively, since it can run before the name 'foo' is bound to it. From kbk at shore.net Thu Feb 26 14:54:23 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Feb 26 14:54:42 2004 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200402261954.i1QJsNhM022777@hydra.localdomain> Patch / Bug Summary ___________________ Patches : 243 open ( +2) / 2319 closed ( +4) / 2562 total ( +6) Bugs : 728 open (+17) / 3904 closed (+12) / 4632 total (+29) RFE : 130 open ( +3) / 119 closed ( +1) / 249 total ( +4) New / Reopened Patches ______________________ Add Py_InitializeEx to allow embedding without signals. (2004-02-19) http://python.org/sf/900727 opened by Dave Wilson catch invalid chunk length in httplib read routine (2004-02-20) http://python.org/sf/900744 opened by Wummel markupbase misses comments (bug 736659) (2004-02-20) http://python.org/sf/901369 opened by Jim Jewett urllib2.parse_http_list bugfix 735248 (2004-02-20) http://python.org/sf/901480 opened by Jim Jewett FreeBSD new pthread problem with system scope (2004-02-23) http://python.org/sf/902444 opened by Hye-Shik Chang dict.update should take a 2-tuple sequence like dict.__init_ (2004-02-25) http://python.org/sf/904720 opened by Bob Ippolito Patches Closed ______________ bdist_wininst pre-install script, and --target-version fix (2004-02-08) http://python.org/sf/892660 closed by theller logging says to call shutdown - make it more likely (2004-01-21) http://python.org/sf/881676 closed by vsajip Refactored patch #815911 for logging with SocketHandler (2003-12-01) http://python.org/sf/851993 closed by vsajip Extension logging.handlers.SocketHandler (2003-10-01) http://python.org/sf/815911 closed by vsajip New / Reopened Bugs ___________________ Duplicated in (2004-02-18) http://python.org/sf/899423 opened by Joseph Walton 1==float('nan') (2004-02-17) http://python.org/sf/899109 reopened by mwh optparse: usage issues (2004-02-19) http://python.org/sf/900071 opened by Simon Dahlbacka hotshot.stats.load (2004-02-19) http://python.org/sf/900092 opened by Simon Dahlbacka cgi.fieldStorage doesn't grok standards env. variables (2004-02-19) http://python.org/sf/900112 opened by Dominique Hazaël-Massieux win shell interpreter error (arithmetic) (2004-02-19) http://python.org/sf/900494 opened by Mark Okamura IDLE docs broken on OSX (2004-02-19) http://python.org/sf/900580 opened by Jonathan Brandmeyer add logging.shutdown() to python logging module example code (2004-02-19) http://python.org/sf/900626 opened by Manuel Schulte urllib2 AuthHandlers can pass a bad host to HTTPPasswordMgr (2004-02-20) http://python.org/sf/900898 opened by James Kruth plat-mac/videoreader.py not working on OS X (2004-02-20) http://python.org/sf/900949 opened by Dinu C. Gherman cygwinccompiler.get_versions fails on `ld -v` output (2004-02-20) http://python.org/sf/900977 opened by Pearu Peterson strange behaviour of xmlrpclib.Server proxy (2004-02-20) http://python.org/sf/901198 opened by Andreas Eisele random.randint fails for some ranges (2004-02-20) http://python.org/sf/901285 opened by paul rubin logging handlers can close without flush (2004-02-20) http://python.org/sf/901330 opened by Jim Jewett random.seed not initialized as advertised (2004-02-21) http://python.org/sf/901605 opened by paul rubin split method documentation can be improved (2004-02-21) http://python.org/sf/901654 opened by Sjoerd Mullender extra_path kwarg to setup() undocumented (2004-02-21) http://python.org/sf/901727 opened by Bob Ippolito relative on-disk SYSTEM id handling is incorrect (2004-02-22) http://python.org/sf/902037 opened by Bob Ippolito pydoc insists upon producing file: URLs (2004-02-22) http://python.org/sf/902061 opened by Matthias Klose fails building C++ extensions when configured --without-cxx (2004-02-22) http://python.org/sf/902065 opened by Matthias Klose urllib2 should be more robust for sloppy proxy URLs (2004-02-22) http://python.org/sf/902075 opened by Matthias Klose Thread start - strange error under Cygwin (2004-02-22) http://python.org/sf/902151 opened by Artur de Sousa Rocha Timezone miscalculation (time.mktime) (2004-02-22) http://python.org/sf/902172 opened by Marti Raudsepp Encodings: koi8-u.py -- errata in decoding map (2004-02-23) http://python.org/sf/902501 opened by Alexander Belchenko doctest confused by super-instances in class-dicts (2004-02-23) http://python.org/sf/902628 opened by Christian Tanzer help("exec") doesn't work (2004-02-23) http://python.org/sf/902998 opened by Tom Lynn OpenVMS file system and UNIVERSAL_NEWLINES support (2004-02-24) http://python.org/sf/903339 opened by Piéronne Jean-François cPickle/pickle incompatibility (2004-02-24) http://python.org/sf/903483 reopened by tismer File read of Chinese utf-16-le treats upper byte 1A as EOF (2004-02-25) http://python.org/sf/904474 opened by Ron Rother threading docs, start error should be specified (2004-02-25) http://python.org/sf/904498 opened by Russell Owen test_tcl fails -- "can't find package Tclx" (2004-02-26) http://python.org/sf/905046 opened by Michael Hudson Bugs Closed ___________ 1==float('nan') (2004-02-17) http://python.org/sf/899109 closed by mwh win shell interpreter error (arithmetic) (2004-02-19) http://python.org/sf/900494 closed by tim_one add logging.shutdown() to python logging module example code (2004-02-19) http://python.org/sf/900626 closed by vsajip logging.shutdown() exception (2003-11-14) http://python.org/sf/842170 closed by vsajip random.randint fails for some ranges (2004-02-20) http://python.org/sf/901285 closed by rhettinger logging handlers can close without flush (2004-02-20) http://python.org/sf/901330 closed by vsajip random.seed not initialized as advertised (2004-02-21) http://python.org/sf/901605 closed by rhettinger Incorrect ZeroDivisionError message with __future__ division (2004-01-26) http://python.org/sf/884841 closed by rhettinger Timezone miscalculation (time.mktime) (2004-02-22) http://python.org/sf/902172 closed by sipsick Encodings: koi8-u.py -- errata in decoding map (2004-02-23) http://python.org/sf/902501 closed by lemburg help("exec") doesn't work (2004-02-23) http://python.org/sf/902998 closed by tlynn cPickle/pickle incompatibility (2004-02-24) http://python.org/sf/903483 closed by tismer codecs.open().readlines(sizehint) bug (2003-11-18) http://python.org/sf/844561 closed by lemburg bsddb btree set_location() semantics changed (2003-08-13) http://python.org/sf/788421 closed by greg New / Reopened RFE __________________ bundlebuilder: some way to add non-py files in packages (2004-02-19) http://python.org/sf/900502 opened by Russell Owen bundlebuilder: an arg to disable zipping the code (2004-02-19) http://python.org/sf/900506 opened by Russell Owen bundlebuilder: easily keep main routine in orig location (2004-02-19) http://python.org/sf/900514 opened by Russell Owen cPickle/pickle incompatibility (2004-02-24) http://python.org/sf/903483 opened by Christian Tismer http://doc.python.org (2004-02-25) http://python.org/sf/904770 opened by Bob Ippolito RFE Closed __________ Enable __slots__ for meta-types (2003-03-02) http://python.org/sf/696184 closed by tismer From pje at telecommunity.com Thu Feb 26 15:12:54 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu Feb 26 15:12:58 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: <403E438C.7040208@stackless.com> <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> <403E3422.3010400@stackless.com> <403E438C.7040208@stackless.com> Message-ID: <5.1.1.6.0.20040226150111.01eaa020@telecommunity.com> At 08:27 PM 2/26/04 +0100, Thomas Heller wrote: >Christian Tismer writes: > > > Thomas Heller wrote: > > > >> The above code looks like as someone forgot to actually initialize the > >> class variable (probably not too likely with the name 'classmethod', but > >> with other names): > >> class klass: > >> classmethod = 42 > >> def foo():... > > > > Yes I know. I just wanted to encourage people to think > > of simpler alternatives but to spoil the language > > for tiny improvements. > > I didn't just want to say "this is ugly", but to think of > > ways to express "the following is meant to be a... method". > > > > Onw thing I could live with if I must is "class method", > > since it is similar to other languages. > > > > reminds-me-of-ternary-operators -- chris > >For me, the advantage of the current > > def foo(cls): ... > foo = classmethod(foo) > >is that it *exactly* describes what's going on. >The difficulty, I would guess, to people coming from other >languages, is that they are more used to (how can I explain it) a mental >model where a compiler parses some syntax and generates code from that. >In python, it *is* simply executable code, and it is executed. What you say is true, but it's also true that assembly code describes *exactly* what is going on. :) But I think there needs to be a distinction betweeen "how" something happens, and "what" is happening. For teaching purposes, it's much easier to say, "if you want to make it a class method, put '[classmethod]' here", than it is to explain the existing technique. If I want to teach someone *how* decorators work (so they can write their own), there is nothing stopping me from demonstrating the foo = classmethod(foo) version. So, there are two separate issues here. One is *using* decorators, the other is *making* them. I do not need to understand how decorators in general work in order to use them, any more than I need to understand metaclasses to use classes, even though every time I make a class I'm using a metaclass! I only need to know what the particular decorator I'm using (or that someone else used) is supposed to *do* for me. But if I want to *make* a decorator, I need to know how they work. (Personally, I'd look at the source of other decorators to figure that out, but I digress.) Anyway, the 'foo = classmethod(foo)' thing is actually *noise* when it comes to teaching, because now you have to explain why you're calling a function on a function and assigning it to the same name as an existing function. We've just digressed into functional programming and the Python namespace mechanisms when all we wanted was to talk about class methods! A syntax that goes in the 'def' block is less distracting, because it just becomes part of the "how to define a class method" rather than opening up all these other issues. This sort of "put the language in the background and the task in the foreground" thing is Python's traditional strength, and I believe that having a reasonable decorator syntax that takes place as part of the 'def' statement will allow that strength to apply to classmethods, staticmethods, and numerous other types of function decorators. From rowen at cesmail.net Thu Feb 26 15:17:17 2004 From: rowen at cesmail.net (Russell E. Owen) Date: Thu Feb 26 15:17:39 2004 Subject: [Python-Dev] Re: threading issue: preventing starting twice References: <200402252140.i1PLeoT08847@guido.python.org> Message-ID: In article <200402252140.i1PLeoT08847@guido.python.org>, Guido van Rossum wrote: > > In the threading module, one must of course only start a thread once. I > > have some questions about handling this gracefully. > > > > First of all, the documentation only says it is an error to start a > > thread more than once. It does not say which exception is raised. On my > > system I find it is AssertionError. So...is that going to be true on all > > systems? Is the test based on "assert", in which case it will go away > > for optimized code (and then what happens)? > > Hey, it's open source! You can look for yourself. :-) > > Yes, the test is based on assert. But if a feature is not documented, it could easily be changed in a future release with no warning. Thus it would be a mistake to assume that this error always raises AssertionError. (Also, since it is based on assert, the error will not even be raised if enables the optimizer?). >... > There is isRunning(). Isn't that enough? I realize it doesn't > distinguish between "not yet running" and "no longer running", but > your application state should be enough to distinguish between the > two, right? I am trying to determine the state of a thread, and in particular determine if it's safe to try to start it. Thus isRunning is explicitly not enough. The result is in my own code I end up writing a status layer around the thread, but I assume many other programmers have to do the same thing. It seems a rather roundabout way to do things when the threading Thread object ought to have a pretty good idea what state it is in. -- Russell From theller at python.net Thu Feb 26 15:32:54 2004 From: theller at python.net (Thomas Heller) Date: Thu Feb 26 15:33:11 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <5.1.1.6.0.20040226150111.01eaa020@telecommunity.com> (Phillip J. Eby's message of "Thu, 26 Feb 2004 15:12:54 -0500") References: <403E438C.7040208@stackless.com> <06480D1E-6882-11D8-8DBD-000A95686CD8@redivi.com> <403E313A.4070102@stackless.com> <2AEDDF24-6885-11D8-8DBD-000A95686CD8@redivi.com> <403E3422.3010400@stackless.com> <403E438C.7040208@stackless.com> <5.1.1.6.0.20040226150111.01eaa020@telecommunity.com> Message-ID: "Phillip J. Eby" writes: >> > Thomas Heller wrote: >>For me, the advantage of the current >> >> def foo(cls): ... >> foo = classmethod(foo) >> >>is that it *exactly* describes what's going on. >>The difficulty, I would guess, to people coming from other >>languages, is that they are more used to (how can I explain it) a mental >>model where a compiler parses some syntax and generates code from that. >>In python, it *is* simply executable code, and it is executed. > > What you say is true, but it's also true that assembly code describes > *exactly* what is going on. :) I consider a language cool where I can really, really understand what's going on. Well, mostly ;-), I still wouldn't say this from Objects/typeobject.c. > But I think there needs to be a distinction betweeen "how" something > happens, and "what" is happening. For teaching purposes, it's much > easier to say, "if you want to make it a class method, put > '[classmethod]' here", than it is to explain the existing technique. > > If I want to teach someone *how* decorators work (so they can write > their own), there is nothing stopping me from demonstrating the foo = > classmethod(foo) version. > > So, there are two separate issues here. One is *using* decorators, > the other is *making* them. I do not need to understand how > decorators in general work in order to use them, any more than I need > to understand metaclasses to use classes, even though every time I > make a class I'm using a metaclass! I only need to know what the > particular decorator I'm using (or that someone else used) is supposed > to *do* for me. But if I want to *make* a decorator, I need to know > how they work. (Personally, I'd look at the source of other > decorators to figure that out, but I digress.) > > Anyway, the 'foo = classmethod(foo)' thing is actually *noise* when it > comes to teaching, because now you have to explain why you're calling > a function on a function and assigning it to the same name as an > existing function. We've just digressed into functional programming > and the Python namespace mechanisms when all we wanted was to talk > about class methods! A syntax that goes in the 'def' block is less > distracting, because it just becomes part of the "how to define a > class method" rather than opening up all these other issues. > > This sort of "put the language in the background and the task in the > foreground" thing is Python's traditional strength, and I believe that > having a reasonable decorator syntax that takes place as part of the > 'def' statement will allow that strength to apply to classmethods, > staticmethods, and numerous other types of function decorators. All this is more or less what I wanted to express. Although I would consider this def func() [__main__]: ... as an ugly hack - it tries to hide what's going. But any advanced feature can be abused, probably. Thomas From guido at python.org Thu Feb 26 15:34:12 2004 From: guido at python.org (Guido van Rossum) Date: Thu Feb 26 15:35:31 2004 Subject: [Python-Dev] Re: threading issue: preventing starting twice In-Reply-To: Your message of "Thu, 26 Feb 2004 12:17:17 PST." References: <200402252140.i1PLeoT08847@guido.python.org> Message-ID: <200402262034.i1QKYDT12353@guido.python.org> > > There is isRunning(). Isn't that enough? I realize it doesn't [I meant isAlive()] > > distinguish between "not yet running" and "no longer running", but > > your application state should be enough to distinguish between the > > two, right? > > I am trying to determine the state of a thread, and in particular > determine if it's safe to try to start it. Thus isRunning is explicitly > not enough. > > The result is in my own code I end up writing a status layer around the > thread, but I assume many other programmers have to do the same thing. I don't know about that. I almost always create a thread and then immediately start it, so I never have this problem. I assume most other programmers do the same thing. > It seems a rather roundabout way to do things when the threading Thread > object ought to have a pretty good idea what state it is in. I admit, the info is there, but not easily accessible. I copied the Java Thread class, which has the same interface "deficiency". I'm not totally against adding a way to determine a thread's state (with three outcomes: initial, started, stopped) but I'm not quite sure that your use case isn't the result of an awkward way of using threads, in which case I don't want to encourage that. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From rowen at cesmail.net Thu Feb 26 15:47:32 2004 From: rowen at cesmail.net (Russell E. Owen) Date: Thu Feb 26 15:47:57 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: In article , Pete Shinners wrote: > I'm not too concerned about the syntax. All current proposals weigh about > equal to me. Although I don't believe "as" makes documentation lookup much > easier. > > I think this feature will be another one of those positive changes that > effects the entire way python is used. The kind of thing like iterators, > string methods, etc. I've been playing with the idea for awhile and can > throw out some extra use cases. > > Some of these may even be bad ideas apon further thought, but it's something > to ponder. > > > def main() [__main__]: > """specify a function to be run if __name__=='__main__'. this > would need some other python tricks to actually work.""" > > def cleanup() [sys.atexit]: > """ensures this function is called when program quits.""" >... This looks like too much magic to me -- a recipe for making tricky and hard-to-read code. I would rather focus on the heart of PEP 318 and try to make class and static methods easier to declare, e.g. something like: def classmethod foo(self, ...): If one supports arbitrary modifier functions and lists of modifier functions I think readability is really going to suffer. I'm usually in favor of generality, but reading the examples above and others presented in this thread really makes me shudder. -- Russell From skip at pobox.com Thu Feb 26 16:06:07 2004 From: skip at pobox.com (Skip Montanaro) Date: Thu Feb 26 16:06:18 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: <16445.3448.583343.884960@montanaro.dyndns.org> <1077745960.1318.3.camel@localhost> <20040225180723.1F8E.JCARLSON@uci.edu> Message-ID: <16446.24511.194281.328926@montanaro.dyndns.org> Russell> I would rather focus on the heart of PEP 318 and try to make class and Russell> static methods easier to declare, e.g. something like: Russell> def classmethod foo(self, ...): Russell> If one supports arbitrary modifier functions and lists of Russell> modifier functions I think readability is really going to Russell> suffer. -1. If you tie the specific decorators into the language syntax I think you will find it too limiting in the long run (perhaps even in the short run). I suspect many of the planned uses for such an extension will not simply be to define class or static methods. Skip From d.holton at vanderbilt.edu Thu Feb 26 17:02:23 2004 From: d.holton at vanderbilt.edu (Doug Holton) Date: Thu Feb 26 17:02:40 2004 Subject: [Python-Dev] other uses for "as" [was Re: new syntax for wrapping (PEP 318)] In-Reply-To: <5.1.1.6.0.20040226143606.024b6730@telecommunity.com> References: <5.1.1.6.0.20040226143606.024b6730@telecommunity.com> Message-ID: <403E6CEF.2080203@vanderbilt.edu> > would also be sensible syntax. But, the real monkeywrench here is that: > > def func1(x as int, y as float) as staticmethod: > > now looks like it *returns* a staticmethod, which is wrong. > > So, thanks to your argument, I'm now leaning a little more towards using > [] rather than "as", because "as" looks like syntax that should be > reserved for adaptation at a future time. :) You're right. I had forgotten that Visual Basic uses "as" to specify the return type of a function too. I guess [] (or any list object) is better for method decorators. Maybe in the future you could have something like: def myfunc(x as int, y as float) [synchronized, staticmethod] as int: ... So "as" is always and only associated with adaptation. x = myfunc(1, 2.0) --> means x = adapt(myfunc(adapt(1,int),adapt(2.0,float)), int) p = q as int --> means p = adapt(q, int) From bob at redivi.com Thu Feb 26 17:30:06 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 17:26:53 2004 Subject: [Python-Dev] other uses for "as" [was Re: new syntax for wrapping (PEP 318)] In-Reply-To: <403E6CEF.2080203@vanderbilt.edu> References: <5.1.1.6.0.20040226143606.024b6730@telecommunity.com> <403E6CEF.2080203@vanderbilt.edu> Message-ID: <53C72AB0-68AB-11D8-8DBD-000A95686CD8@redivi.com> On Feb 26, 2004, at 5:02 PM, Doug Holton wrote: > >> would also be sensible syntax. But, the real monkeywrench here is >> that: >> def func1(x as int, y as float) as staticmethod: >> now looks like it *returns* a staticmethod, which is wrong. > > >> So, thanks to your argument, I'm now leaning a little more towards >> using [] rather than "as", because "as" looks like syntax that should >> be reserved for adaptation at a future time. :) > > You're right. I had forgotten that Visual Basic uses "as" to specify > the return type of a function too. > > I guess [] (or any list object) is better for method decorators. > > Maybe in the future you could have something like: > > def myfunc(x as int, y as float) [synchronized, staticmethod] as int: > ... > > So "as" is always and only associated with adaptation. > > x = myfunc(1, 2.0) > --> means x = adapt(myfunc(adapt(1,int),adapt(2.0,float)), int) > > p = q as int > --> means p = adapt(q, int) def myfunc(x, y) [accepts(int, int), returns(int)]: pass Nothing stops you from (ab)using this syntax to do it.. but you may need to specify kwarg names twice if you want to skip some, and you would probably have a hard time deciding how to spell adaptation of *args or **kwargs :) -bob From tdelaney at avaya.com Thu Feb 26 18:43:43 2004 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Thu Feb 26 18:43:51 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> Wow - I come into work, and the discussion's exploded :) I've read through the entire thread before writing, so I've been able to absorb the various proposals. I'll summarise them as I see them (I'll leave out the most loony ones Christian ;), and add notes. I'll deliberately *not* use 'staticmethod' or 'classmethod' as I don't believe there's any point to restricting the mechanism to those, and use of those will colour perceptions. Instead, I'm going to use metasyntactic variables. For each I'll throw in some basic formatting (normal case) plus some nasty formatting. 1. Single parameter, single decorator: a. def foo (bar) [baz]: pass b. def foo (bar) as baz: pass c. def [baz] foo (bar): pass d. def baz foo (bar): pass e. def foo [baz] (bar): pass (1c) and (1d) don't parse well to me at all. IMO the function name is the most important information in the declaration - it tells you what the function *is*. (1d) is particularly bad, as it makes it very difficult to pick the function name out. Now, much of this could be obviated by a sufficiently-intelligent editor, but most editors are not that intelligent, and it's better if they're not actually needed. (1e) Doesn't work for me because the parameters are IMO the second-most important info in the declaration. It would be too easy to confuse the decorators with the parameters. (1b) is the most visually pleasing, but the lack of syntax means that the decorator doesn't stand out as well as (a). However, it's simple to syntax-colour 'as', so that's close to a non-issue. (1a) seems to be the most understandable, and is quite visually pleasing - the decorator is obviously special, but feels more like it's an explanatory note. 2. Multiple parameters, multiple decorators: a. def foo (bar, baz) [qux, quux]: pass b. def foo (bar, baz) as qux, quux: pass c. def [qux, quux] foo (bar, baz): pass d. def qux, quux foo (bar, baz): pass e. qux_quux = qux, qux def qux_quux food (bar, baz): pass f. def foo [qux, quux] (bar, baz): pass (2c), (2d) and (2e) suffer from the same problems as (1c) and (1d), but worse - the function name is even *more* hidden. Esp. in (2d). (2f) suffers the same problem as (1e) - the parameters are separated from the function name. (2b) again isn't bad - visually pleasing. The decorators actually stand out better here than in (1b) as they are separated by commas. 3. Multi-line declarations: a. def foo (bar, baz) [qux, quux]: pass b. def foo (bar, baz) as qux, quux: pass c. def [qux, quux] foo (bar, baz): pass d. def qux, quux foo (bar, baz): pass e. def foo [qux, quux] (bar, baz): pass Well, they all look horrible ... ;) (3b) and (3d) don't work without changing the parsing rules. It would be necessary to have a bracketing syntax to make them work. In which case, they become almost equivalent to (3a) and (3c) respectively. In (3e) I found that I started typing 'def foo [bar...' which indicates that my earlier feeling that having the parameters separated from the name is a Bad Thing (TM). (3c) actually works better than the previous examples of this IMP because the function name stands out a lot more. But it's still not that noticeable IMO. (3a) allows the function definition to be laid out in a reasonable (not nice) manner. Of them all I find it the best in this situation. So my conclusion is that for me, the decorators work best following the function definition, and my preference is for [] syntax. *Some* form of bracketing syntax needs to be at least allowable in order to break across multiple lines. > From: Pete Shinners [snip] > I think this feature will be another one of those positive > changes that > effects the entire way python is used. The kind of thing like > iterators, > string methods, etc. I've been playing with the idea for > awhile and can > throw out some extra use cases. I think it will be a *very* important advancement to the language - even if some of the constructs below are somewhat ... dubious ;) > def main() [__main__]: > """specify a function to be run if __name__=='__main__'. this > would need some other python tricks to actually work.""" Urgh! Deep magic! Run away! > def cleanup() [sys.atexit]: > """ensures this function is called when program quits.""" This isn't too bad. Probably better with a wrapper around sys.atexit - something like: def atexit (func): sys.atexit(func) return func def cleanup() [atexit]: otherwise it's not obvious that cleanup gets bound to None. Perhaps it should be a requirement of decorators that they return something other than None, and an exception is thrown if this contract is broken? > def handler(event) [self.button.slot('clicked')]: > """automatically make this function a callback for some > gui slot/signal mechanism.""" Could be useful, but I think it's probably better to assign explicitly. It would depend on how the rest of the code was structured though. > def food(arg) [pysco.optimize]: > """specifically control functions that psyco takes over.""" I really like this. Although I must admit, in most of the programs I write, it's most useful to just do a `psyco.full()` at the start :) > def bard(arg, gar) [validate(int, str)]: > """confirm arguments match a type. validate would > have to be a function that returns a function. odd.""" Interesting idea. I have to think about it more. > def corn() [profiler.track]: > """log specific runtime stats on this function.""" Could be really useful. > def sanitytest() [sys.debugfunc]: > """function is ignored at runtime if python not in > "-O" optimize mode (something like assert).""" Hmm - not so sure I like this. I can see how it would work of course ... def debugfunc (func): if __debug__: return func() def sanitytest() [debugfunc]: Overall, I don't think it would be of much use. > def entrypoint(data) [__all__]: > def hidden(data) [__hidden__]: > """select functions for hiding or passing to 'import *'.""" We've already got a mechanism for this - use underscores to hide names, or declare __all__. I wouldn't see a real use for this. Anyway, those are my thoughts on the whole topic. I'm: +1 on syntax for decorators attached to function definitions +1 on def foo (bar) [baz]: +0 on def foo (bar) as baz: if there's a way to extend it multi-line. -1 on all the other proposed syntaxes. -1 on restricting it to specific decorators such as classmethod Cheers. Tim Delaney From mike at nospam.com Thu Feb 26 19:59:59 2004 From: mike at nospam.com (Mike Rovner) Date: Thu Feb 26 20:00:08 2004 Subject: [Python-Dev] Re: other uses for "as" [was Re: new syntax for wrapping(PEP 318)] References: <5.1.1.6.0.20040226143606.024b6730@telecommunity.com><403E6CEF.2080203@vanderbilt.edu> <53C72AB0-68AB-11D8-8DBD-000A95686CD8@redivi.com> Message-ID: Bob Ippolito wrote: > def myfunc(x, y) [accepts(int, int), returns(int)]: > pass > > Nothing stops you from (ab)using this syntax to do it.. but you may > need to specify kwarg names twice if you want to skip some, and you > would probably have a hard time deciding how to spell adaptation of > *args or **kwargs :) Why? Inventing obcure syntax is an easy part: def myfunc(*x, **y) [accepts([int], {string:int}), returns(int)]: pass Hard part is keep it simple and useful. ;) Mike From python at rcn.com Thu Feb 26 20:04:23 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Feb 26 20:06:20 2004 Subject: [Python-Dev] Accumulation module In-Reply-To: <403CF641.6010007@mcs.vuw.ac.nz> Message-ID: <003d01c3fccd$a381e4c0$e1b52c81@oemcomputer> > > * Is there a way to compute the standard deviation without multiple > > passes over the data (one to compute the mean and a second to sum the > > squares of the differences from the mean? > > > > > > I do not understand the problem in getting stdev in a single pass. > Perhaps I not understood your problem. If you have a series of values > you merely have to calculate their sum and sum-of-squares and use the > usual formula. The sandbox contains the latest and greatest version which computes the standard deviation in a single pass using a recurrence formula that has much better numerical performance than the textbook approach. Raymond Hettinger From bob at redivi.com Thu Feb 26 20:10:37 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 20:07:19 2004 Subject: [Python-Dev] Re: other uses for "as" [was Re: new syntax for wrapping(PEP 318)] In-Reply-To: References: <5.1.1.6.0.20040226143606.024b6730@telecommunity.com><403E6CEF.2080203@vanderbilt.edu> <53C72AB0-68AB-11D8-8DBD-000A95686CD8@redivi.com> Message-ID: On Feb 26, 2004, at 7:59 PM, Mike Rovner wrote: > Bob Ippolito wrote: >> def myfunc(x, y) [accepts(int, int), returns(int)]: >> pass >> >> Nothing stops you from (ab)using this syntax to do it.. but you may >> need to specify kwarg names twice if you want to skip some, and you >> would probably have a hard time deciding how to spell adaptation of >> *args or **kwargs :) > > Why? Inventing obcure syntax is an easy part: > > def myfunc(*x, **y) [accepts([int], {string:int}), returns(int)]: pass > > Hard part is keep it simple and useful. ;) I meant getting the 'collective' to decide .. If you check your mail in a few minutes, I'm sure someone will counter the usefulness of this syntax and suggest a different way :) -bob From python at rcn.com Thu Feb 26 20:13:16 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Feb 26 20:15:15 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines()and StringIO.writelines() In-Reply-To: Message-ID: <004001c3fcce$e17e01e0$e1b52c81@oemcomputer> > This is a big YES! Actually, it's a big no. Alex, immediately and correctly pointed out that what is needed in a suffix rather than a separator and the way to get that is with a generator expression: f.writelines(x+'\n' for x in mylines). I had been led astray because I was experimenting with using cStringIO.writelines() as a basis for implementing str.join() for general iterables without creating an intermediate tuple. Right now, ''.join(it) will unexpectedly consume much more memory than really needed. Raymond Hettinger From barry at python.org Thu Feb 26 20:58:43 2004 From: barry at python.org (Barry Warsaw) Date: Thu Feb 26 20:58:50 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> Message-ID: <1077847123.31953.29.camel@anthem.wooz.org> On Thu, 2004-02-26 at 18:43, Delaney, Timothy C (Timothy) wrote: > This isn't too bad. Probably better with a wrapper around sys.atexit - > something like: > > def atexit (func): > sys.atexit(func) > return func > > def cleanup() [atexit]: > > otherwise it's not obvious that cleanup gets bound to None. Perhaps it > should be a requirement of decorators that they return something other > than None, and an exception is thrown if this contract is broken? Would it be better if it returned 1? Just what /is/ the contract for a decorator function? Maybe that it takes a function/method object and must return a descriptor? I do think this needs some clarification in the PEP. -Barry From bob at redivi.com Thu Feb 26 21:14:40 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 21:11:23 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <1077847123.31953.29.camel@anthem.wooz.org> References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> Message-ID: On Feb 26, 2004, at 8:58 PM, Barry Warsaw wrote: > On Thu, 2004-02-26 at 18:43, Delaney, Timothy C (Timothy) wrote: > >> This isn't too bad. Probably better with a wrapper around sys.atexit - >> something like: >> >> def atexit (func): >> sys.atexit(func) >> return func >> >> def cleanup() [atexit]: >> >> otherwise it's not obvious that cleanup gets bound to None. Perhaps it >> should be a requirement of decorators that they return something other >> than None, and an exception is thrown if this contract is broken? > > Would it be better if it returned 1? > > Just what /is/ the contract for a decorator function? Maybe that it > takes a function/method object and must return a descriptor? I do > think > this needs some clarification in the PEP. Why bother? it doesn't have to be used inside of a class body, it shouldn't have to return a descriptor. It's pretty obvious that some of the most useful purposes include wrapping a function in some kind of closure (like synchronized) or returning a descriptor (like classmethod).. but who knows what else it might be useful for, especially when used on classes themselves, as the patch being kicked around allows you to do (I imagine it would be the end to some metaclass abuses, for example). -bob From fincher.8 at osu.edu Thu Feb 26 21:40:27 2004 From: fincher.8 at osu.edu (Jeremy Fincher) Date: Thu Feb 26 21:39:48 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines()and StringIO.writelines() In-Reply-To: <004001c3fcce$e17e01e0$e1b52c81@oemcomputer> References: <004001c3fcce$e17e01e0$e1b52c81@oemcomputer> Message-ID: <4D64FED7-68CE-11D8-AD8E-000A95AB67E4@osu.edu> On Feb 26, 2004, at 8:13 PM, Raymond Hettinger wrote: >> This is a big YES! > > Actually, it's a big no. Alex, immediately and correctly pointed out > that what is needed in a suffix rather than a separator and the way to > get that is with a generator expression: f.writelines(x+'\n' for x in > mylines). That'll still cause the copying of every string in the iterable; interleaving the newline with the strings themselves would probably be a faster solution, I think. > I had been led astray because I was experimenting with using > cStringIO.writelines() as a basis for implementing str.join() for > general iterables without creating an intermediate tuple. Right now, > ''.join(it) will unexpectedly consume much more memory than really > needed. This sounds like a cool idea, are you still going to implement it? Jeremy From python at rcn.com Thu Feb 26 22:03:26 2004 From: python at rcn.com (Raymond Hettinger) Date: Thu Feb 26 22:05:28 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines()and StringIO.writelines() In-Reply-To: <4D64FED7-68CE-11D8-AD8E-000A95AB67E4@osu.edu> Message-ID: <005301c3fcde$458fd820$e1b52c81@oemcomputer> > > I had been led astray because I was experimenting with using > > cStringIO.writelines() as a basis for implementing str.join() for > > general iterables without creating an intermediate tuple. Right now, > > ''.join(it) will unexpectedly consume much more memory than really > > needed. [Jeremy Fincher] > This sounds like a cool idea, are you still going to implement it? One way or another, I'll make str.join() smarter and faster than it is now. Which way proves to be better is still open. Raymond From bob at redivi.com Thu Feb 26 22:21:53 2004 From: bob at redivi.com (Bob Ippolito) Date: Thu Feb 26 22:18:36 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines()and StringIO.writelines() In-Reply-To: <005301c3fcde$458fd820$e1b52c81@oemcomputer> References: <005301c3fcde$458fd820$e1b52c81@oemcomputer> Message-ID: <17246B5D-68D4-11D8-8DBD-000A95686CD8@redivi.com> On Feb 26, 2004, at 10:03 PM, Raymond Hettinger wrote: >>> I had been led astray because I was experimenting with using >>> cStringIO.writelines() as a basis for implementing str.join() for >>> general iterables without creating an intermediate tuple. Right > now, >>> ''.join(it) will unexpectedly consume much more memory than really >>> needed. > > [Jeremy Fincher] >> This sounds like a cool idea, are you still going to implement it? > > > One way or another, I'll make str.join() smarter and faster than it is > now. Which way proves to be better is still open. In trying to time this in pure python, I discovered that cStringIO.writelines doesn't take generators.. apparently only objects that support len(...) are allowed Currently, it seems that ''.join is faster than cStringIO.writelines anyway, even if you just pass it a big list. from cStringIO import StringIO def join2(sep, seq): sio = StringIO() if not sep: sio.writelines(seq) else: # does not work def join2(): yield seq.next() for s in seq: yield sep yield s sio.writelines(join2()) return sio.getvalue() import time from itertools import repeat import operator def timeit(fn, *args, **kwargs): lst = [] for ig in xrange(100): t0 = time.time() fn(*args, **kwargs) lst.append(time.time() - t0) print fn.__name__ print '', 'max:', max(lst) print '', 'min:', min(lst) print '', 'avg:', reduce(operator.add, lst)/len(lst) if __name__ == '__main__': strings = [' ' * 1000] * 1000 print 'no separator' print '------------' timeit(join2, '', strings) timeit(''.join, strings) [crack:~] bob% python strio.py no separator ------------ join2 max: 0.0453701019287 min: 0.0157110691071 avg: 0.0181557154655 join max: 0.0425598621368 min: 0.00337600708008 avg: 0.00432039260864 From raymond.hettinger at verizon.net Fri Feb 27 03:20:17 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri Feb 27 03:22:21 2004 Subject: [Python-Dev] Idea for a fast calling convention Message-ID: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> I've been working on ways to speedup parameter passing for function calls. It's somewhat expensive to pull parameters off of the stack, assemble them into a new tuple, have the function disassemble the tuple, and ultimately free the tuple. METH_NOARGS and METH_O show nice speed-ups by skipping the tuple packing and unpacking. Since they are handled as special cases, that approach doesn't handle the general case with multiple or optional arguments. My idea is for a new flag, METH_STACK, that generalizes (and potentially replaces) METH_NOARGS AND METH_O. When CALL_FUNCTION sees the flag, it dispatches (*meth)(self, &stack, numargs). On the receiving end, the arguments get retrieved with analogues to PyArg_ParseTuple() and PyArg_UnpackTuple() which access the parameters directly and do not need a tuple as a carrier. Raymond Hettinger From python at rcn.com Fri Feb 27 04:12:48 2004 From: python at rcn.com (Raymond Hettinger) Date: Fri Feb 27 04:14:55 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines()and StringIO.writelines() In-Reply-To: <17246B5D-68D4-11D8-8DBD-000A95686CD8@redivi.com> Message-ID: <009601c3fd11$def9f3a0$e1b52c81@oemcomputer> [Bob Ippolito] > Currently, it seems that ''.join is faster than cStringIO.writelines > anyway, even if you just pass it a big list. I suspect something wrong with your timing suite. Under the hood, cStringIO.writelines() actually is ''.join(). It appears that the whole comparison is circular. Raymond Hettinger ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From bob at redivi.com Fri Feb 27 04:30:33 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Feb 27 04:27:23 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines()and StringIO.writelines() In-Reply-To: <009601c3fd11$def9f3a0$e1b52c81@oemcomputer> References: <009601c3fd11$def9f3a0$e1b52c81@oemcomputer> Message-ID: <9770FE02-6907-11D8-8DBD-000A95686CD8@redivi.com> On Feb 27, 2004, at 4:12 AM, Raymond Hettinger wrote: > [Bob Ippolito] >> Currently, it seems that ''.join is faster than cStringIO.writelines >> anyway, even if you just pass it a big list. > > I suspect something wrong with your timing suite. Under the hood, > cStringIO.writelines() actually is ''.join(). It appears that the > whole comparison is circular. Circular in that "ends up doing approximately the same thing but has a lot more overhead" kind of way ;)... I think the difference is easily explained by: * N calls to join2 * N StringIO() * N StringIO.writelines * N StringIO.getvalue * N StringIO dealloc * [ probably also memcpy's 2 or 3 times as much ] -bob -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2357 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20040227/2f05c7b9/smime-0001.bin From theller at python.net Fri Feb 27 05:55:59 2004 From: theller at python.net (Thomas Heller) Date: Fri Feb 27 05:56:07 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules cStringIO.c, 2.45, 2.46 In-Reply-To: (rhettinger@users.sourceforge.net's message of "Fri, 27 Feb 2004 02:31:01 -0800") References: Message-ID: rhettinger@users.sourceforge.net writes: > if (PyObject_Size(args) < 0) return NULL; > > ! args = PyTuple_Pack(1, args); > ! if (args == NULL) > ! return NULL; > ! tmp = PyObject_Call(joiner, args, NULL); > ! Py_DECREF(args); > UNLESS (tmp) return NULL; There are both tabs and spaces in this code. Question: is there a convention about tabs and spaces in C code? Thomas From mwh at python.net Fri Feb 27 06:40:03 2004 From: mwh at python.net (Michael Hudson) Date: Fri Feb 27 06:40:06 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> (Timothy C. Delaney's message of "Fri, 27 Feb 2004 10:43:43 +1100") References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> Message-ID: <2mr7wgoing.fsf@starship.python.net> "Delaney, Timothy C (Timothy)" writes: > Anyway, those are my thoughts on the whole topic. I'm: Nice work. > +1 on syntax for decorators attached to function definitions > +1 on def foo (bar) [baz]: > +0 on def foo (bar) as baz: if there's a way to extend it multi-line. > -1 on all the other proposed syntaxes. > -1 on restricting it to specific decorators such as classmethod I agree, FWIW. Cheers, mwh -- Gullible editorial staff continues to post links to any and all articles that vaguely criticize Linux in any way. -- Reason #4 for quitting slashdot today, from http://www.cs.washington.edu/homes/klee/misc/slashdot.html From mwh at python.net Fri Feb 27 06:44:29 2004 From: mwh at python.net (Michael Hudson) Date: Fri Feb 27 06:44:32 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> (Raymond Hettinger's message of "Fri, 27 Feb 2004 03:20:17 -0500") References: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> Message-ID: <2mn074oig2.fsf@starship.python.net> "Raymond Hettinger" writes: > I've been working on ways to speedup parameter passing for function > calls. It's somewhat expensive to pull parameters off of the stack, > assemble them into a new tuple, have the function disassemble the tuple, > and ultimately free the tuple. Yahbut... that doesn't actually happen all that often. > METH_NOARGS and METH_O show nice speed-ups by skipping the tuple packing > and unpacking. Since they are handled as special cases, that approach > doesn't handle the general case with multiple or optional arguments. Oh, you're talking about builtin functions. > My idea is for a new flag, METH_STACK, that generalizes (and potentially > replaces) METH_NOARGS AND METH_O. > > When CALL_FUNCTION sees the flag, it dispatches (*meth)(self, &stack, > numargs). > > On the receiving end, the arguments get retrieved with analogues to > PyArg_ParseTuple() and PyArg_UnpackTuple() which access the parameters > directly and do not need a tuple as a carrier. Have you seen my "function optimization reorganization" patch on SF? It's a somewhat different result of somewhat similar thinking. http://python.org/sf/876193 Cheers, mwh -- Java sucks. [...] Java on TV set top boxes will suck so hard it might well inhale people from off their sofa until their heads get wedged in the card slots. --- Jon Rabone, ucam.chat From alan.green at cardboard.nu Fri Feb 27 06:25:30 2004 From: alan.green at cardboard.nu (Alan Green) Date: Fri Feb 27 06:50:21 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> Message-ID: Barry Warsaw python.org> writes: > Just what /is/ the contract for a decorator function? Maybe that it > takes a function/method object and must return a descriptor? I do think > this needs some clarification in the PEP. It would be really spiffy if the decorator were able to run the decorated function and then have access to the function's locals dictionary. It would then be possible to define a property like so: class Foo(object): def bar(self) [property]: """ bar property docstring """ def get(self): return self.__bar def set(self, bar): self.__bar = bar The property decorator would reach inside the locals dictionary amd pull out the special names 'get', 'set' and 'delete', and use them to construct the property descriptor. -- Alan. -- From jf.pieronne at laposte.net Fri Feb 27 06:55:41 2004 From: jf.pieronne at laposte.net (=?ISO-8859-1?Q?Jean-Fran=E7ois_Pi=E9ronne?=) Date: Fri Feb 27 06:55:46 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: References: <403A351B.6060007@laposte.net> <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> <403CCFAC.1090809@laposte.net> Message-ID: <403F303D.3060904@laposte.net> Jack Jansen wrote: > > On 25 Feb 2004, at 17:39, Jean-Fran?ois Pi?ronne wrote: > >> >> Before I submit a patch, I have done the following update which seem >> to fix the problem, any comment are welcome. >> >> patch for include/fileobject.h >> @@ -54,3 +54,7 @@ >> */ >> +#ifdef __VMS >> +#define PY_STDIOTEXTMODE "" >> +#else >> #define PY_STDIOTEXTMODE "b" >> +#endif > > [...] > > I wouldn't call it __VMS: if Python still runs on OS/390 or MVS or > whatever they'll have the same problem. And I/O system where the > underlying filesystem isn't bytestream-based is really what we're > talking about, I think. Correct, none of these patches are OpenVMS specific. Any conditional name suggestion is welcome... Someone has also suggest to simplified #ifdef __VMS /* Compatibility: specifying U in a Python without universal ** newlines is allowed, and the file is opened as a normal text ** file. */ mode = "r"; #else mode = "rb"; #endif to mode = "r" PY_STDIOTEXTMODE; Thanks for your help. Jean-Fran?ois From paul-python at svensson.org Fri Feb 27 06:59:17 2004 From: paul-python at svensson.org (Paul Svensson) Date: Fri Feb 27 06:59:33 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <2mr7wgoing.fsf@starship.python.net> References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <2mr7wgoing.fsf@starship.python.net> Message-ID: <20040227065148.F83384@familjen.svensson.org> "Delaney, Timothy C (Timothy)" writes: > +0 on def foo (bar) as baz: if there's a way to extend it multi-line. Thinking of the proposed change to allow parentheses with import, how about: def foo(bar) as baz, gazonk: pass def foo(bar) as (baz, gazonk): pass def foo(bar) as (baz, gazonk): pass /Paul From bob at redivi.com Fri Feb 27 07:07:07 2004 From: bob at redivi.com (Bob Ippolito) Date: Fri Feb 27 07:03:53 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> Message-ID: <76FF390B-691D-11D8-954B-000A95686CD8@redivi.com> On Feb 27, 2004, at 6:25 AM, Alan Green wrote: > Barry Warsaw python.org> writes: > >> Just what /is/ the contract for a decorator function? Maybe that it >> takes a function/method object and must return a descriptor? I do >> think >> this needs some clarification in the PEP. > > It would be really spiffy if the decorator were able to run the > decorated > function and then have access to the function's locals dictionary. It > would > then be possible to define a property like so: > > class Foo(object): > def bar(self) [property]: > """ bar property docstring """ > def get(self): > return self.__bar > def set(self, bar): > self.__bar = bar > > The property decorator would reach inside the locals dictionary amd > pull out > the special names 'get', 'set' and 'delete', and use them to construct > the > property descriptor. That can already be done with class syntax (and an appropriate metaclass), without touching anything's locals. -bob From skip at pobox.com Fri Feb 27 07:29:43 2004 From: skip at pobox.com (Skip Montanaro) Date: Fri Feb 27 07:30:04 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <403F303D.3060904@laposte.net> References: <403A351B.6060007@laposte.net> <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> <403CCFAC.1090809@laposte.net> <403F303D.3060904@laposte.net> Message-ID: <16447.14391.523534.766801@montanaro.dyndns.org> >> I wouldn't call it __VMS: if Python still runs on OS/390 or MVS or >> whatever they'll have the same problem. And I/O system where the >> underlying filesystem isn't bytestream-based is really what we're >> talking about, I think. Jean-Fran?ois> Correct, none of these patches are OpenVMS specific. Jean-Fran?ois> Any conditional name suggestion is welcome... What about Py_HAS_STREAM_IO? It would be defined by default. On platforms like OpenVMS, you could #undef it in pyport.h or through some other mechanism. Or the contr, something like Py_HAS_NO_STREAM_IO. That doesn't read very well though. Skip From Paul.Moore at atosorigin.com Fri Feb 27 07:32:06 2004 From: Paul.Moore at atosorigin.com (Moore, Paul) Date: Fri Feb 27 07:32:08 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <16E1010E4581B049ABC51D4975CEDB8802C09CFC@UKDCX001.uk.int.atosorigin.com> From: Michael Hudson > "Delaney, Timothy C (Timothy)" writes: >> Anyway, those are my thoughts on the whole topic. I'm: > > Nice work. > >> +1 on syntax for decorators attached to function definitions >> +1 on def foo (bar) [baz]: >> +0 on def foo (bar) as baz: if there's a way to extend it multi-line. >> -1 on all the other proposed syntaxes. >> -1 on restricting it to specific decorators such as classmethod > > I agree, FWIW. Agreed, on all counts. Also, while people are currently coming up with some fairly extreme uses of the new feature, I don't think that's a particularly bad thing. As with any new construct, it can be abused, but in practice, good taste will prevail, and in the meantime, the experimentation just proves the flexibility of the feature. Paul. From pedronis at bluewin.ch Fri Feb 27 08:13:40 2004 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Fri Feb 27 08:09:19 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> Message-ID: <5.2.1.1.0.20040227140748.02aef568@pop.bluewin.ch> At 11:25 27.02.2004 +0000, Alan Green wrote: >Barry Warsaw python.org> writes: > > > Just what /is/ the contract for a decorator function? Maybe that it > > takes a function/method object and must return a descriptor? I do think > > this needs some clarification in the PEP. > >It would be really spiffy if the decorator were able to run the decorated >function and then have access to the function's locals dictionary. It would >then be possible to define a property like so: > >class Foo(object): > def bar(self) [property]: > """ bar property docstring """ > def get(self): > return self.__bar > def set(self, bar): > self.__bar = bar > >The property decorator would reach inside the locals dictionary amd pull out >the special names 'get', 'set' and 'delete', and use them to construct the >property descriptor. > please, leave the can of worms of "blocks" etc closed, btw class bar(self) [property]: can achieve this, but it's really a sucky syntax for that. It's really an orthogonal issue wrt PEP318. If someone want to open it, it should really write a separate PEP aftering re-reading the old threads. Thanks. From andersjm at dancontrol.dk Fri Feb 27 08:23:10 2004 From: andersjm at dancontrol.dk (Anders J. Munch) Date: Fri Feb 27 08:23:17 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com><2mr7wgoing.fsf@starship.python.net> <20040227065148.F83384@familjen.svensson.org> Message-ID: <002e01c3fd34$d8785f80$f901a8c0@hirsevej.dk> From: "Paul Svensson" > "Delaney, Timothy C (Timothy)" writes: > > > +0 on def foo (bar) as baz: if there's a way to extend it multi-line. > > Thinking of the proposed change to allow parentheses with import, how about: > > def foo(bar) as baz, gazonk: pass > > def foo(bar) as (baz, gazonk): pass > > def foo(bar) as (baz, > gazonk): pass Unlike import, we are dealing will full-blown expressions here, not just single identifiers, so you'd have to figure out how to deal with e.g.: def foo(bar) as (baz), gazonk: pass # legal? A variation with cleaner implementation: Let the decorator-list be a single expression. If that expression evaluates to a tuple, treat it like a sequence of decorator functions. If not, treat it like a single decorator. +1 on def foo(bar) as baz +0 on def foo(bar)[baz] - Anders From pje at telecommunity.com Fri Feb 27 09:10:45 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Feb 27 09:07:27 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> Message-ID: <5.1.0.14.0.20040227090757.02407410@mail.telecommunity.com> At 11:25 AM 2/27/04 +0000, Alan Green wrote: >Barry Warsaw python.org> writes: > > > Just what /is/ the contract for a decorator function? Maybe that it > > takes a function/method object and must return a descriptor? I do think > > this needs some clarification in the PEP. > >It would be really spiffy if the decorator were able to run the decorated >function and then have access to the function's locals dictionary. It would >then be possible to define a property like so: > >class Foo(object): > def bar(self) [property]: > """ bar property docstring """ > def get(self): > return self.__bar > def set(self, bar): > self.__bar = bar > >The property decorator would reach inside the locals dictionary amd pull out >the special names 'get', 'set' and 'delete', and use them to construct the >property descriptor. -1. There were better alternatives proposed in the previous discussion on PEP 218 and properties, like: def bar(self) [property_get]: # ... def bar(self,value) [property_set]: # ... def bar(self) [property_del]: # ... From barry at python.org Fri Feb 27 09:46:29 2004 From: barry at python.org (Barry Warsaw) Date: Fri Feb 27 09:46:40 2004 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Modules cStringIO.c, 2.45, 2.46 In-Reply-To: References: Message-ID: <1077893189.585.36.camel@anthem.wooz.org> On Fri, 2004-02-27 at 05:55, Thomas Heller wrote: > rhettinger@users.sourceforge.net writes: > > > if (PyObject_Size(args) < 0) return NULL; > > > > ! args = PyTuple_Pack(1, args); > > ! if (args == NULL) > > ! return NULL; > > ! tmp = PyObject_Call(joiner, args, NULL); > > ! Py_DECREF(args); > > UNLESS (tmp) return NULL; > > There are both tabs and spaces in this code. Question: is there a > convention about tabs and spaces in C code? Personally, I'd like to see all the Python C code be reformatted to 4 space indents, or at least, new C modules to be indented that way. However, PEP 7 is the C style guide for Python code, and it clearly lays out the rule: - Use single-tab indents, where a tab is worth 8 spaces. and: (2) To be consistent with surrounding code that also breaks [a rule] (maybe for historic reasons) -- although this is also an opportunity to clean up someone else's mess (in true XP style) So. Be consistent but default to using tabs. -Barry http://www.python.org/peps/pep-0007.html From aahz at pythoncraft.com Fri Feb 27 10:11:21 2004 From: aahz at pythoncraft.com (Aahz) Date: Fri Feb 27 10:11:32 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <16447.14391.523534.766801@montanaro.dyndns.org> References: <403A351B.6060007@laposte.net> <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> <403CCFAC.1090809@laposte.net> <403F303D.3060904@laposte.net> <16447.14391.523534.766801@montanaro.dyndns.org> Message-ID: <20040227151120.GA22220@panix.com> On Fri, Feb 27, 2004, Skip Montanaro wrote: > > What about Py_HAS_STREAM_IO? It would be defined by default. On platforms > like OpenVMS, you could #undef it in pyport.h or through some other > mechanism. > > Or the contr, something like Py_HAS_NO_STREAM_IO. That doesn't read very > well though. Py_HAS_RECORD_IO -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Do not taunt happy fun for loops. Do not change lists you are looping over." --Remco Gerlich, comp.lang.python From jim.jewett at eds.com Fri Feb 27 10:13:38 2004 From: jim.jewett at eds.com (Jewett, Jim J) Date: Fri Feb 27 10:13:36 2004 Subject: [Python-Dev] new syntax for wrapping (PEP 318) - Timothy's summary Message-ID: Delaney, Timothy C (Timothy): > a. def foo (bar) [baz]: > b. def foo (bar) as baz: > c. def [baz] foo (bar): Note that c. is like Quixote > d. def baz foo (bar): > e. def foo [baz] (bar): > (1b) is the most visually pleasing, but the > lack of syntax means that the decorator doesn't > stand out as well as (a). However, it's simple > to syntax-colour 'as', so that's close to a non-issue. > (1a) seems to be the most understandable, and is > quite visually pleasing - the decorator is obviously special, > but feels more like it's an explanatory note. You left out f. def foo (bar) mod [baz]: pass The current patch is not really using a list, it is using magic characters "[" and "]". Given that, it could as easily use magic characters "mod [" and "]" Option 1.f. still has all the syntax-coloring advantages of 1.a., but makes it more obvious that the "mod [baz]" is truly an optional annotation. ("as" is the shortest suggestion for "mod", but not the only one. Also suggested are many variations on "transformed_by", "modified_by", "decorators", "providing", "extended_by", etc.) -jJ From jf.pieronne at laposte.net Fri Feb 27 10:42:24 2004 From: jf.pieronne at laposte.net (=?ISO-8859-1?Q?Jean-Fran=E7ois_Pi=E9ronne?=) Date: Fri Feb 27 10:42:31 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support In-Reply-To: <20040227151120.GA22220@panix.com> References: <403A351B.6060007@laposte.net> <9B4E641A-677B-11D8-9389-000A958D1666@cwi.nl> <403CCFAC.1090809@laposte.net> <403F303D.3060904@laposte.net> <16447.14391.523534.766801@montanaro.dyndns.org> <20040227151120.GA22220@panix.com> Message-ID: <403F6560.2080809@laposte.net> Aahz wrote: > On Fri, Feb 27, 2004, Skip Montanaro wrote: > >>What about Py_HAS_STREAM_IO? It would be defined by default. On platforms >>like OpenVMS, you could #undef it in pyport.h or through some other >>mechanism. >> >>Or the contr, something like Py_HAS_NO_STREAM_IO. That doesn't read very >>well though. > > > Py_HAS_RECORD_IO What about Py_USE_RECORD_IO which would need to be defined only on VMS and probably a few others platforms and is undefined by default. What is the simplest method for you? Cheers, Jean-Fran?ois From pete at shinners.org Fri Feb 27 10:55:51 2004 From: pete at shinners.org (Pete Shinners) Date: Fri Feb 27 10:56:00 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <2mr7wgoing.fsf@starship.python.net> References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <2mr7wgoing.fsf@starship.python.net> Message-ID: Michael Hudson wrote: >> +0 on def foo (bar) as baz: if there's a way to extend it multi-line. Can the Python parser allow "def" and ":" to become a sort of implied braces? This would allow multiline definitions without the need of an extra set of braces. def foo(bar) as x, y, zee: Although neither of those lines look correct in my mind. Just another part of the thought process I guess. From barry at barrys-emacs.org Fri Feb 27 14:31:44 2004 From: barry at barrys-emacs.org (Barry Scott) Date: Fri Feb 27 14:32:17 2004 Subject: [Python-Dev] What is the ":" for in python? Message-ID: <6.0.3.0.2.20040227192602.0309e1f0@torment.chelsea.private> I was asked why python required a ":" in its syntax and then saw Pete Shinners query on changing the parser to treat "def" to ":" as a unit. What is the purpose of the ":"? Why doesn't the parser work as Pete suggests already? Of course the same rules should be true of "if" and ":", "while" and ":" I've always wanted to be able to write: if x > 10 and x < 20: do_in_range() rather then if( x > 10 and x < 20 ): do_in_range() Barry From tree at basistech.com Fri Feb 27 14:39:11 2004 From: tree at basistech.com (Tom Emerson) Date: Fri Feb 27 14:41:16 2004 Subject: [Python-Dev] What is the ":" for in python? In-Reply-To: <6.0.3.0.2.20040227192602.0309e1f0@torment.chelsea.private> References: <6.0.3.0.2.20040227192602.0309e1f0@torment.chelsea.private> Message-ID: <16447.40159.39178.907890@magrathea.basistech.com> Barry Scott writes: > if x > 10 > and x < 20: > do_in_range() > > rather then > > if( x > 10 > and x < 20 ): > do_in_range() You could write if x > 10 \ and x < 20: do_in_range() -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From tismer at stackless.com Fri Feb 27 14:46:11 2004 From: tismer at stackless.com (Christian Tismer) Date: Fri Feb 27 14:46:16 2004 Subject: [Python-Dev] Sprint on Stackless Python March 10-14 in Berlin In-Reply-To: References: <40291EEB.7010201@stackless.com> <402A47CC.1060606@stackless.com> <402A5DD1.1070903@stackless.com> Message-ID: <403F9E83.10502@stackless.com> As already announced, the *********************************************** *** *** *** S t a c k l e s s S p r i n t *** *** *** *********************************************** has now a settled date: It will take place in Berlin, March 10-14 2004. We will have a room in the Free University of Berlin in Berlin-Dahlem. Address and room will be announced, soon. How to attend? See bottom of this message. Contents? Well, there is a lot possible. Current topics which come into mind are * autoscheduling * scheduling object * brain storming! * Zope Wiki and Documentation * making channels really stackless * demo applications * more regression tests * real zope apps? * refactoring Stackless: simpler and configurable * create a Stackless tutorial * minimalist Stackless Python with no hardware dependency * assembly-free Stackless with setjmp/longjmp * spreading the internals between developers but this is completely open for discussion. Level? Well, it will be a bit simpler than the pypy-sprints, which I think are very difficult, but Stackless has its built-in difficulties by nature. Usability? This is by far, now and forever, the best possible way to learn about Stackless Python and to become a core developer. But no guarantee possible :-) Payback? Well, it is intended that Stackless Python should be developed further. If you are using or planning to use Stackless for your products, you are highly encouraged to support Stackless Python by money and by submitting your best developers, regardless how painfully they are going to be missed by the company. Coaching? Yes, it is probable and whished, that Stackless Python will become a topic for training and coaching. First of all, we will try to leverage our knowledge. Fun? Guaranteed. Schedule -------- There is no particular schedule, yet, but for the first day. I am planning to give a talk on Stackless technology, and to be available for questions and discussion during the whole first day. Attendees with small or no knowledge about Stackless Python are strongly encouraged to join that session. Accomodation ------------ There are no special arrangements planned. We hope to find a place for every external sprinter at the home of the Berlin habitants. I'm taking two already, please don't ask. How to attend: -------------- Please subscribe to the mailing list stackless-sprint@stackless.com which you can find via this interface: http://www.stackless.com/mailman/listinfo/stackless-sprint By subscribing, you state that you want to come to the sprint. We have only limited reources left and can take only a few. ------- I hereby wish to thank *Dirk Pape* and *Christoph v. Stuckrad* from FU Berlin for making this event possible! cheers - chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mcherm at mcherm.com Fri Feb 27 14:54:58 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Fri Feb 27 14:55:01 2004 Subject: [Python-Dev] What is the ":" for in python? Message-ID: <1077911698.403fa092c8378@mcherm.com> > I was asked why python required a ":" in its syntax and then saw > Pete Shinners query on changing the parser to treat "def" to ":" as a unit. > > What is the purpose of the ":"? > > Why doesn't the parser work as Pete suggests already? > Of course the same rules should be true of "if" and ":", "while" and ":" If I recall correctly, the original reason for the ":" is that the designers of ABC (which provided much of the original inspiration for Python) conducted a study and found that people recognized block boundaries better if they were introduced by a ":" in addition to being indentation delimited then if they were just indentation delimited. For more details see this posting of Tim's: http://mail.python.org/pipermail/python-list/2000-December/022506.html > if x > 10 > and x < 20: > do_in_range() > > rather then > > if( x > 10 > and x < 20 ): > do_in_range() Interesting. Everyone has their own preferences, but I find BOTH of these very difficult to read. I would prefer if (x > 10 and x < 20): do_in_range() Of course if (10 < x < 20): do_in_range() is even better . -- Michael Chermside From pje at telecommunity.com Fri Feb 27 14:56:22 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Feb 27 14:56:35 2004 Subject: [Python-Dev] Distutils dependencies sprint at PyCon? Message-ID: <5.1.1.6.0.20040227142938.01ea8030@mail.rapidsite.net> Bob Ippolito and I have been talking about working on distutils dependency support while at PyCon. The subject has recently come up on both the Zope3-Dev and Twisted mailing lists, with respect to being able to: 1) break up monolithic systems (PEAK, Zope, Twisted) into smaller package sets with dependencies 2) allow dependencies on other systems (e.g. Twisted using PyProtocols, PEAK using zope.publisher, etc.) 3) support painless install for end users (single command to download and install "everything needed") even at the cost of a little pain for the packager(s). 4) allow dependencies on packages that were packaged with the distutils, but *not* specifically designed to work with the dependency system. In other words, if I want to depend on package X, I should not need to bug package X's author, I should be able to "just do it". 5) Support Python 2.2, while not requiring changes to the distutils, but being a possible candidate for upgrades to the distutils in 2.4. If there's a group that would like to sprint to produce working deliverables for one or more aspects of this problem area (and I think PIMP/PackMan shows that solutions are within reach), I'd like to get some schedule commitments so I can plan a trip. I wasn't originally planning to go to PyCon this year, but I *will* come if it means we can get this thing implemented. If there is sufficient interest, let's take this to an appropriate mailing list (perhaps distutils-sig?) and Wiki to put together a detailed set of stories, as long as we can prioritize the list according to the needs of the folks who will be doing the development. If you're interested in participating in such a sprint, please let me know ASAP, so I can make travel arrangements. Thanks. (Please reply to me personally rather than to the list(s); I will summarize and post an announcement of where to direct further discussion, if there's enough interest to get this off the ground. Thanks!) From mcherm at mcherm.com Fri Feb 27 15:22:53 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Fri Feb 27 15:22:55 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) Message-ID: <1077913373.403fa71d7ed9d@mcherm.com> I've been reading the hailstorm of comments on PEP 318. Yipes... did I help start that off? Lots of good ideas (and some others too) have been raised. I thought I would just weigh in with what I have learned so far. (1) Starting with Pete Shiner's creative list and butressed by subsequent suggestions, I am more convinced than ever that function decorators are actually VERY USEFUL. I'm now strongly +1 on PEP 318 with SOME syntax. (2) I originally objected to "def foo (bar,baz) [qux, quux]:" on the grounds that it relied too heavily on punctuation and using a keyword ("def foo (bar,baz) as qux, quux") would be better. I think that most of my objections have been addressed: * I said "but what would newbies look up", but then Paul Moore pointed out "Who, as a new user, saw 'import something as another' and looked up 'as' rather than 'import'?". He's right. * I said "def marmalade(self) [classmethod]:" versus "def marmalade(self): [classmethod]" is confusing, but then I think that two objections still stand: * "def foo (bar,baz) [qux, quux]:" is certainly not "executable pseudocode". * This is yet another meaning for "[]" in Python, it is certainly not indexing, and it differs subtly from list literals an list comprehensions. I suppose this objection could be mitigated by declaring that any expression evaluating to an iterable could be used (and sane people would use a list literal). However, there are objections to all other syntaxes that have been proposed, and I'm beginning to think that "def foo (bar,baz) [qux, quux]:" may be the best of the bunch despite these shortcomings. -- Michael Chermside From mcherm at mcherm.com Fri Feb 27 15:33:00 2004 From: mcherm at mcherm.com (Michael Chermside) Date: Fri Feb 27 15:33:11 2004 Subject: [Python-Dev] OpenVMS file system and UNIVERSAL_NEWLINES support Message-ID: <1077913980.403fa97c63f02@mcherm.com> Skip writes: > What about Py_HAS_STREAM_IO? [...] > Or the contr, something like Py_HAS_NO_STREAM_IO. That doesn't read very > well though. Py_LACKS_STREAM_IO -- Michael Chermside From alan.green at cardboard.nu Fri Feb 27 15:51:17 2004 From: alan.green at cardboard.nu (Alan Green) Date: Fri Feb 27 15:51:25 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> <5.2.1.1.0.20040227140748.02aef568@pop.bluewin.ch> Message-ID: Samuele Pedroni bluewin.ch> writes: > At 11:25 27.02.2004 +0000, Alan Green wrote: > >It would be really spiffy if the decorator were able to run the decorated > >function and then have access to the function's locals dictionary. It would > >then be possible to define a property like so: > > > >class Foo(object): > > def bar(self) [property]: > > """ bar property docstring """ > > def get(self): > > return self.__bar > > def set(self, bar): > > self.__bar = bar > please, leave the can of worms of "blocks" etc closed, I see you your point. > btw > > class bar(self) [property]: > > can achieve this, but it's really a sucky syntax for that. Agreed. Can already something like that in Python 2.3 with: class Foo: class bar(propertydef): def get(self): ... etc ... And it's ugly too. > It's really an > orthogonal issue wrt PEP318. If someone want to open it, it should really > write a separate PEP aftering re-reading the old threads. OK. Would you be able give me some pointers to the old threads or some clues about what to search on? I've obviously been looking in the wrong places. -- Alan -- From alan.green at cardboard.nu Fri Feb 27 16:00:35 2004 From: alan.green at cardboard.nu (Alan Green) Date: Fri Feb 27 16:12:46 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> <5.1.0.14.0.20040227090757.02407410@mail.telecommunity.com> Message-ID: Phillip J. Eby telecommunity.com> writes: > At 11:25 AM 2/27/04 +0000, Alan Green wrote: > >It would be really spiffy if the decorator were able to run the decorated > >function and then have access to the function's locals dictionary. It would > >then be possible to define a property like so: > > > >class Foo(object): > > def bar(self) [property]: > > """ bar property docstring """ > > def get(self): > > return self.__bar > > def set(self, bar): > > self.__bar = bar > > -1. There were better alternatives proposed in the previous discussion on > PEP 218 and properties, like: > > def bar(self) [property_get]: > # ... > > def bar(self,value) [property_set]: > # ... > > def bar(self) [property_del]: > # ... defs-inside-defs is a bit wierd, but has some advantages over individual methods: * Tells the reader that the get/set/del belong together * Gives a name to the property once, rather than one name three times * Makes a sensible place for the property's docstring Perhaps it might make more sense as: class Foo(object): def [property] bar: def get(self): ...etc... In which case, it truly is getting beyond the scope of PEP318. -- Alan -- From fdrake at acm.org Fri Feb 27 16:49:48 2004 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri Feb 27 16:50:35 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> References: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> Message-ID: <200402271649.48481.fdrake@acm.org> On Friday 27 February 2004 03:20 am, Raymond Hettinger wrote: > I've been working on ways to speedup parameter passing for function > calls. It's somewhat expensive to pull parameters off of the stack, > assemble them into a new tuple, have the function disassemble the tuple, > and ultimately free the tuple. ... > My idea is for a new flag, METH_STACK, that generalizes (and potentially > replaces) METH_NOARGS AND METH_O. How would METH_STACK methods work when called via func(*args) or via the various functions of the C API? The nice thing about METH_NOARGS and METH_O is that they describe what the called function needs. METH_STACK sounds like a core dump waiting to happen. ;-( -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From pf_moore at yahoo.co.uk Fri Feb 27 16:57:19 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Fri Feb 27 16:57:02 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) References: <1077913373.403fa71d7ed9d@mcherm.com> Message-ID: Michael Chermside writes: > I think that two objections still stand: > > * "def foo (bar,baz) [qux, quux]:" is certainly not "executable > pseudocode". This has been mentioned a couple of times. I'm not sure I see the issue, though. With concrete examples, it *does* read like executable pseudocode to me: def factory(cls, arg1, arg2) [classmethod]: ... reads to me as "define a factory function, with arguments cls, arg1, arg2 - it's a class method". def tidy_up() [onexit]: reads as "define the tidy_up function which is called on exit from the program". Getting the decorator names right is the key thing, I think. Paul. -- This signature intentionally left blank From pje at telecommunity.com Fri Feb 27 17:16:06 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Feb 27 17:16:13 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <200402271649.48481.fdrake@acm.org> References: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> Message-ID: <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> At 04:49 PM 2/27/04 -0500, Fred L. Drake, Jr. wrote: >On Friday 27 February 2004 03:20 am, Raymond Hettinger wrote: > > I've been working on ways to speedup parameter passing for function > > calls. It's somewhat expensive to pull parameters off of the stack, > > assemble them into a new tuple, have the function disassemble the tuple, > > and ultimately free the tuple. >.. > > My idea is for a new flag, METH_STACK, that generalizes (and potentially > > replaces) METH_NOARGS AND METH_O. > >How would METH_STACK methods work when called via func(*args) or via the >various functions of the C API? > >The nice thing about METH_NOARGS and METH_O is that they describe what the >called function needs. METH_STACK sounds like a core dump waiting to >happen. >;-( Maybe there should instead be a tp_call_stack slot. Then the various CALL opcodes would call that slot instead of tp_call. C API calls would still go through tp_call. 'object' would define tp_call_stack to call tp_call, using information from the stack. Python function and method objects would then override tp_call_stack to implement the shortcut form. In practice, though, I expect it would be faster to do as Jython and IronPython have done, and define a set of tp_call1, tp_call2, etc. slots that are optimized for specific calling situations, allowing C API calls to be sped up as well, provided you used things like PyObject_Call1(ob,arg), PyObject_Call2(ob,arg1,arg2), and so on. Perhaps there is some information that can be gleaned from the Jython research as to what are the most common number of positional parameters for calls. From bkc at murkworks.com Fri Feb 27 18:00:09 2004 From: bkc at murkworks.com (Brad Clements) Date: Fri Feb 27 17:56:59 2004 Subject: [Python-Dev] How to debug pyexpat SIGSEGV with GDB? Message-ID: <403F84D3.15067.13B6DF33@localhost> Sorry if this is OT. I've hit a repeatable segfault in pyexpat on RH Linux 9 with Python 2.3.3 (I'm trying rss2email). It seems that XML_parse is returning an error, but when XML_GetCurrentLineNumber is called positionPtr is not valid. I catch this in GDB and have looked at the stack (see below), but when I get back up the stack into PyCFunction_Call I don't know what to do. Ideally, I want to find out the Python source file and line number that is currently being executed, then look at the Python source to figure out exactly which pyexpat call is being made just before the call to get_parse_result. It seems strange that (it appears) that Python code is calling get_parse_result directly. So, how can I figure out where in the Python source the function call is coming from using gdb? I'm sure it involves "print" and some casts.. I couldn't find a howto on python.org -- Python 2.3.3 (#1, Dec 22 2003, 14:01:09) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Starting program: /usr/local/bin/python2.3 rss2email.py feeds.dat run --no-send [New Thread 1074948352 (LWP 2379)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1074948352 (LWP 2379)] normal_updatePosition (enc=0x407211c0, ptr=0x40785000
, end=0x823ac09 "?teborg, Sweden, 7-9 June 2004. To help us serve the communit y in the best way possible, we need your input on what you think we should do in the way of tutorials.\"]\n\n\n

\n\n\n

, end=0x823ac09 "?teborg, Sweden, 7-9 June 2004. To help us serve the communit y in the best way possible, we need your input on what you think we should do in the way of tutorials.\"]\n\n\n

\n\n\n

, end=0x823ac09 "?teborg, Sweden, 7-9 June 2004." pos=0x8238954) at xmltok_impl.c:1745 1745 switch (BYTE_TYPE(enc, ptr)) { (gdb) list 1740 const char *ptr, 1741 const char *end, 1742 POSITION *pos) 1743 { 1744 while (ptr != end) { 1745 switch (BYTE_TYPE(enc, ptr)) { 1746 #define LEAD_CASE(n) \ 1747 case BT_LEAD ## n: \ 1748 ptr += n; \ 1749 break; (gdb) (gdb) frame 1 #1 0x40701804 in XML_GetCurrentLineNumber (parser=0x82387c0) at /usr/local/src/Python-2.3.3/Modules/expat/xmlparse.c:1605 1605 XmlUpdatePosition(encoding, positionPtr, eventPtr, &position); (gdb) list 1600 1601 int XMLCALL 1602 XML_GetCurrentLineNumber(XML_Parser parser) 1603 { 1604 if (eventPtr) { 1605 XmlUpdatePosition(encoding, positionPtr, eventPtr, &position); 1606 positionPtr = eventPtr; 1607 } 1608 return position.lineNumber + 1; 1609 } (gdb) frame 2 #2 0x406ff800 in set_error (self=0x4067e8ec, code=XML_ERROR_INVALID_TOKEN) at /usr/local/src/Python-2.3.3/Modules/pyexpat.c:124 124 int lineno = XML_GetErrorLineNumber(parser); (gdb) list 119 set_error(xmlparseobject *self, enum XML_Error code) 120 { 121 PyObject *err; 122 char buffer[256]; 123 XML_Parser parser = self->itself; 124 int lineno = XML_GetErrorLineNumber(parser); 125 int column = XML_GetErrorColumnNumber(parser); 126 127 /* There is no risk of overflowing this buffer, since 128 even for 64-bit integers, there is sufficient space. */ (gdb) print *parser $2 = {m_userData = 0x2, m_handlerArg = 0x40720ac0, m_buffer = 0x82387c0 "??g@??g@\b@v@TA\005\b$D\005\b?E\005\b\b@v@=Dw@\b@x@5\004\001", m_mem = {malloc_fcn = 0x1, realloc_fcn = 0, free_fcn = 0}, m_bufferPtr = 0x0, m_bufferEnd = 0x1

, m_bufferLim = 0x0, m_parseEndByteIndex = 8192, m_parseEndPtr = 0x0, m_dataBuf = 0x0, m_dataBufEnd = 0x81e66b0 "<\224f@?\222f@\204\222f@?\223f@?\222f@\024\224f@, m_characterDataHandler = 0x23, m_processingInstructionHandler = 0xffffffff, m_commentHandler = 0, m_startCdataSectionHandler = 0x7273752f, m_endCdataSectionHandler = 0x636f6c2f, m_defaultHandler = 0x6c2f6c61, m_startDoctypeDeclHandler = 0x702f6269, m_endDoctypeDeclHandler = 0x6f687479, m_unparsedEntityDeclHandler = 0x332e326e, m_notationDeclHandler = 0x6165772f, m_startNamespaceDeclHandler = 0x6665726b, m_endNamespaceDeclHandler = 0x79702e, m_notStandaloneHandler = 0, m_externalEntityRefHandler = 0, m_externalEntityRefHandlerArg = 0x1, m_skippedEntityHandler = 0x8117660 , m_unknownEncodingHandler = 0x2, m_elementDeclHandler = 0x2, m_attlistDeclHandler = 0x2, m_entityDeclHandler = 0x43, m_xmlDeclHandler = 0x4067c3a0, m_encoding = 0x40677fac, m_initEncoding = { initEnc = {scanners = {0x4067d3bc, 0x40643e0c, 0x4012802c, 0x4012802c}, literalScanners = {0x4067e920, 0x4067f180}, sameName = 0x122, nameMatchesAscii = 0x4067f1a0, nameLength = 0x1, skipS = 0x810a7a0 , getAtts = 0x23, charRefNumber = 0xffffffff, predefinedEntityName = 0, updatePosition = 0x7273752f, isPublicId = 0x636f6c2f, utf8Convert = 0x6c2f6c61, utf16Convert = 0x702f6269, minBytesPerChar = 1869116537, isUtf8 = 110 'n', isUtf16 = 50 '2'}, encPtr = 0x6165772f}, m_internalEncoding = 0x6665726b, m_protocolEncodingName = 0x79702e
, m_ns = 0 '\0', m_ns_triplets = 0 '\0', m_unknownEncodingMem = 0x0, m_unknownEncodingData = 0x1, m_unknownEncodingHandlerData = 0x8117660, m_unknownEncodingRelease = 0x1, m_prologState = {handler = 0x4, level = 2, role_none = 67, includeLevel = 1080537744, documentEntity = 1080310604, inEntityValue = 1080546284}, m_processor = 0x40678e14, m_errorCode = 1074954284, m_eventPtr = 0x4012802c "}\v", m_eventEndPtr = 0x4067e9a0 "\001", m_positionPtr = 0x4067f220 "\002", m_openInternalEntities = 0x125, m_defaultExpandInternalEntities = 48 '0', m_tagLevel = 1080552288, m_declEntity = 0x810a7a0, m_doctypeName = 0x23
, m_doctypeSysid = 0xffffffff
, m_doctypePubid = 0x0, m_declAttributeType = 0x7273752f
, m_declNotationName = 0x636f6c2f
, m_declNotationPublicId = 0x6c2f6c61
, m_declElementType = 0x702f6269, m_declAttributeId = 0x6f687479, m_declAttributeIsCdata = 110 'n', m_declAttributeIsId = 50 '2', m_dtd = 0x6165772f, m_curBase = 0x6665726b
, m_tagStack = 0x79702e, m_freeTagList = 0x0, m_inheritedBindings = 0x0, m_freeBindingList = 0x4067ea20, m_attsSize = 135362144, m_nSpecifiedAtts = 0, m_idAttIndex = 0, m_atts = 0x1, m_nsAtts = 0x42, m_nsAttsVersion = 1080542056, m_nsAttsPower = 236 '?', m_position = { lineNumber = 1080528484, columnNumber = 1074954284}, m_tempPool = { blocks = 0x4012802c, freeBlocks = 0x4012802c, end = 0x4067ea20 "`?g@??\020\b#", ptr = 0x4067d410 " \020g@@\215g@????\001", start = 0x121
, mem = 0x4067f1c0}, m_temp2Pool = {blocks = 0x1, freeBlocks = 0x810a7a0, end = 0x22
, ptr = 0x5e84331
, start = 0x0, mem = 0x70747468}, m_groupConnector = 0x6f2f2f3a
, m_groupSize = 1852402798, m_namespaceSeparator = 101 'e', m_parentParser = 0x746f6266, m_isParamEntity = 46 '.', m_useForeignDTD = 111 'o', m_paramEntityParsing = 1852795427} (gdb) (gdb) frame 3 #3 0x406fe324 in xmlparse_Parse (self=0x4067e8ec, args=0x4064456c) at /usr/local/src/Python-2.3.3/Modules/pyexpat.c:888 888 return set_error(self, XML_GetErrorCode(self->itself)); (gdb) list 883 { 884 if (PyErr_Occurred()) { 885 return NULL; 886 } 887 if (rv == 0) { 888 return set_error(self, XML_GetErrorCode(self->itself)); 889 } 890 if (flush_character_buffer(self) < 0) { 891 return NULL; 892 } (gdb) frame 4 #4 0x080ed614 in PyCFunction_Call (func=0x4067f38c, arg=0x4064456c, kw=0x40785000) at ../Python-2.3.3/Objects/methodobject.c:108 108 return (*meth)(self, arg); (gdb) list 103 size = PyTuple_GET_SIZE(arg); 104 if (size == 1) 105 arg = PyTuple_GET_ITEM(arg, 0); 106 else if (size == 0) 107 arg = NULL; 108 return (*meth)(self, arg); 109 } 110 break; 111 default: 112 PyErr_BadInternalCall(); -- Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements From tismer at stackless.com Fri Feb 27 22:12:00 2004 From: tismer at stackless.com (Christian Tismer) Date: Fri Feb 27 22:12:09 2004 Subject: [Python-Dev] How to debug pyexpat SIGSEGV with GDB? In-Reply-To: <403F84D3.15067.13B6DF33@localhost> References: <403F84D3.15067.13B6DF33@localhost> Message-ID: <40400700.3020900@stackless.com> Brad Clements wrote: > Sorry if this is OT. > > I've hit a repeatable segfault in pyexpat on RH Linux 9 with Python 2.3.3 (I'm trying > rss2email). > > It seems that XML_parse is returning an error, but when XML_GetCurrentLineNumber > is called positionPtr is not valid. > > I catch this in GDB and have looked at the stack (see below), but when I get back up > the stack into PyCFunction_Call I don't know what to do. > > Ideally, I want to find out the Python source file and line number that is currently being > executed, then look at the Python source to figure out exactly which pyexpat call is > being made just before the call to get_parse_result. > > It seems strange that (it appears) that Python code is calling get_parse_result directly. > > So, how can I figure out where in the Python source the function call is coming from > using gdb? I'm sure it involves "print" and some casts.. I couldn't find a howto on > python.org Just as a wild idea: Could this be related to the fact that Python 2.3.3 doesn't maintain explicit line numbers, any longer? -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From raymond.hettinger at verizon.net Fri Feb 27 23:06:56 2004 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Fri Feb 27 23:08:49 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <200402271649.48481.fdrake@acm.org> Message-ID: <001401c3fdb0$52af56c0$f63cc797@oemcomputer> > > My idea is for a new flag, METH_STACK, that generalizes (and > potentially > > replaces) METH_NOARGS AND METH_O. > > How would METH_STACK methods work when called via func(*args) The args tuple is converted to an equivalent stack reference. > or via the > various functions of the C API? There would by an analogue to Py_BuildValue(). > The nice thing about METH_NOARGS and METH_O is that they describe what the > called function needs. This is closer to METH_VARARGS which lets the function itself handle the arguments. > METH_STACK sounds like a core dump waiting to > happen. It is no more or less a risk of a core dump than METH_VARARGS. Pretty much if any C function expects one thing and gets another (passing an pointer to the wrong structure or somesuch), then something bad happends. Right now, if you write f(x,y,z) where f is a PyCFunction, then ceval.c will execute CALL_FUNCTION 3 by pulling off three stack variables, load them into a new tuple, pass the tuple to the function, and the function will call either PyArg_ParseTuple() or PyArg_UnpackTuple(). With the new way, CALL_FUNCTION 3 calls the method with a stack pointer and the argument count (essentially the same data as the tuple) and the function calls PyArg_ParseStack() or PyArg_UnpackStack() which verifies the number of arguments and pulls the data off of the stack. All of the steps are encapsulated. Essentially, the only change is transferred the responsibility for pulling data off of the stack from ceval.c to the function that actually uses the data. This saves the unnecessary tuple carrier. Raymond From barry at barrys-emacs.org Sat Feb 28 04:11:10 2004 From: barry at barrys-emacs.org (Barry Scott) Date: Sat Feb 28 04:11:24 2004 Subject: [Python-Dev] What is the ":" for in python? In-Reply-To: <16447.40159.39178.907890@magrathea.basistech.com> References: <6.0.3.0.2.20040227192602.0309e1f0@torment.chelsea.private> <16447.40159.39178.907890@magrathea.basistech.com> Message-ID: <6.0.3.0.2.20040228090304.02e750b0@torment.chelsea.private> At 27-02-2004 19:39, Tom Emerson wrote: >You could write > >if x > 10 \ >and x < 20: > do_in_range() I don't like using backslash line continuation. Too many ways to mess up, putting comments or white space after the \ may or may not work. And are the continuation rules the same of C and make or different? I cannot write: if x > 10 \ # check lower bound and x < 20: # check upper bound do_in_range() File "L:\Users\barry\a.py", line 2 if x > 10 \ # check lower bound ^ SyntaxError: invalid token Barry From mwh at python.net Sat Feb 28 07:23:37 2004 From: mwh at python.net (Michael Hudson) Date: Sat Feb 28 07:23:40 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: (Alan Green's message of "Fri, 27 Feb 2004 20:51:17 +0000 (UTC)") References: <338366A6D2E2CA4C9DAEAE652E12A1DE013E629E@au3010avexu1.global.avaya.com> <1077847123.31953.29.camel@anthem.wooz.org> <5.2.1.1.0.20040227140748.02aef568@pop.bluewin.ch> Message-ID: <2msmgvmlyu.fsf@starship.python.net> Alan Green writes: > OK. Would you be able give me some pointers to the old threads or > some clues about what to search on? I've obviously been looking in > the wrong places. You're looking for the thread "Extended Function Syntax" (and splinters) from Jan/Feb last year. Don't worry, it only ever reached about 100 posts a day... Start here: http://mail.python.org/pipermail/python-dev/2003-January/032337.html Cheers, mwh -- Well, you pretty much need Microsoft stuff to get misbehaviours bad enough to actually tear the time-space continuum. Luckily for you, MS Internet Explorer is available for Solaris. -- Calle Dybedahl, alt.sysadmin.recovery From mwh at python.net Sat Feb 28 07:39:43 2004 From: mwh at python.net (Michael Hudson) Date: Sat Feb 28 07:39:48 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> (Phillip J. Eby's message of "Fri, 27 Feb 2004 17:16:06 -0500") References: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> Message-ID: <2mk727ml80.fsf@starship.python.net> "Phillip J. Eby" writes: > Maybe there should instead be a tp_call_stack slot. Then the various > CALL opcodes would call that slot instead of tp_call. C API calls > would still go through tp_call. People *really* should look at the patch I mentioned... > In practice, though, I expect it would be faster to do as Jython and > IronPython have done, and define a set of tp_call1, tp_call2, > etc. slots that are optimized for specific calling situations, > allowing C API calls to be sped up as well, provided you used things > like PyObject_Call1(ob,arg), PyObject_Call2(ob,arg1,arg2), and so on. I think this only really helps when you have a JIT compiler of some sort? > Perhaps there is some information that can be gleaned from the Jython > research as to what are the most common number of positional > parameters for calls. That's easy: 0 then 1 then 2 then 3 then insignificant. Only a guess, but one I'm fairly confident of. Cheers, mwh -- ARTHUR: Don't ask me how it works or I'll start to whimper. -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From mwh at python.net Sat Feb 28 07:46:39 2004 From: mwh at python.net (Michael Hudson) Date: Sat Feb 28 07:46:43 2004 Subject: [Python-Dev] How to debug pyexpat SIGSEGV with GDB? In-Reply-To: <403F84D3.15067.13B6DF33@localhost> (Brad Clements's message of "Fri, 27 Feb 2004 18:00:09 -0500") References: <403F84D3.15067.13B6DF33@localhost> Message-ID: <2mfzcvmkwg.fsf@starship.python.net> "Brad Clements" writes: > Sorry if this is OT. Marginally. > I've hit a repeatable segfault in pyexpat on RH Linux 9 with Python > 2.3.3 (I'm trying rss2email). > > It seems that XML_parse is returning an error, but when > XML_GetCurrentLineNumber is called positionPtr is not valid. > > I catch this in GDB and have looked at the stack (see below), but > when I get back up the stack into PyCFunction_Call I don't know what > to do. That's possibly because you haven't gone far enough. PyCFunction_Call is what you call to execute a builtin function. You need to work your way a couple more levels up the stack. > Ideally, I want to find out the Python source file and line number > that is currently being executed, then look at the Python source to > figure out exactly which pyexpat call is being made just before the > call to get_parse_result. > > It seems strange that (it appears) that Python code is calling > get_parse_result directly. Build a debug build, maybe? If a function looks like: PyObject* foo() { /* stuff with no function calls, or only inlineable function calls */ return Function(args); } gcc, at least, will not set up a stack frame for foo. > So, how can I figure out where in the Python source the function > call is coming from using gdb? I'm sure it involves "print" and some > casts.. I couldn't find a howto on python.org Read lots of source. Cheers, mwh -- 31. Simplicity does not precede complexity, but follows it. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From pje at telecommunity.com Sat Feb 28 10:53:54 2004 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Feb 28 10:51:05 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <2mk727ml80.fsf@starship.python.net> References: <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> Message-ID: <5.1.0.14.0.20040228103037.038218c0@mail.telecommunity.com> At 12:39 PM 2/28/04 +0000, Michael Hudson wrote: >"Phillip J. Eby" writes: > > > Maybe there should instead be a tp_call_stack slot. Then the various > > CALL opcodes would call that slot instead of tp_call. C API calls > > would still go through tp_call. > >People *really* should look at the patch I mentioned... Ah, I see you've implemented this already. :) > > In practice, though, I expect it would be faster to do as Jython and > > IronPython have done, and define a set of tp_call1, tp_call2, > > etc. slots that are optimized for specific calling situations, > > allowing C API calls to be sped up as well, provided you used things > > like PyObject_Call1(ob,arg), PyObject_Call2(ob,arg1,arg2), and so on. > >I think this only really helps when you have a JIT compiler of some >sort? No, all you need is a mechanism similar to the one in your patch, where the creation of a method or function object includes the selecting of a function pointer for each of the call signatures. So, when creating these objects, you'd select a call0 pointer, a call1 pointer, and so on. If the function requires more arguments than provided for that slot, you just point it to an "insufficient args" version of the function. If the function has defaults that would be filled in, you point it to a version that fills in the defaults, and so on. So, there's not a JIT required, but making the patch could be tedious, repetitive and error prone. Macros could possibly help a bit with the repetition and tedium, although not necessarily the error-prone part. :) > > Perhaps there is some information that can be gleaned from the Jython > > research as to what are the most common number of positional > > parameters for calls. > >That's easy: 0 then 1 then 2 then 3 then insignificant. Actually, I'd find that a bit surprising, as I would expect 1-arg calls to be more popular than 0-argument calls. But I guess if you look at a method call, it could be a 0-argument call to the method, that maps to a 1-argument call on the function. But, I guess that would at any rate put 0-argument calls in the running. Anyway, I guess a 0-argument call for an instancemethod would look something like: PyObject * method_call0(PyObject *self) { if (self->im_self) { return PyObject_Call1(self->im_func, self->im_self); } else { return PyObject_Call0(self->im_func); } } And then you'd repeat this pattern for call1 and call2, but call3 would have to fall back to creating an argument tuple if we're only going up to call3 slots. Interestingly, the patterns involved in making these functions are basically left currying (methods) and right currying (default args). It may be that there's some way to generalize this, perhaps by using wrapping objects. On the other hand, that would tend to go against the intended speedup. > ARTHUR: Don't ask me how it works or I'll start to whimper. > -- The Hitch-Hikers Guide to the Galaxy, Episode 11 Heh. Did you pick this quote intentionally? :) From kbk at shore.net Sat Feb 28 12:37:08 2004 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Feb 28 12:37:12 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> (Phillip J. Eby's message of "Fri, 27 Feb 2004 17:16:06 -0500") References: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> Message-ID: <8765dr85rv.fsf@hydra.localdomain> "Phillip J. Eby" writes: > Perhaps there is some information that can be gleaned from the Jython > research as to what are the most common number of positional > parameters for calls. Queinnec, "Lisp in Small Pieces", p 239: arity 0 1 2 3 4 5 6 7 8 frequency(%) 35 30 18 9 4 1 0 0 0 cumulative(%) 35 66 84 93 97 99 99 99 100 at least for the 1,988 scheme functions in the book. He thinks 0 arity is over-represented due to some thunking techniques that he employs. -- KBK From mwh at python.net Sat Feb 28 12:47:14 2004 From: mwh at python.net (Michael Hudson) Date: Sat Feb 28 12:47:18 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <5.1.0.14.0.20040228103037.038218c0@mail.telecommunity.com> (Phillip J. Eby's message of "Sat, 28 Feb 2004 10:53:54 -0500") References: <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> <5.1.0.14.0.20040228103037.038218c0@mail.telecommunity.com> Message-ID: <2msmgvhza5.fsf@starship.python.net> "Phillip J. Eby" writes: > At 12:39 PM 2/28/04 +0000, Michael Hudson wrote: >>"Phillip J. Eby" writes: >> >> > Maybe there should instead be a tp_call_stack slot. Then the various >> > CALL opcodes would call that slot instead of tp_call. C API calls >> > would still go through tp_call. >> >>People *really* should look at the patch I mentioned... > > Ah, I see you've implemented this already. :) :-) >> > In practice, though, I expect it would be faster to do as Jython and >> > IronPython have done, and define a set of tp_call1, tp_call2, >> > etc. slots that are optimized for specific calling situations, >> > allowing C API calls to be sped up as well, provided you used things >> > like PyObject_Call1(ob,arg), PyObject_Call2(ob,arg1,arg2), and so on. >> >>I think this only really helps when you have a JIT compiler of some >>sort? > > No, all you need is a mechanism similar to the one in your patch, > where the creation of a method or function object includes the > selecting of a function pointer for each of the call signatures. So, > when creating these objects, you'd select a call0 pointer, a call1 > pointer, and so on. If the function requires more arguments than > provided for that slot, you just point it to an "insufficient args" > version of the function. If the function has defaults that would be > filled in, you point it to a version that fills in the defaults, and > so on. Well, this is psyco-style code explosion by hand :-) Doesn't sound like fun. Incidentally, I think it would be goodness if PyCFunctions exposed more info about the arguments they take. > So, there's not a JIT required, but making the patch could be tedious, > repetitive and error prone. Macros could possibly help a bit with the > repetition and tedium, although not necessarily the error-prone part. > :) It probably wouldn't be seriously hard to write a program writing program for this. >> > Perhaps there is some information that can be gleaned from the Jython >> > research as to what are the most common number of positional >> > parameters for calls. >> >>That's easy: 0 then 1 then 2 then 3 then insignificant. > > Actually, I'd find that a bit surprising, as I would expect 1-arg > calls to be more popular than 0-argument calls. But I guess if you > look at a method call, it could be a 0-argument call to the method, > that maps to a 1-argument call on the function. But, I guess that > would at any rate put 0-argument calls in the running. Well, it was just a guess. Maybe METH_O is more common. > Anyway, I guess a 0-argument call for an instancemethod would look > something like: Actually, this isn't much how things work now. > PyObject * > method_call0(PyObject *self) { > if (self->im_self) { > return PyObject_Call1(self->im_func, self->im_self); > } else { > return PyObject_Call0(self->im_func); > } > } but, yes. > And then you'd repeat this pattern for call1 and call2, but call3 > would have to fall back to creating an argument tuple if we're only > going up to call3 slots. > > Interestingly, the patterns involved in making these functions are > basically left currying (methods) and right currying (default args). > It may be that there's some way to generalize this, perhaps by using > wrapping objects. On the other hand, that would tend to go against > the intended speedup. I don't think this approach is going to yield mega speedups whatever. We need Pysco or similar tech for that, I think. >> ARTHUR: Don't ask me how it works or I'll start to whimper. >> -- The Hitch-Hikers Guide to the Galaxy, Episode 11 > > Heh. Did you pick this quote intentionally? :) No, but that quote applies to a worrying fraction of my posts... Cheers, mwh -- Or here's an even simpler indicator of how much C++ sucks: Print out the C++ Public Review Document. Have someone hold it about three feet above your head and then drop it. Thus you will be enlightened. -- Thant Tessman From tismer at stackless.com Sat Feb 28 13:12:54 2004 From: tismer at stackless.com (Christian Tismer) Date: Sat Feb 28 13:12:57 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <2msmgvhza5.fsf@starship.python.net> References: <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> <5.1.1.6.0.20040227170742.01ee7910@telecommunity.com> <5.1.0.14.0.20040228103037.038218c0@mail.telecommunity.com> <2msmgvhza5.fsf@starship.python.net> Message-ID: <4040DA26.3090508@stackless.com> Michael Hudson wrote: ... > Well, this is psyco-style code explosion by hand :-) Doesn't sound > like fun. Well, why not write a little script that generates the code, as part of the build process? And maybe making it an option, for those who fear code bloat? > Incidentally, I think it would be goodness if PyCFunctions exposed > more info about the arguments they take. > > >>So, there's not a JIT required, but making the patch could be tedious, >>repetitive and error prone. Macros could possibly help a bit with the >>repetition and tedium, although not necessarily the error-prone part. >>:) > > > It probably wouldn't be seriously hard to write a program writing > program for this. Ah, oh I see, my message is redundant. I'd like to encourage this. sending it anyway, it's to bad about all the fine characters -- chris p.s.: I believe some automatic source analysis and rewrite might pay off in other areas as well. Grepping through the sources, there are still very many similar patterns of PyArg_ParseTupleXXX calls, which could be replaced by less general, optimized versions. This would even *not* cause code bloat, since all those calling sequences would be smaller than now. -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 mobile +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From jeremy at alum.mit.edu Sat Feb 28 14:04:50 2004 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Sat Feb 28 14:05:05 2004 Subject: [Python-Dev] Idea for a fast calling convention In-Reply-To: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> References: <007b01c3fd0a$88f406a0$e1b52c81@oemcomputer> Message-ID: <1077995090.20154.67.camel@localhost.localdomain> On Fri, 2004-02-27 at 03:20, Raymond Hettinger wrote: > My idea is for a new flag, METH_STACK, that generalizes (and potentially > replaces) METH_NOARGS AND METH_O. > > When CALL_FUNCTION sees the flag, it dispatches (*meth)(self, &stack, > numargs). You're working specifically and calls to C functions, right? The fast_funtion() path essentially does this for functions coded in Python. > On the receiving end, the arguments get retrieved with analogues to > PyArg_ParseTuple() and PyArg_UnpackTuple() which access the parameters > directly and do not need a tuple as a carrier. I like Chris's suggestion to get rid of ParseTuple() where it's still needed. The primary motivation for METH_O IIRC was to avoid the overhead of calling PyArg_ParseTuple(); the ability to eliminate the tuple overhead was gravy. There's one idea I had related to calling Python functions, not sure if it makes much sense. We push arguments onto the caller's stack then copy them into the callee's frame. I wonder if there's a way to pre-allocate the callee's frame and "push" the arguments into that frame directly. It may not make much sense, because copying the arguments from stack to frame probably isn't a big expense. Jeremy From Kevin.Smith at theMorgue.org Sat Feb 28 18:44:38 2004 From: Kevin.Smith at theMorgue.org (Kevin Smith) Date: Sat Feb 28 18:44:46 2004 Subject: [Python-Dev] [UPDATED] PEP 318 - Function/Method Decorator Syntax Message-ID: <125B0D7A-6A48-11D8-BB17-000393829ED2@theMorgue.org> I have updated PEP 318 to include the various other syntaxes suggested in the past few days at python-dev. I have also included an Open Issues section which introduces a question about exactly what a decorator is required to do in order to function properly. PEP: 318 Title: Function/Method Decorator Syntax Abstract The current method for declaring class and static methods is awkward and can lead to code that is difficult to understand. This PEP introduces possible new syntax which will place the translation of instance methods to class/static methods at the same point in the code as the method's declaration. Motivation The current method of translating an instance method into a class/static method places the actual translation at a different point in the code than the declaration of the method. The code below demonstrates this. def foo(self): perform method operation foo = classmethod(foo) When the method is very short, it is easy to look ahead and see that this is a class method. However, if the method is more than 15 lines or so, the translation into a class method is not obvious. A solution to this problem is to move the translation of the method to the same point as the method's declaration. The proposed syntax, shown in the example below, is discussed in the following sections. def foo(self) as synchronized(lock), classmethod: perform method operation Proposal Probably the simplest way to place the decorator that translates an instance method to a class/static method is illustrated in the code below. def classmethod foo(self): perform method operation The code in this example will simply perform the following. def foo(self): perform method operation foo = classmethod(foo) This syntax does not introduce any new keywords and is completely backwards compatible with any existing code. The word between the 'def' and the actual name of the method is simply a reference to a callable object that returns a new function reference. This syntax could also be extended to allow multiple function decorators in the form of a space delimited list as follows: def protected classmethod foo(self): perform method operation which would be equivalent to the current form: def foo(self): perform method operation foo = protected(classmethod(foo)) While this syntax is simple and easy to read, it does become cluttered and more obscure if you wish to allow arguments to be sent to the function decorator. def synchronized(lock) classmethod foo(self): perform method operation Instead of placing the decorators in front of the function name, a better place might be after it, as shown below. The word 'as' is added simply as a separator to assist in readability. def foo(self) as synchronized(lock), classmethod: perform method operation This syntax is quite clear and could probably be interpreted by those not familiar with Python. The proposed syntax can be generalized as follows: 'def' NAME '(' PARAMETERS ')' ['as' DECORATORS] ':' where DECORATORS is a comma-separated list of expressions, or a tuple. Using the latter form, the last example above would look like: def foo(self) as (synchronized(lock), classmethod): perform method operation This form make is possible for the list of decorators to span multiple lines without using the line continuation operator. Alternate Syntaxes Other syntaxes have been proposed in comp.lang.python and python-dev. Unfortunately, no one syntax has come out as a clear winner in the lengthy discussions. The most common suggestions are demonstrated below. The proposed syntax is also included for easy comparison. Proposed Syntax def foo(self) as synchronized(lock), classmethod: perform method operation def foo(self) as (synchronized(lock), classmethod): perform method operation Prefix Forms def [synchronized(lock), classmethod] foo(self): perform method operation def synchronized(lock), classmethod foo(self): perform method operation # Same as above, but only identifiers are allowed sync = synchronized(lock) def sync, classmethod foo(self): perform method operation # Java-like sync = synchronized(lock) def @sync @classmethod foo(self): perform method operation Postfix Forms def foo(self) [synchronized(lock), classmethod]: perform method operation def foo(self) (synchronized(lock), classmethod): perform method operation def foo(self) {'pre': synchronized(lock), 'classmethod': True}: perform method operation I'm not as fond of the forms that use '[ ]' since code like 'foo()[a]' looks as if you are getting the item 'a' from 'foo()'. Although, this isn't as much of an issue when using '[ ]' in a prefix form. The Java-like syntax adds new syntax that is very arbitrary and is almost Perl-ish. In addition, since the order in which the decorators are applied may matter, the last, dictionary-style, syntax must be eliminated. Implementation Issues In the following example there are two function decorators: synchronized(lock) and classmethod. def foo(self) as synchronized(lock), classmethod: perform method operation Since these all appear within the operation of the 'def' itself, it makes sense that synchronized, lock, and classmethod must exist at the time that the definition is executed. In addition, each of these arguments will be evaluated before being applied to the compiled function. This means that arguments like synchronized(lock) must return a descriptor that will be applied to foo. Therefore, the code above translates to: def foo(self): perform method operation foo = classmethod((foo)) In the example above, refers to the descriptor returned by evaluating synchronized(lock). It could easily be argued that the descriptors should be applied in reverse order to make the application of the descriptor look more like the resultant code. I tend to prefer this form. def foo(self): perform method operation foo = (classmethod(foo)) In either case, the modified function is bound to the function name when the 'def' statement is executed. Open Issues It is not clear at the moment if it is even possible to have multiple decorators for a function. If decorators are required to take a function/method and return a descriptor, it might not even be possible to wrap multiple decorators. This should be explored since the best syntax for multiple decorators may not be the same as the best syntax for a single decorator. Current Implementations I am not personally familiar enough with Python's source to implement the proposed syntax; however, Michael Hudson has implemented the "square-bracketed" syntax (see patch at http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar.diff). It should be fairly simple for the Python development team to translate this patch to the proposed syntax. Possible Extensions The proposed syntax is general enough that it could be used on class definitions as well as shown below. class foo(object) as classmodifier: class definition here However, there are no obvious parallels for use with other descriptors such as property(). Conclusion The current method of translating an instance method to a class or static method is awkward. A new syntax for applying function decorators should be implemented (proposed syntax shown below). def foo(self) as synchronized(lock), classmethod: perform method operation The proposed syntax is simple, powerful, easy to read, and therefore preserves those qualities of the Python language. Copyright This document has been placed in the public domain. Kevin Smith Kevin.Smith@themorgue.org From greg at cosc.canterbury.ac.nz Sun Feb 29 19:27:05 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Sun Feb 29 19:27:23 2004 Subject: [Python-Dev] Re: new syntax for wrapping (PEP 318) In-Reply-To: <403D385E.90405@prescod.net> Message-ID: <200403010027.i210R5423024@oma.cosc.canterbury.ac.nz> Paul Prescod : > def foo(self) : > pass Because it looks unspeakably ugly! -1. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Sun Feb 29 20:08:15 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Sun Feb 29 20:08:35 2004 Subject: [Python-Dev] other uses for "as" [was Re: new syntax for wrapping (PEP 318)] In-Reply-To: <5.1.1.6.0.20040226143606.024b6730@telecommunity.com> Message-ID: <200403010108.i2118Fm23565@oma.cosc.canterbury.ac.nz> "Phillip J. Eby" : > "as" looks like syntax that should be reserved > for adaptation at a future time. :) I'd also like to see "as" reserved for possible future use to specify a metaclass in class declarations, which is relevant if we want class decorators with the same syntax as function decorators. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From greg at cosc.canterbury.ac.nz Sun Feb 29 23:06:24 2004 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Sun Feb 29 23:06:37 2004 Subject: [Python-Dev] Re: Optional separatorargument for file.writelines() and StringIO.writelines() In-Reply-To: Message-ID: <200403010406.i2146OW04605@oma.cosc.canterbury.ac.nz> Vernon Cole : > Only in *nix does a linefeed charactor equate to the end of a line in > a text file. So, in every output line of every program I must pretend > to be a C programmer and append a '\n' so that the runtime system can > replace it with the actual End-of-Line used by my operating system. Seems to me the 'n' in '\n' is meant to stand for "newline", not "linefeed". So it's already named in an OS-independent way. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+