From guido at python.org Sun Jun 1 00:29:18 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 31 May 2008 15:29:18 -0700 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) In-Reply-To: <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> Message-ID: I'm willing to meet you halfway. I really don't want isinstance(x, str) to return True for something that doesn't inherit from the concrete str type; this is bound to lead to too much confusion and breakage. But I'm fine with a String ABC (or any other ABC, e.g. Atomic?) that doesn't define any methods but can be used for type testing. How about that? --Guido On Sat, May 31, 2008 at 5:25 AM, Raymond Hettinger wrote: > ISTM, the whole reason people are asking for a String ABC is so you can > write isinstance(obj, String) and allow registered string-like objects to be > accepted. > > The downside is that everytime you want this for a concrete class or type, > it is necessary to write a whole new ABC listing all of the required > methods. Also, you have to change all of the client code's isinstance tests > from concrete to abstract. > > I propose a simpler approach. Provide an alternative registration function > that overrides isinstance() so that objects can explicitly fake any concrete > type: > > s = UserString('whiffleball') > print isinstance(s, str) --> False > register_lookalike(UserString, str) > print isinstance(s, str) --> True > > Besides saving us from writing tons of new ABCs, the approach works with > existing code that already uses isinstance() with concrete types. > > The ABCs that would remain are ones that are truly abstract, that define a > generic interface (like mappings and sequences) and ones that offer some > useful mixin behavior. The remaining ABCs are ones where you have a > fighting chance of actually being able to implement the interface (unlike > String where it would be darned tricky to fully emulate encode(), split(), > etc.) > > This would completely eliminate the need for numbers.Integral for example. > AFAICT, its sole use case is to provide a way for numeric's integral types > (like int8, int32) to pass themselves off as having the same API as regular > ints. Unfortunately, the current approach requires all consumer code to > switch from isinstance(x,int) to isinstance(x,Integral). It would be more > useful if we could simply write register_lookalike(x,int) and be done with > it (no need for numbers.py and its attendant abc machinery). > > If we don't do this, then String won't be the last request. People will > want Datetime for example. Pretty much any concrete type could have a > look-a-like that wanted its own ABC and for all client code to switch from > testing concrete types. > > > Raymond > > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From r.m.oudkerk at googlemail.com Sun Jun 1 00:31:27 2008 From: r.m.oudkerk at googlemail.com (r.m.oudkerk) Date: Sat, 31 May 2008 23:31:27 +0100 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> Message-ID: On 31/05/2008, Paul Moore wrote: > 2008/5/30 Farshid Lashkari : >> I'm not sure if there will be any side affects to modifying >> sys.executable though. Should this be the official way of supporting >> embedded interpreters or should there be a >> multiprocessing.setExecutable() method? > > +1 for setExecutable (I'd prefer set_executable, to be PEP 8 > compliant). Make it explicit, rather than fiddling with stuff in sys > manually. That is easy to do. An issue not mentioned in the PEP is naming conventions. In recent versions I have tried to consistently use mixedCase for functions and methods (other than factory functions) because that is what threading does (give or take settrace(), setprofile(), stack_size().) I am certainly open to using lowercase/lower_case_with_underscores for all functions/methods except for Process's methods and possibly currentProcess(), but I would like some feed back on that. From solipsis at pitrou.net Sun Jun 1 00:45:24 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 31 May 2008 22:45:24 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?optimization_required=3A_=2Eformat=28=29_i?= =?utf-8?q?s_much_slower=09than_=25?= References: Message-ID: Simon Cross gmail.com> writes: > My tests show that the old-style % formatting is much faster when the > final string is 20 characters or less: > > $ ./python -m timeit "'....|....|....|...%s' % '12'" > 10000000 loops, best of 3: 0.0764 usec per loop You are the victim of a constant-folding optimization: $ ./python -m timeit "'....|....|....|...%s' % '12'" 10000000 loops, best of 3: 0.0926 usec per loop $ ./python -m timeit -s "s='12'" "'....|....|....|...%s' % s" 1000000 loops, best of 3: 0.525 usec per loop >>> def f(): return '....|....|....|...%s' % '12' ... >>> dis.dis(f) 1 0 LOAD_CONST 3 ('....|....|....|...12') 3 RETURN_VALUE cheers Antoine. From ncoghlan at gmail.com Sun Jun 1 03:26:52 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 01 Jun 2008 11:26:52 +1000 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) In-Reply-To: <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> Message-ID: <4841FADC.7090005@gmail.com> Raymond Hettinger wrote: > If we don't do this, then String won't be the last request. People will > want Datetime for example. Pretty much any concrete type could have a > look-a-like that wanted its own ABC and for all client code to switch > from testing concrete types. If I remember rightly, the machinery in the ABC's to support registration slows down some other operations with the types - do we want to pay that price all the time? If we do, then it may be a matter of moving some of the registration machinery from ABCMeta up into type itself. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Sun Jun 1 03:41:40 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 31 May 2008 18:41:40 -0700 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> Message-ID: <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1> From: "Guido van Rossum" > I'm willing to meet you halfway. I really don't want isinstance(x, > str) to return True for something that doesn't inherit from the > concrete str type; this is bound to lead to too much confusion and > breakage. Probably true. It was an attractive idea though. Unless all client code converts all its isinstance() tests from concrete to abstract, life is going to be tough for people writing look-alike classes which will have limited applicability. > But I'm fine with a String ABC (or any other ABC, e.g. > Atomic?) that doesn't define any methods but can be used for type > testing. How about that? That's progress! It makes abstract substitution possible while still saving us a lot of code and avoiding over-specification. I propose the following empty abstract classes: String, Datetime, Deque, and Socket. -1 on Atomic though. Earlier in the thread it was made clear that that atomicity is not an intrinsic property of a type; instead it varies across applications (when flattening email folders, a multi-part mime message is atomic, but when flattening individual messages, a multi-part mime message is just another nested container, part of the tree, not one of the leaves). Are you open to considering numbers.Integral to be one of the new empty abstract classes? That would make it much easier for objects wanting to pass themselves as integers. As it stands now, an aspiring Integral class is required to implement a number of arcana including: __rxor__, __rrshift__, __pow__, __invert__, __index__, and __long__. Raymond From mhammond at skippinet.com.au Sun Jun 1 03:28:43 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Sun, 1 Jun 2008 11:28:43 +1000 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> Message-ID: <00ac01c8c386$e10c68e0$a3253aa0$@com.au> > 2008/5/30 Farshid Lashkari : > > I'm not sure if there will be any side affects to modifying > > sys.executable though. Should this be the official way of supporting > > embedded interpreters or should there be a > > multiprocessing.setExecutable() method? > > +1 for setExecutable (I'd prefer set_executable, to be PEP 8 > compliant). Make it explicit, rather than fiddling with stuff in sys > manually. sys.executable typically means the "current" executable, and py2exe etc already fiddles with that. The question in such a context seems to be "what executable should I use for this processing functionality?". In a py2exe like environment, it might not be unreasonable to assume that if a custom executable is to be used, that custom executable may have a different command-line or other special requirements. Further, I could imagine a system that uses an alternative way of starting processes (eg, 'distributed COM') where the concept of 'executable' (or even 'command-line') don't make much sense. So it seems that maybe simply "setExecutable()" isn't the correct abstraction here, but maybe a "factory" approach, so the entire process creation mechanism can be replaced rather than just the name of the executable to spawn? Cheers, Mark From guido at python.org Sun Jun 1 04:10:20 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 31 May 2008 19:10:20 -0700 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) In-Reply-To: <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1> References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1> Message-ID: On Sat, May 31, 2008 at 6:41 PM, Raymond Hettinger wrote: > From: "Guido van Rossum" >> I'm willing to meet you halfway. I really don't want isinstance(x, >> str) to return True for something that doesn't inherit from the >> concrete str type; this is bound to lead to too much confusion and >> breakage. > > Probably true. It was an attractive idea though. Unless all client > code converts all its isinstance() tests from concrete to abstract, > life is going to be tough for people writing look-alike classes > which will have limited applicability. I'd rather require that people rewrite their code to benefit from some new piece of functionality than foisting it upon them regardless, breaking some perfectly fine working in the process. This is how we've always done it. >> But I'm fine with a String ABC (or any other ABC, e.g. >> Atomic?) that doesn't define any methods but can be used for type >> testing. How about that? > > That's progress! It makes abstract substitution possible while > still saving us a lot of code and avoiding over-specification. > I propose the following empty abstract classes: String, Datetime, Deque, > and Socket. Sounds like a mini-PEP is in place. It should focus on the code to actually define these and the intended ways to use them. > -1 on Atomic though. Earlier in the thread it was made clear that > that atomicity is not an intrinsic property of a type; instead it varies > across applications (when flattening email folders, a multi-part mime > message is atomic, but when flattening individual messages, a > multi-part mime message is just another nested container, part > of the tree, not one of the leaves). Fine, it was just an idle thought. > Are you open to considering numbers.Integral to be one of the > new empty abstract classes? That would make it much easier > for objects wanting to pass themselves as integers. As it stands > now, an aspiring Integral class is required to implement a number > of arcana including: __rxor__, __rrshift__, __pow__, __invert__, > __index__, and __long__. I don't think Integer should be completely abstract (what good is an int you can't add 1 to?) but I could be amenable to reducing the set of required operations (which could then resurface as a separate ABC). Please write another mini-PEP. Where did you see __long__? That seems a mistake (at least in 3.0). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sun Jun 1 05:09:10 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 31 May 2008 20:09:10 -0700 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1> Message-ID: [Raymond] >> I propose the following empty abstract classes: String, Datetime, Deque, >> and Socket. [GvR] > Sounds like a mini-PEP is in place. It should focus on the code to > actually define these and the intended ways to use them. Okay, will run a Google code search to see if real code exists that runs isinstance tests on the concrete types. Since the new classes are very lightweight (completely empty), these probably only need minimal justification. The case for String has already been made. And the concept of a Socket is already fully abstract. Not sure I really care about Deque. The Datetime.class is tricky. The existence of many implementations of date/time routines indicates that there is a need; however, they don't share the API so they likely won't fit under a common umbrella. >> Are you open to considering numbers.Integral to be one of the >> new empty abstract classes? That would make it much easier >> for objects wanting to pass themselves as integers. As it stands >> now, an aspiring Integral class is required to implement a number >> of arcana including: __rxor__, __rrshift__, __pow__, __invert__, >> __index__, and __long__. > > I don't think Integer should be completely abstract (what good is an > int you can't add 1 to?) but I could be amenable to reducing the set > of required operations (which could then resurface as a separate ABC). > Please write another mini-PEP. Okay. Will propose to remove the bit flipping methods and anything else that doesn't seem essential to integeriness. Will take a look at the integral types in numeric to see what that actually implement. > Where did you see __long__? That seems > a mistake (at least in 3.0). It's the first listed abstract method in the Py2.6 code. Raymond From guido at python.org Sun Jun 1 05:46:15 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 31 May 2008 20:46:15 -0700 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) In-Reply-To: References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1> Message-ID: On Sat, May 31, 2008 at 8:09 PM, Raymond Hettinger wrote: > [Raymond] >>> >>> I propose the following empty abstract classes: String, Datetime, >>> Deque, >>> and Socket. > > [GvR] >> >> Sounds like a mini-PEP is in place. It should focus on the code to >> actually define these and the intended ways to use them. > > Okay, will run a Google code search to see if real code exists that > runs isinstance tests on the concrete types. I wasn't asking for existing code -- I was asking for the code you propose to add to abc.py (or wherever). > Since the new classes are > very lightweight (completely empty), these probably only need > minimal justification. Again, in a mini-PEP I'm not so much looking for justification but for a precise spec. > The case for String has already been made. Actually I'm not sure. One you know that isinstance(x, String) is true, what can you assume you can do with x? > And the concept of a Socket is already fully abstract. Can you elaborate? There's a very specific API that is assumed of sockets. The code that creates sockets is usually pretty close to the code that consumes it. There are some major classes that cut right through the APIs though: connection or listening (the latter being something on which you call accept()), stream or datagram, and as a special case of stream SSL and the like. > Not sure I really care about Deque. > The Datetime.class is tricky. The existence of many implementations > of date/time routines indicates that there is a need; however, they > don't share the API so they likely won't fit under a common umbrella. Right. I'm now beginning to wonder what exactly you're after here -- saying that something is an "X" without saying anything about an API isn't very useful. You need to have at least *some* API to be able to do anything with that knowledge. >>> Are you open to considering numbers.Integral to be one of the >>> new empty abstract classes? That would make it much easier >>> for objects wanting to pass themselves as integers. As it stands >>> now, an aspiring Integral class is required to implement a number >>> of arcana including: __rxor__, __rrshift__, __pow__, __invert__, >>> __index__, and __long__. >> >> I don't think Integer should be completely abstract (what good is an >> int you can't add 1 to?) but I could be amenable to reducing the set >> of required operations (which could then resurface as a separate ABC). >> Please write another mini-PEP. > > Okay. Will propose to remove the bit flipping methods and anything > else that doesn't seem essential to integeriness. Will take a look at > the integral types in numeric to see what that actually implement. > > > >> Where did you see __long__? That seems >> a mistake (at least in 3.0). > > It's the first listed abstract method in the Py2.6 code. That actually makes sense -- correct interoperability with longs probably requires that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From alexandre at peadrop.com Sun Jun 1 06:11:19 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 1 Jun 2008 00:11:19 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() Message-ID: Would anyone mind if I did add a public C API for gc.disable() and gc.enable()? I would like to use it as an optimization for the pickle module (I found out that I get a good 2x speedup just by disabling the GC while loading large pickles). Of course, I could simply import the gc module and call the functions there, but that seems overkill to me. I included the patch below for review. -- Alexandre Index: Include/objimpl.h =================================================================== --- Include/objimpl.h (revision 63766) +++ Include/objimpl.h (working copy) @@ -221,8 +221,10 @@ * ========================== */ -/* C equivalent of gc.collect(). */ +/* C equivalent of gc.collect(), gc.enable() and gc.disable(). */ PyAPI_FUNC(Py_ssize_t) PyGC_Collect(void); +PyAPI_FUNC(void) PyGC_Enable(void); +PyAPI_FUNC(void) PyGC_Disable(void); /* Test if a type has a GC head */ #define PyType_IS_GC(t) PyType_HasFeature((t), Py_TPFLAGS_HAVE_GC) Index: Modules/gcmodule.c =================================================================== --- Modules/gcmodule.c (revision 63766) +++ Modules/gcmodule.c (working copy) @@ -1252,6 +1252,18 @@ return n; } +void +PyGC_Disable(void) +{ + enabled = 0; +} + +void +PyGC_Enable(void) +{ + enabled = 1; +} + /* for debugging */ void _PyGC_Dump(PyGC_Head *g) From rhamph at gmail.com Sun Jun 1 06:28:35 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 31 May 2008 22:28:35 -0600 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: Message-ID: On Sat, May 31, 2008 at 10:11 PM, Alexandre Vassalotti wrote: > Would anyone mind if I did add a public C API for gc.disable() and > gc.enable()? I would like to use it as an optimization for the pickle > module (I found out that I get a good 2x speedup just by disabling the > GC while loading large pickles). Of course, I could simply import the > gc module and call the functions there, but that seems overkill to me. > I included the patch below for review. I'd rather see it fixed. It behaves quadratically if you load enough to trigger full collection a few times. -- Adam Olsen, aka Rhamphoryncus From python at rcn.com Sun Jun 1 08:15:22 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 31 May 2008 23:15:22 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC Message-ID: Target: Py2.6 and Py3.0 Author: Raymond Hettinger Date: May 31, 2008 Motivation ---------- The principal purpose of an abstract base class is to support multiple implementations of an API; thereby allowing one concrete class to be substitutable for another. This purpose is defeated when useful substitutions are precluded because the ABC is too aggressive in its behavioral requirements -- mandating too many methods, mandating methods that are difficult to implement, or too closely following the full API of a single concrete type. The Integral ABC is somewhat extensive and requires essentially every behavior exhibited by the int concrete class. Usefully, it provides for basic integer behaviors like simple arithmetic and ordering relations. However, the current ABC goes beyond that and requires bit-flipping and other operations that are not fundamental to the notion of being an integer. That makes it challenging to define a new Integral class that isn't already an int. Proposal -------- Remove non-essential abstract methods like __index__, three argument __pow__, __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, __xor__, __rxor__, __or__, __ror__, and __invert__, numerator, and denominator. Discussion ---------- The only known use cases for variants of int are types that limit the range of values to those that fit in a fixed storage width. One existing implementation is in numpy which has types like int0, int8, int16, int32, and int16. The numpy integral types implement all the methods present in Integral except for the new methods like __trunc__, __index__, real, imag, conjugate, numerator, and denominator. For the most part, they are fully substitutable into code that expects regular ints; however, they do have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The wrap-around behaviors make the numpy types totally unsuitable for some applications of Integral such as the fractions module which accepts any integral numerator and denominator. From python at rcn.com Sun Jun 1 08:59:00 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 31 May 2008 23:59:00 -0700 Subject: [Python-Dev] Mini-Pep: An Empty String ABC Message-ID: Mini-Pep: An Empty String ABC Target: Py2.6 and Py3.0 Author: Raymond Hettinger Proposal -------- Add a new collections ABC specified as: class String(Sequence): pass Motivation ---------- Having an ABC for strings allows string look-alike classes to declare themselves as sequences that contain text. Client code (such as a flatten operation or tree searching tool) may use that ABC to usefully differentiate strings from other sequences (i.e. containers vs containees). And in code that only relies on sequence behavior, isinstance(x,str) may be usefully replaced by isinstance(x,String) so that look-alikes can be substituted in calling code. A natural temptation is add other methods to the String ABC, but strings are a tough case. Beyond simple sequence manipulation, the string methods get very complex. An ABC that included those methods would make it tough to write a compliant class that could be registered as a String. The split(), rsplit(), partition(), and rpartition() methods are examples of methods that would be difficult to emulate correctly. Also, starting with Py3.0, strings are essentially abstract sequences of code points, meaning that an encode() method is essential to being able to usefully transform them back into concrete data. Unfortunately, the encode method is so complex that it cannot be readily emulated by an aspiring string look-alike. Besides complexity, another problem with the concrete str API is the extensive number of methods. If string look-alikes were required to emulate the likes of zfill(), ljust(), title(), translate(), join(), etc., it would significantly add to the burden of writing a class complying with the String ABC. The fundamental problem is that of balancing a client function's desire to rely on a broad number of behaviors against the difficulty of writing a compliant look-alike class. For other ABCs, the balance is more easily struck because the behaviors are fewer in number, because they are easier to implement correctly, and because some methods can be provided as mixins. For a String ABC, the balance should lean toward minimalism due to the large number of methods and how difficult it is to implement some of the correctly. A last reason to avoid expanding the String API is that almost none of the candidate methods characterize the notion of "stringiness". With something calling itself an integer, an __add__() method would be expected as it is fundamental to the notion of "integeriness". In contrast, methods like startswith() and title() are non-essential extras -- we would not discount something as being not stringlike if those methods were not present. From fredrik.johansson at gmail.com Sun Jun 1 11:53:05 2008 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Sun, 1 Jun 2008 11:53:05 +0200 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: Message-ID: <3d0cebfb0806010253g77ce1f05i9111d8ad7b11c4ff@mail.gmail.com> On Sun, Jun 1, 2008 at 8:15 AM, Raymond Hettinger wrote: > Discussion > ---------- > The only known use cases for variants of int are types that limit the range > of > values to those that fit in a fixed storage width. Add: * Faster big integers (gmpy) * Integers with exact division to rationals (e.g. sympy) Fredrik From greg.ewing at canterbury.ac.nz Sun Jun 1 12:54:15 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 01 Jun 2008 22:54:15 +1200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> <483FB8F6.4060504@canterbury.ac.nz> Message-ID: <48427FD7.4090005@canterbury.ac.nz> Armin Ronacher wrote: > basestring is not subclassable for example. Also it requires subclassing > which ABCs do not. The use case that was cited was recognising subclasses of UserString, and that's what I was responding to. If basestring were made subclassable and UserString inherited from it, that use case would be covered. Recognising string-like objects *without* requiring subclassing is a hopeless morass to get into, in my opinion. You'll just have endless arguments about which of the zillion methods of str should be in the blessed set which confers string-ness. I also think that the ABC idea in general suffers from that problem, to one degree or another depending on the class involved. Strings are just an extreme case. -- Greg From solipsis at pitrou.net Sun Jun 1 13:42:58 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 1 Jun 2008 11:42:58 +0000 (UTC) Subject: [Python-Dev] Mini-Pep: An Empty String ABC References: Message-ID: Raymond Hettinger rcn.com> writes: > Also, starting with Py3.0, strings are > essentially abstract sequences of code points, meaning that an encode() method > is essential to being able to usefully transform them back into concrete data. Well, that depends: - is a String the specification of a generic range of types which one might want to special-case in some algorithms, e.g. flatten() - or is a String the specification of something which is meant to be used as a replacement of str (or, perhaps, bytes)? If you answer the former, the String API should be very minimal and there is no reason for it to support "encoding" or "decoding". Such a String doesn't have to be a string of characters, it can contain arbitrary objects, e.g. DNA elements. If you answer the latter, what use is a String subclass which isn't a drop-in replacement for either str or bytes? Saying "hello, I'm a String" is not very useful if you can't be used anywhere in existing code. I think most Python coders wouldn't go out of their way to allow arbitrary String instances as parameters for their functions, rather than objects conforming to the full str (or, perhaps, bytes) API. I'd like to know the use cases of a String ABC representing replacements of the str class, though. I must admit I've never used UserString and the like, and don't know how useful they can be. However, the docs have the following to say: ? This UserString class from this module is available for backward compatibility only. If you are writing code that does not need to work with versions of Python earlier than Python 2.2, please consider subclassing directly from the built-in str type instead of using UserString ?. So, apart from compatibility purposes, what is the point currently of *not* directly subclassing str? Regards Antoine. From p.f.moore at gmail.com Sun Jun 1 15:44:59 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 1 Jun 2008 14:44:59 +0100 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> Message-ID: <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com> 2008/5/31 r. m. oudkerk : > I am certainly open to using lowercase/lower_case_with_underscores for > all functions/methods except for Process's methods and possibly > currentProcess(), but I would like some feed back on that. I dislike mixedCase, but consistency with the rest of the library is more important - and as processing is matching the API of threading, which used mixedCase, it should follow that convention. Wasn't there some talk of changing modules to use PEP 8 conventions (lowercase_with_underscore) as part of the Python 3.0 conversion? Did that ever happen? Paul. From p.f.moore at gmail.com Sun Jun 1 15:57:55 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 1 Jun 2008 14:57:55 +0100 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) In-Reply-To: References: <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1> Message-ID: <79990c6b0806010657i75fedd33q8a220dfb339848ae@mail.gmail.com> 2008/6/1 Guido van Rossum : >> The case for String has already been made. > > Actually I'm not sure. One you know that isinstance(x, String) is > true, what can you assume you can do with x? [...] > Right. I'm now beginning to wonder what exactly you're after here -- > saying that something is an "X" without saying anything about an API > isn't very useful. You need to have at least *some* API to be able to > do anything with that knowledge. Apologies to Raymond if I'm putting words into his mouth, but I think it's more about *not* doing things with the type - a String is a Sequence that we don't wish to iterate through (in the flatten case), so the code winds up looking like for elem in seq: if isinstance(elem, Sequence) and not isinstance(elem, String): recurse into the element else: deal with the element as atomic This implies that other "empty" abstract types aren't useful, though, as they are not subclasses of anything else... Paul. From ncoghlan at gmail.com Sun Jun 1 16:28:02 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Jun 2008 00:28:02 +1000 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com> Message-ID: <4842B1F2.6010901@gmail.com> Paul Moore wrote: > Wasn't there some talk of changing modules to use PEP 8 conventions > (lowercase_with_underscore) as part of the Python 3.0 conversion? Did > that ever happen? We fixed the module names that used mixed case - the amount of work that turned out to be involved in just doing that much for PEP 3108 makes me shudder at the thought of trying to fix all of the standard library APIs that currently don't follow the style guide... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From aahz at pythoncraft.com Sun Jun 1 21:53:00 2008 From: aahz at pythoncraft.com (Aahz) Date: Sun, 1 Jun 2008 12:53:00 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: Message-ID: <20080601195300.GA2945@panix.com> On Sat, May 31, 2008, Raymond Hettinger wrote: > > Proposal > -------- > Remove non-essential abstract methods like __index__, three argument > __pow__, __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, > __rand__, __xor__, __rxor__, __or__, __ror__, and __invert__, > numerator, and denominator. The only thing I object to is removing __index__ -- the whole point of an integral class is that it is substitutable as an index for sequences in a way that other numeric types are not. Having an __index__ special method is a key indicator for duck-typing purposes not covered by the ABC. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From g.brandl at gmx.net Sun Jun 1 22:37:07 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 01 Jun 2008 22:37:07 +0200 Subject: [Python-Dev] [Python-3000] Finishing up PEP 3108 In-Reply-To: <5c6f2a5d0805311452v15a87d06g899fa8182dbd9d2a@mail.gmail.com> References: <5c6f2a5d0805311452v15a87d06g899fa8182dbd9d2a@mail.gmail.com> Message-ID: Mark Dickinson schrieb: > On Sat, May 31, 2008 at 11:33 AM, Georg Brandl > wrote: > > > Now that the docs are reST, the source is almost pretty enough to > display > it raw, but I could also imagine a "text" writer that removes the more > obscure markup to present a casual-reader-friendly text version. > > The needed sources could then be distributed with Python -- it shouldn't > be more than about 200 kb. > > > +1 from me. Would this mean that htmllib and sgmllib could be > removed without further ado. OK, I've now implemented this in the trunk (will merge to 3k soon -- htmllib and sgmllib can go then). The topic help is contained in a new module, pydoc_topics.py, which pydoc imports to provide this help. The module can be generated with Sphinx by running "make pydoc-topics" in the Doc/ directory. (This is one more step for the release process, but it is an easy one.) The module is currently ~ 400 kb in size. If this is deemed to be a problem, we could use zlib to compress the contents -- which of course is bad for systems without the zlib module (are there any?). Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Sun Jun 1 23:28:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 01 Jun 2008 23:28:30 +0200 Subject: [Python-Dev] [Python-3000] Finishing up PEP 3108 In-Reply-To: References: <5c6f2a5d0805311452v15a87d06g899fa8182dbd9d2a@mail.gmail.com> Message-ID: <4843147E.8050907@v.loewis.de> > The module is currently ~ 400 kb in size. If this is deemed to be a > problem, > we could use zlib to compress the contents -- which of course is bad for > systems without the zlib module (are there any?). In the distribution, the file gets compressed, anyway. In the installation, I don't think it is a problem. Regards, Martin From guido at python.org Mon Jun 2 00:43:38 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Jun 2008 15:43:38 -0700 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <48427FD7.4090005@canterbury.ac.nz> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> <483FB8F6.4060504@canterbury.ac.nz> <48427FD7.4090005@canterbury.ac.nz> Message-ID: On Sun, Jun 1, 2008 at 3:54 AM, Greg Ewing wrote: > The use case that was cited was recognising subclasses of > UserString, and that's what I was responding to. If > basestring were made subclassable and UserString inherited > from it, that use case would be covered. UserString intentionally doesn't subclass basestring. When basestring was introduced, it was specifically meant to be the base class of *only* str and unicode. There are quite a few core APIs that accept no substitutes, and being an instance of basestring was intended to guarantee that a value is accepted by such APIs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From qgallet at gmail.com Mon Jun 2 00:45:53 2008 From: qgallet at gmail.com (Quentin Gallet-Gilles) Date: Mon, 2 Jun 2008 00:45:53 +0200 Subject: [Python-Dev] [Python-3000] Finishing up PEP 3108 In-Reply-To: <8b943f2b0805290839s7a1f3238g9e21407a56c34159@mail.gmail.com> References: <8b943f2b0805290625w19ab1fd3l48f00f40e630c39d@mail.gmail.com> <483EC414.7080603@ibp.de> <8b943f2b0805290839s7a1f3238g9e21407a56c34159@mail.gmail.com> Message-ID: <8b943f2b0806011545x6a11f019r3c412bc3ebc1a3ab@mail.gmail.com> I've uploaded a patch for the aifc module (http://bugs.python.org/issue2847). I'm still working on the testsuite. Comments are welcome! Quentin On Thu, May 29, 2008 at 5:39 PM, Quentin Gallet-Gilles wrote: > > On Thu, May 29, 2008 at 4:56 PM, Lars Immisch wrote: > >> >> >>> Issue 2847 - the aifc module still imports the cl module in 3.0. >>> Problem is that the cl module is gone. =) So it seems silly to >>> have >>> the imports lying about. This can probably be changed to critical. >>> >>> >>> It shouldn't be a problem to rip everything cl-related out of aifc. >>> The question is how useful aifc will be after that ... >>> >>> >>> Has someone already used that module ? I took a look into it, but I'm a >>> bit confused about the various compression types, case-sensitivity and >>> compatibility issues [1]. Are Apple's "alaw" and SGI's "ALAW" really the >>> same encoding ? Can we use the audioop module for ALAW, just like it's >>> already done for ULAW ? >>> >> >> There is just one alaw I've ever come across (G.711), and the audioop >> implementation could be used (audioop's alaw support is younger than the >> aifc module, BTW) >> >> The capitalisation is confusing, but your document [1] says: "Apple >> Computer's QuickTime player recognize only the Apple compression types. >> Although "ALAW" and "ULAW" contain identical sound samples to the "alaw" and >> "ulaw" formats and were in use long before Apple introduced the new codes, >> QuickTime does not recognize them." >> >> So this seems just a matter of naming in the AIFC, but not a matter of two >> different alaw implementations. >> >> - Lars >> > > Ok, I'll handle this issue. I'll be using the audioop implementation as a > replacement of the SGI compression library. I'll also create a test suite, > as Brett mentioned in the bug tracker the module was missing one. > > Quentin > > >> >> [1] http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/AIFF/AIFF.html >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Jun 2 00:57:09 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Jun 2008 15:57:09 -0700 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) In-Reply-To: <79990c6b0806010657i75fedd33q8a220dfb339848ae@mail.gmail.com> References: <200805311313.11501.steve@pearwood.info> <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> <34093CA46419445EBD6232F96268DFE0@RaymondLaptop1> <79990c6b0806010657i75fedd33q8a220dfb339848ae@mail.gmail.com> Message-ID: On Sun, Jun 1, 2008 at 6:57 AM, Paul Moore wrote: > 2008/6/1 Guido van Rossum : >>> The case for String has already been made. >> >> Actually I'm not sure. One you know that isinstance(x, String) is >> true, what can you assume you can do with x? > [...] >> Right. I'm now beginning to wonder what exactly you're after here -- >> saying that something is an "X" without saying anything about an API >> isn't very useful. You need to have at least *some* API to be able to >> do anything with that knowledge. > > Apologies to Raymond if I'm putting words into his mouth, but I think > it's more about *not* doing things with the type - a String is a > Sequence that we don't wish to iterate through (in the flatten case), > so the code winds up looking like > > for elem in seq: > if isinstance(elem, Sequence) and not isinstance(elem, String): > recurse into the element > else: > deal with the element as atomic I thought that was he meant too, until he said he rejected my offhand suggestion of Atomic with these words: "Earlier in the thread it was made clear that that atomicity is not an intrinsic property of a type; instead it varies across applications [...]" > This implies that other "empty" abstract types aren't useful, though, > as they are not subclasses of anything else... There's a thread on this out now I believe. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Jun 2 01:06:10 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Jun 2008 16:06:10 -0700 Subject: [Python-Dev] Mini-Pep: An Empty String ABC In-Reply-To: References: Message-ID: This PEP is incomplete without specifying exactly which built-in and stdlib types should be registered as String instances. I'm also confused -- the motivation seems mostly "so that you can skip iterating over it when flattening a nested sequence" but earlier you rejected my "Atomic" proposal, saying "Earlier in the thread it was made clear that that atomicity is not an intrinsic property of a type; instead it varies across applications [...]". Isn't this String proposal just that by another name? Finally, I fully expect lots of code writing isinstance(x, String) and making many more assumptions than promised by the String ABC. For example that s[0] has the same type as s (not true for bytes). Or that it is hashable (the Sequence class doesn't define __hash__). Or that s1+s2 will work (not in the Sequence class either). And many more. All this makes me lean towards a rejection of this proposal -- it seems worse than no proposal at all. It could perhaps be rescued by adding some small set of defined operations. --Guido On Sat, May 31, 2008 at 11:59 PM, Raymond Hettinger wrote: > Mini-Pep: An Empty String ABC > Target: Py2.6 and Py3.0 > Author: Raymond Hettinger > > Proposal > -------- > > Add a new collections ABC specified as: > > class String(Sequence): > pass > > Motivation > ---------- > Having an ABC for strings allows string look-alike classes to declare > themselves as sequences that contain text. Client code (such as a flatten > operation or tree searching tool) may use that ABC to usefully differentiate > strings from other sequences (i.e. containers vs containees). And in code > that only relies on sequence behavior, isinstance(x,str) may be usefully > replaced by isinstance(x,String) so that look-alikes can be substituted in > calling code. > > A natural temptation is add other methods to the String ABC, but strings are > a > tough case. Beyond simple sequence manipulation, the string methods get > very > complex. An ABC that included those methods would make it tough to write a > compliant class that could be registered as a String. The split(), > rsplit(), > partition(), and rpartition() methods are examples of methods that would be > difficult to emulate correctly. Also, starting with Py3.0, strings are > essentially abstract sequences of code points, meaning that an encode() > method > is essential to being able to usefully transform them back into concrete > data. > Unfortunately, the encode method is so complex that it cannot be readily > emulated by an aspiring string look-alike. > > Besides complexity, another problem with the concrete str API is the > extensive > number of methods. If string look-alikes were required to emulate the likes > of zfill(), ljust(), title(), translate(), join(), etc., it would > significantly add to the burden of writing a class complying with the String > ABC. > > The fundamental problem is that of balancing a client function's desire to > rely on a broad number of behaviors against the difficulty of writing a > compliant look-alike class. For other ABCs, the balance is more easily > struck > because the behaviors are fewer in number, because they are easier to > implement correctly, and because some methods can be provided as mixins. > For > a String ABC, the balance should lean toward minimalism due to the large > number of methods and how difficult it is to implement some of the > correctly. > > A last reason to avoid expanding the String API is that almost none of the > candidate methods characterize the notion of "stringiness". With something > calling itself an integer, an __add__() method would be expected as it is > fundamental to the notion of "integeriness". In contrast, methods like > startswith() and title() are non-essential extras -- we would not discount > something as being not stringlike if those methods were not present. > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at krypto.org Mon Jun 2 01:30:54 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 1 Jun 2008 16:30:54 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483FBCB4.5020007@egenix.com> References: <48397ECC.9070805@cheimes.de> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> Message-ID: <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg wrote: > On 2008-05-30 00:57, Nick Coghlan wrote: >> >> M.-A. Lemburg wrote: >>> >>> * Why can't we have both PyString *and* PyBytes exposed in 2.x, >>> with one redirecting to the other ? >> >> We do have that - the PyString_* names still work perfectly fine in 2.x. >> They just won't be used in the Python core codebase anymore - everything in >> the Python core will use either PyBytes_* or PyUnicode_* regardless of which >> branch (2.x or 3.x) you're working on. I think that's a good thing for ease >> of maintenance in the future, even if it takes people a while to get their >> heads around it right now. > > Sorry, I probably wasn't clear enough: > > Why can't we have both PyString *and* PyBytes exposed as C > APIs (ie. visible in code and in the linker) in 2.x, with one redirecting > to the other ? > >>> * Why should the 2.x code base turn to hacks, just because 3.x wants >>> to restructure itself ? >> >> With the better explanation from Greg of what the checked in approach >> achieves (i.e. preserving exact ABI compatibility for PyString_*, while >> allowing PyBytes_* to be used at the source code level), I don't see what >> has been done as being any more of a hack than the possibly more common >> "#define " (which *would* break binary compatibility). >> >> The only things that I think would tidy it up further would be to: >> - include an explanation of the approach and its effects on API and ABI >> backward and forward compatibility within 2.x and between 2.x and 3.x in >> stringobject.h >> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0 > > Which is what I was suggesting all along; sorry if I wasn't > clear enough on that. > > The standard approach is that you provide #define redirects from the > old APIs to the new ones (which are then picked up by the compiler) > *and* add function wrappers to the same affect (to make linkers, > dynamic load APIs such ctypes and debuggers happy). > > > Example from pythonrun.h|c: > --------------------------- > > /* Use macros for a bunch of old variants */ > #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL) > > /* Deprecated C API functions still provided for binary compatiblity */ > > #undef PyRun_String > PyAPI_FUNC(PyObject *) > PyRun_String(const char *str, int s, PyObject *g, PyObject *l) > { > return PyRun_StringFlags(str, s, g, l, NULL); > } > Okay, how about this? http://codereview.appspot.com/1521 Using that patch, both PyString_ and PyBytes_ APIs are available using function stubs similar to the above. I opted to define the stub functions right next to the ones they were stubbing rather than putting them all at the end of the file or in another file but they could be moved if someone doesn't like them that way. > I still believe that we should *not* make "easy of merging" the > primary motivation for backporting changes in 3.x to 2.x. Software > design should not be guided by restrictions in the tool chain, > if not absolutely necessary. > > The main argument for a backport needs to be general usefulness > to the 2.x users, IMHO... just like any other feature that > makes it into 2.x. > > If merging is difficult then this needs to be addressed, but > there are more options to that than always going back to the > original 2.x trunk code. I've given a few suggestions on how > this could be approached in other emails on this thread. I am not the one doing the merging or working on merge tools so I'll leave this up to those that are. -gps From greg.ewing at canterbury.ac.nz Mon Jun 2 03:16:03 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 02 Jun 2008 13:16:03 +1200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> <483FB8F6.4060504@canterbury.ac.nz> <48427FD7.4090005@canterbury.ac.nz> Message-ID: <484349D3.2050103@canterbury.ac.nz> Guido van Rossum wrote: > There are quite a few core APIs that accept no > substitutes, and being an instance of basestring was intended to > guarantee that a value is accepted by such APIs. In that case, the idea of a user-defined string class that doesn't inherit from str or unicode seems to be a lost cause, since it will never be acceptable in those places, whatever is done with ABCs. -- Greg From ncoghlan at gmail.com Mon Jun 2 12:47:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Jun 2008 20:47:08 +1000 Subject: [Python-Dev] Mini-Pep: An Empty String ABC In-Reply-To: References: Message-ID: <4843CFAC.8070503@gmail.com> Guido van Rossum wrote: > This PEP is incomplete without specifying exactly which built-in and > stdlib types should be registered as String instances. > > I'm also confused -- the motivation seems mostly "so that you can skip > iterating over it when flattening a nested sequence" but earlier you > rejected my "Atomic" proposal, saying "Earlier in the thread it was > made clear that that atomicity is not an intrinsic property of a type; > instead it varies across applications [...]". Isn't this String > proposal just that by another name? > > Finally, I fully expect lots of code writing isinstance(x, String) and > making many more assumptions than promised by the String ABC. For > example that s[0] has the same type as s (not true for bytes). Or that > it is hashable (the Sequence class doesn't define __hash__). Or that > s1+s2 will work (not in the Sequence class either). And many more. I think the PEP also needs to explain why having multiple small one-off string ABCs is a bad thing. The whole point of providing a standard ABC mechanism is to enable exactly that: allowing a library to say "Here is my concept of what a string class needs to provide - register with this ABC to tell me that I can use your class without blowing up unexpectedly". The library can then preregister a bunch of other classes it knows about that do the right thing (such as the builtin str type) That is, to write a flatten operation with ABC's you might do something along the lines of: from abc import ABCMeta class Atomic(metaclass=ABCMeta): """ABC for iterables that the flatten function will not expand""" Atomic.register(str) # Consider builtin strings to be atomic def flatten(obj, atomic=Atomic): itr = None if not isinstance(obj, atomic): try: itr = iter(obj) except (TypeError, AttributeError): pass if itr is not None: for item in itr: for x in flatten(item): yield x else: yield obj Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From regebro at gmail.com Mon Jun 2 16:52:58 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 2 Jun 2008 16:52:58 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <4838EA2D.7060402@jcea.es> References: <4838EA2D.7060402@jcea.es> Message-ID: <319e029f0806020752necf5acer9866c158ce9ac286@mail.gmail.com> On Sun, May 25, 2008 at 6:25 AM, Jesus Cea wrote: > Since I need to port bsddb3 to py3k, what I need to know?. Is any > *updated* document out there?. No, but there is a not yet complete, but quite updated set of examples. http://code.google.com/p/python-incompatibility/ This is a collection of code snippets that will show code that does work under 2.x but not under 3.x, and the 3.x way of doing it (as well as a way that works under both 2.6 and 3.0, in applicable cases). There is no tests for changes in the C-API, if you (or somebody else) would like to add that (or any other tests) that would be very appreciated! -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From skip at pobox.com Mon Jun 2 14:09:14 2008 From: skip at pobox.com (skip at pobox.com) Date: Mon, 2 Jun 2008 07:09:14 -0500 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <4843B971.6080707@gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com> <4842B1F2.6010901@gmail.com> <18499.18335.638383.979060@montanaro-dyndns-org.local> <4843B971.6080707@gmail.com> Message-ID: <18499.58090.240170.352835@montanaro-dyndns-org.local> >> If the 3.0 API of a module is going to involve breakage which >> requires authors to update their applications wouldn't this be a good >> time to PEP-8-ify the module? (Not suggesting that threading would >> fall into this category.) Nick> Updating application code to deal with a module name change is Nick> easy - just use an "import x as y" statement. Catching the Nick> ImportError for the 3.0 name and falling back to the 2.6 name (or Nick> vice-versa) even makes it possible to support both names fairly Nick> easily. Nick> Changing the names of actual objects within the modules is tougher Nick> though - there are many more ways to access those. I think you misunderstood what I wrote. Suppose you decided that the API for the threading modules needs significant rework, changes which will break much, if not all current usage. I'm only suggesting that if you decide the API change is worthwhile that you take the opportunity to make the class, method and data names PEP8-compliant. I'm not talking about gratuitous change to the identifier names, but situations where the user is going to have to rework their code anyway. Skip From jnoller at gmail.com Mon Jun 2 17:11:35 2008 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 2 Jun 2008 11:11:35 -0400 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> Message-ID: <4222a8490806020811u7ea7d7d5ye66b39bee54b1737@mail.gmail.com> On Sat, May 31, 2008 at 6:31 PM, r.m.oudkerk wrote: > On 31/05/2008, Paul Moore wrote: >> 2008/5/30 Farshid Lashkari : >>> I'm not sure if there will be any side affects to modifying >>> sys.executable though. Should this be the official way of supporting >>> embedded interpreters or should there be a >>> multiprocessing.setExecutable() method? >> >> +1 for setExecutable (I'd prefer set_executable, to be PEP 8 >> compliant). Make it explicit, rather than fiddling with stuff in sys >> manually. > > That is easy to do. Also note - someone just pointed out to me that the executable manipulation as-is breaks when you execute things within IDLE. I'll add all of this to the PEP. -jesse From ncoghlan at gmail.com Mon Jun 2 11:12:17 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Jun 2008 19:12:17 +1000 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <18499.18335.638383.979060@montanaro-dyndns-org.local> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com> <4842B1F2.6010901@gmail.com> <18499.18335.638383.979060@montanaro-dyndns-org.local> Message-ID: <4843B971.6080707@gmail.com> skip at pobox.com wrote: > Nick> We fixed the module names that used mixed case - the amount of > Nick> work that turned out to be involved in just doing that much for > Nick> PEP 3108 makes me shudder at the thought of trying to fix all of > Nick> the standard library APIs that currently don't follow the style > Nick> guide... > > If the 3.0 API of a module is going to involve breakage which requires > authors to update their applications wouldn't this be a good time to > PEP-8-ify the module? (Not suggesting that threading would fall into this > category.) Updating application code to deal with a module name change is easy - just use an "import x as y" statement. Catching the ImportError for the 3.0 name and falling back to the 2.6 name (or vice-versa) even makes it possible to support both names fairly easily. Changing the names of actual objects within the modules is tougher though - there are many more ways to access those. Cheers, Nick -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Mon Jun 2 22:54:08 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 2 Jun 2008 13:54:08 -0700 Subject: [Python-Dev] Mini-Pep: An Empty String ABC References: Message-ID: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1> From: "Guido van Rossum" > All this makes me lean towards a rejection of this proposal -- it > seems worse than no proposal at all. It could perhaps be rescued by > adding some small set of defined operations. By subclassing Sequence, we get index() and count() mixins for free. We can also add other mixin freebies like __hash__(), __eq__(), __ne__(), endswith(), startswith(), find(), rfind(), and rindex(). It's tempting to add center, lust, rjust, and zfill, but those require some sort of constructor that accepts an iterable argument. As important as what is included are the methods intentionally left out. I'm trying to avoid insisting on abstractmethods like encode(), split(), join(), and other methods that place an undue burden on a class being registered as a String. Raymond From guido at python.org Mon Jun 2 23:23:15 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Jun 2008 14:23:15 -0700 Subject: [Python-Dev] Mini-Pep: An Empty String ABC In-Reply-To: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1> References: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1> Message-ID: Please try to find the largest set of methods that you're comfortable with. __add__ comes to mind. Note that if you add __hash__, this rules out bytearray -- is that your intention? __hash__ is intentionally not part of the "read-only" ABCs because read-only doesn't mean immutable. Also, (again) please list which built-in types you want to register. On Mon, Jun 2, 2008 at 1:54 PM, Raymond Hettinger wrote: > From: "Guido van Rossum" >> >> All this makes me lean towards a rejection of this proposal -- it >> seems worse than no proposal at all. It could perhaps be rescued by >> adding some small set of defined operations. > > By subclassing Sequence, we get index() and count() mixins for free. > > We can also add other mixin freebies like __hash__(), __eq__(), __ne__(), > endswith(), startswith(), find(), rfind(), and rindex(). > > It's tempting to add center, lust, rjust, and zfill, but those require some > sort of constructor that accepts an iterable argument. > > As important as what is included are the methods intentionally left out. > I'm trying to avoid insisting on abstractmethods like encode(), split(), > join(), and other methods that place an undue burden on a class being > registered as a String. > > > Raymond > > > > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Mon Jun 2 14:33:08 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 02 Jun 2008 14:33:08 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> Message-ID: <4843E884.1060705@egenix.com> On 2008-06-02 01:30, Gregory P. Smith wrote: > On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg wrote: >> Sorry, I probably wasn't clear enough: >> >> Why can't we have both PyString *and* PyBytes exposed as C >> APIs (ie. visible in code and in the linker) in 2.x, with one redirecting >> to the other ? >> >>>> * Why should the 2.x code base turn to hacks, just because 3.x wants >>>> to restructure itself ? >>> With the better explanation from Greg of what the checked in approach >>> achieves (i.e. preserving exact ABI compatibility for PyString_*, while >>> allowing PyBytes_* to be used at the source code level), I don't see what >>> has been done as being any more of a hack than the possibly more common >>> "#define " (which *would* break binary compatibility). >>> >>> The only things that I think would tidy it up further would be to: >>> - include an explanation of the approach and its effects on API and ABI >>> backward and forward compatibility within 2.x and between 2.x and 3.x in >>> stringobject.h >>> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0 >> Which is what I was suggesting all along; sorry if I wasn't >> clear enough on that. >> >> The standard approach is that you provide #define redirects from the >> old APIs to the new ones (which are then picked up by the compiler) >> *and* add function wrappers to the same affect (to make linkers, >> dynamic load APIs such ctypes and debuggers happy). >> >> >> Example from pythonrun.h|c: >> --------------------------- >> >> /* Use macros for a bunch of old variants */ >> #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL) >> >> /* Deprecated C API functions still provided for binary compatiblity */ >> >> #undef PyRun_String >> PyAPI_FUNC(PyObject *) >> PyRun_String(const char *str, int s, PyObject *g, PyObject *l) >> { >> return PyRun_StringFlags(str, s, g, l, NULL); >> } >> > > Okay, how about this? http://codereview.appspot.com/1521 > > Using that patch, both PyString_ and PyBytes_ APIs are available using > function stubs similar to the above. I opted to define the stub > functions right next to the ones they were stubbing rather than > putting them all at the end of the file or in another file but they > could be moved if someone doesn't like them that way. Thanks. I was working on a similar patch. Looks like you beat me to it. The only thing I'm not sure about is having the wrappers in the same file - this is likely to cause merge conflicts when doing direct merging and even with an automated renaming approach, the extra code would be easier to remove if it were e.g. at the end of the file or even better: in a separate file. My patch worked slightly differently: it adds wrappers PyString* that forward calls to the PyBytes* APIs and they all live in stringobject.c. stringobject.h then also provides aliases so that recompiled extensions pick up the new API names. While working on my patch I ran into an issue that I haven't been able to resolve: the wrapper functions got optimized away by the linker and even though they appear in the libpython2.6.a, they don't end up in the python binary itself. As a result, importing Python 2.5 in the resulting 2.6 binary still fails with a unresolved PyString symbol. Please check whether that's the case for your patch as well. >> I still believe that we should *not* make "easy of merging" the >> primary motivation for backporting changes in 3.x to 2.x. Software >> design should not be guided by restrictions in the tool chain, >> if not absolutely necessary. >> >> The main argument for a backport needs to be general usefulness >> to the 2.x users, IMHO... just like any other feature that >> makes it into 2.x. >> >> If merging is difficult then this needs to be addressed, but >> there are more options to that than always going back to the >> original 2.x trunk code. I've given a few suggestions on how >> this could be approached in other emails on this thread. > > I am not the one doing the merging or working on merge tools so I'll > leave this up to those that are. I'm not sure whether there are any specific merge tools around - apart from the 2to3.py script. There also doesn't seem to be any documentation on the merge process itself (at least nothing that Google can find in the PEPs), so it's difficult to make any suggestions. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 02 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 34 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From dalcinl at gmail.com Mon Jun 2 18:05:07 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 2 Jun 2008 13:05:07 -0300 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <00ac01c8c386$e10c68e0$a3253aa0$@com.au> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> <00ac01c8c386$e10c68e0$a3253aa0$@com.au> Message-ID: On 5/31/08, Mark Hammond wrote: > So it seems that maybe simply "setExecutable()" isn't the correct > abstraction here, but maybe a "factory" approach, so the entire process > creation mechanism can be replaced rather than just the name of the > executable to spawn? Indeed. If the spawn mechanisms (and even the connection mechanisms) were fully abstracted, then I believe that extending pyProcessing to work in clusters with something like MPI would be far easier. But perhaps I'm just dreaming... -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From skip at pobox.com Mon Jun 2 03:06:39 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 1 Jun 2008 20:06:39 -0500 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <4842B1F2.6010901@gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> <79990c6b0806010644q7b07a931wc4b01f5d5a48f47b@mail.gmail.com> <4842B1F2.6010901@gmail.com> Message-ID: <18499.18335.638383.979060@montanaro-dyndns-org.local> Nick> We fixed the module names that used mixed case - the amount of Nick> work that turned out to be involved in just doing that much for Nick> PEP 3108 makes me shudder at the thought of trying to fix all of Nick> the standard library APIs that currently don't follow the style Nick> guide... If the 3.0 API of a module is going to involve breakage which requires authors to update their applications wouldn't this be a good time to PEP-8-ify the module? (Not suggesting that threading would fall into this category.) Skip From dalcinl at gmail.com Mon Jun 2 18:17:47 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 2 Jun 2008 13:17:47 -0300 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> Message-ID: Are you completelly sure of adding those guys: PyBytes_InternXXX ??? On 6/1/08, Gregory P. Smith wrote: > On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg wrote: > > On 2008-05-30 00:57, Nick Coghlan wrote: > >> > >> M.-A. Lemburg wrote: > >>> > >>> * Why can't we have both PyString *and* PyBytes exposed in 2.x, > >>> with one redirecting to the other ? > >> > >> We do have that - the PyString_* names still work perfectly fine in 2.x. > >> They just won't be used in the Python core codebase anymore - everything in > >> the Python core will use either PyBytes_* or PyUnicode_* regardless of which > >> branch (2.x or 3.x) you're working on. I think that's a good thing for ease > >> of maintenance in the future, even if it takes people a while to get their > >> heads around it right now. > > > > Sorry, I probably wasn't clear enough: > > > > Why can't we have both PyString *and* PyBytes exposed as C > > APIs (ie. visible in code and in the linker) in 2.x, with one redirecting > > to the other ? > > > >>> * Why should the 2.x code base turn to hacks, just because 3.x wants > >>> to restructure itself ? > >> > >> With the better explanation from Greg of what the checked in approach > >> achieves (i.e. preserving exact ABI compatibility for PyString_*, while > >> allowing PyBytes_* to be used at the source code level), I don't see what > >> has been done as being any more of a hack than the possibly more common > >> "#define " (which *would* break binary compatibility). > >> > >> The only things that I think would tidy it up further would be to: > >> - include an explanation of the approach and its effects on API and ABI > >> backward and forward compatibility within 2.x and between 2.x and 3.x in > >> stringobject.h > >> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0 > > > > Which is what I was suggesting all along; sorry if I wasn't > > clear enough on that. > > > > The standard approach is that you provide #define redirects from the > > old APIs to the new ones (which are then picked up by the compiler) > > *and* add function wrappers to the same affect (to make linkers, > > dynamic load APIs such ctypes and debuggers happy). > > > > > > Example from pythonrun.h|c: > > --------------------------- > > > > /* Use macros for a bunch of old variants */ > > #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL) > > > > /* Deprecated C API functions still provided for binary compatiblity */ > > > > #undef PyRun_String > > PyAPI_FUNC(PyObject *) > > PyRun_String(const char *str, int s, PyObject *g, PyObject *l) > > { > > return PyRun_StringFlags(str, s, g, l, NULL); > > } > > > > > Okay, how about this? http://codereview.appspot.com/1521 > > Using that patch, both PyString_ and PyBytes_ APIs are available using > function stubs similar to the above. I opted to define the stub > functions right next to the ones they were stubbing rather than > putting them all at the end of the file or in another file but they > could be moved if someone doesn't like them that way. > > > > I still believe that we should *not* make "easy of merging" the > > primary motivation for backporting changes in 3.x to 2.x. Software > > design should not be guided by restrictions in the tool chain, > > if not absolutely necessary. > > > > The main argument for a backport needs to be general usefulness > > to the 2.x users, IMHO... just like any other feature that > > makes it into 2.x. > > > > If merging is difficult then this needs to be addressed, but > > there are more options to that than always going back to the > > original 2.x trunk code. I've given a few suggestions on how > > this could be approached in other emails on this thread. > > > I am not the one doing the merging or working on merge tools so I'll > leave this up to those that are. > > -gps > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From greg at krypto.org Tue Jun 3 00:21:18 2008 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 2 Jun 2008 15:21:18 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <4843E884.1060705@egenix.com> References: <48397ECC.9070805@cheimes.de> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> Message-ID: <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> On Mon, Jun 2, 2008 at 5:33 AM, M.-A. Lemburg wrote: > >> Okay, how about this? http://codereview.appspot.com/1521 >> >> Using that patch, both PyString_ and PyBytes_ APIs are available using >> function stubs similar to the above. I opted to define the stub >> functions right next to the ones they were stubbing rather than >> putting them all at the end of the file or in another file but they >> could be moved if someone doesn't like them that way. >> > > Thanks. I was working on a similar patch. Looks like you beat > me to it. > > The only thing I'm not sure about is having the wrappers in the > same file - this is likely to cause merge conflicts when doing > direct merging and even with an automated renaming approach, > the extra code would be easier to remove if it were e.g. at > the end of the file or even better: in a separate file. > > My patch worked slightly differently: it adds wrappers PyString* > that forward calls to the PyBytes* APIs and they all live in > stringobject.c. stringobject.h then also provides aliases > so that recompiled extensions pick up the new API names. > > While working on my patch I ran into an issue that I haven't > been able to resolve: the wrapper functions got optimized away > by the linker and even though they appear in the libpython2.6.a, > they don't end up in the python binary itself. > > As a result, importing Python 2.5 in the resulting 2.6 > binary still fails with a unresolved PyString symbol. > > Please check whether that's the case for your patch as well. I think that is going to happen no matter which approach is used (yours or mine) unless we force some included code to call each of the stubs (needlessly inefficient). One way to do that is to reference them all from a section of code called conditionally based upon an always false condition that the compiler and linker can never predetermine is false so that it cannot be eliminated as dead code. Given that, should we bother? I don't think we really need PyBytes_ to show up in the binary ABI for 2.x even if that is how we write the calls in the python internals code. The arguments put forth that debugging is easier if you can just set a breakpoint on what you read may be true but including stub functions doesn't help this when most of the time they're compiled under the alternate name using #defines so a breakpoint set on the stub name will not actually trigger. API wise we're really providing the PyBytes* names to make module author's work of writing code that targets 2.6 and 3.x easier but isn't it reasonable for authors to just be told that they're just #defined aliases for PyString*. There is no goal, nor should there be, of a module binary compiled against 2.x loading and working in 3.x. I expect most module authors, code generators and such will want to target Python 2.x earlier than 2.6 as well so should we provide PyBytes_ names as a public API in 2.6 at all? (regardless of if we use the PyBytes names internally for any reason) -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Tue Jun 3 00:22:51 2008 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 2 Jun 2008 15:22:51 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> Message-ID: <52dc1c820806021522u77e94406q7575a34bcaee79c1@mail.gmail.com> -cc: python-3000 I believe those APIs are already there in the existing interface. Why does that concern you? On Mon, Jun 2, 2008 at 9:17 AM, Lisandro Dalcin wrote: > Are you completelly sure of adding those guys: PyBytes_InternXXX ??? > > > On 6/1/08, Gregory P. Smith wrote: > > On Fri, May 30, 2008 at 1:37 AM, M.-A. Lemburg wrote: > > > On 2008-05-30 00:57, Nick Coghlan wrote: > > >> > > >> M.-A. Lemburg wrote: > > >>> > > >>> * Why can't we have both PyString *and* PyBytes exposed in 2.x, > > >>> with one redirecting to the other ? > > >> > > >> We do have that - the PyString_* names still work perfectly fine in > 2.x. > > >> They just won't be used in the Python core codebase anymore - > everything in > > >> the Python core will use either PyBytes_* or PyUnicode_* regardless > of which > > >> branch (2.x or 3.x) you're working on. I think that's a good thing > for ease > > >> of maintenance in the future, even if it takes people a while to get > their > > >> heads around it right now. > > > > > > Sorry, I probably wasn't clear enough: > > > > > > Why can't we have both PyString *and* PyBytes exposed as C > > > APIs (ie. visible in code and in the linker) in 2.x, with one > redirecting > > > to the other ? > > > > > >>> * Why should the 2.x code base turn to hacks, just because 3.x wants > > >>> to restructure itself ? > > >> > > >> With the better explanation from Greg of what the checked in approach > > >> achieves (i.e. preserving exact ABI compatibility for PyString_*, > while > > >> allowing PyBytes_* to be used at the source code level), I don't see > what > > >> has been done as being any more of a hack than the possibly more > common > > >> "#define " (which *would* break binary > compatibility). > > >> > > >> The only things that I think would tidy it up further would be to: > > >> - include an explanation of the approach and its effects on API and > ABI > > >> backward and forward compatibility within 2.x and between 2.x and 3.x > in > > >> stringobject.h > > >> - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0 > > > > > > Which is what I was suggesting all along; sorry if I wasn't > > > clear enough on that. > > > > > > The standard approach is that you provide #define redirects from the > > > old APIs to the new ones (which are then picked up by the compiler) > > > *and* add function wrappers to the same affect (to make linkers, > > > dynamic load APIs such ctypes and debuggers happy). > > > > > > > > > Example from pythonrun.h|c: > > > --------------------------- > > > > > > /* Use macros for a bunch of old variants */ > > > #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, > NULL) > > > > > > /* Deprecated C API functions still provided for binary compatiblity > */ > > > > > > #undef PyRun_String > > > PyAPI_FUNC(PyObject *) > > > PyRun_String(const char *str, int s, PyObject *g, PyObject *l) > > > { > > > return PyRun_StringFlags(str, s, g, l, NULL); > > > } > > > > > > > > > Okay, how about this? http://codereview.appspot.com/1521 > > > > Using that patch, both PyString_ and PyBytes_ APIs are available using > > function stubs similar to the above. I opted to define the stub > > functions right next to the ones they were stubbing rather than > > putting them all at the end of the file or in another file but they > > could be moved if someone doesn't like them that way. > > > > > > > I still believe that we should *not* make "easy of merging" the > > > primary motivation for backporting changes in 3.x to 2.x. Software > > > design should not be guided by restrictions in the tool chain, > > > if not absolutely necessary. > > > > > > The main argument for a backport needs to be general usefulness > > > to the 2.x users, IMHO... just like any other feature that > > > makes it into 2.x. > > > > > > If merging is difficult then this needs to be addressed, but > > > there are more options to that than always going back to the > > > original 2.x trunk code. I've given a few suggestions on how > > > this could be approached in other emails on this thread. > > > > > > I am not the one doing the merging or working on merge tools so I'll > > leave this up to those that are. > > > > -gps > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com > > > > > -- > Lisandro Dalc?n > --------------- > Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) > Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) > Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) > PTLC - G?emes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Tue Jun 3 00:51:46 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 2 Jun 2008 18:51:46 -0400 Subject: [Python-Dev] Postponing the first betas Message-ID: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We are going to postpone the first beta releases by one week. We had some problems with mail.python.org today, which prompted a query to Guido from me about the postponement. mail.python.org should now be back up normally now, as evidenced by the emailfloodl but in the meantime, Guido said: "I'd also like to see PEP 3138 (CJK-friendly repr()) and the pyprocessing PEP implemented by then, and perhaps some other small stuff." So we're going to do the first beta releases next Wednesday, June 11. Please take this time to stabilize all APIs and features, both in Python and C. Next week, I'll do a gut check on critical and release blocker bugs, so please also take a look at those and try to fix what you can. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSER5gnEjvBPtnXfVAQJlBAQAgfmRwGQzwNFrwvMusIoDNVRuyIObkKO0 FeDYb26RAL1jLXt0x/7jE0fBc5FvhDzUJnnNj3sydfyKU5MCb0eB0VeBTmjHU05l yncX6zYSoU14OUW+bkG4y7vf+aLD9zlFsj/ybMEZTQh0RMpZ+HBNhup3NJFEDTBM 97q4SIvltAg= =NBRW -----END PGP SIGNATURE----- From guido at python.org Tue Jun 3 01:09:18 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Jun 2008 16:09:18 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> Message-ID: I will freely admit that I haven't followed this thread in any detail, but if it were up to me, I'd have the 2.6 internal code use PyString (as both what the linker sees and what the human reads in the source code) and the 3.0 code use PyBytes for the same thing. Let the merges be damed -- most changes to 2.6 these days seem to be blocked explicitly from being merged anyway. I'd prefer the 2.6 code base to stay true to 2.x, and the 3.0 code base start afresh where it makes sense. We should reindent more of the 3.0 code base to use 4-space-indents in C code too. I would also add macros that map the PyBytes_* APIs to PyString_*, but I would not start using these internally except in code newly written for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString for 8-bit strings containing text, and PyBytes for 8-bit strings containing binary data. For 8-bit strings that could contain either text or data, I'd use PyString, in the spirit of 2.x. --Guido On Mon, Jun 2, 2008 at 3:21 PM, Gregory P. Smith wrote: > > > On Mon, Jun 2, 2008 at 5:33 AM, M.-A. Lemburg wrote: >>> >>> Okay, how about this? http://codereview.appspot.com/1521 >>> >>> Using that patch, both PyString_ and PyBytes_ APIs are available using >>> function stubs similar to the above. I opted to define the stub >>> functions right next to the ones they were stubbing rather than >>> putting them all at the end of the file or in another file but they >>> could be moved if someone doesn't like them that way. >> >> Thanks. I was working on a similar patch. Looks like you beat >> me to it. >> >> The only thing I'm not sure about is having the wrappers in the >> same file - this is likely to cause merge conflicts when doing >> direct merging and even with an automated renaming approach, >> the extra code would be easier to remove if it were e.g. at >> the end of the file or even better: in a separate file. >> >> My patch worked slightly differently: it adds wrappers PyString* >> that forward calls to the PyBytes* APIs and they all live in >> stringobject.c. stringobject.h then also provides aliases >> so that recompiled extensions pick up the new API names. >> >> While working on my patch I ran into an issue that I haven't >> been able to resolve: the wrapper functions got optimized away >> by the linker and even though they appear in the libpython2.6.a, >> they don't end up in the python binary itself. >> >> As a result, importing Python 2.5 in the resulting 2.6 >> binary still fails with a unresolved PyString symbol. >> >> Please check whether that's the case for your patch as well. > > I think that is going to happen no matter which approach is used (yours or > mine) unless we force some included code to call each of the stubs > (needlessly inefficient). One way to do that is to reference them all from > a section of code called conditionally based upon an always false condition > that the compiler and linker can never predetermine is false so that it > cannot be eliminated as dead code. > > Given that, should we bother? I don't think we really need PyBytes_ to show > up in the binary ABI for 2.x even if that is how we write the calls in the > python internals code. The arguments put forth that debugging is easier if > you can just set a breakpoint on what you read may be true but including > stub functions doesn't help this when most of the time they're compiled > under the alternate name using #defines so a breakpoint set on the stub name > will not actually trigger. > > API wise we're really providing the PyBytes* names to make module author's > work of writing code that targets 2.6 and 3.x easier but isn't it reasonable > for authors to just be told that they're just #defined aliases for > PyString*. There is no goal, nor should there be, of a module binary > compiled against 2.x loading and working in 3.x. > > I expect most module authors, code generators and such will want to target > Python 2.x earlier than 2.6 as well so should we provide PyBytes_ names as a > public API in 2.6 at all? (regardless of if we use the PyBytes names > internally for any reason) > > -gps > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at krypto.org Tue Jun 3 01:29:16 2008 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 2 Jun 2008 16:29:16 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> Message-ID: <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum wrote: > I will freely admit that I haven't followed this thread in any detail, > but if it were up to me, I'd have the 2.6 internal code use PyString ... Should we read this as a BDFL pronouncement and make it so? All that would mean change wise is that trunk r63675 as well as possibly r63672 and r63677 would need to be rolled back and this whole discussion over if such a big change should have happened would turn into a moot point. I would also add macros that map the PyBytes_* APIs to PyString_*, but > I would not start using these internally except in code newly written > for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString > for 8-bit strings containing text, and PyBytes for 8-bit strings > containing binary data. For 8-bit strings that could contain either > text or data, I'd use PyString, in the spirit of 2.x. > > --Guido > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From techtonik at gmail.com Mon Jun 2 08:59:11 2008 From: techtonik at gmail.com (techtonik) Date: Mon, 2 Jun 2008 08:59:11 +0200 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: <79990c6b0805240615l65975730i9a8d289e2315eada@mail.gmail.com> References: <4837E463.3030805@v.loewis.de> <79990c6b0805240615l65975730i9a8d289e2315eada@mail.gmail.com> Message-ID: On Sat, May 24, 2008 at 3:15 PM, Paul Moore wrote: > > > As for PDCurses library itself there is a Makefile in PDCurses > distribution > > for Microsoft Visual C++ 2.0+ named vcwin32.mak I can't afford buying > > Visual Studio to test if it works with newer versions, but logically > Visual > > Studio should be able to convert Makefile to a newer format. > > Visual C++ 9.0 Express Edition builds Python quite happily these days. > So you can certainly do the integration without buying anything. If > you get stuck on technical details, there are people here who would > happily give you advice. Windows developers are always welcome! Finally, I've downloaded Microsoft Visual Studio Express 2008, but it requires Windows XP to install and I have only Windows2000 at hand. I will try to get MS Visual Studio Express 2005 and see if it works. > If you don't have the knowledge needed, and can't spare the time to > learn (which is entirely acceptable) then you are indeed relying on > another Windows developer to pick this up. You may be out of luck > there - nobody has been interested enough to do this before now (it's > not as if PDCurses is new) so there's not much reason to expect things > to have changed. > > I'll try to spare the time to learn how to include curses support in windows, but I will appreciate if anybody could review the patches and convert project to MSVC 2008 in the end. -- --anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Tue Jun 3 02:22:51 2008 From: brett at python.org (Brett Cannon) Date: Mon, 2 Jun 2008 17:22:51 -0700 Subject: [Python-Dev] [Python-3000] Postponing the first betas In-Reply-To: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> Message-ID: On Mon, Jun 2, 2008 at 3:51 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > We are going to postpone the first beta releases by one week. We had some > problems with mail.python.org today, which prompted a query to Guido from me > about the postponement. mail.python.org should now be back up normally now, > as evidenced by the emailfloodl but in the meantime, Guido said: > > "I'd also like to see PEP 3138 (CJK-friendly repr()) and the > pyprocessing PEP implemented by then, and perhaps some other small > stuff." > > So we're going to do the first beta releases next Wednesday, June 11. > Please take this time to stabilize all APIs and features, both in Python > and C. Next week, I'll do a gut check on critical and release blocker bugs, > so please also take a look at those and try to fix what you can. > Now is as good a time as any to mention that on Wednesday I am flying out to help my mother move. I don't know when she is going to have her Internet connection set up, so I might not be back online until June 16. But thanks to all the help I have been receiving on PEP 3108, I trust the various people involved to continue to do the right thing in my absence. -Brett From musiccomposition at gmail.com Tue Jun 3 02:26:22 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 2 Jun 2008 19:26:22 -0500 Subject: [Python-Dev] [Python-3000] Postponing the first betas In-Reply-To: References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> Message-ID: <1afaf6160806021726w2ed8d453p9467ee006b90aaee@mail.gmail.com> On Mon, Jun 2, 2008 at 7:22 PM, Brett Cannon wrote: >> > > Now is as good a time as any to mention that on Wednesday I am flying > out to help my mother move. I don't know when she is going to have her > Internet connection set up, so I might not be back online until June > 16. But thanks to all the help I have been receiving on PEP 3108, I > trust the various people involved to continue to do the right thing in > my absence. That reminds me of those Dilbert cartoons where his mother ends up knowing much more about computers than he does. :) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Tue Jun 3 02:49:40 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 2 Jun 2008 19:49:40 -0500 Subject: [Python-Dev] converting the stdlib to str.format Message-ID: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> Hi all, As a newly converted fan of str.format, it gives me pangs to see the whole stdlib using "%." I realize that str.format is not quite up to the speed standards we'd like, but I'm sure that will change. In any case, I'm willing to give the TLC to convert the whole stdlib to str.format, so I just need your permission! -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From guido at python.org Tue Jun 3 04:04:07 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Jun 2008 19:04:07 -0700 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> Message-ID: On Mon, Jun 2, 2008 at 5:49 PM, Benjamin Peterson wrote: > As a newly converted fan of str.format, it gives me pangs to see the > whole stdlib using "%." I realize that str.format is not quite up to > the speed standards we'd like, but I'm sure that will change. > > In any case, I'm willing to give the TLC to convert the whole stdlib > to str.format, so I just need your permission! So glad you asked! There's a standard answer whenever someone suggests a sweeping conversion of the entire stdlib to some new feature. It's "No." :-) In the past, we've tried things like this a few times, and it was nearly always a mistake. Things like this can't be automated 100%, and doing a manual audit of the changes is so incredibly tedious that inevitably a few incorrect changes slip by. In theory these would be caught by unittests, but in practice the difference between practice and theory is larger than in theory, so inevitably the honor falls to some poor schmuck whose application fails in a non-trivial way, often months later. I recall one particularly ill-fated conversion where we some bogus conversions weren't fixed until several years (and releases!) later. And these were pretty badly botched too, something like foo(bar(x, y)) vs. foo(bar(x), y) -- it was just in rarely executed code and due to the complexity of the expressions (and the fact that the parentheses were balanced ;-) nobody had noticed before. It's much better to do this piecemeal, when you are revising a particular module for some other reason. Then you (presumably) have the time to review every change thoroughly -- and hopefully you can even add unittests for all code you touch. I realize that 2to3 might make such conversions less error-prone. But most 2to3 conversions are not 100% correct either, and the problem of falling asleep while reviewing the changes exists especially where there are so many correct conversions! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Jun 3 09:41:40 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jun 2008 09:41:40 +0200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> Message-ID: <4844F5B4.5090703@v.loewis.de> > In any case, I'm willing to give the TLC to convert the whole stdlib > to str.format, so I just need your permission! Please don't - not before % is actually deprecated (which I hope won't happen until Python 4, with removal of % in Python 5, in the year when I retire, i.e. 2037). Regards, Martin From g.brandl at gmx.net Tue Jun 3 10:21:09 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Jun 2008 10:21:09 +0200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4844F5B4.5090703@v.loewis.de> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >> In any case, I'm willing to give the TLC to convert the whole stdlib >> to str.format, so I just need your permission! > > Please don't - not before % is actually deprecated (which I hope won't > happen until Python 4, with removal of % in Python 5, in the year > when I retire, i.e. 2037). Now this is news to me -- was there a discussion that changed the lifetime expectancy of str.__mod__? I'd always supposed it being deprecated at some point in 3.x. Not that I'm opposed to keeping it... it *will* be a pain to migrate. Georg From solipsis at pitrou.net Tue Jun 3 10:44:29 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Jun 2008 08:44:29 +0000 (UTC) Subject: [Python-Dev] Mini-Pep: An Empty String ABC References: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1> Message-ID: Raymond Hettinger rcn.com> writes: > > By subclassing Sequence, we get index() and count() mixins for free. > It seems to me that Sequence.index()/count() and String.index()/count() shouldn't have the same semantics. In the former case they search for items in the Sequence, in the latter case they search for substrings of the String. From eric+python-dev at trueblade.com Tue Jun 3 11:03:56 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 03 Jun 2008 05:03:56 -0400 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> Message-ID: <484508FC.6070700@trueblade.com> Georg Brandl wrote: > Martin v. L?wis schrieb: >>> In any case, I'm willing to give the TLC to convert the whole stdlib >>> to str.format, so I just need your permission! >> >> Please don't - not before % is actually deprecated (which I hope won't >> happen until Python 4, with removal of % in Python 5, in the year >> when I retire, i.e. 2037). > > Now this is news to me -- was there a discussion that changed the > lifetime expectancy of str.__mod__? I'd always supposed it being > deprecated at some point in 3.x. > > Not that I'm opposed to keeping it... it *will* be a pain to migrate. Guido has previously said he wouldn't mind adding a PendingDeprecationWarning to %-formatting in 3.0. I've attempted to do that in http://bugs.python.org/issue2772. For a reason I don't understand, my change broke test_doctest.py, so I've never applied it. Eric. From python at rcn.com Tue Jun 3 12:09:09 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 3 Jun 2008 03:09:09 -0700 Subject: [Python-Dev] Mini-Pep: An Empty String ABC References: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1> Message-ID: <0D0F7B10924D4B19BB45DDCC121A7435@RaymondLaptop1> From: "Antoine Pitrou" > It seems to me that Sequence.index()/count() and String.index()/count() > shouldn't have the same semantics. > In the former case they search for items in the Sequence, in the latter case > they search for substrings of the String. And the same applies to __contains__(). Raymond From solipsis at pitrou.net Tue Jun 3 12:39:04 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Jun 2008 10:39:04 +0000 (UTC) Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 References: <48397ECC.9070805@cheimes.de> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> Message-ID: Guido van Rossum python.org> writes: > I'd prefer the 2.6 code base to > stay true to 2.x, and the 3.0 code base start afresh where it makes > sense. We should reindent more of the 3.0 code base to use > 4-space-indents in C code too. Is there any reason reindenting shouldn't be done for 2.6 too? (apart from "staying true to 2.x" :-)) Antoine. From g.brandl at gmx.net Tue Jun 3 12:48:54 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 03 Jun 2008 12:48:54 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> Message-ID: Antoine Pitrou schrieb: > Guido van Rossum python.org> writes: >> I'd prefer the 2.6 code base to >> stay true to 2.x, and the 3.0 code base start afresh where it makes >> sense. We should reindent more of the 3.0 code base to use >> 4-space-indents in C code too. > > Is there any reason reindenting shouldn't be done for 2.6 too? > (apart from "staying true to 2.x" :-)) It would make svn blame useless, for a start. (SVN could really use a feature to exclude certain revisions from showing up in svn blame.) Georg From eric+python-dev at trueblade.com Tue Jun 3 12:55:37 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 03 Jun 2008 06:55:37 -0400 Subject: [Python-Dev] Missing checkin emails Message-ID: <48452329.6020502@trueblade.com> Did some checkin emails get lost yesterday? I didn't see mail for a checkin of mine, r63895. Looking at http://mail.python.org/pipermail/python-checkins/2008-June/date.html, it looks like r63887-r63898 are missing from the archive, too. We should probably make an effort to review those checkins. Eric. From barry at python.org Tue Jun 3 13:34:00 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 3 Jun 2008 07:34:00 -0400 Subject: [Python-Dev] Missing checkin emails In-Reply-To: <48452329.6020502@trueblade.com> References: <48452329.6020502@trueblade.com> Message-ID: <4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 3, 2008, at 6:55 AM, Eric Smith wrote: > Did some checkin emails get lost yesterday? I didn't see mail for a > checkin of mine, r63895. Looking at http://mail.python.org/pipermail/python-checkins/2008-June/date.html > , it looks like r63887-r63898 are missing from the archive, too. > > We should probably make an effort to review those checkins. mail.python.org had problems yesterday, so if the mail client doesn't resend, it's very possible that messages got lost. Everything is back to normal now. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSEUsKnEjvBPtnXfVAQKPbwP+MQGUbxyydoa0OAwMM58mS1IhzLubwCpy IUd5AQulh4FPRLn04CdOQQzYlmOJg9gtkQUIlQx4iiap75H4QqM91ubSdEnzPrPH vUXIXvnujKLwEZ4PN08ZQJffp7o5w8ARLso9U0ttYAtlXjx4QYEmxpyKVVe/V/dB zemVnvW4qfE= =eojc -----END PGP SIGNATURE----- From eric+python-dev at trueblade.com Tue Jun 3 14:10:56 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 03 Jun 2008 08:10:56 -0400 Subject: [Python-Dev] Missing checkin emails In-Reply-To: <4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org> References: <48452329.6020502@trueblade.com> <4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org> Message-ID: <484534D0.3060602@trueblade.com> Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jun 3, 2008, at 6:55 AM, Eric Smith wrote: > >> Did some checkin emails get lost yesterday? I didn't see mail for a >> checkin of mine, r63895. Looking at >> http://mail.python.org/pipermail/python-checkins/2008-June/date.html, >> it looks like r63887-r63898 are missing from the archive, too. >> >> We should probably make an effort to review those checkins. > > mail.python.org had problems yesterday, so if the mail client doesn't > resend, it's very possible that messages got lost. Everything is back > to normal now. Thanks, Barry. Does anyone know if we can force svn to resend the checkin notifications for that range? It also looks like r63862 and r63915 are missing. I haven't done an exhaustive analysis of what is and isn't present (I also don't know if it's normal for revisions to be missing). Eric. From guido at python.org Tue Jun 3 16:29:32 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Jun 2008 07:29:32 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> Message-ID: On Tue, Jun 3, 2008 at 3:48 AM, Georg Brandl wrote: > Antoine Pitrou schrieb: >> >> Guido van Rossum python.org> writes: >>> >>> I'd prefer the 2.6 code base to >>> stay true to 2.x, and the 3.0 code base start afresh where it makes >>> sense. We should reindent more of the 3.0 code base to use >>> 4-space-indents in C code too. >> >> Is there any reason reindenting shouldn't be done for 2.6 too? >> (apart from "staying true to 2.x" :-)) > > It would make svn blame useless, for a start. > > (SVN could really use a feature to exclude certain revisions from > showing up in svn blame.) What he said. And "staying true to 2.x" is not a bad rationale. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Jun 3 16:37:11 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Jun 2008 07:37:11 -0700 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <484508FC.6070700@trueblade.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <484508FC.6070700@trueblade.com> Message-ID: On Tue, Jun 3, 2008 at 2:03 AM, Eric Smith wrote: > Georg Brandl wrote: >> >> Martin v. L?wis schrieb: >>>> >>>> In any case, I'm willing to give the TLC to convert the whole stdlib >>>> to str.format, so I just need your permission! >>> >>> Please don't - not before % is actually deprecated (which I hope won't >>> happen until Python 4, with removal of % in Python 5, in the year >>> when I retire, i.e. 2037). >> >> Now this is news to me -- was there a discussion that changed the >> lifetime expectancy of str.__mod__? I'd always supposed it being >> deprecated at some point in 3.x. I think Martin was attempting humor. :-) There's widespread disagreement on when we should retire %. >> Not that I'm opposed to keeping it... it *will* be a pain to migrate. > > Guido has previously said he wouldn't mind adding a > PendingDeprecationWarning to %-formatting in 3.0. I've attempted to do that > in http://bugs.python.org/issue2772. For a reason I don't understand, my > change broke test_doctest.py, so I've never applied it. For 3.0, it should be at best a SilentDeprecationWarning. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mal at egenix.com Tue Jun 3 19:43:45 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 03 Jun 2008 19:43:45 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> Message-ID: <484582D1.7000309@egenix.com> On 2008-06-03 01:09, Guido van Rossum wrote: > I will freely admit that I haven't followed this thread in any detail, > but if it were up to me, I'd have the 2.6 internal code use PyString > (as both what the linker sees and what the human reads in the source > code) and the 3.0 code use PyBytes for the same thing. Let the merges > be damed -- most changes to 2.6 these days seem to be blocked > explicitly from being merged anyway. I'd prefer the 2.6 code base to > stay true to 2.x, and the 3.0 code base start afresh where it makes > sense. We should reindent more of the 3.0 code base to use > 4-space-indents in C code too. > > I would also add macros that map the PyBytes_* APIs to PyString_*, but > I would not start using these internally except in code newly written > for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString > for 8-bit strings containing text, and PyBytes for 8-bit strings > containing binary data. For 8-bit strings that could contain either > text or data, I'd use PyString, in the spirit of 2.x. +1 Let's work on better merge tools that edit the trunk code base into shape for a 3.x checkin. Using automated tools for this is likely going to lower the probability of bugs introduced due to unnoticed merge conflicts and in the end is also going to be a benefit to everyone wanting to maintain a single code base for both targets. Perhaps we could revive the old Tools/scripts/fixcid.py that was used for the 1.4->1.5 renaming ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 03 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 33 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From martin at v.loewis.de Tue Jun 3 20:54:02 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jun 2008 20:54:02 +0200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> Message-ID: <4845934A.6030708@v.loewis.de> >> Please don't - not before % is actually deprecated (which I hope won't >> happen until Python 4, with removal of % in Python 5, in the year >> when I retire, i.e. 2037). > > Now this is news to me -- was there a discussion that changed the > lifetime expectancy of str.__mod__? I'd always supposed it being > deprecated at some point in 3.x. The PEP doesn't specify anything, and I don't recall any discussion, either - the specific timing suggested above is merely my own hopes. Regards, Martin From martin at v.loewis.de Tue Jun 3 20:58:24 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 03 Jun 2008 20:58:24 +0200 Subject: [Python-Dev] Missing checkin emails In-Reply-To: <484534D0.3060602@trueblade.com> References: <48452329.6020502@trueblade.com> <4E4C5FD0-5521-48A5-93B6-DA22036D9CBD@python.org> <484534D0.3060602@trueblade.com> Message-ID: <48459450.7020505@v.loewis.de> > Does anyone know if we can force svn to resend the checkin notifications > for that range? It also looks like r63862 and r63915 are missing. I > haven't done an exhaustive analysis of what is and isn't present (I also > don't know if it's normal for revisions to be missing). I just resent the range that you indicated (r63887-r63898, inclusive). Regards, Martin From r.m.oudkerk at googlemail.com Tue Jun 3 21:16:31 2008 From: r.m.oudkerk at googlemail.com (r.m.oudkerk) Date: Tue, 3 Jun 2008 20:16:31 +0100 Subject: [Python-Dev] Postponing the first betas In-Reply-To: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> Message-ID: On 02/06/2008, Barry Warsaw wrote: > meantime, Guido said: > > "I'd also like to see PEP 3138 (CJK-friendly repr()) and the > pyprocessing PEP implemented by then, and perhaps some other small > stuff." The pyprocessing unit tests crash with a fatal error when run on Linux with a debug version of the interpreter. This is because the GILState stuff is not fork aware. I submitted a patch some months ago: http://bugs.python.org/issue1683 Could somebody review it please. Cheers, Richard. From jnoller at gmail.com Tue Jun 3 23:30:53 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 3 Jun 2008 17:30:53 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion Message-ID: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> Per feedback from Guido, the python-dev list and others, I have sent in an updated version of PEP 371 - the inclusion of the pyprocessing module into the standard library. URL: http://www.python.org/dev/peps/pep-0371/ New highlights: * The module will be renamed to "multiprocessing" * The API will become PEP 8 compliant * Additional Benchmark data added (thanks Alec!) * Additional grammar and terminology changes Please review the new more final draft of the PEP and send out suggestions and feedback. It looks like this is on the slate for 2.6b1 and 3.0b1 so the sooner we get feedback, the better. -Jesse From p.f.moore at gmail.com Tue Jun 3 23:38:42 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 3 Jun 2008 22:38:42 +0100 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> Message-ID: <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> 2008/6/3 Jesse Noller : > Per feedback from Guido, the python-dev list and others, I have sent > in an updated version of PEP 371 - the inclusion of the pyprocessing > module into the standard library. > > URL: http://www.python.org/dev/peps/pep-0371/ > > New highlights: > * The module will be renamed to "multiprocessing" > * The API will become PEP 8 compliant Doesn't that kill the intent that it's a drop-in replacement for threading? I'd say don't do this unless threading is made PEP 8 compliant as well. Paul. From python at rcn.com Tue Jun 3 23:43:27 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 3 Jun 2008 14:43:27 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> Message-ID: >> * The API will become PEP 8 compliant > > Doesn't that kill the intent that it's a drop-in replacement for > threading? IMO, it is essential that the API match the theading module, PEP 8 be damned. Raymond From jnoller at gmail.com Wed Jun 4 00:02:58 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 3 Jun 2008 18:02:58 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> Message-ID: <465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com> What about also including a patch to fix the threading api to be pep 8 compliant? On Jun 3, 2008, at 5:43 PM, "Raymond Hettinger" wrote: >>> * The API will become PEP 8 compliant >> Doesn't that kill the intent that it's a drop-in replacement for >> threading? > > IMO, it is essential that the API match the theading module, PEP 8 > be damned. > > > Raymond > > From musiccomposition at gmail.com Wed Jun 4 00:05:08 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 3 Jun 2008 17:05:08 -0500 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com> Message-ID: <1afaf6160806031505h1ac5975bt6a0f9f7b4e8e922b@mail.gmail.com> On Tue, Jun 3, 2008 at 5:02 PM, Jesse Noller wrote: > What about also including a patch to fix the threading api to be pep 8 > compliant? I doubt that will be accepted because of the closeness threading has to Java's APIs. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From jnoller at gmail.com Wed Jun 4 00:08:05 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 3 Jun 2008 18:08:05 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> Message-ID: Also - we could leave in stubs to match the threading api - Guido, David Goodger and others really prefer not to continue the "broken" API of the threading API On Jun 3, 2008, at 5:43 PM, "Raymond Hettinger" wrote: >>> * The API will become PEP 8 compliant >> Doesn't that kill the intent that it's a drop-in replacement for >> threading? > > IMO, it is essential that the API match the theading module, PEP 8 > be damned. > > > Raymond > > From solipsis at pitrou.net Wed Jun 4 00:27:42 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Jun 2008 22:27:42 +0000 (UTC) Subject: [Python-Dev] PEP 371: Additional Discussion References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <465497BF-CB4A-46F3-B438-3B60C1FF7E3A@gmail.com> <1afaf6160806031505h1ac5975bt6a0f9f7b4e8e922b@mail.gmail.com> Message-ID: Benjamin Peterson gmail.com> writes: > On Tue, Jun 3, 2008 at 5:02 PM, Jesse Noller gmail.com> wrote: > > What about also including a patch to fix the threading api to be pep 8 > > compliant? > > I doubt that will be accepted because of the closeness threading has > to Java's APIs. Is this really a serious concern for some people? If you come from the Java world and are learning Python, I suppose there are things more difficult to assimilate than the change in method naming style... From brett at python.org Wed Jun 4 00:28:02 2008 From: brett at python.org (Brett Cannon) Date: Tue, 3 Jun 2008 15:28:02 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> Message-ID: On Tue, Jun 3, 2008 at 3:08 PM, Jesse Noller wrote: > Also - we could leave in stubs to match the threading api - Guido, David > Goodger and others really prefer not to continue the "broken" API of the > threading API > +1 from me. Gives a transition plan for people to move over to the new API. -Brett > On Jun 3, 2008, at 5:43 PM, "Raymond Hettinger" wrote: > >>>> * The API will become PEP 8 compliant >>> >>> Doesn't that kill the intent that it's a drop-in replacement for >>> threading? >> >> IMO, it is essential that the API match the theading module, PEP 8 be >> damned. >> >> >> Raymond >> >> > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > From musiccomposition at gmail.com Wed Jun 4 00:53:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 3 Jun 2008 17:53:37 -0500 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> Message-ID: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller wrote: > Also - we could leave in stubs to match the threading api - Guido, David > Goodger and others really prefer not to continue the "broken" API of the > threading API I agree that the threading the the pyprocessing APIs should be PEP 8 compliant, but I think 2 APIs is almost worse than one wrong one. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Wed Jun 4 01:00:56 2008 From: brett at python.org (Brett Cannon) Date: Tue, 3 Jun 2008 16:00:56 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> Message-ID: On Tue, Jun 3, 2008 at 3:53 PM, Benjamin Peterson wrote: > On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller wrote: >> Also - we could leave in stubs to match the threading api - Guido, David >> Goodger and others really prefer not to continue the "broken" API of the >> threading API > I agree that the threading the the pyprocessing APIs should be PEP 8 > compliant, but I think 2 APIs is almost worse than one wrong one. > I disagree. If you state upfront that one of them is only for backwards-compatibility/transitioning and will be going away in the future, then I think it is fine to have the extra API. -Brett From mike.klaas at gmail.com Wed Jun 4 01:01:49 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Tue, 3 Jun 2008 16:01:49 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> Message-ID: <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com> On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote: > On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller > wrote: >> Also - we could leave in stubs to match the threading api - Guido, >> David >> Goodger and others really prefer not to continue the "broken" API >> of the >> threading API > I agree that the threading the the pyprocessing APIs should be PEP 8 > compliant, but I think 2 APIs is almost worse than one wrong one. A cleaner way to effectuate the transition would be to leave the camelCase API in 2.6 (for both modules), switch to PEP 8 in py3k (for both modules), and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade the 2.6 API with the PEP 8 API. 2to3 would rewrite 'import threading3k' to 'import threading' and everything would work (it would warn about 'import threading' in 2.6 code). -Mike From musiccomposition at gmail.com Wed Jun 4 01:20:48 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 3 Jun 2008 18:20:48 -0500 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> Message-ID: <1afaf6160806031620g42a31199g1d284783a07a21b2@mail.gmail.com> On Tue, Jun 3, 2008 at 6:00 PM, Brett Cannon wrote: > On Tue, Jun 3, 2008 at 3:53 PM, Benjamin Peterson > wrote: >> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller wrote: >>> Also - we could leave in stubs to match the threading api - Guido, David >>> Goodger and others really prefer not to continue the "broken" API of the >>> threading API >> I agree that the threading the the pyprocessing APIs should be PEP 8 >> compliant, but I think 2 APIs is almost worse than one wrong one. >> > > I disagree. If you state upfront that one of them is only for > backwards-compatibility/transitioning and will be going away in the > future, then I think it is fine to have the extra API. In that case, I'm +1 as long as we implement a full DeprecationWarning on one. > > -Brett > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From guido at python.org Wed Jun 4 01:21:10 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Jun 2008 16:21:10 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com> Message-ID: I'm curious why people thing that strict API compatibility is important at all. In my view, having the APIs be similar is really helpful because it helps people quickly understand what you can do with the new module. But I honestly don't expect anyone to take an existing app using threading and turning it into one using multiprocessing by changing a single line. There are too many things that threads let you do that aren't supported by processes (e.g. shared mutable objects protected by locks). IMO the multiprocessing module should only provide a PEP-8-compliant API, and optionally we could start adding such an API to the threading module. Strict compatibility with the Java API really isn't important there either -- that's just how I got started with it, and I gladly admit I went overboard there. --Guido On Tue, Jun 3, 2008 at 4:01 PM, Mike Klaas wrote: > > On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote: > >> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller wrote: >>> >>> Also - we could leave in stubs to match the threading api - Guido, David >>> Goodger and others really prefer not to continue the "broken" API of the >>> threading API >> >> I agree that the threading the the pyprocessing APIs should be PEP 8 >> compliant, but I think 2 APIs is almost worse than one wrong one. > > A cleaner way to effectuate the transition would be to leave the camelCase > API in 2.6 (for both modules), switch to PEP 8 in py3k (for both modules), > and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade the > 2.6 API with the PEP 8 API. > > 2to3 would rewrite 'import threading3k' to 'import threading' and everything > would work (it would warn about 'import threading' in 2.6 code). > > -Mike > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Wed Jun 4 03:48:59 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 3 Jun 2008 18:48:59 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com><79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com><1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com><1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com> Message-ID: <5062908B5F0144528DC9F35A19D425B3@RaymondLaptop1> I think its a small disaster to have the APIs be almost the same but with trivial differences in spelling. PEP 8 is a nice guideline but it seems to have become an end in an of itself. The point of the PEP is to use consistency as a memory cue, but having two sets of method names that are almost parallel but spelled differently is at odds with that goal. Moving both modules to PEP 8 compliance makes more sense, but I would hope that gets left for Py3.0. There's too much existing code, too many existing tutorials, and too many hardcopy books already in Py2.x. Raymond ----- Original Message ----- From: "Guido van Rossum" To: "Mike Klaas" Cc: Sent: Tuesday, June 03, 2008 4:21 PM Subject: Re: [Python-Dev] PEP 371: Additional Discussion I'm curious why people thing that strict API compatibility is important at all. In my view, having the APIs be similar is really helpful because it helps people quickly understand what you can do with the new module. But I honestly don't expect anyone to take an existing app using threading and turning it into one using multiprocessing by changing a single line. There are too many things that threads let you do that aren't supported by processes (e.g. shared mutable objects protected by locks). IMO the multiprocessing module should only provide a PEP-8-compliant API, and optionally we could start adding such an API to the threading module. Strict compatibility with the Java API really isn't important there either -- that's just how I got started with it, and I gladly admit I went overboard there. --Guido On Tue, Jun 3, 2008 at 4:01 PM, Mike Klaas wrote: > > On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote: > >> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller wrote: >>> >>> Also - we could leave in stubs to match the threading api - Guido, David >>> Goodger and others really prefer not to continue the "broken" API of the >>> threading API >> >> I agree that the threading the the pyprocessing APIs should be PEP 8 >> compliant, but I think 2 APIs is almost worse than one wrong one. > > A cleaner way to effectuate the transition would be to leave the camelCase > API in 2.6 (for both modules), switch to PEP 8 in py3k (for both modules), > and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade the > 2.6 API with the PEP 8 API. > > 2to3 would rewrite 'import threading3k' to 'import threading' and everything > would work (it would warn about 'import threading' in 2.6 code). > > -Mike > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com From python at rcn.com Wed Jun 4 03:56:28 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 3 Jun 2008 18:56:28 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com><79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com><1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com> Message-ID: <3DADCFCEEBBB4695A6A50646D1FD15C1@RaymondLaptop1> From: "Mike Klaas" > A cleaner way to effectuate the transition would be to leave > the camelCase API in 2.6 (for both modules), switch to PEP 8 > in py3k (for both modules) +1 That makes good sense. > , and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade the 2.6 API with the PEP 8 API. +0 PEP 8 is nice but it's not that important. Raymond From greg.ewing at canterbury.ac.nz Wed Jun 4 04:28:22 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 04 Jun 2008 14:28:22 +1200 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> Message-ID: <4845FDC6.2050909@canterbury.ac.nz> Benjamin Peterson wrote: > I agree that the threading the the pyprocessing APIs should be PEP 8 > compliant, but I think 2 APIs is almost worse than one wrong one. So change them both to be PEP 8 compliant, and leave aliases in both for existing code to use. -- Greg From guido at python.org Wed Jun 4 06:36:34 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Jun 2008 21:36:34 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <5062908B5F0144528DC9F35A19D425B3@RaymondLaptop1> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com> <5062908B5F0144528DC9F35A19D425B3@RaymondLaptop1> Message-ID: On Tue, Jun 3, 2008 at 6:48 PM, Raymond Hettinger wrote: > I think its a small disaster to have the APIs be almost > the same but with trivial differences in spelling. Strong words -- and it now seems just your gut feelings against mine. I have come to trust mine quite a bit. > PEP 8 is a nice guideline but it seems to have become > an end in an of itself. The point of the PEP is to use > consistency as a memory cue, but having two sets of > method names that are almost parallel but spelled > differently is at odds with that goal. PEP 8 helps by making exact method spellings easier to remember. The recommendation against abbreviations also comes from this goal. It should be used for all new APIs. (Except of course when adding to a set of existing APIs that use a different convention.) I consider multiprocessing a new API -- while it bears a superficial resemblance with the threading API the similarities are just that, and it should not be constrained by mistakes in that API. > Moving both modules to PEP 8 compliance makes more > sense, but I would hope that gets left for Py3.0. There's > too much existing code, too many existing tutorials, and > too many hardcopy books already in Py2.x. It was decided not to "fix" APIs for Py3k beyond the module naming issue. We can add a new PEP-8-compliant API to the threading module in 3.0 but the old API should stay for another release or more. I'm neutral on whether it makes sense to backport the new threading.py APIs to 2.6. --Guido > Raymond > > > ----- Original Message ----- From: "Guido van Rossum" > To: "Mike Klaas" > Cc: > Sent: Tuesday, June 03, 2008 4:21 PM > Subject: Re: [Python-Dev] PEP 371: Additional Discussion > > > I'm curious why people thing that strict API compatibility is > important at all. In my view, having the APIs be similar is really > helpful because it helps people quickly understand what you can do > with the new module. But I honestly don't expect anyone to take an > existing app using threading and turning it into one using > multiprocessing by changing a single line. There are too many things > that threads let you do that aren't supported by processes (e.g. > shared mutable objects protected by locks). > > IMO the multiprocessing module should only provide a PEP-8-compliant > API, and optionally we could start adding such an API to the threading > module. Strict compatibility with the Java API really isn't important > there either -- that's just how I got started with it, and I gladly > admit I went overboard there. > > --Guido > > On Tue, Jun 3, 2008 at 4:01 PM, Mike Klaas wrote: >> >> On 3-Jun-08, at 3:53 PM, Benjamin Peterson wrote: >> >>> On Tue, Jun 3, 2008 at 5:08 PM, Jesse Noller wrote: >>>> >>>> Also - we could leave in stubs to match the threading api - Guido, David >>>> Goodger and others really prefer not to continue the "broken" API of the >>>> threading API >>> >>> I agree that the threading the the pyprocessing APIs should be PEP 8 >>> compliant, but I think 2 APIs is almost worse than one wrong one. >> >> A cleaner way to effectuate the transition would be to leave the camelCase >> API in 2.6 (for both modules), switch to PEP 8 in py3k (for both modules), >> and provide threading3k and multiprocessing3k modules in 2.6 that fa?ade >> the >> 2.6 API with the PEP 8 API. >> >> 2to3 would rewrite 'import threading3k' to 'import threading' and >> everything >> would work (it would warn about 'import threading' in 2.6 code). >> >> -Mike >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python%40rcn.com > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jun 4 06:37:27 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 3 Jun 2008 21:37:27 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <3DADCFCEEBBB4695A6A50646D1FD15C1@RaymondLaptop1> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <79990c6b0806031438g2128d5b5n6f8aea83183bed2b@mail.gmail.com> <1afaf6160806031553n60c7aef7l4d26fd0538cf42b6@mail.gmail.com> <1979FD88-4E65-4D21-818B-E2AFC655B596@gmail.com> <3DADCFCEEBBB4695A6A50646D1FD15C1@RaymondLaptop1> Message-ID: On Tue, Jun 3, 2008 at 6:56 PM, Raymond Hettinger wrote: > From: "Mike Klaas" >> >> A cleaner way to effectuate the transition would be to leave >> the camelCase API in 2.6 (for both modules), switch to PEP 8 >> in py3k (for both modules) > > +1 > That makes good sense. No. It makes *no* sense to introduce a new API in 2.6 and change it again in 3.0. That sounds like another foolish consistency. >> , and provide threading3k and multiprocessing3k modules in 2.6 that >> fa?ade the 2.6 API with the PEP 8 API. > > +0 > PEP 8 is nice but it's not that important. > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Wed Jun 4 10:54:25 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 4 Jun 2008 01:54:25 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC References: Message-ID: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> The only comment so far was to keep the __index__ method. Other than that, is this good to go? Raymond ----- Original Message ----- > Target: Py2.6 and Py3.0 > Author: Raymond Hettinger > Date: May 31, 2008 > > Motivation > ---------- > The principal purpose of an abstract base class is to support multiple > implementations of an API; thereby allowing one concrete class to be > substitutable for another. This purpose is defeated when useful substitutions > are precluded because the ABC is too aggressive in its behavioral requirements > -- mandating too many methods, mandating methods that are difficult to > implement, or too closely following the full API of a single concrete type. > > The Integral ABC is somewhat extensive and requires essentially every behavior > exhibited by the int concrete class. Usefully, it provides for basic integer > behaviors like simple arithmetic and ordering relations. However, the current > ABC goes beyond that and requires bit-flipping and other operations that are > not fundamental to the notion of being an integer. That makes it challenging > to define a new Integral class that isn't already an int. > > Proposal > -------- > Remove non-essential abstract methods like __index__, three argument __pow__, > __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, __xor__, > __rxor__, __or__, __ror__, and __invert__, numerator, and denominator. > > Discussion > ---------- > The only known use cases for variants of int are types that limit the range of > values to those that fit in a fixed storage width. > > One existing implementation is in numpy which has types like int0, int8, > int16, int32, and int16. The numpy integral types implement all the methods > present in Integral except for the new methods like __trunc__, __index__, > real, imag, conjugate, numerator, and denominator. For the most part, they > are fully substitutable into code that expects regular ints; however, they do > have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The > wrap-around behaviors make the numpy types totally unsuitable for some > applications of Integral such as the fractions module which accepts any > integral numerator and denominator. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com From ncoghlan at gmail.com Wed Jun 4 12:23:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Jun 2008 20:23:08 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4845934A.6030708@v.loewis.de> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> Message-ID: <48466D0C.40709@gmail.com> Martin v. L?wis wrote: >>> Please don't - not before % is actually deprecated (which I hope won't >>> happen until Python 4, with removal of % in Python 5, in the year >>> when I retire, i.e. 2037). >> Now this is news to me -- was there a discussion that changed the >> lifetime expectancy of str.__mod__? I'd always supposed it being >> deprecated at some point in 3.x. > > The PEP doesn't specify anything, and I don't recall any discussion, > either - the specific timing suggested above is merely my own hopes. While str.format has an awful lot of points in its favour (no tuple/dict special casing, much cleaner handling of named arguments, access to format options for non-builtin types), there are some debugging cases where I suspect the basic version of % formatting will prove to be more convenient. At this point in time, my personal preference would be that later in the 3.x series certain aspects of % formatting will be deprecated (acceptance of non-tuples in favour of the format() builtin, acceptance of dicts in favour of str.format), but that simple % formatting of a tuple of values will still be permitted. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Jun 4 12:33:07 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Jun 2008 20:33:07 +1000 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> Message-ID: <48466F63.3020902@gmail.com> Jesse Noller wrote: > Per feedback from Guido, the python-dev list and others, I have sent > in an updated version of PEP 371 - the inclusion of the pyprocessing > module into the standard library. > > URL: http://www.python.org/dev/peps/pep-0371/ > > New highlights: > * The module will be renamed to "multiprocessing" > * The API will become PEP 8 compliant My 2 cents on this: PEP 8 compliance in the multiprocessing API is a good idea, but threading should be updated at the same time to provide PEP 8 compliant aliases for the existing names. Using the old Java-based names should provoke a Py3k warning in 2.6 (note: to avoid a runtime performance hit when not using -3, the actual function definitions should be made conditional, rather than checking whether or not to emit the warning every time the legacy API is invoked) The PyGILState_* bug in debug builds that RMO pointed out should also be mentioned in the PEP as something that needs to be fixed in order to implement the PEP. Cheers, Nick. [1] http://bugs.python.org/issue1683 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fuzzyman at voidspace.org.uk Wed Jun 4 13:36:48 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 04 Jun 2008 12:36:48 +0100 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <48466D0C.40709@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> Message-ID: <48467E50.9030202@voidspace.org.uk> Nick Coghlan wrote: > Martin v. L?wis wrote: >>>> Please don't - not before % is actually deprecated (which I hope won't >>>> happen until Python 4, with removal of % in Python 5, in the year >>>> when I retire, i.e. 2037). >>> Now this is news to me -- was there a discussion that changed the >>> lifetime expectancy of str.__mod__? I'd always supposed it being >>> deprecated at some point in 3.x. >> >> The PEP doesn't specify anything, and I don't recall any discussion, >> either - the specific timing suggested above is merely my own hopes. > > While str.format has an awful lot of points in its favour (no > tuple/dict special casing, much cleaner handling of named arguments, > access to format options for non-builtin types), there are some > debugging cases where I suspect the basic version of % formatting will > prove to be more convenient. > > At this point in time, my personal preference would be that later in > the 3.x series certain aspects of % formatting will be deprecated > (acceptance of non-tuples in favour of the format() builtin, > acceptance of dicts in favour of str.format), but that simple % > formatting of a tuple of values will still be permitted. > +1 Simple string formatting with %s and a single object or a tuple meets >90% of my string formatting needs. Michael Foord > Cheers, > Nick. > From solipsis at pitrou.net Wed Jun 4 14:44:10 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 4 Jun 2008 12:44:10 +0000 (UTC) Subject: [Python-Dev] converting the stdlib to str.format References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> Message-ID: Michael Foord voidspace.org.uk> writes: > Simple string formatting with %s and a single object or a tuple meets > >90% of my string formatting needs. Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s) (if I got the format spec right). cheers Antoine. From p.f.moore at gmail.com Wed Jun 4 14:50:48 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 4 Jun 2008 13:50:48 +0100 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> Message-ID: <79990c6b0806040550w1550194crbe2dc4d16ed45c39@mail.gmail.com> 2008/6/4 Antoine Pitrou : > Michael Foord voidspace.org.uk> writes: >> Simple string formatting with %s and a single object or a tuple meets >> >90% of my string formatting needs. > > Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s) > (if I got the format spec right). repr(s) is simpler still, and doesn't break if s is a tuple :-) Paul. From ncoghlan at gmail.com Wed Jun 4 15:28:40 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Jun 2008 23:28:40 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> Message-ID: <48469888.4070603@gmail.com> Antoine Pitrou wrote: > Michael Foord voidspace.org.uk> writes: >> Simple string formatting with %s and a single object or a tuple meets >> >90% of my string formatting needs. > > Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s) > (if I got the format spec right). repr(s) is even simpler :) The basic idea for conversions to the new formatting: '%r' % v --> repr(v) # or ascii(v) if more appropriate '%s' % v --> str(v) # or format(v), or format(v, '') The str/unicode stability that the latter provided in 2.x is no longer needed in 3.x. The first one never had any advantage that I can see over invoking repr() directly (since repr() and %r always produce an 8-bit string in 2.x) The conversion of fmt % v depends on both fmt and v: v is a single value, fmt is only a formatting code: '%x' % v--> format(v, 'x') v is a single value, fmt is both a formatting code and additional text: 'Value: %x' % v--> 'Value: {0:x}'.format(v) Note that the old code in this case wouldn't display dict or tuple instances correctly. Avoiding that ambiguity is a major advantage of the new approach. v is a tuple* (more on this below): 'Key: %s Value: %s' % k, v --> 'Key: {0} Value: {1}'.format(k, v) fmt uses named parameters: 'Value: %(val)s' % dict(val=v) --> 'Value: {val}'.format(val=v) * I still think the new str.format approach is too verbose and requires too much thought for simple use cases where the printf-style % replacement hits a sweet spot between being easy to write and easy to read. So I see the introduction of str.format in 3.0 as an opportunity to *fix* %-formatting later in the 3.x series rather than get rid of it entirely. The fixes I would apply: - remove support for %(name)s based formatting (str.format is vastly superior once you start naming the parameters). - remove support for passing a single value to a format string without wrapping it in an iterable first - accept any iterable as the right hand argument str.__mod__ This approach would eliminate the current ambiguity in fmt.v, and would allow fmt % [x] to be used to box an argument for safe use in a single-parameter format string instead of having to muck around with singleton tuples. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at pearwood.info Wed Jun 4 16:19:10 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 5 Jun 2008 00:19:10 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <48469888.4070603@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <48469888.4070603@gmail.com> Message-ID: <200806050019.11367.steve@pearwood.info> On Wed, 4 Jun 2008 11:28:40 pm Nick Coghlan wrote: > > Not to mention that e.g. "%r" % s is much simpler than > > "{0!r}".format(s) (if I got the format spec right). > > repr(s) is even simpler :) Fair enough, and I see your smiley, but consider: "X %r X" % s "X {0!r} X".format(s) "X " + repr(s) + " X" I for one am excited to see advanced string formatting via .format(), but would be very disappointed if future Python versions removed the simplicity of %. -- Steven From solipsis at pitrou.net Wed Jun 4 16:08:20 2008 From: solipsis at pitrou.net (Antoine) Date: Wed, 4 Jun 2008 16:08:20 +0200 (CEST) Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <48469888.4070603@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> Message-ID: <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> > Antoine Pitrou wrote: >> >> Not to mention that e.g. "%r" % s is much simpler than "{0!r}".format(s) >> (if I got the format spec right). > > repr(s) is even simpler :) Yes, of course, but in the non-trivial case, e.g. "value=%r" % my_value, it's much easier to use %-style formatting than playing with repr() and the other string formatting facilities. > Note that the old code in this case wouldn't display dict or tuple > instances correctly. Avoiding that ambiguity is a major advantage of the > new approach. I know. For me the problem is not about ditching the % operator for an intuitively-named method like format(). It's the format syntax which has become much more complicated and error-prone without any clear advantage. Regards Antoine. From jyasskin at gmail.com Wed Jun 4 17:05:09 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Wed, 4 Jun 2008 08:05:09 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> Message-ID: <5d44f72f0806040805l5cd8f2dbn1fcaa5e9b29ad793@mail.gmail.com> Because .numerator and .denominator are defined by the Rational ABC, and Integral inherits from that, removing them from Integral is a larger change than removing the other methods. You'd have to remove them from Rational or break the inheritance relation. Removing the bitwise operators sounds fine to me. On Wed, Jun 4, 2008 at 1:54 AM, Raymond Hettinger wrote: > The only comment so far was to keep the __index__ method. > > Other than that, is this good to go? > > Raymond > > > ----- Original Message ----- >> >> Target: Py2.6 and Py3.0 >> Author: Raymond Hettinger >> Date: May 31, 2008 >> >> Motivation >> ---------- >> The principal purpose of an abstract base class is to support multiple >> implementations of an API; thereby allowing one concrete class to be >> substitutable for another. This purpose is defeated when useful >> substitutions >> are precluded because the ABC is too aggressive in its behavioral >> requirements >> -- mandating too many methods, mandating methods that are difficult to >> implement, or too closely following the full API of a single concrete >> type. >> >> The Integral ABC is somewhat extensive and requires essentially every >> behavior >> exhibited by the int concrete class. Usefully, it provides for basic >> integer >> behaviors like simple arithmetic and ordering relations. However, the >> current >> ABC goes beyond that and requires bit-flipping and other operations that >> are >> not fundamental to the notion of being an integer. That makes it >> challenging >> to define a new Integral class that isn't already an int. >> >> Proposal >> -------- >> Remove non-essential abstract methods like __index__, three argument >> __pow__, >> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, >> __xor__, >> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator. >> >> Discussion >> ---------- >> The only known use cases for variants of int are types that limit the >> range of >> values to those that fit in a fixed storage width. >> >> One existing implementation is in numpy which has types like int0, int8, >> int16, int32, and int16. The numpy integral types implement all the >> methods >> present in Integral except for the new methods like __trunc__, __index__, >> real, imag, conjugate, numerator, and denominator. For the most part, >> they >> are fully substitutable into code that expects regular ints; however, they >> do >> have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The >> wrap-around behaviors make the numpy types totally unsuitable for some >> applications of Integral such as the fractions module which accepts any >> integral numerator and denominator. >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/python%40rcn.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com > -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ From guido at python.org Wed Jun 4 17:33:48 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Jun 2008 08:33:48 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> Message-ID: I haven't seen anyone applaud either. Unless more folks actually say they agree I don't want to go forward with this. There was quite a bit of discussion about PEP 3141 and it was accepted; striking this much from it with virtually no discussion seems wrong to me. Jeffrey made a good point: .numerator and .denominator need to be kept in the interface. I really don't want to divorce Integer from Rational. They're trivial to implement, and I won't complain if you leave them unimplemented while claiming conformance. :-) My position: I'm -1 on removing __index__, numerator, denominator, and on removing anything you didn't mention explicitly. You used the phrase "methods like", which seems inappropriate for a specification. Note also that these happen to be concrete methods, not abstract ones like you claim. I'm -0 on removing the bitwise operators, i.e. could be swayed by some +1 votes. --Guido On Wed, Jun 4, 2008 at 1:54 AM, Raymond Hettinger wrote: > The only comment so far was to keep the __index__ method. > > Other than that, is this good to go? > > Raymond > > > ----- Original Message ----- >> >> Target: Py2.6 and Py3.0 >> Author: Raymond Hettinger >> Date: May 31, 2008 >> >> Motivation >> ---------- >> The principal purpose of an abstract base class is to support multiple >> implementations of an API; thereby allowing one concrete class to be >> substitutable for another. This purpose is defeated when useful >> substitutions >> are precluded because the ABC is too aggressive in its behavioral >> requirements >> -- mandating too many methods, mandating methods that are difficult to >> implement, or too closely following the full API of a single concrete >> type. >> >> The Integral ABC is somewhat extensive and requires essentially every >> behavior >> exhibited by the int concrete class. Usefully, it provides for basic >> integer >> behaviors like simple arithmetic and ordering relations. However, the >> current >> ABC goes beyond that and requires bit-flipping and other operations that >> are >> not fundamental to the notion of being an integer. That makes it >> challenging >> to define a new Integral class that isn't already an int. >> >> Proposal >> -------- >> Remove non-essential abstract methods like __index__, three argument >> __pow__, >> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, >> __xor__, >> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator. >> >> Discussion >> ---------- >> The only known use cases for variants of int are types that limit the >> range of >> values to those that fit in a fixed storage width. >> >> One existing implementation is in numpy which has types like int0, int8, >> int16, int32, and int16. The numpy integral types implement all the >> methods >> present in Integral except for the new methods like __trunc__, __index__, >> real, imag, conjugate, numerator, and denominator. For the most part, >> they >> are fully substitutable into code that expects regular ints; however, they >> do >> have wrap-around behaviors such as int8(200)+int8(100) --> int8(44). The >> wrap-around behaviors make the numpy types totally unsuitable for some >> applications of Integral such as the fractions module which accepts any >> integral numerator and denominator. >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/python%40rcn.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jun 4 19:42:54 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Jun 2008 10:42:54 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <48466F63.3020902@gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> Message-ID: Sounds good. (Maybe you want to contribute a patch to threading.py? Your implementation notes are important. It could be quite independent from PEP 371.) On Wed, Jun 4, 2008 at 3:33 AM, Nick Coghlan wrote: > Jesse Noller wrote: >> >> Per feedback from Guido, the python-dev list and others, I have sent >> in an updated version of PEP 371 - the inclusion of the pyprocessing >> module into the standard library. >> >> URL: http://www.python.org/dev/peps/pep-0371/ >> >> New highlights: >> * The module will be renamed to "multiprocessing" >> * The API will become PEP 8 compliant > > My 2 cents on this: PEP 8 compliance in the multiprocessing API is a good > idea, but threading should be updated at the same time to provide PEP 8 > compliant aliases for the existing names. Using the old Java-based names > should provoke a Py3k warning in 2.6 (note: to avoid a runtime performance > hit when not using -3, the actual function definitions should be made > conditional, rather than checking whether or not to emit the warning every > time the legacy API is invoked) > > The PyGILState_* bug in debug builds that RMO pointed out should also be > mentioned in the PEP as something that needs to be fixed in order to > implement the PEP. > > Cheers, > Nick. > > [1] http://bugs.python.org/issue1683 > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Wed Jun 4 21:57:11 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 4 Jun 2008 12:57:11 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> Message-ID: <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> From: "Guido van Rossum" > Unless more folks actually say they agree I don't want to go forward > with this. There was quite a bit of discussion about PEP 3141 and it > was accepted; striking this much from it with virtually no discussion > seems wrong to me. Not sure how to generate more discussion. It seems self-evident that an abc with lots of abstract methods is inherently less usable and that bitwise operations go beyond the basic notion of "integeriness". Requiring all those methods to be defined makes it harder to write a compliant class. Other than int/long, no currently existing type matches-up with Integral. Does anyone care about this? If something like numpy's int8 eventually grows the required methods, it is worriesome that the numerator/denominator properties will be automatically supplied for fractions.py to accept as inputs eventhough int8's wrap-around behavior makes them entirely unsuitable. I don't know if this bothers anyone. It would seem that a consumer of an Integral can assume the existence of methods but nothing about whether the result is usable. That might not be a big deal except that numpy is t he only known use case. Hopefully, some discussion gets generated. But if no one cares, I'll happily drop it. Raymond From python at rcn.com Wed Jun 4 22:28:10 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 4 Jun 2008 13:28:10 -0700 Subject: [Python-Dev] converting the stdlib to str.format References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><48469888.4070603@gmail.com> <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> Message-ID: <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> From: "Antoine" For me the problem is not about ditching the % operator for an > intuitively-named method like format(). It's the format syntax which has > become much more complicated and error-prone without any clear advantage. It's seems that way to me too. But, it may be one of those things that you quickly get used to. Raymond From guido at python.org Wed Jun 4 22:50:37 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Jun 2008 13:50:37 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> Message-ID: On Wed, Jun 4, 2008 at 12:57 PM, Raymond Hettinger wrote: > From: "Guido van Rossum" >> >> Unless more folks actually say they agree I don't want to go forward >> with this. There was quite a bit of discussion about PEP 3141 and it >> was accepted; striking this much from it with virtually no discussion >> seems wrong to me. > > Not sure how to generate more discussion. It seems self-evident > that an abc with lots of abstract methods is inherently less usable > and that bitwise operations go beyond the basic notion of "integeriness". > Requiring all those methods to be defined makes it harder to write > a compliant class. In general it is good to require that more thought goes into the design and implementation of a class, so that less thought needs to go into using it. > Other than int/long, no currently existing type > matches-up with Integral. Does anyone care about this? Does "this" refer to "the Integral ABC" or "no type matches up with it" ? > If something like numpy's int8 eventually grows the required methods, > it is worrysome that the numerator/denominator properties will be > automatically supplied for fractions.py to accept as inputs eventhough > int8's wrap-around behavior makes them entirely unsuitable. I'm not sure what you mean. This is probably just my lack of imagination. Can you give a small code example where using an int8 with the fractions module would cause problems? The Fraction class appears to be calling __index__ on its arguments, which would convert the int8 to a proper int (or long, in Python 2.6). > I don't > know if this bothers anyone. It would seem that a consumer of an Integral > can assume the existence of methods but nothing about whether the > result is usable. That might not be a big deal except that numpy is t > he only known use case. Any integer type that performs arithmetic modulo some number is problematic. Perhaps a totally separate ABC needs to be defined for this purpose, one that doesn't inherit from Rational. > Hopefully, some discussion gets generated. But if no one cares, I'll > happily drop it. Have you asked the numpy folks? If enough people care, we could easily create a BinaryInteger ABC that defines the bitwise operations. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jun 4 22:52:14 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Jun 2008 13:52:14 -0700 Subject: [Python-Dev] Mini-Pep: An Empty String ABC In-Reply-To: <0D0F7B10924D4B19BB45DDCC121A7435@RaymondLaptop1> References: <07CDACB613B34584ADE74568D50ECAF6@RaymondLaptop1> <0D0F7B10924D4B19BB45DDCC121A7435@RaymondLaptop1> Message-ID: Given the lack of responses to my questions, let's reject this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amcnabb at mcnabbs.org Thu Jun 5 00:24:17 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Wed, 4 Jun 2008 16:24:17 -0600 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> Message-ID: <20080604222416.GD26992@mcnabbs.org> On Wed, Jun 4, 2008 at 12:57 PM, Raymond Hettinger wrote: > > Not sure how to generate more discussion. It seems self-evident > that an abc with lots of abstract methods is inherently less usable > and that bitwise operations go beyond the basic notion of "integeriness". > Requiring all those methods to be defined makes it harder to write > a compliant class. > > Other than int/long, no currently existing type > matches-up with Integral. Does anyone care about this? > > Hopefully, some discussion gets generated. But if no one cares, I'll > happily drop it. I'm just responding out of duty to add more discussion on the topic. I think I agree with Raymond on the basic principle that simple ABC's are easier to use than simple ones. Anyway, it's hard to add anything to the discussion when I haven't seen many applications of the Integral ABC, but in principle, but if I had to vote I'd say that the bitwise operators aren't particularly integery. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From henning.vonbargen at arcor.de Wed Jun 4 21:37:22 2008 From: henning.vonbargen at arcor.de (Henning von Bargen) Date: Wed, 4 Jun 2008 21:37:22 +0200 Subject: [Python-Dev] converting the stdlib to str.format References: Message-ID: <002801c8c67a$68aa7350$6401a8c0@max> Please, please, please: Keep the % formatting! In 99% of the Python code I have seen, the str.format wouldn't gain any advantage (not even regarding the code readability). Yes, there may be special cases where str.format offers more flexibility, but until now I never missed anything in the % formatting. One thing to keep in mind: What if you have a lot of %-format strings translated and stored them inside a database, for example? If % would be deprecated, these database entries would have to be converted, so you would have to write yet another program to do this. So if the % formating really is going to vanish, then please not before April 2099 (or when Duke Nukem Forever will be released ;-). Just my 0,02 EUR Henning From tjreedy at udel.edu Thu Jun 5 02:59:47 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 4 Jun 2008 20:59:47 -0400 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> Message-ID: "Raymond Hettinger" wrote in message news:2745D7EB7C064B16A88E433588021756 at RaymondLaptop1... | From: "Guido van Rossum" | > Unless more folks actually say they agree I don't want to go forward | > with this. There was quite a bit of discussion about PEP 3141 and it | > was accepted; striking this much from it with virtually no discussion | > seems wrong to me. | | Not sure how to generate more discussion. It seems self-evident | that an abc with lots of abstract methods is inherently less usable | and that bitwise operations go beyond the basic notion of "integeriness". On reading PEP3141 some months ago and again today, I thought and still do that all the methods that depend on a 2s-complement representation and implementation really belong to an implentation-defined subclass of Integral. But I am not sure of the purpose of the class and of including such concrete methods in an ABC, and so said nothing ;-). I notice that the similarly implementation-based frexp and ldexp math methods are *not* in the Real class. | Requiring all those methods to be defined makes it harder to write | a compliant class. Other than int/long, no currently existing type | matches-up with Integral. Does anyone care about this? I might some day, for didactic purposes, write integer classes based on prime-factorization or fibonacci representations. I certainly would skip all the 2s-complement methods. But I do not know that I would care much about being 'non-compliant'. | If something like numpy's int8 eventually grows the required methods, | it is worriesome that the numerator/denominator properties will be | automatically supplied for fractions.py to accept as inputs eventhough | int8's wrap-around behavior makes them entirely unsuitable. In my view, the members of residue/remainder classes are not really integers unless their usage is carefully restricted to avoid invocation of the mod operation. Terry Jan Reedy From greg.ewing at canterbury.ac.nz Thu Jun 5 03:48:07 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 05 Jun 2008 13:48:07 +1200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <48469888.4070603@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> Message-ID: <484745D7.3000808@canterbury.ac.nz> Nick Coghlan wrote: > - remove support for passing a single value to a format string without > wrapping it in an iterable first But won't that clobber a large number of the simple use cases that you want to keep %-formatting for? -- Greg From steve at pearwood.info Thu Jun 5 04:30:33 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 5 Jun 2008 12:30:33 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <484745D7.3000808@canterbury.ac.nz> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> Message-ID: <200806051230.33243.steve@pearwood.info> On Thu, 5 Jun 2008 11:48:07 am Greg Ewing wrote: > Nick Coghlan wrote: > > - remove support for passing a single value to a format string > > without wrapping it in an iterable first > > But won't that clobber a large number of the simple > use cases that you want to keep %-formatting for? Yes, I don't particularly see the advantage of writing: "spam %s spam" % (value,) over "spam %s spam" % value Why require the first version? -- Steven From andymac at bullseye.apana.org.au Thu Jun 5 04:32:48 2008 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Thu, 05 Jun 2008 13:32:48 +1100 Subject: [Python-Dev] wrt the beta deadline and freelist management In-Reply-To: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> Message-ID: <48475050.9090602@bullseye.andymac.org> There are 2 disparate approaches to clearing/compacting free lists for basic types: - APIs of the form Py_ClearFreeList() called from gc.collect() [class, frame, method, tuple & unicode types]; - APIs of the form Py_CompactFreeList() called from sys._compact_freelists() [int & float types]; Both approaches are new for 2.6 & 3.0. The patch at http://bugs.python.org/issue2862 is geared towards bring the int/float free list management into line with the others. The patch at http://bugs.python.org/issue3029 adds free list management to the dict, list & set types. The value of this is probably minor, and this patch is thus not significant in its own right other than for consistency. However Raymond's comment to issue 3029 that the management routines shouldn't be public APIs is IMO important (& I happen to agree). It would be nice to reduce the 2 approaches to one. I would rather the gc.collect() approach, as this seems to be more obvious to many users who have gone looking for free list management in prior versions, however my opinion isn't particularly valuable on this. Can this be resolved for 2.6/3.0? Regards, Andrew. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From dickinsm at gmail.com Thu Jun 5 05:44:09 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 4 Jun 2008 23:44:09 -0400 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: Message-ID: <5c6f2a5d0806042044p73cd93c2j39b37f214fa91c05@mail.gmail.com> On Sun, Jun 1, 2008 at 2:15 AM, Raymond Hettinger wrote: > Proposal > -------- > Remove non-essential abstract methods like __index__, three argument > __pow__, > __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, > __xor__, > __rxor__, __or__, __ror__, and __invert__, numerator, and denominator. > +1 from me. I'd support removing all these, minus the exceptions already pointed out (__index__, numerator, denominator). As a (so far incomplete) effort to speed up the Decimal type I recently implemented a decimal-based integer type; this type would seem a natural candidate to inherit from Integral, but the logical and shift operators above make less sense for this type. The other odd man out here is three-argument pow; this *is* a method that makes sense for integers without reference to the way they're stored. So maybe this should stay. (Though I've occasionally wondered why three-argument pow is part of the core language, rather than being in the standard library.) Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Jun 5 06:13:34 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Jun 2008 21:13:34 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <5c6f2a5d0806042044p73cd93c2j39b37f214fa91c05@mail.gmail.com> References: <5c6f2a5d0806042044p73cd93c2j39b37f214fa91c05@mail.gmail.com> Message-ID: I think it's fine to remove 3-arg pow() from the ABC; implementations are of course free to provide it. Raymond, why don't you cook up a patch? It should patch both the PEP and numbers.py. On Wed, Jun 4, 2008 at 8:44 PM, Mark Dickinson wrote: > On Sun, Jun 1, 2008 at 2:15 AM, Raymond Hettinger wrote: >> >> Proposal >> -------- >> Remove non-essential abstract methods like __index__, three argument >> __pow__, >> __lshift__, __rlshift__, __rshift__, __rrshift__, __and__, __rand__, >> __xor__, >> __rxor__, __or__, __ror__, and __invert__, numerator, and denominator. > > +1 from me. > I'd support removing all these, minus the exceptions already pointed out > (__index__, numerator, denominator). As a (so far incomplete) effort > to speed up the Decimal type I recently implemented a decimal-based > integer type; this type would seem a natural candidate to inherit > from Integral, but the logical and shift operators above make less sense > for this type. > The other odd man out here is three-argument pow; this *is* a > method that makes sense for integers without reference to the > way they're stored. So maybe this should stay. (Though I've > occasionally wondered why three-argument pow is part of the > core language, rather than being in the standard library.) > Mark > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From p.f.moore at gmail.com Thu Jun 5 12:50:32 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 5 Jun 2008 11:50:32 +0100 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <200806051230.33243.steve@pearwood.info> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> <200806051230.33243.steve@pearwood.info> Message-ID: <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com> On 05/06/2008, Steven D'Aprano wrote: > Yes, I don't particularly see the advantage of writing: > > "spam %s spam" % (value,) > > over > > "spam %s spam" % value > > Why require the first version? Because the second breaks if value is a tuple: Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> value = (1,2) >>> "spam %s spam" % value Traceback (most recent call last): File "", line 1, in TypeError: not all arguments converted during string formatting Paul. From ncoghlan at gmail.com Thu Jun 5 14:19:44 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Jun 2008 22:19:44 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <484745D7.3000808@canterbury.ac.nz> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> Message-ID: <4847D9E0.4020805@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: > >> - remove support for passing a single value to a format string without >> wrapping it in an iterable first > > But won't that clobber a large number of the simple > use cases that you want to keep %-formatting for? Note the part of the proposal that allows *any* iterable on the right hand side rather than the current insistence on a tuple. So the single-value use cases can be boxed easily with a list. To my mind salvaging %-formatting requires removing the ambiguity over whether or not the right hand side will be iterated over. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fuzzyman at voidspace.org.uk Thu Jun 5 14:29:21 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 05 Jun 2008 13:29:21 +0100 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4847D9E0.4020805@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> <4847D9E0.4020805@gmail.com> Message-ID: <4847DC21.8030508@voidspace.org.uk> Nick Coghlan wrote: > Greg Ewing wrote: >> Nick Coghlan wrote: >> >>> - remove support for passing a single value to a format string >>> without wrapping it in an iterable first >> >> But won't that clobber a large number of the simple >> use cases that you want to keep %-formatting for? > > Note the part of the proposal that allows *any* iterable on the right > hand side rather than the current insistence on a tuple. So the > single-value use cases can be boxed easily with a list. > > To my mind salvaging %-formatting requires removing the ambiguity over > whether or not the right hand side will be iterated over. > But that either needs doing for 3.0 or waiting until 4.0 right? Personally I only *occasionally* find the tuple interpolation a problem and am perfectly happy with the current % string formatting... Michael Foord > Cheers, > Nick. > From ncoghlan at gmail.com Thu Jun 5 14:43:20 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Jun 2008 22:43:20 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><48469888.4070603@gmail.com> <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> Message-ID: <4847DF68.1090205@gmail.com> Raymond Hettinger wrote: > From: "Antoine" > For me the problem is not about ditching the % operator for an >> intuitively-named method like format(). It's the format syntax which has >> become much more complicated and error-prone without any clear advantage. > > It's seems that way to me too. But, it may be one of those things that > you quickly get used to. %r is about the worst case for the new syntax relative to the old - two characters become 5. It's worth looking at what those extra characters buy us though: {0!r} {}: Conversion to a bracketed format is what allows us the flexibility to permit arbitrary format strings (such as datetime formatting), as well as the 0: Explicit positional argument references allow arguments to be re-used (not quite sold on this one personally - surely named arguments are even better for that?) !: Explicit separators (: or !) allow the option of flexible object-controlled formatting, while still permitting the basic formatting of str/repr/ascii if desired. I'm really starting to wonder if supporting positional arguments to str.format() *at all* is a mistake. Maybe we should ditch support for positional arguments and just accept a single dictionary as the sole parameter to format(). For dictionary formatting, str.format() is a clear winner over str.__mod__(). For positional formatting I'm not so sure - if someone decided to convert from %-formatting to str.format, would it be such a burden to ask them to name their substitution variables in the process? Silly example: "%s occurs %s times in this format string" % (2, 2) "{0} occurs {0} times in this format string".format(2) "{num} occurs {num} times in this format string".format(dict(num=2)) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 5 14:59:38 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Jun 2008 22:59:38 +1000 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> Message-ID: <4847E33A.1050806@gmail.com> Guido van Rossum wrote: > Sounds good. (Maybe you want to contribute a patch to threading.py? > Your implementation notes are important. It could be quite independent > from PEP 371.) I created issue 3042 as an RFE to add PEP 8 compliant aliases to the threading module (including the notes about how to avoid the runtime performance hit on the old names). Jesse may want to reference that from the PEP as well. I'm not sure I'll be able to get to it for the first beta though - I'm about to go away for a few days, and plan to enjoy my computer-free holiday :) Cheers, Nick. http://bugs.python.org/issue3042 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jnoller at gmail.com Thu Jun 5 15:15:51 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 5 Jun 2008 09:15:51 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4847E33A.1050806@gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> Message-ID: <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> On Thu, Jun 5, 2008 at 8:59 AM, Nick Coghlan wrote: > Guido van Rossum wrote: >> >> Sounds good. (Maybe you want to contribute a patch to threading.py? >> Your implementation notes are important. It could be quite independent >> from PEP 371.) > > I created issue 3042 as an RFE to add PEP 8 compliant aliases to the > threading module (including the notes about how to avoid the runtime > performance hit on the old names). Jesse may want to reference that from the > PEP as well. > > I'm not sure I'll be able to get to it for the first beta though - I'm about > to go away for a few days, and plan to enjoy my computer-free holiday :) > > Cheers, > Nick. > > http://bugs.python.org/issue3042 I'll add it to the PEP. I had to take a day to do "real work" so I can circle back to this today. The continuing outstanding question is if we should put Processing into 2.6 with the threading-like API or PEP 8 compliant names. Richard has already converted the package to PEP 8 style naming, which means I'll need to add aliases to for the original API. Ideally, both threading and processing would loose the non-PEP 8 APIs in py3k or 2.7 Before I go back to the PEP though - I'd like to see if we can reach some consensus on the API naming. My personal thought is that for many tasks, the processing module *is* a drop-in replacement (I had to do this very thing yesterday) which means that putting it in with an API which matches the current threading module's API is a Good Thing. However, the flip side of this is that no one really "likes" the threading API as-is, so putting the module into the standard library with the matching API simply adds another "broken" API. Other than the API naming - I don't think there are any other issues which need to be addressed, minus some cleanup within the PEP for consistency. -Jesse From ncoghlan at gmail.com Thu Jun 5 15:20:40 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Jun 2008 23:20:40 +1000 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> Message-ID: <4847E828.9060001@gmail.com> Terry Reedy wrote: > "Raymond Hettinger" wrote in message > news:2745D7EB7C064B16A88E433588021756 at RaymondLaptop1... > | From: "Guido van Rossum" > | > Unless more folks actually say they agree I don't want to go forward > | > with this. There was quite a bit of discussion about PEP 3141 and it > | > was accepted; striking this much from it with virtually no discussion > | > seems wrong to me. > | > | Not sure how to generate more discussion. It seems self-evident > | that an abc with lots of abstract methods is inherently less usable > | and that bitwise operations go beyond the basic notion of "integeriness". > > On reading PEP3141 some months ago and again today, I thought and still do > that all the methods that depend on a 2s-complement representation and > implementation really belong to an implentation-defined subclass of > Integral. But I am not sure of the purpose of the class and of including > such concrete methods in an ABC, and so said nothing ;-). I think it definitely makes sense to separate out the number-as-sequence-of-bits operations from the main Integral ABC. This would involve moving: lshift, rshift, and, or, xor, invert (along with their reversed and in-place counterparts) Note that this leaves the Integral ABC adding only __long__, __index__ and 3-argument __pow__ over and above the Rational ABC. If 3-argument __pow__ goes (which appears likely), we're left with __long__ and __index__. However, there's still a few additional public properties and methods inherited from higher up in the numeric stack which most existing integral types are unlikely to provide: .real, .imag, .conjugate(). Unlike the methods being relocated, however, these are absolutely trivial for all non-complex numeric types. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 5 15:53:51 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 05 Jun 2008 23:53:51 +1000 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> Message-ID: <4847EFEF.6040302@gmail.com> Jesse Noller wrote: > However, the flip side of this is that no one really "likes" the > threading API as-is, so putting the module into the standard library > with the matching API simply adds another "broken" API. Provided threading gets a PEP 8 style API in 2.6 (which it looks like it is going to), then I don't see a problem with multiprocessing only providing the new API. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at pearwood.info Thu Jun 5 16:05:57 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 6 Jun 2008 00:05:57 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <200806051230.33243.steve@pearwood.info> <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com> Message-ID: <200806060005.58301.steve@pearwood.info> On Thu, 5 Jun 2008 08:50:32 pm Paul Moore wrote: > On 05/06/2008, Steven D'Aprano wrote: > > Yes, I don't particularly see the advantage of writing: > > > > "spam %s spam" % (value,) > > > > over > > > > "spam %s spam" % value > > > > Why require the first version? > > Because the second breaks if value is a tuple: That's a good reason for not using the second form if value is an arbitrary object that could be any type at all. But how often does that happen in real code? It's rare that the right hand argument can be any arbitrary type. I've looked at my code, and there's not one occasion that I've needed to write "foo %s" % (obj,) to guard against obj unexpectedly being a tuple. If obj was a tuple, it would fail long before it reached the string expression. Of course, your mileage may vary. As I see it, your argument is one for consistency: since *sometimes* you need to wrap the right-hand argument in a tuple, then we should insist in *always* wrapping it in a tuple. But of course you can do that right now: >>> n = 1 >>> "number %d" % (n,) # just in case n is a tuple 'number 1' Just don't force me to do it. -- Steven From solipsis at pitrou.net Thu Jun 5 16:49:35 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 5 Jun 2008 14:49:35 +0000 (UTC) Subject: [Python-Dev] converting the stdlib to str.format References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><48469888.4070603@gmail.com> <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> <4847DF68.1090205@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > > %r is about the worst case for the new syntax relative to the old - two > characters become 5. Well there are other very common "worst cases" - namely %d, %f (and probably a few less commonly used ones such as %X). > For dictionary formatting, str.format() is a clear winner over > str.__mod__(). Agreed. %(something)s is quirky - you are bound to forget the final "s" (or whatever other specifier) from time to time. Antoine. From guido at python.org Thu Jun 5 16:54:41 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Jun 2008 07:54:41 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <4847E828.9060001@gmail.com> References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> Message-ID: On Thu, Jun 5, 2008 at 6:20 AM, Nick Coghlan wrote: > Terry Reedy wrote: >> >> "Raymond Hettinger" wrote in message >> news:2745D7EB7C064B16A88E433588021756 at RaymondLaptop1... >> | From: "Guido van Rossum" >> | > Unless more folks actually say they agree I don't want to go forward >> | > with this. There was quite a bit of discussion about PEP 3141 and it >> | > was accepted; striking this much from it with virtually no discussion >> | > seems wrong to me. >> | >> | Not sure how to generate more discussion. It seems self-evident >> | that an abc with lots of abstract methods is inherently less usable >> | and that bitwise operations go beyond the basic notion of >> "integeriness". >> >> On reading PEP3141 some months ago and again today, I thought and still do >> that all the methods that depend on a 2s-complement representation and >> implementation really belong to an implentation-defined subclass of >> Integral. But I am not sure of the purpose of the class and of including >> such concrete methods in an ABC, and so said nothing ;-). > > I think it definitely makes sense to separate out the > number-as-sequence-of-bits operations from the main Integral ABC. This would > involve moving: > > lshift, rshift, and, or, xor, invert (along with their reversed and in-place > counterparts) Agreed. Let's move these into a separate BinaryInteger class. > Note that this leaves the Integral ABC adding only __long__, __index__ and > 3-argument __pow__ over and above the Rational ABC. If 3-argument __pow__ > goes (which appears likely), we're left with __long__ and __index__. Let's ditch 3-arg pow, but keep __long__ (in 2.6) and __index__. Actually __long__ can go too. > However, there's still a few additional public properties and methods > inherited from higher up in the numeric stack which most existing integral > types are unlikely to provide: .real, .imag, .conjugate(). Unlike the > methods being relocated, however, these are absolutely trivial for all > non-complex numeric types. I definitely want to keep these. They're essential for people who want to use the higher-up classes in the numeric tower. I think this is settled now; Raymond can update PEP 3141 (if he refrains from editorializing) and patch numbers.py. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at ctypes.org Thu Jun 5 17:08:10 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 05 Jun 2008 17:08:10 +0200 Subject: [Python-Dev] PEP3118, python 2.6 Message-ID: <4848015A.5080009@ctypes.org> Hi Travis, I'm currently trying to port the pep3118 ctypes changes which are already in the py3k branch to the trunk. In py3k the tests for this stuff (in Lib/ctypes/test/test_pep3118.py) use the memoryview object which exposes attributes like .format, .shape, .strides and so on for objects implementing the new buffer interface. In Python 2.6 memoryview does not exist so the question is how to write a test that checks for the correct attributes. My idea is to implement the __array_interface__ property for ctypes instances, as described in this document http://numpy.scipy.org/array_interface.shtml. Do you think this is a good idea, or would it be better to invent another, private, mechanism for the tests? In other words: Would the __array_interface__ property (version 3) on these objects be useful for external code or would it disrupt external code? BTW: Did you have a chance to look over the test_pep3118 module in py3k, to make sure that I got things correctly (it should be pretty easy to read, imo)? -- Thanks, Thomas From jnoller at gmail.com Thu Jun 5 17:10:41 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 5 Jun 2008 11:10:41 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4847EFEF.6040302@gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> Message-ID: <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> On Thu, Jun 5, 2008 at 9:53 AM, Nick Coghlan wrote: > Jesse Noller wrote: >> >> However, the flip side of this is that no one really "likes" the >> threading API as-is, so putting the module into the standard library >> with the matching API simply adds another "broken" API. > > Provided threading gets a PEP 8 style API in 2.6 (which it looks like it is > going to), then I don't see a problem with multiprocessing only providing > the new API. > > Cheers, > Nick. > Fantastic. If no one has any other arguments to the contrary, I'll run with the new API naming scheme. Now I have a lot of tests and documentation to rewrite. -jesse From steve at pearwood.info Thu Jun 5 17:21:22 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 6 Jun 2008 01:21:22 +1000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4847DF68.1090205@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> <4847DF68.1090205@gmail.com> Message-ID: <200806060121.22560.steve@pearwood.info> On Thu, 5 Jun 2008 10:43:20 pm Nick Coghlan wrote: > I'm really starting to wonder if supporting positional arguments to > str.format() *at all* is a mistake. Maybe we should ditch support for > positional arguments and just accept a single dictionary as the sole > parameter to format(). > > For dictionary formatting, str.format() is a clear winner over > str.__mod__(). For positional formatting I'm not so sure - if someone > decided to convert from %-formatting to str.format, would it be such > a burden to ask them to name their substitution variables in the > process? If positional arguments are dropped, I would expect to see a proliferation of meaningless names used in the dicts: "items {a} {b}".format(dict(a=spam, b=eggs)) In other words, the names will be just aliases for the position. I would also expect a performance hit. On my computer, dict(a=spam, b=eggs) is two times slower than {'a':spam, 'b':eggs} and nearly three times slower than (spam, eggs). I don't imagine building a dict will ever be faster than building a tuple. Given that format() is already significantly slower than %, why make it slower still? -1 on dropping positional arguments. -- Steven From theller at ctypes.org Thu Jun 5 17:42:45 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 05 Jun 2008 17:42:45 +0200 Subject: [Python-Dev] PEP3118, python 2.6 In-Reply-To: <4848015A.5080009@ctypes.org> References: <4848015A.5080009@ctypes.org> Message-ID: Thomas Heller schrieb: > I'm currently trying to port the pep3118 ctypes changes which are already in > the py3k branch to the trunk. > > In py3k the tests for this stuff (in Lib/ctypes/test/test_pep3118.py) use > the memoryview object which exposes attributes like .format, .shape, .strides > and so on for objects implementing the new buffer interface. > > In Python 2.6 memoryview does not exist so the question is how to write a test > that checks for the correct attributes. My idea is to implement the > __array_interface__ property for ctypes instances, as described in this document > http://numpy.scipy.org/array_interface.shtml. In private email Travis told me that he has no time to backport the memoryview object to Python 2.6. Maybe there is someone who could do this (Myself I have no time either for this)? Thomas From musiccomposition at gmail.com Thu Jun 5 18:23:15 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 5 Jun 2008 11:23:15 -0500 Subject: [Python-Dev] on Python's tests (and making them better) Message-ID: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> Hi, This summer, I am going to be revamping Python's test suite. Major things I plan to do include - rewriting regrtest.py to be a simple test driver - implementing CPython only decorators - moving skipping data to the tests themselves (perhaps with decorators) - completing the transition to unittest/docttest - removing old test_support crud (like verify) - increasing the testing of unicode inputs and fixing bugs the appear - write new testing docs for newbie contributors - merging related tests - reorganizing the tests into separate directories Other suggestions? I also wanted to ask about the API compatibility requirements for test.test_support. IMO, we shouldn't include the testing tools as part of the stdlib. However, test_support is documented. Does anyone know any projects using test_support? -- Thanks, Benjamin Peterson "There's no place like 127.0.0.1." From fijall at gmail.com Thu Jun 5 18:41:33 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 5 Jun 2008 18:41:33 +0200 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> Message-ID: <693bc9ab0806050941q10a73354iccca0d7694ed9a5e@mail.gmail.com> On Thu, Jun 5, 2008 at 6:23 PM, Benjamin Peterson wrote: > Hi, > This summer, I am going to be revamping Python's test suite. Major > things I plan to do include > - rewriting regrtest.py to be a simple test driver > - implementing CPython only decorators > - moving skipping data to the tests themselves (perhaps with decorators) > - completing the transition to unittest/docttest > - removing old test_support crud (like verify) > - increasing the testing of unicode inputs and fixing bugs the appear > - write new testing docs for newbie contributors > - merging related tests > - reorganizing the tests into separate directories > > Other suggestions? Cool! I can help you with some info which tests are unlikely to pass on top of ie pypy (and *we* decided those are implementation-specific, but I might be wrong). Cheers, fijal From janssen at parc.com Thu Jun 5 18:41:44 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 5 Jun 2008 09:41:44 PDT Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <20080604222416.GD26992@mcnabbs.org> References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <20080604222416.GD26992@mcnabbs.org> Message-ID: <08Jun5.094145pdt."58698"@synergy1.parc.xerox.com> > I think I agree with Raymond on the basic principle that simple ABC's > are easier to use than simple ones. I don't think he was saying that. He was saying that simple ABC's are easier to implement to. It's not at all clear to me that simple ABC's are good in and of themselves. I think a String ABC should probably include all the methods that basestring provides, perhaps by reducing the methods that basestring provides, and moving the removed methods to the String ABC, with implementations that call the methods provided by basestring. That way, String classes could be implemented by overriding the "base" set of methods, but still inherit from String to get the other methods. But, then, isn't basestring the true simple ABC for strings? Bill From amcnabb at mcnabbs.org Thu Jun 5 18:57:13 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Thu, 5 Jun 2008 10:57:13 -0600 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <08Jun5.094145pdt."58698"@synergy1.parc.xerox.com> References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <20080604222416.GD26992@mcnabbs.org> <08Jun5.094145pdt."58698"@synergy1.parc.xerox.com> Message-ID: <20080605165713.GH30223@mcnabbs.org> On Thu, Jun 05, 2008 at 09:41:44AM -0700, Bill Janssen wrote: > > I think I agree with Raymond on the basic principle that simple ABC's > > are easier to use than simple ones. > > I don't think he was saying that. He was saying that simple ABC's are > easier to implement to. Sorry; I used the wrong word. I should have said "implement to" rather than "use." I agree with you. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From guido at python.org Thu Jun 5 19:22:13 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Jun 2008 10:22:13 -0700 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> Message-ID: I've accepted your PEP. I think it still needs some clean-up and perhaps clarification of the agreement reached about API style, but there is nothing now that keeps you from implementing it! Hopefully you'll make the beta release early next week. --Guido On Thu, Jun 5, 2008 at 8:10 AM, Jesse Noller wrote: > On Thu, Jun 5, 2008 at 9:53 AM, Nick Coghlan wrote: >> Jesse Noller wrote: >>> >>> However, the flip side of this is that no one really "likes" the >>> threading API as-is, so putting the module into the standard library >>> with the matching API simply adds another "broken" API. >> >> Provided threading gets a PEP 8 style API in 2.6 (which it looks like it is >> going to), then I don't see a problem with multiprocessing only providing >> the new API. >> >> Cheers, >> Nick. >> > > Fantastic. If no one has any other arguments to the contrary, I'll run > with the new API naming scheme. > > Now I have a lot of tests and documentation to rewrite. > > -jesse > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dalcinl at gmail.com Thu Jun 5 19:28:36 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 5 Jun 2008 14:28:36 -0300 Subject: [Python-Dev] PEP3118, python 2.6 In-Reply-To: References: <4848015A.5080009@ctypes.org> Message-ID: On 6/5/08, Thomas Heller wrote: Thomas, Iff this helps, you have attached the backport but as an extension module. Sorry, I do not have time for the full backport. But at least, I've found that the Py3K implementation works You will need to build it with 'python2.6 setup.py build' and copy 'memvwr.so' somewhere in your $PYTHONPATH, then in you 'site.py' or whatever, you can do 'import memvwr; del memvwr' . This will automaticall add the module to the __builtin__ module. Some notes: * I had to add Py_TPFLAGS_HAVE_NEWBUFFER to the type object structure. Should'nt this be already included in Py_TPFLAGS_DEFAULT ? * I have to monkey-copy two auxiliary functions 'abstrac.c' for Py3k, as ther are employed in memoryobject.c Travis, I believe you should have to review this. I'm not sure iff that is the right approach, regarding symbol visibility in shared libs in different OS's, compilers, etc.. But perhaps all is OK. Just a warning about possible problems :-). * I've not implemented the former buffer interface, I just put NULL's for the slots. Not sure if implementing the former interface is relly needed. * I've used PyString_XXX API calls. Not sure what will be the final status of the C API cleanup for Py2.6. Perhaps this should be replaced with PyBytes_XXXX > Thomas Heller schrieb: > > > I'm currently trying to port the pep3118 ctypes changes which are already in > > the py3k branch to the trunk. > > > > In py3k the tests for this stuff (in Lib/ctypes/test/test_pep3118.py) use > > the memoryview object which exposes attributes like .format, .shape, .strides > > and so on for objects implementing the new buffer interface. > > > > In Python 2.6 memoryview does not exist so the question is how to write a test > > that checks for the correct attributes. My idea is to implement the > > __array_interface__ property for ctypes instances, as described in this document > > http://numpy.scipy.org/array_interface.shtml. > > > In private email Travis told me that he has no time to backport the memoryview > object to Python 2.6. Maybe there is someone who could do this (Myself I have > no time either for this)? > > > Thomas > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 -------------- next part -------------- A non-text attachment was scrubbed... Name: memoryview-py2.6.tar.gz Type: application/x-gzip Size: 6427 bytes Desc: not available URL: From jnoller at gmail.com Thu Jun 5 19:48:12 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 5 Jun 2008 13:48:12 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> Message-ID: <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com> I'll have a cleaned version to you by end of day, and by day, I mean by 10:00 EST :) On Thu, Jun 5, 2008 at 1:22 PM, Guido van Rossum wrote: > I've accepted your PEP. I think it still needs some clean-up and > perhaps clarification of the agreement reached about API style, but > there is nothing now that keeps you from implementing it! Hopefully > you'll make the beta release early next week. > > --Guido > > On Thu, Jun 5, 2008 at 8:10 AM, Jesse Noller wrote: >> On Thu, Jun 5, 2008 at 9:53 AM, Nick Coghlan wrote: >>> Jesse Noller wrote: >>>> >>>> However, the flip side of this is that no one really "likes" the >>>> threading API as-is, so putting the module into the standard library >>>> with the matching API simply adds another "broken" API. >>> >>> Provided threading gets a PEP 8 style API in 2.6 (which it looks like it is >>> going to), then I don't see a problem with multiprocessing only providing >>> the new API. >>> >>> Cheers, >>> Nick. >>> >> >> Fantastic. If no one has any other arguments to the contrary, I'll run >> with the new API naming scheme. >> >> Now I have a lot of tests and documentation to rewrite. >> >> -jesse >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From musiccomposition at gmail.com Thu Jun 5 20:54:01 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 5 Jun 2008 13:54:01 -0500 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com> Message-ID: <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com> On Thu, Jun 5, 2008 at 12:48 PM, Jesse Noller wrote: > I'll have a cleaned version to you by end of day, and by day, I mean > by 10:00 EST :) Great! Please post that on the bug tracker when you're ready. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From martin at v.loewis.de Thu Jun 5 21:32:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Jun 2008 21:32:00 +0200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com><48467E50.9030202@voidspace.org.uk><48469888.4070603@gmail.com> <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> <4847DF68.1090205@gmail.com> Message-ID: <48483F30.6030101@v.loewis.de> > Agreed. %(something)s is quirky - you are bound to forget the final "s" > (or whatever other specifier) from time to time. But that gives a ValueError the first time you try it, no? Regards, Martin From theller at ctypes.org Thu Jun 5 22:05:30 2008 From: theller at ctypes.org (Thomas Heller) Date: Thu, 05 Jun 2008 22:05:30 +0200 Subject: [Python-Dev] PEP3118, python 2.6 In-Reply-To: References: <4848015A.5080009@ctypes.org> Message-ID: Lisandro Dalcin schrieb: > On 6/5/08, Thomas Heller wrote: > Thomas, Iff this helps, you have attached the backport but as an > extension module. Sorry, I do not have time for the full backport. > But at least, I've found that the Py3K implementation works Thanks for the effort. Maybe I will look into that later. (And, if you want to make sure that your patch doesn't get lost and hopefully reviewed you should add it to the bug tracker.) In the meantime I found that I already had a _ctypes private function that can expose the needed info to Python, for ctypes pep3118 testing. Thanks again, Thomas From jnoller at gmail.com Thu Jun 5 23:44:36 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 5 Jun 2008 17:44:36 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com> <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com> Message-ID: <4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com> On Thu, Jun 5, 2008 at 2:54 PM, Benjamin Peterson wrote: > On Thu, Jun 5, 2008 at 12:48 PM, Jesse Noller wrote: >> I'll have a cleaned version to you by end of day, and by day, I mean >> by 10:00 EST :) > > Great! Please post that on the bug tracker when you're ready. > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > I meant a cleaned version of the PEP - I still have docs and tests to redo From ncoghlan at gmail.com Fri Jun 6 04:06:41 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Jun 2008 12:06:41 +1000 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <48466F63.3020902@gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com> <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com> <4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com> Message-ID: <48489BB1.7080702@gmail.com> Jesse Noller wrote: > I meant a cleaned version of the PEP - I still have docs and tests to redo > It would also be good if you could check Benjamin's patch on issue 3402 to give threading a PEP 8 compliant API and make sure the names are the same as the new names in multiprocessing. Cheers, Nick. P.S. OK, now I really am going to be offline for the next few days ;) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jun 6 04:07:53 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Jun 2008 12:07:53 +1000 Subject: [Python-Dev] PEP3118, python 2.6 In-Reply-To: References: <4848015A.5080009@ctypes.org> Message-ID: <48489BF9.3070407@gmail.com> Thomas Heller wrote: > Lisandro Dalcin schrieb: >> On 6/5/08, Thomas Heller wrote: >> Thomas, Iff this helps, you have attached the backport but as an >> extension module. Sorry, I do not have time for the full backport. >> But at least, I've found that the Py3K implementation works > > Thanks for the effort. Maybe I will look into that later. (And, if > you want to make sure that your patch doesn't get lost and hopefully > reviewed you should add it to the bug tracker.) There's an existing issue for the memoryview backport. Maybe it's priority should be bumped up a bit? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jnoller at gmail.com Fri Jun 6 04:08:35 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 5 Jun 2008 22:08:35 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <48489BB1.7080702@gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <4847E33A.1050806@gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com> <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com> <4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com> <48489BB1.7080702@gmail.com> Message-ID: <4222a8490806051908r20c93293v2761ed61c70dbeef@mail.gmail.com> On Thu, Jun 5, 2008 at 10:06 PM, Nick Coghlan wrote: > Jesse Noller wrote: >> >> I meant a cleaned version of the PEP - I still have docs and tests to redo >> > It would also be good if you could check Benjamin's patch on issue 3402 to > give threading a PEP 8 compliant API and make sure the names are the same as > the new names in multiprocessing. > > Cheers, > Nick. > > P.S. OK, now I really am going to be offline for the next few days ;) > Good idea, I'll add that to the queue - right now I'm focusing on getting the module into the build on trunk (locally) and getting all of the docs/tests updated and up to par. -jesse From jimjjewett at gmail.com Fri Jun 6 05:19:34 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 5 Jun 2008 23:19:34 -0400 Subject: [Python-Dev] PEP 8 vs PEP 371: Additional Discussion Message-ID: Guido van Rossum wrote: > I consider multiprocessing a new API -- while it bears > a superficial resemblance with the threading API the > similarities are just that, and it should not be > constrained by mistakes in that API. The justification for including it is precisely that it is *not* a new API. For multiple processes in general, there are competing APIs, which may well be better. The advantage of this API is that (in many cases) it is a drop-in replacement for threading. If that breaks, then there really isn't any reason to include it in the stdlib yet. This doesn't prevent changing the joint API to conform with PEP 8. But why clean this module while leaving the rest of the stdlib? "Because there is a volunteer" only makes sense if changes to the other modules would also be welcomed. Is there some reason to believe that changes in the threading API are much less disruptive than changes elsewhere in the stdlib? -jJ From greg.ewing at canterbury.ac.nz Fri Jun 6 05:32:35 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 06 Jun 2008 15:32:35 +1200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> <200806051230.33243.steve@pearwood.info> <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com> Message-ID: <4848AFD3.5020606@canterbury.ac.nz> Paul Moore wrote: > Because the second breaks if value is a tuple: However, changing it now is going to break a huge amount of existing code that uses %-formatting, and in ways that 2to3 can't reliably fix. Keeping %-formatting but breaking a large proportion of its uses doesn't seem like a good idea to me. -- Greg From greg.ewing at canterbury.ac.nz Fri Jun 6 05:37:50 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 06 Jun 2008 15:37:50 +1200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4847DF68.1090205@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> <4847DF68.1090205@gmail.com> Message-ID: <4848B10E.5000902@canterbury.ac.nz> Nick Coghlan wrote: > %r is about the worst case for the new syntax relative to the old - two > characters become 5. It's worth looking at what those extra characters > buy us though: However, those benefits are only realised some of the time, and it's only in rare cases that they're all realised at once. The cost-benefit analysis needs to take the entropy into account somehow. -- Greg From greg.ewing at canterbury.ac.nz Fri Jun 6 05:39:33 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 06 Jun 2008 15:39:33 +1200 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4847DF68.1090205@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> <4847DF68.1090205@gmail.com> Message-ID: <4848B175.9050307@canterbury.ac.nz> Nick Coghlan wrote: > Maybe we should ditch support for > positional arguments and just accept a single dictionary as the sole > parameter to format(). > > "{num} occurs {num} times in this format string".format(dict(num=2)) If named arguments were to become mandatory, I'd want to be able to write that as "...{num}...".format(num = 2) -- Greg From alexandre at peadrop.com Fri Jun 6 05:43:59 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Thu, 5 Jun 2008 23:43:59 -0400 Subject: [Python-Dev] [Python-3000] How to specify keyword-only arguments from C? In-Reply-To: References: <02ee01c8c77b$1553d6e0$3ffb84a0$@com.au> Message-ID: On Thu, Jun 5, 2008 at 11:18 PM, Alexandre Vassalotti wrote: > On Thu, Jun 5, 2008 at 10:14 PM, Mark Hammond wrote: >> Set an error if the 'arg' tuple doesn't have a length of zero? >> > > Oh, that isn't a bad idea at all. I will try this. Thanks! > Worked flawlessly! Just for the archives, here's how it looks like: static int Unpickler_init(UnpicklerObject *self, PyObject *args, PyObject *kwds) { static char *kwlist[] = {"file", "encoding", "errors", 0}; PyObject *file; char *encoding = NULL; char *errors = NULL; if (Py_SIZE(args) != 1) { PyErr_Format(PyExc_TypeError, "%s takes exactly one 1 positional argument (%zd given)", Py_TYPE(self)->tp_name, Py_SIZE(args)); return -1; } if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|ss:Unpickler", kwlist, &file, &encoding, &errors)) return -1; ... Thank you, Mark, for the tip! -- Alexandre From python at rcn.com Fri Jun 6 05:45:27 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 5 Jun 2008 20:45:27 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> Message-ID: <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> [Terry Reedy] >> On reading PEP3141 some months ago and again today, I thought and still do >> that all the methods that depend on a 2s-complement representation and >> implementation really belong to an implentation-defined subclass of >> Integral. But I am not sure of the purpose of the class and of including >> such concrete methods in an ABC, and so said nothing ;-). [Nick Coghlan] > I think it definitely makes sense to separate out the > number-as-sequence-of-bits operations from the main Integral ABC. This > would involve moving: > > lshift, rshift, and, or, xor, invert (along with their reversed and > in-place counterparts) I vote for complete removal of the bit flipping methods. There's a cost to creating new ABCs that are devoid of use cases. When Terry says "I am not sure of the purpose of the class ...", I think he meant that a binary abc wasn't proposed because of its questionable utility. At some point, if you want an int type, you just use an int type. What is the purpose of having yet another layer in the "numeric tower"? Does anyone actually need an int lookalike with binary methods but cannot just inherit from int? Looking back at PEP 3119, I see there was an early decision "between standardizing more, fine-grained ABCs or fewer, course-grained ones" and that the choice was resolved in favor the few. That is how we avoided urban sprawl with the likes of "ComposableSet, MutableSet, HashableSet, MutableComposableSet, HashableComposableSet". I had thought everyone was on-board with the notion that thin abcs that capture the essence of a type are better than fat abcs that seek emulate every aspect of a single concrete type. Without agreement on that premise, simplification is a lost cause. If the group wants to go forward with a Binary subclass of Integral, then I would like to withdraw this simplification mini-pep. Better to stick with what we have now than to addcomplexity to a module that is already use case challenged. Raymond From jimjjewett at gmail.com Fri Jun 6 05:39:39 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 5 Jun 2008 23:39:39 -0400 Subject: [Python-Dev] Mini-Pep: An Empty String ABC Message-ID: > So, apart from compatibility purposes, what is the > point currently of *not* directly subclassing str? To provide your own storage format, such as a views into existing data. Whether or not this is actually practical is a different question; plenty C code tends to assume it can use the internals of str directly, which breaks on even some subclasses. -jJ From g.brandl at gmx.net Fri Jun 6 13:08:25 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 06 Jun 2008 11:08:25 +0000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4848B175.9050307@canterbury.ac.nz> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> <33649.192.165.213.18.1212588500.squirrel@webmail.nerim.net> <2A332D5D0BD54A8EACE1B95BC6AD3FA4@RaymondLaptop1> <4847DF68.1090205@gmail.com> <4848B175.9050307@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Nick Coghlan wrote: >> Maybe we should ditch support for >> positional arguments and just accept a single dictionary as the sole >> parameter to format(). >> >> "{num} occurs {num} times in this format string".format(dict(num=2)) > > If named arguments were to become mandatory, I'd want > to be able to write that as > > "...{num}...".format(num = 2) You already are -- in fact, that is the only way this works. (I'd wondered why nobody brought this error up in this discussion before. Probably it wouldn't have been ugly enough.) I don't see a significant speed difference between positional and keyword arguments to format() when testing with two arguments. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Jun 6 13:12:27 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 06 Jun 2008 11:12:27 +0000 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4848AFD3.5020606@canterbury.ac.nz> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> <200806051230.33243.steve@pearwood.info> <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com> <4848AFD3.5020606@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Paul Moore wrote: > >> Because the second breaks if value is a tuple: > > However, changing it now is going to break a huge > amount of existing code that uses %-formatting, > and in ways that 2to3 can't reliably fix. > > Keeping %-formatting but breaking a large > proportion of its uses doesn't seem like a good > idea to me. Exactly. If % formatting is removed in 3k, code breaks, so we keep it. Changing it now so that code breaks anyway is not productive :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From mal at egenix.com Fri Jun 6 11:19:29 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 06 Jun 2008 11:19:29 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com> Message-ID: <48490121.5030701@egenix.com> On 2008-06-03 01:29, Gregory P. Smith wrote: > On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum wrote: > >> I will freely admit that I haven't followed this thread in any detail, >> but if it were up to me, I'd have the 2.6 internal code use PyString > > ... > > Should we read this as a BDFL pronouncement and make it so? > > All that would mean change wise is that trunk r63675 as well as possibly > r63672 and r63677 would need to be rolled back and this whole discussion > over if such a big change should have happened would turn into a moot point. I would certainly welcome reverting the change. All that's needed to support PyBytes API in 2.x is a set of #defines that map the new APIs to the PyString names. That's a clean and easily understandable solution. Programmers interested in the code for a PyString API can then still look up the code in stringobject.c, e.g. to find out how a certain special case is handled or to check the ref counting - just like they did for years. Developer who want to start differentiating between mixed byte/text data and bytes-only can start using PyBytes for byte data. >> I would also add macros that map the PyBytes_* APIs to PyString_*, but >> I would not start using these internally except in code newly written >> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString >> for 8-bit strings containing text, and PyBytes for 8-bit strings >> containing binary data. For 8-bit strings that could contain either >> text or data, I'd use PyString, in the spirit of 2.x. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 06 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 30 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From g.brandl at gmx.net Fri Jun 6 13:27:29 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 06 Jun 2008 11:27:29 +0000 Subject: [Python-Dev] Modules for 2.6 inclusion Message-ID: Hi, PEP 361 lists the following modules for possible inclusion in 2.6 (next to pyprocessing, which is now accepted): - winerror http://python.org/sf/1505257 (Owner: MAL) This patch has been marked as rejected, so I'll remove the entry from the PEP. - setuptools BDFL pronouncement for inclusion in 2.5: http://mail.python.org/pipermail/python-dev/2006-April/063964.html PJE's withdrawal from 2.5 for inclusion in 2.6: http://mail.python.org/pipermail/python-dev/2006-April/064145.html I guess this will be deferred? - ast http://mail.python.org/pipermail/python-dev/2008-April/078950.html If there's no objection, I'll go over the interface with Thomas, who's working on AST optimization for 2.7, to make sure the ast interface can stay the same after his branch is merged, write the docs and commit it before beta1. - bdist_deb in distutils package http://mail.python.org/pipermail/python-dev/2006-February/060926.html - bdist_egg in distutils package - pure python pgen module (Owner: Guido) Deferral to 2.6: http://mail.python.org/pipermail/python-dev/2006-April/064528.html There are also several other "possible" todo items in PEP 361, but they all look as if they are not required to be in before beta1. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From p.f.moore at gmail.com Fri Jun 6 11:31:11 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Jun 2008 10:31:11 +0100 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> <200806051230.33243.steve@pearwood.info> <79990c6b0806050350x3cb5c579qd692dc354cd0f24d@mail.gmail.com> <4848AFD3.5020606@canterbury.ac.nz> Message-ID: <79990c6b0806060231k4316e21aof9b9e3103d64c79@mail.gmail.com> On 06/06/2008, Georg Brandl wrote: > Greg Ewing schrieb: > > Paul Moore wrote: > > > > > > > Because the second breaks if value is a tuple: > > > > > > > However, changing it now is going to break a huge > > amount of existing code that uses %-formatting, > > and in ways that 2to3 can't reliably fix. > > > > Keeping %-formatting but breaking a large > > proportion of its uses doesn't seem like a good > > idea to me. > > > > Exactly. If % formatting is removed in 3k, code breaks, so we keep it. > Changing it now so that code breaks anyway is not productive :) Just to clarify, I'm not advocating change here. My comment (above) about code breaking was in reference to an issue with a chunk of user-level code which got clipped from the quote, and is completely unrelated to any diiscussion of changing Python. Paul. From mal at egenix.com Fri Jun 6 13:07:31 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 06 Jun 2008 13:07:31 +0200 Subject: [Python-Dev] Modules for 2.6 inclusion In-Reply-To: References: Message-ID: <48491A73.7060104@egenix.com> On 2008-06-06 13:27, Georg Brandl wrote: > Hi, > > PEP 361 lists the following modules for possible inclusion in 2.6 (next > to pyprocessing, which is now accepted): > > - winerror > http://python.org/sf/1505257 > (Owner: MAL) > > This patch has been marked as rejected, so I'll remove the entry from > the PEP. Note that the idea is still valid - the implementation of the module should be written in C and the patch only comes with a Python module. If anyone would like to work on a (generated) C module, please have a look at the patch. winerror is meant to provide access to the Windows error codes which are currently not available in Python. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 06 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 30 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From steve at holdenweb.com Fri Jun 6 13:14:07 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 06 Jun 2008 07:14:07 -0400 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: <4847D9E0.4020805@gmail.com> References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4844F5B4.5090703@v.loewis.de> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> <4847D9E0.4020805@gmail.com> Message-ID: Nick Coghlan wrote: > Greg Ewing wrote: >> Nick Coghlan wrote: >> >>> - remove support for passing a single value to a format string >>> without wrapping it in an iterable first >> >> But won't that clobber a large number of the simple >> use cases that you want to keep %-formatting for? > > Note the part of the proposal that allows *any* iterable on the right > hand side rather than the current insistence on a tuple. So the > single-value use cases can be boxed easily with a list. > > To my mind salvaging %-formatting requires removing the ambiguity over > whether or not the right hand side will be iterated over. > But then this breaks code where iterables are intended to be output using a single %s format specifier, for example. I don't see why this would be regarded as helpful. To avoid breakage I'd rather keep the %-formatting ability as it is, and label it a legacy feature, rather than "salvaging" it. It's going to be too tricky to convert using 2to3 otherwise. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ggpolo at gmail.com Fri Jun 6 13:43:10 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 6 Jun 2008 08:43:10 -0300 Subject: [Python-Dev] Modules for 2.6 inclusion In-Reply-To: References: Message-ID: 2008/6/6 Georg Brandl : > Hi, > > PEP 361 lists the following modules for possible inclusion in 2.6 (next > to pyprocessing, which is now accepted): > > - winerror > http://python.org/sf/1505257 > (Owner: MAL) > > This patch has been marked as rejected, so I'll remove the entry from > the PEP. > > - setuptools > BDFL pronouncement for inclusion in 2.5: > http://mail.python.org/pipermail/python-dev/2006-April/063964.html > > PJE's withdrawal from 2.5 for inclusion in 2.6: > http://mail.python.org/pipermail/python-dev/2006-April/064145.html > > I guess this will be deferred? > > - ast > http://mail.python.org/pipermail/python-dev/2008-April/078950.html > > If there's no objection, I'll go over the interface with Thomas, who's > working on AST optimization for 2.7, to make sure the ast interface can > stay the same after his branch is merged, write the docs and commit it > before beta1. > > - bdist_deb in distutils package > http://mail.python.org/pipermail/python-dev/2006-February/060926.html > > - bdist_egg in distutils package > > - pure python pgen module > (Owner: Guido) > Deferral to 2.6: > http://mail.python.org/pipermail/python-dev/2006-April/064528.html > I created an issue 1 week ago (http://bugs.python.org/issue2983) suggesting the addition of the ttk module to lib-tk, and to the new tkinter package. Is there any chance to this be accepted for Python 2.6 ? > > There are also several other "possible" todo items in PEP 361, but they > all look as if they are not required to be in before beta1. > > Georg > > -- > Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. > Four shall be the number of spaces thou shalt indent, and the number of thy > indenting shall be four. Eight shalt thou not indent, nor either indent thou > two, excepting that thou then proceed to four. Tabs are right out. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com > -- -- Guilherme H. Polo Goncalves From jnoller at gmail.com Fri Jun 6 14:05:20 2008 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 6 Jun 2008 08:05:20 -0400 Subject: [Python-Dev] PEP 371: Additional Discussion In-Reply-To: <4222a8490806051908r20c93293v2761ed61c70dbeef@mail.gmail.com> References: <4222a8490806031430k26b43686ua6724b7d6e5981fb@mail.gmail.com> <4222a8490806050615y3039ae23n362a2a780b5f16fd@mail.gmail.com> <4847EFEF.6040302@gmail.com> <4222a8490806050810re44924et7df61fd2070d4dca@mail.gmail.com> <4222a8490806051048k2d0acfa1t607821c6d2b707e2@mail.gmail.com> <1afaf6160806051154u50b2ce20wca3caa4fec00f9c@mail.gmail.com> <4222a8490806051444qd561e68j72f034d214412aa@mail.gmail.com> <48489BB1.7080702@gmail.com> <4222a8490806051908r20c93293v2761ed61c70dbeef@mail.gmail.com> Message-ID: <4222a8490806060505h2be38ef4k27f346e8981a2391@mail.gmail.com> On Thu, Jun 5, 2008 at 10:08 PM, Jesse Noller wrote: > On Thu, Jun 5, 2008 at 10:06 PM, Nick Coghlan wrote: >> Jesse Noller wrote: >>> >>> I meant a cleaned version of the PEP - I still have docs and tests to redo >>> >> It would also be good if you could check Benjamin's patch on issue 3402 to >> give threading a PEP 8 compliant API and make sure the names are the same as >> the new names in multiprocessing. >> >> Cheers, >> Nick. >> >> P.S. OK, now I really am going to be offline for the next few days ;) >> > > Good idea, I'll add that to the queue - right now I'm focusing on > getting the module into the build on trunk (locally) and getting all > of the docs/tests updated and up to par. > > -jesse > Just to follow up - I have added a ticket to the tracker (http://bugs.python.org/issue3050) to track current status and work, as issues are resolve both the ticket and the PEP will be updated to reflect status. I still need to (or have someone with dev privs) add dependencies to issue 3042 and 1683 -jesse From guido at python.org Fri Jun 6 17:52:26 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jun 2008 08:52:26 -0700 Subject: [Python-Dev] converting the stdlib to str.format In-Reply-To: References: <1afaf6160806021749xba33304t9ef5bf2936fcc7bf@mail.gmail.com> <4845934A.6030708@v.loewis.de> <48466D0C.40709@gmail.com> <48467E50.9030202@voidspace.org.uk> <48469888.4070603@gmail.com> <484745D7.3000808@canterbury.ac.nz> <4847D9E0.4020805@gmail.com> Message-ID: +1 On 6/6/08, Steve Holden wrote: > Nick Coghlan wrote: >> Greg Ewing wrote: >>> Nick Coghlan wrote: >>> >>>> - remove support for passing a single value to a format string >>>> without wrapping it in an iterable first >>> >>> But won't that clobber a large number of the simple >>> use cases that you want to keep %-formatting for? >> >> Note the part of the proposal that allows *any* iterable on the right >> hand side rather than the current insistence on a tuple. So the >> single-value use cases can be boxed easily with a list. >> >> To my mind salvaging %-formatting requires removing the ambiguity over >> whether or not the right hand side will be iterated over. >> > But then this breaks code where iterables are intended to be output > using a single %s format specifier, for example. I don't see why this > would be regarded as helpful. To avoid breakage I'd rather keep the > %-formatting ability as it is, and label it a legacy feature, rather > than "salvaging" it. It's going to be too tricky to convert using 2to3 > otherwise. > > regards > Steve > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > Holden Web LLC http://www.holdenweb.com/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- Sent from Gmail for mobile | mobile.google.com --Guido van Rossum (home page: http://www.python.org/~guido/) From status at bugs.python.org Fri Jun 6 18:06:28 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 6 Jun 2008 18:06:28 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080606160628.DE7D078096@psf.upfronthosting.co.za> ACTIVITY SUMMARY (05/30/08 - 06/06/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1895 open (+28) / 12981 closed (+15) / 14876 total (+43) Open issues with patches: 575 Average duration of open issues: 708 days. Median duration of open issues: 1451 days. Open Issues Breakdown open 1872 (+27) pending 23 ( +1) Issues Created Or Reopened (45) _______________________________ tkinter, assorted fixes 06/02/08 CLOSED http://bugs.python.org/issue2906 reopened gpolo patch, patch PyNumberMethods has left-over fields in Py3 06/02/08 http://bugs.python.org/issue2997 reopened georg.brandl patch locale module alias table needs to be updated 05/30/08 CLOSED http://bugs.python.org/issue3011 created lemburg disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523 05/30/08 CLOSED http://bugs.python.org/issue3012 created jameinel disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523 05/30/08 http://bugs.python.org/issue3013 created jameinel patch, easy file_dealloc() assumes errno is set when EOF is returned 05/30/08 http://bugs.python.org/issue3014 created johansen tkinter with wantobjects=False has been broken for some time 05/30/08 http://bugs.python.org/issue3015 created gpolo patch tarfile.py incurs exception from self.chmod() when tarball has g 05/30/08 CLOSED http://bugs.python.org/issue3016 created zooko Verify doc updates for the decimal module 05/31/08 http://bugs.python.org/issue3017 created rhettinger tkinter demos fixed 05/31/08 http://bugs.python.org/issue3018 created gpolo patch Python3a5 compile failing due to high memory usage 05/31/08 CLOSED http://bugs.python.org/issue3019 created segfaulthunter doctest should have lib2to3 integration 06/01/08 http://bugs.python.org/issue3020 created scoder Lexical exception handlers 06/01/08 http://bugs.python.org/issue3021 created pitrou patch mailbox module, two small fixes 06/01/08 http://bugs.python.org/issue3022 created gpolo patch, patch Problem with invalidly-encoded command-line arguments (Unix) 06/01/08 http://bugs.python.org/issue3023 created baikie Integer conversion inconsistent 06/01/08 CLOSED http://bugs.python.org/issue3024 created helminthe batch/IDLE differ: print broken for chraracters>ascii 06/01/08 CLOSED http://bugs.python.org/issue3025 created jimjjewett mmap broken with large files on 64bit system 06/02/08 http://bugs.python.org/issue3026 created donut if extended c module is calling Windows API waitForSingleObject, 06/02/08 CLOSED http://bugs.python.org/issue3027 created patrick tokenize module: normal lines, not "logical" 06/02/08 http://bugs.python.org/issue3028 created noam free list management - list, dict, set 06/02/08 http://bugs.python.org/issue3029 created aimacintyre patch, patch compiler warning on HP-UX 06/03/08 http://bugs.python.org/issue3030 created theller distutils package_dir/package_data failure 06/03/08 http://bugs.python.org/issue3031 created fma tkFont added displayof where necessary 06/03/08 CLOSED http://bugs.python.org/issue3032 created gpolo patch tkFont added displayof where necessary 06/03/08 http://bugs.python.org/issue3033 created gpolo patch, patch tkFont added displayof where necessary 06/03/08 CLOSED http://bugs.python.org/issue3034 created gpolo patch, patch Removing apparently unwanted functions from Tkinter 06/03/08 http://bugs.python.org/issue3035 created gpolo patch, patch docs - print() example wrong in whatsnew 06/04/08 CLOSED http://bugs.python.org/issue3036 created evilnick in output 06/04/08 CLOSED http://bugs.python.org/issue3037 created ianb Return results from Python callbacks to Tcl as Tcl objects, plea 06/05/08 http://bugs.python.org/issue3038 created gpolo tarfile.TarFileCompat.writestr(ZipInfo, str) raises AttributeErr 06/05/08 http://bugs.python.org/issue3039 created jkankiewicz patch optparse documentation: variable arguments example doesn't work 06/05/08 http://bugs.python.org/issue3040 created meyerowitz autodoc does not support unicode docstrings 06/05/08 CLOSED http://bugs.python.org/issue3041 created cdevienne Add PEP 8 compliant aliases to threading module 06/05/08 http://bugs.python.org/issue3042 created ncoghlan patch Recursion bug in deepcopy 06/05/08 http://bugs.python.org/issue3043 created Zeroth Simplify the Integral ABC 06/05/08 CLOSED http://bugs.python.org/issue3044 created rhettinger patch Windows online help broken when spaces in TEMP environ 06/06/08 http://bugs.python.org/issue3045 created peta Locking should be removed from the new buffer protocol 06/06/08 http://bugs.python.org/issue3046 created scoder patch string object's lstrip function 06/06/08 CLOSED http://bugs.python.org/issue3047 created jsostok getsizeof incorrect for Unicode strings 06/06/08 http://bugs.python.org/issue3048 created loewis Some 3k sizeof tests fail 06/06/08 CLOSED http://bugs.python.org/issue3049 created loewis Implement PEP 371: multiprocessing module 06/06/08 http://bugs.python.org/issue3050 created jnoller heapq change breaking compatibility 06/06/08 http://bugs.python.org/issue3051 created therve Mac Modules failing to compile 06/06/08 http://bugs.python.org/issue3052 created benjamin.peterson test_shutil fails under AIX 06/06/08 http://bugs.python.org/issue3053 created pitrou patch Issues Now Closed (42) ______________________ py3k: Completely remove nb_coerce slot 260 days http://bugs.python.org/issue1185 scoder patch Add ctypes calling convention that allows safe access of errno 147 days http://bugs.python.org/issue1798 theller patch Adapt pydoc to new doc system 132 days http://bugs.python.org/issue1883 georg.brandl patch Read support for Records in msilib 108 days http://bugs.python.org/issue2125 loewis patch Backport oct() and hex() to use __index__ 20 days http://bugs.python.org/issue2337 benjamin.peterson patch, 26backport numeric overflow in IDLE 53 days http://bugs.python.org/issue2584 kbk PyOS_vsnprintf() underflow leads to memory corruption 54 days http://bugs.python.org/issue2588 gregory.p.smith patch PyOS_vsnprintf() potential integer overflow leads to memory corr 54 days http://bugs.python.org/issue2589 gregory.p.smith inconsistency with bare * in parameter list 54 days http://bugs.python.org/issue2613 bgolemon datetime/date strftime() method and time.strftime() inconsistenc 26 days http://bugs.python.org/issue2782 gregory.p.smith subprocess.py leaks fd in communicate 24 days http://bugs.python.org/issue2791 gregory.p.smith patch store thread.get_ident() thread identifier inside threading.Thre 17 days http://bugs.python.org/issue2871 gregory.p.smith patch Remove htmllib use in the stdlib 17 days http://bugs.python.org/issue2873 georg.brandl Create the tkinter package 16 days http://bugs.python.org/issue2884 kbk patch Add memory footprint query 15 days http://bugs.python.org/issue2898 schuppenies patch tkinter, assorted fixes 0 days http://bugs.python.org/issue2906 georg.brandl patch, patch ctypes.util.find_library() doesn't consult LD_LIBRARY_PATH 12 days http://bugs.python.org/issue2936 theller EasyDialogs - documentation enhancement 2 days http://bugs.python.org/issue3005 georg.brandl Module cmd documentation enhancement 1 days http://bugs.python.org/issue3010 georg.brandl patch locale module alias table needs to be updated 0 days http://bugs.python.org/issue3011 lemburg disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523 0 days http://bugs.python.org/issue3012 benjamin.peterson tarfile.py incurs exception from self.chmod() when tarball has g 1 days http://bugs.python.org/issue3016 lars.gustaebel Python3a5 compile failing due to high memory usage 0 days http://bugs.python.org/issue3019 georg.brandl Integer conversion inconsistent 0 days http://bugs.python.org/issue3024 rhettinger batch/IDLE differ: print broken for chraracters>ascii 0 days http://bugs.python.org/issue3025 loewis if extended c module is calling Windows API waitForSingleObject, 1 days http://bugs.python.org/issue3027 georg.brandl tkFont added displayof where necessary 0 days http://bugs.python.org/issue3032 gpolo patch tkFont added displayof where necessary 0 days http://bugs.python.org/issue3034 gpolo patch, patch docs - print() example wrong in whatsnew 0 days http://bugs.python.org/issue3036 georg.brandl in output 0 days http://bugs.python.org/issue3037 georg.brandl autodoc does not support unicode docstrings 0 days http://bugs.python.org/issue3041 georg.brandl Simplify the Integral ABC 0 days http://bugs.python.org/issue3044 gvanrossum patch string object's lstrip function 0 days http://bugs.python.org/issue3047 jsostok Some 3k sizeof tests fail 0 days http://bugs.python.org/issue3049 schuppenies Tk.quit and sys.exit cause Fatal Error 1669 days http://bugs.python.org/issue837234 georg.brandl patch 1079734 broke cgi.FieldStorage w/ multipart post req. 1218 days http://bugs.python.org/issue1112856 hdiogenes new turtle module 707 days http://bugs.python.org/issue1513695 loewis patch Prevent textwrap from breaking words at hyphens 413 days http://bugs.python.org/issue1702681 gpolo access to unicodedata (via codepoints or 2-char surrogates) 406 days http://bugs.python.org/issue1706460 doerwalter tiny addition to peephole optimizer 305 days http://bugs.python.org/issue1764087 rhettinger patch generic and more efficient removal of unreachable code 304 days http://bugs.python.org/issue1764986 rhettinger patch small improvement for peephole conditional jump optimizer 304 days http://bugs.python.org/issue1765558 rhettinger patch Top Issues Most Discussed (10) ______________________________ 16 Let bin() show floats 7 days open http://bugs.python.org/issue3008 14 repr() should not escape non-ASCII characters 53 days open http://bugs.python.org/issue2630 6 Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1 518 days open http://bugs.python.org/issue1628484 5 Integer conversion inconsistent 0 days closed http://bugs.python.org/issue3024 5 PyNumberMethods has left-over fields in Py3 4 days open http://bugs.python.org/issue2997 5 Remove PyUnicode_AsString(), rework PyUnicode_AsStringAndSize() 28 days open http://bugs.python.org/issue2799 5 Add ctypes calling convention that allows safe access of errno 147 days closed http://bugs.python.org/issue1798 4 New class special method lookup change 2020 days open http://bugs.python.org/issue643841 4 Recursion bug in deepcopy 1 days pending http://bugs.python.org/issue3043 4 Add PEP 8 compliant aliases to threading module 1 days open http://bugs.python.org/issue3042 From guido at python.org Fri Jun 6 19:22:24 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jun 2008 10:22:24 -0700 Subject: [Python-Dev] PEP 8 vs PEP 371: Additional Discussion In-Reply-To: References: Message-ID: On Thu, Jun 5, 2008 at 8:19 PM, Jim Jewett wrote: > Guido van Rossum wrote: > >> I consider multiprocessing a new API -- while it bears >> a superficial resemblance with the threading API the >> similarities are just that, and it should not be >> constrained by mistakes in that API. > > The justification for including it is precisely that it is *not* a new API. It is new in the stdlib. Insofar it is not new, renaming the methods is a very superficial thing. > For multiple processes in general, there are competing APIs, which may > well be better. The advantage of this API is that (in many cases) it > is a drop-in replacement for threading. If that breaks, then there > really isn't any reason to include it in the stdlib yet. I've said before, and nobody has disagreed, that while familiarity with the threading module helps people understand how to use the multiprocessing module (as opposed to other APIs, which differ more deeply than just in their method names), but that I don't expect anyone to take an existing multi-threaded app of any complexity and turn it into a multi-processing app by changing one import. > This doesn't prevent changing the joint API to conform with PEP 8. > But why clean this module while leaving the rest of the stdlib? Have you actually read the thread? Many arguments were put forward for cleaning up the API -- there is no need to perpetuate mistakes of the past when introducing anything "new" to the stdlib. > "Because there is a volunteer" only makes sense if changes to the > other modules would also be welcomed. Is there some reason to believe > that changes in the threading API are much less disruptive than > changes elsewhere in the stdlib? We'll keep the compatible API as long as needed. This means in 3.0 too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 6 20:01:20 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jun 2008 11:01:20 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger wrote: > Does anyone actually need an int lookalike with binary methods but > cannot just inherit from int? Does anyone actually need an int lookalike with operations like +, - etc. but cannot just inherit from int? If the answer is yes, is there a compelling reason why they wouldn't want to support binary methods as well? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From p.f.moore at gmail.com Fri Jun 6 20:36:03 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 6 Jun 2008 19:36:03 +0100 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: <79990c6b0806061136h67d54fd7yaab4db9d216e1d6c@mail.gmail.com> 2008/6/6 Guido van Rossum : > On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger wrote: >> Does anyone actually need an int lookalike with binary methods but >> cannot just inherit from int? > > Does anyone actually need an int lookalike with operations like +, - > etc. but cannot just inherit from int? If the answer is yes, is there > a compelling reason why they wouldn't want to support binary methods > as well? I have no vested interest either way, but I think someone mentioned creating a long int equivalent based on Decimal earlier in the thread. Wrappers for the gmp library might also want to do this. It's hardly the world's biggest use case, though... Paul. From aleaxit at gmail.com Fri Jun 6 20:40:01 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Fri, 6 Jun 2008 11:40:01 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum wrote: > On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger wrote: >> Does anyone actually need an int lookalike with binary methods but >> cannot just inherit from int? > > Does anyone actually need an int lookalike with operations like +, - > etc. but cannot just inherit from int? If the answer is yes, is there > a compelling reason why they wouldn't want to support binary methods > as well? Yes, there's a use case for implementing long integers as arrays of decimal digits -- addition is roughly as efficient as for binary integers (x86 chips still have instructions to help with that), and emitting as decimal digits is MUCH more efficient of course -- so if I/O in decimal form is the most common operation, with a little arithmetic (particularly sums), you could gain performance; binary operations, however, would be as inefficient as decimal form conversion is for ordinary binary ints, and not needed for the typical applications that would use these "decimal coded integers" (accounting), so why not save the implementer of such an extension from having to write that unneeded and slow extra code? Alex From jjl at pobox.com Fri Jun 6 21:12:27 2008 From: jjl at pobox.com (John J Lee) Date: Fri, 6 Jun 2008 20:12:27 +0100 (BST) Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> Message-ID: On Thu, 5 Jun 2008, Benjamin Peterson wrote: > - reorganizing the tests into separate directories Why this one? John From musiccomposition at gmail.com Fri Jun 6 21:20:24 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 6 Jun 2008 14:20:24 -0500 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> Message-ID: <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> On Fri, Jun 6, 2008 at 2:12 PM, John J Lee wrote: > On Thu, 5 Jun 2008, Benjamin Peterson wrote: > >> - reorganizing the tests into separate directories > > Why this one? I always find it hard to find a test I'm looking for in a directory with 365 different tests in it. Also grouping tests by function will hopefully help reduce duplication and it more intuitive. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From jflatow at northwestern.edu Fri Jun 6 22:10:58 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Fri, 6 Jun 2008 15:10:58 -0500 Subject: [Python-Dev] Assumed reflexivity of rich comparison operators Message-ID: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu> Hi all, PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the fourth clause of the proposed resolutions to concerns: "The reflexivity rules *are* assumed by Python. Thus, the interpreter may swap y>x with x=x with x<=y, and may swap the arguments of x==y and x!=y." However, if this is the case, why does Python allow the definition of both pairs of __le__, __ge__ and __lt__, __gt__ for a single class, since users have no guarantee over which will be called? Currently, if I do not want x >= y to mean the same thing as y <= x (and believe it or not I believe I have a good use case for doing this), there is no reliable way of doing this. However, if the decision is to not allow users to do this at all using operators (and force them to create specially named methods), what is the point of allowing the definition of both? It seems very confusing to me (and indeed I was at first very confused what was going on), to tempt users to be able to define both but give no promise that if they do, the appropriate one will be called. Does anyone have a good explanation for this? Thanks! jared From guido at python.org Fri Jun 6 22:18:13 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jun 2008 13:18:13 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: On Fri, Jun 6, 2008 at 11:40 AM, Alex Martelli wrote: > On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum wrote: >> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger wrote: >>> Does anyone actually need an int lookalike with binary methods but >>> cannot just inherit from int? >> >> Does anyone actually need an int lookalike with operations like +, - >> etc. but cannot just inherit from int? If the answer is yes, is there >> a compelling reason why they wouldn't want to support binary methods >> as well? > > Yes, there's a use case for implementing long integers as arrays of > decimal digits -- addition is roughly as efficient as for binary > integers (x86 chips still have instructions to help with that), and > emitting as decimal digits is MUCH more efficient of course -- so if > I/O in decimal form is the most common operation, with a little > arithmetic (particularly sums), you could gain performance; I tried this with ABC in '83. We didn't see any of the hoped-for benefits though. That's why Python has binary long integers. :-) > binary > operations, however, would be as inefficient as decimal form > conversion is for ordinary binary ints, and not needed for the typical > applications that would use these "decimal coded integers" > (accounting), so why not save the implementer of such an extension > from having to write that unneeded and slow extra code? You could just raise an exception. This is common in Java when an Interface requires you implement a method you can't. Or use virtual inheritance from the Integral class and leave them unimplemented. See if anyone cares. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 6 22:24:32 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jun 2008 13:24:32 -0700 Subject: [Python-Dev] Assumed reflexivity of rich comparison operators In-Reply-To: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu> References: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu> Message-ID: On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow wrote: > PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the fourth > clause of the proposed resolutions to concerns: > > "The reflexivity rules *are* assumed by Python. Thus, the interpreter may > swap y>x with x=x with x<=y, and may swap the arguments of x==y and > x!=y." > > However, if this is the case, why does Python allow the definition of both > pairs of __le__, __ge__ and __lt__, __gt__ for a single class, since users > have no guarantee over which will be called? Currently, if I do not want x >>= y to mean the same thing as y <= x (and believe it or not I believe I > have a good use case for doing this), I find it hard to believe that your users will be happy with that though. :-) > there is no reliable way of doing this. Does it help if I tell you that for "x y" we always try x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the case where y is an instance of a subclass of the class of x? > However, if the decision is to not allow users to do this at all using > operators (and force them to create specially named methods), what is the > point of allowing the definition of both? The same reason we allow (require) you to define __add__ and __radd_. It is quite possible that for any particular binary operation "x y", the class of x doesn't know how to implement it, and then the class of y is tried with the reversed operation. > It seems very confusing to me (and > indeed I was at first very confused what was going on), to tempt users to be > able to define both but give no promise that if they do, the appropriate one > will be called. Does anyone have a good explanation for this? I have explained it as well as I can. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From cesare at pronto.it Fri Jun 6 22:18:44 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Fri, 06 Jun 2008 22:18:44 +0200 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: In data 06 giugno 2008 alle ore 20:40:01, Alex Martelli ha scritto: > On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum wrote: >> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger wrote: >>> Does anyone actually need an int lookalike with binary methods but >>> cannot just inherit from int? >> >> Does anyone actually need an int lookalike with operations like +, - >> etc. but cannot just inherit from int? If the answer is yes, is there >> a compelling reason why they wouldn't want to support binary methods >> as well? > > Yes, there's a use case for implementing long integers as arrays of > decimal digits -- addition is roughly as efficient as for binary > integers (x86 chips still have instructions to help with that), and > emitting as decimal digits is MUCH more efficient of course -- so if > I/O in decimal form is the most common operation, with a little > arithmetic (particularly sums), you could gain performance; binary > operations, however, would be as inefficient as decimal form > conversion is for ordinary binary ints, and not needed for the typical > applications that would use these "decimal coded integers" > (accounting), so why not save the implementer of such an extension > from having to write that unneeded and slow extra code? > > > Alex I don't know if you are talking about BCD numbers, but they are quite inefficient and slow in x86 architecture. There are instructions only to add and subtract packed BCD numbers which uses just two decimal digits (packed in two nibbles into a single byte). For unpacked BCDs, there are instructions to add, subtract, multiply and divide numbers, but which uses only one digit at the time. So using packed BCDs to store 8 decimal digits in 32 bits, for example, requires 4 instructions to make addictions or subractions, plus the required shift & mask instructions to put every couple digits into the AL register to execute BCD operations. Unpacked BCDs need double of them. Also, these instructions still use microcode to execute on modern processors, slowing down the execution pipeline (most of the simpler instructions do not require microcode, and execute "directly"). Last but not least, on x86-64 architecture BCD instructions were completely removed from the ISA; opcodes are assigned to new instructions. Obviously, binary operations can be performed twice faster thanks to the 64 bit registers and ALUs. The only practical advantage on using BCD numbers is the conversion-to-string operation, which can be done faster than binary numbers. Binary addition, subtraction, multiplication and division are greatly faster than BCD ones, and should be the preferred way to do integer math. Cesare From python at rcn.com Fri Jun 6 22:33:05 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 6 Jun 2008 13:33:05 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: New idea! I missed an obvious solution. Let the binary methods in Integral be mixins instead of abstract methods. That minimizes the burden on the class implementer while providing maximum support for clients. class Integral(Rational): ... def __lshift__(self, other): """self << other""" return long(self) << long(other) def __xor__(self, other): """self ^ other""" return long(self) ^ long(other) I worried a bit about changing type, but this kind of thing is already baked into numbers.py: @property def imag(self): """Real numbers have no imaginary component.""" return 0 @property def denominator(self): """Integers have a denominator of 1.""" return 1 Raymond ----- Original Message ----- From: "Alex Martelli" To: "Guido van Rossum" Cc: "Raymond Hettinger" ; Sent: Friday, June 06, 2008 11:40 AM Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC > On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum wrote: >> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger wrote: >>> Does anyone actually need an int lookalike with binary methods but >>> cannot just inherit from int? >> >> Does anyone actually need an int lookalike with operations like +, - >> etc. but cannot just inherit from int? If the answer is yes, is there >> a compelling reason why they wouldn't want to support binary methods >> as well? > > Yes, there's a use case for implementing long integers as arrays of > decimal digits -- addition is roughly as efficient as for binary > integers (x86 chips still have instructions to help with that), and > emitting as decimal digits is MUCH more efficient of course -- so if > I/O in decimal form is the most common operation, with a little > arithmetic (particularly sums), you could gain performance; binary > operations, however, would be as inefficient as decimal form > conversion is for ordinary binary ints, and not needed for the typical > applications that would use these "decimal coded integers" > (accounting), so why not save the implementer of such an extension > from having to write that unneeded and slow extra code? > > > Alex From jflatow at northwestern.edu Fri Jun 6 22:44:14 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Fri, 6 Jun 2008 15:44:14 -0500 Subject: [Python-Dev] Assumed reflexivity of rich comparison operators In-Reply-To: References: <57C82822-A384-47F3-9E2C-9730E1471B43@northwestern.edu> Message-ID: On Jun 6, 2008, at 3:24 PM, Guido van Rossum wrote: > On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow > wrote: >> PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the >> fourth >> clause of the proposed resolutions to concerns: >> >> "The reflexivity rules *are* assumed by Python. Thus, the >> interpreter may >> swap y>x with x=x with x<=y, and may swap the arguments of >> x==y and >> x!=y." >> >> However, if this is the case, why does Python allow the definition >> of both >> pairs of __le__, __ge__ and __lt__, __gt__ for a single class, >> since users >> have no guarantee over which will be called? Currently, if I do not >> want x >>> = y to mean the same thing as y <= x (and believe it or not I >>> believe I >> have a good use case for doing this), > > I find it hard to believe that your users will be happy with that > though. :-) In this case, I am my users and I would be very happy with it (but I won't try to justify it :). >> there is no reliable way of doing this. > > Does it help if I tell you that for "x y" we always try > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the > case where y is an instance of a subclass of the class of x? Yes, actually that explains quite a bit, and now I see that is the case I am dealing with. y is an instance of a subclass of x, but the class of x is the one that defines both the binops. I suppose it is too much to ask to only call the __reverse_binop__ if the subclass overrides it. >> However, if the decision is to not allow users to do this at all >> using >> operators (and force them to create specially named methods), what >> is the >> point of allowing the definition of both? > > The same reason we allow (require) you to define __add__ and __radd_. > It is quite possible that for any particular binary operation "x > y", the class of x doesn't know how to implement it, and then > the class of y is tried with the reversed operation. > >> It seems very confusing to me (and >> indeed I was at first very confused what was going on), to tempt >> users to be >> able to define both but give no promise that if they do, the >> appropriate one >> will be called. Does anyone have a good explanation for this? > > I have explained it as well as I can. Thanks very much, at least that is enough information to work around reliably. jared From guido at python.org Fri Jun 6 22:47:01 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jun 2008 13:47:01 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: Make that int() instead of long() and I'm okay with it. On Fri, Jun 6, 2008 at 1:33 PM, Raymond Hettinger wrote: > New idea! I missed an obvious solution. Let the binary methods in Integral > be mixins instead of abstract methods. That minimizes the burden on the > class implementer while providing maximum support for clients. > > class Integral(Rational): > ... > def __lshift__(self, other): > """self << other""" > return long(self) << long(other) > def __xor__(self, other): > """self ^ other""" > return long(self) ^ long(other) > > I worried a bit about changing type, but this kind of thing is already baked > into numbers.py: > > @property > def imag(self): > """Real numbers have no imaginary component.""" > return 0 > > @property > def denominator(self): > """Integers have a denominator of 1.""" > return 1 > > Raymond > > > ----- Original Message ----- From: "Alex Martelli" > To: "Guido van Rossum" > Cc: "Raymond Hettinger" ; > Sent: Friday, June 06, 2008 11:40 AM > Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC > > >> On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum >> wrote: >>> >>> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger wrote: >>>> >>>> Does anyone actually need an int lookalike with binary methods but >>>> cannot just inherit from int? >>> >>> Does anyone actually need an int lookalike with operations like +, - >>> etc. but cannot just inherit from int? If the answer is yes, is there >>> a compelling reason why they wouldn't want to support binary methods >>> as well? >> >> Yes, there's a use case for implementing long integers as arrays of >> decimal digits -- addition is roughly as efficient as for binary >> integers (x86 chips still have instructions to help with that), and >> emitting as decimal digits is MUCH more efficient of course -- so if >> I/O in decimal form is the most common operation, with a little >> arithmetic (particularly sums), you could gain performance; binary >> operations, however, would be as inefficient as decimal form >> conversion is for ordinary binary ints, and not needed for the typical >> applications that would use these "decimal coded integers" >> (accounting), so why not save the implementer of such an extension >> from having to write that unneeded and slow extra code? >> >> >> Alex > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Jun 6 23:13:49 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 6 Jun 2008 14:13:49 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC References: <90B6514494134BEAB0191E8423A986D6@RaymondLaptop1> <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> Message-ID: <1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1> From: "Guido van Rossum" > Make that int() instead of long() and I'm okay with it. Does anyone know why Integral says that __long__ is a required abstract method, but not __int__? Likewise, why is index() defined as long(self) instead of int(self)? There may be some design nuance that I'm not seeing. Raymond From martin at v.loewis.de Fri Jun 6 23:30:20 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Jun 2008 23:30:20 +0200 Subject: [Python-Dev] Modules for 2.6 inclusion In-Reply-To: References: Message-ID: <4849AC6C.7020004@v.loewis.de> > I created an issue 1 week ago (http://bugs.python.org/issue2983) > suggesting the addition of the ttk module to lib-tk, and to the new > tkinter package. Is there any chance to this be accepted for Python > 2.6 ? Is it complete? In principle, it's for the mentor to decide (who would need to decide before the first beta, of course - afterwards it's for the release manager to decide). Regards, Martin From martin at v.loewis.de Fri Jun 6 23:32:40 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Jun 2008 23:32:40 +0200 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> Message-ID: <4849ACF8.20305@v.loewis.de> >>> - reorganizing the tests into separate directories >> Why this one? > > I always find it hard to find a test I'm looking for in a directory > with 365 different tests in it. Also grouping tests by function will > hopefully help reduce duplication and it more intuitive. Still, I don't think this should be done. Flat is better than nested, and adding hierarchy will make it *more* difficult to find anything (except perhaps for the one person who did the rearrangement). I personally use grep to find the place where to add a new test. Regards, Martin From ggpolo at gmail.com Fri Jun 6 23:40:41 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 6 Jun 2008 18:40:41 -0300 Subject: [Python-Dev] Modules for 2.6 inclusion In-Reply-To: <4849AC6C.7020004@v.loewis.de> References: <4849AC6C.7020004@v.loewis.de> Message-ID: 2008/6/6 "Martin v. L?wis" : >> I created an issue 1 week ago (http://bugs.python.org/issue2983) >> suggesting the addition of the ttk module to lib-tk, and to the new >> tkinter package. Is there any chance to this be accepted for Python >> 2.6 ? > > Is it complete? In principle, it's for the mentor to decide (who > would need to decide before the first beta, of course - afterwards > it's for the release manager to decide). > It is complete Martin. But I will wait for Fredrik's decision then, not sure if it will be soon enough tho. > Regards, > Martin > -- -- Guilherme H. Polo Goncalves From jjt009 at yahoo.com Fri Jun 6 23:40:24 2008 From: jjt009 at yahoo.com (James Thomas) Date: Fri, 6 Jun 2008 14:40:24 -0700 (PDT) Subject: [Python-Dev] New Developer In-Reply-To: Message-ID: <443348.68183.qm@web30903.mail.mud.yahoo.com> Hi All, I'm a new developer with Python. I have a solid knowledge of both Python and C. Are there any tasks that would be suitable for a beginner to the Python codebase? I haven't been able to find a list of such tasks on python.org or bugs.python.org. Thanks, James Thomas --- On Fri, 6/6/08, python-dev-request at python.org <python-dev-request at python.org> wrote: From: python-dev-request at python.org <python-dev-request at python.org> Subject: Python-Dev Digest, Vol 59, Issue 24 To: python-dev at python.org Date: Friday, June 6, 2008, 2:30 PM Send Python-Dev mailing list submissions to python-dev at python.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request at python.org You can reach the person managing the list at python-dev-owner at python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..."Today's Topics: 1. Re: Assumed reflexivity of rich comparison operators (Guido van Rossum) 2. Re: Mini-Pep: Simplifying the Integral ABC (Cesare Di Mauro) 3. Re: Mini-Pep: Simplifying the Integral ABC (Raymond Hettinger) 4. Re: Assumed reflexivity of rich comparison operators (Jared Flatow) 5. Re: Mini-Pep: Simplifying the Integral ABC (Guido van Rossum) 6. Re: Mini-Pep: Simplifying the Integral ABC (Raymond Hettinger) 7. Re: Modules for 2.6 inclusion (Martin v. L?wis)On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow <jflatow at northwestern.edu> wrote: > PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the fourth > clause of the proposed resolutions to concerns: > > "The reflexivity rules *are* assumed by Python. Thus, the interpreter may > swap y>x with x<y, y>=x with x<=y, and may swap the arguments of x==y and > x!=y." > > However, if this is the case, why does Python allow the definition of both > pairs of __le__, __ge__ and __lt__, __gt__ for a single class, since users > have no guarantee over which will be called? Currently, if I do not want x >>= y to mean the same thing as y <= x (and believe it or not I believe I > have a good use case for doing this), I find it hard to believe that your users will be happy with that though. :-) > there is no reliable way of doing this. Does it help if I tell you that for "x <binop> y" we always try x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the case where y is an instance of a subclass of the class of x? > However, if the decision is to not allow users to do this at all using > operators (and force them to create specially named methods), what is the > point of allowing the definition of both? The same reason we allow (require) you to define __add__ and __radd_. It is quite possible that for any particular binary operation "x <binop> y", the class of x doesn't know how to implement it, and then the class of y is tried with the reversed operation. > It seems very confusing to me (and > indeed I was at first very confused what was going on), to tempt users to be > able to define both but give no promise that if they do, the appropriate one > will be called. Does anyone have a good explanation for this? I have explained it as well as I can. -- --Guido van Rossum (home page: http://www.python.org/~guido/)In data 06 giugno 2008 alle ore 20:40:01, Alex Martelli <aleaxit at gmail.com> ha scritto: > On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org> wrote: >> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote: >>> Does anyone actually need an int lookalike with binary methods but >>> cannot just inherit from int? >> >> Does anyone actually need an int lookalike with operations like +, - >> etc. but cannot just inherit from int? If the answer is yes, is there >> a compelling reason why they wouldn't want to support binary methods >> as well? > > Yes, there's a use case for implementing long integers as arrays of > decimal digits -- addition is roughly as efficient as for binary > integers (x86 chips still have instructions to help with that), and > emitting as decimal digits is MUCH more efficient of course -- so if > I/O in decimal form is the most common operation, with a little > arithmetic (particularly sums), you could gain performance; binary > operations, however, would be as inefficient as decimal form > conversion is for ordinary binary ints, and not needed for the typical > applications that would use these "decimal coded integers" > (accounting), so why not save the implementer of such an extension > from having to write that unneeded and slow extra code? > > > Alex I don't know if you are talking about BCD numbers, but they are quite inefficient and slow in x86 architecture. There are instructions only to add and subtract packed BCD numbers which uses just two decimal digits (packed in two nibbles into a single byte). For unpacked BCDs, there are instructions to add, subtract, multiply and divide numbers, but which uses only one digit at the time. So using packed BCDs to store 8 decimal digits in 32 bits, for example, requires 4 instructions to make addictions or subractions, plus the required shift & mask instructions to put every couple digits into the AL register to execute BCD operations. Unpacked BCDs need double of them. Also, these instructions still use microcode to execute on modern processors, slowing down the execution pipeline (most of the simpler instructions do not require microcode, and execute "directly"). Last but not least, on x86-64 architecture BCD instructions were completely removed from the ISA; opcodes are assigned to new instructions. Obviously, binary operations can be performed twice faster thanks to the 64 bit registers and ALUs. The only practical advantage on using BCD numbers is the conversion-to-string operation, which can be done faster than binary numbers. Binary addition, subtraction, multiplication and division are greatly faster than BCD ones, and should be the preferred way to do integer math. CesareNew idea! I missed an obvious solution. Let the binary methods in Integral be mixins instead of abstract methods. That minimizes the burden on the class implementer while providing maximum support for clients. class Integral(Rational): ... def __lshift__(self, other): """self << other""" return long(self) << long(other) def __xor__(self, other): """self ^ other""" return long(self) ^ long(other) I worried a bit about changing type, but this kind of thing is already baked into numbers.py: @property def imag(self): """Real numbers have no imaginary component.""" return 0 @property def denominator(self): """Integers have a denominator of 1.""" return 1 Raymond ----- Original Message ----- From: "Alex Martelli" <aleaxit at gmail.com> To: "Guido van Rossum" <guido at python.org> Cc: "Raymond Hettinger" <python at rcn.com>; <python-dev at python.org> Sent: Friday, June 06, 2008 11:40 AM Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC > On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org> wrote: >> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote: >>> Does anyone actually need an int lookalike with binary methods but >>> cannot just inherit from int? >> >> Does anyone actually need an int lookalike with operations like +, - >> etc. but cannot just inherit from int? If the answer is yes, is there >> a compelling reason why they wouldn't want to support binary methods >> as well? > > Yes, there's a use case for implementing long integers as arrays of > decimal digits -- addition is roughly as efficient as for binary > integers (x86 chips still have instructions to help with that), and > emitting as decimal digits is MUCH more efficient of course -- so if > I/O in decimal form is the most common operation, with a little > arithmetic (particularly sums), you could gain performance; binary > operations, however, would be as inefficient as decimal form > conversion is for ordinary binary ints, and not needed for the typical > applications that would use these "decimal coded integers" > (accounting), so why not save the implementer of such an extension > from having to write that unneeded and slow extra code? > > > AlexOn Jun 6, 2008, at 3:24 PM, Guido van Rossum wrote: > On Fri, Jun 6, 2008 at 1:10 PM, Jared Flatow > <jflatow at northwestern.edu> wrote: >> PEP 207 (http://www.python.org/dev/peps/pep-0207/) states in the >> fourth >> clause of the proposed resolutions to concerns: >> >> "The reflexivity rules *are* assumed by Python. Thus, the >> interpreter may >> swap y>x with x<y, y>=x with x<=y, and may swap the arguments of >> x==y and >> x!=y." >> >> However, if this is the case, why does Python allow the definition >> of both >> pairs of __le__, __ge__ and __lt__, __gt__ for a single class, >> since users >> have no guarantee over which will be called? Currently, if I do not >> want x >>> = y to mean the same thing as y <= x (and believe it or not I >>> believe I >> have a good use case for doing this), > > I find it hard to believe that your users will be happy with that > though. :-) In this case, I am my users and I would be very happy with it (but I won't try to justify it :). >> there is no reliable way of doing this. > > Does it help if I tell you that for "x <binop> y" we always try > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the > case where y is an instance of a subclass of the class of x? Yes, actually that explains quite a bit, and now I see that is the case I am dealing with. y is an instance of a subclass of x, but the class of x is the one that defines both the binops. I suppose it is too much to ask to only call the __reverse_binop__ if the subclass overrides it. >> However, if the decision is to not allow users to do this at all >> using >> operators (and force them to create specially named methods), what >> is the >> point of allowing the definition of both? > > The same reason we allow (require) you to define __add__ and __radd_. > It is quite possible that for any particular binary operation "x > <binop> y", the class of x doesn't know how to implement it, and then > the class of y is tried with the reversed operation. > >> It seems very confusing to me (and >> indeed I was at first very confused what was going on), to tempt >> users to be >> able to define both but give no promise that if they do, the >> appropriate one >> will be called. Does anyone have a good explanation for this? > > I have explained it as well as I can. Thanks very much, at least that is enough information to work around reliably. jaredMake that int() instead of long() and I'm okay with it. On Fri, Jun 6, 2008 at 1:33 PM, Raymond Hettinger <python at rcn.com> wrote: > New idea! I missed an obvious solution. Let the binary methods in Integral > be mixins instead of abstract methods. That minimizes the burden on the > class implementer while providing maximum support for clients. > > class Integral(Rational): > ... > def __lshift__(self, other): > """self << other""" > return long(self) << long(other) > def __xor__(self, other): > """self ^ other""" > return long(self) ^ long(other) > > I worried a bit about changing type, but this kind of thing is already baked > into numbers.py: > > @property > def imag(self): > """Real numbers have no imaginary component.""" > return 0 > > @property > def denominator(self): > """Integers have a denominator of 1.""" > return 1 > > Raymond > > > ----- Original Message ----- From: "Alex Martelli" <aleaxit at gmail.com> > To: "Guido van Rossum" <guido at python.org> > Cc: "Raymond Hettinger" <python at rcn.com>; <python-dev at python.org> > Sent: Friday, June 06, 2008 11:40 AM > Subject: Re: [Python-Dev] Mini-Pep: Simplifying the Integral ABC > > >> On Fri, Jun 6, 2008 at 11:01 AM, Guido van Rossum <guido at python.org> >> wrote: >>> >>> On Thu, Jun 5, 2008 at 8:45 PM, Raymond Hettinger <python at rcn.com> wrote: >>>> >>>> Does anyone actually need an int lookalike with binary methods but >>>> cannot just inherit from int? >>> >>> Does anyone actually need an int lookalike with operations like +, - >>> etc. but cannot just inherit from int? If the answer is yes, is there >>> a compelling reason why they wouldn't want to support binary methods >>> as well? >> >> Yes, there's a use case for implementing long integers as arrays of >> decimal digits -- addition is roughly as efficient as for binary >> integers (x86 chips still have instructions to help with that), and >> emitting as decimal digits is MUCH more efficient of course -- so if >> I/O in decimal form is the most common operation, with a little >> arithmetic (particularly sums), you could gain performance; binary >> operations, however, would be as inefficient as decimal form >> conversion is for ordinary binary ints, and not needed for the typical >> applications that would use these "decimal coded integers" >> (accounting), so why not save the implementer of such an extension >> from having to write that unneeded and slow extra code? >> >> >> Alex > > -- --Guido van Rossum (home page: http://www.python.org/~guido/)From: "Guido van Rossum" <guido at python.org> > Make that int() instead of long() and I'm okay with it. Does anyone know why Integral says that __long__ is a required abstract method, but not __int__? Likewise, why is index() defined as long(self) instead of int(self)? There may be some design nuance that I'm not seeing. Raymond> I created an issue 1 week ago (http://bugs.python.org/issue2983) > suggesting the addition of the ttk module to lib-tk, and to the new > tkinter package. Is there any chance to this be accepted for Python > 2.6 ? Is it complete? In principle, it's for the mentor to decide (who would need to decide before the first beta, of course - afterwards it's for the release manager to decide). Regards, Martin_______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From facundobatista at gmail.com Sat Jun 7 00:10:00 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 6 Jun 2008 19:10:00 -0300 Subject: [Python-Dev] New Developer In-Reply-To: <443348.68183.qm@web30903.mail.mud.yahoo.com> References: <443348.68183.qm@web30903.mail.mud.yahoo.com> Message-ID: 2008/6/6 James Thomas : > Hi All, > I'm a new developer with Python. I have a solid knowledge of both Python and > C. Are there any tasks that would be suitable for a beginner to the Python > codebase? I haven't been able to find a list of such tasks on python.org or > bugs.python.org. Welcome! The issue tracker [1] is *the* place to find such tasks! Note that there're some tasks tagged as easy [2], take a look! Next week we'll have a Python Bug Weekend [3], it's a good moment to gain speed. (ah, what is not welcomed are HTML mails) Regards, [1] http://bugs.python.org/ [2] http://www.taniquetil.com.ar/cgi-bin/pytickets.py?nropag=0&priority=0&severity=0&component=0&version=0&keyword=6 [3] http://wiki.python.org/moin/PythonBugDay -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From guido at python.org Sat Jun 7 00:23:12 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Jun 2008 15:23:12 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1> References: <2745D7EB7C064B16A88E433588021756@RaymondLaptop1> <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> <1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1> Message-ID: Both of these seem 2.6-specific quirks. Those lines wereJeffrey's; maybe he remembers? I'm guessing that adding __long__ was done since 2.6 supports it, and the removal of __int__ was an oversight. I also think that there's no reason to change __index__ to call long(); int() will automatically return a long as needed. Maybe changing __long__ back to __int__ is also harmless. On Fri, Jun 6, 2008 at 2:13 PM, Raymond Hettinger wrote: > From: "Guido van Rossum" >> >> Make that int() instead of long() and I'm okay with it. > > Does anyone know why Integral says that __long__ is a required abstract > method, but not __int__? > > Likewise, why is index() defined as long(self) instead of int(self)? > > There may be some design nuance that I'm not seeing. > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Sat Jun 7 00:51:10 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 06 Jun 2008 18:51:10 -0400 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849ACF8.20305@v.loewis.de> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> Message-ID: <4849BF5E.9020205@trueblade.com> Martin v. L?wis wrote: >>>> - reorganizing the tests into separate directories >>> Why this one? >> I always find it hard to find a test I'm looking for in a directory >> with 365 different tests in it. Also grouping tests by function will >> hopefully help reduce duplication and it more intuitive. > > Still, I don't think this should be done. Flat is better than nested, > and adding hierarchy will make it *more* difficult to find anything > (except perhaps for the one person who did the rearrangement). > > I personally use grep to find the place where to add a new test. I agree. There's not much chance I'd know which directory a test for a given piece of functionality is in, so instead of grepping in a single directory, I'd have to grep in all of them. Definitely more hassle. Eric. From musiccomposition at gmail.com Sat Jun 7 00:52:05 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 6 Jun 2008 17:52:05 -0500 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849BF5E.9020205@trueblade.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com> Message-ID: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> On Fri, Jun 6, 2008 at 5:51 PM, Eric Smith wrote: > Martin v. L?wis wrote: >>> >>> I always find it hard to find a test I'm looking for in a directory >>> with 365 different tests in it. Also grouping tests by function will >>> hopefully help reduce duplication and it more intuitive. >> >> Still, I don't think this should be done. Flat is better than nested, >> and adding hierarchy will make it *more* difficult to find anything >> (except perhaps for the one person who did the rearrangement). >> >> I personally use grep to find the place where to add a new test. > > I agree. There's not much chance I'd know which directory a test for a > given piece of functionality is in, so instead of grepping in a single > directory, I'd have to grep in all of them. Definitely more hassle. Really? Given the choice between core_language (divided into syntax and builtins) and stdlib you wouldn't know where to look? > > Eric. > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From eric+python-dev at trueblade.com Sat Jun 7 00:58:06 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 06 Jun 2008 18:58:06 -0400 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com> <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> Message-ID: <4849C0FE.3070208@trueblade.com> Benjamin Peterson wrote: > On Fri, Jun 6, 2008 at 5:51 PM, Eric Smith > wrote: >> Martin v. L?wis wrote: >>>> I always find it hard to find a test I'm looking for in a directory >>>> with 365 different tests in it. Also grouping tests by function will >>>> hopefully help reduce duplication and it more intuitive. >>> Still, I don't think this should be done. Flat is better than nested, >>> and adding hierarchy will make it *more* difficult to find anything >>> (except perhaps for the one person who did the rearrangement). >>> >>> I personally use grep to find the place where to add a new test. >> I agree. There's not much chance I'd know which directory a test for a >> given piece of functionality is in, so instead of grepping in a single >> directory, I'd have to grep in all of them. Definitely more hassle. > > Really? Given the choice between core_language (divided into syntax > and builtins) and stdlib you wouldn't know where to look? Really. I wouldn't bother thinking about it. I'd just grep. Eric. From facundobatista at gmail.com Sat Jun 7 00:58:48 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 6 Jun 2008 19:58:48 -0300 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> Message-ID: 2008/6/6 Benjamin Peterson : > I always find it hard to find a test I'm looking for in a directory > with 365 different tests in it. Also grouping tests by function will > hopefully help reduce duplication and it more intuitive. Intuitive to who taking into account which grouping? Start to group using *your* intuitive way, and a lot of people will find it unintuitive. -1 to group the tests. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From jnoller at gmail.com Sat Jun 7 01:02:46 2008 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 6 Jun 2008 19:02:46 -0400 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com> <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> Message-ID: <49570F5A-35A9-4349-A26D-F1CF8FE14767@gmail.com> On Jun 6, 2008, at 6:52 PM, "Benjamin Peterson" wrote: > On Fri, Jun 6, 2008 at 5:51 PM, Eric Smith > wrote: >> Martin v. L?wis wrote: >>>> >>>> I always find it hard to find a test I'm looking for in a directory >>>> with 365 different tests in it. Also grouping tests by function >>>> will >>>> hopefully help reduce duplication and it more intuitive. >>> >>> Still, I don't think this should be done. Flat is better than >>> nested, >>> and adding hierarchy will make it *more* difficult to find anything >>> (except perhaps for the one person who did the rearrangement). >>> >>> I personally use grep to find the place where to add a new test. >> >> I agree. There's not much chance I'd know which directory a test >> for a >> given piece of functionality is in, so instead of grepping in a >> single >> directory, I'd have to grep in all of them. Definitely more hassle. > > Really? Given the choice between core_language (divided into syntax > and builtins) and stdlib you wouldn't know where to look? > >> >> Eric. >> > > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > ______________________________ I'd tend to agree with ben here - a single directory for all tests does scale well and makes the purpose of each test unclear. Besides, grep -r could traverse the directory tree. +1 nested :) Jesse From jnoller at gmail.com Sat Jun 7 01:04:40 2008 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 6 Jun 2008 19:04:40 -0400 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> Message-ID: How does 1 directory scale when one day you have possibly thousands of tests? On Jun 6, 2008, at 6:58 PM, "Facundo Batista" wrote: > 2008/6/6 Benjamin Peterson : > >> I always find it hard to find a test I'm looking for in a directory >> with 365 different tests in it. Also grouping tests by function will >> hopefully help reduce duplication and it more intuitive. > > Intuitive to who taking into account which grouping? Start to group > using *your* intuitive way, and a lot of people will find it > unintuitive. > > -1 to group the tests. > > Regards, > > -- > . Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com From martin at v.loewis.de Sat Jun 7 01:19:52 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 07 Jun 2008 01:19:52 +0200 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com> <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> Message-ID: <4849C618.8000209@v.loewis.de> > Really? Given the choice between core_language (divided into syntax > and builtins) and stdlib you wouldn't know where to look? Exactly so. I would not normally guess that the builtins belong to the core language - they belong to the stdlib, IMO. "core language" is just syntax to me, perhaps plus the data types that have literals or displays, and perhaps plus a few selected exceptions. (just to test your own intuition: when reversed() got added, was that more of a language change than when sys.meta_path got added?) Regards, Martin From martin at v.loewis.de Sat Jun 7 01:26:38 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 07 Jun 2008 01:26:38 +0200 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> Message-ID: <4849C7AE.4070408@v.loewis.de> > How does 1 directory scale when one day you have possibly thousands of > tests? I find this a theoretical question. It took 18 years to arrive at 500 test files. Assuming a linear growth, we get 1000 tests in 2025, and 2000 tests in 2060. People can worry about reorganizing them then. Regards, Martin From rrr at ronadam.com Sat Jun 7 02:30:10 2008 From: rrr at ronadam.com (Ron Adam) Date: Fri, 06 Jun 2008 19:30:10 -0500 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849C7AE.4070408@v.loewis.de> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849C7AE.4070408@v.loewis.de> Message-ID: <4849D692.1050004@ronadam.com> Martin v. L?wis wrote: >> How does 1 directory scale when one day you have possibly thousands of >> tests? > > I find this a theoretical question. It took 18 years to arrive at 500 > test files. Assuming a linear growth, we get 1000 tests in 2025, and > 2000 tests in 2060. People can worry about reorganizing them then. Personally I'd like to see packages have their own test directory. This keeps things related to each other together. Top level modules of course would have their tests in the top level test directory as they are now. I don't see any need to subdivide tests further than that. From ggpolo at gmail.com Sat Jun 7 02:40:15 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 6 Jun 2008 21:40:15 -0300 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849D692.1050004@ronadam.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849C7AE.4070408@v.loewis.de> <4849D692.1050004@ronadam.com> Message-ID: 2008/6/6 Ron Adam : > > > Martin v. L?wis wrote: >>> >>> How does 1 directory scale when one day you have possibly thousands of >>> tests? >> >> I find this a theoretical question. It took 18 years to arrive at 500 >> test files. Assuming a linear growth, we get 1000 tests in 2025, and >> 2000 tests in 2060. People can worry about reorganizing them then. > > > Personally I'd like to see packages have their own test directory. This > keeps things related to each other together. Top level modules of course > would have their tests in the top level test directory as they are now. > I really dislike having a test directory inside a python package. You have my -1 on that idea. > I don't see any need to subdivide tests further than that. > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com > -- -- Guilherme H. Polo Goncalves From python at rcn.com Sat Jun 7 03:21:07 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 6 Jun 2008 18:21:07 -0700 Subject: [Python-Dev] on Python's tests (and making them better) References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com><1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com><4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com> <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> Message-ID: >>> Still, I don't think this should be done. Flat is better than nested, >>> and adding hierarchy will make it *more* difficult to find anything >>> (except perhaps for the one person who did the rearrangement). Yes. Grep is your friend. Raymond From rrr at ronadam.com Sat Jun 7 02:30:10 2008 From: rrr at ronadam.com (Ron Adam) Date: Fri, 06 Jun 2008 19:30:10 -0500 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849C7AE.4070408@v.loewis.de> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849C7AE.4070408@v.loewis.de> Message-ID: <4849D692.1050004@ronadam.com> Martin v. L?wis wrote: >> How does 1 directory scale when one day you have possibly thousands of >> tests? > > I find this a theoretical question. It took 18 years to arrive at 500 > test files. Assuming a linear growth, we get 1000 tests in 2025, and > 2000 tests in 2060. People can worry about reorganizing them then. Personally I'd like to see packages have their own test directory. This keeps things related to each other together. Top level modules of course would have their tests in the top level test directory as they are now. I don't see any need to subdivide tests further than that. From steve at pearwood.info Sat Jun 7 05:24:21 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 7 Jun 2008 13:24:21 +1000 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849ACF8.20305@v.loewis.de> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> Message-ID: <200806071324.21468.steve@pearwood.info> On Sat, 7 Jun 2008 07:32:40 am Martin v. L?wis wrote: > Flat is better than nested, > and adding hierarchy will make it *more* difficult to find anything > (except perhaps for the one person who did the rearrangement). Do you have a filing cabinet with everything filed under "F" for File? *wink* I've often thought that "Flat is better than nested" was one of the weaker Zens... often good advice, but nearly as often not. Lists of lists and binary trees can be useful. If you've ever used an operating system with a flat file system (and I have) you'll know what a pain it is dealing with the lack of hierarchical directories. > I personally use grep to find the place where to add a new test. I think, and this is just my opinion, that if you are forced to do a linear search through the entire test suite in order to find the file you want, that perhaps the test suite needs a tad better organisation. -- Steven From jyasskin at gmail.com Sat Jun 7 05:25:03 2008 From: jyasskin at gmail.com (Jeffrey Yasskin) Date: Fri, 6 Jun 2008 20:25:03 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: References: <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> <1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1> Message-ID: <5d44f72f0806062025s17fc0e26yc61cbf17e8b13172@mail.gmail.com> Well, it seems like Integral instances should be able to be passed to either int() or long(), so __long__ should probably stay. I have no idea why I didn't include __int__, but its absence was probably the only reason __index__ calls long() instead of int(). On Fri, Jun 6, 2008 at 3:23 PM, Guido van Rossum wrote: > Both of these seem 2.6-specific quirks. Those lines wereJeffrey's; > maybe he remembers? I'm guessing that adding __long__ was done since > 2.6 supports it, and the removal of __int__ was an oversight. I also > think that there's no reason to change __index__ to call long(); int() > will automatically return a long as needed. Maybe changing __long__ > back to __int__ is also harmless. > > On Fri, Jun 6, 2008 at 2:13 PM, Raymond Hettinger wrote: >> From: "Guido van Rossum" >>> >>> Make that int() instead of long() and I'm okay with it. >> >> Does anyone know why Integral says that __long__ is a required abstract >> method, but not __int__? >> >> Likewise, why is index() defined as long(self) instead of int(self)? >> >> There may be some design nuance that I'm not seeing. >> >> >> Raymond >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- Namast?, Jeffrey Yasskin http://jeffrey.yasskin.info/ From steve at pearwood.info Sat Jun 7 05:42:19 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 7 Jun 2008 13:42:19 +1000 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849C0FE.3070208@trueblade.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> <4849C0FE.3070208@trueblade.com> Message-ID: <200806071342.19766.steve@pearwood.info> On Sat, 7 Jun 2008 08:58:06 am Eric Smith wrote: > > Really? Given the choice between core_language (divided into syntax > > and builtins) and stdlib you wouldn't know where to look? > > Really. I wouldn't bother thinking about it. I'd just grep. Since you'll do the same thing regardless of whether the tests are nested or not, I don't think you should object to the proposal. "I'm against it because it will make no difference to me" is not what I call a good objection. Some people might prefer to cut down their false positives by splitting the tests into sub-directories. You can still get all the false positives you want by writing grep -r -I parrot * instead of grep -I parrot * -- Steven From cs at zip.com.au Sat Jun 7 06:18:22 2008 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 7 Jun 2008 14:18:22 +1000 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <200806071324.21468.steve@pearwood.info> Message-ID: <20080607041822.GA4843@cskk.homeip.net> On 07Jun2008 13:24, Steven D'Aprano wrote: | On Sat, 7 Jun 2008 07:32:40 am Martin v. L?wis wrote: | > Flat is better than nested, | > and adding hierarchy will make it *more* difficult to find anything | > (except perhaps for the one person who did the rearrangement). | | Do you have a filing cabinet with everything filed under "F" for File? | *wink* No, but I always want to put things in more than one folder in my filing cabinet, and cannot. Choosing a primary axis for organisation can be arbitrary. | I've often thought that "Flat is better than nested" was one of the | weaker Zens... often good advice, but nearly as often not. Lists of | lists and binary trees can be useful. If you've ever used an operating | system with a flat file system (and I have) you'll know what a pain it | is dealing with the lack of hierarchical directories. Aye, but I've frequently tripped over people blithely saying "general to specific" for nesting without realising that the axes were orthoganal, not inherently nested. Not arguing for or against, btw. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ From jjt009 at yahoo.com Sat Jun 7 06:57:58 2008 From: jjt009 at yahoo.com (James Thomas) Date: Fri, 6 Jun 2008 21:57:58 -0700 (PDT) Subject: [Python-Dev] Location of uname in os.py Message-ID: <7934.12022.qm@web30904.mail.mud.yahoo.com> Can anyone give me an idea as to where the function uname is located in os? I am unable to find it explicitly defined within the os.py file in the current svn repository. I also need to know how a call to os.uname() interacts with platform.uname(). I apologize if there is an obvious answer to this question; I just began to work on the Python project a few days ago and therefore am relatively unacqainted with the codebase. Thanks, James Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Sat Jun 7 10:51:22 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Jun 2008 08:51:22 +0000 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849C7AE.4070408@v.loewis.de> <4849D692.1050004@ronadam.com> Message-ID: Guilherme Polo schrieb: > 2008/6/6 Ron Adam : >> >> >> Martin v. L?wis wrote: >>>> >>>> How does 1 directory scale when one day you have possibly thousands of >>>> tests? >>> >>> I find this a theoretical question. It took 18 years to arrive at 500 >>> test files. Assuming a linear growth, we get 1000 tests in 2025, and >>> 2000 tests in 2060. People can worry about reorganizing them then. >> >> Personally I'd like to see packages have their own test directory. This >> keeps things related to each other together. Top level modules of course >> would have their tests in the top level test directory as they are now. >> > > I really dislike having a test directory inside a python package. You > have my -1 on that idea. I'm also not in favor of this, especially since you still need a wrapper in Lib/test to make them execute. However, I can understand package maintainers wanting "everything in one place". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Sat Jun 7 09:07:21 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 07 Jun 2008 09:07:21 +0200 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849D692.1050004@ronadam.com> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849C7AE.4070408@v.loewis.de> <4849D692.1050004@ronadam.com> Message-ID: <484A33A9.1020909@v.loewis.de> > Personally I'd like to see packages have their own test directory. That's a good idea, and it is already implemented for several packages. > This > keeps things related to each other together. Top level modules of > course would have their tests in the top level test directory as they > are now. Indeed. Martin From martin at v.loewis.de Sat Jun 7 09:16:23 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 07 Jun 2008 09:16:23 +0200 Subject: [Python-Dev] Location of uname in os.py In-Reply-To: <7934.12022.qm@web30904.mail.mud.yahoo.com> References: <7934.12022.qm@web30904.mail.mud.yahoo.com> Message-ID: <484A35C7.3070803@v.loewis.de> > Can anyone give me an idea as to where the function uname is located in > os? This question is out-of-scope for python-dev; please use python-list instead when asking how something is currently implemented. However, there is an easy answer, so I'll give it anyway: it's implemented in Modules/posixmodule.c:posix_uname. Finding out why it still shows up in os is left as an exercise for the reader. Regards, Martin From martin at v.loewis.de Sat Jun 7 09:18:18 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 07 Jun 2008 09:18:18 +0200 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <200806071324.21468.steve@pearwood.info> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> <200806071324.21468.steve@pearwood.info> Message-ID: <484A363A.2000406@v.loewis.de> > I think, and this is just my opinion, that if you are forced to do a > linear search through the entire test suite in order to find the file > you want, that perhaps the test suite needs a tad better organisation. Perhaps. However, if the linear search is then replaced with a recursive one, nothing is gained. Regards, Martin From jjl at pobox.com Sat Jun 7 14:45:01 2008 From: jjl at pobox.com (John J Lee) Date: Sat, 7 Jun 2008 13:45:01 +0100 (BST) Subject: [Python-Dev] Python bug day (was Re: New Developer) In-Reply-To: References: <443348.68183.qm@web30903.mail.mud.yahoo.com> Message-ID: On Fri, 6 Jun 2008, Facundo Batista wrote: [...] > Next week we'll have a Python Bug Weekend [3], it's a good moment to gain speed. [...] > [3] http://wiki.python.org/moin/PythonBugDay That page says the next bug day will be on Sat, June 21st-22nd 2008, which is in two weeks' time. Has that plan changed? John From musiccomposition at gmail.com Sat Jun 7 15:02:44 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 7 Jun 2008 08:02:44 -0500 Subject: [Python-Dev] on Python's tests (and making them better) In-Reply-To: <4849C618.8000209@v.loewis.de> References: <1afaf6160806050923x61128cdft55ff109e80ecf9fd@mail.gmail.com> <1afaf6160806061220x38037c3en84f1746194baa435@mail.gmail.com> <4849ACF8.20305@v.loewis.de> <4849BF5E.9020205@trueblade.com> <1afaf6160806061552l151c586fpfc063460868f19a4@mail.gmail.com> <4849C618.8000209@v.loewis.de> Message-ID: <1afaf6160806070602h26fccefcv7e9aa5a9f659b7cd@mail.gmail.com> On Fri, Jun 6, 2008 at 6:19 PM, "Martin v. L?wis" wrote: > (just to test your own intuition: when reversed() got added, was > that more of a language change than when sys.meta_path got > added?) I would say they were both core language changes. Anyway, I will happily drop this part of this proposal, since it such a small part. > > Regards, > Martin > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From greg at krypto.org Sat Jun 7 17:28:58 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 7 Jun 2008 08:28:58 -0700 Subject: [Python-Dev] wrt the beta deadline and freelist management In-Reply-To: <48475050.9090602@bullseye.andymac.org> References: <06BB1BB8-2C9C-4588-9B38-49DE234752E0@python.org> <48475050.9090602@bullseye.andymac.org> Message-ID: <52dc1c820806070828x26e85a0bg6dad2c3188414315@mail.gmail.com> On Wed, Jun 4, 2008 at 7:32 PM, Andrew MacIntyre < andymac at bullseye.apana.org.au> wrote: > There are 2 disparate approaches to clearing/compacting free lists for > basic types: > - APIs of the form Py_ClearFreeList() called from gc.collect() > [class, frame, method, tuple & unicode types]; > - APIs of the form Py_CompactFreeList() called from > sys._compact_freelists() [int & float types]; > > Both approaches are new for 2.6 & 3.0. > > The patch at http://bugs.python.org/issue2862 is geared towards bring > the int/float free list management into line with the others. > > The patch at http://bugs.python.org/issue3029 adds free list management > to the dict, list & set types. The value of this is probably minor, > and this patch is thus not significant in its own right other than for > consistency. > > However Raymond's comment to issue 3029 that the management routines > shouldn't be public APIs is IMO important (& I happen to agree). > > It would be nice to reduce the 2 approaches to one. > > I would rather the gc.collect() approach, as this seems to be more > obvious to many users who have gone looking for free list management in > prior versions, however my opinion isn't particularly valuable on this. > > Can this be resolved for 2.6/3.0? > I agree with the gc.collect approach taken in your issue2862 patch. Barring any objections, I suggest we accept it and will commit it in the next couple days. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Sat Jun 7 20:16:16 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Jun 2008 18:16:16 +0000 Subject: [Python-Dev] ABC issues In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > On Mon, May 26, 2008 at 11:59 AM, Raymond Hettinger wrote: >>> * The 2.6-backported Mapping ABC has the 3.0 dict API, >>> that is, it uses keys() that returns a view etc. >> >> Curious to hear what Guido thinks about this one. >> A nice use of the Mapping ABC is to be able to >> get 3.0 behaviors. I thought that was the whole >> point of all these backports. If the ABC gets altered, >> then it just makes the 2-to-3 conversion harder. > > It's wrong if the ABC doesn't describe the behavior of actual > implementations; that is its primary purpose, the mixin class is a > nice side benefit. > > We could make the incompatible mixin classes available separately > though, if you think they're useful. > >>> * The 2.6 UserDict is not registered as a mapping. >> >> Since the API's are not currently the same, it makes >> sense that UserDict is not registered. >> If the Mapping ABC does get changed, only IterableUserDict >> should get registered. A regular UserDict does not comply. > > Fair enough. I recomment to fix the Mapping ABC and register IterableUserDict. As a followup: - The Mapping fix is in http://bugs.python.org/issue3057. - Registering IterableUserDict is now in SVN. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From pfreixes at gmail.com Sat Jun 7 19:14:36 2008 From: pfreixes at gmail.com (Pau Freixes) Date: Sat, 7 Jun 2008 19:14:36 +0200 Subject: [Python-Dev] Different cpu python code usage between embedded mode and standalone mode Message-ID: <207312b70806071014yff950c3j8faa5577462e127d@mail.gmail.com> Hi list, First Hello to all, I have a serious problem for understand some results when I'm comparing cpu usages between same python code in embedded mode and standalone mode ( python name_script.py ) This last months I have been writing a program in c like to mod_python for embedding python language, it's a middleware for dispatch and execute python batch programs into several nodes. Now I'm writing some python program for test how scale this into several nodes and comparing with "standalone" performance. I found a very strange problem with one application named md5challenge, this aplication try to calculate the max number md5 digest in several seconds, md5challenge use a simple signal alarm for stop program when time has passed. This is the code of python script _nrdigest = 0 _const_b = 20 _f = None _signal = False def handler_alrm(signum, frame): global _signal global _nrdigest global _f _signal = True def try_me(): global _nrdigest global _f global _signal _f = open("/dev/urandom","r") while _signal is not True: buff = _f.read(_const_b) md5.md5(buff).hexdigest() _nrdigest = _nrdigest + 1 if _f is not None : _f.close() # Define entry point with one input variable # req is a instance of Request object, usefull members of this object are: # req.input is a dictionary with input.xml variables # req.constants is a dictionary with constants defined into signature.xml # req.output is void dictionary for full with output variables # req.config is a dictionary with config values take from namespace # req.apdn_pid is a pid of aplication def main( req ): global _nrdigest signal.signal(signal.SIGALRM, handler_alrm) signal.alarm(req.input['time']) try_me() req.output['count'] = _nrdigest return req.OK if __name__ == "__main__": # test code class test_req: pass req = test_req() req.input = { 'time' : 10 } req.output = { 'ret' : 0, 'count' : 0 } req.OK = 1 main(req) print "Reached %d digests" % req.output['count'] When I try to run this program in standalone into my Pentium Dual Core md5challenge reached 1.000.000 milion keys in 10 seconds but when i try to run this code in embedded mode md5challenge reached about 200.000 more keys !!! I repeat this test many times and always wins embedded mode !!! What's happen ? Also I tested to erase read dependencies from /dev/random, and calculate all keys from same buffer. In this case embedded mode win always also, and the difference are more bigger !!! The alarm time expires in both case in 10 seconds. Thks to all, can anybody help to me for understand this results ? -- Pau Freixes Linux GNU/User -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sat Jun 7 19:27:07 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Jun 2008 10:27:07 -0700 Subject: [Python-Dev] Different cpu python code usage between embedded mode and standalone mode In-Reply-To: <207312b70806071014yff950c3j8faa5577462e127d@mail.gmail.com> References: <207312b70806071014yff950c3j8faa5577462e127d@mail.gmail.com> Message-ID: This is not an issue for python-dev, but I have to ask: what do you mean by "embedded mode"? On Sat, Jun 7, 2008 at 10:14 AM, Pau Freixes wrote: > Hi list, > > First Hello to all, I have a serious problem for understand some results > when I'm comparing cpu usages between same python code in embedded mode and > standalone mode ( python name_script.py ) > > This last months I have been writing a program in c like to mod_python for > embedding python language, it's a middleware for dispatch and execute python > batch programs into several nodes. Now I'm writing some python program for > test how scale this into several nodes and comparing with "standalone" > performance. > > I found a very strange problem with one application named md5challenge, this > aplication try to calculate the max number md5 digest in several seconds, > md5challenge use a simple signal alarm for stop program when time has > passed. This is the code of python script > > _nrdigest = 0 > _const_b = 20 > _f = None > _signal = False > > > def handler_alrm(signum, frame): > global _signal > global _nrdigest > global _f > > > _signal = True > > def try_me(): > global _nrdigest > global _f > global _signal > > _f = open("/dev/urandom","r") > while _signal is not True: > buff = _f.read(_const_b) > md5.md5(buff).hexdigest() > _nrdigest = _nrdigest + 1 > > if _f is not None : > _f.close() > > # Define entry point with one input variable > # req is a instance of Request object, usefull members of this object are: > # req.input is a dictionary with input.xml variables > # req.constants is a dictionary with constants defined into signature.xml > # req.output is void dictionary for full with output variables > # req.config is a dictionary with config values take from namespace > # req.apdn_pid is a pid of aplication > > > def main( req ): > global _nrdigest > > > signal.signal(signal.SIGALRM, handler_alrm) > signal.alarm(req.input['time']) > > try_me() > > req.output['count'] = _nrdigest > > return req.OK > > > if __name__ == "__main__": > > # test code > class test_req: > pass > > req = test_req() > req.input = { 'time' : 10 } > req.output = { 'ret' : 0, 'count' : 0 } > req.OK = 1 > > main(req) > > print "Reached %d digests" % req.output['count'] > > > When I try to run this program in standalone into my Pentium Dual Core > md5challenge reached 1.000.000 milion keys in 10 seconds but when i try to > run this code in embedded mode md5challenge reached about 200.000 more keys > !!! I repeat this test many times and always wins embedded mode !!! > What's happen ? > > Also I tested to erase read dependencies from /dev/random, and calculate all > keys from same buffer. In this case embedded mode win always also, and the > difference are more bigger !!! > > The alarm time expires in both case in 10 seconds. > > Thks to all, can anybody help to me for understand this results ? > > -- > Pau Freixes > Linux GNU/User > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jun 7 19:30:16 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Jun 2008 10:30:16 -0700 Subject: [Python-Dev] Mini-Pep: Simplifying the Integral ABC In-Reply-To: <5d44f72f0806062025s17fc0e26yc61cbf17e8b13172@mail.gmail.com> References: <4847E828.9060001@gmail.com> <545E2549BAF34779BC22AFAA0AB4F6CD@RaymondLaptop1> <1A488C5E25A14DC5A0DAC585C073E8BC@RaymondLaptop1> <5d44f72f0806062025s17fc0e26yc61cbf17e8b13172@mail.gmail.com> Message-ID: I recommend switching back to __int__ and int; even in 2.5, these are expected to return either an int or a long as required. There's no need to mess with __long__ at all. On Fri, Jun 6, 2008 at 8:25 PM, Jeffrey Yasskin wrote: > Well, it seems like Integral instances should be able to be passed to > either int() or long(), so __long__ should probably stay. I have no > idea why I didn't include __int__, but its absence was probably the > only reason __index__ calls long() instead of int(). > > On Fri, Jun 6, 2008 at 3:23 PM, Guido van Rossum wrote: >> Both of these seem 2.6-specific quirks. Those lines wereJeffrey's; >> maybe he remembers? I'm guessing that adding __long__ was done since >> 2.6 supports it, and the removal of __int__ was an oversight. I also >> think that there's no reason to change __index__ to call long(); int() >> will automatically return a long as needed. Maybe changing __long__ >> back to __int__ is also harmless. >> >> On Fri, Jun 6, 2008 at 2:13 PM, Raymond Hettinger wrote: >>> From: "Guido van Rossum" >>>> >>>> Make that int() instead of long() and I'm okay with it. >>> >>> Does anyone know why Integral says that __long__ is a required abstract >>> method, but not __int__? >>> >>> Likewise, why is index() defined as long(self) instead of int(self)? >>> >>> There may be some design nuance that I'm not seeing. >>> >>> >>> Raymond >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/guido%40python.org >>> >> >> >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) >> > > > > -- > Namast?, > Jeffrey Yasskin > http://jeffrey.yasskin.info/ > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Sat Jun 7 22:13:48 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Jun 2008 20:13:48 +0000 Subject: [Python-Dev] Deprecate parser's "ast" functions? Message-ID: The parser module exports each function and type twice, once with "AST" in the name, once with "ST". Since "AST" now has a different meaning for Python code compilation, I propose to deprecate the "ast" variants in 2.6 and remove them in Python 3000. (Also, all keyword arguments are called "ast", that cannot change in 2.6 but should in 3.0.) I'm at the moment changing the documentation of parser to only refer to the "st" variants anymore. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From gnewsg at gmail.com Sat Jun 7 22:11:34 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Sat, 7 Jun 2008 13:11:34 -0700 (PDT) Subject: [Python-Dev] Modules for 2.6 inclusion In-Reply-To: References: <4849AC6C.7020004@v.loewis.de> Message-ID: <905d05c9-eb24-467f-8a24-d9d0c7e3b439@c58g2000hsc.googlegroups.com> On 6 Giu, 13:27, Georg Brandl wrote: > - setuptools > BDFL pronouncement for inclusion in 2.5: > http://mail.python.org/pipermail/python-dev/2006-April/063964.html I'd like to see more interest about this issue since it's a real shame that the current distutils is not even able to solve dependencies and has such a poor integration with Pypi. Having a *valid* distribution tool like setuptools integrated with Python would be the first step to have one of the most important things Python is currently missing finally available: a centralized archive of softwares like Perl CPAN or Ruby Gems. IMHO this should be put on top of the "TODO" list and I'm honestly surprised it raised so little clamor so far. --- Giampaolo http://code.google.com/p/pyftpdlib From g.brandl at gmx.net Sun Jun 8 00:27:46 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 07 Jun 2008 22:27:46 +0000 Subject: [Python-Dev] Modules for 2.6 inclusion In-Reply-To: References: Message-ID: Guilherme Polo schrieb: > I created an issue 1 week ago (http://bugs.python.org/issue2983) > suggesting the addition of the ttk module to lib-tk, and to the new > tkinter package. Is there any chance to this be accepted for Python > 2.6 ? This may be a good thing to have since it can show that Tkinter is not as dead (and dead ugly) as many people think it is. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From armin.ronacher at active-4.com Sat Jun 7 23:10:14 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sat, 7 Jun 2008 21:10:14 +0000 (UTC) Subject: [Python-Dev] =?utf-8?b?cGVwOGl0eSBfX2Z1dHVyZV9f?= Message-ID: Hi, That's just a flaming-sword thread but I want to mention it nonetheless :-) Basically I propose getting rid of __future__._Feature.getMandatoryRelease() in favour of __future__._Feature.mandatory. That's backwards compatibile and much more pythonic. Getters/Setters are considered unpythonic and the getting rid of all that Java-like naming sounds like a good idea to me :) If threading/processing gets a pep8ified API with 2.6, why not __future__? Proposed patch: http://paste.pocoo.org/show/64512/ Regards, Armin From pfreixes at gmail.com Sun Jun 8 13:15:08 2008 From: pfreixes at gmail.com (Pau Freixes) Date: Sun, 8 Jun 2008 13:15:08 +0200 Subject: [Python-Dev] GIL cpu usage problem, confirm me Message-ID: <207312b70806080415n5a628179o157e755eb08ea82b@mail.gmail.com> Hi List, Surly this is a recurring theme into python dev world, but I need your help for confirm if the follow image it's really http://www.milnou.net/~pfreixes/img/cpu_usage_gil_problem.png I'm writing a brief article for my blog and I need to make sure about the current problem with GIL and multi core environments, this picture try to explain with images the problem for scheduling multiple threads running python code of same interpreter into multiple cpu cores. Can anyone confirm to me this picture ? And if it's possible answer this two questions I will be happy :/ 1) When this situation it's produced into one core environment whats happens when thread library or os switch context into other python thread and this don't have a GIL ? 2) Exist some PEP or plans for modify this and run multiple thread python for same interpreter at current time ? for python 3000? Thanks and excuse for this break. -- Pau Freixes Linux GNU/User -------------- next part -------------- An HTML attachment was scrubbed... URL: From musiccomposition at gmail.com Sun Jun 8 15:37:20 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 8 Jun 2008 08:37:20 -0500 Subject: [Python-Dev] GIL cpu usage problem, confirm me In-Reply-To: <207312b70806080415n5a628179o157e755eb08ea82b@mail.gmail.com> References: <207312b70806080415n5a628179o157e755eb08ea82b@mail.gmail.com> Message-ID: <1afaf6160806080637v2816e03g1f7bf15ca2306f1c@mail.gmail.com> On Sun, Jun 8, 2008 at 6:15 AM, Pau Freixes wrote: > Hi List, > > Surly this is a recurring theme into python dev world, but I need your help > for confirm if the follow image it's really > > http://www.milnou.net/~pfreixes/img/cpu_usage_gil_problem.png > > I'm writing a brief article for my blog and I need to make sure about the > current problem with GIL and multi core environments, this picture try to > explain with images the problem for scheduling multiple threads running > python code of same interpreter into multiple cpu cores. Can anyone confirm > to me this picture ? > > And if it's possible answer this two questions I will be happy :/ Next time, comp.lang.python would be the place to ask these questions. > > 1) When this situation it's produced into one core environment whats happens > when thread library or os switch context into other python thread and this > don't have a GIL ? They run together. > > 2) Exist some PEP or plans for modify this and run multiple thread python > for same interpreter at current time ? for python 3000? Certainly not in core Python. Have a look http://code.google.com/p/python-threadsafe/, though. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From exarkun at divmod.com Sun Jun 8 15:49:33 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sun, 8 Jun 2008 09:49:33 -0400 Subject: [Python-Dev] GIL cpu usage problem, confirm me In-Reply-To: <1afaf6160806080637v2816e03g1f7bf15ca2306f1c@mail.gmail.com> Message-ID: <20080608134933.4714.378522420.divmod.quotient.6949@ohm> On Sun, 8 Jun 2008 08:37:20 -0500, Benjamin Peterson wrote: > >Certainly not in core Python. Have a look >http://code.google.com/p/python-threadsafe/, though. > http://code.google.com/p/python-safethread/ Jean-Paul From curt at hagenlocher.org Mon Jun 9 04:24:55 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Sun, 8 Jun 2008 19:24:55 -0700 Subject: [Python-Dev] Assignment to None Message-ID: My apologies if this is one of those "dead horse" issues. The following seems a little inconsistent to me: >>> c = C() >>> c.None Traceback (most recent call last): File "", line 1, in AttributeError: C instance has no attribute 'None' >>> c.None = 'foo' File "", line 1 SyntaxError: assignment to None >>> setattr(c, 'None', 'foo') >>> c.None 'foo' >>> So, it's okay to setattr the attribute name "None" but not okay to set it directly? Is this deliberate or is it an unintentional side effect of parser changes to prevent assignment to None? -- Curt Hagenlocher curt at hagenlocher.org From musiccomposition at gmail.com Mon Jun 9 05:10:12 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 8 Jun 2008 22:10:12 -0500 Subject: [Python-Dev] Assignment to None In-Reply-To: References: Message-ID: <1afaf6160806082010w5ca671ctfa4c259cacb61351@mail.gmail.com> On Sun, Jun 8, 2008 at 9:24 PM, Curt Hagenlocher wrote: > My apologies if this is one of those "dead horse" issues. The > following seems a little inconsistent to me: > >>>> c = C() >>>> c.None > Traceback (most recent call last): > File "", line 1, in > AttributeError: C instance has no attribute 'None' >>>> c.None = 'foo' > File "", line 1 > SyntaxError: assignment to None >>>> setattr(c, 'None', 'foo') >>>> c.None > 'foo' >>>> > > So, it's okay to setattr the attribute name "None" but not okay to set > it directly? Is this deliberate or is it an unintentional side effect > of parser changes to prevent assignment to None? I believe that runtime checks like the one needed for setattr would be too much of a performance hit to be practical. The syntax changes are meant to avoid mistakes and confusion rather than completely prevent something from ever happening. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at pearwood.info Mon Jun 9 05:48:44 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 9 Jun 2008 13:48:44 +1000 Subject: [Python-Dev] Assignment to None In-Reply-To: References: Message-ID: <200806091348.44361.steve@pearwood.info> On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote: > So, it's okay to setattr the attribute name "None" but not okay to > set it directly? ? I suspect this is off-topic for python-dev, and would be better on comp.lang.python or similar, but for what it's worth, I consider having an attribute named 'None' bad practise, regardless of any runtime checks. But as part of Python's "we're all consenting adults here" philosophy, I wouldn't approve of expensive or extensive run-time checks specifically to prevent it. If you really have to name your attribute None, and are prepared to live with the consequences, then go ahead. In a similar fashion: >>> class Parrot(object): ... pass ... >>> p = Parrot() >>> p.1 = 'spam' File "", line 1 p.1 ^ SyntaxError: invalid syntax >>> setattr(p, '1', 'spam') >>> -- Steven From martin at v.loewis.de Mon Jun 9 07:19:44 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jun 2008 07:19:44 +0200 Subject: [Python-Dev] Assignment to None In-Reply-To: References: Message-ID: <484CBD70.3000808@v.loewis.de> > So, it's okay to setattr the attribute name "None" but not okay to set > it directly? Is this deliberate or is it an unintentional side effect > of parser changes to prevent assignment to None? It's deliberate. setattr(o, "foo bar", "baz") also works, even though "foo bar" is not an identifier. setattr doesn't take the Python grammar into account, but only the object's structure. Regards, Martin From greg at krypto.org Mon Jun 9 07:20:36 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 8 Jun 2008 22:20:36 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <48490121.5030701@egenix.com> References: <48397ECC.9070805@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com> <48490121.5030701@egenix.com> Message-ID: <52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com> On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg wrote: > On 2008-06-03 01:29, Gregory P. Smith wrote: > >> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum >> wrote: >> >> I will freely admit that I haven't followed this thread in any detail, >>> but if it were up to me, I'd have the 2.6 internal code use PyString >>> >> >> ... >> >> Should we read this as a BDFL pronouncement and make it so? >> >> All that would mean change wise is that trunk r63675 as well as possibly >> r63672 and r63677 would need to be rolled back and this whole discussion >> over if such a big change should have happened would turn into a moot >> point. >> > > I would certainly welcome reverting the change. > > All that's needed to support PyBytes API in 2.x is a set of #defines > that map the new APIs to the PyString names. That's a clean and > easily understandable solution. > Okay, I've reverted r63675 in trunk revision r64048. That leaves all of the python modules and internals using PyString_ api names instead of PyBytes_ api names as they were before. PyBytes_ #define's exist for the appropriate PyString methods incase anyone wants to use those. Programmers interested in the code > for a PyString API can then still look up the code in stringobject.c, > e.g. to find out how a certain special case is handled or to check > the ref counting - just like they did for years. The files still exist with the new names. bytesobject.c instead of stringobject.c. Those renames were done in the other CLs i mentioned which have not yet been reverted. The current state seems a bit odd because they depend on the #defines to cause method definitions to be the PyString_ names instead of the PyBytes_ names. > > > Developer who want to start differentiating between mixed byte/text > data and bytes-only can start using PyBytes for byte data. > > I would also add macros that map the PyBytes_* APIs to PyString_*, but >>> I would not start using these internally except in code newly written >>> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString >>> for 8-bit strings containing text, and PyBytes for 8-bit strings >>> containing binary data. For 8-bit strings that could contain either >>> text or data, I'd use PyString, in the spirit of 2.x. >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From musiccomposition at gmail.com Mon Jun 9 01:38:05 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 8 Jun 2008 18:38:05 -0500 Subject: [Python-Dev] Nab those release blockers! Message-ID: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> As we approach the planned first betas for 2.6/3.0, my heart sinks when I see the imposing list of 16 release blockers. [1] Luckily, most of the issues have patches that just need *your* review. Others, namely the Py3k exception issues, may require a little more discussion, but I'm sure we can get it done. Let's get this release on time! [clapping] [1] http://bugs.python.org/issue?%40sort0=activity&%40sortdir0=on&%40sort1=&%40group0=&%40group1=&%40columns=title%2Cid%2Cactivity%2Cversions%2Cstatus&%40filter=priority%2Cstatus&priority=1&status=1&%40pagesize=50&%40startwith=0 -- Thanks Benjamin Peterson "There's no place like 127.0.0.1." From fuzzyman at voidspace.org.uk Mon Jun 9 10:28:03 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 09 Jun 2008 09:28:03 +0100 Subject: [Python-Dev] Assignment to None In-Reply-To: <200806091348.44361.steve@pearwood.info> References: <200806091348.44361.steve@pearwood.info> Message-ID: <484CE993.5050506@voidspace.org.uk> Steven D'Aprano wrote: > On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote: > > >> So, it's okay to setattr the attribute name "None" but not okay to >> set it directly? >> > > I suspect this is off-topic for python-dev, and would be better on > comp.lang.python or similar, but for what it's worth, I consider having > an attribute named 'None' bad practise, regardless of any runtime > checks. But as part of Python's "we're all consenting adults here" > philosophy, I wouldn't approve of expensive or extensive run-time > checks specifically to prevent it. If you really have to name your > attribute None, and are prepared to live with the consequences, then go > ahead. > > FWIW Curt is asking because he is on the team responsible for implementing IronPython. In .NET you have several enumerations with members called None, for example 'Alignment.None'. It would be inconvenient for IronPython users if it enforced this particular syntax rule of Python's; that not only is assignment to None disallowed but that members named None are invalid syntax. The question is, what is the specification for Python. Is it that assignment to None is disallowed and the naming of members called None being invalid syntax is merely an artefact of the implementation of this, or does Python require this... Michael Foord > In a similar fashion: > > >>>> class Parrot(object): >>>> > ... pass > ... > >>>> p = Parrot() >>>> p.1 = 'spam' >>>> > File "", line 1 > p.1 > ^ > SyntaxError: invalid syntax > >>>> setattr(p, '1', 'spam') >>>> >>>> > > > From mal at egenix.com Mon Jun 9 10:44:38 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 09 Jun 2008 10:44:38 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com> <48490121.5030701@egenix.com> <52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com> Message-ID: <484CED76.3070703@egenix.com> On 2008-06-09 07:20, Gregory P. Smith wrote: > On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg wrote: > >> On 2008-06-03 01:29, Gregory P. Smith wrote: >> >>> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum >>> wrote: >>> >>> I will freely admit that I haven't followed this thread in any detail, >>>> but if it were up to me, I'd have the 2.6 internal code use PyString >>>> >>> ... >>> >>> Should we read this as a BDFL pronouncement and make it so? >>> >>> All that would mean change wise is that trunk r63675 as well as possibly >>> r63672 and r63677 would need to be rolled back and this whole discussion >>> over if such a big change should have happened would turn into a moot >>> point. >>> >> I would certainly welcome reverting the change. >> >> All that's needed to support PyBytes API in 2.x is a set of #defines >> that map the new APIs to the PyString names. That's a clean and >> easily understandable solution. >> > > Okay, I've reverted r63675 in trunk revision r64048. That leaves all of the > python modules and internals using PyString_ api names instead of PyBytes_ > api names as they were before. PyBytes_ #define's exist for the appropriate > PyString methods incase anyone wants to use those. Thanks. > Programmers interested in the code >> for a PyString API can then still look up the code in stringobject.c, >> e.g. to find out how a certain special case is handled or to check >> the ref counting - just like they did for years. > > > The files still exist with the new names. bytesobject.c instead of > stringobject.c. Those renames were done in the other CLs i mentioned which > have not yet been reverted. The current state seems a bit odd because they > depend on the #defines to cause method definitions to be the PyString_ names > instead of the PyBytes_ names. Please restore the original state, ie. PyString APIs live in stringobject.h and stringobject.c. bytesobject.h should then have the #defines for PyBytes APIs, pointing them to the PyString names (basically what's currently in stringobject.h). >> Developer who want to start differentiating between mixed byte/text >> data and bytes-only can start using PyBytes for byte data. >> >> I would also add macros that map the PyBytes_* APIs to PyString_*, but >>>> I would not start using these internally except in code newly written >>>> for 2.6 and intended to be "in the spirit of 3.0". IOW use PyString >>>> for 8-bit strings containing text, and PyBytes for 8-bit strings >>>> containing binary data. For 8-bit strings that could contain either >>>> text or data, I'd use PyString, in the spirit of 2.x. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 09 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 27 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From jnoller at gmail.com Mon Jun 9 12:33:33 2008 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 9 Jun 2008 06:33:33 -0400 Subject: [Python-Dev] Nab those release blockers! In-Reply-To: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> Message-ID: <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com> For the multiprocessing module all I have left is the docs in rest format - given I'm a rest newbie, it's taking longer than I thought On Jun 8, 2008, at 7:38 PM, "Benjamin Peterson" wrote: > As we approach the planned first betas for 2.6/3.0, my heart sinks > when I see the imposing list of 16 release blockers. [1] Luckily, most > of the issues have patches that just need *your* review. Others, > namely the Py3k exception issues, may require a little more > discussion, but I'm sure we can get it done. > > Let's get this release on time! [clapping] > > [1] http://bugs.python.org/issue?%40sort0=activity&%40sortdir0=on&%40sort1=&%40group0=&%40group1=&%40columns=title%2Cid%2Cactivity%2Cversions%2Cstatus&%40filter=priority%2Cstatus&priority=1&status=1&%40pagesize=50&%40startwith=0 > > -- > Thanks > Benjamin Peterson > "There's no place like 127.0.0.1." > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com From ncoghlan at gmail.com Mon Jun 9 13:28:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 09 Jun 2008 21:28:57 +1000 Subject: [Python-Dev] Nab those release blockers! In-Reply-To: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> Message-ID: <484D13F9.30105@gmail.com> Benjamin Peterson wrote: > As we approach the planned first betas for 2.6/3.0, my heart sinks > when I see the imposing list of 16 release blockers. [1] Luckily, most > of the issues have patches that just need *your* review. Others, > namely the Py3k exception issues, may require a little more > discussion, but I'm sure we can get it done. For issue 643841 (providing a ProxyMixin class in a new typetools module), aside from the minor tweak mentioned in the last couple of comments, I'm basically looking for a yea or nay from Barry or Guido. I think it's the best answer we're going to find to the problem of how to write type-agnostic delegation classes in a Python world without classic classes. Cheers, Nick. http://bugs.python.org/issue643841 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From musiccomposition at gmail.com Mon Jun 9 15:37:42 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 9 Jun 2008 08:37:42 -0500 Subject: [Python-Dev] Nab those release blockers! In-Reply-To: <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com> References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com> Message-ID: <1afaf6160806090637w77786a23q9cac4ee7e12277cc@mail.gmail.com> On Mon, Jun 9, 2008 at 5:33 AM, Jesse Noller wrote: > For the multiprocessing module all I have left is the docs in rest format - > given I'm a rest newbie, it's taking longer than I thought If you want to email what you have done to docs at python.org, we'd love to help! -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From tom at vector-seven.com Mon Jun 9 15:43:27 2008 From: tom at vector-seven.com (Thomas Lee) Date: Mon, 09 Jun 2008 23:43:27 +1000 Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign nodes Message-ID: <484D337F.8090201@vector-seven.com> In porting one of the old peephole optimizations to the new AST compiler I noticed something weird going on with the following code: a, b, c = 1, 2, 3 Now, as you would expect this gets parsed into an Assign node. That Assign node looks like the following: Assign.targets = [Tuple(Name(a), Name(b), Name(c))] Assign.value = Tuple(1, 2, 3) What's weird here is that Assign.targets is an asdl_seq ... why are we wrapping the names in a Tuple() node? Shouldn't it look something more like this: Assign.targets = [Name(a), Name(b), Name(c)] I understand that parsing the testlist might yield a tuple and it was thus easier to just use the tuple rather than unpack it into an asdl_seq ... but if this was the intention, then why is Assign.targets an expr* rather than a plain old expr? Cheers, Tom From ncoghlan at gmail.com Mon Jun 9 16:04:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Jun 2008 00:04:05 +1000 Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign nodes In-Reply-To: <484D337F.8090201@vector-seven.com> References: <484D337F.8090201@vector-seven.com> Message-ID: <484D3855.703@gmail.com> Thomas Lee wrote: > In porting one of the old peephole optimizations to the new AST compiler > I noticed something weird going on with the following code: > > a, b, c = 1, 2, 3 > > Now, as you would expect this gets parsed into an Assign node. That > Assign node looks like the following: > > Assign.targets = [Tuple(Name(a), Name(b), Name(c))] > Assign.value = Tuple(1, 2, 3) > > What's weird here is that Assign.targets is an asdl_seq ... why are we > wrapping the names in a Tuple() node? Shouldn't it look something more > like this: > > Assign.targets = [Name(a), Name(b), Name(c)] > > I understand that parsing the testlist might yield a tuple and it was > thus easier to just use the tuple rather than unpack it into an asdl_seq > ... but if this was the intention, then why is Assign.targets an expr* > rather than a plain old expr? I haven't looked at that code recently, but I believe the ADSL sequence in the assignment node is for statements where there are actually multiple assignment targets, such as: >>> p = x, y = 1, 2 >>> p, x, y ((1, 2), 1, 2) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tom at vector-seven.com Mon Jun 9 16:21:14 2008 From: tom at vector-seven.com (Thomas Lee) Date: Tue, 10 Jun 2008 00:21:14 +1000 Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign nodes In-Reply-To: <484D3855.703@gmail.com> References: <484D337F.8090201@vector-seven.com> <484D3855.703@gmail.com> Message-ID: <484D3C5A.6020808@vector-seven.com> Nick Coghlan wrote: > > I haven't looked at that code recently, but I believe the ADSL > sequence in the assignment node is for statements where there are > actually multiple assignment targets, such as: > > >>> p = x, y = 1, 2 > >>> p, x, y > ((1, 2), 1, 2) > > Cheers, > Nick. > Ah I see. A quick test verifies exactly this: >>> import _ast >>> ast = compile("p = x, y = 1, 2", "", "exec", _ast.PyCF_ONLY_AST) >>> ast.body[0] <_ast.Assign object at 0xb7d0122c> >>> ast.body[0].targets [<_ast.Name object at 0xb7d0124c>, <_ast.Tuple object at 0xb7d0128c>] >>> ast.body[0].value <_ast.Tuple object at 0xb7d0132c> >>> I thought this would have been implemented as nested Assign nodes, but I'm obviously wrong. :) Thanks for the clarification. Cheers, T From aleaxit at gmail.com Mon Jun 9 17:12:42 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 9 Jun 2008 08:12:42 -0700 Subject: [Python-Dev] Assignment to None In-Reply-To: <484CE993.5050506@voidspace.org.uk> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> Message-ID: The problem is more general: what if a member (of some external object we're proxying one way or another) is named print (in Python < 3), or class, or...? To allow foo.print or bar.class would require pretty big changes to Python's parser -- I have vague memories that the issue was discussed ages ago (possibly in conjunction with some early release of Jython) but never went anywhere much (including proposals to automatically append an underscore to such IDs in the proxying layer, etc etc). Maybe None in particular is enough of a special case (if it just happens to be hugely often used in dotNET libraries)? Alex On Mon, Jun 9, 2008 at 1:28 AM, Michael Foord wrote: > Steven D'Aprano wrote: >> >> On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote: >> >> >>> >>> So, it's okay to setattr the attribute name "None" but not okay to set it >>> directly? >> >> I suspect this is off-topic for python-dev, and would be better on >> comp.lang.python or similar, but for what it's worth, I consider having an >> attribute named 'None' bad practise, regardless of any runtime checks. But >> as part of Python's "we're all consenting adults here" philosophy, I >> wouldn't approve of expensive or extensive run-time checks specifically to >> prevent it. If you really have to name your attribute None, and are prepared >> to live with the consequences, then go ahead. >> >> > > FWIW Curt is asking because he is on the team responsible for implementing > IronPython. > > In .NET you have several enumerations with members called None, for example > 'Alignment.None'. It would be inconvenient for IronPython users if it > enforced this particular syntax rule of Python's; that not only is > assignment to None disallowed but that members named None are invalid > syntax. > > The question is, what is the specification for Python. Is it that assignment > to None is disallowed and the naming of members called None being invalid > syntax is merely an artefact of the implementation of this, or does Python > require this... > > Michael Foord > >> In a similar fashion: >> >> >>>>> >>>>> class Parrot(object): >>>>> >> >> ... pass >> ... >> >>>>> >>>>> p = Parrot() >>>>> p.1 = 'spam' >>>>> >> >> File "", line 1 >> p.1 >> ^ >> SyntaxError: invalid syntax >> >>>>> >>>>> setattr(p, '1', 'spam') >>>>> >>>>> >> >> >> > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > From fuzzyman at voidspace.org.uk Mon Jun 9 17:46:20 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 09 Jun 2008 16:46:20 +0100 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> Message-ID: <484D504C.7060406@voidspace.org.uk> Alex Martelli wrote: > The problem is more general: what if a member (of some external > object we're proxying one way or another) is named print (in Python < > 3), or class, or...? To allow foo.print or bar.class would require > pretty big changes to Python's parser -- I have vague memories that > the issue was discussed ages ago (possibly in conjunction with some > early release of Jython) but never went anywhere much (including > proposals to automatically append an underscore to such IDs in the > proxying layer, etc etc). Maybe None in particular is enough of a > special case (if it just happens to be hugely often used in dotNET > libraries)? > 'None' as a member does occur particularly frequently in the .NET world. A halfway house might be to state (something like): Python as a language disallows you from having names the same as keywords or 'None'. An implementation restriction specific to CPython is that the same restriction also applies to member names. Alternative implementations are free to not implement this restriction, with the caveat that code using reserved member names directly will be invalid syntax for CPython. Although this appears to raise the spectre of code written for Jython or IronPython not being compatible with CPython, in reality if they are using Java or .NET objects then they are already incompatible with CPython (IronPython doesn't work by proxying .NET objects by the way - you work directly with them but the DLR 'catches' attribute look ups etc to add Python methods to basic types and do 'other magic' like wrapping Python functions as delegates). This at least enshrines the current IronPython behaviour with the veneer of respectability. Michael Foord > Alex > > On Mon, Jun 9, 2008 at 1:28 AM, Michael Foord wrote: > >> Steven D'Aprano wrote: >> >>> On Mon, 9 Jun 2008 12:24:55 pm Curt Hagenlocher wrote: >>> >>> >>> >>>> So, it's okay to setattr the attribute name "None" but not okay to set it >>>> directly? >>>> >>> I suspect this is off-topic for python-dev, and would be better on >>> comp.lang.python or similar, but for what it's worth, I consider having an >>> attribute named 'None' bad practise, regardless of any runtime checks. But >>> as part of Python's "we're all consenting adults here" philosophy, I >>> wouldn't approve of expensive or extensive run-time checks specifically to >>> prevent it. If you really have to name your attribute None, and are prepared >>> to live with the consequences, then go ahead. >>> >>> >>> >> FWIW Curt is asking because he is on the team responsible for implementing >> IronPython. >> >> In .NET you have several enumerations with members called None, for example >> 'Alignment.None'. It would be inconvenient for IronPython users if it >> enforced this particular syntax rule of Python's; that not only is >> assignment to None disallowed but that members named None are invalid >> syntax. >> >> The question is, what is the specification for Python. Is it that assignment >> to None is disallowed and the naming of members called None being invalid >> syntax is merely an artefact of the implementation of this, or does Python >> require this... >> >> Michael Foord >> >> >>> In a similar fashion: >>> >>> >>> >>>>>> class Parrot(object): >>>>>> >>>>>> >>> ... pass >>> ... >>> >>> >>>>>> p = Parrot() >>>>>> p.1 = 'spam' >>>>>> >>>>>> >>> File "", line 1 >>> p.1 >>> ^ >>> SyntaxError: invalid syntax >>> >>> >>>>>> setattr(p, '1', 'spam') >>>>>> >>>>>> >>>>>> >>> >>> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com >> >> From tonynelson at georgeanelson.com Mon Jun 9 18:46:31 2008 From: tonynelson at georgeanelson.com (Tony Nelson) Date: Mon, 9 Jun 2008 12:46:31 -0400 Subject: [Python-Dev] Assignment to None In-Reply-To: <484D504C.7060406@voidspace.org.uk> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D504C.7060406@voidspace.org.uk> Message-ID: At 4:46 PM +0100 6/9/08, Michael Foord wrote: >Alex Martelli wrote: >> The problem is more general: what if a member (of some external >> object we're proxying one way or another) is named print (in Python < >> 3), or class, or...? To allow foo.print or bar.class would require >> pretty big changes to Python's parser -- I have vague memories that >> the issue was discussed ages ago (possibly in conjunction with some >> early release of Jython) but never went anywhere much (including >> proposals to automatically append an underscore to such IDs in the >> proxying layer, etc etc). Maybe None in particular is enough of a >> special case (if it just happens to be hugely often used in dotNET >> libraries)? >> > >'None' as a member does occur particularly frequently in the .NET world. > >A halfway house might be to state (something like): > > Python as a language disallows you from having names the same as >keywords or 'None'. An implementation restriction specific to CPython is >that the same restriction also applies to member names. Alternative >implementations are free to not implement this restriction, with the >caveat that code using reserved member names directly will be invalid >syntax for CPython. ... Or perhaps CPython should just stop trying to detect this at compile time. Note that while assignment to ".None" is not allowed, setattr(foo, "None", 1) then referencing ".None" is allowed. >>> f.None = 1 SyntaxError: assignment to None >>> f.None Traceback (most recent call last): File "", line 1, in ? AttributeError: 'Foo' object has no attribute 'None' >>> setattr(f, 'None', 1) > f.None 1 >>> -- ____________________________________________________________________ TonyN.:' ' From tom at vector-seven.com Mon Jun 9 19:02:00 2008 From: tom at vector-seven.com (Thomas Lee) Date: Tue, 10 Jun 2008 03:02:00 +1000 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D504C.7060406@voidspace.org.uk> Message-ID: <484D6208.40503@vector-seven.com> Tony Nelson wrote: > At 4:46 PM +0100 6/9/08, Michael Foord wrote: > >> Alex Martelli wrote: >> >>> The problem is more general: what if a member (of some external >>> object we're proxying one way or another) is named print (in Python < >>> 3), or class, or...? To allow foo.print or bar.class would require >>> pretty big changes to Python's parser -- I have vague memories that >>> the issue was discussed ages ago (possibly in conjunction with some >>> early release of Jython) but never went anywhere much (including >>> proposals to automatically append an underscore to such IDs in the >>> proxying layer, etc etc). Maybe None in particular is enough of a >>> special case (if it just happens to be hugely often used in dotNET >>> libraries)? >>> >>> >> 'None' as a member does occur particularly frequently in the .NET world. >> >> A halfway house might be to state (something like): >> >> Python as a language disallows you from having names the same as >> keywords or 'None'. An implementation restriction specific to CPython is >> that the same restriction also applies to member names. Alternative >> implementations are free to not implement this restriction, with the >> caveat that code using reserved member names directly will be invalid >> syntax for CPython. >> > ... > > Or perhaps CPython should just stop trying to detect this at compile time. > Note that while assignment to ".None" is not allowed, setattr(foo, "None", > 1) then referencing ".None" is allowed. > > I'm +0 on this at the moment, but I can understand the desire for it. Maybe we should stop trying to check for this assignment on attributes? Currently there are separate checks for assignment to None: one for the "foo.None = ..." form, another for the "None = ..." form. Removing the check for the former looks like it would be a one-liner. Cheers, T From tnelson at onresolve.com Mon Jun 9 19:50:01 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Mon, 9 Jun 2008 18:50:01 +0100 Subject: [Python-Dev] Obtaining short file path In-Reply-To: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int> References: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int> Message-ID: <6167796BFEB5D0438720AC212E89A6B007872732@exchange.onresolve.com> If you want a short path name, you should use win32api.GetShortPathName(). Attempting to compute it yourself isn?t as straight forward as you think. From: python-dev-bounces+tnelson=onresolve.com at python.org [mailto:python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Hartwell Bryan Sent: 27 May 2008 08:00 To: python-dev at python.org Subject: [Python-Dev] Obtaining short file path Hi, Purpose: obtaining the system (?short?) path from a full path Background: File dialogs (visual studio) return a full path (e.g. f=?C:\this path has spaces\thisfilenameislongerthan8char.txt?). If this value is provided to Python, it will not recongize this as a file. In fact os.path.isfile(f) doesn?t return false, it crashes. Likewise, when calling executables (from Python) with files as arguments a short path is required. VB FileSystemObject has the ShortPath method, while os.path and path (www.jorendorff.com) modules do not (at least as far as my googling could determine). Why bother creating a COM interface when you?re just going to pass as shell run-time arguments all the values the server is better at computing? System: Python 2.3; Windows XP Sample Code: import win32com.client import time import os,sys import os.path #------------------------------------------------------------- def shortpath(x): z='' for y in x.split('\\'): if len(y.split('.')[0])>8: if ('.' in y): z=z+'\\'+y.split('.')[0][:6].upper()+'~1'+'.'+y.split('.')[1] else: z=z+'\\'+y[:6].upper()+'~1' else: z=z+'\\'+y return z[1:] #------------------------------------------------------------- xlApp = win32com.client.Dispatch("Excel.Application") xlBook = xlApp.ActiveWorkbook savFile = str(sys.argv[1]) rawFile = str(xlBook.Sheets("Timestamp").TextBox2) #print os.path.isfile(savFile) r=shortpath(rawFile) print r try: print os.path.isfile(r) except: print 'something rude' time.sleep(7) Notes: This code does not account for peer paths or files that share the first 8 characters (and file extension). I?m also aware that this is not the normal means for submitting a ?patch?, but in my job function I don?t see myself regularly participating in python development (and I?m probably not savvy enough) so the effort wasn?t worth it. However I still thought others might benefit from what seems to be (to me) a fundamental path function. Do with it, or ignore it, as you please. Cheers, Bryan Hartwell ________________________________ This message is intended only for the use of the intended recipients, and it may be privileged and confidential. If you are not the intended recipient, you are hereby notified that any review, retransmission, conversion to hard copy, copying, circulation or other use of this message is strictly prohibited. If you are not the intended recipient, please notify me immediately by return e-mail, and delete this message from your system. -------------- next part -------------- An HTML attachment was scrubbed... URL: From gisle at activestate.com Mon Jun 9 19:18:10 2008 From: gisle at activestate.com (Gisle Aas) Date: Mon, 9 Jun 2008 19:18:10 +0200 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> Message-ID: On Jun 9, 2008, at 17:12, Alex Martelli wrote: > The problem is more general: what if a member (of some external > object we're proxying one way or another) is named print (in Python < > 3), or class, or...? To allow foo.print or bar.class would require > pretty big changes to Python's parser I simple solution to this would be to introduce syntax that allow you to "quote" identifiers that would otherwise be lexed as keywords. For instance C# uses a prefix '@' for such quoting, so you could write foo. at print, bar. at class and baz. at None if you insist on using such identifiers. Too ugly for Python I guess. Regards, Gisle From tjreedy at udel.edu Mon Jun 9 21:22:39 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 9 Jun 2008 15:22:39 -0400 Subject: [Python-Dev] Assignment to None References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk><484D504C.7060406@voidspace.org.uk> Message-ID: "Tony Nelson" wrote in message news:p04330107c4730af274e7@[192.168.123.162]... | At 4:46 PM +0100 6/9/08, Michael Foord wrote: | Or perhaps CPython should just stop trying to detect this at compile time. | Note that while assignment to ".None" is not allowed, setattr(foo, "None", | 1) then referencing ".None" is allowed. | | >>> f.None = 1 | SyntaxError: assignment to None | >>> f.None | Traceback (most recent call last): | File "", line 1, in ? | AttributeError: 'Foo' object has no attribute 'None' | >>> setattr(f, 'None', 1) | > f.None | 1 I was a little surprised by this. ISTM that f.None should consistently work or not work both for getting and setting. The last result could be considered a bug rusulting from getting disabling depending on None not being set, but having a backdoor that does allow it to be set. I recently read someone (Guido?) saying that a language should start 'strict' because it is much easier to allow something new than disallow something old. Allowing 'None' and possibly other keywords as attributes will not break code (that does not depend on the exceptions). But I presume there was some thought that the restriction might prevent buggy code. At the global level, the subversion does not work: >>> globals()['None'] = 'ha' >>> None >>> dir() ['None', '__builtins__', '__doc__', '__name__', '__package__'] >>> globals()['None'] 'ha' tjr From martin at v.loewis.de Mon Jun 9 22:36:05 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jun 2008 22:36:05 +0200 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk><484D504C.7060406@voidspace.org.uk> Message-ID: <484D9435.5020907@v.loewis.de> > At the global level, the subversion does not work: I think you are misinterpreting what you are seeing. When you refer to the global identifier None, the compiler just knows that it must be the NoneType singleton, and returns it as a constant, without doing any name lookup. So it isn't that assignment of the global None fails to be performed; instead, the compiler just ignores any global with the name None that might have been set. The compiler doesn't do the same thing for attributes, and rightfully so: it would surely be surprising if foo.None would yield the NoneType singleton for any arbitrary foo. Instead, the compiler generates a regular attribute, which then regularly finds the attribute in the object's dictionary if it is there. > I was a little surprised by this. ISTM that f.None should > consistently work or not work both for getting and setting. That will only happen when (if) None becomes a keywords: then foo.None will be a syntax error. You will still be able to put attributes named 'None' (or 'import' or 'if') into any object using getattr/setattr, or by direct access to __dict__. Regards, Martin From martin at v.loewis.de Mon Jun 9 23:15:05 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 09 Jun 2008 23:15:05 +0200 Subject: [Python-Dev] Assignment to None In-Reply-To: <484CE993.5050506@voidspace.org.uk> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> Message-ID: <484D9D59.903@v.loewis.de> > The question is, what is the specification for Python. Now, that's a more interesting question than the question originally asked (which I interpreted as "why does it work the way it works"). The only indication in the specification of that feature I could find was: http://docs.python.org/dev/library/constants.html "Changed in version 2.4: Assignments to None are illegal and raise a SyntaxError." Now, given that this talks about the built-in namespace, this *doesn't* specify that foo.None=1 should also raise a syntax error. So the implementation apparently deviates from the specification. In Python 3, None, True, and False are keywords, so clearly, the intended semantics is also the implemented one (and the language description for 2.x needs to be updated/clarified). Regards, Martin From josiah.carlson at gmail.com Tue Jun 10 03:42:32 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Mon, 9 Jun 2008 18:42:32 -0700 Subject: [Python-Dev] asyncore patch Message-ID: As we approach the 2.6 beta date, and after getting my updated public key pushed to the python.org servers, I would really like to get the asyncore/asynchat patch (with documentation) committed. Previously, we were waiting on documentation, which the last patch had, but which was > 80 columns. That is now fixed. Neal had promised to commit if I got everything in order, but now that I have access, I can commit it. Would it be ok if I committed the changes? Neal, do you want to commit the changes if I post an updated patch with a blurb for the NEWS file? What can I do to get these patch in and included with the beta? - Josiah From musiccomposition at gmail.com Tue Jun 10 04:19:01 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 9 Jun 2008 21:19:01 -0500 Subject: [Python-Dev] asyncore patch In-Reply-To: References: Message-ID: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson wrote: > > Would it be ok if I committed the changes? Neal, do you want to > commit the changes if I post an updated patch with a blurb for the > NEWS file? You are the asyncore maintainer, correct? I believe it's pretty much up to you, then. :) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From josiah.carlson at gmail.com Tue Jun 10 07:01:59 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Mon, 9 Jun 2008 22:01:59 -0700 Subject: [Python-Dev] asyncore patch In-Reply-To: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> Message-ID: On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson wrote: > On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson wrote: >> >> Would it be ok if I committed the changes? Neal, do you want to >> commit the changes if I post an updated patch with a blurb for the >> NEWS file? > > You are the asyncore maintainer, correct? I believe it's pretty much > up to you, then. :) Indeed, but I didn't want to step on anyone's toes. It's committed in revision 64062 for anyone who cares. - Josiah From tjreedy at udel.edu Tue Jun 10 08:11:12 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 10 Jun 2008 02:11:12 -0400 Subject: [Python-Dev] Assignment to None References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk><484D504C.7060406@voidspace.org.uk> <484D9435.5020907@v.loewis.de> Message-ID: ""Martin v. L?wis"" wrote in message news:484D9435.5020907 at v.loewis.de... |> At the global level, the subversion does not work: | | I think you are misinterpreting what you are seeing. No, you didn't understand the code I posted which explicitely demonstrated the same point you repeated. Again: >>> globals()['None'] = 'ha' >>> None >>> dir() ['None', '__builtins__', '__doc__', '__name__', '__package__'] >>> globals()['None'] 'ha' The point of the the last two lines is that the assignment *was* made but was ignored in line 2 [because, obviously, the interpreter bypasses the normal lookup procedure]. From ncoghlan at gmail.com Tue Jun 10 13:04:32 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Jun 2008 21:04:32 +1000 Subject: [Python-Dev] Tuple pack/unpack and the definition of AST Assign nodes In-Reply-To: <484D3C5A.6020808@vector-seven.com> References: <484D337F.8090201@vector-seven.com> <484D3855.703@gmail.com> <484D3C5A.6020808@vector-seven.com> Message-ID: <484E5FC0.80900@gmail.com> Thomas Lee wrote: > Nick Coghlan wrote: >> >> I haven't looked at that code recently, but I believe the ADSL >> sequence in the assignment node is for statements where there are >> actually multiple assignment targets, such as: >> >> >>> p = x, y = 1, 2 >> >>> p, x, y >> ((1, 2), 1, 2) >> >> Cheers, >> Nick. >> > Ah I see. A quick test verifies exactly this: > > >>> import _ast > >>> ast = compile("p = x, y = 1, 2", "", "exec", > _ast.PyCF_ONLY_AST) > >>> ast.body[0] > <_ast.Assign object at 0xb7d0122c> > >>> ast.body[0].targets > [<_ast.Name object at 0xb7d0124c>, <_ast.Tuple object at 0xb7d0128c>] > >>> ast.body[0].value > <_ast.Tuple object at 0xb7d0132c> > >>> > > I thought this would have been implemented as nested Assign nodes, but > I'm obviously wrong. :) Thanks for the clarification. It's one of the key differences between Python's "assignment as a statement" and C-style "assignment as an expression" - nested Assignment nodes are meaningful with the latter approach, but nonsensical with the way Python defines assignment (the rightmost expression is evaluated once and then assigned to each of the target expressions from left to right). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Jun 10 13:12:11 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 10 Jun 2008 21:12:11 +1000 Subject: [Python-Dev] asyncore patch In-Reply-To: References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> Message-ID: <484E618B.60502@gmail.com> Josiah Carlson wrote: > On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson > wrote: >> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson wrote: >>> Would it be ok if I committed the changes? Neal, do you want to >>> commit the changes if I post an updated patch with a blurb for the >>> NEWS file? >> You are the asyncore maintainer, correct? I believe it's pretty much >> up to you, then. :) > > Indeed, but I didn't want to step on anyone's toes. > > It's committed in revision 64062 for anyone who cares. I'm sure asyncore users will appreciate the work put into this, even if there's something of a shortage of such people on this list :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From g.brandl at gmx.net Tue Jun 10 15:08:37 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 10 Jun 2008 15:08:37 +0200 Subject: [Python-Dev] asyncore patch In-Reply-To: <484E618B.60502@gmail.com> References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> <484E618B.60502@gmail.com> Message-ID: Nick Coghlan schrieb: > Josiah Carlson wrote: >> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson >> wrote: >>> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson wrote: >>>> Would it be ok if I committed the changes? Neal, do you want to >>>> commit the changes if I post an updated patch with a blurb for the >>>> NEWS file? >>> You are the asyncore maintainer, correct? I believe it's pretty much >>> up to you, then. :) >> >> Indeed, but I didn't want to step on anyone's toes. >> >> It's committed in revision 64062 for anyone who cares. > > I'm sure asyncore users will appreciate the work put into this, even if > there's something of a shortage of such people on this list :) I don't want to be picky, but it seems the commit broke the tests: test test_asyncore failed -- Traceback (most recent call last): File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py", line 387, in test_recv self.assertEqual(w.fd, fd) AssertionError: 6 != 3 Looks like a minor oversight. Georg From barry at python.org Tue Jun 10 15:10:26 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 10 Jun 2008 09:10:26 -0400 Subject: [Python-Dev] asyncore patch In-Reply-To: References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> <484E618B.60502@gmail.com> Message-ID: <3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 10, 2008, at 9:08 AM, Georg Brandl wrote: > Nick Coghlan schrieb: >> Josiah Carlson wrote: >>> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson >>> wrote: >>>> On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson >>> > wrote: >>>>> Would it be ok if I committed the changes? Neal, do you want to >>>>> commit the changes if I post an updated patch with a blurb for the >>>>> NEWS file? >>>> You are the asyncore maintainer, correct? I believe it's pretty >>>> much >>>> up to you, then. :) >>> Indeed, but I didn't want to step on anyone's toes. >>> It's committed in revision 64062 for anyone who cares. >> I'm sure asyncore users will appreciate the work put into this, >> even if there's something of a shortage of such people on this >> list :) > > I don't want to be picky, but it seems the commit broke the tests: > > test test_asyncore failed -- Traceback (most recent call last): > File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py", > line 387, in test_recv > self.assertEqual(w.fd, fd) > AssertionError: 6 != 3 > > Looks like a minor oversight. This needs to be fixed or backed out if we're going to make beta tomorrow. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSE59Q3EjvBPtnXfVAQLVsQQAsbk6VvlabljNtDX63e9DvBnt8Gh15l0W /yw+lRp9BEThLwitp1ZllFbZElPHsyKaFJn16yOhrysjp/Hf1zWJYgYcHee1m/P4 o4eJMgYlkgIymolJFiviirPj/4YPC8XYkGVcPs1so7XvwVfRvn/u+uhmf1y/+Uwo ql6fkLjXVMo= =iLdD -----END PGP SIGNATURE----- From musiccomposition at gmail.com Tue Jun 10 15:12:33 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 10 Jun 2008 08:12:33 -0500 Subject: [Python-Dev] asyncore patch In-Reply-To: <3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org> References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> <484E618B.60502@gmail.com> <3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org> Message-ID: <1afaf6160806100612l50581aefx736d3e72bafcbeab@mail.gmail.com> On Tue, Jun 10, 2008 at 8:10 AM, Barry Warsaw wrote: >> I don't want to be picky, but it seems the commit broke the tests: >> >> test test_asyncore failed -- Traceback (most recent call last): >> File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py", line >> 387, in test_recv >> self.assertEqual(w.fd, fd) >> AssertionError: 6 != 3 >> >> Looks like a minor oversight. > > This needs to be fixed or backed out if we're going to make beta tomorrow. Oh, I think there are more pressing issues than this, but FWIW I made a issue for it: 3074 -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From josiah.carlson at gmail.com Tue Jun 10 16:56:31 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Tue, 10 Jun 2008 07:56:31 -0700 Subject: [Python-Dev] asyncore patch In-Reply-To: <1afaf6160806100612l50581aefx736d3e72bafcbeab@mail.gmail.com> References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> <484E618B.60502@gmail.com> <3555721E-D086-49E8-8E1B-F79D1967E7A1@python.org> <1afaf6160806100612l50581aefx736d3e72bafcbeab@mail.gmail.com> Message-ID: I'm working on it now. I'll do my best to have a fix by the time I go to work this morning. - Josiah On Tue, Jun 10, 2008 at 6:12 AM, Benjamin Peterson wrote: > On Tue, Jun 10, 2008 at 8:10 AM, Barry Warsaw wrote: >>> I don't want to be picky, but it seems the commit broke the tests: >>> >>> test test_asyncore failed -- Traceback (most recent call last): >>> File "/tmp/python-test/local/lib/python2.6/test/test_asyncore.py", line >>> 387, in test_recv >>> self.assertEqual(w.fd, fd) >>> AssertionError: 6 != 3 >>> >>> Looks like a minor oversight. >> >> This needs to be fixed or backed out if we're going to make beta tomorrow. > > Oh, I think there are more pressing issues than this, but FWIW I made > a issue for it: 3074 > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com > From gnewsg at gmail.com Tue Jun 10 17:26:59 2008 From: gnewsg at gmail.com (Giampaolo Rodola') Date: Tue, 10 Jun 2008 08:26:59 -0700 (PDT) Subject: [Python-Dev] asyncore patch In-Reply-To: References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> Message-ID: On 10 Giu, 07:01, "Josiah Carlson" wrote: > On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson > > wrote: > > On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson wrote: > > >> Would it be ok if I committed the changes? ?Neal, do you want to > >> commit the changes if I post an updated patch with a blurb for the > >> NEWS file? > > > You are the asyncore maintainer, correct? I believe it's pretty much > > up to you, then. :) > > Indeed, but I didn't want to step on anyone's toes. > > It's committed in revision 64062 for anyone who cares. > > ?- Josiah I've started to test the new code by using the pyftpdlib test suite. On Windows all tests passed but on Linux I get some "EBADF Bad file descriptor" errors occurring when using poll() instead of select(). I'll try to look into them today and open a report if necessary. In the meanwhile I noticed some minor bugs in asyncore.py. Take a look at the patch below: Index: Lib/asyncore.py =================================================================== --- Lib/asyncore.py (revisione 64069) +++ Lib/asyncore.py (copia locale) @@ -228,7 +228,7 @@ # passed be connected. try: self.addr = sock.getpeername() - except socket.error: + except socket.error, err: if err[0] == ENOTCONN: # To handle the case where we got an unconnected # socket. @@ -424,7 +424,7 @@ #check for errors err = self.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) if err != 0: - raise socket.error(err, strerror(err)) + raise socket.error(err, _strerror(err)) self.handle_connect_event() self.handle_write() --- Giampaolo http://code.google.com/p/pyftpdlib/ From tom at vector-seven.com Tue Jun 10 17:52:07 2008 From: tom at vector-seven.com (Thomas Lee) Date: Wed, 11 Jun 2008 01:52:07 +1000 Subject: [Python-Dev] Assignment to None In-Reply-To: <484D9D59.903@v.loewis.de> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> Message-ID: <484EA327.2050704@vector-seven.com> Martin v. L?wis wrote: >> The question is, what is the specification for Python. >> > > Now, that's a more interesting question than the question originally > asked (which I interpreted as "why does it work the way it works"). > > The only indication in the specification of that feature I could find > was: > > http://docs.python.org/dev/library/constants.html > > "Changed in version 2.4: Assignments to None are illegal and raise a > SyntaxError." > > Now, given that this talks about the built-in namespace, this *doesn't* > specify that foo.None=1 should also raise a syntax error. > > So the implementation apparently deviates from the specification. > > In Python 3, None, True, and False are keywords, so clearly, the > intended semantics is also the implemented one (and the language > description for 2.x needs to be updated/clarified). > > Interestingly enough, the semantics of True, False and None are different from one another in 2.6: True = "blah" and False = 6 are perfectly legal in Python <=2.6. Funny, I just ran into this. I was trying to figure out why the AST optimization code was breaking test_xmlrpc ... turns out xmlrpclib defines xmlrpclib.True and xmlrpclib.False and the optimizer was trying to resolve them as constants while compiling the module. Ouch. What happened in 3k? Were the constants in xmlrpclib renamed/removed? Cheers, T From josiah.carlson at gmail.com Tue Jun 10 17:58:33 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Tue, 10 Jun 2008 08:58:33 -0700 Subject: [Python-Dev] asyncore patch In-Reply-To: References: <1afaf6160806091919l6a574696x12c5d005ea4a31eb@mail.gmail.com> Message-ID: On Tue, Jun 10, 2008 at 8:26 AM, Giampaolo Rodola' wrote: > On 10 Giu, 07:01, "Josiah Carlson" wrote: >> On Mon, Jun 9, 2008 at 7:19 PM, Benjamin Peterson >> >> wrote: >> > On Mon, Jun 9, 2008 at 8:42 PM, Josiah Carlson wrote: >> >> >> Would it be ok if I committed the changes? Neal, do you want to >> >> commit the changes if I post an updated patch with a blurb for the >> >> NEWS file? >> >> > You are the asyncore maintainer, correct? I believe it's pretty much >> > up to you, then. :) >> >> Indeed, but I didn't want to step on anyone's toes. >> >> It's committed in revision 64062 for anyone who cares. >> >> - Josiah > > > I've started to test the new code by using the pyftpdlib test suite. > On Windows all tests passed but on Linux I get some "EBADF Bad file > descriptor" errors occurring when using poll() instead of select(). > I'll try to look into them today and open a report if necessary. > In the meanwhile I noticed some minor bugs in asyncore.py. Take a look > at the patch below: > > > Index: Lib/asyncore.py > =================================================================== > --- Lib/asyncore.py (revisione 64069) > +++ Lib/asyncore.py (copia locale) > @@ -228,7 +228,7 @@ > # passed be connected. > try: > self.addr = sock.getpeername() > - except socket.error: > + except socket.error, err: > if err[0] == ENOTCONN: > # To handle the case where we got an unconnected > # socket. > @@ -424,7 +424,7 @@ > #check for errors > err = self.socket.getsockopt(socket.SOL_SOCKET, > socket.SO_ERROR) > if err != 0: > - raise socket.error(err, strerror(err)) > + raise socket.error(err, _strerror(err)) > > self.handle_connect_event() > self.handle_write() I have fixed the testcase (it was failing due to the new semantics of the filewrapper automatically duping the file handle), as well as the two NameErrors that Giampaolo pointed out. See revision 64080. - Josiah From exarkun at divmod.com Tue Jun 10 18:04:03 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 10 Jun 2008 12:04:03 -0400 Subject: [Python-Dev] segfault in struct module In-Reply-To: 0 Message-ID: <20080610160403.4714.104530664.divmod.quotient.7701@ohm> exarkun at charm:~$ ~/Projects/python/trunk/python Python 2.6a3+ (trunk:63964, Jun 5 2008, 16:49:12) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import struct >>> struct.pack("357913941c", 'a') Segmentation fault exarkun at charm:~$ The unit test for exactly this case was deleted in r60892. I would like to suggest that just deleting unit tests isn't a very good idea. Jean-Paul From hodgestar+pythondev at gmail.com Tue Jun 10 18:31:13 2008 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Tue, 10 Jun 2008 18:31:13 +0200 Subject: [Python-Dev] Exception.__unicode__ and tp_unicode Message-ID: Originally Python exceptions had no __unicode__ method. In Python 2.5 __unicode__ was added. This led to "unicode(Exception)" failing and so the addition of __unicode__ was reverted [1]. This leaves Python 2.6 in a position where calls to "unicode(Exception(u'\xe1'))" fail as they are equivalent to "uncode(str(Exception(u'\xe1'))" which cannot convert the non-ASCII character to ASCII (or other default encoding) [2]. >From here there are 3 options: 1) Leave things as they are. 2) Add back __unicode__ and have "unicode(Exception)" fail. 3) Add a tp_unicode slot to Python objects and have everything work (at the cost of adding the slot). Each option has its draw backs. Ideally I'd like to see 3) implemented (there are already two volunteers for and some initial stabs at implementing it) but a change to Object is going to need an okay from someone quite high up. Also, if you know of any code this would break, now is the time to let me know. If we can't have 3) I'd like to see us fall back to option 2). Passing unicode exceptions back is useful in a number of common situations (non-English exception messages, database errors, pretty much anywhere that something goes wrong while dealing with potentially non-ASCII text) and encoding to some specific format is usually not an option since there is no way to know where the exception will eventually be caught. Also, unicode(ClassA) already fails for any class that implements __unicode__ so even without this effecting Exception it's already not safe to do u"%s" % SomeClass. Also, there is a readily available work around by doing u"%s" % str(SomeClass). I'm opposed to 1) because a full work around means doing something like: def unicode_exception(e): if len(e.args) == 0: return u"" elif len(e.args) == 1: return unicode(e.args[0]) else: return unicode(e.args) and then using unicode_exception(...) instead of unicode(...) whenever one needs to get a unicode value for an exception. The issue doesn't affect Python 3.0 where unicode(...) is replaced by str(...). [1] http://bugs.python.org/issue1551432 [2] http://bugs.python.org/issue2517 Schiavo Simon From g.brandl at gmx.net Tue Jun 10 18:39:22 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 10 Jun 2008 18:39:22 +0200 Subject: [Python-Dev] Assignment to None In-Reply-To: <484EA327.2050704@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> Message-ID: Thomas Lee schrieb: > What happened in 3k? Were the constants in xmlrpclib renamed/removed? They were removed, as there is no way they can be accessed as attributes of a module now. Georg From dje at google.com Tue Jun 10 19:13:46 2008 From: dje at google.com (Doug Evans) Date: Tue, 10 Jun 2008 10:13:46 -0700 (PDT) Subject: [Python-Dev] bogus comment in Python.h Message-ID: <20080610171346.52EE01C7641@localhost> I spent a bit of time trying to figure out what's going on here (was getting errors regarding missing uintptr_t while trying to compile an external module with Python 2.4). pyport.h now includes stdint.h but can we fix up this in Python.h? /* For uintptr_t, intptr_t */ #ifdef HAVE_STDDEF_H #include #endif I'm guessing removing the inclusion of stddef.h will break something. It's a bit of a wart (or not) that Python.h includes stdlib.h, stddef.h, et.al. but pyport.h includes stdint.h (why not just include them all in one place?). Anyways, to save some confusion for someone trying to track down a problem in the future, could the above comment be removed? --- Python.h (revision 64082) +++ Python.h (working copy) @@ -43,8 +43,6 @@ #ifdef HAVE_UNISTD_H #include #endif - -/* For uintptr_t, intptr_t */ #ifdef HAVE_STDDEF_H #include #endif [I'd suggest rewording it except that I'm not sure exactly what stddef.h is used for. Presumably it's size_t but I didn't want to guess.] From jnoller at gmail.com Tue Jun 10 21:11:55 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 10 Jun 2008 15:11:55 -0400 Subject: [Python-Dev] PEP 371 trunk implementation Message-ID: <4222a8490806101211m5999112ct4a2adb9f92a57789@mail.gmail.com> I've uploaded the first "rolled up" patch against trunk for PEP 371's implementation. This includes docs, tests, module, the works. The only outstanding issues with this are: * Had to comment out several unit tests due to threading._Event and multiprocessing Event API differences. * One test failing consistently on OS/X due to a change that came with rolling into trunk, I'm working on this * Bug http://bugs.python.org/issue1683 needs to have the patch reviewed and applied to trunk. * Full doc-content review The patch is both on the codereview system and the bug tracker: http://bugs.python.org/issue3050 http://codereview.appspot.com/2061 -jesse From barry at python.org Wed Jun 11 01:23:17 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 10 Jun 2008 19:23:17 -0400 Subject: [Python-Dev] Nab those release blockers! In-Reply-To: <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com> References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 9, 2008, at 6:33 AM, Jesse Noller wrote: > For the multiprocessing module all I have left is the docs in rest > format - given I'm a rest newbie, it's taking longer than I thought It's okay if docs don't make it into the beta. That's the kind of thing that can (but still, must) be fixed up before the final release. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSE8M5XEjvBPtnXfVAQIBcQP/S5BMgNS/sOXajnU7eLfXGM39/PwStL8b HOv/MZiHtjmRx5Rtro7lDnjaf9u4Ae+V90a5Pt+aAIANSLenzl1GVcJuuQNP+baa trrTvvBGD3C78au76yOIKRZ0xfvv+rxQXo3mzQ4Yf3GiyOaDdhHMiar6qHlQhesK 1j6rghX/J0Q= =r31d -----END PGP SIGNATURE----- From musiccomposition at gmail.com Wed Jun 11 01:26:25 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 10 Jun 2008 18:26:25 -0500 Subject: [Python-Dev] Nab those release blockers! In-Reply-To: References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com> Message-ID: <1afaf6160806101626y11c5bb4btfbf053f6ee66fdfe@mail.gmail.com> On Tue, Jun 10, 2008 at 6:23 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jun 9, 2008, at 6:33 AM, Jesse Noller wrote: > >> For the multiprocessing module all I have left is the docs in rest format >> - given I'm a rest newbie, it's taking longer than I thought > > It's okay if docs don't make it into the beta. That's the kind of thing > that can (but still, must) be fixed up before the final release. Luckily, it's done now, and I think I will be able to check in multiprocessing tonight or tomorrow. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From jnoller at gmail.com Wed Jun 11 01:44:39 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 10 Jun 2008 19:44:39 -0400 Subject: [Python-Dev] Nab those release blockers! In-Reply-To: <1afaf6160806101626y11c5bb4btfbf053f6ee66fdfe@mail.gmail.com> References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> <788FB19E-65C1-4765-96B0-EC67CB1400D0@gmail.com> <1afaf6160806101626y11c5bb4btfbf053f6ee66fdfe@mail.gmail.com> Message-ID: I'll try to roll an updated patch with the review info tonight for you On Jun 10, 2008, at 7:26 PM, "Benjamin Peterson" wrote: > On Tue, Jun 10, 2008 at 6:23 PM, Barry Warsaw > wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Jun 9, 2008, at 6:33 AM, Jesse Noller wrote: >> >>> For the multiprocessing module all I have left is the docs in rest >>> format >>> - given I'm a rest newbie, it's taking longer than I thought >> >> It's okay if docs don't make it into the beta. That's the kind of >> thing >> that can (but still, must) be fixed up before the final release. > > Luckily, it's done now, and I think I will be able to check in > multiprocessing tonight or tomorrow. > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." From guido at python.org Wed Jun 11 02:07:28 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jun 2008 17:07:28 -0700 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me Message-ID: All, I've been sick since Saturday and still don't feel up to much. I've collected a severe email backlog going back to June 6th. If there's anything someone really needs me to look at ASAP (e.g. a BDFL decision affecting the impending beta release) please send me an email (a followup to this thread is fine) and I'll try to look at it soon. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fijall at gmail.com Wed Jun 11 03:10:17 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 11 Jun 2008 03:10:17 +0200 Subject: [Python-Dev] bug or a feature? Message-ID: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> What do you think about this code: class A: locals()[42] = 98 Seems people rely on it working. Do we consider it part of python language? (Note that you cannot do the same with getattr/setattr which checks if argument is a string) Cheers, fijal From musiccomposition at gmail.com Wed Jun 11 03:14:39 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 10 Jun 2008 20:14:39 -0500 Subject: [Python-Dev] bug or a feature? In-Reply-To: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> Message-ID: <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com> On Tue, Jun 10, 2008 at 8:10 PM, Maciej Fijalkowski wrote: > What do you think about this code: > > class A: > locals()[42] = 98 > > Seems people rely on it working. Do we consider it part of python > language? (Note that you cannot do the same with getattr/setattr which > checks if argument is a string) Seems like a bug to me, but I don't think there is much we can do about it short of making locals a custom dict which rejects none string keys. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at holdenweb.com Wed Jun 11 03:30:56 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 10 Jun 2008 21:30:56 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com> Message-ID: <484F2AD0.5010106@holdenweb.com> Benjamin Peterson wrote: > On Tue, Jun 10, 2008 at 8:10 PM, Maciej Fijalkowski wrote: >> What do you think about this code: >> >> class A: >> locals()[42] = 98 >> >> Seems people rely on it working. Do we consider it part of python >> language? (Note that you cannot do the same with getattr/setattr which >> checks if argument is a string) > > Seems like a bug to me, but I don't think there is much we can do > about it short of making locals a custom dict which rejects none > string keys. > > Given that the documentation has for a long time said: """locals( ) Update and return a dictionary representing the current local symbol table. Warning: The contents of this dictionary should not be modified; changes may not affect the values of local variables used by the interpreter. """ it would be hard for the people who have relied on this behavior to complain if it stopped working. Basically retaining this behavior is pandering to people who have ignored the language specification! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From scott+python-dev at scottdial.com Wed Jun 11 03:36:27 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 10 Jun 2008 21:36:27 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> Message-ID: <484F2C1B.2090404@scottdial.com> Maciej Fijalkowski wrote: > What do you think about this code: > > class A: > locals()[42] = 98 > > Seems people rely on it working. I apologize for my ignorance, but who? Could you please cite something reputable that relies on this detail? -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From fijall at gmail.com Wed Jun 11 03:37:45 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 11 Jun 2008 03:37:45 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <484F2C1B.2090404@scottdial.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> Message-ID: <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> On Wed, Jun 11, 2008 at 3:36 AM, Scott Dial wrote: > Maciej Fijalkowski wrote: >> >> What do you think about this code: >> >> class A: >> locals()[42] = 98 >> >> Seems people rely on it working. > > I apologize for my ignorance, but who? Could you please cite something > reputable that relies on this detail? > It's in tests of sqlalchemy. My question is among the lines "should I bug sqlalchemy guys to remove this, or should I change pypy to accept this". From musiccomposition at gmail.com Wed Jun 11 03:40:16 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 10 Jun 2008 20:40:16 -0500 Subject: [Python-Dev] bug or a feature? In-Reply-To: <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> Message-ID: <1afaf6160806101840j56928e7fiad88dd73f26668c4@mail.gmail.com> On Tue, Jun 10, 2008 at 8:37 PM, Maciej Fijalkowski wrote: > On Wed, Jun 11, 2008 at 3:36 AM, Scott Dial >> >> I apologize for my ignorance, but who? Could you please cite something >> reputable that relies on this detail? >> > > It's in tests of sqlalchemy. My question is among the lines "should I > bug sqlalchemy guys to remove this, or should I change pypy to accept > this". If you have that power, I recommend you tell them to get rid of it. :) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at holdenweb.com Wed Jun 11 03:30:56 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 10 Jun 2008 21:30:56 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com> Message-ID: <484F2AD0.5010106@holdenweb.com> Benjamin Peterson wrote: > On Tue, Jun 10, 2008 at 8:10 PM, Maciej Fijalkowski wrote: >> What do you think about this code: >> >> class A: >> locals()[42] = 98 >> >> Seems people rely on it working. Do we consider it part of python >> language? (Note that you cannot do the same with getattr/setattr which >> checks if argument is a string) > > Seems like a bug to me, but I don't think there is much we can do > about it short of making locals a custom dict which rejects none > string keys. > > Given that the documentation has for a long time said: """locals( ) Update and return a dictionary representing the current local symbol table. Warning: The contents of this dictionary should not be modified; changes may not affect the values of local variables used by the interpreter. """ it would be hard for the people who have relied on this behavior to complain if it stopped working. Basically retaining this behavior is pandering to people who have ignored the language specification! regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From python at rcn.com Wed Jun 11 04:18:28 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Jun 2008 19:18:28 -0700 Subject: [Python-Dev] bug or a feature? References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com> Message-ID: >> What do you think about this code: >> >> class A: >> locals()[42] = 98 >> >> Seems people rely on it working. Do we consider it part of python >> language? (Note that you cannot do the same with getattr/setattr which >> checks if argument is a string) > > Seems like a bug to me, but I don't think there is much we can do > about it short of making locals a custom dict which rejects none > string keys. I see no reason to invent a new custom dict to prevent people from doing something they find to be useful. If you can't segfault with it, who cares. Also, adding another custom type just makes it more difficult to remember what is returned by locals() or globals(). Raymond From musiccomposition at gmail.com Wed Jun 11 04:20:32 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 10 Jun 2008 21:20:32 -0500 Subject: [Python-Dev] bug or a feature? In-Reply-To: References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <1afaf6160806101814t655a83e3i5ba5f6279ef93dbf@mail.gmail.com> Message-ID: <1afaf6160806101920l21ede0d4rdf785088e7076917@mail.gmail.com> On Tue, Jun 10, 2008 at 9:18 PM, Raymond Hettinger wrote: > I see no reason to invent a new custom dict to prevent people from > doing something they find to be useful. If you can't segfault with it, > who cares. Don't worry; I'm not suggesting it. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From guido at python.org Wed Jun 11 05:12:48 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Jun 2008 20:12:48 -0700 Subject: [Python-Dev] pep8ity __future__ In-Reply-To: References: Message-ID: I don't see it the same way; this is a terribly unimportant API, so let's not mess with it. threading.py is worth fixing (a) because it's so popular, and (b) because some folks insisted that the new multiprocessing module have an API that is as similar as possible to threading. IOW The general moratorium on pep8ifying remains; we made a specific exception for threading.py for very specific reasons. --Guido On Sat, Jun 7, 2008 at 2:10 PM, Armin Ronacher wrote: > Hi, > > That's just a flaming-sword thread but I want to mention it nonetheless :-) > > Basically I propose getting rid of __future__._Feature.getMandatoryRelease() > in favour of __future__._Feature.mandatory. That's backwards compatibile > and much more pythonic. Getters/Setters are considered unpythonic and the > getting rid of all that Java-like naming sounds like a good idea to me :) > > If threading/processing gets a pep8ified API with 2.6, why not __future__? > > Proposed patch: http://paste.pocoo.org/show/64512/ > > > Regards, > Armin > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at krypto.org Wed Jun 11 05:42:57 2008 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 10 Jun 2008 20:42:57 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <484CED76.3070703@egenix.com> References: <48397ECC.9070805@cheimes.de> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com> <48490121.5030701@egenix.com> <52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com> <484CED76.3070703@egenix.com> Message-ID: <52dc1c820806102042x19c935cev61739772fddb2173@mail.gmail.com> On Mon, Jun 9, 2008 at 1:44 AM, M.-A. Lemburg wrote: > On 2008-06-09 07:20, Gregory P. Smith wrote: > >> On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg wrote: >> >> On 2008-06-03 01:29, Gregory P. Smith wrote: >>> >>> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum >>>> wrote: >>>> >>>> I will freely admit that I haven't followed this thread in any detail, >>>> >>>>> but if it were up to me, I'd have the 2.6 internal code use PyString >>>>> >>>>> ... >>>> >>>> Should we read this as a BDFL pronouncement and make it so? >>>> >>>> All that would mean change wise is that trunk r63675 as well as possibly >>>> r63672 and r63677 would need to be rolled back and this whole discussion >>>> over if such a big change should have happened would turn into a moot >>>> point. >>>> >>>> I would certainly welcome reverting the change. >>> >>> All that's needed to support PyBytes API in 2.x is a set of #defines >>> that map the new APIs to the PyString names. That's a clean and >>> easily understandable solution. >>> >>> >> Okay, I've reverted r63675 in trunk revision r64048. That leaves all of >> the >> python modules and internals using PyString_ api names instead of PyBytes_ >> api names as they were before. PyBytes_ #define's exist for the >> appropriate >> PyString methods incase anyone wants to use those. >> > > Thanks. > > Programmers interested in the code >> >>> for a PyString API can then still look up the code in stringobject.c, >>> e.g. to find out how a certain special case is handled or to check >>> the ref counting - just like they did for years. >>> >> >> >> The files still exist with the new names. bytesobject.c instead of >> stringobject.c. Those renames were done in the other CLs i mentioned >> which >> have not yet been reverted. The current state seems a bit odd because >> they >> depend on the #defines to cause method definitions to be the PyString_ >> names >> instead of the PyBytes_ names. >> > > Please restore the original state, ie. PyString APIs live in > stringobject.h and stringobject.c. bytesobject.h should then have > the #defines for PyBytes APIs, pointing them to the PyString > names (basically what's currently in stringobject.h). > all done as of 64105 -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Wed Jun 11 06:02:13 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 11 Jun 2008 16:02:13 +1200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> Message-ID: <484F4E45.9060004@canterbury.ac.nz> Maciej Fijalkowski wrote: > What do you think about this code: > > class A: > locals()[42] = 98 > > Seems people rely on it working. Do we consider it part of python > language? Modifying the dict returned by locals() is documented as NOT being guaranteed to work, isn't it? -- Greg From stefan_ml at behnel.de Wed Jun 11 07:15:30 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 11 Jun 2008 07:15:30 +0200 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: References: Message-ID: Hi Guido, glad to hear you're recovering. Guido van Rossum wrote: > anything someone really needs me to look at ASAP (e.g. a BDFL decision > affecting the impending beta release) please send me an email This seems to require a BDFL decision: http://bugs.python.org/issue2997 Executive Summary: PyNumberMethods has been changed on py3k back in 2006 with the nb_divide removal, so it's now incompatible with Py2. But there are three more unused struct members *behind* that field that can be removed for beta1, but have not been removed yet. Should they be removed for cleanliness (patch in the issue) or should nb_divide and nb_inplace_divide instead be restored (rev 43285) to restore backwards compatibility? Thanks, Stefan From greg at krypto.org Wed Jun 11 08:31:23 2008 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 10 Jun 2008 23:31:23 -0700 Subject: [Python-Dev] segfault in struct module In-Reply-To: <20080610160403.4714.104530664.divmod.quotient.7701@ohm> References: <20080610160403.4714.104530664.divmod.quotient.7701@ohm> Message-ID: <52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com> On Tue, Jun 10, 2008 at 9:04 AM, Jean-Paul Calderone wrote: > > exarkun at charm:~$ ~/Projects/python/trunk/python > Python 2.6a3+ (trunk:63964, Jun 5 2008, 16:49:12) > [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import struct > >>> struct.pack("357913941c", 'a') > Segmentation fault > exarkun at charm:~$ > > The unit test for exactly this case was deleted in r60892. I would like to > suggest that just deleting unit tests isn't a very good idea. > > Jean-Paul > That deletion was apparently in the release25-maint branch where this bug has been fixed in order to avoid the test case from allocating 8gb on 64bit hosts where it wouldn't need to raise a MemoryError ( http://bugs.python.org/issue2137). However it looks like it was never committed to trunk. (release25-maint r60793 has a lot of fixes) I'm looking into merging the fixes into trunk. Martin, since you committed 60793 in Feb, any others like this that need merging from release25-maint to trunk off the top of your head? -Greg -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Wed Jun 11 08:32:14 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 11 Jun 2008 08:32:14 +0200 Subject: [Python-Dev] multiprocessing problems Message-ID: Currently, multiprocessing cannot be imported: >>> import multiprocessing Traceback (most recent call last): File "", line 1, in File "/home/gbr/devel/python/Lib/multiprocessing/__init__.py", line 63, in import _multiprocessing AttributeError: 'module' object has no attribute 'BufferTooShort' The test suite passes (at least for some buildbots) because it imports _multiprocessing first, which then in its init function imports multiprocessing to get the BufferTooShort exception. Since BufferTooShort and other exceptions inheriting from ProcessError are simple derived exceptions, why aren't they created in the C module in the first place? Georg From martin at v.loewis.de Wed Jun 11 10:19:06 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Jun 2008 10:19:06 +0200 Subject: [Python-Dev] segfault in struct module In-Reply-To: <52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com> References: <20080610160403.4714.104530664.divmod.quotient.7701@ohm> <52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com> Message-ID: <484F8A7A.7060407@v.loewis.de> > Martin, since you committed 60793 in Feb, any others like this that need > merging from release25-maint to trunk off the top of your head? That's the entire chunk of "Google bug fixes", and yes, all of it needs to be ported to 2.6 still. I'll look into it, unless you volunteer :-) (it doesn't need to be done before the beta, as it's bug fixes only). Regards, Martin From mal at egenix.com Wed Jun 11 10:52:06 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 11 Jun 2008 10:52:06 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <52dc1c820806102042x19c935cev61739772fddb2173@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <483FBCB4.5020007@egenix.com> <52dc1c820806011630y7957ef90n2b7b3441ba9451b5@mail.gmail.com> <4843E884.1060705@egenix.com> <52dc1c820806021521g810d9f1wd282508f8452c13@mail.gmail.com> <52dc1c820806021629k5491e8c0u67e8a6f5247d2368@mail.gmail.com> <48490121.5030701@egenix.com> <52dc1c820806082220n2f0832e1l7af4bc2fecf31987@mail.gmail.com> <484CED76.3070703@egenix.com> <52dc1c820806102042x19c935cev61739772fddb2173@mail.gmail.com> Message-ID: <484F9236.4020101@egenix.com> On 2008-06-11 05:42, Gregory P. Smith wrote: > On Mon, Jun 9, 2008 at 1:44 AM, M.-A. Lemburg wrote: > >> On 2008-06-09 07:20, Gregory P. Smith wrote: >> >>> On Fri, Jun 6, 2008 at 2:19 AM, M.-A. Lemburg wrote: >>> >>> On 2008-06-03 01:29, Gregory P. Smith wrote: >>>> On Mon, Jun 2, 2008 at 4:09 PM, Guido van Rossum >>>>> wrote: >>>>> >>>>> I will freely admit that I haven't followed this thread in any detail, >>>>> >>>>>> but if it were up to me, I'd have the 2.6 internal code use PyString >>>>>> >>>>>> ... >>>>> Should we read this as a BDFL pronouncement and make it so? >>>>> >>>>> All that would mean change wise is that trunk r63675 as well as possibly >>>>> r63672 and r63677 would need to be rolled back and this whole discussion >>>>> over if such a big change should have happened would turn into a moot >>>>> point. >>>>> >>>>> I would certainly welcome reverting the change. >>>> All that's needed to support PyBytes API in 2.x is a set of #defines >>>> that map the new APIs to the PyString names. That's a clean and >>>> easily understandable solution. >>>> >>>> >>> Okay, I've reverted r63675 in trunk revision r64048. That leaves all of >>> the >>> python modules and internals using PyString_ api names instead of PyBytes_ >>> api names as they were before. PyBytes_ #define's exist for the >>> appropriate >>> PyString methods incase anyone wants to use those. >>> >> Thanks. >> >> Programmers interested in the code >>>> for a PyString API can then still look up the code in stringobject.c, >>>> e.g. to find out how a certain special case is handled or to check >>>> the ref counting - just like they did for years. >>>> >>> >>> The files still exist with the new names. bytesobject.c instead of >>> stringobject.c. Those renames were done in the other CLs i mentioned >>> which >>> have not yet been reverted. The current state seems a bit odd because >>> they >>> depend on the #defines to cause method definitions to be the PyString_ >>> names >>> instead of the PyBytes_ names. >>> >> Please restore the original state, ie. PyString APIs live in >> stringobject.h and stringobject.c. bytesobject.h should then have >> the #defines for PyBytes APIs, pointing them to the PyString >> names (basically what's currently in stringobject.h). >> > > all done as of 64105 Thank you ! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 11 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 25 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ncoghlan at gmail.com Wed Jun 11 11:21:12 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Jun 2008 19:21:12 +1000 Subject: [Python-Dev] Assignment to None In-Reply-To: <484EA327.2050704@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> Message-ID: <484F9908.9020009@gmail.com> Thomas Lee wrote: > Martin v. L?wis wrote: >> In Python 3, None, True, and False are keywords, so clearly, the >> intended semantics is also the implemented one (and the language >> description for 2.x needs to be updated/clarified). >> >> > Interestingly enough, the semantics of True, False and None are > different from one another in 2.6: > > True = "blah" and False = 6 are perfectly legal in Python <=2.6. True and False didn't get the same treatment as None because we didn't want to break the follow compatibility workaround for Python version prior to 2.2.2: try: True except NameError: True, False = 1, 0 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Jun 11 11:34:35 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Jun 2008 19:34:35 +1000 Subject: [Python-Dev] Exception.__unicode__ and tp_unicode In-Reply-To: References: Message-ID: <484F9C2B.1070805@gmail.com> Simon Cross wrote: > Originally Python exceptions had no __unicode__ method. In Python 2.5 > __unicode__ was added. This led to "unicode(Exception)" failing and so > the addition of __unicode__ was reverted [1]. > > This leaves Python 2.6 in a position where calls to > "unicode(Exception(u'\xe1'))" fail as they are equivalent to > "uncode(str(Exception(u'\xe1'))" which cannot convert the non-ASCII > character to ASCII (or other default encoding) [2]. > >>From here there are 3 options: > > 1) Leave things as they are. > 2) Add back __unicode__ and have "unicode(Exception)" fail. > 3) Add a tp_unicode slot to Python objects and have everything work > (at the cost of adding the slot). 4) Fix PyObject_Unicode to not retrieve __unicode__ from new-style instances, and instead only look for the method on their types (similar to the way PyObject_Format looks up the __format__ method). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Jun 11 11:40:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Jun 2008 19:40:57 +1000 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: References: Message-ID: <484F9DA9.3030606@gmail.com> Guido van Rossum wrote: > All, > > I've been sick since Saturday and still don't feel up to much. I've > collected a severe email backlog going back to June 6th. If there's > anything someone really needs me to look at ASAP (e.g. a BDFL decision > affecting the impending beta release) please send me an email (a > followup to this thread is fine) and I'll try to look at it soon. > As it involves adding a new standard library module, my proposal [1] to add a typetools.ProxyMixin class to 2.6/3.0 to make it practical to write type-agnostic interface adapter classes as new-style classes needs a pronouncement from yourself or Barry as to whether or not it can go in. I think it's an important tool to add since 3.0 no longer has classic classes around to handle the task, and I'm suggesting a new module for it because none of the existing standard library modules seemed like a good fit for the functionality. Cheers, Nick. [1] http://bugs.python.org/issue643841 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Jun 11 11:50:59 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Jun 2008 19:50:59 +1000 Subject: [Python-Dev] bug or a feature? In-Reply-To: <484F4E45.9060004@canterbury.ac.nz> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F4E45.9060004@canterbury.ac.nz> Message-ID: <484FA003.5020703@gmail.com> Greg Ewing wrote: > Maciej Fijalkowski wrote: >> What do you think about this code: >> >> class A: >> locals()[42] = 98 >> >> Seems people rely on it working. Do we consider it part of python >> language? > > Modifying the dict returned by locals() is documented > as NOT being guaranteed to work, isn't it? > Yep - it just happens to work for class and module level namespaces in CPython. Implementations are also permitted to restrict namespace dictionaries to only accept string keys (I believe Jython does this for performance reasons - CPython just optimised the hell out of normal dictionaries that happen to only contain string keys instead). So I don't see any reason for Maciej to mess with PyPy to support code which deliberately makes use of formally undefined behaviour. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Jun 11 11:54:43 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Jun 2008 19:54:43 +1000 Subject: [Python-Dev] pep8ity __future__ In-Reply-To: References: Message-ID: <484FA0E3.709@gmail.com> Guido van Rossum wrote: > I don't see it the same way; this is a terribly unimportant API, so > let's not mess with it. threading.py is worth fixing (a) because it's > so popular, and (b) because some folks insisted that the new > multiprocessing module have an API that is as similar as possible to > threading. IOW The general moratorium on pep8ifying remains; we made a > specific exception for threading.py for very specific reasons. We're also only *adding* the PEP 8 compliant API to threading, and leaving the old API in-place indefinitely in the 2.x series. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Wed Jun 11 13:35:25 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 07:35:25 -0400 Subject: [Python-Dev] Betas today - I hope Message-ID: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 So I had planned to do a bunch of work last night looking at the release blocker issues, but nature intervened. A bunch of severe thunderstorms knock out my 'net access until this morning. I'll try to find some time during the day to look at the RB issues. Hopefully we can get Guido to look at them too and Pronounce on some of them. Guido please start with: http://bugs.python.org/issue643841 My plan is to begin building the betas tonight, at around 9 or 10pm EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll email the list. If you think we really aren't ready for beta, then I would still like to get a release out today. In that case, we'll call it alpha and delay the betas. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSE+4fXEjvBPtnXfVAQJ4AgP+MD0o3Mw5borZRnQffomXfsAFOMLD0PDr fS5PwhxkVL/qJU7ZM7v9l8j5walI7Boj1aH7w8UQdV0KpEc0ruqWhsPFCOny3L0W WgHMtyD0Wc+GAf6ckpOxxWI4Blg52MVzkhplKHaRJ5c4lNriBKt1o9xeupElWtHq awcHgApvTms= =iNK1 -----END PGP SIGNATURE----- From hodgestar+pythondev at gmail.com Wed Jun 11 13:56:31 2008 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Wed, 11 Jun 2008 13:56:31 +0200 Subject: [Python-Dev] Exception.__unicode__ and tp_unicode In-Reply-To: <484F9C2B.1070805@gmail.com> References: <484F9C2B.1070805@gmail.com> Message-ID: On Wed, Jun 11, 2008 at 11:34 AM, Nick Coghlan wrote: > 4) Fix PyObject_Unicode to not retrieve __unicode__ from new-style > instances, and instead only look for the method on their types (similar to > the way PyObject_Format looks up the __format__ method). Thanks for the suggestion Nick. I've added a patch which implements it to the bug. Schiavo Simon From barry at python.org Wed Jun 11 14:04:43 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 08:04:43 -0400 Subject: [Python-Dev] Nab those release blockers! In-Reply-To: <484D13F9.30105@gmail.com> References: <1afaf6160806081638i6fdaf181if6756476b3a4e7da@mail.gmail.com> <484D13F9.30105@gmail.com> Message-ID: <0E8A956A-07AA-4B3C-A6AE-5E0E04912B5A@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 9, 2008, at 7:28 AM, Nick Coghlan wrote: > Benjamin Peterson wrote: >> As we approach the planned first betas for 2.6/3.0, my heart sinks >> when I see the imposing list of 16 release blockers. [1] Luckily, >> most >> of the issues have patches that just need *your* review. Others, >> namely the Py3k exception issues, may require a little more >> discussion, but I'm sure we can get it done. > > For issue 643841 (providing a ProxyMixin class in a new typetools > module), aside from the minor tweak mentioned in the last couple of > comments, I'm basically looking for a yea or nay from Barry or > Guido. I think it's the best answer we're going to find to the > problem of how to write type-agnostic delegation classes in a Python > world without classic classes. I've updated the issue with my own opinion. Let's see what Guido thinks. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSE+/W3EjvBPtnXfVAQIP0QP7BNuejYBEZcp9n/iF8KicfKIJ9L39pNDA QWSW72ApVnwaLEebl1QPEfF5LBzjVSYqWu5RcuxYnFro61FIi6QSX91LN+fq+o0a YuPTa3P9VAIbrfpZQVT9MsLNTqcxtsrA6YtGyv5K48oPrkKCm/9/8+7EFOXeRmXQ gXBv7K+3ZT4= =qsrW -----END PGP SIGNATURE----- From barry at python.org Wed Jun 11 14:06:55 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 08:06:55 -0400 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: References: Message-ID: <7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 10, 2008, at 8:07 PM, Guido van Rossum wrote: > I've been sick since Saturday and still don't feel up to much. I've > collected a severe email backlog going back to June 6th. If there's > anything someone really needs me to look at ASAP (e.g. a BDFL decision > affecting the impending beta release) please send me an email (a > followup to this thread is fine) and I'll try to look at it soon. Hi Guido, sorry you've been sick! If you're up to it, please pronounce on this issue: http://bugs.python.org/issue643841 It's a long bug thread, but needs a decision. Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSE+/4nEjvBPtnXfVAQJDzQP9FtWLT3+MIBoeRLRqFtXEZxphi+IDU7wp Jtw82cFgKz/wPPxyyGEtnLSmt6duj3MTTftuVsTKS1plqsd6puC2S1VbEOvS9otp xQ0E1mn/xb4B/BvH9n3xkS9FTiYlePkmU61IdAdmSAQ3gtFlwDPB8o5k30RuvhLl S+Ai+mBqPq4= =TUyr -----END PGP SIGNATURE----- From musiccomposition at gmail.com Wed Jun 11 14:15:45 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 11 Jun 2008 07:15:45 -0500 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: <1afaf6160806102017u21d508d3u4e1bfbf3ff155ca2@mail.gmail.com> References: <1afaf6160806101714j3fa18965h44cd2f3092ba3f3c@mail.gmail.com> <1afaf6160806101852v1a53f4f0yebd32f68150fe726@mail.gmail.com> <1afaf6160806101919p774e1dbfm43762e3d09bceff7@mail.gmail.com> <1afaf6160806102017u21d508d3u4e1bfbf3ff155ca2@mail.gmail.com> Message-ID: <1afaf6160806110515w4db61294tf49ceda4c54401f6@mail.gmail.com> [just ccing python-dev] On Tue, Jun 10, 2008 at 10:17 PM, Benjamin Peterson wrote: > On Tue, Jun 10, 2008 at 10:08 PM, Guido van Rossum wrote: >> On Tue, Jun 10, 2008 at 7:19 PM, Benjamin Peterson >> wrote: >>> >>> I think that we should accept Antoine's patch and begin the twilight >>> years of sys.exc_info in favor of passing the exception instances >>> around. This makes for more explicit and less magical code. I don't >>> think there's any sys.exc_info case that can't be rewritten without >>> it. >> >> OK, assuming it works and doesn't break any unittests (or fixes the >> ones it expects to break), and has unittests for the new behavior, I'd >> say go for it. > > Excellent! > >> >>> I think the implicit chaining is assuming a little too much >>> about the needs of the program. >> >> That's why it's on a separate attribute. It can be handy to use when >> you need to debug an exception that happens in an exception handle. >> Sometimes it just helps to know why the handler was being invoked in >> the first place, other times you really want to know the original >> exception because that's the problem you're trying to track down. But >> I believe this is where Collin ran into a brick wall. I still think it >> could be implemented post beta 1. > > Ok. I will make an issue for it in the morning. > > Thanks for your pronouncements. > > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Wed Jun 11 14:23:20 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 11 Jun 2008 07:23:20 -0500 Subject: [Python-Dev] multiprocessing problems In-Reply-To: References: Message-ID: <1afaf6160806110523r707ffbeejf00ecef9aff3e615@mail.gmail.com> On Wed, Jun 11, 2008 at 1:32 AM, Georg Brandl wrote: > Currently, multiprocessing cannot be imported: > >>>> import multiprocessing > Traceback (most recent call last): > File "", line 1, in > File "/home/gbr/devel/python/Lib/multiprocessing/__init__.py", line 63, in > > import _multiprocessing > AttributeError: 'module' object has no attribute 'BufferTooShort' > > The test suite passes (at least for some buildbots) because it imports > _multiprocessing first, which then in its init function imports > multiprocessing > to get the BufferTooShort exception. I fixed it by moving the _multiprocessing import below the exception definitions. > > Since BufferTooShort and other exceptions inheriting from ProcessError are > simple derived exceptions, why aren't they created in the C module in the > first place? They should eventually go to C, but I'll worry about it after the betas. :) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From jnoller at gmail.com Wed Jun 11 14:28:03 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 11 Jun 2008 08:28:03 -0400 Subject: [Python-Dev] multiprocessing problems In-Reply-To: <1afaf6160806110523r707ffbeejf00ecef9aff3e615@mail.gmail.com> References: <1afaf6160806110523r707ffbeejf00ecef9aff3e615@mail.gmail.com> Message-ID: <4222a8490806110528x19e50295h2a9d6f92b7d5a337@mail.gmail.com> On Wed, Jun 11, 2008 at 8:23 AM, Benjamin Peterson wrote: > On Wed, Jun 11, 2008 at 1:32 AM, Georg Brandl wrote: >> Currently, multiprocessing cannot be imported: >> >>>>> import multiprocessing >> Traceback (most recent call last): >> File "", line 1, in >> File "/home/gbr/devel/python/Lib/multiprocessing/__init__.py", line 63, in >> >> import _multiprocessing >> AttributeError: 'module' object has no attribute 'BufferTooShort' >> >> The test suite passes (at least for some buildbots) because it imports >> _multiprocessing first, which then in its init function imports >> multiprocessing >> to get the BufferTooShort exception. > > I fixed it by moving the _multiprocessing import below the exception > definitions. > >> >> Since BufferTooShort and other exceptions inheriting from ProcessError are >> simple derived exceptions, why aren't they created in the C module in the >> first place? > > They should eventually go to C, but I'll worry about it after the betas. :) > Thanks Ben, you beat me to it. I'll add it to the tracker. From ncoghlan at gmail.com Wed Jun 11 14:59:09 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Jun 2008 22:59:09 +1000 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: <7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org> References: <7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org> Message-ID: <484FCC1D.1010605@gmail.com> Barry Warsaw wrote: > On Jun 10, 2008, at 8:07 PM, Guido van Rossum wrote: > >> I've been sick since Saturday and still don't feel up to much. I've >> collected a severe email backlog going back to June 6th. If there's >> anything someone really needs me to look at ASAP (e.g. a BDFL decision >> affecting the impending beta release) please send me an email (a >> followup to this thread is fine) and I'll try to look at it soon. > > Hi Guido, sorry you've been sick! If you're up to it, please pronounce > on this issue: > > http://bugs.python.org/issue643841 > > It's a long bug thread, but needs a decision. I just added a couple of messages at the end that recap the more recent discussions (the early responses to the issue are *really* old...) Cheers, Nick. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Wed Jun 11 16:32:09 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 11 Jun 2008 16:32:09 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> Message-ID: <484FE1E9.2080904@egenix.com> On 2008-06-11 13:35, Barry Warsaw wrote: > So I had planned to do a bunch of work last night looking at the release > blocker issues, but nature intervened. A bunch of severe thunderstorms > knock out my 'net access until this morning. > > I'll try to find some time during the day to look at the RB issues. > Hopefully we can get Guido to look at them too and Pronounce on some of > them. Guido please start with: > > http://bugs.python.org/issue643841 > > My plan is to begin building the betas tonight, at around 9 or 10pm EDT > (0100 to 0200 UTC Thursday). If a showstopper comes up before then, > I'll email the list. If you think we really aren't ready for beta, then > I would still like to get a release out today. In that case, we'll call > it alpha and delay the betas. There are two things I'd like to get in to 3.0: * .transform()/.untransform() methods (this is mostly done, just need to add the methods to PyUnicode, PyBytes and PyByteArray) * cleanup of the PyUnicode_AsString() and PyUnicode_AsStringAndSize() C APIs (these APIs don't fit into the naming scheme used in the Unicode API and have a few other issues as well, see issue 2799; at the very least they should be made interpreter internal, ie. rename them to _PyUnicode_AsString() and _PyUnicode_AsStringAndSize() to prevent their use in extensions) I did not have time in the last few days to work on these and won't in the next few days either. Next week looks much better. If it's ok to make the above changes after the release (whatever you call it ;-), that would be great. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 11 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 25 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ironfroggy at socialserve.com Wed Jun 11 16:06:35 2008 From: ironfroggy at socialserve.com (Calvin Spealman) Date: Wed, 11 Jun 2008 10:06:35 -0400 Subject: [Python-Dev] update_wrapper should preserve staticmethod behavior Message-ID: I'd like to make a claim about the following example, that update_wrapper should be improved to preserve the behavior of known, built-in decorators. In this case, I'm talking about staticmethod. The order I list here feels natural, but it obviously doesn't work. The only reason it doesn't seems to be that it is trying to decorate the descriptor, not the function itself. This is expected, but it could certainly be smart enough to detect a descriptor and attempt to get the actual callable underneath, could it not? It would not work for all cases, of course. >>> def d(f): ... def nf(*a, **kw): ... print "decorated function called" ... return f(*a, **kwargs) ... functools.update_wrapper(nf, f) ... return nf ... >>> class A(object): ... @d ... @staticmethod ... def a(self): ... print "a" ... Traceback (most recent call last): File "", line 1, in File "", line 3, in A File "", line 5, in d File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ python2.5/functools.py", line 33, in update_wrapper setattr(wrapper, attr, getattr(wrapped, attr)) AttributeError: 'staticmethod' object has no attribute '__module__' From walter at livinglogic.de Wed Jun 11 17:15:56 2008 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 11 Jun 2008 17:15:56 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <484FE1E9.2080904@egenix.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> Message-ID: <484FEC2C.6070409@livinglogic.de> M.-A. Lemburg wrote: > On 2008-06-11 13:35, Barry Warsaw wrote: >> So I had planned to do a bunch of work last night looking at the >> release blocker issues, but nature intervened. A bunch of severe >> thunderstorms knock out my 'net access until this morning. >> >> I'll try to find some time during the day to look at the RB issues. >> Hopefully we can get Guido to look at them too and Pronounce on some >> of them. Guido please start with: >> >> http://bugs.python.org/issue643841 >> >> My plan is to begin building the betas tonight, at around 9 or 10pm >> EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before >> then, I'll email the list. If you think we really aren't ready for >> beta, then I would still like to get a release out today. In that >> case, we'll call it alpha and delay the betas. > > There are two things I'd like to get in to 3.0: > > * .transform()/.untransform() methods (this is mostly done, just need > to add the methods to PyUnicode, PyBytes and PyByteArray) What would these methods do? Use the codec machinery without any type checks? I think for transformations we don't need the full codec machinery: We probably don't need extensible error handling. There are transformation that are not invertible, so it doesn't make sense to have both operations in one object. If the operation *is* invertible, two tranformers can be used. Do we really need a registry that maps function named to functions? A simple API might look like this: class TransformInfo: # stateless transformer def transform(self, input): # return stateful incremental transformer def incrementaltransformer(self): # wrap stream in a transforming stream def streamtransformer(self, stream): incrementaltransformer() would return an object that has one method: def transform(self, input, final=False); > [...] Servus, Walter From g.brandl at gmx.net Wed Jun 11 17:48:08 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 11 Jun 2008 17:48:08 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <484FE1E9.2080904@egenix.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> Message-ID: M.-A. Lemburg schrieb: > On 2008-06-11 13:35, Barry Warsaw wrote: >> So I had planned to do a bunch of work last night looking at the release >> blocker issues, but nature intervened. A bunch of severe thunderstorms >> knock out my 'net access until this morning. >> >> I'll try to find some time during the day to look at the RB issues. >> Hopefully we can get Guido to look at them too and Pronounce on some of >> them. Guido please start with: >> >> http://bugs.python.org/issue643841 >> >> My plan is to begin building the betas tonight, at around 9 or 10pm EDT >> (0100 to 0200 UTC Thursday). If a showstopper comes up before then, >> I'll email the list. If you think we really aren't ready for beta, then >> I would still like to get a release out today. In that case, we'll call >> it alpha and delay the betas. > > There are two things I'd like to get in to 3.0: Another thing that should get in is the PEP 3138 implementation, that is, the ascii() builtin and repr() changes. Georg From pje at telecommunity.com Wed Jun 11 18:15:25 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 11 Jun 2008 12:15:25 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com > References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> Message-ID: <20080611161455.CBD703A405F@sparrow.telecommunity.com> At 03:37 AM 6/11/2008 +0200, Maciej Fijalkowski wrote: >On Wed, Jun 11, 2008 at 3:36 AM, Scott Dial > wrote: > > Maciej Fijalkowski wrote: > >> > >> What do you think about this code: > >> > >> class A: > >> locals()[42] = 98 > >> > >> Seems people rely on it working. > > > > I apologize for my ignorance, but who? Could you please cite something > > reputable that relies on this detail? > > > >It's in tests of sqlalchemy. My question is among the lines "should I >bug sqlalchemy guys to remove this, or should I change pypy to accept >this". That test is there to ensure that it interoperates with code using the AddOns library from the Cheeseshop; SQLAlchemy is not the source of the usage. From greg at krypto.org Wed Jun 11 18:55:26 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 11 Jun 2008 09:55:26 -0700 Subject: [Python-Dev] segfault in struct module In-Reply-To: <484F8A7A.7060407@v.loewis.de> References: <20080610160403.4714.104530664.divmod.quotient.7701@ohm> <52dc1c820806102331t7a07e9d4v76f9a3734e953d32@mail.gmail.com> <484F8A7A.7060407@v.loewis.de> Message-ID: <52dc1c820806110955p200b9153x601045f9ae01dd71@mail.gmail.com> On Wed, Jun 11, 2008 at 1:19 AM, "Martin v. L?wis" wrote: > > Martin, since you committed 60793 in Feb, any others like this that need > > merging from release25-maint to trunk off the top of your head? > > That's the entire chunk of "Google bug fixes", and yes, all of it needs > to be ported to 2.6 still. > > I'll look into it, unless you volunteer :-) (it doesn't need to be done > before the beta, as it's bug fixes only). > > Regards, > Martin > No problem, I already did it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Wed Jun 11 18:57:28 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 11 Jun 2008 18:57:28 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <484FEC2C.6070409@livinglogic.de> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> <484FEC2C.6070409@livinglogic.de> Message-ID: <485003F8.40803@egenix.com> On 2008-06-11 17:15, Walter D?rwald wrote: > M.-A. Lemburg wrote: >> On 2008-06-11 13:35, Barry Warsaw wrote: >>> So I had planned to do a bunch of work last night looking at the >>> release blocker issues, but nature intervened. A bunch of severe >>> thunderstorms knock out my 'net access until this morning. >>> >>> I'll try to find some time during the day to look at the RB issues. >>> Hopefully we can get Guido to look at them too and Pronounce on some >>> of them. Guido please start with: >>> >>> http://bugs.python.org/issue643841 >>> >>> My plan is to begin building the betas tonight, at around 9 or 10pm >>> EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before >>> then, I'll email the list. If you think we really aren't ready for >>> beta, then I would still like to get a release out today. In that >>> case, we'll call it alpha and delay the betas. >> >> There are two things I'd like to get in to 3.0: >> >> * .transform()/.untransform() methods (this is mostly done, just need >> to add the methods to PyUnicode, PyBytes and PyByteArray) > > What would these methods do? Use the codec machinery without any type > checks? As discussed in another thread some weeks ago: .transform() and .untransform() use the codecs to apply same-type conversions. They do apply type checks to make sure that the codec does indeed return the same type. E.g. text.transform('xml-escape') or data.transform('base64'). > I think for transformations we don't need the full codec machinery: > ... No need to invent another wheel :-) The codecs already exist for Py2.x and can be used by the .encode()/.decode() methods in Py2.x (where no type checks occur). In Py3.x, .encode()/.decode() only allow conversions of the type unicode <-> bytes. .transform()/.untransform() add conversions of the type unicode <-> unicode or bytes <-> bytes. All other conversions in Py3.x have to go through codecs.encode() and codecs.decode() which are the generic codec access functions from the codec registry. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 11 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 25 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From guido at python.org Wed Jun 11 19:02:55 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 10:02:55 -0700 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> Message-ID: On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw wrote: > So I had planned to do a bunch of work last night looking at the release > blocker issues, but nature intervened. A bunch of severe thunderstorms > knock out my 'net access until this morning. > > I'll try to find some time during the day to look at the RB issues. > Hopefully we can get Guido to look at them too and Pronounce on some of > them. Guido please start with: > > http://bugs.python.org/issue643841 I've added a comment. Let me know if anything I said is unclear. > My plan is to begin building the betas tonight, at around 9 or 10pm EDT > (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll > email the list. If you think we really aren't ready for beta, then I would > still like to get a release out today. In that case, we'll call it alpha > and delay the betas. I'd rather call it beta even if certain things are still known to change in the next release. Beta means we'll have a much easier time pushing back on random other feature change proposals. > - -Barry -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jun 11 19:15:04 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 10:15:04 -0700 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <484FE1E9.2080904@egenix.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> Message-ID: On Wed, Jun 11, 2008 at 7:32 AM, M.-A. Lemburg wrote: > There are two things I'd like to get in to 3.0: > > * .transform()/.untransform() methods (this is mostly done, just need > to add the methods to PyUnicode, PyBytes and PyByteArray) I'm +0 on this. It is very minor syntactic sugar that typically saves you only one import. I expect the argument will nearly always be a literal, e.g. data.transform('gzip') rather than a variable like data.transform(compression_method). But it *is* convenient and can make code more readable, e.g. if compressed: data = data.untransform('gzip') Nobody will have to guess what that does. (IMO the confusion about which direction the transformation goes is theoretical. except perhaps in the case of rot13. :-) > * cleanup of the PyUnicode_AsString() and PyUnicode_AsStringAndSize() > C APIs (these APIs don't fit into the naming scheme used in the > Unicode API and have a few other issues as well, see issue 2799; > at the very least they should be made interpreter internal, ie. > rename them to _PyUnicode_AsString() and _PyUnicode_AsStringAndSize() > to prevent their use in extensions) I'm okay with this too. > I did not have time in the last few days to work on these and won't > in the next few days either. Next week looks much better. > > If it's ok to make the above changes after the release (whatever you > call it ;-), that would be great. That's up to the release manager, but IMO we could have a small list of *specific* things that are not yet implemented in beta 1 but that we know will be in beta 2. This is IMO better than just calling it another alpha, because that keeps the floodgates for more feature change requests open for another month. --Guido > Thanks, > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Jun 11 2008) >>>> >>>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > > ________________________________________________________________________ > 2008-07-07: EuroPython 2008, Vilnius, Lithuania 25 days to go > > :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: > > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jun 11 19:27:20 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 10:27:20 -0700 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: References: Message-ID: On Tue, Jun 10, 2008 at 10:15 PM, Stefan Behnel wrote: > This seems to require a BDFL decision: > > http://bugs.python.org/issue2997 > > Executive Summary: PyNumberMethods has been changed on py3k back in 2006 with > the nb_divide removal, so it's now incompatible with Py2. But there are three > more unused struct members *behind* that field that can be removed for beta1, > but have not been removed yet. Should they be removed for cleanliness (patch > in the issue) or should nb_divide and nb_inplace_divide instead be restored > (rev 43285) to restore backwards compatibility? I've added a recommendation to apply this patch to the bug. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Jun 11 19:29:42 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 10:29:42 -0700 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: <484FCC1D.1010605@gmail.com> References: <7F90796A-E7D2-4C32-A03A-794EFCE94C14@python.org> <484FCC1D.1010605@gmail.com> Message-ID: On Wed, Jun 11, 2008 at 5:59 AM, Nick Coghlan wrote: >> http://bugs.python.org/issue643841 >> >> It's a long bug thread, but needs a decision. > > I just added a couple of messages at the end that recap the more recent > discussions (the early responses to the issue are *really* old...) I've added my opinion to the bug: I am in favor of documenting the current behavior (making it mandatory for all implementations). I would like to hold back on adding a proxy mixin class until we've found some actual use for it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Wed Jun 11 19:37:01 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 13:37:01 -0400 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 11, 2008, at 1:02 PM, Guido van Rossum wrote: > On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw > wrote: >> So I had planned to do a bunch of work last night looking at the >> release >> blocker issues, but nature intervened. A bunch of severe >> thunderstorms >> knock out my 'net access until this morning. >> >> I'll try to find some time during the day to look at the RB issues. >> Hopefully we can get Guido to look at them too and Pronounce on >> some of >> them. Guido please start with: >> >> http://bugs.python.org/issue643841 > > I've added a comment. Let me know if anything I said is unclear. Nope, it was perfect, thanks. >> My plan is to begin building the betas tonight, at around 9 or 10pm >> EDT >> (0100 to 0200 UTC Thursday). If a showstopper comes up before >> then, I'll >> email the list. If you think we really aren't ready for beta, then >> I would >> still like to get a release out today. In that case, we'll call it >> alpha >> and delay the betas. > > I'd rather call it beta even if certain things are still known to > change in the next release. Beta means we'll have a much easier time > pushing back on random other feature change proposals. Sounds good. I'm going to go through the other release critical issues and will follow up on this thread if there are any other questions. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFANPXEjvBPtnXfVAQIMfgQAovATNsivTKAMIUPsue+Tq5OwVlE8zxIf KqmNfXDdyAWhWjnWrao0V73A2v6TKvgL2H6SON6UT1oHvRus1ahLWXQWpTUIdn4L jiYfeek14XoVim8mRz7L8mhxThADPMj3YkhWK448QyZbMkColMqUIVCgvfaQYSxM 2+3zOWQJ/AQ= =eqAr -----END PGP SIGNATURE----- From guido at python.org Wed Jun 11 19:39:26 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 10:39:26 -0700 Subject: [Python-Dev] update_wrapper should preserve staticmethod behavior In-Reply-To: References: Message-ID: Please submit a fix to the issue tracker at bugs.python.org if you care about this. On Wed, Jun 11, 2008 at 7:06 AM, Calvin Spealman wrote: > I'd like to make a claim about the following example, that update_wrapper > should be improved to preserve the behavior of known, built-in decorators. > In this case, I'm talking about staticmethod. The order I list here feels > natural, but it obviously doesn't work. The only reason it doesn't seems to > be that it is trying to decorate the descriptor, not the function itself. > This is expected, but it could certainly be smart enough to detect a > descriptor and attempt to get the actual callable underneath, could it not? > It would not work for all cases, of course. > >>>> def d(f): > ... def nf(*a, **kw): > ... print "decorated function called" > ... return f(*a, **kwargs) > ... functools.update_wrapper(nf, f) > ... return nf > ... >>>> class A(object): > ... @d > ... @staticmethod > ... def a(self): > ... print "a" > ... > Traceback (most recent call last): > File "", line 1, in > File "", line 3, in A > File "", line 5, in d > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/functools.py", > line 33, in update_wrapper > setattr(wrapper, attr, getattr(wrapped, attr)) > AttributeError: 'staticmethod' object has no attribute '__module__' > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Wed Jun 11 19:48:54 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 11 Jun 2008 17:48:54 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?update=5Fwrapper_should_preserve_staticmet?= =?utf-8?q?hod_behavior?= References: Message-ID: Calvin Spealman socialserve.com> writes: > Traceback (most recent call last): > File "", line 1, in > File "", line 3, in A > File "", line 5, in d > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ > python2.5/functools.py", line 33, in update_wrapper > setattr(wrapper, attr, getattr(wrapped, attr)) > AttributeError: 'staticmethod' object has no attribute '__module__' Well, if staticmethod doesn't mirror the original function's __module__ attribute, I'd say staticmethod is the culprit. Since Python grew the update_wrapper function, it seems reasonable to ask that all decorators (or decorator-alikes) provided with Python call update_wrapper. Of course staticmethod is written in C, so is there a C function somewhere providing the same functionality as update_wrapper does? From barry at python.org Wed Jun 11 19:54:49 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 13:54:49 -0400 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 11, 2008, at 1:15 AM, Stefan Behnel wrote: > Guido van Rossum wrote: >> anything someone really needs me to look at ASAP (e.g. a BDFL >> decision >> affecting the impending beta release) please send me an email > > This seems to require a BDFL decision: > > http://bugs.python.org/issue2997 Guido's approved the patch. Please go ahead and apply it. If no one gets to it before tonight, I'll put it in beta 1 if it applies cleanly. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFARanEjvBPtnXfVAQJM/AP+I7gb3VM90c6iCAkvx8EijC2LmrIPtmN2 kSMtMLlyObEZLdGGiPOQdafwx+SeuKxwY2d/RF1VvM2K6fWyWjbw0wt2ZMLxs1UB AVz4PmHSeS23jao2io75wBx4iUTmccte0dDBx6JJYojq6PZeMe4W5lJHuzuyrHWQ i4EnPJHiqag= =HaZV -----END PGP SIGNATURE----- From musiccomposition at gmail.com Wed Jun 11 19:58:25 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 11 Jun 2008 12:58:25 -0500 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: References: Message-ID: <1afaf6160806111058w2be5e0bbh60321954f752c420@mail.gmail.com> On Wed, Jun 11, 2008 at 12:54 PM, Barry Warsaw wrote: >> >> http://bugs.python.org/issue2997 > > Guido's approved the patch. Please go ahead and apply it. If no one gets > to it before tonight, I'll put it in beta 1 if it applies cleanly. I'm going to handle it. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From barry at python.org Wed Jun 11 20:00:42 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 14:00:42 -0400 Subject: [Python-Dev] Have been sick, am behind on mail, let me know if there's anything urgent for me In-Reply-To: <1afaf6160806111058w2be5e0bbh60321954f752c420@mail.gmail.com> References: <1afaf6160806111058w2be5e0bbh60321954f752c420@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 11, 2008, at 1:58 PM, Benjamin Peterson wrote: > On Wed, Jun 11, 2008 at 12:54 PM, Barry Warsaw > wrote: >>> >>> http://bugs.python.org/issue2997 >> >> Guido's approved the patch. Please go ahead and apply it. If no >> one gets >> to it before tonight, I'll put it in beta 1 if it applies cleanly. > > I'm going to handle it. Thanks! You're awesome. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFASynEjvBPtnXfVAQK9YgQAuVbLsvZd+O2RGac/Tqr0DHcCPHm7lOL9 QyqUMzideytmuGhw4IjaIIKhP2G1Dg7iskuCEqkosm5niRKz4vw6/R53kMC5xPRQ Adm+KfCxohXzJoN7+ON89wnM5AqfT1AxeYlhGkKGvxRfjTydIXMTTTSJLC4ztyT6 BWIG49zqEJY= =kLlY -----END PGP SIGNATURE----- From greg at krypto.org Wed Jun 11 20:02:53 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 11 Jun 2008 11:02:53 -0700 Subject: [Python-Dev] bogus comment in Python.h In-Reply-To: <20080610171346.52EE01C7641@localhost> References: <20080610171346.52EE01C7641@localhost> Message-ID: <52dc1c820806111102k1db831b7n19037e4f2050cf91@mail.gmail.com> On Tue, Jun 10, 2008 at 10:13 AM, Doug Evans wrote: > I spent a bit of time trying to figure out what's going on here > (was getting errors regarding missing uintptr_t while trying to compile > an external module with Python 2.4). > pyport.h now includes stdint.h but can we fix up this in Python.h? > > /* For uintptr_t, intptr_t */ > #ifdef HAVE_STDDEF_H > #include > #endif > > I'm guessing removing the inclusion of stddef.h will break something. > It's a bit of a wart (or not) that Python.h includes stdlib.h, stddef.h, > et.al. > but pyport.h includes stdint.h (why not just include them all in one > place?). > > Anyways, to save some confusion for someone trying to track down > a problem in the future, could the above comment be removed? Okay. I changed it to /* For size_t? */ in trunk (2.6). > --- Python.h (revision 64082) > +++ Python.h (working copy) > @@ -43,8 +43,6 @@ > #ifdef HAVE_UNISTD_H > #include > #endif > - > -/* For uintptr_t, intptr_t */ > #ifdef HAVE_STDDEF_H > #include > #endif > > [I'd suggest rewording it except that I'm not sure exactly what stddef.h > is used for. Presumably it's size_t but I didn't want to guess.] > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Wed Jun 11 20:03:17 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 11 Jun 2008 11:03:17 -0700 Subject: [Python-Dev] PEP 8 and optional underscores Message-ID: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> "Function names should be lowercase, with words separated by underscores as necessary to improve readability." -- PEP 8 If I'm reading this correctly, then underscores are not required everywhere. Can some of these be shortened? function:: active_count() method:: Thread.get_name() method:: Thread.is_alive() method:: Thread.is_daemon() method:: Thread.set_daemon(daemonic) In some cases, the mental pronounciation changes and affects my perception of meaning. For example, Thread.setName or Thread.setname both feel like a setter to me, but Thread.set_name causes a mental pause and a momentary double-take (is it the name of a set?). A few months ago, I think there was a PEP 8 discussion rejecting suggestions to make underscores required everywhere (leading to getattr-->get_attr, iteritems-->iter_items, staticmethod->static_method, setdefault->set_default, popitem->pop_item, splitlines->split_lines etc.) Perhaps underscores should only be used when the contracted form lacks clarity. Food for thought, Raymond From musiccomposition at gmail.com Wed Jun 11 20:21:13 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 11 Jun 2008 13:21:13 -0500 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> Message-ID: <1afaf6160806111121p37fd0071ne17cef0013a3412@mail.gmail.com> On Wed, Jun 11, 2008 at 1:03 PM, Raymond Hettinger wrote: > "Function names should be lowercase, with words separated by underscores as > necessary to improve readability." -- PEP 8 > > If I'm reading this correctly, then underscores are not required > everywhere. Can some of these be shortened? > > function:: active_count() > method:: Thread.get_name() > method:: Thread.is_alive() > method:: Thread.is_daemon() > method:: Thread.set_daemon(daemonic) > > In some cases, the mental pronounciation changes and affects my perception > of meaning. For example, Thread.setName or Thread.setname both feel like a > setter to me, but Thread.set_name causes a mental pause and a momentary > double-take (is it the name of a set?). Actually, in this case, I think the Pythonic thing to do would be to use properties. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From theller at ctypes.org Wed Jun 11 20:36:50 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 11 Jun 2008 20:36:50 +0200 Subject: [Python-Dev] rest markup in ctypes docs Message-ID: There are a few cases where the ctypes docs are rendered incorrectly: http://docs.python.org/dev/library/ctypes.html#function-prototypes This looks as if 'prototype' would be a symbol exposed by ctypes; it is not - it is used as a placeholder for the object returned by calls to the ctypes.CFUNCTYPE, ctypes.WINFUNCTYPE, and ctypes.PYFUNCTYPE functions above. The rest markup is like this: .. function:: prototype(address) :noindex: Returns a foreign function at the specified address. Some lines below, it looks like ctypes would export '1', '2', and '4'. Here is the markup: .. function:: prototype(address) :noindex: Returns a foreign function at the specified address. How can this be fixed? -- Thanks, Thomas From theller at ctypes.org Wed Jun 11 20:40:23 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 11 Jun 2008 20:40:23 +0200 Subject: [Python-Dev] rest markup in ctypes docs In-Reply-To: References: Message-ID: Thomas Heller schrieb: > There are a few cases where the ctypes docs are rendered incorrectly: > > http://docs.python.org/dev/library/ctypes.html#function-prototypes > > This looks as if 'prototype' would be a symbol exposed by ctypes; it is > not - it is used as a placeholder for the object returned by calls to > the ctypes.CFUNCTYPE, ctypes.WINFUNCTYPE, and ctypes.PYFUNCTYPE functions above. > > The rest markup is like this: > > > .. function:: prototype(address) > :noindex: > > Returns a foreign function at the specified address. > > Some lines below, it looks like ctypes would export '1', '2', and '4'. > Here is the markup: > Sorry, I meant this: .. data:: 1 :noindex: Specifies an input parameter to the function. > > How can this be fixed? > -- Thanks, Thomas From ironfroggy at socialserve.com Wed Jun 11 20:56:25 2008 From: ironfroggy at socialserve.com (Calvin Spealman) Date: Wed, 11 Jun 2008 14:56:25 -0400 Subject: [Python-Dev] update_wrapper should preserve staticmethod behavior In-Reply-To: References: Message-ID: <58D5E407-8BCC-45C3-BFE8-96F0728DC412@socialserve.com> staticmethod doesn't wrap anything, it just creates a descriptor on the class with a __get__ that returns the original, untouched callable. Doesn't even care _what_ the thing you use it on is (function, other callable, or something else entirely.) This actually shouldn't be attempted on non-staticmethod descriptors, after thinking about it. Can't be sure that desc.__get__(cls) is usable to wrap when, at the end, you will be doing some_instance.a and now had the wrong __get__() signature used. Oh, no! class A(object): @d @some_decorator_returns_a_descriptor def a(): pass What should probably happen here is that d needs to see its decorating a descriptor and itself return a descriptor to pass along the right behavior. So, when you do A().a() you should have d.__get__ (cls, inst) calling some_decorator_returns_a_descriptor.__get__(cls, inst) and acting as if that was the thing it decorated. Of course, this would have the probably unexpected behavior of decorating such things at binding time (ie, when a classmethod is bound) rather than after definition. Not good. They could be cached and this used to implement new functionality that the decorator can be applied to the class method once for each class its bound to (interesting? useful?), but I can't think of a justification myself. Unless any of this other behavior could be justified, I'll provide an update_wrapper() patch to at least become staticmethod smart. On Jun 11, 2008, at 1:48 PM, Antoine Pitrou wrote: > Calvin Spealman socialserve.com> writes: >> Traceback (most recent call last): >> File "", line 1, in >> File "", line 3, in A >> File "", line 5, in d >> File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ >> python2.5/functools.py", line 33, in update_wrapper >> setattr(wrapper, attr, getattr(wrapped, attr)) >> AttributeError: 'staticmethod' object has no attribute '__module__' > > Well, if staticmethod doesn't mirror the original function's > __module__ > attribute, I'd say staticmethod is the culprit. > > Since Python grew the update_wrapper function, it seems reasonable > to ask > that all decorators (or decorator-alikes) provided with Python call > update_wrapper. Of course staticmethod is written in C, so is there a > C function somewhere providing the same functionality as > update_wrapper does? > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ironfroggy%40socialserve.com From solipsis at pitrou.net Wed Jun 11 21:22:49 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 11 Jun 2008 19:22:49 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?update=5Fwrapper_should_preserve_staticmet?= =?utf-8?q?hod=09behavior?= References: <58D5E407-8BCC-45C3-BFE8-96F0728DC412@socialserve.com> Message-ID: Calvin Spealman socialserve.com> writes: > > staticmethod doesn't wrap anything, it just creates a descriptor on > the class with a __get__ that returns the original, untouched > callable. Doesn't even care _what_ the thing you use it on is > (function, other callable, or something else entirely.) FWIW, I still disagree. Technically, it might not "wrap" anything (in the sense that it isn't defined as a function returning another function - which is a narrow definition of a wrapper by the way), but semantically it does. To the non-expert programmer, it is a decorator like any other one. The fact that it is implemented differently from other decorators is not an excuse for it to follow different rules... Unless, of course, there is a good *semantic* reason for staticmethod not to mirror the __module__ attribute. (by the way, does the same apply to classmethod as well?) Regards Antoine. From scott+python-dev at scottdial.com Wed Jun 11 21:23:38 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Wed, 11 Jun 2008 15:23:38 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <20080611161455.CBD703A405F@sparrow.telecommunity.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> Message-ID: <4850263A.3040805@scottdial.com> Phillip J. Eby wrote: > That test is there to ensure that it interoperates with code using the > AddOns library from the Cheeseshop; SQLAlchemy is not the source of the > usage. Now that's interesting. The AddOns library uses class objects as keys in the __dict__, but that doesn't says anything about the usage of locals(). At no point in the AddOns library is locals() abused like this, so even if one asserts that assignment to the dict returned by locals() is a bug, the underlying behavior of interest is whether __dict__ is allowed to have non-string keys. >>> from peak.util.addons import AddOn >>> class C: pass >>> class A(AddOn): pass >>> spam = C() >>> print spam.__dict__ {} >>> A(spam) >>> print spam.__dict__ {: } If non-string keys are not allowed in __dict__, then the AddOns library should be changed to add another dict to the object of interest to track these AddOn instances. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From g.brandl at gmx.net Wed Jun 11 21:25:16 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 11 Jun 2008 21:25:16 +0200 Subject: [Python-Dev] rest markup in ctypes docs In-Reply-To: References: Message-ID: Thomas Heller schrieb: > Thomas Heller schrieb: >> There are a few cases where the ctypes docs are rendered incorrectly: >> >> http://docs.python.org/dev/library/ctypes.html#function-prototypes >> >> This looks as if 'prototype' would be a symbol exposed by ctypes; it is >> not - it is used as a placeholder for the object returned by calls to >> the ctypes.CFUNCTYPE, ctypes.WINFUNCTYPE, and ctypes.PYFUNCTYPE functions above. >> >> The rest markup is like this: >> >> >> .. function:: prototype(address) >> :noindex: >> >> Returns a foreign function at the specified address. You can use .. function:: prototype(address) :noindex: :module: here. (The :module: option gives the function's module name which is empty in this case.) >> Some lines below, it looks like ctypes would export '1', '2', and '4'. >> Here is the markup: >> > Sorry, I meant this: > > ... data:: 1 > :noindex: > > Specifies an input parameter to the function. A simple definition list or table should suffice here. Georg From alexandre at peadrop.com Wed Jun 11 22:54:48 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Wed, 11 Jun 2008 16:54:48 -0400 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> Message-ID: On Wed, Jun 11, 2008 at 7:35 AM, Barry Warsaw wrote: > My plan is to begin building the betas tonight, at around 9 or 10pm EDT > (0100 to 0200 UTC Thursday). If a showstopper comes up before then, I'll > email the list. If you think we really aren't ready for beta, then I would > still like to get a release out today. In that case, we'll call it alpha > and delay the betas. I have two release blockers pending review: http://bugs.python.org/issue2918 http://bugs.python.org/issue2917 I believe both patches are ready to be committed to the py3k branch. However, I would certainly like that someone would review the patches (or at least test them). Right now, I am currently looking at fixing issue 2919 (http://bugs.python.org/issue2919). The profile and the cProfile module differ much more than I originally expected. So, I won't be able to get these two for the beta. I have also been looking at http://bugs.python.org/issue2874, in which Benjamin Peterson proposed an simple solution to fix it. Although I haven't tried his approach, I think I could get this one done for today. Finally, I would like to commit the patch in http://bugs.python.org/issue2523 which fix the quadratic behavior in BufferedReader.read(). It would also be nice to have someone else experienced with the io module to review the patch. Cheers, -- Alexandre From curt at hagenlocher.org Wed Jun 11 23:27:51 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Wed, 11 Jun 2008 14:27:51 -0700 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <484CBD70.3000808@v.loewis.de> Message-ID: On Sun, Jun 8, 2008 at 10:19 PM, "Martin v. L?wis" wrote: > > > So, it's okay to setattr the attribute name "None" but not okay to set > > it directly? Is this deliberate or is it an unintentional side effect > > of parser changes to prevent assignment to None? > > It's deliberate. setattr(o, "foo bar", "baz") also works, even though > "foo bar" is not an identifier. setattr doesn't take the Python grammar > into account, but only the object's structure. Thanks for this example; I found it useful. As Michael says, my primary interest in asking this question is because I'm working on IronPython. Compatibility with CPython is extremely important for us, so I need to understand exactly what behavior is mandated. Here's a recap of the ground this thread has covered: 1. In all versions, direct access to members whose names are reserved keywords (such as "print") is not allowed. 2. In Python 2.5 and 2.6, "True", "False" and "None" are not keywords, but direct assignment to a member named "None" is specifically prevented by the parser. Members named "None" may, however, be read directly if they are present. There is no special handling for "True" or "False". 3. In Python 3.0, "True", "False" and "None" are keywords. This eventually leads to a problem for us in interoperability with other CLR code. One example, as Michael points out, is that "None" is a relatively common member of enumeration types. Now for IronPython 2.0, we're targeting compatibility with CPython 2.5. If we duplicate 2.5's behavior and allow direct reads but not direct writes for a member named None, we'd be okay for the enumeration example. Enumerated values aren't writable anyway. However, this sets us up for a problem with some hypothetical future version of IronPython that targets 3.0 compabililty. At that point, we'll face the unpleasant task of having to choose between compability and interoperability. We haven't really had to do this before now because the convention in CLR-based code is typically that publicly-exposed symbols start with a capital letter -- and all of the existing reserved words in Python are lower-case. It's likely to be a much bigger issue with Jython, given the Java convention of having lower-cased method names. If I recall correctly, Jython handles this by appending a trailing underscore to the imported name and there's no reason why we couldn't do something similar. -- Curt Hagenlocher curt at hagenlocher.org From guido at python.org Wed Jun 11 23:39:21 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 14:39:21 -0700 Subject: [Python-Dev] update_wrapper should preserve staticmethod behavior In-Reply-To: References: <58D5E407-8BCC-45C3-BFE8-96F0728DC412@socialserve.com> Message-ID: On Wed, Jun 11, 2008 at 12:22 PM, Antoine Pitrou wrote: > Calvin Spealman socialserve.com> writes: >> >> staticmethod doesn't wrap anything, it just creates a descriptor on >> the class with a __get__ that returns the original, untouched >> callable. Doesn't even care _what_ the thing you use it on is >> (function, other callable, or something else entirely.) > > FWIW, I still disagree. Technically, it might not "wrap" anything (in the sense > that it isn't defined as a function returning another function - which is a > narrow definition of a wrapper by the way), but semantically it does. To the > non-expert programmer, it is a decorator like any other one. The fact that it is > implemented differently from other decorators is not an excuse for it to follow > different rules... > > Unless, of course, there is a good *semantic* reason for staticmethod not to > mirror the __module__ attribute. > > (by the way, does the same apply to classmethod as well?) Remember (a) these are implemented in C, and (b) they were created in Python 2.2, before we even had decorators (first introduced in 2.4). I'm fine with making them more transparent and conformant to emerging protocols, but they will always be a little different, due to being implemented as objects rather than functional wrappers. The latter approach *can* be used for decorator implementations to, it just is done rarely. I see this as something that can be done post beta 1. It has to be done while carefully remaining backwards compatible though. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Wed Jun 11 23:42:59 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 17:42:59 -0400 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> Message-ID: <45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote: > On Wed, Jun 11, 2008 at 7:35 AM, Barry Warsaw > wrote: >> My plan is to begin building the betas tonight, at around 9 or 10pm >> EDT >> (0100 to 0200 UTC Thursday). If a showstopper comes up before >> then, I'll >> email the list. If you think we really aren't ready for beta, then >> I would >> still like to get a release out today. In that case, we'll call it >> alpha >> and delay the betas. > > I have two release blockers pending review: > > http://bugs.python.org/issue2918 > http://bugs.python.org/issue2917 > > I believe both patches are ready to be committed to the py3k branch. > However, I would certainly like that someone would review the patches > (or at least test them). In IRC, Benjamin agreed to review these and commit them if they look okay. I'm still 3-4 hours away from making the betas, so there's time to do this and still see how the buildbots react. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFBG43EjvBPtnXfVAQJSYAP9FwvFppbeIQX8IhCUtXAXf5jSoGkpTKHQ FT5PBhI/WmM1HTwtx3i06EA/3P+rB2yIVOJhCuK1ORoLvAZO1C8gKUp/8yvgH2KD 0OtlVYgfs4iXwgXe36dI9uBt9AIaohHNnuEgzqzH/IXIcdZ21bp3WKJTbvSgruFX q0K3SkO9ano= =TI1T -----END PGP SIGNATURE----- From barry at python.org Thu Jun 12 00:13:49 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 18:13:49 -0400 Subject: [Python-Dev] Potential beta showstopper (was Re: Have been sick, am behind on mail, let me know if there's anything urgent for me) In-Reply-To: References: Message-ID: <0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Okay, we have a potential showstopper for the betas today. All the 3.0 buildbots are bloody red: http://www.python.org/dev/buildbot/stable/ Running the tests locally on OS X 10.5 and Ubuntu 8.04 confirm that the tests hang after test_xmlrpc. The tests are uninterruptible. I hereby give permission to any dev with commit privileges to checkin a fix, or back out the offending revision. I will be off-line for about 3 more hours and if this can't be fixed by then, we will have to postpone the releases. Once I'm back, I'll be on #python-dev @ freenode. Thanks to Benjamin and Alexandre for helping me get through the current crop of release critical issues today. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFBOHXEjvBPtnXfVAQK6zQP+Nnjt6Q97heAuquZgKEfMxXBL/QzodDgB Th58+w04h0gTVW39Y9NEeKynVPrjsbjxKdrbdZ0i+1pEbqJMXyV65jFmxqb+mYdd 194s5hm+U0KEk7h+64aEkmvFPAgeZ2TGBrqCu5CWDNbWmqerPWuNDzqPxQrWd4Mt wcaic8LFaaU= =4MTf -----END PGP SIGNATURE----- From musiccomposition at gmail.com Thu Jun 12 00:18:26 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 11 Jun 2008 17:18:26 -0500 Subject: [Python-Dev] [Python-3000] Potential beta showstopper (was Re: Have been sick, am behind on mail, let me know if there's anything urgent for me) In-Reply-To: <0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org> References: <0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org> Message-ID: <1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com> On Wed, Jun 11, 2008 at 5:13 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Okay, we have a potential showstopper for the betas today. All the 3.0 > buildbots are bloody red: > > http://www.python.org/dev/buildbot/stable/ > > Running the tests locally on OS X 10.5 and Ubuntu 8.04 confirm that the > tests hang after test_xmlrpc. The tests are uninterruptible. > > I hereby give permission to any dev with commit privileges to checkin a fix, > or back out the offending revision. I will be off-line for about 3 more > hours and if this can't be fixed by then, we will have to postpone the > releases. Already done. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From tjreedy at udel.edu Thu Jun 12 02:32:45 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 11 Jun 2008 20:32:45 -0400 Subject: [Python-Dev] bug or a feature? References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com><20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> Message-ID: "Scott Dial" wrote in message news:4850263A.3040805 at scottdial.com... || If non-string keys are not allowed in __dict__, then the AddOns library | should be changed to add another dict to the object of interest to track | these AddOn instances. There are three possibilities with respect to __dict__ and non-string keys. 1. All implementations must reject such. 2. Each implementation decides for itself. 3. All implementations must allow such. Current, CPython does not reject, eliminating 1). Since, as I understand it, at least 1 other implementation does reject, 3) is also eliminated until Guido decrees otherwise and such implementation(s) change. This leaves 2) as the de facto situation, but this could be made clearer in the docs: "The result of trying to add non-string keys to any __dict__ attribute is implementation defined." This means that portable Python code must act as if 1) were the case. The Data Model chapter of the Reference Manual lists .__dict__ as a special attribute of callables, modules, classes, and instances. It describes __dict__ as a "namespace dictionary" or "implementation of the namespace" thereof. Since namespaces map names (or possibly non-name strings) to objects, this to me implies that an implementation is and should not be required to allow non-strings in __dict__. The same chapter has more than one sentence saying something like "o.x is equivalent to o.__dict__['x']". While one could read this as prohibiting o.__dict__[non_string], one could also read it as being silent, neither allowing nor prohibiting. The builtin interface functions setattr, hasattr, getattr all require strings for accessing the underlying __dict__. Since they could have been defined otherwise, to accept any object as an attribute 'name' (key), this again implies (to me) that __dict__s are only intended and only required to have string keys. Hence, I was initially surprised that 1) above was not true. So I might add something stronger to the docs, something like "The special __dict__ attributes are only intended to hold string keys. If an implementation allows other keys, that is only an current accidental side-effect of considerations of parsimony and efficiency." Contrariwise, if 3) were mandated, then I would think that the xxxattr functions should be changed. Terry Jan Reedy From fijall at gmail.com Thu Jun 12 02:59:27 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 12 Jun 2008 02:59:27 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> Message-ID: <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> On Thu, Jun 12, 2008 at 2:32 AM, Terry Reedy wrote: > > "Scott Dial" wrote in message > news:4850263A.3040805 at scottdial.com... > || If non-string keys are not allowed in __dict__, then the AddOns library > | should be changed to add another dict to the object of interest to track > | these AddOn instances. > > There are three possibilities with respect to __dict__ and non-string keys. > 1. All implementations must reject such. > 2. Each implementation decides for itself. > 3. All implementations must allow such. > > Current, CPython does not reject, eliminating 1). Since, as I understand > it, at least 1 other implementation does reject, 3) is also eliminated > until Guido decrees otherwise and such implementation(s) change. This > leaves 2) as the de facto situation, but this could be made clearer in the > docs: "The result of trying to add non-string keys to any __dict__ > attribute is implementation defined." This means that portable Python code > must act as if 1) were the case. > > The Data Model chapter of the Reference Manual lists .__dict__ as a special > attribute of callables, modules, classes, and instances. It describes > __dict__ as a "namespace dictionary" or "implementation of the namespace" > thereof. Since namespaces map names (or possibly non-name strings) to > objects, this to me implies that an implementation is and should not be > required to allow non-strings in __dict__. > > The same chapter has more than one sentence saying something like "o.x is > equivalent to o.__dict__['x']". While one could read this as prohibiting > o.__dict__[non_string], one could also read it as being silent, neither > allowing nor prohibiting. > > The builtin interface functions setattr, hasattr, getattr all require > strings for accessing the underlying __dict__. Since they could have been > defined otherwise, to accept any object as an attribute 'name' (key), this > again implies (to me) that __dict__s are only intended and only required to > have string keys. Hence, I was initially surprised that 1) above was not > true. So I might add something stronger to the docs, something like "The > special __dict__ attributes are only intended to hold string keys. If an > implementation allows other keys, that is only an current accidental > side-effect of considerations of parsimony and efficiency." > > Contrariwise, if 3) were mandated, then I would think that the xxxattr > functions should be changed. > > Terry Jan Reedy > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com > This is completely irrelevant. This post is not about assigning non-string stuff to __dict__ of class which works completely fine. It's about abusing locals, which are not even given that they'll modify this dict. From tjreedy at udel.edu Thu Jun 12 04:01:50 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 11 Jun 2008 22:01:50 -0400 Subject: [Python-Dev] bug or a feature? References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com><484F2C1B.2090404@scottdial.com><693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com><20080611161455.CBD703A405F@sparrow.telecommunity.com><4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> Message-ID: "Maciej Fijalkowski" wrote in message news:693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5 at mail.gmail.com... | On Thu, Jun 12, 2008 at 2:32 AM, Terry Reedy wrote: | > | > "Scott Dial" wrote in message | > news:4850263A.3040805 at scottdial.com... | > || If non-string keys are not allowed in __dict__, then the AddOns library | > | should be changed to add another dict to the object of interest to track | > | these AddOn instances. | > | > There are three possibilities with respect to __dict__ and non-string keys. | > 1. All implementations must reject such. | > 2. Each implementation decides for itself. | > 3. All implementations must allow such. | > | > Current, CPython does not reject, eliminating 1). Since, as I understand | > it, at least 1 other implementation does reject, 3) is also eliminated | > until Guido decrees otherwise and such implementation(s) change. This | > leaves 2) as the de facto situation, but this could be made clearer in the | > docs: "The result of trying to add non-string keys to any __dict__ | > attribute is implementation defined." This means that portable Python code | > must act as if 1) were the case. | > | > The Data Model chapter of the Reference Manual lists .__dict__ as a special | > attribute of callables, modules, classes, and instances. It describes | > __dict__ as a "namespace dictionary" or "implementation of the namespace" | > thereof. Since namespaces map names (or possibly non-name strings) to | > objects, this to me implies that an implementation is and should not be | > required to allow non-strings in __dict__. | > | > The same chapter has more than one sentence saying something like "o.x is | > equivalent to o.__dict__['x']". While one could read this as prohibiting | > o.__dict__[non_string], one could also read it as being silent, neither | > allowing nor prohibiting. | > | > The builtin interface functions setattr, hasattr, getattr all require | > strings for accessing the underlying __dict__. Since they could have been | > defined otherwise, to accept any object as an attribute 'name' (key), this | > again implies (to me) that __dict__s are only intended and only required to | > have string keys. Hence, I was initially surprised that 1) above was not | > true. So I might add something stronger to the docs, something like "The | > special __dict__ attributes are only intended to hold string keys. If an | > implementation allows other keys, that is only an current accidental | > side-effect of considerations of parsimony and efficiency." | > | > Contrariwise, if 3) were mandated, then I would think that the xxxattr | > functions should be changed. | This is completely irrelevant. This post is not about assigning | non-string stuff to __dict__ of class which works completely fine. My apologies for clipping too much of Scott's post. Here is the rest that came before what I quoted, which makes clear, from first sentence to last line, that *he* *is* talking about assigning non-string stuff to __dict__ of a class. "The AddOns library uses class objects as keys in the __dict__, but that doesn't says anything about the usage of locals(). At no point in the AddOns library is locals() abused like this, so even if one asserts that assignment to the dict returned by locals() is a bug, the underlying behavior of interest is whether __dict__ is allowed to have non-string keys. >>> from peak.util.addons import AddOn >>> class C: pass >>> class A(AddOn): pass >>> spam = C() >>> print spam.__dict__ {} >>> A(spam) >>> print spam.__dict__ {: } " Whether non-strings keys in o.__dict__ 'works fine' depends on one's definition of 'works fine'. In any case, as of 3.0a5, I thinks the docs could better clarify the status of *this* 'feature'. Whatever pronouncement Guido has made has not, that I could find, made it in. | It's about abusing locals, which are not even given that they'll | modify this dict. I thought it settled that that is a bad thing to do. Here the doc is pretty clear. tjr From guido at python.org Thu Jun 12 04:16:36 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 19:16:36 -0700 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org> Message-ID: On Wed, Jun 11, 2008 at 2:42 PM, Barry Warsaw wrote: > On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote: >> I have two release blockers pending review: >> >> http://bugs.python.org/issue2918 >> http://bugs.python.org/issue2917 >> >> I believe both patches are ready to be committed to the py3k branch. >> However, I would certainly like that someone would review the patches >> (or at least test them). > > In IRC, Benjamin agreed to review these and commit them if they look okay. > I'm still 3-4 hours away from making the betas, so there's time to do this > and still see how the buildbots react. That's great, and I see these were committed and the bugs closed. Next time, can we also have some kind of OK from the reviewer (Benjamin, I presume) in the issue tracker? IRC does not leave a usable trail of decisions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jun 12 04:18:53 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 11 Jun 2008 19:18:53 -0700 Subject: [Python-Dev] [Python-3000] Potential beta showstopper (was Re: Have been sick, am behind on mail, let me know if there's anything urgent for me) In-Reply-To: <1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com> References: <0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org> <1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com> Message-ID: On Wed, Jun 11, 2008 at 3:18 PM, Benjamin Peterson wrote: > On Wed, Jun 11, 2008 at 5:13 PM, Barry Warsaw wrote: >> Okay, we have a potential showstopper for the betas today. All the 3.0 >> buildbots are bloody red: >> >> http://www.python.org/dev/buildbot/stable/ >> >> Running the tests locally on OS X 10.5 and Ubuntu 8.04 confirm that the >> tests hang after test_xmlrpc. The tests are uninterruptible. >> >> I hereby give permission to any dev with commit privileges to checkin a fix, >> or back out the offending revision. I will be off-line for about 3 more >> hours and if this can't be fixed by then, we will have to postpone the >> releases. > > Already done. Done what? Fixed, or backed out? Any more details? Old farts who aren't on IRC want to know. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Thu Jun 12 04:21:27 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Jun 2008 22:21:27 -0400 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <45AC8DA6-353F-41F5-997C-15E419D1D1F8@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 11, 2008, at 10:16 PM, Guido van Rossum wrote: > On Wed, Jun 11, 2008 at 2:42 PM, Barry Warsaw > wrote: >> On Jun 11, 2008, at 4:54 PM, Alexandre Vassalotti wrote: >>> I have two release blockers pending review: >>> >>> http://bugs.python.org/issue2918 >>> http://bugs.python.org/issue2917 >>> >>> I believe both patches are ready to be committed to the py3k branch. >>> However, I would certainly like that someone would review the >>> patches >>> (or at least test them). >> >> In IRC, Benjamin agreed to review these and commit them if they >> look okay. >> I'm still 3-4 hours away from making the betas, so there's time to >> do this >> and still see how the buildbots react. > > That's great, and I see these were committed and the bugs closed. Next > time, can we also have some kind of OK from the reviewer (Benjamin, I > presume) in the issue tracker? IRC does not leave a usable trail of > decisions. Good point. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFCIJ3EjvBPtnXfVAQIvXAP/fB6WAD4lS3XEKyEt/FwEoBfgileV4/bj 4km/LPOEtjN4jNks8dSoL+JHsym/76r0uylx3s3jH1sLCKmM7i9tD/SNo/Cim6r9 5fgDmreZkDU+zyyvhdiuxUl+jxmroDPd3R6vPptVbwly4SHsftceZ6jToJUvcNPd p0nDoyCI8VU= =xktw -----END PGP SIGNATURE----- From musiccomposition at gmail.com Thu Jun 12 04:32:46 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 11 Jun 2008 21:32:46 -0500 Subject: [Python-Dev] [Python-3000] Potential beta showstopper (was Re: Have been sick, am behind on mail, let me know if there's anything urgent for me) In-Reply-To: References: <0BFACB06-37F1-447A-8828-2B41DB7C98D1@python.org> <1afaf6160806111518t3655df6bl80a6b4dc589d08c3@mail.gmail.com> Message-ID: <1afaf6160806111932v2371bdbbhca3e25ccea7c5620@mail.gmail.com> On Wed, Jun 11, 2008 at 9:18 PM, Guido van Rossum wrote: > On Wed, Jun 11, 2008 at 3:18 PM, Benjamin Peterson >> >> Already done. > > Done what? Fixed, or backed out? Any more details? Old farts who > aren't on IRC want to know. :-) That would be fixed. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From scott+python-dev at scottdial.com Thu Jun 12 04:44:17 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Wed, 11 Jun 2008 22:44:17 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> Message-ID: <48508D81.2050304@scottdial.com> Maciej Fijalkowski wrote: > On Thu, Jun 12, 2008 at 2:32 AM, Terry Reedy wrote: >> "Scott Dial" wrote in message >> news:4850263A.3040805 at scottdial.com... >> || If non-string keys are not allowed in __dict__, then the AddOns library >> | should be changed to add another dict to the object of interest to track >> | these AddOn instances. >> >> There are three possibilities with respect to __dict__ and non-string keys. >> 1. All implementations must reject such. >> 2. Each implementation decides for itself. >> 3. All implementations must allow such. >> >> Current, CPython does not reject, eliminating 1). Since, as I understand >> it, at least 1 other implementation does reject, 3) is also eliminated >> until Guido decrees otherwise and such implementation(s) change. This >> leaves 2) as the de facto situation, but this could be made clearer in the >> docs: "The result of trying to add non-string keys to any __dict__ >> attribute is implementation defined." This means that portable Python code >> must act as if 1) were the case. > > This is completely irrelevant. This post is not about assigning > non-string stuff to __dict__ of class which works completely fine. > It's about abusing locals, which are not even given that they'll > modify this dict. Not withstanding Terry's respond, this is not as off-topic as you make it out to be. The test case you cited is in fact test this exact 'feature'. And as Terry expounded on it, it was unclear to me whether that was even of allowed. The only reason the test used locals() was because it was the only way to insert a non-string key into the class namespace. >>> class A: ... locals()[42] = 98 >>> A.__dict__ {'__module__': '__main__', 43: 1, '__doc__': None} locals() has to be used because __dict__ is unavailable at definition. >>> class A: ... __dict__[42] = 98 NameError: name '__dict__' is not defined So, while one can write: >>> class A: pass >>> a.__dict__[42] = 98 But that's not quite the same. Nevertheless, it was still unclear whether there was any pronouncements on non-string keys. Sorry for wasting your time. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From barry at python.org Thu Jun 12 07:00:19 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Jun 2008 01:00:19 -0400 Subject: [Python-Dev] No betas today Message-ID: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Much thanks to Benjamin Peterson and Alexandre Vassalotti for providing much help today trying to get the betas out. Unfortunately, it's not gonna happen today. There are two issues blocking the release. http://bugs.python.org/issue3088 http://bugs.python.org/issue3089 The main problem (issue 3089) is that none of the Python 3.0 buildbots are green. Issue 3088 may only happen for me, but test_multiprocessing simply hangs for me on OS X 10.5.3. I'm going to bed now. As soon as we can turn the bots green we can try again. This weekend might be a good time for me. Please help out by resolving these two issues. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFCtZHEjvBPtnXfVAQLrHQQAoq1RCNeawpJHaefuvmZiS4VZzzvtV5Bm HFW6FWyO86NNEcCK6Delthf+H1mgjCaoymHojxBJhWm7UJK0WjMz+0RpF8zcLo0R i8I8SiFLy6kRB5UxlZHS/VXDi8QS92SogRvlnWLBwXNNK6wlgeqK8C0zt1ilL0q6 Fqk/AsGO/fA= =HlEA -----END PGP SIGNATURE----- From thehazard at gmail.com Wed Jun 11 22:25:21 2008 From: thehazard at gmail.com (Daniel Bonekeeper) Date: Wed, 11 Jun 2008 17:25:21 -0300 Subject: [Python-Dev] OFF-TOPIC-BUT-IMPORTANT: End of free internet model by 2012 ? Message-ID: Hi everybody on the list ! Sorry about the off-topic message, but giving the nature of the message and the nature of the list (after all, everybody here uses Internet or develops for web in some way), I decided to post this here and see what you guys think about it. http://ipower.ning.com/netneutrality To quote the article (so you know if it interests you or not): "Bell Canada and TELUS (formerly owned by Verizon) employees officially confirm that by 2012 ISP's all over the globe will reduce Internet access to a TV-like subscription model, only offering access to a small standard amount of commercial sites and require extra fees for every other site you visit. These 'other' sites would then lose all their exposure and eventually shut down, resulting in what could be seen as the end of the Internet." It is important to note that, although the article mentions Bell Canada, it is not that hard to imagine that this will eventually be expanded to all big ISPs. Thanks and again, I apologize for this off-topic message ! Daniel -- What this world needs is a good five-dollar plasma weapon. From ncoghlan at gmail.com Thu Jun 12 11:48:52 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 12 Jun 2008 19:48:52 +1000 Subject: [Python-Dev] update_wrapper should preserve staticmethod behavior In-Reply-To: References: Message-ID: <4850F104.60402@gmail.com> Calvin Spealman wrote: > I'd like to make a claim about the following example, that > update_wrapper should be improved to preserve the behavior of known, > built-in decorators. In this case, I'm talking about staticmethod. The > order I list here feels natural, but it obviously doesn't work. The only > reason it doesn't seems to be that it is trying to decorate the > descriptor, not the function itself. This is expected, but it could > certainly be smart enough to detect a descriptor and attempt to get the > actual callable underneath, could it not? It would not work for all > cases, of course. > > >>> def d(f): > ... def nf(*a, **kw): > ... print "decorated function called" > ... return f(*a, **kwargs) > ... functools.update_wrapper(nf, f) > ... return nf > ... > >>> class A(object): > ... @d > ... @staticmethod > ... def a(self): > ... print "a" > ... > Traceback (most recent call last): > File "", line 1, in > File "", line 3, in A > File "", line 5, in d > File > "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/functools.py", > line 33, in update_wrapper > setattr(wrapper, attr, getattr(wrapped, attr)) > AttributeError: 'staticmethod' object has no attribute '__module__' d expects a function object, this code gives it a nonfunction object - that's a bug in the function definition. Decorators differ in whether or not they are stackable with other function decorators (i.e. they return a function object, or another callable with a compatible API), or whether they can only be used as terminal decorators (i.e. they return something that doesn't look like a function). classmethod and staticmethod fit in the latter category, and I don't see any reason to change them. Consider what happens in your example without the update_wrapper line: >>> def d(f): ... def new(*args, **kwds): ... print "Calling decorated function" ... return f(*args, **kwds) ... return new ... >>> class A(object): ... @d ... @staticmethod ... def a(*args, **kwds): ... print "Method called" ... print "Args:", args ... print "Keywords:", kwds ... >>> A().a() Calling decorated function Traceback (most recent call last): File "", line 1, in File "", line 4, in new TypeError: 'staticmethod' object is not callable If anything, the AttributeError from update wrapper is a net win since the buggy code breaks at class definition time rather than when you try to call the broken method. Cheers, Nick. P.S. Checking for __get__ won't help with detecting non-function descriptors anyway - function objects themselves define that method in order to provide Python's normal instance method binding behaviour. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 12 11:59:04 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 12 Jun 2008 19:59:04 +1000 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <1afaf6160806111121p37fd0071ne17cef0013a3412@mail.gmail.com> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <1afaf6160806111121p37fd0071ne17cef0013a3412@mail.gmail.com> Message-ID: <4850F368.60106@gmail.com> Benjamin Peterson wrote: > On Wed, Jun 11, 2008 at 1:03 PM, Raymond Hettinger wrote: >> "Function names should be lowercase, with words separated by underscores as >> necessary to improve readability." -- PEP 8 >> >> If I'm reading this correctly, then underscores are not required >> everywhere. Can some of these be shortened? >> >> function:: active_count() >> method:: Thread.get_name() >> method:: Thread.is_alive() >> method:: Thread.is_daemon() >> method:: Thread.set_daemon(daemonic) >> >> In some cases, the mental pronounciation changes and affects my perception >> of meaning. For example, Thread.setName or Thread.setname both feel like a >> setter to me, but Thread.set_name causes a mental pause and a momentary >> double-take (is it the name of a set?). > > Actually, in this case, I think the Pythonic thing to do would be to > use properties. Thus exposing those details as thread.name, thread.alive, thread.daemonic (with the first two being read-only and the last read/write)? +1 for that approach on the threading.Thread and multiprocessing.Process classes. Not sure what to do about the active thread count function - shortening it as Raymond suggests would probably be fine. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pje at telecommunity.com Thu Jun 12 12:01:18 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 12 Jun 2008 06:01:18 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> Message-ID: <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote: >The Data Model chapter of the Reference Manual lists .__dict__ as a special >attribute of callables, modules, classes, and instances. It describes >__dict__ as a "namespace dictionary" or "implementation of the namespace" >thereof. Since namespaces map names (or possibly non-name strings) to >objects, this to me implies that an implementation is and should not be >required to allow non-strings in __dict__. > >The same chapter has more than one sentence saying something like "o.x is >equivalent to o.__dict__['x']". While one could read this as prohibiting >o.__dict__[non_string], one could also read it as being silent, neither >allowing nor prohibiting. As it happens, most objects' __dict__ slots are settable by default, and *require* that you set it to a dict or subclass thereof. This seems (to me) to imply that a standard dictionary (i.e. one supporting keys of any type) is required. (In the sense that a dict subclass which rejects non-strings would be violating the Liskov principle.) From ncoghlan at gmail.com Thu Jun 12 12:02:00 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 12 Jun 2008 20:02:00 +1000 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <484CBD70.3000808@v.loewis.de> Message-ID: <4850F418.2040301@gmail.com> Curt Hagenlocher wrote: > If I recall correctly, Jython handles this by appending a trailing > underscore to the imported name and there's no reason why we couldn't > do something similar. It also has the virtue of being the common convention for attribute names that shadow keywords even in CPython (e.g. unittest.TestCase.assert_). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pje at telecommunity.com Thu Jun 12 12:06:32 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 12 Jun 2008 06:06:32 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.co m> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> Message-ID: <20080612100603.7A8203A40B0@sparrow.telecommunity.com> At 02:59 AM 6/12/2008 +0200, Maciej Fijalkowski wrote: >It's about abusing locals, which are not even given that they'll >modify this dict. Note that class bodies are a special case: as of PEP 3115, it's possible for a class body's locals to be a non-dictionary object, so it makes no sense to make a class body's locals() or f_locals return some *other* object. Meanwhile, as a practicality-beats-purity matter, you're going to run into compatibility problems with a fair number of libraries in the field (including Zope and Twisted, and anything using them) if you *don't* support locals() modification in class bodies. (Those other libraries don't use non-string keys like AddOns does, but they *do* depend on modifying a class body's frame locals.) From ncoghlan at gmail.com Thu Jun 12 12:35:40 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 12 Jun 2008 20:35:40 +1000 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> Message-ID: <4850FBFC.9070501@gmail.com> Guido van Rossum wrote: > On Wed, Jun 11, 2008 at 4:35 AM, Barry Warsaw wrote: >> So I had planned to do a bunch of work last night looking at the release >> blocker issues, but nature intervened. A bunch of severe thunderstorms >> knock out my 'net access until this morning. >> >> I'll try to find some time during the day to look at the RB issues. >> Hopefully we can get Guido to look at them too and Pronounce on some of >> them. Guido please start with: >> >> http://bugs.python.org/issue643841 > > I've added a comment. Let me know if anything I said is unclear. The bugtracker seems to be offline atm - I'll reply there once I can get to it again (as well as switching this issue back to being a documentation one). I don't think we're going to see a major clamor for a value-based delegation mixin in the standard library until people using classic classes for value-based delegation start making serious attempts to port to Py3k (where that approach is no longer available). At the moment, such classes only need to care about the methods they want to fiddle with, leaving everything else to __getattr__ based delegation. I've pushed as hard as I'm personally willing to for this without convincing anyone else that it's worth doing, so I'll start working on a documentation patch for the language reference instead which explicitly splits the special methods into the following categories: 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in the instance dictionary MUST NOT have any ill effects. (All tp_* C-level slots and slots looked up via _PyType_Lookup will fit into this category) 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the instance dictionary MAY have ill effects. (slots such as __enter__ and __exit__ that are looked up via normal attribute lookup in CPython will fit into this category) 3. Technically a subcategory of group 1, these are special methods which can affect the interpreter's behaviour by their mere definition on a type. (The __get__, __set__ and __delete__ descriptor protocol methods fall into this category) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From antonxx at gmx.de Thu Jun 12 11:48:40 2008 From: antonxx at gmx.de (anton) Date: Thu, 12 Jun 2008 09:48:40 +0000 (UTC) Subject: [Python-Dev] bugs.python.org down Message-ID: Hi, could someone look for this ... or I am the only one which noticed that. ping works but http://bugs.python.org giges after a timeout: Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET /. Reason: Error reading from remote server Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3 OpenSSL/0.9.8c Server at bugs.python.org Port 80 From jnoller at gmail.com Thu Jun 12 13:08:52 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Jun 2008 07:08:52 -0400 Subject: [Python-Dev] [Python-3000] No betas today In-Reply-To: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org> References: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org> Message-ID: <5F4CD1A3-867F-4E37-863A-C2B9324F02DD@gmail.com> That's odd - I've been doing all of the mp work on osx 10.5.3 - I'll run the units in a loop today to see if I can catch it. -Jesse On Jun 12, 2008, at 1:00 AM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Much thanks to Benjamin Peterson and Alexandre Vassalotti for > providing much help today trying to get the betas out. > Unfortunately, it's not gonna happen today. There are two issues > blocking the release. > > http://bugs.python.org/issue3088 > http://bugs.python.org/issue3089 > > The main problem (issue 3089) is that none of the Python 3.0 > buildbots are green. Issue 3088 may only happen for me, but > test_multiprocessing simply hangs for me on OS X 10.5.3. > > I'm going to bed now. As soon as we can turn the bots green we can > try again. This weekend might be a good time for me. > > Please help out by resolving these two issues. > > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iQCVAwUBSFCtZHEjvBPtnXfVAQLrHQQAoq1RCNeawpJHaefuvmZiS4VZzzvtV5Bm > HFW6FWyO86NNEcCK6Delthf+H1mgjCaoymHojxBJhWm7UJK0WjMz+0RpF8zcLo0R > i8I8SiFLy6kRB5UxlZHS/VXDi8QS92SogRvlnWLBwXNNK6wlgeqK8C0zt1ilL0q6 > Fqk/AsGO/fA= > =HlEA > -----END PGP SIGNATURE----- > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/jnoller%40gmail.com From hodgestar+pythondev at gmail.com Thu Jun 12 13:31:43 2008 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Thu, 12 Jun 2008 13:31:43 +0200 Subject: [Python-Dev] bugs.python.org down In-Reply-To: References: Message-ID: On Thu, Jun 12, 2008 at 11:48 AM, anton wrote: > ping works but http://bugs.python.org giges after a timeout: Seeing the same thing here, so it's not just you. Schiavo Simon From barry at python.org Thu Jun 12 13:33:13 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Jun 2008 07:33:13 -0400 Subject: [Python-Dev] bugs.python.org down In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 12, 2008, at 5:48 AM, anton wrote: > could someone look for this ... or I am the only one which noticed > that. > > ping works but http://bugs.python.org giges after a timeout: > > > Proxy Error > > The proxy server received an invalid response from an upstream server. > The proxy server could not handle the request GET /. > > Reason: Error reading from remote server > > Apache/2.2.3 (Debian) mod_python/3.2.10 Python/2.4.4 mod_ssl/2.2.3 > OpenSSL/0.9.8c Server at bugs.python.org Port 80 I can't even ssh into the machine this morning, so I've forwarded this off to the python.org sysadmins. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFEJenEjvBPtnXfVAQL1DwP/cpAACpa7KTL8BZcuQGcqgmbez1PmtXBG jl85PaULlm6LAvTjtZMy/n2ribZ7TTGLRAHkEoGiQyN03xHFn+w8v6XpXwmvRMsw 19oLBxc3zOptNSfQwSUwdiSHwAdTPW9XjoR0xf7TwOQJZQ5PFtCPniQT2YxVnj4X ZD/VmrIasUo= =Jr3d -----END PGP SIGNATURE----- From cfbolz at gmx.de Thu Jun 12 13:34:44 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 12 Jun 2008 13:34:44 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> Message-ID: <485109D4.2070407@gmx.de> Phillip J. Eby wrote: > At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote: >> The Data Model chapter of the Reference Manual lists .__dict__ as a >> special >> attribute of callables, modules, classes, and instances. It describes >> __dict__ as a "namespace dictionary" or "implementation of the namespace" >> thereof. Since namespaces map names (or possibly non-name strings) to >> objects, this to me implies that an implementation is and should not be >> required to allow non-strings in __dict__. >> >> The same chapter has more than one sentence saying something like "o.x is >> equivalent to o.__dict__['x']". While one could read this as prohibiting >> o.__dict__[non_string], one could also read it as being silent, neither >> allowing nor prohibiting. > > As it happens, most objects' __dict__ slots are settable by default, and > *require* that you set it to a dict or subclass thereof. This is wrong for types: $ python Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class A(object): pass ... >>> A.__dict__ = {} Traceback (most recent call last): File "", line 1, in AttributeError: attribute '__dict__' of 'type' objects is not writable In fact it seems to me that there is no way to set non-string keys into a type dict after class creation: Cls.__dict__ supports no setitem, setattr checks for a string argument. I think there are good arguments for not allowing strings keys in type dicts, or at least leaving it up to the implementation. Using non-string keys in type dicts is relatively awkward and allowing them makes many interesting optimizations (like method caches) a _lot_ harder to get right. Cheers, Carl Friedrich Bolz From barry at python.org Thu Jun 12 13:35:15 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Jun 2008 07:35:15 -0400 Subject: [Python-Dev] [Python-3000] No betas today In-Reply-To: <5F4CD1A3-867F-4E37-863A-C2B9324F02DD@gmail.com> References: <3F811617-7E55-4290-BB1E-14229E563C5F@python.org> <5F4CD1A3-867F-4E37-863A-C2B9324F02DD@gmail.com> Message-ID: <217F21B5-0747-46A4-8B44-11FA81D83F68@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 12, 2008, at 7:08 AM, Jesse Noller wrote: > That's odd - I've been doing all of the mp work on osx 10.5.3 - I'll > run the units in a loop today to see if I can catch it. > -Jesse Thanks. It's definitely reproducible on both my Leopard boxes. I haven't tried it on my Tiger box. The buildbots seem unhappy too. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFEJ9HEjvBPtnXfVAQJlNwQAsG2M/hNORYSuCsmKHU+nvbX3cASiFlhj IIhNfdrtkRbjXBaDgxrr/Ps1+Q3I56lpSg5SDEWxXVufNdBdHXGAKKxnrYN22a1x Nwuu+DXOGYmzBcQc2BS5L9/31yeDmSFBMP257BBstCZww97pt8VRLKpUL2Wzfzny MZb6Km/NXwQ= =Ab3l -----END PGP SIGNATURE----- From cfbolz at gmx.de Thu Jun 12 13:34:44 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 12 Jun 2008 13:34:44 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> Message-ID: <485109D4.2070407@gmx.de> Phillip J. Eby wrote: > At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote: >> The Data Model chapter of the Reference Manual lists .__dict__ as a >> special >> attribute of callables, modules, classes, and instances. It describes >> __dict__ as a "namespace dictionary" or "implementation of the namespace" >> thereof. Since namespaces map names (or possibly non-name strings) to >> objects, this to me implies that an implementation is and should not be >> required to allow non-strings in __dict__. >> >> The same chapter has more than one sentence saying something like "o.x is >> equivalent to o.__dict__['x']". While one could read this as prohibiting >> o.__dict__[non_string], one could also read it as being silent, neither >> allowing nor prohibiting. > > As it happens, most objects' __dict__ slots are settable by default, and > *require* that you set it to a dict or subclass thereof. This is wrong for types: $ python Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class A(object): pass ... >>> A.__dict__ = {} Traceback (most recent call last): File "", line 1, in AttributeError: attribute '__dict__' of 'type' objects is not writable In fact it seems to me that there is no way to set non-string keys into a type dict after class creation: Cls.__dict__ supports no setitem, setattr checks for a string argument. I think there are good arguments for not allowing strings keys in type dicts, or at least leaving it up to the implementation. Using non-string keys in type dicts is relatively awkward and allowing them makes many interesting optimizations (like method caches) a _lot_ harder to get right. Cheers, Carl Friedrich Bolz From barry at python.org Thu Jun 12 13:49:42 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Jun 2008 07:49:42 -0400 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> Message-ID: <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 11, 2008, at 2:03 PM, Raymond Hettinger wrote: > "Function names should be lowercase, with words separated by > underscores as necessary to improve readability." -- PEP 8 > > If I'm reading this correctly, then underscores are not required > everywhere. Can some of these be shortened? > > function:: active_count() > method:: Thread.get_name() > method:: Thread.is_alive() > method:: Thread.is_daemon() > method:: Thread.set_daemon(daemonic) +1 on opting for properties in the specific cases here where it makes sense. > In some cases, the mental pronounciation changes and affects my > perception of meaning. For example, Thread.setName or > Thread.setname both feel like a setter to me, but Thread.set_name > causes a mental pause and a momentary double-take (is it the name of > a set?). > > A few months ago, I think there was a PEP 8 discussion rejecting > suggestions to make underscores required everywhere (leading to > getattr-->get_attr, iteritems-->iter_items, staticmethod- > >static_method, setdefault->set_default, popitem->pop_item, > splitlines->split_lines etc.) > > Perhaps underscores should only be used when the contracted form > lacks clarity. There are some builtins that I wish had used underscores to make them easier to read. isinstance() and issubclass() come to mind. OTOH, getattr and iteritems are already contractions, so leaving them underscoreless doesn't bother me too much. For the others you mention, well I'll invoke ZoP 12 and 13. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFENVnEjvBPtnXfVAQIVhAQAn6liMZxAFKPmfvHp7UQLBdCEiI0rkGjR OCxaEQLxpeuHfDGMS47kVJhyAwo3skKzv9Yv1+Vuor7SaP9hAO3h4r3SO2wM9jDF jwrx6ajm9wtsTzSrh9QkYzOyegeMAfj3p8gLZ+eHiRdFHXj++biunvnn4GPl7O5x E84j8rNlWkU= =LYDl -----END PGP SIGNATURE----- From pje at telecommunity.com Thu Jun 12 14:25:02 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 12 Jun 2008 08:25:02 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <485109D4.2070407@gmx.de> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> <485109D4.2070407@gmx.de> Message-ID: <20080612122436.4D18F3A405F@sparrow.telecommunity.com> At 01:34 PM 6/12/2008 +0200, Carl Friedrich Bolz wrote: >Phillip J. Eby wrote: > > As it happens, most objects' __dict__ slots are settable by default, and > > *require* that you set it to a dict or subclass thereof. > >This is wrong for types: Which is why I said "most" - to exclude types, and objects that don't have a __dict__ slot to begin with. >I think there are good arguments for not allowing strings keys in type >dicts, or at least leaving it up to the implementation. That may well be, but there is nothing in Python's spec that I'm aware of that *forbids* it. For example the type() constructor doc doesn't say anything about using string-only keys in the class dictionary. >Using non-string >keys in type dicts is relatively awkward and allowing them makes many >interesting optimizations (like method caches) a _lot_ harder to get right. Really? Why? Having non-string dict keys is NOT the same thing as having non-string attribute names, so attribute name lookups should be unaffected. (Attribute names *are* required to be strings, and -- as far as I know -- always have been.) From facundobatista at gmail.com Thu Jun 12 14:29:24 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 12 Jun 2008 09:29:24 -0300 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> Message-ID: 2008/6/12 Barry Warsaw : >> function:: active_count() >> method:: Thread.get_name() >> method:: Thread.is_alive() >> method:: Thread.is_daemon() >> method:: Thread.set_daemon(daemonic) > > +1 on opting for properties in the specific cases here where it makes sense. I'm +1 too... but which is the normal procedure here? Should it be... 2.n : .is_alive() 2.n+1 : .is_alive() (deprecated), .alive (recommended) 2.n+2 : .alive ...? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From facundobatista at gmail.com Thu Jun 12 14:34:49 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 12 Jun 2008 09:34:49 -0300 Subject: [Python-Dev] bugs.python.org down In-Reply-To: References: Message-ID: 2008/6/12 anton : > ping works but http://bugs.python.org giges after a timeout: I can enter into it. Also note that my issues showing system [1] was updated 6.5 hours ago, so it was up at that time (my system goes and get some info twice a day, you have the "last updated" info down on that page). Regards, [1] http://www.taniquetil.com.ar/cgi-bin/pytickets.py -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From musiccomposition at gmail.com Thu Jun 12 14:54:02 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 12 Jun 2008 07:54:02 -0500 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> Message-ID: <1afaf6160806120554p1e1d617fidcdecee16a0c63b7@mail.gmail.com> On Thu, Jun 12, 2008 at 7:29 AM, Facundo Batista wrote: >> +1 on opting for properties in the specific cases here where it makes sense. If you can get Guido to agree to it, I'll implement it. > > I'm +1 too... but which is the normal procedure here? > > Should it be... > > 2.n : .is_alive() > 2.n+1 : .is_alive() (deprecated), .alive (recommended) > 2.n+2 : .alive I think: 2.x isAlive() (Py3kWarning), .alive (recommended for compatibility with 3.x) 3.x .alive -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From cfbolz at gmx.de Thu Jun 12 14:56:46 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 12 Jun 2008 14:56:46 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <20080612122436.4D18F3A405F@sparrow.telecommunity.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> <485109D4.2070407@gmx.de> <20080612122436.4D18F3A405F@sparrow.telecommunity.com> Message-ID: <48511D0E.9050309@gmx.de> Phillip J. Eby wrote: > At 01:34 PM 6/12/2008 +0200, Carl Friedrich Bolz wrote: >> Phillip J. Eby wrote: >> > As it happens, most objects' __dict__ slots are settable by default, >> and >> > *require* that you set it to a dict or subclass thereof. >> >> This is wrong for types: > > Which is why I said "most" - to exclude types, and objects that don't > have a __dict__ slot to begin with. Sorry, I thought we were mostly discussing type dictionaries at the moment. >> I think there are good arguments for not allowing strings keys in type >> dicts, or at least leaving it up to the implementation. > > That may well be, but there is nothing in Python's spec that I'm aware > of that *forbids* it. For example the type() constructor doc doesn't > say anything about using string-only keys in the class dictionary. Fine, but as far as I can see there is also nothing that *mandates* it. >> Using non-string >> keys in type dicts is relatively awkward and allowing them makes many >> interesting optimizations (like method caches) a _lot_ harder to get >> right. > > Really? Why? Having non-string dict keys is NOT the same thing as > having non-string attribute names, so attribute name lookups should be > unaffected. (Attribute names *are* required to be strings, and -- as > far as I know -- always have been.) Of course attribute name lookups are affected, because you can have a non-string key that has a __hash__ and __eq__ method to make it look sufficiently like a string to the dict. Then the attribute lookup needs to take these into account. This makes a method cache implementation much more complex, because suddenly arbitrary user code can run during the lookup and your classes (and thus your method caches) can change under your feed during the lookup. In this situation getting the corner cases right is very hard. Cheers, Carl Friedrich From barry at python.org Thu Jun 12 15:04:14 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Jun 2008 09:04:14 -0400 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> Message-ID: <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 12, 2008, at 8:29 AM, Facundo Batista wrote: > 2008/6/12 Barry Warsaw : > >>> function:: active_count() >>> method:: Thread.get_name() >>> method:: Thread.is_alive() >>> method:: Thread.is_daemon() >>> method:: Thread.set_daemon(daemonic) >> >> +1 on opting for properties in the specific cases here where it >> makes sense. > > I'm +1 too... but which is the normal procedure here? > > Should it be... > > 2.n : .is_alive() > 2.n+1 : .is_alive() (deprecated), .alive (recommended) > 2.n+2 : .alive Personally, I'd go with a property .is_alive - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFEeznEjvBPtnXfVAQKEeQP8CdF+N1wX4Qwxn7iglGYq0peZEYagn4JG dOkP+TOkAzSciNfIotPaHJTGbyXsFtxLe3Rpq8r56/sPhHKS66+vCMojLBK64Iue 7/PDhZ300KRpPtbJOOA4OmqI2rz1fO+vflavICZlx7oIwC25L7dQSteu/NBJYGJN QX/Z8WutBng= =1B3v -----END PGP SIGNATURE----- From cfbolz at gmx.de Thu Jun 12 14:56:46 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 12 Jun 2008 14:56:46 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <20080612122436.4D18F3A405F@sparrow.telecommunity.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> <485109D4.2070407@gmx.de> <20080612122436.4D18F3A405F@sparrow.telecommunity.com> Message-ID: <48511D0E.9050309@gmx.de> Phillip J. Eby wrote: > At 01:34 PM 6/12/2008 +0200, Carl Friedrich Bolz wrote: >> Phillip J. Eby wrote: >> > As it happens, most objects' __dict__ slots are settable by default, >> and >> > *require* that you set it to a dict or subclass thereof. >> >> This is wrong for types: > > Which is why I said "most" - to exclude types, and objects that don't > have a __dict__ slot to begin with. Sorry, I thought we were mostly discussing type dictionaries at the moment. >> I think there are good arguments for not allowing strings keys in type >> dicts, or at least leaving it up to the implementation. > > That may well be, but there is nothing in Python's spec that I'm aware > of that *forbids* it. For example the type() constructor doc doesn't > say anything about using string-only keys in the class dictionary. Fine, but as far as I can see there is also nothing that *mandates* it. >> Using non-string >> keys in type dicts is relatively awkward and allowing them makes many >> interesting optimizations (like method caches) a _lot_ harder to get >> right. > > Really? Why? Having non-string dict keys is NOT the same thing as > having non-string attribute names, so attribute name lookups should be > unaffected. (Attribute names *are* required to be strings, and -- as > far as I know -- always have been.) Of course attribute name lookups are affected, because you can have a non-string key that has a __hash__ and __eq__ method to make it look sufficiently like a string to the dict. Then the attribute lookup needs to take these into account. This makes a method cache implementation much more complex, because suddenly arbitrary user code can run during the lookup and your classes (and thus your method caches) can change under your feed during the lookup. In this situation getting the corner cases right is very hard. Cheers, Carl Friedrich From metawilm at gmail.com Thu Jun 12 15:47:59 2008 From: metawilm at gmail.com (Willem Broekema) Date: Thu, 12 Jun 2008 15:47:59 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <48511D0E.9050309@gmx.de> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> <485109D4.2070407@gmx.de> <20080612122436.4D18F3A405F@sparrow.telecommunity.com> <48511D0E.9050309@gmx.de> Message-ID: On Thu, Jun 12, 2008 at 2:56 PM, Carl Friedrich Bolz wrote: > Of course attribute name lookups are affected, because you can have a > non-string key that has a __hash__ and __eq__ method to make it look > sufficiently like a string to the dict. Then the attribute lookup needs > to take these into account. This makes a method cache implementation > much more complex, because suddenly arbitrary user code can run during > the lookup and your classes (and thus your method caches) can change > under your feed during the lookup. In this situation getting the corner > cases right is very hard. I fully agree with this assessment: the theoretical possibilities make attribute handling very hairy. Note that besides non-strings, also user-defined subclasses of string can have their own hash/eq logic. If namespace keys and attributes could be locked down to only be regular strings, not of another type and not a string subclass instance, then implementing attributes becomes a lot less hairy, and optimizations much more straightforward. - Willem (CLPython) From guido at python.org Thu Jun 12 15:55:07 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 06:55:07 -0700 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <4850FBFC.9070501@gmail.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <4850FBFC.9070501@gmail.com> Message-ID: [Barry] >>> http://bugs.python.org/issue643841 [Guido] >> I've added a comment. Let me know if anything I said is unclear. On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan wrote: > The bugtracker seems to be offline atm - I'll reply there once I can get to > it again (as well as switching this issue back to being a documentation > one). > > I don't think we're going to see a major clamor for a value-based delegation > mixin in the standard library until people using classic classes for > value-based delegation start making serious attempts to port to Py3k (where > that approach is no longer available). At the moment, such classes only need > to care about the methods they want to fiddle with, leaving everything else > to __getattr__ based delegation. Whether they'll care about this issue of course depends on whether they need overloaded operators and other special delegations to be delegated transparently. We'll have to see how important this is. New-style classes have been around and recommended for a long time -- why haven't people pushed for a proxy class before? > I've pushed as hard as I'm personally willing to for this without convincing > anyone else that it's worth doing, What does *it* refer to? Changing the behavior, or adding a proxy class to the stdlib? I'm -1000 on the former, but only -0 on the latter -- as I wrote in the tracker, I just don't want to see an unproven proxy class (and I don't like the module name). > so I'll start working on a documentation > patch for the language reference instead which explicitly splits the special > methods into the following categories: Thanks for doing this, it is needed regardless! > 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in > the instance dictionary MUST NOT have any ill effects. (All tp_* C-level > slots and slots looked up via _PyType_Lookup will fit into this category) Watch out: I think the term "method lookup" may be confusing here. Certainly when the user writes x.__foo__(), the instance dict *is* consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where the instance dict is bypassed. > 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the > instance dictionary MAY have ill effects. (slots such as __enter__ and > __exit__ that are looked up via normal attribute lookup in CPython will fit > into this category) Why even have a MAY category? Are you expecting these to become tp_ slots at some point? > 3. Technically a subcategory of group 1, these are special methods which can > affect the interpreter's behaviour by their mere definition on a type. (The > __get__, __set__ and __delete__ descriptor protocol methods fall into this > category) I don't follow why this is relevant. This is a different, AFAIK orthogonal issue, used in many places: *if* an object used in a certain context has a specific attribute, *then* that attribute is used, *otherwise* a default action is taken. Applies to __repr__ just as much. These belong in category 1 if and only if the lookup bypasses the instance dict. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jun 12 16:17:42 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 07:17:42 -0700 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> Message-ID: If you want to have a meaningful discussion about this, the addition of the multiprocessing package and the recent threading.py API changes must be rolled back, so we can design a proper API without the beta 1 pressure. Some observations: - If it's isAlive() in Java style, it should be is_alive in Python style. - I like *methods* (and functions) named isfoo or is_foo, but I like the corresponding properties to be called just foo. - I think properties instead of simple setters and getters is great (as long as the setters implied in the property enforce the conditions). - I hate typing underscores, since they require using the shift key *and* moving my hands away from the home position on the keybord. --Guido On Thu, Jun 12, 2008 at 6:04 AM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Jun 12, 2008, at 8:29 AM, Facundo Batista wrote: > >> 2008/6/12 Barry Warsaw : >> >>>> function:: active_count() >>>> method:: Thread.get_name() >>>> method:: Thread.is_alive() >>>> method:: Thread.is_daemon() >>>> method:: Thread.set_daemon(daemonic) >>> >>> +1 on opting for properties in the specific cases here where it makes >>> sense. >> >> I'm +1 too... but which is the normal procedure here? >> >> Should it be... >> >> 2.n : .is_alive() >> 2.n+1 : .is_alive() (deprecated), .alive (recommended) >> 2.n+2 : .alive > > Personally, I'd go with a property .is_alive > > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iQCVAwUBSFEeznEjvBPtnXfVAQKEeQP8CdF+N1wX4Qwxn7iglGYq0peZEYagn4JG > dOkP+TOkAzSciNfIotPaHJTGbyXsFtxLe3Rpq8r56/sPhHKS66+vCMojLBK64Iue > 7/PDhZ300KRpPtbJOOA4OmqI2rz1fO+vflavICZlx7oIwC25L7dQSteu/NBJYGJN > QX/Z8WutBng= > =1B3v > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Thu Jun 12 16:43:02 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 12 Jun 2008 14:43:02 -0000 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <4850FBFC.9070501@gmail.com> Message-ID: <48EE15E7.7020701@voidspace.org.uk> Guido van Rossum wrote: > [Barry] > >>>> http://bugs.python.org/issue643841 >>>> > > [Guido] > >>> I've added a comment. Let me know if anything I said is unclear. >>> > > On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan wrote: > >> The bugtracker seems to be offline atm - I'll reply there once I can get to >> it again (as well as switching this issue back to being a documentation >> one). >> >> I don't think we're going to see a major clamor for a value-based delegation >> mixin in the standard library until people using classic classes for >> value-based delegation start making serious attempts to port to Py3k (where >> that approach is no longer available). At the moment, such classes only need >> to care about the methods they want to fiddle with, leaving everything else >> to __getattr__ based delegation. >> > > Whether they'll care about this issue of course depends on whether > they need overloaded operators and other special delegations to be > delegated transparently. We'll have to see how important this is. > New-style classes have been around and recommended for a long time -- > why haven't people pushed for a proxy class before? > > It's only in Python 3 that old style classes are going away fully, so up until now you could at least use a classic class to do the proxying. I've written my own proxy classes before that look very similar to this, and there are other proxy classes around that do the same (I thought one was by Phillip J Eby but can't find a reference easily). The last one I wrote was to proxy CPython objects from IronPython via Python.NET... I would prefer it if the proxy class wrapped the return values of inplace operations. Michael Foord >> I've pushed as hard as I'm personally willing to for this without convincing >> anyone else that it's worth doing, >> > > What does *it* refer to? Changing the behavior, or adding a proxy > class to the stdlib? I'm -1000 on the former, but only -0 on the > latter -- as I wrote in the tracker, I just don't want to see an > unproven proxy class (and I don't like the module name). > > >> so I'll start working on a documentation >> patch for the language reference instead which explicitly splits the special >> methods into the following categories: >> > > Thanks for doing this, it is needed regardless! > > >> 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in >> the instance dictionary MUST NOT have any ill effects. (All tp_* C-level >> slots and slots looked up via _PyType_Lookup will fit into this category) >> > > Watch out: I think the term "method lookup" may be confusing here. > Certainly when the user writes x.__foo__(), the instance dict *is* > consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where > the instance dict is bypassed. > > >> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the >> instance dictionary MAY have ill effects. (slots such as __enter__ and >> __exit__ that are looked up via normal attribute lookup in CPython will fit >> into this category) >> > > Why even have a MAY category? Are you expecting these to become tp_ > slots at some point? > > >> 3. Technically a subcategory of group 1, these are special methods which can >> affect the interpreter's behaviour by their mere definition on a type. (The >> __get__, __set__ and __delete__ descriptor protocol methods fall into this >> category) >> > > I don't follow why this is relevant. This is a different, AFAIK > orthogonal issue, used in many places: *if* an object used in a > certain context has a specific attribute, *then* that attribute is > used, *otherwise* a default action is taken. Applies to __repr__ just > as much. These belong in category 1 if and only if the lookup bypasses > the instance dict. > > From ncoghlan at gmail.com Thu Jun 12 16:50:36 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jun 2008 00:50:36 +1000 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <4850FBFC.9070501@gmail.com> Message-ID: <485137BC.2070505@gmail.com> Guido van Rossum wrote: > [Barry] >>>> http://bugs.python.org/issue643841 > > [Guido] >>> I've added a comment. Let me know if anything I said is unclear. > > On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan wrote: >> The bugtracker seems to be offline atm - I'll reply there once I can get to >> it again (as well as switching this issue back to being a documentation >> one). >> >> I don't think we're going to see a major clamor for a value-based delegation >> mixin in the standard library until people using classic classes for >> value-based delegation start making serious attempts to port to Py3k (where >> that approach is no longer available). At the moment, such classes only need >> to care about the methods they want to fiddle with, leaving everything else >> to __getattr__ based delegation. > > Whether they'll care about this issue of course depends on whether > they need overloaded operators and other special delegations to be > delegated transparently. We'll have to see how important this is. > New-style classes have been around and recommended for a long time -- > why haven't people pushed for a proxy class before? There was an easier way to do it in the form of classic classes - the 2,x interpreter is riddled with special cases that ensure that __getattr__ is always consulted when looking for special methods on a classic class. The tracker issue regarding the fact that things aren't so simple with new style classes was actually raised way back in 2002 when someone tried to port such a class to new-style and discovered that overriding __getattribute__ was no longer enough. >> I've pushed as hard as I'm personally willing to for this without convincing >> anyone else that it's worth doing, > > What does *it* refer to? Changing the behavior, or adding a proxy > class to the stdlib? I'm -1000 on the former, but only -0 on the > latter -- as I wrote in the tracker, I just don't want to see an > unproven proxy class (and I don't like the module name). "It" referred to adding the proxy class - I'm personally ambivalent on adding it at this point, because the complexity of it reduces my confidence that I got it right, but it also makes it seem unfair to users of this feature of classic classes to take it away in 3.0 without giving them some kind of functional replacement. As for as the module name goes, I don't particularly like it either - dropping something in the types module instead would be an alternative option. >> so I'll start working on a documentation >> patch for the language reference instead which explicitly splits the special >> methods into the following categories: > > Thanks for doing this, it is needed regardless! > >> 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in >> the instance dictionary MUST NOT have any ill effects. (All tp_* C-level >> slots and slots looked up via _PyType_Lookup will fit into this category) > > Watch out: I think the term "method lookup" may be confusing here. > Certainly when the user writes x.__foo__(), the instance dict *is* > consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where > the instance dict is bypassed. Ah good point, I'll make sure to be careful with that. >> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the >> instance dictionary MAY have ill effects. (slots such as __enter__ and >> __exit__ that are looked up via normal attribute lookup in CPython will fit >> into this category) > > Why even have a MAY category? Are you expecting these to become tp_ > slots at some point? Either tp_* slots, or just having the invocation bypass the instance attributes and only look at the object's type. I think it would actually be desirable for this category to be empty from a purity point of view (with all the special methods in category 1), but given that CPython itself currently doesn't conform to such a language spec, this seems to be the next best option (i.e. allowing other implementations or later versions of CPython to put these special methods in category 1 along with the rest of the special methods) >> 3. Technically a subcategory of group 1, these are special methods which can >> affect the interpreter's behaviour by their mere definition on a type. (The >> __get__, __set__ and __delete__ descriptor protocol methods fall into this >> category) > > I don't follow why this is relevant. This is a different, AFAIK > orthogonal issue, used in many places: *if* an object used in a > certain context has a specific attribute, *then* that attribute is > used, *otherwise* a default action is taken. Applies to __repr__ just > as much. These belong in category 1 if and only if the lookup bypasses > the instance dict. Actual hasattr() checks aren't a problem - those hit __getattribute__ and a delegating class can correctly check them against the target object. Methods like __str__ or __repr__ also aren't a major issue - those are easy to delegate in a way that reproduces the same behaviour as if the delegating class wasn't there (just reinvoke the appropriate builtin on your target object). This category is specifically for method checks in category 1 which bypass __getattribute__ *and* have significant effects on the way an object gets handled that can't be readily dealt with by a value-based delegation class - the most significant methods I've actually found in that category so far are the descriptor protocol methods (that's why my ProxyMixin class skipped delegating them). As long as the callable() builtin is around, __call__ actually lands in this category as well (since defining it will affect the answer returned by the callable() builtin). Being able to return different proxy classes with and without __callable__ defined is actually the reason weakref.proxy is a factory function rather than a type in its own right. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 12 16:54:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jun 2008 00:54:08 +1000 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <48EE15E7.7020701@voidspace.org.uk> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <4850FBFC.9070501@gmail.com> <48EE15E7.7020701@voidspace.org.uk> Message-ID: <48513890.6080303@gmail.com> Michael Foord wrote: > Guido van Rossum wrote: >> Whether they'll care about this issue of course depends on whether >> they need overloaded operators and other special delegations to be >> delegated transparently. We'll have to see how important this is. >> New-style classes have been around and recommended for a long time -- >> why haven't people pushed for a proxy class before? > > It's only in Python 3 that old style classes are going away fully, so up > until now you could at least use a classic class to do the proxying. > > I've written my own proxy classes before that look very similar to this, > and there are other proxy classes around that do the same (I thought one > was by Phillip J Eby but can't find a reference easily). The last one I > wrote was to proxy CPython objects from IronPython via Python.NET... > > I would prefer it if the proxy class wrapped the return values of > inplace operations. Yeah, the latest version on the issue tracker does that, and allows subclasses to define a return_inplace() method to alter the behaviour (e.g. a weakref.proxy equivalent wouldn't want to wrap the return values so that it can ensure there is always at least one strong reference to the result of the operation). Since you can also replace the .target attribute with a property to affect how the target object is stored and accessed, it's a reasonably flexible approach. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From walter at livinglogic.de Thu Jun 12 16:59:54 2008 From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=) Date: Thu, 12 Jun 2008 16:59:54 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <485003F8.40803@egenix.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> <484FEC2C.6070409@livinglogic.de> <485003F8.40803@egenix.com> Message-ID: <485139EA.1000502@livinglogic.de> M.-A. Lemburg wrote: > On 2008-06-11 17:15, Walter D?rwald wrote: >> M.-A. Lemburg wrote: >>> On 2008-06-11 13:35, Barry Warsaw wrote: >>>> So I had planned to do a bunch of work last night looking at the >>>> release blocker issues, but nature intervened. A bunch of severe >>>> thunderstorms knock out my 'net access until this morning. >>>> >>>> I'll try to find some time during the day to look at the RB issues. >>>> Hopefully we can get Guido to look at them too and Pronounce on some >>>> of them. Guido please start with: >>>> >>>> http://bugs.python.org/issue643841 >>>> >>>> My plan is to begin building the betas tonight, at around 9 or 10pm >>>> EDT (0100 to 0200 UTC Thursday). If a showstopper comes up before >>>> then, I'll email the list. If you think we really aren't ready for >>>> beta, then I would still like to get a release out today. In that >>>> case, we'll call it alpha and delay the betas. >>> >>> There are two things I'd like to get in to 3.0: >>> >>> * .transform()/.untransform() methods (this is mostly done, just need >>> to add the methods to PyUnicode, PyBytes and PyByteArray) >> >> What would these methods do? Use the codec machinery without any type >> checks? > > As discussed in another thread some weeks ago: > > .transform() and .untransform() use the codecs to apply same-type > conversions. They do apply type checks to make sure that the > codec does indeed return the same type. > > E.g. text.transform('xml-escape') or data.transform('base64'). So what would a base64 codec do with the errors argument? >> I think for transformations we don't need the full codec machinery: > > ... > > No need to invent another wheel :-) The codecs already exist for > Py2.x and can be used by the .encode()/.decode() methods in Py2.x > (where no type checks occur). By using a new API we could get rid of old warts. For example: Why does the stateless encoder/decoder return how many input characters/bytes it has consumed? It must consume *all* bytes anyway! > In Py3.x, .encode()/.decode() only allow conversions of the type > unicode <-> bytes. .transform()/.untransform() add conversions > of the type unicode <-> unicode or bytes <-> bytes. > > All other conversions in Py3.x have to go through codecs.encode() and > codecs.decode() which are the generic codec access functions from > the codec registry. Servus, Walter From skip at pobox.com Thu Jun 12 17:21:05 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 12 Jun 2008 10:21:05 -0500 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> Message-ID: <18513.16097.394123.312491@montanaro-dyndns-org.local> >> Should it be... >> >> 2.n : .is_alive() >> 2.n+1 : .is_alive() (deprecated), .alive (recommended) >> 2.n+2 : .alive Barry> Personally, I'd go with a property .is_alive I'm not fond of using a property for this since it can lull you into the false belief that what you are doing is less expensive than it really is (attribute access vs method call). I think this is a case where explicit is better than implicit. Skip From arigo at tunes.org Thu Jun 12 17:24:43 2008 From: arigo at tunes.org (Armin Rigo) Date: Thu, 12 Jun 2008 17:24:43 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: <48508D81.2050304@scottdial.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> <48508D81.2050304@scottdial.com> Message-ID: <20080612152443.GA22637@code0.codespeak.net> Hi, On Wed, Jun 11, 2008 at 10:44:17PM -0400, Scott Dial wrote: > The only reason the test used locals() was > because it was the only way to insert a non-string key into the class > namespace. This discussion is mistakenly focused on locals(). There is a direct way to have arbitrary keys in the dict of a type: >>> MyClass = type('MyClass', (Base,), {42: 64}) >>> MyClass.__dict__[42] 64 There is, however, no way to modify or add non-string keys in the type after its creation. So the question is whether the type() constructor is allowed to fail with a TypeError when the initial dict contains non-string keys (this is PyPy's current behavior). A bientot, Armin. From barry at python.org Thu Jun 12 17:40:26 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Jun 2008 11:40:26 -0400 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <18513.16097.394123.312491@montanaro-dyndns-org.local> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 12, 2008, at 11:21 AM, skip at pobox.com wrote: > >>> Should it be... >>> >>> 2.n : .is_alive() >>> 2.n+1 : .is_alive() (deprecated), .alive (recommended) >>> 2.n+2 : .alive > > Barry> Personally, I'd go with a property .is_alive > > I'm not fond of using a property for this since it can lull you into > the > false belief that what you are doing is less expensive than it > really is > (attribute access vs method call). I think this is a case where > explicit is > better than implicit. I've heard this argument before, and it's one I'm not unsympathetic to. Usually I've heard it in the context of remote network access, e.g. let's say .alive had to make a REST call over the net each time. Well, maybe there's caching involved which amortizes the costs, so the argument gets trickier. Ideally, I would like for those considerations to not enter into the API design. I'd rather keep it clean, with sufficient documentation to give hints about any additional costs involved. Logically .alive is a property, so I'd like to write it that way. Perhaps function annotations help out here. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFFDanEjvBPtnXfVAQJxbAQAq2BDNE7m+J+5Jy3E4XlHaw8JXsEaukEA YvZ1M1R1mREGCstIf4tlScKko1eu9PfJIk7+kCmFYezighJ1tPZunyu5zLemlgQe 9rJ9keFIJBLtao8yv+FPkn56idixtkTGq+14Ef+EhjIhOnbonp+eDuhScbE2tdzM 5tuFO/hsrqw= =66R7 -----END PGP SIGNATURE----- From mal at egenix.com Thu Jun 12 17:43:36 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 12 Jun 2008 17:43:36 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <485139EA.1000502@livinglogic.de> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> <484FEC2C.6070409@livinglogic.de> <485003F8.40803@egenix.com> <485139EA.1000502@livinglogic.de> Message-ID: <48514428.7040001@egenix.com> On 2008-06-12 16:59, Walter D?rwald wrote: > M.-A. Lemburg wrote: >> .transform() and .untransform() use the codecs to apply same-type >> conversions. They do apply type checks to make sure that the >> codec does indeed return the same type. >> >> E.g. text.transform('xml-escape') or data.transform('base64'). > > So what would a base64 codec do with the errors argument? It could use it to e.g. try to recover as much data as possible from broken input data. Currently (in Py2.x), it raises an exception if you pass in anything but "strict". >>> I think for transformations we don't need the full codec machinery: >> > ... >> >> No need to invent another wheel :-) The codecs already exist for >> Py2.x and can be used by the .encode()/.decode() methods in Py2.x >> (where no type checks occur). > > By using a new API we could get rid of old warts. For example: Why does > the stateless encoder/decoder return how many input characters/bytes it > has consumed? It must consume *all* bytes anyway! No, it doesn't and that's the point in having those return values :-) Even though the encoder/decoders are stateless, that doesn't mean they have to consume all input data. The caller is responsible to make sure that all input data was in fact consumed. You could for example have a decoder that stops decoding after having seen a block end indicator, e.g. a base64 line end or XML closing element. Just because all codecs that ship with Python always try to decode the complete input doesn't mean that the feature isn't being used. The interface was designed to allow for the above situations. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 12 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 24 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From adlaiff6 at gmail.com Thu Jun 12 19:13:45 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 12 Jun 2008 10:13:45 -0700 Subject: [Python-Dev] Assignment to None In-Reply-To: <4850F418.2040301@gmail.com> References: <484CBD70.3000808@v.loewis.de> <4850F418.2040301@gmail.com> Message-ID: I haven't been following this thread very closely, so I'm not sure what the status is, but I'd just like to point out that yesterday I used the fact that a[None] = b works, when I used the @memoize decorator from the wiki. This seems to provide an argument that, for symmetry's sake, we might want to keep a.None = b as well. -- Cheers, Leif From guido at python.org Thu Jun 12 19:28:18 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 10:28:18 -0700 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <484CBD70.3000808@v.loewis.de> <4850F418.2040301@gmail.com> Message-ID: On Thu, Jun 12, 2008 at 10:13 AM, Leif Walsh wrote: > I haven't been following this thread very closely, so I'm not sure > what the status is, but I'd just like to point out that yesterday I > used the fact that a[None] = b works, when I used the @memoize > decorator from the wiki. This seems to provide an argument that, for > symmetry's sake, we might want to keep a.None = b as well. That makes about as much sense as wanting to support a.42 = b since a[42] = b works. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From adlaiff6 at gmail.com Thu Jun 12 19:55:38 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 12 Jun 2008 10:55:38 -0700 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <484CBD70.3000808@v.loewis.de> <4850F418.2040301@gmail.com> Message-ID: On Thu, Jun 12, 2008 at 10:28 AM, Guido van Rossum wrote: > That makes about as much sense as wanting to support a.42 = b since > a[42] = b works. :-) Well don't I feel silly now. -- Cheers, Leif From jimjjewett at gmail.com Thu Jun 12 20:11:14 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 12 Jun 2008 14:11:14 -0400 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <4850FBFC.9070501@gmail.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <4850FBFC.9070501@gmail.com> Message-ID: On 6/12/08, Nick Coghlan wrote: > documentation patch for the language reference ... > following categories: ... > 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in > the instance dictionary MAY have ill effects. (slots such as __enter__ and > __exit__ that are looked up via normal attribute lookup in CPython will fit > into this category) Should this category really be enumerated? I thought that was the default meaning of __name__, so the real clarification is: (1) Requiring that the specific names in category 1 MUST be treated this way. (2) Mentioning __*__ and listing any known exceptions. (Can "next" be treated this way despite the lack of __*__? Is it forbidden to treat __context__ this way?) -jJ From tjreedy at udel.edu Thu Jun 12 21:38:09 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Jun 2008 15:38:09 -0400 Subject: [Python-Dev] bug or a feature? References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com><484F2C1B.2090404@scottdial.com><693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com><20080611161455.CBD703A405F@sparrow.telecommunity.com><4850263A.3040805@scottdial.com> <20080612100048.7EEEA3A405F@sparrow.telecommunity.com> Message-ID: "Phillip J. Eby" wrote in message news:20080612100048.7EEEA3A405F at sparrow.telecommunity.com... | At 08:32 PM 6/11/2008 -0400, Terry Reedy wrote: | >The Data Model chapter of the Reference Manual lists .__dict__ as a special | >attribute of callables, modules, classes, and instances. It describes | >__dict__ as a "namespace dictionary" or "implementation of the namespace" | >thereof. Since namespaces map names (or possibly non-name strings) to | >objects, this to me implies that an implementation is and should not be | >required to allow non-strings in __dict__. | > | >The same chapter has more than one sentence saying something like "o.x is | >equivalent to o.__dict__['x']". While one could read this as prohibiting | >o.__dict__[non_string], one could also read it as being silent, neither | >allowing nor prohibiting. | | As it happens, most objects' __dict__ slots are settable by default, | and *require* that you set it to a dict or subclass thereof. This | seems (to me) to imply that a standard dictionary (i.e. one | supporting keys of any type) is required. (In the sense that a dict | subclass which rejects non-strings would be violating the Liskov principle.) Is this requirement a documented Python requirement (not that I found), an undocumented Python requirement, or a CPython-specific requirement (that other implementations may freely ignore)? I don't have much opinion on what the rules for __dict__ attributes should be, just that whatever they are should be documented. This include implementation-defined aspects. This will help both implementors and users who wish to write Python code rather that CPython code. If there is a consensus and BDFL pronouncement, I will make or help review suggested doc changes. tjr From guido at python.org Thu Jun 12 21:46:18 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 12:46:18 -0700 Subject: [Python-Dev] bug or a feature? In-Reply-To: <20080612152443.GA22637@code0.codespeak.net> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> <48508D81.2050304@scottdial.com> <20080612152443.GA22637@code0.codespeak.net> Message-ID: On Thu, Jun 12, 2008 at 8:24 AM, Armin Rigo wrote: > This discussion is mistakenly focused on locals(). There is a direct > way to have arbitrary keys in the dict of a type: > >>>> MyClass = type('MyClass', (Base,), {42: 64}) >>>> MyClass.__dict__[42] > 64 > > There is, however, no way to modify or add non-string keys in the type > after its creation. So the question is whether the type() constructor > is allowed to fail with a TypeError when the initial dict contains > non-string keys (this is PyPy's current behavior). The intention was for these dicts to be used as namespaces. I think of it as follows: (a) Using non-string keys is a no-no, but the implementation isn't required to go out of its way to forbid it. (b) Using non-empty string keys that aren't well-formed identifiers should be allowed. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From metawilm at gmail.com Thu Jun 12 22:01:28 2008 From: metawilm at gmail.com (Willem Broekema) Date: Thu, 12 Jun 2008 22:01:28 +0200 Subject: [Python-Dev] bug or a feature? In-Reply-To: References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> <48508D81.2050304@scottdial.com> <20080612152443.GA22637@code0.codespeak.net> Message-ID: On Thu, Jun 12, 2008 at 9:46 PM, Guido van Rossum wrote: > The intention was for these dicts to be used as namespaces. I think of > it as follows: > > (a) Using non-string keys is a no-no, but the implementation isn't > required to go out of its way to forbid it. That will allow easier and more efficient implementation, good! > (b) Using non-empty string keys that aren't well-formed identifiers > should be allowed. ok. Is it allowed to "normalize" subclasses of strings to regular string, e.g. after: class mystring(str): pass class C: pass x = C() setattr(x, mystring('foo'), 42) is it allowed that the dict of x contains a regular string 'foo' instead of the mystring instance? - Willem From pje at telecommunity.com Thu Jun 12 23:42:50 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 12 Jun 2008 17:42:50 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F2C1B.2090404@scottdial.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> <48508D81.2050304@scottdial.com> <20080612152443.GA22637@code0.codespeak.net> Message-ID: <20080612214219.3C37A3A405F@sparrow.telecommunity.com> At 12:46 PM 6/12/2008 -0700, Guido van Rossum wrote: >The intention was for these dicts to be used as namespaces. By "these" do you mean type object __dict__ attributes, or *all* __dict__ attributes? From guido at python.org Fri Jun 13 00:05:27 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 15:05:27 -0700 Subject: [Python-Dev] bug or a feature? In-Reply-To: References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> <48508D81.2050304@scottdial.com> <20080612152443.GA22637@code0.codespeak.net> Message-ID: On Thu, Jun 12, 2008 at 1:01 PM, Willem Broekema wrote: > On Thu, Jun 12, 2008 at 9:46 PM, Guido van Rossum wrote: >> The intention was for these dicts to be used as namespaces. I think of >> it as follows: >> >> (a) Using non-string keys is a no-no, but the implementation isn't >> required to go out of its way to forbid it. > > That will allow easier and more efficient implementation, good! > >> (b) Using non-empty string keys that aren't well-formed identifiers >> should be allowed. > > ok. > > Is it allowed to "normalize" subclasses of strings to regular string, > e.g. after: > > class mystring(str): pass > class C: pass > > x = C() > setattr(x, mystring('foo'), 42) > > is it allowed that the dict of x contains a regular string 'foo' > instead of the mystring instance? I think yes, as this would allow for a more efficient implementation of the custom dict class. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 13 00:07:37 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 15:07:37 -0700 Subject: [Python-Dev] bug or a feature? In-Reply-To: <20080612214219.3C37A3A405F@sparrow.telecommunity.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <693bc9ab0806101837rb13fc37p7041ba9dc8c05804@mail.gmail.com> <20080611161455.CBD703A405F@sparrow.telecommunity.com> <4850263A.3040805@scottdial.com> <693bc9ab0806111759k14f7e672r1ebd20a85d4d5af5@mail.gmail.com> <48508D81.2050304@scottdial.com> <20080612152443.GA22637@code0.codespeak.net> <20080612214219.3C37A3A405F@sparrow.telecommunity.com> Message-ID: On Thu, Jun 12, 2008 at 2:42 PM, Phillip J. Eby wrote: > At 12:46 PM 6/12/2008 -0700, Guido van Rossum wrote: >> >> The intention was for these dicts to be used as namespaces. > > By "these" do you mean type object __dict__ attributes, or *all* __dict__ > attributes? Hadn't thought of that one! Since __dict__ is widely advertised as being, well, a dict, it may be harder to enforce this for __dict__ instances of other objects. I'd like to tighten the rules here too, but only if it actually helps Python implementations other than CPython. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tlesher at gmail.com Thu Jun 12 15:46:43 2008 From: tlesher at gmail.com (Tim Lesher) Date: Thu, 12 Jun 2008 09:46:43 -0400 Subject: [Python-Dev] OFF-TOPIC-BUT-IMPORTANT: End of free internet model by 2012 ? In-Reply-To: References: Message-ID: <9613db600806120646l21965124v5ae5577cb84f812@mail.gmail.com> On Wed, Jun 11, 2008 at 4:25 PM, Daniel Bonekeeper wrote: > Hi everybody on the list ! > > Sorry about the off-topic message, but giving the nature of the > message and the nature of the list (after all, everybody here uses > Internet or develops for web in some way), I decided to post this here > and see what you guys think about it. > My gut feeling is that you are being taken by some fairly amateurish viral marketing, and that you've played into it by uncritically regurgitating the message onto these mailing lists. Googling "Dylan Pattyn" yields only references to this very video and a site for a performance art/activist group called IPower, which boasts such "projects" as "an suicide-awareness campaign disguised as a personal suicide blog", a self-described "online stunt" called "40,000 Blowjobs", and a "wacky online reality series". Really, do you honestly think that Time Magazine would use a twenty-something freelancer, who has never published anything in any major periodical, to break a news article that would at the least create a decade or more of anti-trust lawsuits? -- Tim Lesher -------------- next part -------------- An HTML attachment was scrubbed... URL: From fwierzbicki at gmail.com Fri Jun 13 05:06:23 2008 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Thu, 12 Jun 2008 23:06:23 -0400 Subject: [Python-Dev] Assignment to None In-Reply-To: References: <484CBD70.3000808@v.loewis.de> Message-ID: <4dab5f760806122006t5bfa9fd7h18dd6ef3f1300391@mail.gmail.com> On Wed, Jun 11, 2008 at 5:27 PM, Curt Hagenlocher wrote: > If I recall correctly, Jython handles this by appending a trailing > underscore to the imported name and there's no reason why we couldn't > do something similar. In truth the current implementation of Jython allows keywords in many strange places, I expect this was done to allow for method names that are not keywords in Java so, for example, if there is a method called "print" in a Java class that we want to call (quite common) then it can be called. As far as I know appended underscores don't enter into it. Some poking around reveals that the current Jython is probably too lax here, even keywords that are common to both Python and Java can be method names (like "if"). I plan to continue to allow Python keywords that are not Java keywords to behave this way at least for the 2.x series, though I don't think I'm going to go through the effort of allowing keywords common to both Java and Python like "if" (The 2.5 version of Jython will have a new parser so I do actually need to make these explicit choices right now). I think changing this behavior would hurt backwards compatibility too much. Maybe I'll rethink it in our 3.0 timeframe. If a convention like a trailing underscore is adopted we might move to that in the move to 3.0. -Frank From curt at hagenlocher.org Fri Jun 13 05:17:34 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Thu, 12 Jun 2008 20:17:34 -0700 Subject: [Python-Dev] Assignment to None In-Reply-To: <4dab5f760806122006t5bfa9fd7h18dd6ef3f1300391@mail.gmail.com> References: <484CBD70.3000808@v.loewis.de> <4dab5f760806122006t5bfa9fd7h18dd6ef3f1300391@mail.gmail.com> Message-ID: On Thu, Jun 12, 2008 at 8:06 PM, Frank Wierzbicki wrote: > > On Wed, Jun 11, 2008 at 5:27 PM, Curt Hagenlocher wrote: > > If I recall correctly, Jython handles this by appending a trailing > > underscore to the imported name and there's no reason why we couldn't > > do something similar. > > In truth the current implementation of Jython allows keywords in many > strange places, I expect this was done to allow for method names that > are not keywords in Java so, for example, if there is a method called > "print" in a Java class that we want to call (quite common) then it > can be called. As far as I know appended underscores don't enter into > it. After posting that message, I did what I should have done initially which was to ask Jim Hugunin about it. He said that Jython had gotten Guido's blessing to parse keywords in a context-sensitive fashion -- so that "foo.{keyword}" might be considered legal under certain circumstances. I don't, alas, have any specific cites to that end, but I suspect that we'll be following that precedent :). -- Curt Hagenlocher curt at hagenlocher.org From fwierzbicki at gmail.com Fri Jun 13 05:19:15 2008 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Thu, 12 Jun 2008 23:19:15 -0400 Subject: [Python-Dev] bug or a feature? In-Reply-To: <484FA003.5020703@gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F4E45.9060004@canterbury.ac.nz> <484FA003.5020703@gmail.com> Message-ID: <4dab5f760806122019ob74feexbe60a4191630afd6@mail.gmail.com> On Wed, Jun 11, 2008 at 5:50 AM, Nick Coghlan wrote: > Greg Ewing wrote: > Implementations are also permitted to restrict namespace dictionaries to > only accept string keys (I believe Jython does this for performance reasons > - CPython just optimised the hell out of normal dictionaries that happen to > only contain string keys instead). Jython's current version (2.2) and back did indeed restrict these dictionaries to accept string keys only, but future versions will allow non-string keys (the trunk version already does). Many frameworks use non-string keys in these dictionaries. -Frank From guido at python.org Fri Jun 13 05:41:36 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 20:41:36 -0700 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins Message-ID: My colleague and SVN developer Ben Sussman-Collins occasionally blogs about the social side of (mostly open source) software development. He just posted a new one that struck a chord: http://blog.red-bean.com/sussman/?p=96 The story's main moral: submit your code for review early and often; work in a branch if you need to, but don't hide your code from review in a local repository until it's "perfect". Let's all remember this and make sure not to drop "code bombs" on each other. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 13 05:42:58 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 20:42:58 -0700 Subject: [Python-Dev] bug or a feature? In-Reply-To: <4dab5f760806122019ob74feexbe60a4191630afd6@mail.gmail.com> References: <693bc9ab0806101810r60f7a23dl48bce0ea3fe32ba4@mail.gmail.com> <484F4E45.9060004@canterbury.ac.nz> <484FA003.5020703@gmail.com> <4dab5f760806122019ob74feexbe60a4191630afd6@mail.gmail.com> Message-ID: On Thu, Jun 12, 2008 at 8:19 PM, Frank Wierzbicki wrote: > On Wed, Jun 11, 2008 at 5:50 AM, Nick Coghlan wrote: >> Greg Ewing wrote: >> Implementations are also permitted to restrict namespace dictionaries to >> only accept string keys (I believe Jython does this for performance reasons >> - CPython just optimised the hell out of normal dictionaries that happen to >> only contain string keys instead). > Jython's current version (2.2) and back did indeed restrict these > dictionaries to accept string keys only, but future versions will > allow non-string keys (the trunk version already does). Many > frameworks use non-string keys in these dictionaries. I think that's laudable from a compatibility POV, but at the same time I think frameworks should rethink such usage and switch to strings containing non-identifier characters, e.g. '$' or '.' or some such convention. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Fri Jun 13 05:49:11 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 12 Jun 2008 20:49:11 -0700 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins In-Reply-To: References: Message-ID: On Thu, Jun 12, 2008 at 8:41 PM, Guido van Rossum wrote: > My colleague and SVN developer Ben Sussman-Collins occasionally blogs > about the social side of (mostly open source) software development. He > just posted a new one that struck a chord: > > http://blog.red-bean.com/sussman/?p=96 > > The story's main moral: submit your code for review early and often; > work in a branch if you need to, but don't hide your code from review > in a local repository until it's "perfect". > > Let's all remember this and make sure not to drop "code bombs" on each > other. :-) Ben mentions this in the post, but it's a good reminder: comments on python-checkins are *not* personal. The goal is to make the code better and/or gain better understanding. We all make mistakes, better to correct them early before they become big problems.. n From guido at python.org Fri Jun 13 05:51:38 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 20:51:38 -0700 Subject: [Python-Dev] [Python-3000-checkins] r64217 - in python/branches/py3k/Lib: bsddb/test/test_associate.py bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtempfile.py thread Message-ID: It's water under the bridge now, but IMO it was too rash to *remove* the old threading API from Py3k, and doubly rash to do so one day before the beta release. Running up to a release (whether alpha, beta or final) we should practice extra restraint, not rush to get things in right before the deadline. Let's all be more careful the rest of this release cycle! (I think it wasn't just Benjamin who raced to get things in...) --Guido On Thu, Jun 12, 2008 at 7:00 PM, benjamin.peterson wrote: > Author: benjamin.peterson > Date: Fri Jun 13 04:00:47 2008 > New Revision: 64217 > > Log: > fix more threading API related bugs > > Modified: > python/branches/py3k/Lib/bsddb/test/test_associate.py > python/branches/py3k/Lib/bsddb/test/test_join.py > python/branches/py3k/Lib/bsddb/test/test_lock.py > python/branches/py3k/Lib/bsddb/test/test_thread.py > python/branches/py3k/Lib/idlelib/rpc.py > python/branches/py3k/Lib/idlelib/run.py > python/branches/py3k/Lib/socketserver.py > python/branches/py3k/Lib/test/test_threadedtempfile.py > python/branches/py3k/Lib/threading.py > > Modified: python/branches/py3k/Lib/bsddb/test/test_associate.py > ============================================================================== > --- python/branches/py3k/Lib/bsddb/test/test_associate.py (original) > +++ python/branches/py3k/Lib/bsddb/test/test_associate.py Fri Jun 13 04:00:47 2008 > @@ -9,7 +9,7 @@ > from pprint import pprint > > try: > - from threading import Thread, currentThread > + from threading import Thread, current_thread > have_threads = 1 > except ImportError: > have_threads = 0 > [etc.] -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 13 05:53:39 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 12 Jun 2008 20:53:39 -0700 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins In-Reply-To: References: Message-ID: On Thu, Jun 12, 2008 at 8:49 PM, Neal Norwitz wrote: > On Thu, Jun 12, 2008 at 8:41 PM, Guido van Rossum wrote: >> My colleague and SVN developer Ben Sussman-Collins occasionally blogs >> about the social side of (mostly open source) software development. He >> just posted a new one that struck a chord: >> >> http://blog.red-bean.com/sussman/?p=96 >> >> The story's main moral: submit your code for review early and often; >> work in a branch if you need to, but don't hide your code from review >> in a local repository until it's "perfect". >> >> Let's all remember this and make sure not to drop "code bombs" on each >> other. :-) > > Ben mentions this in the post, but it's a good reminder: comments on > python-checkins are *not* personal. The goal is to make the code > better and/or gain better understanding. We all make mistakes, better > to correct them early before they become big problems.. And this reminder applies to reviewer *and* reviewees! (I know I've made this mistake in both roles. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Fri Jun 13 10:54:12 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jun 2008 18:54:12 +1000 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <18513.16097.394123.312491@montanaro-dyndns-org.local> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> Message-ID: <485235B4.6030401@gmail.com> skip at pobox.com wrote: > I'm not fond of using a property for this since it can lull you into the > false belief that what you are doing is less expensive than it really is > (attribute access vs method call). I think this is a case where explicit is > better than implicit. Have you looked at what the methods we're proposing to turn into properties are actually doing?: def getName(self): assert self.__initialized, "Thread.__init__() not called" return self.__name def setName(self, name): assert self.__initialized, "Thread.__init__() not called" self.__name = str(name) def getIdent(self): assert self.__initialized, "Thread.__init__() not called" return self.__ident def isAlive(self): assert self.__initialized, "Thread.__init__() not called" return self.__started.isSet() and not self.__stopped def isDaemon(self): assert self.__initialized, "Thread.__init__() not called" return self.__daemonic def setDaemon(self, daemonic): if not self.__initialized: raise RuntimeError("Thread.__init__() not called") if self.__started.isSet(): raise RuntimeError("cannot set daemon status of active thread"); self.__daemonic = daemonic Checking whether or not an Event is set is hardly an expensive operation and well within the limits of what I think it is reasonable for a property access to do implicitly (e.g. it would also be reasonable for a thread safe object to acquire and release a synchronisation lock while retrieving or changing an attribute). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jun 13 10:56:01 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jun 2008 18:56:01 +1000 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <4850FBFC.9070501@gmail.com> Message-ID: <48523621.6050604@gmail.com> Georg Brandl wrote: > Nick Coghlan schrieb: > >> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute >> in the instance dictionary MAY have ill effects. (slots such as >> __enter__ and __exit__ that are looked up via normal attribute lookup >> in CPython will fit into this category) > > I would argue that the __enter__ and __exit__ behavior should be changed > too. > The reason for the current behavior is this: > > 2 0 LOAD_GLOBAL 0 (a) > 3 DUP_TOP > 4 LOAD_ATTR 1 (__exit__) > 7 STORE_FAST 0 (_[1]) > 10 LOAD_ATTR 2 (__enter__) > 13 CALL_FUNCTION 0 > 16 STORE_FAST 1 (_[2]) > 19 SETUP_FINALLY 18 (to 40) > > IOW, when "with" is compiled, the attributes are retrieved using bytecode. > It wouldn't be hard to have a WITH_SETUP opcode (along with the already > existing WITH_CLEANUP) that would make the bytecode read like: > > 2 0 LOAD_GLOBAL 0 (a) > 3 WITH_SETUP 0 (_[1]) > 6 STORE_FAST 1 (_[2]) > 9 SETUP_FINALLY 18 (to 40) No argument here - the PEP 343 implementation is the way it is mainly because it involved the least messing around with the performance-sensitive ceval loop. That said, the with statement implementation is already a bit different in 2.6/3.0 (it moves things around on the stack so it can avoid the STORE_FAST/LOAD_FAST/DELETE_FAST operations): def f(): with a: pass Python 2.5 disassembly: 2 0 LOAD_GLOBAL 0 (a) 3 DUP_TOP 4 LOAD_ATTR 1 (__exit__) 7 STORE_FAST 0 (_[1]) 10 LOAD_ATTR 2 (__enter__) 13 CALL_FUNCTION 0 16 POP_TOP 17 SETUP_FINALLY 4 (to 24) 20 POP_BLOCK 21 LOAD_CONST 0 (None) >> 24 LOAD_FAST 0 (_[1]) 27 DELETE_FAST 0 (_[1]) 30 WITH_CLEANUP 31 END_FINALLY Python 2.6 disassembly: 2 0 LOAD_GLOBAL 0 (a) 3 DUP_TOP 4 LOAD_ATTR 1 (__exit__) 7 ROT_TWO 8 LOAD_ATTR 2 (__enter__) 11 CALL_FUNCTION 0 14 POP_TOP 15 SETUP_FINALLY 4 (to 22) 18 POP_BLOCK 19 LOAD_CONST 0 (None) >> 22 WITH_CLEANUP 23 END_FINALLY Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jun 13 11:06:00 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jun 2008 19:06:00 +1000 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins In-Reply-To: References: Message-ID: <48523878.9030104@gmail.com> Guido van Rossum wrote: > On Thu, Jun 12, 2008 at 8:49 PM, Neal Norwitz > wrote: >> Ben mentions this in the post, but it's a good reminder: comments >> on python-checkins are *not* personal. The goal is to make the >> code better and/or gain better understanding. We all make >> mistakes, better to correct them early before they become big >> problems.. > > And this reminder applies to reviewer *and* reviewees! (I know I've > made this mistake in both roles. :-) I still love the last entry from Raymond's school of hard knocks [1]: """do everything right (formatting, procedure, profiling, testing, etc) and watch the Timbot come along five minutes later and improve your code making it faster, clearer, more conformant, more elegant, and also gel neatly with the vaguaries of memory allocation, cache performance, and compilers you've never heard of.""" Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2002-September/028725.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From walter at livinglogic.de Fri Jun 13 11:32:21 2008 From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=) Date: Fri, 13 Jun 2008 11:32:21 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <48514428.7040001@egenix.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> <484FEC2C.6070409@livinglogic.de> <485003F8.40803@egenix.com> <485139EA.1000502@livinglogic.de> <48514428.7040001@egenix.com> Message-ID: <48523EA5.2070407@livinglogic.de> M.-A. Lemburg wrote: > On 2008-06-12 16:59, Walter D?rwald wrote: >> M.-A. Lemburg wrote: >>> .transform() and .untransform() use the codecs to apply same-type >>> conversions. They do apply type checks to make sure that the >>> codec does indeed return the same type. >>> >>> E.g. text.transform('xml-escape') or data.transform('base64'). >> >> So what would a base64 codec do with the errors argument? > > It could use it to e.g. try to recover as much data as possible > from broken input data. > > Currently (in Py2.x), it raises an exception if you pass in anything > but "strict". > >>>> I think for transformations we don't need the full codec machinery: >>> > ... >>> >>> No need to invent another wheel :-) The codecs already exist for >>> Py2.x and can be used by the .encode()/.decode() methods in Py2.x >>> (where no type checks occur). >> >> By using a new API we could get rid of old warts. For example: Why >> does the stateless encoder/decoder return how many input >> characters/bytes it has consumed? It must consume *all* bytes anyway! > > No, it doesn't and that's the point in having those return values :-) > > Even though the encoder/decoders are stateless, that doesn't mean > they have to consume all input data. The caller is responsible to > make sure that all input data was in fact consumed. > > You could for example have a decoder that stops decoding after > having seen a block end indicator, e.g. a base64 line end or > XML closing element. So how should the UTF-8 decoder know that it has to stop at a closing XML element? > Just because all codecs that ship with Python always try to decode > the complete input doesn't mean that the feature isn't being used. I know of no other code that does. Do you have an example for this use. > The interface was designed to allow for the above situations. Then could we at least have a new codec method that does: def statelesencode(self, input): (output, consumed) = self.encode(input) assert len(input) == consumed return output Servus, Walter From fuzzyman at voidspace.org.uk Fri Jun 13 12:02:53 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 13 Jun 2008 11:02:53 +0100 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <4850FBFC.9070501@gmail.com> Message-ID: <485245CD.9000602@voidspace.org.uk> Guido van Rossum wrote: > [Barry] > >>>> http://bugs.python.org/issue643841 >>>> > > [Guido] > >>> I've added a comment. Let me know if anything I said is unclear. >>> > > On Thu, Jun 12, 2008 at 3:35 AM, Nick Coghlan wrote: > >> The bugtracker seems to be offline atm - I'll reply there once I can get to >> it again (as well as switching this issue back to being a documentation >> one). >> >> I don't think we're going to see a major clamor for a value-based delegation >> mixin in the standard library until people using classic classes for >> value-based delegation start making serious attempts to port to Py3k (where >> that approach is no longer available). At the moment, such classes only need >> to care about the methods they want to fiddle with, leaving everything else >> to __getattr__ based delegation. >> > > Whether they'll care about this issue of course depends on whether > they need overloaded operators and other special delegations to be > delegated transparently. We'll have to see how important this is. > New-style classes have been around and recommended for a long time -- > why haven't people pushed for a proxy class before? > > It's only in Python 3 that old style classes are going away fully, so up until now you could at least use a classic class to do the proxying. I've written my own proxy classes before that look very similar to this, and there are other proxy classes around that do the same (I thought one was by Phillip J Eby but can't find a reference easily). The last one I wrote was to proxy CPython objects from IronPython via Python.NET... I would prefer it if the proxy class wrapped the return values of inplace operations. Michael Foord >> I've pushed as hard as I'm personally willing to for this without convincing >> anyone else that it's worth doing, >> > > What does *it* refer to? Changing the behavior, or adding a proxy > class to the stdlib? I'm -1000 on the former, but only -0 on the > latter -- as I wrote in the tracker, I just don't want to see an > unproven proxy class (and I don't like the module name). > > >> so I'll start working on a documentation >> patch for the language reference instead which explicitly splits the special >> methods into the following categories: >> > > Thanks for doing this, it is needed regardless! > > >> 1. Method lookup MUST bypass __getattribute__, shadowing the attribute in >> the instance dictionary MUST NOT have any ill effects. (All tp_* C-level >> slots and slots looked up via _PyType_Lookup will fit into this category) >> > > Watch out: I think the term "method lookup" may be confusing here. > Certainly when the user writes x.__foo__(), the instance dict *is* > consulted. It is only on *implied* lookups (e.g. x[y] or x+y) where > the instance dict is bypassed. > > >> 2. Method lookup MAY bypass __getattribute__, shadowing the attribute in the >> instance dictionary MAY have ill effects. (slots such as __enter__ and >> __exit__ that are looked up via normal attribute lookup in CPython will fit >> into this category) >> > > Why even have a MAY category? Are you expecting these to become tp_ > slots at some point? > > >> 3. Technically a subcategory of group 1, these are special methods which can >> affect the interpreter's behaviour by their mere definition on a type. (The >> __get__, __set__ and __delete__ descriptor protocol methods fall into this >> category) >> > > I don't follow why this is relevant. This is a different, AFAIK > orthogonal issue, used in many places: *if* an object used in a > certain context has a specific attribute, *then* that attribute is > used, *otherwise* a default action is taken. Applies to __repr__ just > as much. These belong in category 1 if and only if the lookup bypasses > the instance dict. > > From mal at egenix.com Fri Jun 13 12:14:49 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 13 Jun 2008 12:14:49 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <48523EA5.2070407@livinglogic.de> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> <484FEC2C.6070409@livinglogic.de> <485003F8.40803@egenix.com> <485139EA.1000502@livinglogic.de> <48514428.7040001@egenix.com> <48523EA5.2070407@livinglogic.de> Message-ID: <48524899.2030609@egenix.com> On 2008-06-13 11:32, Walter D?rwald wrote: > M.-A. Lemburg wrote: >> On 2008-06-12 16:59, Walter D?rwald wrote: >>> M.-A. Lemburg wrote: >>>> .transform() and .untransform() use the codecs to apply same-type >>>> conversions. They do apply type checks to make sure that the >>>> codec does indeed return the same type. >>>> >>>> E.g. text.transform('xml-escape') or data.transform('base64'). >>> >>> So what would a base64 codec do with the errors argument? >> >> It could use it to e.g. try to recover as much data as possible >> from broken input data. >> >> Currently (in Py2.x), it raises an exception if you pass in anything >> but "strict". >> >>>>> I think for transformations we don't need the full codec machinery: >>>> > ... >>>> >>>> No need to invent another wheel :-) The codecs already exist for >>>> Py2.x and can be used by the .encode()/.decode() methods in Py2.x >>>> (where no type checks occur). >>> >>> By using a new API we could get rid of old warts. For example: Why >>> does the stateless encoder/decoder return how many input >>> characters/bytes it has consumed? It must consume *all* bytes anyway! >> >> No, it doesn't and that's the point in having those return values :-) >> >> Even though the encoder/decoders are stateless, that doesn't mean >> they have to consume all input data. The caller is responsible to >> make sure that all input data was in fact consumed. >> >> You could for example have a decoder that stops decoding after >> having seen a block end indicator, e.g. a base64 line end or >> XML closing element. > > So how should the UTF-8 decoder know that it has to stop at a closing > XML element? The UTF-8 decoder doesn't support this, but you could write a codec that applies this kind of detection, e.g. to not try to decode partial UTF-8 byte sequences at the end of input, which would then result in error. >> Just because all codecs that ship with Python always try to decode >> the complete input doesn't mean that the feature isn't being used. > > I know of no other code that does. Do you have an example for this use. I already gave you a few examples. >> The interface was designed to allow for the above situations. > > Then could we at least have a new codec method that does: > > def statelesencode(self, input): > (output, consumed) = self.encode(input) > assert len(input) == consumed > return output You mean as method to the Codec class ? Sure, we could do that, but please use a different name, e.g. .encodeall() and .decodeall() - .encode() and .decode() are already stateles (and so would the new methods be), so "stateless" isn't all that meaningful in this context. We could also add such a check to the PyCodec_Encode() and _Decode() functions. They currently do not apply the above check. In Python, those two functions are exposed as codecs.encode() and codecs.decode(). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 13 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 23 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From steve at holdenweb.com Fri Jun 13 12:53:14 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 13 Jun 2008 06:53:14 -0400 Subject: [Python-Dev] [Python-3000-checkins] r64217 - in python/branches/py3k/Lib: bsddb/test/test_associate.py bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtempfile.py thread In-Reply-To: References: Message-ID: <4852519A.6070506@holdenweb.com> Guido van Rossum wrote: > It's water under the bridge now, but IMO it was too rash to *remove* > the old threading API from Py3k, and doubly rash to do so one day > before the beta release. Running up to a release (whether alpha, beta > or final) we should practice extra restraint, not rush to get things > in right before the deadline. Let's all be more careful the rest of > this release cycle! (I think it wasn't just Benjamin who raced to get > things in...) Well, it wouldn't be "adding a new feature" to reinstate the old API for beta two, would it, as long as we retain the new one too? It does seem that change was a little precipitate. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jun 13 12:53:14 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 13 Jun 2008 06:53:14 -0400 Subject: [Python-Dev] [Python-3000-checkins] r64217 - in python/branches/py3k/Lib: bsddb/test/test_associate.py bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtempfile.py thread In-Reply-To: References: Message-ID: <4852519A.6070506@holdenweb.com> Guido van Rossum wrote: > It's water under the bridge now, but IMO it was too rash to *remove* > the old threading API from Py3k, and doubly rash to do so one day > before the beta release. Running up to a release (whether alpha, beta > or final) we should practice extra restraint, not rush to get things > in right before the deadline. Let's all be more careful the rest of > this release cycle! (I think it wasn't just Benjamin who raced to get > things in...) Well, it wouldn't be "adding a new feature" to reinstate the old API for beta two, would it, as long as we retain the new one too? It does seem that change was a little precipitate. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Fri Jun 13 15:07:47 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 13 Jun 2008 23:07:47 +1000 Subject: [Python-Dev] [Python-3000] [Python-3000-checkins] r64217 - in python/branches/py3k/Lib: bsddb/test/test_associate.py bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtempfile.py thread In-Reply-To: <4852519A.6070506@holdenweb.com> References: <4852519A.6070506@holdenweb.com> Message-ID: <48527123.9010802@gmail.com> Steve Holden wrote: > Guido van Rossum wrote: >> It's water under the bridge now, but IMO it was too rash to *remove* >> the old threading API from Py3k, and doubly rash to do so one day >> before the beta release. Running up to a release (whether alpha, beta >> or final) we should practice extra restraint, not rush to get things >> in right before the deadline. Let's all be more careful the rest of >> this release cycle! (I think it wasn't just Benjamin who raced to get >> things in...) > > Well, it wouldn't be "adding a new feature" to reinstate the old API for > beta two, would it, as long as we retain the new one too? It does seem > that change was a little precipitate. Although if we weren't actually planning on removing the old API in 3.0, I'm a little confused as to why we were adding Py3k warnings to it in 2.6... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From walter at livinglogic.de Fri Jun 13 15:26:47 2008 From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=) Date: Fri, 13 Jun 2008 15:26:47 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <48524899.2030609@egenix.com> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> <484FEC2C.6070409@livinglogic.de> <485003F8.40803@egenix.com> <485139EA.1000502@livinglogic.de> <48514428.7040001@egenix.com> <48523EA5.2070407@livinglogic.de> <48524899.2030609@egenix.com> Message-ID: <48527597.4090206@livinglogic.de> M.-A. Lemburg wrote: > On 2008-06-13 11:32, Walter D?rwald wrote: >> M.-A. Lemburg wrote: >>> On 2008-06-12 16:59, Walter D?rwald wrote: >>>> M.-A. Lemburg wrote: >>>>> .transform() and .untransform() use the codecs to apply same-type >>>>> conversions. They do apply type checks to make sure that the >>>>> codec does indeed return the same type. >>>>> >>>>> E.g. text.transform('xml-escape') or data.transform('base64'). >>>> >>>> So what would a base64 codec do with the errors argument? >>> >>> It could use it to e.g. try to recover as much data as possible >>> from broken input data. >>> >>> Currently (in Py2.x), it raises an exception if you pass in anything >>> but "strict". >>> >>>>>> I think for transformations we don't need the full codec machinery: >>>>> > ... >>>>> >>>>> No need to invent another wheel :-) The codecs already exist for >>>>> Py2.x and can be used by the .encode()/.decode() methods in Py2.x >>>>> (where no type checks occur). >>>> >>>> By using a new API we could get rid of old warts. For example: Why >>>> does the stateless encoder/decoder return how many input >>>> characters/bytes it has consumed? It must consume *all* bytes anyway! >>> >>> No, it doesn't and that's the point in having those return values :-) >>> >>> Even though the encoder/decoders are stateless, that doesn't mean >>> they have to consume all input data. The caller is responsible to >>> make sure that all input data was in fact consumed. >>> >>> You could for example have a decoder that stops decoding after >>> having seen a block end indicator, e.g. a base64 line end or >>> XML closing element. >> >> So how should the UTF-8 decoder know that it has to stop at a closing >> XML element? > > The UTF-8 decoder doesn't support this, but you could write a codec > that applies this kind of detection, e.g. to not try to decode > partial UTF-8 byte sequences at the end of input, which would then > result in error. > >>> Just because all codecs that ship with Python always try to decode >>> the complete input doesn't mean that the feature isn't being used. >> >> I know of no other code that does. Do you have an example for this use. > > I already gave you a few examples. Maybe I was unclear, I meant real world examples, not hypothetical ones. >>> The interface was designed to allow for the above situations. >> >> Then could we at least have a new codec method that does: >> >> def statelesencode(self, input): >> (output, consumed) = self.encode(input) >> assert len(input) == consumed >> return output > > You mean as method to the Codec class ? No, I meant as a method for the CodecInfo clas. > Sure, we could do that, but please use a different name, > e.g. .encodeall() and .decodeall() - .encode() and .decode() > are already stateles (and so would the new methods be), so > "stateless" isn't all that meaningful in this context. I like the names encodeall/decodeall! > We could also add such a check to the PyCodec_Encode() and _Decode() > functions. They currently do not apply the above check. > > In Python, those two functions are exposed as codecs.encode() > and codecs.decode(). This change will probably have to wait for the 2.7 cycle. Servus, Walter From barry at python.org Fri Jun 13 15:47:40 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Jun 2008 09:47:40 -0400 Subject: [Python-Dev] First betas (perhaps) on June 18 Message-ID: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It's unfortunate that I was not able to release the first betas on Wednesday. The primary reason was that none of the 3.0 buildbots were green. My proposal is this: I will spin another release this coming Wednesday, June 18. If we can get both the 2.6 and 3.0 buildbots green by then, and close out all remaining release critical bugs, then Wednesday's release will be beta 1. In that case, I will update PEP 361 and push the entire schedule back by 2 weeks to account for our slippage. If we can't get these releases stable by then, Wednesday's release will be alpha and I will make the July 2 release the first beta, updating and pushing back the PEP 361 schedule accordingly. Thanks for all your help in getting 3.0 stable! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFJ6fXEjvBPtnXfVAQJekgP/XzGR0h6Kcgwe/eo/hlAP2RJiEBny7iIT avSkq5sDD4YiUHeSSVy/8A5rahjR3Bsrzgh0MSePrf8coUHz3N+OCGsJK07BVJIT 9wan/OXTkpsrqa2xQgrokk0PqCE80S2Lp6+P5jMW06tujx3fFn+NJNDjaNUOKPUc nlfjFpdwYxc= =Nwmx -----END PGP SIGNATURE----- From facundobatista at gmail.com Fri Jun 13 15:54:08 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 13 Jun 2008 10:54:08 -0300 Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18 In-Reply-To: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> Message-ID: 2008/6/13 Barry Warsaw : > My proposal is this: I will spin another release this coming Wednesday, June > 18. If we can get both the 2.6 and 3.0 buildbots green by then, and close > out all remaining release critical bugs, then Wednesday's release will be > beta 1. In that case, I will update PEP 361 and push the entire schedule > back by 2 weeks to account for our slippage. Next weekend, 21 and 22, it will be the Python Bug days. Maybe it's worth to delay the betas one week to include this work? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From guido at python.org Fri Jun 13 16:20:09 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 07:20:09 -0700 Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18 In-Reply-To: References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> Message-ID: On Fri, Jun 13, 2008 at 6:54 AM, Facundo Batista wrote: > 2008/6/13 Barry Warsaw : > >> My proposal is this: I will spin another release this coming Wednesday, June >> 18. If we can get both the 2.6 and 3.0 buildbots green by then, and close >> out all remaining release critical bugs, then Wednesday's release will be >> beta 1. In that case, I will update PEP 361 and push the entire schedule >> back by 2 weeks to account for our slippage. > > Next weekend, 21 and 22, it will be the Python Bug days. > > Maybe it's worth to delay the betas one week to include this work? You mean one more week? (Barry's already delaying them one week.) I'm fine with whatever delay Barry approves, but I want EVERYONE to stick to the purpose of the delay: get the buildbots green and close out release critical bugs. This means no new last-minute features or big refactorings (unless there's really no more conservative way to fix a critical bug). It also means try to get at least one other person to review every change, no matter how small (or big!) *before* submitting. It also means run the tests before submitting. On the specific purpose of delaying for the bug day, I'm hesitant -- bug days tend to cause a great deal of commits of code by relatively new developers, which is counter to our purpose of stability. I would almost say it's better for the bug day to fall right after the beta. That also has the advantage that the bug day can focus on new bugs in the beta, and squash them right away... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 13 16:33:37 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 07:33:37 -0700 Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18 In-Reply-To: References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> Message-ID: On Fri, Jun 13, 2008 at 7:27 AM, Georg Brandl wrote: > Guido van Rossum schrieb: >> >> On Fri, Jun 13, 2008 at 6:54 AM, Facundo Batista >> wrote: >>> >>> 2008/6/13 Barry Warsaw : >>> >>>> My proposal is this: I will spin another release this coming Wednesday, >>>> June >>>> 18. If we can get both the 2.6 and 3.0 buildbots green by then, and >>>> close >>>> out all remaining release critical bugs, then Wednesday's release will >>>> be >>>> beta 1. In that case, I will update PEP 361 and push the entire >>>> schedule >>>> back by 2 weeks to account for our slippage. >>> >>> Next weekend, 21 and 22, it will be the Python Bug days. >>> >>> Maybe it's worth to delay the betas one week to include this work? >> >> You mean one more week? (Barry's already delaying them one week.) >> >> I'm fine with whatever delay Barry approves, but I want EVERYONE to >> stick to the purpose of the delay: get the buildbots green and close >> out release critical bugs. This means no new last-minute features or >> big refactorings (unless there's really no more conservative way to >> fix a critical bug). It also means try to get at least one other >> person to review every change, no matter how small (or big!) *before* >> submitting. It also means run the tests before submitting. > > That is a new policy, isn't it? Yes -- one that seems appropriate during a beta release period (i.e. until 2.6 and 3.0 final are out). >> On the specific purpose of delaying for the bug day, I'm hesitant -- >> bug days tend to cause a great deal of commits of code by relatively >> new developers, which is counter to our purpose of stability. I would >> almost say it's better for the bug day to fall right after the beta. >> That also has the advantage that the bug day can focus on new bugs in >> the beta, and squash them right away... > > +1. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Fri Jun 13 16:35:25 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 09:35:25 -0500 Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18 In-Reply-To: References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> Message-ID: <1afaf6160806130735v40252796laf478e9e99f45148@mail.gmail.com> On Fri, Jun 13, 2008 at 9:20 AM, Guido van Rossum wrote: > > I'm fine with whatever delay Barry approves, but I want EVERYONE to > stick to the purpose of the delay: get the buildbots green and close > out release critical bugs. This means no new last-minute features or > big refactorings (unless there's really no more conservative way to > fix a critical bug). It also means try to get at least one other > person to review every change, no matter how small (or big!) *before* > submitting. It also means run the tests before submitting. I will quite happily abide by this; it's already causing more than enough stress over here. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Fri Jun 13 16:47:32 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 09:47:32 -0500 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins In-Reply-To: References: Message-ID: <1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com> On Thu, Jun 12, 2008 at 10:41 PM, Guido van Rossum wrote: > > Let's all remember this and make sure not to drop "code bombs" on each > other. :-) Ok! I'd like everybody to know that I'm working on Python tests on a Bazaar branch. [1] (That's only a bzr co --lightweight away.) Go ahead and send me angry emails about how it's totally wrong, and your cat could do better. [1] http://code.python.org/python/users/benjamin.peterson/new_testing/main/ -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From solipsis at pitrou.net Fri Jun 13 17:54:31 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 13 Jun 2008 15:54:31 +0000 (UTC) Subject: [Python-Dev] [OT] Interesting blog post by Ben Sussman-Collins References: <1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com> Message-ID: Benjamin Peterson gmail.com> writes: > > On Thu, Jun 12, 2008 at 10:41 PM, Guido van Rossum python.org> wrote: > > > > Let's all remember this and make sure not to drop "code bombs" on each > > other. > > Ok! I'd like everybody to know that I'm working on Python tests on a > Bazaar branch. [1] (That's only a bzr co --lightweight away.) Go ahead > and send me angry emails about how it's totally wrong, and your cat > could do better. If I had a cat it would probably use Mercurial instead :-) From status at bugs.python.org Fri Jun 13 18:06:27 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 13 Jun 2008 18:06:27 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080613160627.4766B7818E@psf.upfronthosting.co.za> ACTIVITY SUMMARY (06/06/08 - 06/13/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1899 open (+36) / 13028 closed (+15) / 14927 total (+51) Open issues with patches: 574 Average duration of open issues: 713 days. Median duration of open issues: 1464 days. Open Issues Breakdown open 1873 (+35) pending 26 ( +1) Issues Created Or Reopened (54) _______________________________ merge pickle and cPickle in 3.0 06/12/08 CLOSED http://bugs.python.org/issue2917 reopened benjamin.peterson patch Implement PEP 371: multiprocessing module 06/11/08 CLOSED http://bugs.python.org/issue3050 reopened benjamin.peterson patch heapq change breaking compatibility 06/09/08 http://bugs.python.org/issue3051 reopened rhettinger patch test_disutils fails 06/06/08 http://bugs.python.org/issue3054 created ronaldoussoren patch test_list on 64-bit platforms 06/06/08 http://bugs.python.org/issue3055 created ronaldoussoren Simplify the Integral ABC 06/07/08 http://bugs.python.org/issue3056 created rhettinger patch 2.6 abc fixes 06/07/08 CLOSED http://bugs.python.org/issue3057 created georg.brandl patch, 26backport Let SimpleXMLRPCServer pass client_address to called functions. 06/07/08 http://bugs.python.org/issue3058 created cloverprince Removing .decode calls from Lib/calendar.py 06/07/08 CLOSED http://bugs.python.org/issue3059 created gpolo patch Warn about tuple parameters 06/08/08 CLOSED http://bugs.python.org/issue3060 created benjamin.peterson patch time.strftime() always decodes result with UTF-8 06/08/08 http://bugs.python.org/issue3061 created georg.brandl Turtle speed() function has no effect under Mac OS X 06/08/08 http://bugs.python.org/issue3062 created delroth memory leak in random number generation 06/08/08 CLOSED http://bugs.python.org/issue3063 created gtang new turtle module for Python 3.0 06/08/08 CLOSED http://bugs.python.org/issue3064 created gregorlingl patch Fix pickling bug of collections.namedtuple 06/08/08 CLOSED http://bugs.python.org/issue3065 created alexandre.vassalotti patch, patch FD leak in urllib2 06/09/08 http://bugs.python.org/issue3066 created bohdan setlocale Tracebacks on unicode locale strings 06/09/08 http://bugs.python.org/issue3067 created vincent.chute IDLE - Add an extension configuration dialog 06/09/08 http://bugs.python.org/issue3068 created taleinat patch Let set.union and set.intersection accept multiple arguments 06/09/08 http://bugs.python.org/issue3069 created arno patch Wrong size calculation in posix_execve 06/09/08 http://bugs.python.org/issue3070 created Rhamphoryncus The ValueError raised by failing to unpack sequence should have 06/10/08 http://bugs.python.org/issue3071 created jml Assignment to list of lists gives incorrect result 06/10/08 CLOSED http://bugs.python.org/issue3072 created stesteve Cookie.Morsel breaks in parsing cookie values with whitespace 06/10/08 http://bugs.python.org/issue3073 created rhymes test_asyncore is failing 06/10/08 CLOSED http://bugs.python.org/issue3074 created benjamin.peterson make minidom.toxml() encoding argument useful 06/10/08 http://bugs.python.org/issue3075 created janssen easy test_xmlrpc_net fails 06/10/08 CLOSED http://bugs.python.org/issue3076 created cartman h2py char literal doesn work 06/11/08 http://bugs.python.org/issue3077 created grossetti patch tokenize.py improvements 06/11/08 http://bugs.python.org/issue3078 created Indy patch sys.exit() called from optparse - bad, bad, bad 06/11/08 http://bugs.python.org/issue3079 created skip.montanaro easy Full unicode import system 06/11/08 http://bugs.python.org/issue3080 created amaury.forgeotdarc Py_(X)SETREF macros 06/11/08 http://bugs.python.org/issue3081 created pitrou patch test_multiprocessing broken 06/11/08 CLOSED http://bugs.python.org/issue3082 created gregory.p.smith Add alternate (#) formatting for bin, oct, hex output for str.fo 06/11/08 http://bugs.python.org/issue3083 created eric.smith sys.exit() called from optparse - bad, bad, bad 06/11/08 CLOSED http://bugs.python.org/issue3084 created skip.montanaro chapter 17.1.3.5 'Replacing os.popen*' in the Python library ref 06/11/08 http://bugs.python.org/issue3085 created afoo sys.maxsize not available by using the latest Win32 build 06/12/08 CLOSED http://bugs.python.org/issue3086 created giampaolo.rodola Clean up Demos and Tools 06/12/08 http://bugs.python.org/issue3087 created benjamin.peterson easy test_multiprocessing hangs on OS X 10.5.3 06/12/08 http://bugs.python.org/issue3088 created barry All 3.0 buildbots are red 06/12/08 http://bugs.python.org/issue3089 created barry ARCHFLAGS parsing/concatenation in unixccompiler.py breaks when 06/12/08 http://bugs.python.org/issue3090 created jnoller patch Can't build datetime or time on py3k (r64171) 06/12/08 CLOSED http://bugs.python.org/issue3091 created skip.montanaro Wrong unicode size detection in pybench 06/12/08 http://bugs.python.org/issue3092 created pitrou patch Namespace polution from multiprocessing 06/12/08 http://bugs.python.org/issue3093 created Rhamphoryncus By default, HTTPSConnection should send header "Host: somehost" 06/12/08 http://bugs.python.org/issue3094 created steven.wong patch multiprocessing initializes flags dict unsafely 06/12/08 CLOSED http://bugs.python.org/issue3095 created Rhamphoryncus patch sphinx: sort warnings by filename 06/12/08 CLOSED http://bugs.python.org/issue3096 created mgedmin patch sphinx: config option for exclude_dirnames 06/12/08 http://bugs.python.org/issue3097 created mgedmin sys.sizeof test fails with wide unicode 06/12/08 http://bugs.python.org/issue3098 created benjamin.peterson On windows, "import nul" always succeed 06/13/08 http://bugs.python.org/issue3099 created amaury.forgeotdarc weakref subclass segfault 06/13/08 http://bugs.python.org/issue3100 created Rhamphoryncus patch global function _add_one_to_C 06/13/08 http://bugs.python.org/issue3101 created loewis ctypes defines global symbols 06/13/08 http://bugs.python.org/issue3102 created loewis sqlite defines a few global symbols. 06/13/08 http://bugs.python.org/issue3103 created loewis overzealous garbage collector (dict) 06/13/08 http://bugs.python.org/issue3104 created GreaseMonkey Issues Now Closed (53) ______________________ Crash on Windows if Python runs from a directory with umlauts 229 days http://bugs.python.org/issue1342 amaury.forgeotdarc patch Thread local storage and PyGILState_* mucked up by os.fork() 174 days http://bugs.python.org/issue1683 benjamin.peterson test_crasher in test_struct uses 8 GB memory on 64 bit systems 114 days http://bugs.python.org/issue2137 gregory.p.smith patch, 64bit, easy Add a factorial function 83 days http://bugs.python.org/issue2138 rhettinger Update PEP 3000 with new release schedule 85 days http://bugs.python.org/issue2330 georg.brandl Py3K warn against assigning to True/False 83 days http://bugs.python.org/issue2349 benjamin.peterson patch, 26backport Remove all IRIX dependant modules from aifc module 85 days http://bugs.python.org/issue2419 georg.brandl No way to disable socket timeouts in httplib, etc. 72 days http://bugs.python.org/issue2451 facundobatista patch Remove unused constants from optimized code objects 78 days http://bugs.python.org/issue2493 rhettinger patch Exception state lives too long in 3.0 29 days http://bugs.python.org/issue2507 benjamin.peterson patch itertools.permutations docstring is misleading 69 days http://bugs.python.org/issue2536 georg.brandl Unpickling of range objects fail in Py3k 63 days http://bugs.python.org/issue2582 alexandre.vassalotti patch Descriptor instance attributes not interpreted consistently 62 days http://bugs.python.org/issue2605 gvanrossum Tile module: Add support for themed widgets 58 days http://bugs.python.org/issue2618 benjamin.peterson repr() should not escape non-ASCII characters 59 days http://bugs.python.org/issue2630 ishimoto patch __exit__ silences the active exception 30 days http://bugs.python.org/issue2833 benjamin.peterson Remove cl usage from aifc 25 days http://bugs.python.org/issue2847 georg.brandl patch Remove mimetools usage from the stdlib 30 days http://bugs.python.org/issue2848 benjamin.peterson patch Remove usage of rfc822 from the stdlib 30 days http://bugs.python.org/issue2849 benjamin.peterson patch rewrite test_struct as a unittest 24 days http://bugs.python.org/issue2911 benjamin.peterson patch, easy let platform.uname try harder 26 days http://bugs.python.org/issue2912 benjamin.peterson patch, easy merge pickle and cPickle in 3.0 1 days http://bugs.python.org/issue2917 alexandre.vassalotti patch Merge StringIO/cStringIO in 3.0 23 days http://bugs.python.org/issue2918 alexandre.vassalotti patch test_pydoc fails in trunk 15 days http://bugs.python.org/issue2976 benjamin.peterson PyNumberMethods has left-over fields in Py3 10 days http://bugs.python.org/issue2997 benjamin.peterson patch disutils fails with GNU ld (GNU Binutils) 2.18.50.20080523 12 days http://bugs.python.org/issue3013 georg.brandl patch, easy Lexical exception handlers 10 days http://bugs.python.org/issue3021 benjamin.peterson patch tokenize module: normal lines, not "logical" 6 days http://bugs.python.org/issue3028 noam free list management - list, dict, set 5 days http://bugs.python.org/issue3029 gregory.p.smith patch, patch Add PEP 8 compliant aliases to threading module 6 days http://bugs.python.org/issue3042 benjamin.peterson patch Some 3k sizeof tests fail 1 days http://bugs.python.org/issue3049 loewis Implement PEP 371: multiprocessing module 0 days http://bugs.python.org/issue3050 jnoller patch Mac Modules failing to compile 1 days http://bugs.python.org/issue3052 benjamin.peterson 2.6 abc fixes 0 days http://bugs.python.org/issue3057 georg.brandl patch, 26backport Removing .decode calls from Lib/calendar.py 1 days http://bugs.python.org/issue3059 georg.brandl patch Warn about tuple parameters 1 days http://bugs.python.org/issue3060 benjamin.peterson patch memory leak in random number generation 0 days http://bugs.python.org/issue3063 tim_one new turtle module for Python 3.0 1 days http://bugs.python.org/issue3064 loewis patch Fix pickling bug of collections.namedtuple 0 days http://bugs.python.org/issue3065 rhettinger patch, patch Assignment to list of lists gives incorrect result 0 days http://bugs.python.org/issue3072 loewis test_asyncore is failing 0 days http://bugs.python.org/issue3074 josiahcarlson test_xmlrpc_net fails 0 days http://bugs.python.org/issue3076 benjamin.peterson test_multiprocessing broken 0 days http://bugs.python.org/issue3082 gregory.p.smith sys.exit() called from optparse - bad, bad, bad 0 days http://bugs.python.org/issue3084 skip.montanaro sys.maxsize not available by using the latest Win32 build 1 days http://bugs.python.org/issue3086 benjamin.peterson Can't build datetime or time on py3k (r64171) 0 days http://bugs.python.org/issue3091 skip.montanaro multiprocessing initializes flags dict unsafely 0 days http://bugs.python.org/issue3095 Rhamphoryncus patch sphinx: sort warnings by filename 0 days http://bugs.python.org/issue3096 georg.brandl patch SimpleHTTPServer reports wrong content-length for text files 1677 days http://bugs.python.org/issue839496 gagenellina patch winerror module 725 days http://bugs.python.org/issue1505257 giampaolo.rodola patch __unicode__ breaks for exception class objects 645 days http://bugs.python.org/issue1551432 davidfraser urlparse.urlunparse forms file urls incorrectly 387 days http://bugs.python.org/issue1722348 georg.brandl "%d" format handling for long values 353 days http://bugs.python.org/issue1742669 pixelbeat patch Top Issues Most Discussed (10) ______________________________ 24 Implement PEP 371: multiprocessing module 0 days closed http://bugs.python.org/issue3050 17 let platform.uname try harder 26 days closed http://bugs.python.org/issue2912 17 Error when printing an exception containing a Unicode string 74 days open http://bugs.python.org/issue2517 14 test_multiprocessing hangs on OS X 10.5.3 1 days open http://bugs.python.org/issue3088 14 heapq change breaking compatibility 4 days open http://bugs.python.org/issue3051 12 Remove mimetools usage from the stdlib 30 days closed http://bugs.python.org/issue2848 10 New class special method lookup change 2027 days open http://bugs.python.org/issue643841 10 weakref subclass segfault 1 days open http://bugs.python.org/issue3100 9 memory leak in random number generation 0 days closed http://bugs.python.org/issue3063 9 Lexical exception handlers 10 days closed http://bugs.python.org/issue3021 From skip at pobox.com Fri Jun 13 18:21:15 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 13 Jun 2008 11:21:15 -0500 Subject: [Python-Dev] multiprocessing source not "Unix clean" Message-ID: <18514.40571.3934.984310@montanaro-dyndns-org.local> In trying to solve a build problem with the multiprocessing code on Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all appear to have Windows line endings. Should those maybe be stripped to conform to the other Python source? FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does define _SEM_VALUE_MAX in sys/params.h. .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing': .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function) .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.) On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX. I used a little cpp action to define it: #ifndef SEM_VALUE_MAX # ifdef _SEM_VALUE_MAX # define SEM_VALUE_MAX _SEM_VALUE_MAX # else # define SEM_VALUE_MAX INT_MAX # endif #endif Skip From skip at pobox.com Fri Jun 13 18:29:07 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 13 Jun 2008 11:29:07 -0500 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <485235B4.6030401@gmail.com> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> <485235B4.6030401@gmail.com> Message-ID: <18514.41043.913466.680598@montanaro-dyndns-org.local> Nick> Have you looked at what the methods we're proposing to turn into Nick> properties are actually doing?: Nick> def getName(self): Nick> assert self.__initialized, "Thread.__init__() not called" Nick> return self.__name ... Nick> Checking whether or not an Event is set is hardly an expensive Nick> operation... Precisely my point. The fact that the body of the method does almost nothing means that the cost of method lookup and function call will likely be much greater than the actual work being done. Skip From python at rcn.com Fri Jun 13 18:40:16 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 13 Jun 2008 09:40:16 -0700 Subject: [Python-Dev] PEP 8 and optional underscores References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1><222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org><367889A5-6BD0-4106-948D-B5B181D4466D@python.org><18513.16097.394123.312491@montanaro-dyndns-org.local><485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> Message-ID: <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> > Nick> def getName(self): > Nick> assert self.__initialized, "Thread.__init__() not called" > Nick> return self.__name Why is __name private to begin with? Any reason for the getters and setters? Why isn't this just an attribute? Raymond From musiccomposition at gmail.com Fri Jun 13 18:42:12 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 11:42:12 -0500 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> <485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> Message-ID: <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger wrote: >> Nick> def getName(self): >> Nick> assert self.__initialized, "Thread.__init__() not called" >> Nick> return self.__name > > Why is __name private to begin with? Any reason for the getters and > setters? Why isn't this just an attribute? In 3.x, it's just an attribute. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From skip at pobox.com Fri Jun 13 18:58:10 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 13 Jun 2008 11:58:10 -0500 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins In-Reply-To: <1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com> References: <1afaf6160806130747k4835ab2cmb99ab3b87136142d@mail.gmail.com> Message-ID: <18514.42786.53127.271780@montanaro-dyndns-org.local> >> Let's all remember this and make sure not to drop "code bombs" on >> each other. :-) Benjamin> Ok! I'd like everybody to know that I'm working on Python Benjamin> tests on a Bazaar branch. [1] (That's only a bzr co Benjamin> --lightweight away.) Go ahead and send me angry emails about Benjamin> how it's totally wrong, and your cat could do better. No problem, as long as it doesn't take five months and touch every file in the distribution. I don't think Ben was suggesting there is never a good reason to work in relative isolation. I interpreted it as a rule of thumb for joint projects. In your case, while it can have a wide-ranging effect on the unit test code it's likely that you will be able to manage syncing from trunk/py3k just fine and you already have plenty of examples of how things work now and how they should work when you're done. That said, if you can commit in chunks that would probably still be a win. Where I work most of the projects are one-person, due largely to the business we are in and thus how management wants things done. I think that contributes to a number of issues, the most problematic being that APIs are developed largely in isolation, often with the needs of only one or two applications in mind. That creates problems for people who come along later and need to use those APIs, only to discover that they are fundamentally flawed in some way. Skip From guido at python.org Fri Jun 13 19:12:01 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 10:12:01 -0700 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> Message-ID: On Thu, Jun 12, 2008 at 8:40 AM, Barry Warsaw wrote: > Ideally, I would like for those considerations [i.e. whether an access is expensive] > to not enter into the API > design. I'd rather keep it clean, with sufficient documentation to give > hints about any additional costs involved. Logically .alive is a property, > so I'd like to write it that way. I beg to differ. Ideally, API design should suggest which operations are efficient and which ones aren't, in order to (subconsciously) help the developer develop a mental model about the performance characteristics of an API. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 13 19:14:14 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 10:14:14 -0700 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> <485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> Message-ID: On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson wrote: > On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger wrote: >>> Nick> def getName(self): >>> Nick> assert self.__initialized, "Thread.__init__() not called" >>> Nick> return self.__name >> >> Why is __name private to begin with? Any reason for the getters and >> setters? Why isn't this just an attribute? > > In 3.x, it's just an attribute. Oh, is it? Where's the PEP with the API redesign? Did I miss some kind of decision-making process, weighing compatibility concerns against other issues? Really, I think this all went way too fast, and should be rolled back and discussed properly *in detail* first. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 13 19:17:22 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 10:17:22 -0700 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <18514.40571.3934.984310@montanaro-dyndns-org.local> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> Message-ID: On Fri, Jun 13, 2008 at 9:21 AM, wrote: > In trying to solve a build problem with the multiprocessing code on > Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all > appear to have Windows line endings. Should those maybe be stripped to > conform to the other Python source? Ow. definitely. > FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does define > _SEM_VALUE_MAX in sys/params.h. > > .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing': > .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function) > .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once > .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.) > > On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX. I used > a little cpp action to define it: > > #ifndef SEM_VALUE_MAX > # ifdef _SEM_VALUE_MAX > # define SEM_VALUE_MAX _SEM_VALUE_MAX > # else > # define SEM_VALUE_MAX INT_MAX > # endif > #endif Does this enable you to submit a patch? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Fri Jun 13 19:18:08 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 12:18:08 -0500 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> <485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> Message-ID: <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com> On Fri, Jun 13, 2008 at 12:14 PM, Guido van Rossum wrote: > On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson > wrote: >> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger wrote: >>>> Nick> def getName(self): >>>> Nick> assert self.__initialized, "Thread.__init__() not called" >>>> Nick> return self.__name >>> >>> Why is __name private to begin with? Any reason for the getters and >>> setters? Why isn't this just an attribute? >> >> In 3.x, it's just an attribute. > > Oh, is it? Where's the PEP with the API redesign? Did I miss some kind > of decision-making process, weighing compatibility concerns against > other issues? meaning that it only has one underscore. They methods still live. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From walter at livinglogic.de Fri Jun 13 19:20:03 2008 From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=) Date: Fri, 13 Jun 2008 19:20:03 +0200 Subject: [Python-Dev] [Python-3000] Betas today - I hope In-Reply-To: <48527597.4090206@livinglogic.de> References: <8D250561-B5D9-4943-AFE3-06EE872E6E11@python.org> <484FE1E9.2080904@egenix.com> <484FEC2C.6070409@livinglogic.de> <485003F8.40803@egenix.com> <485139EA.1000502@livinglogic.de> <48514428.7040001@egenix.com> <48523EA5.2070407@livinglogic.de> <48524899.2030609@egenix.com> <48527597.4090206@livinglogic.de> Message-ID: <4852AC43.1090304@livinglogic.de> Walter D?rwald wrote: > [...] >> Sure, we could do that, but please use a different name, >> e.g. .encodeall() and .decodeall() - .encode() and .decode() >> are already stateles (and so would the new methods be), so >> "stateless" isn't all that meaningful in this context. > > I like the names encodeall/decodeall! > >> We could also add such a check to the PyCodec_Encode() and _Decode() >> functions. They currently do not apply the above check. >> >> In Python, those two functions are exposed as codecs.encode() >> and codecs.decode(). > > This change will probably have to wait for the 2.7 cycle. BTW, what I noticed is that the unicode-internal codec seems to be broken: >>> import codecs >>> codecs.getencoder("unicode-internal")(u"abc") ('a\x00b\x00c\x00', 6) I would have expected it to return: >>> import codecs >>> codecs.getencoder("unicode-internal")(u"abc") ('a\x00b\x00c\x00', 3) Servus, Walter From guido at python.org Fri Jun 13 19:25:41 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 10:25:41 -0700 Subject: [Python-Dev] [Python-3000] [Python-3000-checkins] r64217 - in python/branches/py3k/Lib: bsddb/test/test_associate.py bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtemp Message-ID: On Fri, Jun 13, 2008 at 6:07 AM, Nick Coghlan wrote: > Steve Holden wrote: >> >> Guido van Rossum wrote: >>> >>> It's water under the bridge now, but IMO it was too rash to *remove* >>> the old threading API from Py3k, and doubly rash to do so one day >>> before the beta release. Running up to a release (whether alpha, beta >>> or final) we should practice extra restraint, not rush to get things >>> in right before the deadline. Let's all be more careful the rest of >>> this release cycle! (I think it wasn't just Benjamin who raced to get >>> things in...) >> >> Well, it wouldn't be "adding a new feature" to reinstate the old API for >> beta two, would it, as long as we retain the new one too? It does seem that >> change was a little precipitate. > > Although if we weren't actually planning on removing the old API in 3.0, I'm > a little confused as to why we were adding Py3k warnings to it in 2.6... It all went way too fast for anyone to think about the decision, so I suspect these two issues are directly related. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 13 19:30:59 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 10:30:59 -0700 Subject: [Python-Dev] [Python-3000] [Python-3000-checkins] r64217 - in python/branches/py3k/Lib: bsddb/test/test_associate.py bsddb/test/test_join.py bsddb/test/test_lock.py bsddb/test/test_thread.py idlelib/rpc.py idlelib/run.py socketserver.py test/test_threadedtemp Message-ID: On Thu, Jun 12, 2008 at 11:32 PM, Georg Brandl wrote: > Guido van Rossum schrieb: >> >> It's water under the bridge now, but IMO it was too rash to *remove* >> the old threading API from Py3k, and doubly rash to do so one day >> before the beta release. Running up to a release (whether alpha, beta >> or final) we should practice extra restraint, not rush to get things >> in right before the deadline. Let's all be more careful the rest of >> this release cycle! (I think it wasn't just Benjamin who raced to get >> things in...) > > Also, for any method or module renaming, there is no way around a full > grep through the code base to really catch all uses of the old API. Eh? Even a full grep won't reveal 3rd party uses of the old API. While 3.x is intentionally breaking some things, this is not (and was never meant to be) as a blanket approval to break any API we feel like breaking. Each specific breakage must have a good motivation, e.g. the old API is confusing, or inefficient, or disallows certain uses -- non-PEP-8-conformance alone is not enough! In addition, each specific breakage must be weighed against the cost of updating 3rd party code, and whether a good (or at least fair) automatic conversion can be added to 2to3. > It may be tedious, especially with common names, but such bugs really > must be avoided as they can easily be -- I still could find uses of > old-style threading names, even in the stdlib. (Fixed in r64222.) All the more reason not to rush into API renamings without due process and lots of discussion. Let's err on the side of the status quo -- otherwise we'll never be ready for beta. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jnoller at gmail.com Fri Jun 13 19:36:47 2008 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 13 Jun 2008 13:36:47 -0400 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <18514.40571.3934.984310@montanaro-dyndns-org.local> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> Message-ID: <4222a8490806131036u708a5306lcd88bbd0e49e54aa@mail.gmail.com> I don't know where the windows line endings came from honestly - I'm on mac and linux boxes. Yes, they should be stripped. The solaris issue should be addressed in the setup.py section casing the various sem implementations for the platforms On Fri, Jun 13, 2008 at 12:21 PM, wrote: > In trying to solve a build problem with the multiprocessing code on > Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all > appear to have Windows line endings. Should those maybe be stripped to > conform to the other Python source? > > FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does define > _SEM_VALUE_MAX in sys/params.h. > > .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing': > .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function) > .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once > .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.) > > On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX. I used > a little cpp action to define it: > > #ifndef SEM_VALUE_MAX > # ifdef _SEM_VALUE_MAX > # define SEM_VALUE_MAX _SEM_VALUE_MAX > # else > # define SEM_VALUE_MAX INT_MAX > # endif > #endif > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com > From musiccomposition at gmail.com Fri Jun 13 21:32:41 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 14:32:41 -0500 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: References: <18514.40571.3934.984310@montanaro-dyndns-org.local> Message-ID: <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> On Fri, Jun 13, 2008 at 12:17 PM, Guido van Rossum wrote: > On Fri, Jun 13, 2008 at 9:21 AM, wrote: >> In trying to solve a build problem with the multiprocessing code on >> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all >> appear to have Windows line endings. Should those maybe be stripped to >> conform to the other Python source? > > Ow. definitely. I've converted the line endings in the trunk and py3k. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From guido at python.org Fri Jun 13 22:02:18 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 13:02:18 -0700 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> Message-ID: On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson wrote: > On Fri, Jun 13, 2008 at 12:17 PM, Guido van Rossum wrote: >> On Fri, Jun 13, 2008 at 9:21 AM, wrote: >>> In trying to solve a build problem with the multiprocessing code on >>> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all >>> appear to have Windows line endings. Should those maybe be stripped to >>> conform to the other Python source? >> >> Ow. definitely. > > I've converted the line endings in the trunk and py3k. Thanks! Can we have a post-mortem of this? How did they get in there in the first place without anybody noticing? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Fri Jun 13 22:06:52 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 13 Jun 2008 13:06:52 -0700 Subject: [Python-Dev] Advice on numbers.py implementation of binary mixins. Message-ID: <1984993647E144708D965F1C7AED77C6@RaymondLaptop1> PEP-3141 outlines an approach to writing binary operators to allow the right operand to override the operation if the left operand inherits the operation from the ABC. Here is my first approximation at how to write them for the Integral mixins: class Integral(Rational): def __and__(self, other): if isinstance(other, (type(self), int, long)): # XXX return int(self) & int(other) return NotImplemented def __rand__(self, other): if isinstance(other, Integral): return int(other) & int(self) return NotImplemented The question for the group is what to put on the XXX line. 1. Limit it to type(self). This approach claims the least knowledge about other types and uses their __rand__ methods(). 2. Use type(self), int, and long. This approach says that we know that ints and longs can be reasonably converted to an int; however, it means that any int/long subtype cannot use __rand__ to override the mixin's __and__. 3. Emulate the PEP as closely as possible and accomodate subclassing without changing behaviors. This approach is a bit complex: knowntypes = (set(type(self).__mro__) & set(type(other.__mro__)) - set(type(Integral.__mro__)) | set([int, long])) if isinstance(other, tuple(knowntypes)): I got this by analyzing the possible paths in an inheritance tree: class P(Integral): pass class Q(P): pass class R(Q): pass r = R(3) class S(Q): def __rand__(s,o) ... s = S(6) class T(Integral): def __rand__(s,o) ... t = T(5) With r&t, there is no common ancestor below Integral, so we want Integral.__and__() to return NotImplemented and allow T.__rand__() to do its thing. With r&s, both r and s share Q as a common ancenstor. By using the mixin and not overriding __and__(), Q specifies that __and__() should mean int(r)&int(s) whenever isinstance(r,Q) and isinstance(s,Q). Approaches 1 & 2 don't search the mro for shared ancestors below the level of Integral. So, when evaluating r&s, Integral.__and__() only checks that r is not an instance of S, int or long, and it erroneously returns NotImplemented , leaving s.__rand__() to take over. What do you guys think? Raymond From musiccomposition at gmail.com Fri Jun 13 22:09:09 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 15:09:09 -0500 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> Message-ID: <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> On Fri, Jun 13, 2008 at 3:02 PM, Guido van Rossum wrote: > On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson > wrote: >> I've converted the line endings in the trunk and py3k. > > Thanks! Can we have a post-mortem of this? How did they get in there > in the first place without anybody noticing? svn cat tells me that they were CRLF when I initially imported them. I wonder if this has something to do with the patch utility I used to apply Jesse's patch. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From guido at python.org Fri Jun 13 22:21:48 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 13:21:48 -0700 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> Message-ID: On Fri, Jun 13, 2008 at 1:09 PM, Benjamin Peterson wrote: > On Fri, Jun 13, 2008 at 3:02 PM, Guido van Rossum wrote: >> On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson >> wrote: >>> I've converted the line endings in the trunk and py3k. >> >> Thanks! Can we have a post-mortem of this? How did they get in there >> in the first place without anybody noticing? > > svn cat tells me that they were CRLF when I initially imported them. I > wonder if this has something to do with the patch utility I used to > apply Jesse's patch. I'm guessing Jesse uploaded the patch from a Windows box. It would be good to have something in the toolchain to change this. Perhaps a submit trigger (like the whitespace check) can look for this? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Fri Jun 13 22:23:19 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 15:23:19 -0500 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> Message-ID: <1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com> On Fri, Jun 13, 2008 at 3:21 PM, Guido van Rossum wrote: > > I'm guessing Jesse uploaded the patch from a Windows box. Actually he just told me that he uploaded it from a Mac, so it's my fault. :) > > It would be good to have something in the toolchain to change this. > Perhaps a submit trigger (like the whitespace check) can look for > this? I'll figure out how to do that if you give me shell access. ;) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From guido at python.org Fri Jun 13 22:24:38 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 13:24:38 -0700 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> <1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com> Message-ID: On Fri, Jun 13, 2008 at 1:23 PM, Benjamin Peterson wrote: > On Fri, Jun 13, 2008 at 3:21 PM, Guido van Rossum wrote: >> >> I'm guessing Jesse uploaded the patch from a Windows box. > > Actually he just told me that he uploaded it from a Mac, so it's my fault. :) You're on Windows?! >> It would be good to have something in the toolchain to change this. >> Perhaps a submit trigger (like the whitespace check) can look for >> this? > > I'll figure out how to do that if you give me shell access. ;) Talk to MvL. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Fri Jun 13 22:26:32 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 13 Jun 2008 15:26:32 -0500 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> <1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com> Message-ID: <1afaf6160806131326s50a5e365q96485d325dc57486@mail.gmail.com> On Fri, Jun 13, 2008 at 3:24 PM, Guido van Rossum wrote: > On Fri, Jun 13, 2008 at 1:23 PM, Benjamin Peterson > wrote: >> >> Actually he just told me that he uploaded it from a Mac, so it's my fault. :) > > You're on Windows?! Me!? I cringe at the accusation. No, I just think I have emacs set up wrong. > >>> It would be good to have something in the toolchain to change this. >>> Perhaps a submit trigger (like the whitespace check) can look for >>> this? >> >> I'll figure out how to do that if you give me shell access. ;) > > Talk to MvL. I will do. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From benji at benjiyork.com Fri Jun 13 22:28:29 2008 From: benji at benjiyork.com (Benji York) Date: Fri, 13 Jun 2008 16:28:29 -0400 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> Message-ID: On Fri, Jun 13, 2008 at 4:21 PM, Guido van Rossum wrote: > On Fri, Jun 13, 2008 at 1:09 PM, Benjamin Peterson > wrote: >> On Fri, Jun 13, 2008 at 3:02 PM, Guido van Rossum wrote: >>> On Fri, Jun 13, 2008 at 12:32 PM, Benjamin Peterson >>> wrote: >>>> I've converted the line endings in the trunk and py3k. >>> >>> Thanks! Can we have a post-mortem of this? How did they get in there >>> in the first place without anybody noticing? >> >> svn cat tells me that they were CRLF when I initially imported them. I >> wonder if this has something to do with the patch utility I used to >> apply Jesse's patch. > > I'm guessing Jesse uploaded the patch from a Windows box. > > It would be good to have something in the toolchain to change this. Subversion can be configured to normalize line endings, either manually through properties, or automatically via ~/.subversion/config: [miscellany] enable-auto-props = yes [auto-props] *.c = svn:eol-style=native *.h = svn:eol-style=native etc. -- Benji York From guido at python.org Fri Jun 13 22:28:34 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Jun 2008 13:28:34 -0700 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <1afaf6160806131326s50a5e365q96485d325dc57486@mail.gmail.com> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> <1afaf6160806131323u12ee0567qd9b72d85b0fd5426@mail.gmail.com> <1afaf6160806131326s50a5e365q96485d325dc57486@mail.gmail.com> Message-ID: On Fri, Jun 13, 2008 at 1:26 PM, Benjamin Peterson wrote: > On Fri, Jun 13, 2008 at 3:24 PM, Guido van Rossum wrote: >> On Fri, Jun 13, 2008 at 1:23 PM, Benjamin Peterson >> wrote: >>> >>> Actually he just told me that he uploaded it from a Mac, so it's my fault. :) >> >> You're on Windows?! > > Me!? I cringe at the accusation. No, I just think I have emacs set up wrong. Ow. Are you sure you haven't made this mistake in other situations? Please do try to debug your setup and avoid repeats! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From benji at benjiyork.com Fri Jun 13 22:31:52 2008 From: benji at benjiyork.com (Benji York) Date: Fri, 13 Jun 2008 16:31:52 -0400 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <1afaf6160806131232p7298435ep13ff6b8a304ee4e9@mail.gmail.com> <1afaf6160806131309y4428e354ie2ff9bf662533099@mail.gmail.com> Message-ID: On Fri, Jun 13, 2008 at 4:28 PM, Benji York wrote: > Subversion can be configured to normalize line endings, either manually > through properties, or automatically via ~/.subversion/config: After sending this I though "surely this is in the developer docs", and indeed it is: http://www.python.org/dev/faq/#what-configuration-settings-should-i-use -- Benji York From barry at python.org Sat Jun 14 00:22:42 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Jun 2008 18:22:42 -0400 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins In-Reply-To: References: Message-ID: <0CC2A0C7-BA24-4C1E-B85E-BCA0B771B00B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 12, 2008, at 11:41 PM, Guido van Rossum wrote: > My colleague and SVN developer Ben Sussman-Collins occasionally blogs > about the social side of (mostly open source) software development. He > just posted a new one that struck a chord: > > http://blog.red-bean.com/sussman/?p=96 > > The story's main moral: submit your code for review early and often; > work in a branch if you need to, but don't hide your code from review > in a local repository until it's "perfect". > > Let's all remember this and make sure not to drop "code bombs" on each > other. :-) Very interesting article. I'm short on time and don't want to rant (though I likely will ;), but I whole-heartedly agree with the moral of the story! I disagree with some of the details though. I actually think that a dvcs is /better/ suited to transparency, when used right, and when coupled with a public open code hosting facility. Sure, a lot depends on social engineering, and I agree with Ben that the tools make a difference, I just think that a good dvcs solves more problems than it creates. Also, there are a few things we do at my job that I think contribute significantly and positively to our productivity, quality and sense of shared community code. Briefly: * pre-implementation "calls" - you do not start hacking code until you've discussed your design or approach with at least one other person, either over the phone or on irc (preferably the former). Yes, there are exceptions but they are discouraged. This means that when you actually sit in front of your editor, you have a much better idea of what you are trying to accomplish. * small branches - we have a strict limit on diffs no greater than 800 lines. Yes we have exceptions, but they are rare and pre-arranged. Having such a strict limit really forces you to be disciplined, organized and very effectively diffuses code bombs. * everyone can see (lots of) everyone else's code - this is great because everyone needs some advice or guidance along the way. If you get stuck, you can push a branch and I can pull it and look at it, run it, test it, even modify it and push my own branch for you to see. This is /much/ more effective than trading patches, and I don't see how this could even work without a dvcs. * nothing lands without being reviewed - this is a hard and fast rule, no exceptions. Someone else has to review your code, and most developers are also reviewers (we have a mentoring program to train new reviewers). You get over the fear pretty quickly, and learn /a lot/ both by reviewing and getting reviewed. Coding standards emerge, best practices are established, and overall team productivity goes way up. Small branches are critical to this process, as is our goal of reviewing every branch within 24 hours of its submission. * nothing lands without passing all tests - speaking from experience, this is the one thing I wish Python would adopt! This means the trunk is /always/ releasable and stable. The trade-off is that it can take quite a while for your branch to land once it's been approved, since this process is serialized and is dependent on full test suite execution time. Python's challenge here is that what passes on one platform does not necessarily pass on another. Still, if this week is any indication, passing on /any/ platform would be nice. ;) I'm not saying Python can or should adopt these guidelines. An open source volunteer project is different than a corporate environment, even if the latter is very open-source-y. But it is worthwhile to continually evaluate and improve the process because over time, you definitely improve efficiency in ways that are happily adopted by the majority of the community. - -Barry P.S. I can't leave this without one little plug. Some folks really like the model that a non-dvcs imposes on development, others thrive on the freedom a dvcs gives you. Bazaar is flexible enough to support both models, even at the same time. It's not either-or. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFLzMnEjvBPtnXfVAQLm3QQAptABUXBoWeshMJAvHno1IDMZavL9D2BG q9d3wz8O5u2pbvuZyh44w4fhm7w7fscGvmPygifNbjPTVMeUYQUkunuEfWEIzf6B f6hm1KQm5gi9B4eqSUh3ItIAjGAnkVnCJ8VHeRH/Hff9FZhHqPhtP26LBrecMoho q0g80DrALB8= =J936 -----END PGP SIGNATURE----- From fuzzyman at voidspace.org.uk Sat Jun 14 00:42:52 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 13 Jun 2008 23:42:52 +0100 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? Message-ID: <4852F7EC.6040403@voidspace.org.uk> The Python FAQ has the following entry: 4.26 Why doesn't Python have a "with" statement like some other languages? From: http://www.python.org/doc/faq/general/ Even if the content is still relevant (I know nothing of the use of 'with' outside Python), the entry probably needs at least clarifying now that Python does have a 'with' statement. Michael Foord -- http://www.ironpythoninaction.com/ http://www.theotherdelia.co.uk/ http://www.voidspace.org.uk/ http://www.ironpython.info/ http://www.resolverhacks.net/ From exarkun at divmod.com Sat Jun 14 01:22:07 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 13 Jun 2008 19:22:07 -0400 Subject: [Python-Dev] Interesting blog post by Ben Sussman-Collins In-Reply-To: <0CC2A0C7-BA24-4C1E-B85E-BCA0B771B00B@python.org> Message-ID: <20080613232207.4714.1814579539.divmod.quotient.8878@ohm> On Fri, 13 Jun 2008 18:22:42 -0400, Barry Warsaw wrote: > [snip] > >* small branches - we have a strict limit on diffs no greater than 800 >lines. Yes we have exceptions, but they are rare and pre-arranged. Having >such a strict limit really forces you to be disciplined, organized and very >effectively diffuses code bombs. > >* everyone can see (lots of) everyone else's code - this is great because >everyone needs some advice or guidance along the way. If you get stuck, >you can push a branch and I can pull it and look at it, run it, test it, >even modify it and push my own branch for you to see. This is /much/ more >effective than trading patches, and I don't see how this could even work >without a dvcs. > >* nothing lands without being reviewed - this is a hard and fast rule, no >exceptions. Someone else has to review your code, and most developers are >also reviewers (we have a mentoring program to train new reviewers). You >get over the fear pretty quickly, and learn /a lot/ both by reviewing and >getting reviewed. Coding standards emerge, best practices are established, >and overall team productivity goes way up. Small branches are critical to >this process, as is our goal of reviewing every branch within 24 hours of >its submission. > >* nothing lands without passing all tests - speaking from experience, this >is the one thing I wish Python would adopt! This means the trunk is >/always/ releasable and stable. The trade-off is that it can take quite a >while for your branch to land once it's been approved, since this process >is serialized and is dependent on full test suite execution time. Python's >challenge here is that what passes on one platform does not necessarily >pass on another. Still, if this week is any indication, passing on /any/ >platform would be nice. ;) > >I'm not saying Python can or should adopt these guidelines. An open source >volunteer project is different than a corporate environment, even if the >latter is very open-source-y. But it is worthwhile to continually evaluate >and improve the process because over time, you definitely improve >efficiency in ways that are happily adopted by the majority of the >community. A big +1 on all these points. I can also add that Twisted is developed following many of these rules so it *can* work for open source volunteer projects, if the developers want it to. Jean-Paul From ncoghlan at gmail.com Sat Jun 14 03:23:19 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 14 Jun 2008 11:23:19 +1000 Subject: [Python-Dev] Advice on numbers.py implementation of binary mixins. In-Reply-To: <1984993647E144708D965F1C7AED77C6@RaymondLaptop1> References: <1984993647E144708D965F1C7AED77C6@RaymondLaptop1> Message-ID: <48531D87.8020004@gmail.com> Raymond Hettinger wrote: > The question for the group is what to put on the XXX line. > > 1. Limit it to type(self). This approach claims the least knowledge > about other types and uses their __rand__ methods(). > > 2. Use type(self), int, and long. This approach says that we know that > ints and longs can be reasonably converted to an int; however, it means > that any int/long subtype cannot use __rand__ to override the mixin's > __and__. > > 3. Emulate the PEP as closely as possible and accomodate subclassing > without changing behaviors. This approach is a bit complex: > > knowntypes = (set(type(self).__mro__) > & set(type(other.__mro__)) > - set(type(Integral.__mro__)) > | set([int, long])) > if isinstance(other, tuple(knowntypes)): 4. As 2, but skip int/long subclasses: if type(other) in (int, long) or isinstance(other, type(self)): Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Sat Jun 14 06:31:34 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 13 Jun 2008 23:31:34 -0500 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: References: <18514.40571.3934.984310@montanaro-dyndns-org.local> Message-ID: <18515.18854.711188.716549@montanaro-dyndns-org.local> >>>>> "Guido" == Guido van Rossum writes: Guido> On Fri, Jun 13, 2008 at 9:21 AM, wrote: >> In trying to solve a build problem with the multiprocessing code on >> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all >> appear to have Windows line endings. Should those maybe be stripped to >> conform to the other Python source? Guido> Ow. definitely. Yes. If I have some time Saturday and nobody beats me to it I'll fix this. >> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does >> define _SEM_VALUE_MAX in sys/params.h. >> >> .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing': >> .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function) >> .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once >> .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.) >> >> On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX. I used >> a little cpp action to define it: >> >> #ifndef SEM_VALUE_MAX >> # ifdef _SEM_VALUE_MAX >> # define SEM_VALUE_MAX _SEM_VALUE_MAX >> # else >> # define SEM_VALUE_MAX INT_MAX >> # endif >> #endif Guido> Does this enable you to submit a patch? Sure, I can submit a patch, though while that got me going I have no real idea if that is the correct way to worm around the problem. I sort of think this is something which should be tested for in configure. The code above was just a guess. Skip From cesare at pronto.it Sat Jun 14 08:19:30 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sat, 14 Jun 2008 08:19:30 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <4852F7EC.6040403@voidspace.org.uk> References: <4852F7EC.6040403@voidspace.org.uk> Message-ID: I agree: Python's with statement does have a completely different meaning of the same Pascal & Co 's statement. However, I don't agree with the FAQ on this point. I think that a Pascal-like with statement can be achieved, even with a dynamic language such as Python, and in a simple way. We know that name resolution in Python uses the LEGB rule: first it starts looking at locals, then to enclosing functions, then to globals, and finally to the built-ins. Assignament usually acts on locals (and with global statement, we can can even change globals). A "with" statement (such as Pascal's one) can be implemented adding a new scope resolution on top of LEGB. So, taking the FAQ's example, and using a new keyword (because with is already assigned to a different kind of work): def foo(a): on a: print x the block defined by the "on" statement first must starts looking at the object's namespace. If no symbol was defined inside a, then it follows the traditional LEGB name resolution. Assignament must work on the object's namespace, of course: def foo(a): on a: x += 1 print x will be equivalent to: def foo(a): a.x += 1 print a.x A more complex example (taking "A Simple Hello World Program" in http://docs.python.org/lib/node688.html ) will show a major benefit on using this statement: from Tkinter import * class Application(Frame): def say_hi(self): print "hi there, everyone!" def createWidgets(self): on Button(self): text = "QUIT" fg = "red" command = self.quit pack({"side": "left"}) on Button(self): text = "Hello", command = self.say_hi pack({"side": "left"}) def __init__(self, master=None): Frame.__init__(self, master) self.pack() self.createWidgets() root = Tk() on Application(master=root): mainloop() root.destroy() Notice that adding a new step in scope resolution doesn't necessarily increase execution speed. Taking a look at the above example, accessing to the Button instance's attributes can be as fast as the original example, if not better. Also, the new step in scope resolution will be needed only inside the block defined by the "on" statement. Outside the "on" statement, the usual LEGB method will be used. Obviously "on" statements can be nested, adding each one a new step on scope resolution. IMO such kind of statement can be very useful to make the source code more readable, especially working with GUIs. If somebody knows SmallTalk, it looks similar to the "cascade messages" syntax: (Window new) label: 'Hello'; open Cesare Di Mauro In data 14 giugno 2008 alle ore 00:42:52, Michael Foord ha scritto: > The Python FAQ has the following entry: > > 4.26 Why doesn't Python have a "with" statement like some other languages? > > From: > > http://www.python.org/doc/faq/general/ > > Even if the content is still relevant (I know nothing of the use of > 'with' outside Python), the entry probably needs at least clarifying now > that Python does have a 'with' statement. > > Michael Foord > From martin at v.loewis.de Sat Jun 14 08:55:58 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Jun 2008 08:55:58 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> Message-ID: <48536B7E.30103@v.loewis.de> > the block defined by the "on" statement first must starts looking at > the object's namespace. If no symbol was defined inside a, then it > follows the traditional LEGB name resolution. > > Assignament must work on the object's namespace, of course: This probably belongs to python-ideas or some such, but I don't think this approach can work. People will want to assign to local variables in an "ob" block, and then be surprised that the assignment actually modified their object: def f(L): total = 0 for h in L: on h: more code accessing h's attributes if x: # reads h.x total = total+1 return total People will be surprised that total is always 0 (and that some objects have an attribute total with a value of 1). Likewise on x: for e in L: counts[e] += 1 # modifies x.counts People will be surprised that x also grows an attribute e, as the for loop involves an assignment, which you say goes to the object's namespace, of course. Regards, Martin From steve at pearwood.info Sat Jun 14 10:16:17 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 14 Jun 2008 18:16:17 +1000 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> Message-ID: <200806141816.18526.steve@pearwood.info> On Sat, 14 Jun 2008 04:19:30 pm Cesare Di Mauro wrote: > A "with" statement (such as Pascal's one) can be implemented adding a > new scope resolution on top of LEGB. So, taking the FAQ's example, > and using a new keyword (because with is already assigned to a > different kind of work): > > def foo(a): > on a: > print x > > the block defined by the "on" statement first must starts looking at > the object's namespace. If no symbol was defined inside a, then it > follows the traditional LEGB name resolution. I am a great fan of Pascal-style with blocks, and I'd love Python to get something like them. But I have concluded that they simply aren't practical in Python. Your suggestion sounds reasonable at first glance, but it isn't really. def foo(a): on a: x = round(x, n) return x What does this function do? There's no way of telling, because it depends on what attributes a has. It might be equivalent to any of: a.x = a.round(a.x, a.n) # all attributes a.x = round(a.x, a.n) # built-in function round a.x = round(x, a.n) # global x and attribute n etc. and there's no way of predicting which it will be until you know what attributes a has. Even something like this can fail unexpectedly: on a: import math y = math.sin(x) if object a happens to have an attribute called 'math'. > Assignament must work on the object's namespace, of course: There's no "of course" about it. That rule implies that it is impossible to modify the local namespace inside an 'on block', because the object's namespace will always take priority: def foo(a): y = 1 on a: y = x + 1 # creates a new a.y attribute! return y # always returns 1 Perhaps you want this behaviour, but I don't think it is either obvious or desirable. Pascal doesn't have this problem because you always explicitly define your local variables. -- Steven From hodgestar+pythondev at gmail.com Sat Jun 14 11:03:09 2008 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Sat, 14 Jun 2008 11:03:09 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <200806141816.18526.steve@pearwood.info> References: <4852F7EC.6040403@voidspace.org.uk> <200806141816.18526.steve@pearwood.info> Message-ID: On Sat, Jun 14, 2008 at 10:16 AM, Steven D'Aprano wrote: > I am a great fan of Pascal-style with blocks, and I'd love Python to get > something like them. But I have concluded that they simply aren't > practical in Python. Your suggestion sounds reasonable at first glance, > but it isn't really. > > def foo(a): > on a: > x = round(x, n) > return x > > What does this function do? There's no way of telling, because it > depends on what attributes a has. It might be equivalent to any of: After having read all the examples in the thread so far, let's never add anything like this to Python ever. It breaks so many of the "import this" guidelines it's scary. The biggest loss is readability. Sure in a statically typed language it might be possible for the compiler figure out which things are attributes and which are not, but I have to be able to do the same. And lets not even think about what happens when some adds a new attribute to a and code using "on a" suddenly starts silently breaking. Schiavo Simon From musiccomposition at gmail.com Sat Jun 14 14:33:27 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 14 Jun 2008 07:33:27 -0500 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <18515.18854.711188.716549@montanaro-dyndns-org.local> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <18515.18854.711188.716549@montanaro-dyndns-org.local> Message-ID: <1afaf6160806140533w6efa9adarcb854ff3a3318556@mail.gmail.com> On Fri, Jun 13, 2008 at 11:31 PM, wrote: >>>>>> "Guido" == Guido van Rossum writes: > > Guido> Ow. definitely. > > Yes. If I have some time Saturday and nobody beats me to it I'll fix this. Sorry, it seems I've beaten you. :) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From pje at telecommunity.com Sat Jun 14 16:43:46 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 14 Jun 2008 10:43:46 -0400 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> Message-ID: <20080614144312.EA7C63A405F@sparrow.telecommunity.com> At 08:19 AM 6/14/2008 +0200, Cesare Di Mauro wrote: >Assignament must work on the object's namespace, of course: > >def foo(a): > on a: > x += 1 > print x > will be equivalent to: > >def foo(a): > a.x += 1 > print a.x Er, you need a syntactic disambiguation here to distinguish attributes from locals or globals: def foo(a): on a: .x += 1 print .x Otherwise, this leads to all sorts of craziness. You'd also have to restrict what could be referenced in a nested "on" block, in order to avoid further ambiguities. From cesare at pronto.it Sat Jun 14 21:31:23 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sat, 14 Jun 2008 21:31:23 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <48536B7E.30103@v.loewis.de> References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: In data 14 giugno 2008 alle ore 08:55:58, Martin v. L?wis ha scritto: > This probably belongs to python-ideas or some such, but I don't > think this approach can work. People will want to assign to local > variables in an "ob" block, and then be surprised that the > assignment actually modified their object: > > def f(L): > total = 0 > for h in L: > on h: > more code accessing h's attributes > if x: # reads h.x > total = total+1 > return total > > People will be surprised that total is always 0 (and that > some objects have an attribute total with a value of 1). > Likewise > > on x: > for e in L: > counts[e] += 1 # modifies x.counts > > People will be surprised that x also grows an attribute > e, as the for loop involves an assignment, which you say > goes to the object's namespace, of course. > > Regards, > Martin I think there can be two solutions to the local variable assignament. The first is to declare them with an "local" instruction, similar to the global one. So we can have: def f(L): total = 0 for h in L: on h: local total more code accessing h's attributes if x: # reads h.x total = total+1 return total on x: local e for e in L: counts[e] += 1 # modifies x.counts The second solution is to leave assignament to the local namespace, and let assignament to object's attributes happens. So your examples will work without changes, and the object never "inherits" new attributes. Also, taking the Tk example that I used, it can be changed in the following way: on Button(self) as b: b.text = "QUIT" b.fg = "red" b.command = self.quit pack({"side": "left"}) on Button(self) as b: b.text = "Hello" b.command = self.say_hi pack({"side": "left"}) Using a syntax which reseambles the with one. Cesare Di Mauro From g.brandl at gmx.net Sat Jun 14 21:33:40 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 14 Jun 2008 21:33:40 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: Cesare Di Mauro schrieb: > Also, taking the Tk example that I used, it can be changed in the following way: > > on Button(self) as b: > b.text = "QUIT" > b.fg = "red" > b.command = self.quit > > pack({"side": "left"}) > > on Button(self) as b: > b.text = "Hello" > b.command = self.say_hi > > pack({"side": "left"}) > > Using a syntax which reseambles the with one. So what is the advantage to b = Button(self) b.text = "QUIT" b.fg = "red" b.command = self.quit ? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From jnoller at gmail.com Sat Jun 14 21:38:53 2008 From: jnoller at gmail.com (Jesse Noller) Date: Sat, 14 Jun 2008 15:38:53 -0400 Subject: [Python-Dev] multiprocessing source not "Unix clean" In-Reply-To: <18515.18854.711188.716549@montanaro-dyndns-org.local> References: <18514.40571.3934.984310@montanaro-dyndns-org.local> <18515.18854.711188.716549@montanaro-dyndns-org.local> Message-ID: <4222a8490806141238x31ee38abk17f20e0ecaa052b8@mail.gmail.com> On Sat, Jun 14, 2008 at 12:31 AM, wrote: >>>>>> "Guido" == Guido van Rossum writes: > > Guido> On Fri, Jun 13, 2008 at 9:21 AM, wrote: > >> In trying to solve a build problem with the multiprocessing code on > >> Solaris10 I visited multiprocessing.c in XEmacs and noticed the files all > >> appear to have Windows line endings. Should those maybe be stripped to > >> conform to the other Python source? > > Guido> Ow. definitely. > > Yes. If I have some time Saturday and nobody beats me to it I'll fix this. > > >> FWIW, it appears that Solaris doesn't define SEM_VALUE_MAX but does > >> define _SEM_VALUE_MAX in sys/params.h. > >> > >> .../Modules/_multiprocessing/multiprocessing.c: In function 'init_multiprocessing': > >> .../Modules/_multiprocessing/multiprocessing.c:253: error: 'SEM_VALUE_MAX' undeclared (first use in this function) > >> .../Modules/_multiprocessing/multiprocessing.c:253: error: (Each undeclared identifier is reported only once > >> .../Modules/_multiprocessing/multiprocessing.c:253: error: for each function it appears in.) > >> > >> On Windows the author simple #defines SEM_VALUE_MAX to be LONG_MAX. I used > >> a little cpp action to define it: > >> > >> #ifndef SEM_VALUE_MAX > >> # ifdef _SEM_VALUE_MAX > >> # define SEM_VALUE_MAX _SEM_VALUE_MAX > >> # else > >> # define SEM_VALUE_MAX INT_MAX > >> # endif > >> #endif > > Guido> Does this enable you to submit a patch? > > Sure, I can submit a patch, though while that got me going I have no real > idea if that is the correct way to worm around the problem. I sort of think > this is something which should be tested for in configure. The code above > was just a guess. > > Skip I filed http://bugs.python.org/issue3110 for the solaris issue. The best way to fix this is to patch setup.py to in the multiprocessing section to account for the solaris issue. -jesse From cesare at pronto.it Sat Jun 14 21:53:14 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sat, 14 Jun 2008 21:53:14 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <200806141816.18526.steve@pearwood.info> Message-ID: In data 14 giugno 2008 alle ore 11:03:09, Simon Cross ha scritto: > After having read all the examples in the thread so far, let's never > add anything like this to Python ever. It breaks so many of the > "import this" guidelines it's scary. The biggest loss is readability. > Sure in a statically typed language it might be possible for the > compiler figure out which things are attributes and which are not, but > I have to be able to do the same. And lets not even think about what > happens when some adds a new attribute to a and code using "on a" > suddenly starts silently breaking. > > Schiavo > Simon It was just a "rough" idea, and I have suggested two solutions to make it better. Obviously there can be situations where the new instruction will be useful, and other where it will not. In my experience I found so many times coding "patterns" where I work "inside" an object, calling many methods and accessing its variabiles (primarily reading them), and the new instruction which I proposed to have in Python can make this kind of code much easier to write and managed (especially with GUIs, there are many common "patterns" that can benefit from it). Also, I think that the code can be more readable. Just take a look at the example I reported: don't you find it easier to read? Cesare Di Mauro From cesare at pronto.it Sat Jun 14 22:05:56 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sat, 14 Jun 2008 22:05:56 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: In data 14 giugno 2008 alle ore 21:33:40, Georg Brandl ha scritto: > So what is the advantage to > > b = Button(self) > b.text = "QUIT" > b.fg = "red" > b.command = self.quit > > ? > > Georg In this example there are many assignaments, so there aren't many advantages. But this t = ScrolledText.ScrolledText(master, width=60, height=37) t.insert(Tkinter.END, self.log.getText()) t.configure(state=Tkinter.DISABLED) t.see(Tkinter.END) t.pack(fill=Tkinter.BOTH) can look like: on Tkinter: on ScrolledText.ScrolledText(master, width=60, height=37): insert(END, self.log.getText()) configure(state=DISABLED) see(END) pack(fill=BOTH) Cesare Di Mauro From ggpolo at gmail.com Sat Jun 14 22:09:28 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Sat, 14 Jun 2008 17:09:28 -0300 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: On Sat, Jun 14, 2008 at 5:05 PM, Cesare Di Mauro wrote: > In data 14 giugno 2008 alle ore 21:33:40, Georg Brandl ha scritto: > >> So what is the advantage to >> >> b = Button(self) >> b.text = "QUIT" >> b.fg = "red" >> b.command = self.quit >> >> ? >> >> Georg > > In this example there are many assignaments, so there aren't many advantages. > > But this > > t = ScrolledText.ScrolledText(master, width=60, height=37) > t.insert(Tkinter.END, self.log.getText()) > t.configure(state=Tkinter.DISABLED) > t.see(Tkinter.END) > t.pack(fill=Tkinter.BOTH) > > can look like: > > on Tkinter: > on ScrolledText.ScrolledText(master, width=60, height=37): > insert(END, self.log.getText()) > configure(state=DISABLED) > see(END) > pack(fill=BOTH) > Then you have to start guessing from where these names came from. > Cesare Di Mauro > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com > -- -- Guilherme H. Polo Goncalves From aleaxit at gmail.com Sat Jun 14 22:09:25 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 14 Jun 2008 13:09:25 -0700 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <20080614144312.EA7C63A405F@sparrow.telecommunity.com> References: <4852F7EC.6040403@voidspace.org.uk> <20080614144312.EA7C63A405F@sparrow.telecommunity.com> Message-ID: Yep. Javascript's totally botched and unusable 'with' statement is an excellent warning of what horrors that kind of thing can wreck in a dynamic language unless carefully designed. I also agree that .foo is a good disambiguation syntax -- unfortunately it doesn't disambiguate among *nested* with-blocks; maybe one should use multiple dots for the purpose, i.e. on a: on b: .foo = 23 would assign to b.foo, while on a: on b: ..foo = 23 would assign to a.foo ...? Disambiguation might be required only for assignment, maybe -- with all other accesses to identifiers without leading dots doing nested lookup (so that e.g. "print foo" might emit b.foo if present, else a.foo if present, else local foo if present, else global foo if present). I'm not sure I like this whole arrangement, but maybe overall I do, perhaps +0 ... Alex On Sat, Jun 14, 2008 at 7:43 AM, Phillip J. Eby wrote: > At 08:19 AM 6/14/2008 +0200, Cesare Di Mauro wrote: >> >> Assignament must work on the object's namespace, of course: >> >> def foo(a): >> on a: >> x += 1 >> print x >> will be equivalent to: >> >> def foo(a): >> a.x += 1 >> print a.x > > Er, you need a syntactic disambiguation here to distinguish attributes from > locals or globals: > > def foo(a): > on a: > .x += 1 > print .x > > Otherwise, this leads to all sorts of craziness. You'd also have to > restrict what could be referenced in a nested "on" block, in order to avoid > further ambiguities. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > From martin at v.loewis.de Sat Jun 14 22:09:56 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Jun 2008 22:09:56 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <200806141816.18526.steve@pearwood.info> Message-ID: <48542594.4000107@v.loewis.de> > Just take a look at the example I reported: don't you find it easier to read? No, not at all. I actually *like* having to specify the object on a method call. I find C++/Java code very hard to read which invokes a method without specifying this. infront of the call - in particular when you are unfamiliar with the class hierarchy you are looking at. (due to late binding, it is still difficult to find out what specific method is being called, but atleast you can be certain it's a method, and you might even have a clue what the dynamic type of the object is). So I think the FAQ should continue to claim that Python cannot grow a Pascal-like with statement. Regards, Martin From cesare at pronto.it Sat Jun 14 22:19:49 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sat, 14 Jun 2008 22:19:49 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: In data 14 giugno 2008 alle ore 22:09:28, Guilherme Polo ha scritto: >> on Tkinter: >> on ScrolledText.ScrolledText(master, width=60, height=37): >> insert(END, self.log.getText()) >> configure(state=DISABLED) >> see(END) >> pack(fill=BOTH) >> > > Then you have to start guessing from where these names came from. The same happens with: from Tkinter import * which is a fair common instruction... P.S. Object hierarchy can be a knightmare with or without the new instruction: it's a matter of programmer's knowledge base. Cesare Di Mauro From cesare at pronto.it Sat Jun 14 22:20:44 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sat, 14 Jun 2008 22:20:44 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <48542594.4000107@v.loewis.de> References: <4852F7EC.6040403@voidspace.org.uk> <200806141816.18526.steve@pearwood.info> <48542594.4000107@v.loewis.de> Message-ID: In data 14 giugno 2008 alle ore 22:09:56, Martin v. L?wis ha scritto: > No, not at all. I actually *like* having to specify the object on a > method call. I find C++/Java code very hard to read which invokes a > method without specifying this. infront of the call - in particular > when you are unfamiliar with the class hierarchy you are looking at. > (due to late binding, it is still difficult to find out what specific > method is being called, but atleast you can be certain it's a method, > and you might even have a clue what the dynamic type of the object > is). > > So I think the FAQ should continue to claim that Python cannot grow > a Pascal-like with statement. > > Regards, > Martin OK. It was just an idea... Thanks Cesare Di Mauro From hodgestar at gmail.com Sat Jun 14 22:22:22 2008 From: hodgestar at gmail.com (Simon Cross) Date: Sat, 14 Jun 2008 22:22:22 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <200806141816.18526.steve@pearwood.info> Message-ID: On Sat, Jun 14, 2008 at 9:53 PM, Cesare Di Mauro wrote: > Just take a look at the example I reported: don't you find it easier to read? Sure, it's perhaps a bit easier on the eyes, but readability includes understanding what's the code does. Let's take an example: >on Tkinter: > on ScrolledText.ScrolledText(master, width=60, height=37): > insert(END, self.log.getText()) > configure(state=DISABLED) > see(END) > pack(fill=BOTH) Is END an attribute of ScrolledText? Tkinter? Neither? Both? Who knows. If we allow assignments it's even more painful. Throwing random junk into the current scope isn't my idea of fun. Schiavo Simon From hodgestar at gmail.com Sat Jun 14 22:24:25 2008 From: hodgestar at gmail.com (Simon Cross) Date: Sat, 14 Jun 2008 22:24:25 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: On Sat, Jun 14, 2008 at 10:19 PM, Cesare Di Mauro wrote: > from Tkinter import * I discourage this too. :) Schiavo Simob From barry at python.org Sat Jun 14 23:18:34 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 14 Jun 2008 17:18:34 -0400 Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18 In-Reply-To: References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> Message-ID: <4F7E8331-D77E-4449-B8DD-305F0D3A50AB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 13, 2008, at 9:54 AM, Facundo Batista wrote: > 2008/6/13 Barry Warsaw : > >> My proposal is this: I will spin another release this coming >> Wednesday, June >> 18. If we can get both the 2.6 and 3.0 buildbots green by then, >> and close >> out all remaining release critical bugs, then Wednesday's release >> will be >> beta 1. In that case, I will update PEP 361 and push the entire >> schedule >> back by 2 weeks to account for our slippage. > > Next weekend, 21 and 22, it will be the Python Bug days. > > Maybe it's worth to delay the betas one week to include this work? I think I'd rather stick to a one week delay, otherwise I think we're really talking about pushing the whole schedule back a month. I think we should focus right now on getting the buildbots as green as possible (yay for some 3.0 greenness! thanks everyone!) and closing all the release critical bugs. I think this will actually free us up a bit more on bug day to concentrate on fixing other issues. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFQ1qnEjvBPtnXfVAQI13wP/Xj+my8YUNHuLG8i7pggHNC1Vn7/j5K7H 4l5vT9EMgkT6OHXKvrVPxOJSbm7TkoHd4k3M524IYwnKigFAVH/e9WmgTChp4hPv 7UsF9MGKcSsgnjB2LpWvbV9A6lSRLsNLjuLqqJSQgS7TvrD+0Omd91RM9twmV9io BndCSNAALyE= =g3RH -----END PGP SIGNATURE----- From musiccomposition at gmail.com Sat Jun 14 23:22:35 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 14 Jun 2008 16:22:35 -0500 Subject: [Python-Dev] [Python-3000] First betas (perhaps) on June 18 In-Reply-To: <4F7E8331-D77E-4449-B8DD-305F0D3A50AB@python.org> References: <26D65059-EDD6-4C90-98FE-111B5135E725@python.org> <4F7E8331-D77E-4449-B8DD-305F0D3A50AB@python.org> Message-ID: <1afaf6160806141422y14bfd5dcn1bc900bf984cbdf2@mail.gmail.com> On Sat, Jun 14, 2008 at 4:18 PM, Barry Warsaw wrote: > On Jun 13, 2008, at 9:54 AM, Facundo Batista wrote: >> Next weekend, 21 and 22, it will be the Python Bug days. >> >> Maybe it's worth to delay the betas one week to include this work? > > I think I'd rather stick to a one week delay, otherwise I think we're really > talking about pushing the whole schedule back a month. I think we should > focus right now on getting the buildbots as green as possible (yay for some > 3.0 greenness! thanks everyone!) and closing all the release critical bugs. > I think this will actually free us up a bit more on bug day to concentrate > on fixing other issues. I kinda hate to do this to you, but I had to file another release blocker: #3114 -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From aleaxit at gmail.com Sun Jun 15 01:12:47 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 14 Jun 2008 16:12:47 -0700 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: On Sat, Jun 14, 2008 at 1:24 PM, Simon Cross wrote: > On Sat, Jun 14, 2008 at 10:19 PM, Cesare Di Mauro wrote: >> from Tkinter import * > > I discourage this too. :) And so do I, every single time I teach Python (which is pretty often, even though many of those cases don't make it into YouTube videos). I don't need to rant against it within Google, because Google's Python Style Guide already absolutely forbids it, fortunately;-). Alex From armin.ronacher at active-4.com Sun Jun 15 01:39:11 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sat, 14 Jun 2008 23:39:11 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections Message-ID: Hi, I noticed lately that quite a few projects are implementing their own subclasses of `dict` that retain the order of the key/value pairs. However half of the implementations I came across are not implementing the whole dict interface which leads to weird bugs, also the performance of a Python implementation is not that great. To fight that problem I want to proposed a new class in "collections" called odict which is a dict that keeps the items sorted, similar to a PHP array. The interface would be fully compatible with dict and implemented as dict subclass. Updates to existing keys does not change the order of a key but new keys are inserted at the end. Additionally it would support slicing where a list of key, value tuples is returned and sort/reverse/index methods that work like their list equivalents. Index based lookup could work via odict.byindex(). An implementation of that exists as part of the ordereddict implementation which however goes beyond that and is pretty much a fork of the python dict[1]. Some reasons why ordered dicts are a useful feature: - in XML/HTML processing it's often desired to keep the attributes of an tag ordered during processing. So that input ordering is the same as the output ordering. - Form data transmitted via HTTP is usually ordered by the position of the input/textarea/select field in the HTML document. That information is currently lost in most Python web applications / frameworks. - Eaiser transition of code from Ruby/PHP which have sorted associative arrays / hashmaps. - Having an ordered dict in the standard library would allow other libraries support them. For example a PHP serializer could return odicts rather then dicts which drops the ordering information. XML libraries such as etree could add support for it when creating elements or return attribute dicts. Regards, Armin [1]: http://www.xs4all.nl/~anthon/Python/ordereddict/ From fuzzyman at voidspace.org.uk Sun Jun 15 01:44:24 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 15 Jun 2008 00:44:24 +0100 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <485457D8.2090205@voidspace.org.uk> Armin Ronacher wrote: > Hi, > > I noticed lately that quite a few projects are implementing their own > subclasses of `dict` that retain the order of the key/value pairs. > However half of the implementations I came across are not implementing > the whole dict interface which leads to weird bugs, also the performance > of a Python implementation is not that great. > > I'm +1 - but this proposal has been made many times before and people always argue about what features are needed or desirable. :-( Michael Foord > To fight that problem I want to proposed a new class in "collections" > called odict which is a dict that keeps the items sorted, similar to > a PHP array. > > The interface would be fully compatible with dict and implemented as > dict subclass. Updates to existing keys does not change the order of > a key but new keys are inserted at the end. > > Additionally it would support slicing where a list of key, value tuples > is returned and sort/reverse/index methods that work like their list > equivalents. Index based lookup could work via odict.byindex(). > > An implementation of that exists as part of the ordereddict implementation > which however goes beyond that and is pretty much a fork of the python > dict[1]. > > Some reasons why ordered dicts are a useful feature: > > - in XML/HTML processing it's often desired to keep the attributes of > an tag ordered during processing. So that input ordering is the > same as the output ordering. > > - Form data transmitted via HTTP is usually ordered by the position > of the input/textarea/select field in the HTML document. That > information is currently lost in most Python web applications / > frameworks. > > - Eaiser transition of code from Ruby/PHP which have sorted > associative arrays / hashmaps. > > - Having an ordered dict in the standard library would allow other > libraries support them. For example a PHP serializer could return > odicts rather then dicts which drops the ordering information. > XML libraries such as etree could add support for it when creating > elements or return attribute dicts. > > Regards, > Armin > > > [1]: http://www.xs4all.nl/~anthon/Python/ordereddict/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.theotherdelia.co.uk/ http://www.voidspace.org.uk/ http://www.ironpython.info/ http://www.resolverhacks.net/ From greg.ewing at canterbury.ac.nz Sun Jun 15 01:42:14 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 15 Jun 2008 11:42:14 +1200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> Message-ID: <48545756.9060901@canterbury.ac.nz> Cesare Di Mauro wrote: > However, I don't agree with the FAQ on this point. I think that a > Pascal-like with statement can be achieved, even with a dynamic > language such as Python, and in a simple way. It's not so much a matter of whether it *can* be done, but whether there's any substantial need for it. I don't believe there is in Python, because you can always put a reference to the target object into a succinctly-named local and access it through that. You can't always do that in Pascal, because it provides no way of obtaining a pointer a record that wasn't heap-allocated. -- Greg From nd at perlig.de Sun Jun 15 01:57:23 2008 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Sun, 15 Jun 2008 01:57:23 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <200806150157.23985@news.perlig.de> * Armin Ronacher wrote: > Some reasons why ordered dicts are a useful feature: > > - in XML/HTML processing it's often desired to keep the attributes of > an tag ordered during processing. So that input ordering is the > same as the output ordering. > > - Form data transmitted via HTTP is usually ordered by the position > of the input/textarea/select field in the HTML document. That > information is currently lost in most Python web applications / > frameworks. > > - Eaiser transition of code from Ruby/PHP which have sorted > associative arrays / hashmaps. > > - Having an ordered dict in the standard library would allow other > libraries support them. For example a PHP serializer could return > odicts rather then dicts which drops the ordering information. > XML libraries such as etree could add support for it when creating > elements or return attribute dicts. I find this collection of cases pretty weak as an argument for implementing that in the stdlib. A lot of special purpose types would fit into such reasoning, but do you want to have all of them maintained here? nd -- Da f?llt mir ein, wieso gibt es eigentlich in Unicode kein "i" mit einem Herzchen als T?pfelchen? Das w?r sooo s??ss! -- Bj?rn H?hrmann in darw From marek at xivilization.net Sun Jun 15 01:53:39 2008 From: marek at xivilization.net (Marek Kubica) Date: Sat, 14 Jun 2008 23:53:39 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: Message-ID: Hi, Armin Ronacher active-4.com> writes: > To fight that problem I want to proposed a new class in "collections" > called odict which is a dict that keeps the items sorted, similar to > a PHP array. I'm also +1 on this. I've used the implementation you mentioned in a compiler project of mine and found it to be quite useful. It is hard for me to mention any other uses but there definitely are many occasions where people need them and either go for (key, value)-tuples or use some lib which does only provide this single data type. I am pretty optimistic that the addition will find its usecases, once it is in the stdlib. regards, Marek From greg.ewing at canterbury.ac.nz Sun Jun 15 02:24:43 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 15 Jun 2008 12:24:43 +1200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> Message-ID: <4854614B.9070903@canterbury.ac.nz> Cesare Di Mauro wrote: > The same happens with: > > from Tkinter import * > > which is a fair common instruction... ...and which should *not* be used in most cases, for the same reason. All those tutorials that start out with 'from something import *' are doing a lot of harm to the impressionable minds of new programmers, IMO. -- Greg From jimjjewett at gmail.com Sun Jun 15 03:54:36 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Sat, 14 Jun 2008 21:54:36 -0400 Subject: [Python-Dev] Advice on numbers.py implementation of binary mixins. Message-ID: Raymond Hettinger wrote: > PEP-3141 outlines an approach to writing binary > operators to allow the right operand to override > the operation if the left operand inherits the > operation from the ABC. > Here is my first approximation at how to write > them for the Integral mixins: > class Integral(Rational): > def __and__(self, other): > if isinstance(other, (type(self), int, long)): # XXX > return int(self) & int(other) I think for this mixin, it doesn't matter whether other is an Integral instance; it matter whether it is has a more specific solution. So instead of checking whether isinstance, check whether its __rand__ method is Integral.__rand__. I think you also may want to guard against incomplete right-hand operations, by doing something like replacing the simple > return NotImplemented with try: val = other.__rand__(self) if val is not NotImplemented: return val except (TypeError, AttributeError): pass # Use the generic fallback after all return int(self) & int(other) From jimjjewett at gmail.com Sun Jun 15 04:06:31 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Sat, 14 Jun 2008 22:06:31 -0400 Subject: [Python-Dev] Proposal: add odict to collections Message-ID: The odict (as proposed here, ordered on time of key insertion) looks like a close match to the dlict needed by some of the optimization proposals. http://python.org/dev/peps/pep-0267/ -jJ From guido at python.org Sun Jun 15 05:26:58 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 14 Jun 2008 20:26:58 -0700 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <200806150157.23985@news.perlig.de> References: <200806150157.23985@news.perlig.de> Message-ID: On Sat, Jun 14, 2008 at 4:57 PM, Andr? Malo wrote: > * Armin Ronacher wrote: > >> Some reasons why ordered dicts are a useful feature: >> >> - in XML/HTML processing it's often desired to keep the attributes of >> an tag ordered during processing. So that input ordering is the >> same as the output ordering. >> >> - Form data transmitted via HTTP is usually ordered by the position >> of the input/textarea/select field in the HTML document. That >> information is currently lost in most Python web applications / >> frameworks. >> >> - Eaiser transition of code from Ruby/PHP which have sorted >> associative arrays / hashmaps. >> >> - Having an ordered dict in the standard library would allow other >> libraries support them. For example a PHP serializer could return >> odicts rather then dicts which drops the ordering information. >> XML libraries such as etree could add support for it when creating >> elements or return attribute dicts. > > I find this collection of cases pretty weak as an argument for implementing > that in the stdlib. A lot of special purpose types would fit into such > reasoning, but do you want to have all of them maintained here? No, but an ordered dict happens to be a *very* common thing to need, for a variety of reasons. So I'm +0.5 on adding this to the collections module. However someone needs to contribute working code. It would also be useful to verify that it actually fulfills the needs of some actual use case. Perhaps looking at how Django uses its version would be helpful. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Sun Jun 15 05:42:24 2008 From: talin at acm.org (Talin) Date: Sat, 14 Jun 2008 20:42:24 -0700 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <485457D8.2090205@voidspace.org.uk> References: <485457D8.2090205@voidspace.org.uk> Message-ID: <48548FA0.5020009@acm.org> Michael Foord wrote: > Armin Ronacher wrote: >> Hi, >> >> I noticed lately that quite a few projects are implementing their own >> subclasses of `dict` that retain the order of the key/value pairs. >> However half of the implementations I came across are not implementing >> the whole dict interface which leads to weird bugs, also the performance >> of a Python implementation is not that great. >> >> > > I'm +1 - but this proposal has been made many times before and people > always argue about what features are needed or desirable. :-( There's been a lot of controversy/confusion about ordered dicts. One of the sources of confusion is that people mean different things when they use the term "ordered dict": In some cases, the term is used to mean a dictionary that remembers the order of insertions, and in other cases it is used to mean a sorted dict, i.e. an associative data structure in which the entries are kept sorted. (And I'm not sure that those are the only two possibilities.) I would be more in favor of the idea if we could come up with a less ambiguous naming scheme. -- Talin From bjourne at gmail.com Sun Jun 15 06:37:11 2008 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Sun, 15 Jun 2008 04:37:11 +0000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <740c3aec0806142137r79bb8a84k411155d75b2df592@mail.gmail.com> On Sat, Jun 14, 2008 at 11:39 PM, Armin Ronacher wrote: > Some reasons why ordered dicts are a useful feature: And for metaclasses or for LRU caches or for removing duplicates in a list while maintaining order. I think that the name ordereddict would be more readable though, and match the existing defaultdict class in the collections module. -- mvh Bj?rn From hasan.diwan at gmail.com Sun Jun 15 06:38:01 2008 From: hasan.diwan at gmail.com (Hasan Diwan) Date: Sat, 14 Jun 2008 21:38:01 -0700 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <48548FA0.5020009@acm.org> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> Message-ID: <2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com> 2008/6/14 Talin : > There's been a lot of controversy/confusion about ordered dicts. One of the > sources of confusion is that people mean different things when they use the > term "ordered dict": In some cases, the term is used to mean a dictionary > that remembers the order of insertions, and in other cases it is used to > mean a sorted dict, i.e. an associative data structure in which the entries > are kept sorted. (And I'm not sure that those are the only two > possibilities.) Have the comparison function passed in as a parameter then, if it's None, then have it maintain the order of insertion? Something like: def __init__(self, cmpfunc = None): self.dict = dict() def __getattr__(self, key): try: return self.key -- Cheers, Hasan Diwan From martin at v.loewis.de Sun Jun 15 06:55:22 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jun 2008 06:55:22 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com> Message-ID: <4854A0BA.4070801@v.loewis.de> > Have the comparison function passed in as a parameter then, if it's > None, then have it maintain the order of insertion? No. This would contribute to the confusion, not resolve it. If it's called "ordered dictionary", it shouldn't do sorting at all. If it does sorting, it should be called "sorted dictionary", and mandate a comparison function (which might default to cmp). Regards, Martin From ncoghlan at gmail.com Sun Jun 15 07:14:41 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Jun 2008 15:14:41 +1000 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> Message-ID: <4854A541.5080101@gmail.com> Cesare Di Mauro wrote: > I agree: Python's with statement does have a completely different > meaning of the same Pascal & Co 's statement. > > However, I don't agree with the FAQ on this point. I think that a > Pascal-like with statement can be achieved, even with a dynamic > language such as Python, and in a simple way. It isn't that a Pascal-with-style statement can't be achieved, it's that it is pointless syntactic sugar in Python. Just use o = long_complicated_object_name instead. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Jun 15 07:20:32 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Jun 2008 15:20:32 +1000 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <4854614B.9070903@canterbury.ac.nz> References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> <4854614B.9070903@canterbury.ac.nz> Message-ID: <4854A6A0.3030302@gmail.com> Greg Ewing wrote: > Cesare Di Mauro wrote: > >> The same happens with: >> >> from Tkinter import * >> >> which is a fair common instruction... > > ...and which should *not* be used in most cases, for > the same reason. > > All those tutorials that start out with 'from something > import *' are doing a lot of harm to the impressionable > minds of new programmers, IMO. > Yeah, the only remotely legitimate usage of it that I am aware of is for modules with a hybrid implementation where the public Python module does a "from _native_module import *" to get the rest of the implementation. And even that is somewhat arguable. To go back to Cesare's most recent example: t = ScrolledText.ScrolledText(master, width=60, height=37) t.insert(Tkinter.END, self.log.getText()) t.configure(state=Tkinter.DISABLED) t.see(Tkinter.END) t.pack(fill=Tkinter.BOTH) can look like: tk = Tkinter: st = ScrolledText.ScrolledText(master, width=60, height=37): st.insert(tk.END, self.log.getText()) st.configure(state=tk.DISABLED) st.see(tk.END) st.pack(fill=tk.BOTH) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Jun 15 07:25:31 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Jun 2008 15:25:31 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <48548FA0.5020009@acm.org> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> Message-ID: <4854A7CB.7050508@gmail.com> Talin wrote: > Michael Foord wrote: >> Armin Ronacher wrote: >>> Hi, >>> >>> I noticed lately that quite a few projects are implementing their own >>> subclasses of `dict` that retain the order of the key/value pairs. >>> However half of the implementations I came across are not implementing >>> the whole dict interface which leads to weird bugs, also the performance >>> of a Python implementation is not that great. >>> >>> >> >> I'm +1 - but this proposal has been made many times before and people >> always argue about what features are needed or desirable. :-( > > There's been a lot of controversy/confusion about ordered dicts. One of > the sources of confusion is that people mean different things when they > use the term "ordered dict": In some cases, the term is used to mean a > dictionary that remembers the order of insertions, and in other cases it > is used to mean a sorted dict, i.e. an associative data structure in > which the entries are kept sorted. (And I'm not sure that those are the > only two possibilities.) > > I would be more in favor of the idea if we could come up with a less > ambiguous naming scheme. I think Armin's proposal addresses this nicely by the analogy to lists: the ordered dict is in key insertion order by default, but you can invoke odict.sort() to sort it instead. Given the synergy with the Py3k metaclass enhancements, I believe this would be a good thing to have. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Jun 15 07:33:49 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Jun 2008 15:33:49 +1000 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> <485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com> Message-ID: <4854A9BD.5040905@gmail.com> Benjamin Peterson wrote: > On Fri, Jun 13, 2008 at 12:14 PM, Guido van Rossum wrote: >> On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson >> wrote: >>> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger wrote: >>>>> Nick> def getName(self): >>>>> Nick> assert self.__initialized, "Thread.__init__() not called" >>>>> Nick> return self.__name >>>> Why is __name private to begin with? Any reason for the getters and >>>> setters? Why isn't this just an attribute? >>> In 3.x, it's just an attribute. >> Oh, is it? Where's the PEP with the API redesign? Did I miss some kind >> of decision-making process, weighing compatibility concerns against >> other issues? > > meaning that it only has one underscore. They methods still live. That's worse - it means it can now collide with _name attributes on subclasses. This has become a lot more complicated than what I thought we were doing: adding PEP 8 compliant aliases for the existing methods without otherwise changing the threading implementation. As far as I can recall, that is all that was contained in the 2.x patch I reviewed. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From armin.ronacher at active-4.com Sun Jun 15 07:34:45 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 05:34:45 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <200806150157.23985@news.perlig.de> Message-ID: Guido van Rossum python.org> writes: > > On Sat, Jun 14, 2008 at 4:57 PM, Andr? Malo perlig.de> wrote: > > I find this collection of cases pretty weak as an argument for implementing > > that in the stdlib. A lot of special purpose types would fit into such > > reasoning, but do you want to have all of them maintained here? > > No, but an ordered dict happens to be a *very* common thing to need, > for a variety of reasons. So I'm +0.5 on adding this to the > collections module. However someone needs to contribute working code. > It would also be useful to verify that it actually fulfills the needs > of some actual use case. Perhaps looking at how Django uses its > version would be helpful. I compared multiple ordered dicts now (including Babel, Django and the C-implementation I mentioned earlier) and implemented a python version of the ordered dict as reference implementation: http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py Regards, Armin From armin.ronacher at active-4.com Sun Jun 15 07:37:20 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 05:37:20 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <740c3aec0806142137r79bb8a84k411155d75b2df592@mail.gmail.com> Message-ID: BJ?rn Lindqvist gmail.com> writes: > I think that the name ordereddict would be more readable though, and > match the existing defaultdict class in the collections module. While I agree that "ordered" makes more sense, it's pretty stupid to type with two 'd's right after another. Regards, Armin From python at rcn.com Sun Jun 15 07:43:32 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 14 Jun 2008 22:43:32 -0700 Subject: [Python-Dev] Proposal: add odict to collections References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> Message-ID: <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> From: "Talin" > There's been a lot of controversy/confusion about ordered dicts. I think that is why all earlier proposals all died. > One of > the sources of confusion is that people mean different things when they > use the term "ordered dict": In some cases, the term is used to mean a > dictionary that remembers the order of insertions, and in other cases it > is used to mean a sorted dict, i.e. an associative data structure in > which the entries are kept sorted. (And I'm not sure that those are the > only two possibilities.) For an insertion order dictionary, there was also a question about how to handle duplicate keys. Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the odict.items() return? [(k1,v3), (k2,v2)] [(k2,v2), (k1,v3)] The first maintains the original sort order and is consistent with the usual idea that d[k]=v simply replaces the value but does not alter the hash table. It is a bit weird though because v3 appears earlier than v2 which was inserted earlier. It's especially weird for keys that are equal but not identical: d[0]=v1 d[0.0]=v3. Do you want 0.0 to remain associated with v3 or should the items list contain a pair that was not in the original insertion list, (0, v3)? The second one is a bit weird because a key "loses its place" whenever the value is updated. IIRC, previous discussions showed an interest in odicts but that there were a lot of questions of the specific semantics of the API. This would no doubt be compounded by attempts to emulate dict views. Regardless, there should probably be a PEP and alternate pure python versions should be posted on ASPN so people can try them out. post Raymond From armin.ronacher at active-4.com Sun Jun 15 07:42:19 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 05:42:19 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <2cda2fc90806142138x253894e5vc199609db4244829@mail.gmail.com> Message-ID: Hasan Diwan gmail.com> writes: > > 2008/6/14 Talin acm.org>: > > There's been a lot of controversy/confusion about ordered dicts. One of the > > sources of confusion is that people mean different things when they use the > > term "ordered dict": In some cases, the term is used to mean a dictionary > > that remembers the order of insertions, and in other cases it is used to > > mean a sorted dict, i.e. an associative data structure in which the entries > > are kept sorted. (And I'm not sure that those are the only two > > possibilities.) > > Have the comparison function passed in as a parameter then, if it's > None, then have it maintain the order of insertion? Something like: > def __init__(self, cmpfunc = None): > self.dict = dict() > > I think that would be contraproductive and would make the constructor incompatible with the normal dict constructor which accepts keyword arguments too. Also that dict is just in order, but not sorted by something. You could still do something like this: d = odict() d['Pinguin'] = 'telly' d['Parrot'] = 'cage' d['Mouse'] = 'Napoleon' d.sort(key=lambda x: x[1].lower()) That of course would not sort items inserted after the sort call. Regards, Armin From wolever at cs.toronto.edu Sun Jun 15 06:59:51 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Sun, 15 Jun 2008 01:59:51 -0300 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: On 14-Jun-08, at 8:39 PM, Armin Ronacher wrote: > ... > I noticed lately that quite a few projects are implementing their own > subclasses of `dict` that retain the order of the key/value pairs. > ... I'm +1 on this one too, as there are at least a couple of times in recent memory when I would have found this useful. And, as far as questions about the definition of an ordered dictionary, is there any good reason not to simply treat the dict as a list? Something like (with the obvious bits left out): class odict(dict): def __init__(self, *args): self._order = [] def __setitem__(self, key, val): if key not in self: self._order.append(key) def __iter__(self): return self._order def items(self): return ([item, self[item] for item in self._order]) def sort(self): self._order.sort() ... and so on ... That way all the order-related functions are well defined, so it would be hard claim that it doesn't do the "right thing" without claiming that lists don't do the "right thing". From armin.ronacher at active-4.com Sun Jun 15 08:07:47 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 06:07:47 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: Raymond Hettinger rcn.com> writes: > For an insertion order dictionary, there was also a question about > how to handle duplicate keys. > > Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the odict.items() return? > > [(k1,v3), (k2,v2)] > [(k2,v2), (k1,v3)] All the ordered dict implementations I saw behave like this: >>> odict([(1, 'foo'), (2, 'bar'), (1, 'baz')]).items() [(1, 'baz'), (2, 'bar')] And that makes sense because it's consistent with the dict interface. But I guess that is a pretty small issue and most of the time you are not dealing with double keys when using an ordered dict. > IIRC, previous discussions showed an interest in odicts but that > there were a lot of questions of the specific semantics of the API. No doubt there. But > This would no doubt be compounded by attempts to emulate > dict views. Regardless, there should probably be a PEP and > alternate pure python versions should be posted on ASPN so people > can try them out. That's true, but by now there are countless of ordered dict implementations with a mostly-compatible interface and applications and libraries are using them already. I have an example implementation here that incorporates the ideas from ordereddict, Django's OrderedDict and Babel's odict: http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py Regards, Armin From cesare at pronto.it Sun Jun 15 08:16:38 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sun, 15 Jun 2008 08:16:38 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <4854614B.9070903@canterbury.ac.nz> References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> <4854614B.9070903@canterbury.ac.nz> Message-ID: In data 15 giugno 2008 alle ore 02:24:43, Greg Ewing ha scritto: > ...and which should *not* be used in most cases, for > the same reason. > > All those tutorials that start out with 'from something > import *' are doing a lot of harm to the impressionable > minds of new programmers, IMO. OK, but nobody have questioned about removing 'from something import *' just to help noobs... That's because the instruction *can* be useful in *some* (hopely limited, but existent) contexts. It's a matter of programmer choises. Anyway (and dropping my proposal), I think that the FAQ needs to be changed to advice that the 'with' keyword in Python makes a completely different kind of work. Cesare From cesare at pronto.it Sun Jun 15 08:20:22 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Sun, 15 Jun 2008 08:20:22 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: In data 15 giugno 2008 alle ore 07:43:32, Raymond Hettinger ha scritto: > For an insertion order dictionary, there was also a question about > how to handle duplicate keys. > > Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the odict.items() return? > > [(k1,v3), (k2,v2)] > [(k2,v2), (k1,v3)] > > The first maintains the original sort order and is consistent with the usual > idea that d[k]=v simply replaces the value but does not alter the hash table. > It is a bit weird though because v3 appears earlier than v2 which was > inserted earlier. It's especially weird for keys that are equal but not > identical: d[0]=v1 d[0.0]=v3. Do you want 0.0 to remain associated > with v3 or should the items list contain a pair that was not in the original > insertion list, (0, v3)? > > The second one is a bit weird because a key "loses its place" whenever > the value is updated. > > IIRC, previous discussions showed an interest in odicts but that > there were a lot of questions of the specific semantics of the API. > This would no doubt be compounded by attempts to emulate > dict views. Regardless, there should probably be a PEP and > alternate pure python versions should be posted on ASPN so people > can try them out. > post > > > Raymond The same problem happens with dictionary updates: d = {} d[k1] = v1 d[k2] = v2 d[k1] = v3 The last instruction just replaces the existing entry, so I'm +0 for the first result. Cesare From dbpokorny at gmail.com Sun Jun 15 09:09:59 2008 From: dbpokorny at gmail.com (dbpokorny at gmail.com) Date: Sun, 15 Jun 2008 00:09:59 -0700 (PDT) Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com> On Jun 14, 4:39 pm, Armin Ronacher wrote: > - in XML/HTML processing it's often desired to keep the attributes of > an tag ordered during processing. So that input ordering is the > same as the output ordering. Munging XML is a niche. > > - Form data transmitted via HTTP is usually ordered by the position > of the input/textarea/select field in the HTML document. That > information is currently lost in most Python web applications / > frameworks. If you don't like the fact that your web application framework loses the order of its key:value pairs relative to the page, then you should bring it up with the developers. Ordered dicts, dicts that remember the chronological order of their insertion, don't sound generally useful. In contrast, sorted dicts sound appealing, but I can't think of any use cases where sorted(mydict.keys()) fails to be effective; it seems to me the main advantage of a sorted dict is that you don't have to remember to sort the keys every time you want to use them. > Additionally it would support slicing where a list of key, value tuples > is returned and sort/reverse/index methods that work like their list > equivalents. Index based lookup could work via odict.byindex(). Slicing tuples, lists, and strings return objects of the same type, so slicing a sorted dict should return a sorted dict. A sorted dict is the same data structure as a Berkeley DB table. Why not use the term 'table' instead of 'sorteddict'? Unless I am missing something big, the implementation of new key insert for sorteddict looks inefficient - on average you have to move half of your elements over one space. A B tree or B+ tree (or other tree) would be a more efficient algorithm. -1 for ordered dict +1 for sorted dict David From g.brandl at gmx.net Sun Jun 15 09:11:43 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 15 Jun 2008 09:11:43 +0200 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <4854A9BD.5040905@gmail.com> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> <485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com> <4854A9BD.5040905@gmail.com> Message-ID: Nick Coghlan schrieb: > Benjamin Peterson wrote: >> On Fri, Jun 13, 2008 at 12:14 PM, Guido van Rossum wrote: >>> On Fri, Jun 13, 2008 at 9:42 AM, Benjamin Peterson >>> wrote: >>>> On Fri, Jun 13, 2008 at 11:40 AM, Raymond Hettinger wrote: >>>>>> Nick> def getName(self): >>>>>> Nick> assert self.__initialized, "Thread.__init__() not called" >>>>>> Nick> return self.__name >>>>> Why is __name private to begin with? Any reason for the getters and >>>>> setters? Why isn't this just an attribute? >>>> In 3.x, it's just an attribute. >>> Oh, is it? Where's the PEP with the API redesign? Did I miss some kind >>> of decision-making process, weighing compatibility concerns against >>> other issues? >> >> meaning that it only has one underscore. They methods still live. > > That's worse - it means it can now collide with _name attributes on > subclasses. > > This has become a lot more complicated than what I thought we were > doing: adding PEP 8 compliant aliases for the existing methods without > otherwise changing the threading implementation. As far as I can recall, > that is all that was contained in the 2.x patch I reviewed. The move to _attributes instead of __attributes was done quite a while earlier, in r57222, by Guido himself. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From python at rcn.com Sun Jun 15 09:12:38 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 15 Jun 2008 00:12:38 -0700 Subject: [Python-Dev] Proposal: add odict to collections References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org><758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: From: "Cesare Di Mauro" > The same problem happens with dictionary updates: > > d = {} > d[k1] = v1 > d[k2] = v2 > d[k1] = v3 > > The last instruction just replaces the existing entry, so I'm +0 for the first result. There's a difference. With dicts, the third insertion *replaces* the value while leaving data structure unmolested. Also, the key doesn't update (an equal but identical key doesn't get substituted). With an odict that preserves insertion order, you're talking about *deleting* the old entry and *appending* the new one, complete with both the new key and new value. If that is the desired behavior, then it becomes impossible to update the value of an odict entry while leaving its insertion order unchanged. What a bummer. An alternative behavior is to replace the value, leaving the key in its original position. But then, you've messed-up the expectation that v2 occurs before v3 eventhough v3 was inserted later. This is especially weird because you keep k1 which was inserted earlier, not k3 which is equivalent but not necessarily identical. Neither behavior is de facto, TheRightWay(tm). Each has its uses. Each has its oddities. Each will have its fans who find that particular way to be the best and most intuitive. One other issue arises if you choose the approach where updating a value triggers re-ordering -- the the dict view concept no longer translates very well. With regular dicts, you can update values while iterating. Losing that would be a bummer too. I don't favor one over the other. Am just pointing-out that the proposal is a little more complex than simply wishing for an ordered verion of a dictionary and expecting that that wish is self-defining in a way the matches everyone's intuition, use cases, and expectations. Raymond From martin at v.loewis.de Sun Jun 15 09:41:32 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 15 Jun 2008 09:41:32 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <200806150157.23985@news.perlig.de> Message-ID: <4854C7AC.4000508@v.loewis.de> > I compared multiple ordered dicts now (including Babel, Django and the > C-implementation I mentioned earlier) and implemented a python version > of the ordered dict as reference implementation: > > http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py I find the slicing API surprising. IMO, if you do support slicing, then a) the start and end indices should be the same ones that you also use in regular indexing. b) the result should be of the same kind as the thing being sliced, i.e. an odict. So I would rather expect >>> d['c':'spam'] odict.odict([('c', 'd'), ('foo', 'bar')]) The slicing operation that you provide should be spelled as d.items()[1:3], or, if you don't want to pay the cost of creating the full items list, then add a method such as d.slice_by_index(1,3). What's the use case for this operation, anyway? Regards, Martin From ncoghlan at gmail.com Sun Jun 15 09:49:25 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Jun 2008 17:49:25 +1000 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <222B74C3-CC7B-4AEC-8084-10B62E7705B4@python.org> <367889A5-6BD0-4106-948D-B5B181D4466D@python.org> <18513.16097.394123.312491@montanaro-dyndns-org.local> <485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com> <4854A9BD.5040905@gmail.com> Message-ID: <4854C985.5000201@gmail.com> Georg Brandl wrote: > Nick Coghlan schrieb: >> This has become a lot more complicated than what I thought we were >> doing: adding PEP 8 compliant aliases for the existing methods without >> otherwise changing the threading implementation. As far as I can >> recall, that is all that was contained in the 2.x patch I reviewed. > > The move to _attributes instead of __attributes was done quite a while > earlier, in r57222, by Guido himself. Well, that would explain why it wasn't in the patch I reviewed then :) Raymond's right that a bunch of those methods are fairly pointless though (relics of the module's Java origins I guess, just like the camelCase naming). Perhaps PEP 371 should be expanded a bit to give some details on the cleaned up threading API, as well as a deprecation schedule for the old API? My suggestion would be that we keep the old API around in 3.0 and just give it a vanilla PendingDeprecationWarning, then look at actually deprecating it in 3.1 for removal in 3.2 - by that time, the need for compatibility with versions prior to 2.6 should be fairly rare. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From 2008a at usenet.alexanderweb.de Sun Jun 15 09:53:08 2008 From: 2008a at usenet.alexanderweb.de (Alexander Schremmer) Date: Sun, 15 Jun 2008 09:53:08 +0200 Subject: [Python-Dev] Proposal: add odict to collections References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: <5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net> Armin Ronacher wrote: > That's true, but by now there are countless of ordered dict > implementations with a mostly-compatible interface and applications and > libraries are using them already. Even worse, most of them are slow, i.e. show a wrong algorithmic complexity ... > I have an example implementation here that incorporates the ideas > from ordereddict, Django's OrderedDict and Babel's odict: > > http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py ... like your implementation. It is not too hard to get the delitem O(log n) compared to your O(n), see here: http://codespeak.net/svn/user/arigo/hack/pyfuse/OrderedDict.py So many people are implementing this kind of data type but do not really care about making as fast as it could be ... IMHO yet another reason to ship a usable implementation with Python. Kind regards, Alexander From ncoghlan at gmail.com Sun Jun 15 10:03:02 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Jun 2008 18:03:02 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org><758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: <4854CCB6.9050308@gmail.com> Raymond Hettinger wrote: > I don't favor one over the other. Am just pointing-out that the > proposal is a little more complex than simply wishing for an ordered > verion of a dictionary and expecting that that wish is self-defining in > a way the matches everyone's intuition, use cases, and expectations. If you have an odict with first-insertion ordering, it's fairly trivial to convert it to a dictionary with last-insertion ordering: class odictlastinsertion(odict): def __setitem__(self, k, v): self.pop(k, None) self[k] = v As you note, going the other way would be rather difficult, suggesting that the version ordered by the first key insertion is the more fundamental structure. A PEP would definitely be needed to thrash out those kind of issues and decisions though Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sun Jun 15 10:14:12 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 15 Jun 2008 18:14:12 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com> References: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com> Message-ID: <4854CF54.7090506@gmail.com> dbpokorny at gmail.com wrote: > -1 for ordered dict > +1 for sorted dict Build the ordered dict, then sort it in-place afterwards, just like a list (alternatively, build a normal dict then feed sorted(orig.items()) to the ordered dict constructor). The point of an ordered dict is that unlike a normal dict, the order the keys are returned is well defined at the interface level, rather than arbitrary and implementation-defined. David Wolever's example of a normal dict() with an associated key list is an excellent way of thinking about the proposed semantics. Having a fast ordered dict in the standard library also opens up the possibility of eventually using such a thing for keyword arguments at some point in the future. How nice would it be to be able to just write: t = namedtuple(a=1, b=2, c=3) instead of: c = namedtuple("a b c") t = c(a=1, b=2, c=3) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at pearwood.info Sun Jun 15 10:38:55 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 15 Jun 2008 18:38:55 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <200806151838.56078.steve@pearwood.info> On Sun, 15 Jun 2008 02:59:51 pm David Wolever wrote: > And, as far as questions about the definition of an ordered > dictionary, is there any good reason not to simply treat the dict as > a list? Something like (with the obvious bits left out): Yes. (1) If you implement the ordered dict as a list of key/value pairs, then you get order for free, but searching is slow, and so is deletion. If we wanted O(N) searches, we'd just use a list of tuples :) (2) If you implement it as a hash table plus a list of keys in insertion order, then searching the dict is fast, but deletions are slow. Also you double (?) the amount of memory needed for the keys. On the other hand... a tree-based implementation would require more work, and many more decisions (what kind of tree?), would lose the O(1) behaviour of the hash table, and may end up using just as much memory. So I wouldn't discount your suggestion. -- Steven From lgautier at gmail.com Sun Jun 15 10:42:32 2008 From: lgautier at gmail.com (laurent) Date: Sun, 15 Jun 2008 10:42:32 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: <1213519352.6748.20.camel@hot-spring> On Sun, 2008-06-15 at 00:12 -0700, Raymond Hettinger wrote: > From: "Cesare Di Mauro" > > The same problem happens with dictionary updates: > > > > d = {} > > d[k1] = v1 > > d[k2] = v2 > > d[k1] = v3 > > > > The last instruction just replaces the existing entry, so I'm +0 for the first result. > > There's a difference. With dicts, the third insertion *replaces* the value while leaving data structure unmolested. Also, the key > doesn't update (an equal but identical key doesn't get substituted). > > With an odict that preserves insertion order, you're talking about *deleting* the old entry and *appending* the new one, complete > with both the new key and new value. If that is the desired behavior, then it becomes impossible to update the value of an odict > entry while leaving its insertion order unchanged. What a bummer. > > An alternative behavior is to replace the value, leaving the key in its original position. But then, you've messed-up the > expectation that v2 occurs before v3 eventhough v3 was inserted later. This is especially weird because you keep k1 which was > inserted earlier, not k3 which is equivalent but not necessarily identical. An other behavior making sense would be to append a new position for the key. With your example above: d.index(k1) = [0, 2] d.index(k2) = [1, ] Deletion of a key would have to be explicit (and delete all associated indexes). An other way to think of such a structure would be as a list, for which each item would also have an associated key. ?I think someone mentioned the link with a list during this thread, but here I am not referring to implementation, but the feature of a container-like class that would allow to access elements either by position (index), and that would be a one-to-one association, or key, and that would be a one-to-possibly-many association. > Neither behavior is de facto, TheRightWay(tm). Each has its uses. Each has its oddities. Each will have its fans who find that > particular way to be the best and most intuitive. > > One other issue arises if you choose the approach where updating a value triggers re-ordering -- the the dict view concept no longer > translates very well. With regular dicts, you can update values while iterating. Losing that would be a bummer too. > > I don't favor one over the other. Am just pointing-out that the proposal is a little more complex than simply wishing for an > ordered verion of a dictionary and expecting that that wish is self-defining in a way the matches everyone's intuition, use cases, > and expectations. > > > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/lgautier%40gmail.com From steve at pearwood.info Sun Jun 15 10:44:13 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 15 Jun 2008 18:44:13 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <200806151844.13685.steve@pearwood.info> On Sun, 15 Jun 2008 05:12:38 pm Raymond Hettinger wrote: > With an odict that preserves insertion order, you're talking about > *deleting* the old entry and *appending* the new one, complete with > both the new key and new value. I certainly don't consider updating an ordered dictionary entry as a deletion followed by an append. In fact, I'd be very surprised, and dismayed, if that was the default behaviour. Conceptually, I would expect the following behaviour: >>> od = odict() >>> od[1] = 'spam' # insert a new key >>> od[2] = 'parrot' # insert a new key >>> od[1] = 'ham' # modify existing key >>> od.items() [(1, 'ham'), (2, 'parrot')] If I wanted the alternative behaviour, I could easily get it: >>> od = odict() >>> od[1] = 'spam' # insert a new key >>> od[2] = 'parrot' # insert a new key >>> od.pop(1, None); od[1] = 'ham' # remove and re-insert a key >>> od.items() [(2, 'parrot'), (1, 'ham')] Earlier, Raymond also asked what to do about keys with equal but not identical keys. Since I consider setting the value to be an update rather than a deletion plus re-insertion, then the behaviour is obvious. >>> od = odict([(1, 'norwegian blue'), (2, 'parrot')]) >>> od[1.0] = 'norwegian red' >>> od.items() [(1, 'norwegian red'), (2, 'parrot')] This is close to the behaviour of regular dicts, and to do differently would be very surprising to me. Again, anyone who wants the alternative behaviour can get it easily, with a pop and a set. +1 for an ordered dictionary. As for a sorted dictionary, I don't care much, so +0. You can always sort the keys when you need them. -- Steven From martin at v.loewis.de Sun Jun 15 10:48:25 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jun 2008 10:48:25 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> <5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net> Message-ID: <4854D759.2040202@v.loewis.de> > ... like your implementation. It is not too hard to get the delitem > O(log n) compared to your O(n), see here: > > http://codespeak.net/svn/user/arigo/hack/pyfuse/OrderedDict.py > > So many people are implementing this kind of data type but do not really > care about making as fast as it could be ... IMHO yet another reason to > ship a usable implementation with Python. If you use a linked list instead of Python list, you can even do deletion in O(1). Of course, memory consumption will be higher. Regards, Martin From steve at pearwood.info Sun Jun 15 11:03:15 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 15 Jun 2008 19:03:15 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <1213519352.6748.20.camel@hot-spring> References: <1213519352.6748.20.camel@hot-spring> Message-ID: <200806151903.16005.steve@pearwood.info> On Sun, 15 Jun 2008 06:42:32 pm laurent wrote: > An other way to think of such a structure would be as a list, for > which each item would also have an associated key. > ?I think someone mentioned the link with a list during this thread, > but here I am not referring to implementation, but the feature of a > container-like class that would allow to access elements either by > position (index), and that would be a one-to-one association, or key, > and that would be a one-to-possibly-many association. I think the quickest way to kill this proposal again will be to start overloading it with more and more complicated behaviour. Python dicts are one-to-one (ignoring edge cases like dict[1] vs dict[1.0]). If you want one-to-many, you can subclass dict and get that behaviour. I think that an ordered dict in the standard library should follow the same philosophy: define the simplest, most fundamental behaviour which is an ordered dictionary, and then let people sub-class it to make more complicated types. So I vote -1 on one-to-many associations, and +1 to one-to-one like regular dicts. As for a API to access items by position rather than by key, I'm neutral on it. You can always get the nth key by extracting the keys into a list. Provided it doesn't become a point of contention, then I'm +0 on giving the ordered dict index-based methods in addition to the key-based methods, but if it becomes a sticking point and puts the whole concept in jeopardy, then I vote -1 on the index-based API. This applies also to slicing. -- Steven From nd at perlig.de Sun Jun 15 11:31:54 2008 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Sun, 15 Jun 2008 11:31:54 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <200806150157.23985@news.perlig.de> Message-ID: <200806151131.55036@news.perlig.de> * Guido van Rossum wrote: > On Sat, Jun 14, 2008 at 4:57 PM, Andr? Malo wrote: > > * Armin Ronacher wrote: > >> Some reasons why ordered dicts are a useful feature: > >> > >> - in XML/HTML processing it's often desired to keep the attributes > >> of an tag ordered during processing. So that input ordering is the > >> same as the output ordering. > >> > >> - Form data transmitted via HTTP is usually ordered by the position > >> of the input/textarea/select field in the HTML document. That > >> information is currently lost in most Python web applications / > >> frameworks. > >> > >> - Eaiser transition of code from Ruby/PHP which have sorted > >> associative arrays / hashmaps. > >> > >> - Having an ordered dict in the standard library would allow other > >> libraries support them. For example a PHP serializer could return > >> odicts rather then dicts which drops the ordering information. > >> XML libraries such as etree could add support for it when creating > >> elements or return attribute dicts. > > > > I find this collection of cases pretty weak as an argument for > > implementing that in the stdlib. A lot of special purpose types would > > fit into such reasoning, but do you want to have all of them maintained > > here? > > No, but an ordered dict happens to be a *very* common thing to need, > for a variety of reasons. So I'm +0.5 on adding this to the > collections module. However someone needs to contribute working code. > It would also be useful to verify that it actually fulfills the needs > of some actual use case. Perhaps looking at how Django uses its > version would be helpful. FWIW, I'm working a lot in the contexts described above and I never needed ordered dicts so far (what do I have to do in order to need them?). I've found myself implementing, for example, mutlivaluedicts instead, several times. nd -- Real programmers confuse Christmas and Halloween because DEC 25 = OCT 31. -- Unknown (found in ssl_engine_mutex.c) From armin.ronacher at active-4.com Sun Jun 15 11:39:05 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 09:39:05 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <200806151844.13685.steve@pearwood.info> Message-ID: Steven D'Aprano pearwood.info> writes: > Conceptually, I would expect the following behaviour: > > >>> od = odict() > >>> od[1] = 'spam' # insert a new key > >>> od[2] = 'parrot' # insert a new key > >>> od[1] = 'ham' # modify existing key > >>> od.items() > [(1, 'ham'), (2, 'parrot')] That behavior is different to any ordered-dict implementation out there ;-) Regards, Armin From armin.ronacher at active-4.com Sun Jun 15 12:06:34 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 10:06:34 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> <5cuei5-dd7.ln1@tuxpaddy.alexanderweb.homeip.net> Message-ID: Hi, Alexander Schremmer <2008a usenet.alexanderweb.de> writes: > Even worse, most of them are slow, i.e. show a wrong algorithmic > complexity ... I already said that in the initial post. > > I have an example implementation here that incorporates the ideas > > from ordereddict, Django's OrderedDict and Babel's odict: > > > > http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py > > ... like your implementation. It is not too hard to get the delitem > O(log n) compared to your O(n), see here: That implementation is intended as example implementation for a possible API not one you should actually use. Regards, Armin From armin.ronacher at active-4.com Sun Jun 15 12:13:26 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 10:13:26 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: Message-ID: There are far more responses for that topic than I imagined so I would love to write a PEP about that topic, incorporating the ideas/questions and suggestions discussed here. Regards, Armin From armin.ronacher at active-4.com Sun Jun 15 12:11:39 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 15 Jun 2008 10:11:39 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <200806150157.23985@news.perlig.de> <4854C7AC.4000508@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > > I compared multiple ordered dicts now (including Babel, Django and the > > C-implementation I mentioned earlier) and implemented a python version > > of the ordered dict as reference implementation: > > > > http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py > > I find the slicing API surprising. IMO, if you do support slicing, then > a) the start and end indices should be the same ones that you also use > in regular indexing. > b) the result should be of the same kind as the thing being sliced, i.e. > an odict. > > So I would rather expect > > >>> d['c':'spam'] > odict.odict([('c', 'd'), ('foo', 'bar')]) > > The slicing operation that you provide should be spelled as > d.items()[1:3], or, if you don't want to pay the cost of creating > the full items list, then add a method such as d.slice_by_index(1,3). > What's the use case for this operation, anyway? The use case in my particular case is a ordered dict for messages of a .po file I want to display page-wise in an application. However I agree that it's not useful most of the time so dropping it makes sense. If an application really needs slicing it can still subclass it and implement that. Furthermore with dict-views in Python 3 it would be possible to implement an efficient slicing operation on the dict view returned. Regards, Armin From python at rcn.com Sun Jun 15 12:49:34 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 15 Jun 2008 03:49:34 -0700 Subject: [Python-Dev] Proposal: add odict to collections References: Message-ID: <590A66CD9F4648BB8127A1CBAABA93EA@RaymondLaptop1> From: "Armin Ronacher" > There are far more responses for that topic than I imagined so I would love > to write a PEP about that topic, incorporating the ideas/questions and > suggestions discussed here. Instead of going straight to a PEP, I recommend opening a new wiki page on the topic, letting people post sample pure python implementations, pros/cons of various APIs, showing sample use cases, and analyzing the O(n) behavior of various implementation strategies. As a reference point, the collections.namedtuple() recipe was much simpler but took over six months to mature. Its development process started by combining the best of all previously posted attempts at the same thing, then it was exercised heavily in real apps, then it was posted on ASPN and underwent significant improvements based on feedback from a number of expert developers. Then, it was proposed on python-dev and improved again based on feedback received there. Upon writing the docs and creating examples, more refinements ensued. Upon applying it to the standard library, more was learned. After the alpha, we started getting user feedback and it got refined even further. Raymond From barry at python.org Sun Jun 15 14:01:40 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 15 Jun 2008 08:01:40 -0400 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 15, 2008, at 1:43 AM, Raymond Hettinger wrote: > For an insertion order dictionary, there was also a question about > how to handle duplicate keys. > > Given odict([(k1,v1), (k2,v2), (k1,v3)]), what should the > odict.items() return? > > [(k1,v3), (k2,v2)] > [(k2,v2), (k1,v3)] > > The first maintains the original sort order and is consistent with > the usual > idea that d[k]=v simply replaces the value but does not alter the > hash table. > It is a bit weird though because v3 appears earlier than v2 which was > inserted earlier. It's especially weird for keys that are equal but > not > identical: d[0]=v1 d[0.0]=v3. Do you want 0.0 to remain associated > with v3 or should the items list contain a pair that was not in the > original > insertion list, (0, v3)? > > The second one is a bit weird because a key "loses its place" whenever > the value is updated. Heh, neither of these would work for the email package's own flavor of "ordered" dictionary. Its .items() will return all three key/val pairs, but it's __getitem__ interface would only return the first two, and there are additional interfaces exposed for .get_all() (which is like .get() but returns a list of values matching the given key). Okay, so email couldn't use whatever stdlib odict gets added, but I think that just shows that this may not be as fundamental a data structure as we think it is. I'd much rather see a package on the cheeseshop gain overwhelming popularity, practically forcing us to include it in the stdlib. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFUEpHEjvBPtnXfVAQIj9QQAtuvlC5YtcSBsddztqD8kwokSrvKz7Nef oM0JpxjVBQ7oT0F9MnWEvu9Rf8aTVdXsR/zWTf0yw1jt4HtM50Yu4RGqyjghFJ/Z +Gz+hAqyCerJE6Y9AlW4UdJbQ47wD/Sp1AbMafHCubbt3Sp1AxKmr1tN84WAFefw 8rkP6LbpP64= =dwAi -----END PGP SIGNATURE----- From stefan_ml at behnel.de Sun Jun 15 14:40:36 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 15 Jun 2008 14:40:36 +0200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: Armin Ronacher wrote: > - in XML/HTML processing it's often desired to keep the attributes of > an tag ordered during processing. So that input ordering is the > same as the output ordering. > > - [...] > XML libraries such as etree could add support for it when creating > elements or return attribute dicts. I hope that what you call "etree" here (I assume you mean ElementTree) would stay compatible to lxml.etree in that case. Meaning: adding or replacing an attribute removes a possibly existing entry and appends the new value at the end. Stefan From jnoller at gmail.com Sun Jun 15 14:59:23 2008 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 15 Jun 2008 08:59:23 -0400 Subject: [Python-Dev] PEP 8 and optional underscores In-Reply-To: <4854C985.5000201@gmail.com> References: <7619EA98CE4545DFA4F477B7714C2FA1@RaymondLaptop1> <485235B4.6030401@gmail.com> <18514.41043.913466.680598@montanaro-dyndns-org.local> <89BCC1A20CF14EB3BE9CCDB6EE286599@RaymondLaptop1> <1afaf6160806130942o2cd5bb94q70c1810bb6d49cac@mail.gmail.com> <1afaf6160806131018w7f2e16cfo69307d086f0f4@mail.gmail.com> <4854A9BD.5040905@gmail.com> <4854C985.5000201@gmail.com> Message-ID: <4222a8490806150559qfe7496er184af3af3043bb37@mail.gmail.com> On Sun, Jun 15, 2008 at 3:49 AM, Nick Coghlan wrote: > Georg Brandl wrote: >> >> Nick Coghlan schrieb: >>> >>> This has become a lot more complicated than what I thought we were doing: >>> adding PEP 8 compliant aliases for the existing methods without otherwise >>> changing the threading implementation. As far as I can recall, that is all >>> that was contained in the 2.x patch I reviewed. >> >> The move to _attributes instead of __attributes was done quite a while >> earlier, in r57222, by Guido himself. > > Well, that would explain why it wasn't in the patch I reviewed then :) > > Raymond's right that a bunch of those methods are fairly pointless though > (relics of the module's Java origins I guess, just like the camelCase > naming). > > Perhaps PEP 371 should be expanded a bit to give some details on the cleaned > up threading API, as well as a deprecation schedule for the old API? > > My suggestion would be that we keep the old API around in 3.0 and just give > it a vanilla PendingDeprecationWarning, then look at actually deprecating it > in 3.1 for removal in 3.2 - by that time, the need for compatibility with > versions prior to 2.6 should be fairly rare. > > Cheers, > Nick. Rather than piling onto pep 371, perhaps a secondary PEP covering the renaming of *both* modules and the cleaning up the new API should be done, that way pep 371 can simply conform to the latter? -jesse From tom at vector-seven.com Sun Jun 15 15:11:36 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sun, 15 Jun 2008 23:11:36 +1000 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <484EA327.2050704@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> Message-ID: <48551508.5060500@vector-seven.com> My work on the AST optimizer has led me down the path of attempting to replace things like Name("True") with Const(Py_True) nodes. This works fine most of the time, with the exception of the xmlrpclib module, where True and False are actually redefined: True, False = True, False As I stated in an earlier email, the optimizer tries to replace the tuple of Name nodes on the LHS with Py_True and Py_False respectively, which has the net effect of removing xmlrpclib.{True, False}. Obviously undesirable. The simplest options I can think of to remedy this: 1. A setattr hack: setattr(__import__(__name__), "True", True) 2. Remove all optimization of Name("True") and Name("False") 3. Skip AST optimization entirely for the LHS of Assignment nodes (effectively removing any optimization of the "targets" tuple) I'm leaning towards #3 at the moment as it seems like it's going to be the cleanest approach and makes a lot of sense -- at least on the surface. Can anybody think of problems with this approach? Cheers, T Thomas Lee wrote: > Martin v. L?wis wrote: >>> The question is, what is the specification for Python. >>> >> >> Now, that's a more interesting question than the question originally >> asked (which I interpreted as "why does it work the way it works"). >> >> The only indication in the specification of that feature I could find >> was: >> >> http://docs.python.org/dev/library/constants.html >> >> "Changed in version 2.4: Assignments to None are illegal and raise a >> SyntaxError." >> >> Now, given that this talks about the built-in namespace, this *doesn't* >> specify that foo.None=1 should also raise a syntax error. >> >> So the implementation apparently deviates from the specification. >> >> In Python 3, None, True, and False are keywords, so clearly, the >> intended semantics is also the implemented one (and the language >> description for 2.x needs to be updated/clarified). >> >> > Interestingly enough, the semantics of True, False and None are > different from one another in 2.6: > > True = "blah" and False = 6 are perfectly legal in Python <=2.6. > > Funny, I just ran into this. I was trying to figure out why the AST > optimization code was breaking test_xmlrpc ... turns out xmlrpclib > defines xmlrpclib.True and xmlrpclib.False and the optimizer was > trying to resolve them as constants while compiling the module. Ouch. > > What happened in 3k? Were the constants in xmlrpclib renamed/removed? > > Cheers, > T > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com From musiccomposition at gmail.com Sun Jun 15 15:33:02 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sun, 15 Jun 2008 08:33:02 -0500 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <48551508.5060500@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> Message-ID: <1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com> On Sun, Jun 15, 2008 at 8:11 AM, Thomas Lee wrote: > > The simplest options I can think of to remedy this: > > 1. A setattr hack: setattr(__import__(__name__), "True", True) > 2. Remove all optimization of Name("True") and Name("False") > 3. Skip AST optimization entirely for the LHS of Assignment nodes > (effectively removing any optimization of the "targets" tuple) You're working on optimization for the 2.6 branch, correct? In that case, why don't we take option 3 in 2.x and just reenable it in 3.x where it's completely forbidden to assign to True or False? > > I'm leaning towards #3 at the moment as it seems like it's going to be the > cleanest approach and makes a lot of sense -- at least on the surface. Can > anybody think of problems with this approach? > > Cheers, > T -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From tom at vector-seven.com Sun Jun 15 15:34:14 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sun, 15 Jun 2008 23:34:14 +1000 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <48551508.5060500@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> Message-ID: <48551A56.1020704@vector-seven.com> Option 4 just struck me: only optimize Name nodes if they have a Load ctx. This makes even more sense: in a Store context, we almost invariably want the name rather than the constant. Cheers, T Thomas Lee wrote: > My work on the AST optimizer has led me down the path of attempting to > replace things like Name("True") with Const(Py_True) nodes. This works > fine most of the time, with the exception of the xmlrpclib module, > where True and False are actually redefined: > > True, False = True, False > > As I stated in an earlier email, the optimizer tries to replace the > tuple of Name nodes on the LHS with Py_True and Py_False respectively, > which has the net effect of removing xmlrpclib.{True, False}. > Obviously undesirable. > > The simplest options I can think of to remedy this: > > 1. A setattr hack: setattr(__import__(__name__), "True", True) > 2. Remove all optimization of Name("True") and Name("False") > 3. Skip AST optimization entirely for the LHS of Assignment nodes > (effectively removing any optimization of the "targets" tuple) > > I'm leaning towards #3 at the moment as it seems like it's going to be > the cleanest approach and makes a lot of sense -- at least on the > surface. Can anybody think of problems with this approach? > > Cheers, > T > > Thomas Lee wrote: >> Martin v. L?wis wrote: >>>> The question is, what is the specification for Python. >>>> >>> >>> Now, that's a more interesting question than the question originally >>> asked (which I interpreted as "why does it work the way it works"). >>> >>> The only indication in the specification of that feature I could find >>> was: >>> >>> http://docs.python.org/dev/library/constants.html >>> >>> "Changed in version 2.4: Assignments to None are illegal and raise a >>> SyntaxError." >>> >>> Now, given that this talks about the built-in namespace, this *doesn't* >>> specify that foo.None=1 should also raise a syntax error. >>> >>> So the implementation apparently deviates from the specification. >>> >>> In Python 3, None, True, and False are keywords, so clearly, the >>> intended semantics is also the implemented one (and the language >>> description for 2.x needs to be updated/clarified). >>> >>> >> Interestingly enough, the semantics of True, False and None are >> different from one another in 2.6: >> >> True = "blah" and False = 6 are perfectly legal in Python <=2.6. >> >> Funny, I just ran into this. I was trying to figure out why the AST >> optimization code was breaking test_xmlrpc ... turns out xmlrpclib >> defines xmlrpclib.True and xmlrpclib.False and the optimizer was >> trying to resolve them as constants while compiling the module. Ouch. >> >> What happened in 3k? Were the constants in xmlrpclib renamed/removed? >> >> Cheers, >> T >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com From tom at vector-seven.com Sun Jun 15 15:37:25 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sun, 15 Jun 2008 23:37:25 +1000 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com> Message-ID: <48551B15.8070702@vector-seven.com> Benjamin Peterson wrote: > On Sun, Jun 15, 2008 at 8:11 AM, Thomas Lee wrote: > >> The simplest options I can think of to remedy this: >> >> 1. A setattr hack: setattr(__import__(__name__), "True", True) >> 2. Remove all optimization of Name("True") and Name("False") >> 3. Skip AST optimization entirely for the LHS of Assignment nodes >> (effectively removing any optimization of the "targets" tuple) >> > > You're working on optimization for the 2.6 branch, correct? In that > case, why don't we take option 3 in 2.x and just reenable it in 3.x > where it's completely forbidden to assign to True or False? > > Sorry, that's correct. This is against 2.6 trunk. That's the idea -- in 3.x this will be a non-issue. Cheers, T From solipsis at pitrou.net Sun Jun 15 16:19:19 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 15 Jun 2008 14:19:19 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com> Message-ID: dbpokorny gmail.com gmail.com> writes: > > If you don't like the fact that your web application framework loses > the order of its key:value pairs relative to the page, then you should > bring it up with the developers. Who will then point up that to retain that order while providing you with a dict-like API, they need to write some kind of ordered dict implementation. Then you'll complain that their implementation is sub-optimal and isn't 100% compatible with the original dict API, and post on python-dev to ask that a standard odict implementation is considered for the stdlib. > Ordered dicts, dicts that remember the chronological order of their > insertion, don't sound generally useful. They are generally useful in any case where you want to handle key-value pairs while not confusing a human operator by messing up the original order. Think e.g. configuration files. A common complaint against ConfigParser is that writing a configuration file does not preserve the order of the original file, which is harmless for the computer but very annoying for the human being who maintains that file. As for sorted dicts with general O(log(n)) behaviour, you could try to combine the blist type (http://pypi.python.org/pypi/blist/) with the standard bisect module and see what that gives. From g.brandl at gmx.net Sun Jun 15 16:23:27 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 15 Jun 2008 16:23:27 +0200 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <48551A56.1020704@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <48551A56.1020704@vector-seven.com> Message-ID: Remember that it must still be possible to write (in 2.6) True = 0 assert not True Georg Thomas Lee schrieb: > Option 4 just struck me: only optimize Name nodes if they have a Load > ctx. This makes even more sense: in a Store context, we almost > invariably want the name rather than the constant. > > Cheers, > T > > Thomas Lee wrote: >> My work on the AST optimizer has led me down the path of attempting to >> replace things like Name("True") with Const(Py_True) nodes. This works >> fine most of the time, with the exception of the xmlrpclib module, >> where True and False are actually redefined: >> >> True, False = True, False >> >> As I stated in an earlier email, the optimizer tries to replace the >> tuple of Name nodes on the LHS with Py_True and Py_False respectively, >> which has the net effect of removing xmlrpclib.{True, False}. >> Obviously undesirable. >> >> The simplest options I can think of to remedy this: >> >> 1. A setattr hack: setattr(__import__(__name__), "True", True) >> 2. Remove all optimization of Name("True") and Name("False") >> 3. Skip AST optimization entirely for the LHS of Assignment nodes >> (effectively removing any optimization of the "targets" tuple) >> >> I'm leaning towards #3 at the moment as it seems like it's going to be >> the cleanest approach and makes a lot of sense -- at least on the >> surface. Can anybody think of problems with this approach? >> >> Cheers, >> T >> >> Thomas Lee wrote: >>> Martin v. L?wis wrote: >>>>> The question is, what is the specification for Python. >>>>> >>>> >>>> Now, that's a more interesting question than the question originally >>>> asked (which I interpreted as "why does it work the way it works"). >>>> >>>> The only indication in the specification of that feature I could find >>>> was: >>>> >>>> http://docs.python.org/dev/library/constants.html >>>> >>>> "Changed in version 2.4: Assignments to None are illegal and raise a >>>> SyntaxError." >>>> >>>> Now, given that this talks about the built-in namespace, this *doesn't* >>>> specify that foo.None=1 should also raise a syntax error. >>>> >>>> So the implementation apparently deviates from the specification. >>>> >>>> In Python 3, None, True, and False are keywords, so clearly, the >>>> intended semantics is also the implemented one (and the language >>>> description for 2.x needs to be updated/clarified). >>>> >>>> >>> Interestingly enough, the semantics of True, False and None are >>> different from one another in 2.6: >>> >>> True = "blah" and False = 6 are perfectly legal in Python <=2.6. >>> >>> Funny, I just ran into this. I was trying to figure out why the AST >>> optimization code was breaking test_xmlrpc ... turns out xmlrpclib >>> defines xmlrpclib.True and xmlrpclib.False and the optimizer was >>> trying to resolve them as constants while compiling the module. Ouch. >>> >>> What happened in 3k? Were the constants in xmlrpclib renamed/removed? >>> >>> Cheers, >>> T >>> >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From pje at telecommunity.com Sun Jun 15 16:29:05 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 15 Jun 2008 10:29:05 -0400 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com> Message-ID: <20080615142830.0882A3A402D@sparrow.telecommunity.com> At 02:19 PM 6/15/2008 +0000, Antoine Pitrou wrote: > > Ordered dicts, dicts that remember the chronological order of their > > insertion, don't sound generally useful. > >They are generally useful in any case where you want to handle key-value >pairs while not confusing a human operator by messing up the original order. >Think e.g. configuration files. A common complaint against ConfigParser is >that writing a configuration file does not preserve the order of the original >file, which is harmless for the computer but very annoying for the human >being who maintains that file. You don't need an ordered dictionary for that; you need a save routine that stream-edits the old file contents. That way, you don't lose comments and spacing either. As for the other uses for ordered dictionaries, I find it simplest to just use a list of key,value pairs, and only transform it to a dictionary or dictionary-like structure as needed, using tools like the cgi module, the email package, or wsgiref.headers. From solipsis at pitrou.net Sun Jun 15 16:34:56 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 15 Jun 2008 14:34:56 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com> <20080615142830.0882A3A402D@sparrow.telecommunity.com> Message-ID: Phillip J. Eby telecommunity.com> writes: > > As for the other uses for ordered dictionaries, I find it simplest to > just use a list of key,value pairs, and only transform it to a > dictionary or dictionary-like structure as needed, using tools like > the cgi module, the email package, or wsgiref.headers. What you are saying is that there are already generally useful container types in the stdlib, but isn't it a good argument in favor of ripping them out of domain-specific packages and provide them as generic classes in the collections module? Someone never using cgi or wsgiref wouldn't know that some of the code there can be useful for other purposes. From tom at vector-seven.com Sun Jun 15 16:40:47 2008 From: tom at vector-seven.com (Thomas Lee) Date: Mon, 16 Jun 2008 00:40:47 +1000 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <48551A56.1020704@vector-seven.com> Message-ID: <485529EF.3080209@vector-seven.com> Georg Brandl wrote: > Remember that it must still be possible to write (in 2.6) > > True = 0 > assert not True Ah of course. Looks like I should just avoid optimizations of Name("True") and Name("False") all together. That's a shame! Cheers, T > > Georg > > Thomas Lee schrieb: >> Option 4 just struck me: only optimize Name nodes if they have a Load >> ctx. This makes even more sense: in a Store context, we almost >> invariably want the name rather than the constant. >> >> Cheers, >> T >> >> Thomas Lee wrote: >>> My work on the AST optimizer has led me down the path of attempting >>> to replace things like Name("True") with Const(Py_True) nodes. This >>> works fine most of the time, with the exception of the xmlrpclib >>> module, where True and False are actually redefined: >>> >>> True, False = True, False >>> >>> As I stated in an earlier email, the optimizer tries to replace the >>> tuple of Name nodes on the LHS with Py_True and Py_False >>> respectively, which has the net effect of removing xmlrpclib.{True, >>> False}. Obviously undesirable. >>> >>> The simplest options I can think of to remedy this: >>> >>> 1. A setattr hack: setattr(__import__(__name__), "True", True) >>> 2. Remove all optimization of Name("True") and Name("False") >>> 3. Skip AST optimization entirely for the LHS of Assignment nodes >>> (effectively removing any optimization of the "targets" tuple) >>> >>> I'm leaning towards #3 at the moment as it seems like it's going to >>> be the cleanest approach and makes a lot of sense -- at least on the >>> surface. Can anybody think of problems with this approach? >>> >>> Cheers, >>> T >>> >>> Thomas Lee wrote: >>>> Martin v. L?wis wrote: >>>>>> The question is, what is the specification for Python. >>>>>> >>>>> >>>>> Now, that's a more interesting question than the question originally >>>>> asked (which I interpreted as "why does it work the way it works"). >>>>> >>>>> The only indication in the specification of that feature I could find >>>>> was: >>>>> >>>>> http://docs.python.org/dev/library/constants.html >>>>> >>>>> "Changed in version 2.4: Assignments to None are illegal and raise a >>>>> SyntaxError." >>>>> >>>>> Now, given that this talks about the built-in namespace, this >>>>> *doesn't* >>>>> specify that foo.None=1 should also raise a syntax error. >>>>> >>>>> So the implementation apparently deviates from the specification. >>>>> >>>>> In Python 3, None, True, and False are keywords, so clearly, the >>>>> intended semantics is also the implemented one (and the language >>>>> description for 2.x needs to be updated/clarified). >>>>> >>>>> >>>> Interestingly enough, the semantics of True, False and None are >>>> different from one another in 2.6: >>>> >>>> True = "blah" and False = 6 are perfectly legal in Python <=2.6. >>>> >>>> Funny, I just ran into this. I was trying to figure out why the AST >>>> optimization code was breaking test_xmlrpc ... turns out xmlrpclib >>>> defines xmlrpclib.True and xmlrpclib.False and the optimizer was >>>> trying to resolve them as constants while compiling the module. Ouch. >>>> >>>> What happened in 3k? Were the constants in xmlrpclib renamed/removed? >>>> >>>> Cheers, >>>> T >>>> >>>> _______________________________________________ >>>> Python-Dev mailing list >>>> Python-Dev at python.org >>>> http://mail.python.org/mailman/listinfo/python-dev >>>> Unsubscribe: >>>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com >>>> >>> >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com >>> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org >> >> > > From g.brandl at gmx.net Sun Jun 15 16:47:41 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 15 Jun 2008 16:47:41 +0200 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <485529EF.3080209@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <48551A56.1020704@vector-seven.com> <485529EF.3080209@vector-seven.com> Message-ID: Thomas Lee schrieb: > Georg Brandl wrote: >> Remember that it must still be possible to write (in 2.6) >> >> True = 0 >> assert not True > > Ah of course. Looks like I should just avoid optimizations of > Name("True") and Name("False") all together. That's a shame! We can of course decide to make assignment to True and False illegal in 2.7 :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From pje at telecommunity.com Sun Jun 15 16:52:20 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 15 Jun 2008 10:52:20 -0400 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <09dc54f7-aaeb-4037-87a3-41c7e10a2af0@c19g2000prf.googlegroups.com> <20080615142830.0882A3A402D@sparrow.telecommunity.com> Message-ID: <20080615145147.31B213A402D@sparrow.telecommunity.com> At 02:34 PM 6/15/2008 +0000, Antoine Pitrou wrote: >Phillip J. Eby telecommunity.com> writes: > > > > As for the other uses for ordered dictionaries, I find it simplest to > > just use a list of key,value pairs, and only transform it to a > > dictionary or dictionary-like structure as needed, using tools like > > the cgi module, the email package, or wsgiref.headers. > >What you are saying is that there are already generally useful container >types in the stdlib, but isn't it a good argument in favor of ripping them >out of domain-specific packages and provide them as generic classes in the >collections module? > >Someone never using cgi or wsgiref wouldn't know that some of the code there >can be useful for other purposes. I didn't say I used them for other purposes, or that they were generally useful. Rather, they're specifically useful for the things they're useful for. More often than not, the use case calls for not merely ordering, but ordering of *values*, with multiple values for each key. But the precise API desired for manipulating such structures tends to be highly app-specific (like email, CGI form values, HTTP headers, etc.), so it's actually IMO an argument *against* a general odict type. From tom at vector-seven.com Sun Jun 15 16:55:40 2008 From: tom at vector-seven.com (Thomas Lee) Date: Mon, 16 Jun 2008 00:55:40 +1000 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <48551A56.1020704@vector-seven.com> <485529EF.3080209@vector-seven.com> Message-ID: <48552D6C.1080507@vector-seven.com> Georg Brandl wrote: > > We can of course decide to make assignment to True and False > illegal in 2.7 :) > > Georg > Great to know that's an option. There's little-to-no chance of this making 2.6. I might just avoid trying to treat True/False as "real" constants until there's been a proper discussion about changing these semantics -- just to get test_xmlrpc passing. Cheers, T From rhamph at gmail.com Sun Jun 15 18:45:35 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 15 Jun 2008 10:45:35 -0600 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: On Sun, Jun 15, 2008 at 6:01 AM, Barry Warsaw wrote: > On Jun 15, 2008, at 1:43 AM, Raymond Hettinger wrote: >> The second one is a bit weird because a key "loses its place" whenever >> the value is updated. > > Heh, neither of these would work for the email package's own flavor of > "ordered" dictionary. Its .items() will return all three key/val pairs, but > it's __getitem__ interface would only return the first two, and there are > additional interfaces exposed for .get_all() (which is like .get() but > returns a list of values matching the given key). > > Okay, so email couldn't use whatever stdlib odict gets added, but I think > that just shows that this may not be as fundamental a data structure as we > think it is. I'd much rather see a package on the cheeseshop gain > overwhelming popularity, practically forcing us to include it in the stdlib. +1 on this comment, -0 on adding an ordered dict to 2.6/3.0. An ordered (or sorted) dict seems to be more of a gut reaction. They have some data in a dict, they realize they want it ordered/sorted for some purpose, so the first thing they check is if the dict already provides it. It doesn't, but putting together a combination of a dict and a list is often a fair bit of work - nevermind if you want O(1) removal - so we hear about it in the mean time. But my point is that we we need to focus on finding real use cases and exploring how best to solve them. Otherwise we might as well throw in my OrderedSet[1] as-is, despite that it's got no comments and no ratings. Even I don't seem to use it. Everybody has an opinion on the colour of this bikeshed. [1] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/528878 -- Adam Olsen, aka Rhamphoryncus From p.f.moore at gmail.com Sun Jun 15 18:54:08 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 15 Jun 2008 17:54:08 +0100 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <590A66CD9F4648BB8127A1CBAABA93EA@RaymondLaptop1> References: <590A66CD9F4648BB8127A1CBAABA93EA@RaymondLaptop1> Message-ID: <79990c6b0806150954v24cde6f9v9196db9976717321@mail.gmail.com> 2008/6/15 Raymond Hettinger : > From: "Armin Ronacher" >> >> There are far more responses for that topic than I imagined so I would >> love >> to write a PEP about that topic, incorporating the ideas/questions and >> suggestions discussed here. > > Instead of going straight to a PEP, I recommend opening a new wiki page > on the topic, letting people post sample pure python implementations, > pros/cons of various APIs, showing sample use cases, and analyzing > the O(n) behavior of various implementation strategies. That sounds reasonable. I'm +1 in principle, but would like to see the details thrashed out. BTW, as a meta-question, where is this proposal targeted for? I'd assume 2.6/3.0 are closed now, as we're hitting betas - so are we looking at 2.7/3.1? (Just a quick sanity check on the timing...) Paul. From martin at v.loewis.de Sun Jun 15 19:02:26 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 15 Jun 2008 19:02:26 +0200 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: <48551B15.8070702@vector-seven.com> References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <1afaf6160806150633n76e2f9d2ud83be8d5d30786b@mail.gmail.com> <48551B15.8070702@vector-seven.com> Message-ID: <48554B22.5010708@v.loewis.de> > Sorry, that's correct. This is against 2.6 trunk. > > That's the idea -- in 3.x this will be a non-issue. It's perhaps even less of an issue than you think: In 3.0, True and False are *already* compiled into constants. So any additional special casing would make no difference. Regards, Martin From guido at python.org Sun Jun 15 19:50:13 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 15 Jun 2008 10:50:13 -0700 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <4854CCB6.9050308@gmail.com> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> <4854CCB6.9050308@gmail.com> Message-ID: On Sun, Jun 15, 2008 at 1:03 AM, Nick Coghlan wrote: > Raymond Hettinger wrote: >> >> I don't favor one over the other. Am just pointing-out that the proposal >> is a little more complex than simply wishing for an ordered verion of a >> dictionary and expecting that that wish is self-defining in a way the >> matches everyone's intuition, use cases, and expectations. > > If you have an odict with first-insertion ordering, it's fairly trivial to > convert it to a dictionary with last-insertion ordering: > > class odictlastinsertion(odict): > def __setitem__(self, k, v): > self.pop(k, None) > self[k] = v > > As you note, going the other way would be rather difficult, suggesting that > the version ordered by the first key insertion is the more fundamental > structure. > > A PEP would definitely be needed to thrash out those kind of issues and > decisions though Right. Though on this particular issue, my gut instinct tells me that first-insertion-order is more useful (matching your assertion above). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From zooko at zooko.com Sun Jun 15 21:20:19 2008 From: zooko at zooko.com (zooko) Date: Sun, 15 Jun 2008 12:20:19 -0700 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <200806150157.23985@news.perlig.de> Message-ID: <5CA3F464-102F-432C-92B3-98145D71DF68@zooko.com> On Jun 14, 2008, at 8:26 PM, Guido van Rossum wrote: > No, but an ordered dict happens to be a *very* common thing to need, > for a variety of reasons. So I'm +0.5 on adding this to the > collections module. However someone needs to contribute working code. Here's an LRU cache that I've used a few times over the years: http://allmydata.org/trac/pyutil/browser/pyutil/pyutil/cache.py This is just like a dict ordered by insertion, except: 1. That it removes the oldest entry if it grows beyond a limit. 2. That it moves an entry to the head of the queue when has_key() is called on that item. So, it would be easy to change those two behaviors in order to use this implementation. There are actually three implementations in that file: one that is asymptotically O(1) for all operations (using a double-linked list woven into the values of the dict), and one that uses a Python list to hold the order, so it is faster for small enough dicts. The third implementation is an implementation that someone else wrote that I included just for comparison purposes -- the comparison showed that each of mine was better. Regards, Zooko From leif.walsh at gmail.com Sun Jun 15 22:07:37 2008 From: leif.walsh at gmail.com (Leif Walsh) Date: Sun, 15 Jun 2008 13:07:37 -0700 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <4854A541.5080101@gmail.com> References: <4852F7EC.6040403@voidspace.org.uk> <4854A541.5080101@gmail.com> Message-ID: -1 to this 'on' statement. On Sat, Jun 14, 2008 at 10:14 PM, Nick Coghlan wrote: > It isn't that a Pascal-with-style statement can't be achieved, it's that it > is pointless syntactic sugar in Python. Just use o = > long_complicated_object_name instead. +1 to this reason. -- Cheers, Leif From zooko at zooko.com Mon Jun 16 00:02:12 2008 From: zooko at zooko.com (zooko) Date: Sun, 15 Jun 2008 15:02:12 -0700 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <5CA3F464-102F-432C-92B3-98145D71DF68@zooko.com> References: <200806150157.23985@news.perlig.de> <5CA3F464-102F-432C-92B3-98145D71DF68@zooko.com> Message-ID: <3F26BFF2-5756-4A7F-A2E7-04CF4A44B716@zooko.com> On Jun 15, 2008, at 12:20 PM, zooko wrote: > So, it would be easy to change those two behaviors in order to use > this implementation. There are actually three implementations in > that file: one that is asymptotically O(1) for all operations > (using a double-linked list woven into the values of the dict), and > one that uses a Python list to hold the order, so it is faster for > small enough dicts. P.S. I didn't mean to fall for the common misunderstanding that hash table operations are O(1). What I should have written is that my ordered dict technique *adds* only O(1) time to the time of the dict on which it is built. As to the question of how important or common this data structure is, I have to admit that while I implemented this one and used it several times (always exclusively for LRU caching), I currently don't use it for anything. Nowadays I try to avoid doing transparent caching (such as LRU caches are often used for) in favor of explicit management of the resource. Regards, Zooko From aleaxit at gmail.com Mon Jun 16 01:12:46 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 15 Jun 2008 16:12:46 -0700 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> <4854614B.9070903@canterbury.ac.nz> Message-ID: +1 on updating the FAQ. Maybe we could even have it notice that a read-only version of the desired semantic for 'with' is easily hacked with the *current* semantic of 'with'...: @contextlib.contextmanager def readonly(anobj): caller_globals = sys._getframe(2).f_globals saved_globals = caller_globals.copy() caller_globals.update((n, getattr(anobj, n)) for n in dir(anobj)) yield caller_globals.clear() caller_globals.update(saved_globals) and then, of course, with readonly(someobj): ... (local variables take precedence, and in particular all assignments define local variables, as usual -- but you can say e.g. 'zz' to mean 'someobj.zz', if you're sufficiently keen on giving up the advantage of having many well-separated namespaces;-). Alex On Sat, Jun 14, 2008 at 11:16 PM, Cesare Di Mauro wrote: > In data 15 giugno 2008 alle ore 02:24:43, Greg Ewing ha scritto: > >> ...and which should *not* be used in most cases, for >> the same reason. >> >> All those tutorials that start out with 'from something >> import *' are doing a lot of harm to the impressionable >> minds of new programmers, IMO. > > OK, but nobody have questioned about removing 'from something import *' just to help noobs... > That's because the instruction *can* be useful in *some* (hopely limited, but existent) contexts. > It's a matter of programmer choises. > > Anyway (and dropping my proposal), I think that the FAQ needs to be changed to advice that the > 'with' keyword in Python makes a completely different kind of work. > > Cesare > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com > From steve at pearwood.info Mon Jun 16 02:07:17 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 16 Jun 2008 10:07:17 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <200806151844.13685.steve@pearwood.info> Message-ID: <200806161007.18697.steve@pearwood.info> On Sun, 15 Jun 2008 07:39:05 pm Armin Ronacher wrote: > Steven D'Aprano pearwood.info> writes: > > Conceptually, I would expect the following behaviour: > > >>> od = odict() > > >>> od[1] = 'spam' # insert a new key > > >>> od[2] = 'parrot' # insert a new key > > >>> od[1] = 'ham' # modify existing key > > >>> od.items() > > > > [(1, 'ham'), (2, 'parrot')] > > That behavior is different to any ordered-dict implementation > out there ;-) I beg to differ. It's the same behaviour as in this implementation: http://dev.pocoo.org/hg/sandbox/raw-file/tip/odict.py which I understand was written by you. -- Steven From greg.ewing at canterbury.ac.nz Mon Jun 16 03:00:28 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 16 Jun 2008 13:00:28 +1200 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <48548FA0.5020009@acm.org> References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> Message-ID: <4855BB2C.30902@canterbury.ac.nz> Talin wrote: > In some cases, the term is used to mean a > dictionary that remembers the order of insertions, and in other cases it > is used to mean a sorted dict, > > I would be more in favor of the idea if we could come up with a less > ambiguous naming scheme. Perhaps "indexed list" or maybe "keyed list" would be a better term for the first variety. And "sorted dict" seems like a good enough term for the other one. -- Greg From greg.ewing at canterbury.ac.nz Mon Jun 16 03:08:55 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 16 Jun 2008 13:08:55 +1200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> <4854614B.9070903@canterbury.ac.nz> Message-ID: <4855BD27.1080605@canterbury.ac.nz> Cesare Di Mauro wrote: > OK, but nobody have questioned about removing 'from something import *' just to help noobs... I'm not advocating removing it from the language, far from it. I agree there are legitimate uses for it in rare cases. I just wish that people wouldn't use it in tutorials, where it's the first thing the newcomer will see, and get the impression that it's the normal thing to do. -- Greg From apt.shansen at gmail.com Mon Jun 16 06:16:59 2008 From: apt.shansen at gmail.com (Stephen Hansen) Date: Sun, 15 Jun 2008 21:16:59 -0700 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: <485457D8.2090205@voidspace.org.uk> <48548FA0.5020009@acm.org> <758DCE76527549B7879CC378065C3EC3@RaymondLaptop1> Message-ID: <7a9c25c20806152116u5cbc7370te17b2c493f3e8a7f@mail.gmail.com> > But my point is that we we need to focus on finding real use cases and > exploring how best to solve them. Otherwise we might as well throw in > my OrderedSet[1] as-is, despite that it's got no comments and no > ratings. Even I don't seem to use it. > I'm mostly lurking on these threads, so as to be semi-prepared when new versions come out.. in fifty years, since we *just* migrated from 2.3 to 2.4 on our product, so. :) Anyways, we've had an OrderedDictionary sort of implementation in our library for eons. The product is commercial and a mix between a desktop application and a web one, with an application server and cross-platform availability... so there's a slightly bizarre and wide range of uses that we've found for putting ordered dictionaries to. If some various use-cases and our reasoning helps, here it is. If not, ignore :) - One is that the system is modular, with various parts able to be activated or deactivated in configuration. The order of module load is important, as we've never quite bothered to put in automatic dependency checking -- that's just overboard for us. Further, the modules can't really be referred to each other via "import" or in code, but instead need to go through a layer of indirection through a name-- so-- the system maintains an ordered dict of modules, a la sys.modules, with the order determining load when it goes through to initialize itself. - There's several more places with a similar pattern; a mapping between "Component Name" and "Module" for generic systems. A processing host which is meant to be able to load and run any kind of service or task. - There's several places where a user is configuring a slightly complex set of actions-- he gives these actions a name, which is shown to the user, and then we have the configuration options itself we use if that is chosen. Its just natural/easy to store that in an ordered dict after we pull it out of the DB, as we want its order to be whatever the user chooses in their setup, and the key->value association is clear. - In a modular application with a generic outer interface(meaning the main app can't even fathom what all it might be asked to load), there's things like a "Windows" menu item that's easily stored internally as a mapping between window names and the window object itself, so the menu can be readily re-generated at will and the window found to switch to. - In fact, we use a lot of OrderedDictionaries as a part of that UI to data/configuration mapping, come to think of it. We know the order of "fields" that someone can search on in the database in advance, and have them written into the searchUI code as an ordered dict because it just works nicely. The keys() become a drop-down list, the value a structure identifying to the central server what field it is they're searching on. - Fiiinally (sorta), we find passing ordered dictionaries to our Genshi web templating layer very lovely for making easy-designable web templates for the web client. We even let customers edit them sometimes! Basically, after looking at all of these, my impressions of an "ordered dictionary" for the various use cases we use are: - The ordered dictionary is used almost exclusively in situations where we are getting the order implicitly from some other source. Be it a SQL query (with its own ORDER BY statements), a configuration option, the order of lines in a file, an auto-sequenced table, or hard-coded data.... Thus, we've always found "insertion order" to be important. - Much to my surprise, we actually aren't ever using an ordered dictionary in a situation where the value ends up being modified after the dictionary is loaded. - The only time we use dictionaries where we are updating them after the fact and their order is -expected- to change is when we are using a *sorted* dictionary. - As such, I'd be quite surprised if I was updating the value of an ordered dictionary and it were to change its order. Meaning: >>> d = odict() >>> d["hello"] = 1 >>> d["there"] = 2 >>> d["hello"] = 3 >>> d.keys() ['hello', 'there'] And not: ['there', 'hello'] An ordered dictionary that does not simply preserve initial insertion order strikes me as a *sorted* dictionary-- sorting on insertion time. I'd expect a sorted dictionary to shift itself around as appropriate. I'd not expect an ordered dictionary to change the order without some explicit action. To me, "ordered dictionary" is in fact a *preordered* dictionary. The order is defined before the data in, and the dictionary's job is to just preserve it. Anyways. No idea if that'll help the discussion, but a couple people kept talking about use cases :) --Stephen P.S. Our implementation, incidentally, is essentially the same as one mentioned above though its subclassing from UserDict (historical reasons for now). We just track the keys in _keys, and if someone's setting something in the dictionary not in keys, we append. It seemed the most natural/obvious way to do it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dbpokorny at gmail.com Mon Jun 16 08:26:35 2008 From: dbpokorny at gmail.com (dbpokorny at gmail.com) Date: Sun, 15 Jun 2008 23:26:35 -0700 (PDT) Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <5edc7d74-5507-43d4-8f12-1410b0dc5c3b@z16g2000prn.googlegroups.com> It is possible to get both ordered dict and sorted dict semantics in the same type if you replace (key, value) pairs for dictionary entries with (key,value,order) triples. The third component is a value that indicates the place of the entry relative to the other entries. To get an ordered dict, __setitem__ assigns 1+ max(order) to the new entry's order. To get a sorted dict, order = key. To get a dict sorted by some key function, order = keyfunc(key), etc... David From armin.ronacher at active-4.com Mon Jun 16 11:05:47 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Mon, 16 Jun 2008 09:05:47 +0000 (UTC) Subject: [Python-Dev] Proposal: add odict to collections References: Message-ID: Armin Ronacher active-4.com> writes: > > There are far more responses for that topic than I imagined so I would love > to write a PEP about that topic, incorporating the ideas/questions and > suggestions discussed here. There is now a PEP for the ordered dict: - PEP: http://www.python.org/dev/peps/pep-0372/ - Thread on comp.lang.python: http://thread.gmane.org/gmane.comp.python.general/577894 Regards, Armin From ncoghlan at gmail.com Mon Jun 16 11:29:13 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 16 Jun 2008 19:29:13 +1000 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: References: Message-ID: <48563269.3050609@gmail.com> Armin Ronacher wrote: > Armin Ronacher active-4.com> writes: > >> There are far more responses for that topic than I imagined so I would love >> to write a PEP about that topic, incorporating the ideas/questions and >> suggestions discussed here. > > There is now a PEP for the ordered dict: > > - PEP: http://www.python.org/dev/peps/pep-0372/ > - Thread on comp.lang.python: > http://thread.gmane.org/gmane.comp.python.general/577894 Another use case: in 2.6, RawConfigParser accepts a dict_type argument that allows an application to set the type of dictionary used internally. The motivation for this addition was expressly to allow users to provide an ordered dictionary [1]. Cheers, Nick. [1] http://bugs.python.org/issue1371075 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From rasky at develer.com Mon Jun 16 18:18:23 2008 From: rasky at develer.com (Giovanni Bajo) Date: Mon, 16 Jun 2008 18:18:23 +0200 Subject: [Python-Dev] Troubles with Roundup Message-ID: <4856924F.6080302@develer.com> Hello, I'm trying to login into the tracker but it gives me "invalid login" even after multiple password resets. I can't submit a proper bugreport because... I can't login :) Who can I privately contact to avoid spamming this list? Thanks! -- Giovanni Bajo Develer S.r.l. http://www.develer.com From stephen at xemacs.org Mon Jun 16 20:45:19 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 17 Jun 2008 03:45:19 +0900 Subject: [Python-Dev] Proposal: add odict to collections In-Reply-To: <5edc7d74-5507-43d4-8f12-1410b0dc5c3b@z16g2000prn.googlegroups.com> References: <5edc7d74-5507-43d4-8f12-1410b0dc5c3b@z16g2000prn.googlegroups.com> Message-ID: <87bq21i5hc.fsf@uwakimon.sk.tsukuba.ac.jp> dbpokorny at gmail.com writes: > It is possible to get both ordered dict and sorted dict semantics in > the same type if you replace (key, value) pairs for dictionary entries > with (key,value,order) triples. Roundup uses something like this concept for its value choice menus. I don't actually think it's used, though, except as a way to allow admin user input that inserts an item at a position. From brett at python.org Mon Jun 16 21:07:38 2008 From: brett at python.org (Brett Cannon) Date: Mon, 16 Jun 2008 12:07:38 -0700 Subject: [Python-Dev] Troubles with Roundup In-Reply-To: <4856924F.6080302@develer.com> References: <4856924F.6080302@develer.com> Message-ID: On Mon, Jun 16, 2008 at 9:18 AM, Giovanni Bajo wrote: > Hello, > > I'm trying to login into the tracker but it gives me "invalid login" even > after multiple password resets. I can't submit a proper bugreport because... > I can't login :) > > Who can I privately contact to avoid spamming this list? > The tracker-discuss mailing list is the best place. Or file a bug on the meta-tracker. -Brett From barry at python.org Mon Jun 16 22:19:59 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 16 Jun 2008 16:19:59 -0400 Subject: [Python-Dev] Blow your mind Message-ID: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A colleague just forward this to me and it blew my fscking mind to smithereens. It also brings back a lot of memories. Enjoy! - -Barry http://www.vimeo.com/1093745 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFbK8XEjvBPtnXfVAQLPhwP/XQitqKSHHHUsYyrqN77G2FfIK0adyGNI /NUY0DGJQUKrL0Wy+LQQzXUqY0B9a2uk6hxyyA7qWYzd57/l7JCGCgWdEJPhOyxb Y8LW1N9z2SqdngaZHrQz1j5rRj5x2BFZpwt9/c/fqhbBhXIUrLnX/CZ342jQzneL WrKzvqemGlY= =JSCL -----END PGP SIGNATURE----- From simgidacav at gmail.com Mon Jun 16 08:42:18 2008 From: simgidacav at gmail.com (Simgi Dacav) Date: Mon, 16 Jun 2008 08:42:18 +0200 Subject: [Python-Dev] Let me introduce myself! :) Message-ID: <20080616084218.9150832e.simgidacav@gmail.com> Hi there! I'm Giovanni Simoni, known as Dacav. I'm a student of Computer Science at the University of Trento (Italy), a GNU/Linux user and, since I've tryied it for the first time, I've appreciated the expressive power of Python, so I'd like to delve into it. I subscribed this mailing list after reading the introduction to development (http://www.python.org/dev/intro/). I feel that it wolud be a good mean to improve my skillness with Python, even if I'm probably not enough experienced to bring a real contribute, for the moment. In the meanwhile I inform you that you have one more lurker! :) Thanks for your efforts, and regards Giovanni - -- Giovanni [Dacav] Simoni Linux User #414144 Molti pensano che l'apprendere a fare le cose, diventando uno scienziato, faccia sparire il mistero. Ci? che io sento ? che il mistero motiva e nutre la scienza. La scoperta approfondisce il mistero. -- Manfred Eigen From musiccomposition at gmail.com Mon Jun 16 22:24:10 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 16 Jun 2008 15:24:10 -0500 Subject: [Python-Dev] Blow your mind In-Reply-To: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> Message-ID: <1afaf6160806161324y1c82cf5fu3b262a07d9a79ac7@mail.gmail.com> On Mon, Jun 16, 2008 at 3:19 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > A colleague just forward this to me and it blew my fscking mind to > smithereens. It also brings back a lot of memories. Enjoy! Darn! I'm not on there yet. Anyway, it's nice to see Python development is exploding like fireworks now! -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From mal at egenix.com Mon Jun 16 22:27:40 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 16 Jun 2008 22:27:40 +0200 Subject: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None) In-Reply-To: References: <200806091348.44361.steve@pearwood.info> <484CE993.5050506@voidspace.org.uk> <484D9D59.903@v.loewis.de> <484EA327.2050704@vector-seven.com> <48551508.5060500@vector-seven.com> <48551A56.1020704@vector-seven.com> <485529EF.3080209@vector-seven.com> Message-ID: <4856CCBC.5040003@egenix.com> On 2008-06-15 16:47, Georg Brandl wrote: > Thomas Lee schrieb: >> Georg Brandl wrote: >>> Remember that it must still be possible to write (in 2.6) >>> >>> True = 0 >>> assert not True >> >> Ah of course. Looks like I should just avoid optimizations of >> Name("True") and Name("False") all together. That's a shame! > > We can of course decide to make assignment to True and False > illegal in 2.7 :) Raising a run-time exception would be fine, but not a SyntaxError at compile time - this would effectively make it impossible to keep code compatible to Python 2.1. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 20 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From brett at python.org Tue Jun 17 00:11:43 2008 From: brett at python.org (Brett Cannon) Date: Mon, 16 Jun 2008 15:11:43 -0700 Subject: [Python-Dev] Blow your mind In-Reply-To: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> Message-ID: On Mon, Jun 16, 2008 at 1:19 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > A colleague just forward this to me and it blew my fscking mind to > smithereens. It also brings back a lot of memories. Enjoy! > In case anyone cares to download the video, it's easy with Safari. Just go to the page and under Window -> Activity there is a list of all files downloaded for the page. Just double-click the multi-MB FLV file and Safari will download the Flash video file directly. -Brett From curt at hagenlocher.org Tue Jun 17 01:08:53 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Mon, 16 Jun 2008 16:08:53 -0700 Subject: [Python-Dev] Epoch and Platform Message-ID: The documentation for the time module says that "the epoch is the point where the time starts. On January 1st of that year, at 0 hours, the ``time since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the epoch is, look at gmtime(0)." This confirms that the epoch is platform-specific. As such, the only legal uses of the timestamp should be 1) comparing with another timestamp to determine elapsed time in seconds 2) passing to another standard Python library function which expects a timestamp 3) as a source of randomness. However, the following files in the standard library have hardcoded the assumption that the Python epoch will always be the same as the Unix epoch: In gzip.py, method GzipFile._write_gzip_header In tarfile.py, method _Stream._init_write_gz In uuid.py, function uuid1 Additionally, the following files in the standard library have hardcoded the assumption that the Python epoch will cause timestamps to fall within the range of a 32-bit unsigned integer value: In imputil.py, function _compile In py_compile.py, function compile So there's some kind of a potential discrepancy here, albeit a minor one. This discrepancy can be resolved in one of at least three ways: 1) The documentation for the time module is wrong, and the epoch for Python (at least versions 2.x) should be the Unix epoch. 2) These library functions are slightly wrong and should be modified by subtracing an "epoch offset" before doing other calculations. This offset can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - time.timezone". 3) These library files should be considered part of the platform-specific implementation, and an alternate platform should provide its own version of these files if necessary. Any thoughts on this? >From the perspective of implementing IronPython, I'd prefer that the answer is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as possible, while respecting CLR-specific norms for functionality which is left up to individual implementations. -- Curt Hagenlocher curt at hagenlocher.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Jun 17 01:38:46 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Jun 2008 16:38:46 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: ISTR that we force the epoch to be 1970 on all major platforms -- or perhaps it happens to be 1970 even on Windows when using MS's C runtime. On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher wrote: > The documentation for the time module says that "the epoch is the point > where the time starts. On January 1st of that year, at 0 hours, the ``time > since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the > epoch is, look at gmtime(0)." This confirms that the epoch is > platform-specific. As such, the only legal uses of the timestamp should be > > 1) comparing with another timestamp to determine elapsed time in seconds > 2) passing to another standard Python library function which expects a > timestamp > 3) as a source of randomness. > > However, the following files in the standard library have hardcoded the > assumption that the Python epoch will always be the same as the Unix epoch: > In gzip.py, method GzipFile._write_gzip_header > In tarfile.py, method _Stream._init_write_gz > In uuid.py, function uuid1 > > Additionally, the following files in the standard library have hardcoded the > assumption that the Python epoch will cause timestamps to fall within the > range of a 32-bit unsigned integer value: > In imputil.py, function _compile > In py_compile.py, function compile > > So there's some kind of a potential discrepancy here, albeit a minor one. > This discrepancy can be resolved in one of at least three ways: > > 1) The documentation for the time module is wrong, and the epoch for Python > (at least versions 2.x) should be the Unix epoch. > 2) These library functions are slightly wrong and should be modified by > subtracing an "epoch offset" before doing other calculations. This offset > can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - > time.timezone". > 3) These library files should be considered part of the platform-specific > implementation, and an alternate platform should provide its own version of > these files if necessary. > > Any thoughts on this? > > From the perspective of implementing IronPython, I'd prefer that the answer > is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as > possible, while respecting CLR-specific norms for functionality which is > left up to individual implementations. > > -- > Curt Hagenlocher > curt at hagenlocher.org > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From curt at hagenlocher.org Tue Jun 17 01:51:19 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Mon, 16 Jun 2008 16:51:19 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: >From what I remember, the Microsoft CLIB has been consistent with the Unix epoch since the bad old days of 16-bit. I believe that the Macintosh CLIB used to be based on January 1, 1904 -- but it's been a long time since I did any Mac development and I'm sure it would have changed with OS X. On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum wrote: > ISTR that we force the epoch to be 1970 on all major platforms -- or > perhaps it happens to be 1970 even on Windows when using MS's C > runtime. > > On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher wrote: >> The documentation for the time module says that "the epoch is the point >> where the time starts. On January 1st of that year, at 0 hours, the ``time >> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the >> epoch is, look at gmtime(0)." This confirms that the epoch is >> platform-specific. As such, the only legal uses of the timestamp should be >> >> 1) comparing with another timestamp to determine elapsed time in seconds >> 2) passing to another standard Python library function which expects a >> timestamp >> 3) as a source of randomness. >> >> However, the following files in the standard library have hardcoded the >> assumption that the Python epoch will always be the same as the Unix epoch: >> In gzip.py, method GzipFile._write_gzip_header >> In tarfile.py, method _Stream._init_write_gz >> In uuid.py, function uuid1 >> >> Additionally, the following files in the standard library have hardcoded the >> assumption that the Python epoch will cause timestamps to fall within the >> range of a 32-bit unsigned integer value: >> In imputil.py, function _compile >> In py_compile.py, function compile >> >> So there's some kind of a potential discrepancy here, albeit a minor one. >> This discrepancy can be resolved in one of at least three ways: >> >> 1) The documentation for the time module is wrong, and the epoch for Python >> (at least versions 2.x) should be the Unix epoch. >> 2) These library functions are slightly wrong and should be modified by >> subtracing an "epoch offset" before doing other calculations. This offset >> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - >> time.timezone". >> 3) These library files should be considered part of the platform-specific >> implementation, and an alternate platform should provide its own version of >> these files if necessary. >> >> Any thoughts on this? >> >> From the perspective of implementing IronPython, I'd prefer that the answer >> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as >> possible, while respecting CLR-specific norms for functionality which is >> left up to individual implementations. >> >> -- >> Curt Hagenlocher >> curt at hagenlocher.org >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From bonelake at gmail.com Tue Jun 17 16:32:35 2008 From: bonelake at gmail.com (Brad Miller) Date: Tue, 17 Jun 2008 09:32:35 -0500 Subject: [Python-Dev] Error building 3.0 on OS-X Message-ID: I have been updating the 3.0 source using bzr and after my latest bzr pull I get an error on make install. I'm running 10.5.3. ./configure --enable-framework make make install ... running install_egg_info Writing /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/lib-dynload/Python-3.0a5_-py3.0.egg-info ln -fs "../../../Python" "/Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/config/libpython3.0.a" cd Mac && make installmacsubtree DESTDIR="" DYLD_FRAMEWORK_PATH=/Users/bmiller/src/python/3.0: ../python.exe ./scripts/cachersrc.py -v /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/plat-mac /Library/Frameworks/Python.framework/Versions/3.0/Mac/Tools ../python.exe: can't open file './scripts/cachersrc.py': [Errno 2] No such file or directory make[1]: *** [installmacsubtree] Error 2 make: *** [frameworkinstallmaclib] Error 2 Any hints? Thanks, Brad From p.f.moore at gmail.com Tue Jun 17 17:54:34 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 17 Jun 2008 16:54:34 +0100 Subject: [Python-Dev] Blow your mind In-Reply-To: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> Message-ID: <79990c6b0806170854n55cbb72bxe788d11c644c8a10@mail.gmail.com> 2008/6/16 Barry Warsaw : > A colleague just forward this to me and it blew my fscking mind to > smithereens. It also brings back a lot of memories. Enjoy! Wow! One thing that surprised me was that I never saw Tim appear... Paul. From curt at hagenlocher.org Tue Jun 17 18:26:55 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 17 Jun 2008 09:26:55 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: Any chance of an Official Pronouncement on this topic? It would help us greatly -- even if only to figure out who'll be paying for the next round of beer. On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum wrote: > ISTR that we force the epoch to be 1970 on all major platforms -- or > perhaps it happens to be 1970 even on Windows when using MS's C > runtime. > > On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher wrote: >> The documentation for the time module says that "the epoch is the point >> where the time starts. On January 1st of that year, at 0 hours, the ``time >> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the >> epoch is, look at gmtime(0)." This confirms that the epoch is >> platform-specific. As such, the only legal uses of the timestamp should be >> >> 1) comparing with another timestamp to determine elapsed time in seconds >> 2) passing to another standard Python library function which expects a >> timestamp >> 3) as a source of randomness. >> >> However, the following files in the standard library have hardcoded the >> assumption that the Python epoch will always be the same as the Unix epoch: >> In gzip.py, method GzipFile._write_gzip_header >> In tarfile.py, method _Stream._init_write_gz >> In uuid.py, function uuid1 >> >> Additionally, the following files in the standard library have hardcoded the >> assumption that the Python epoch will cause timestamps to fall within the >> range of a 32-bit unsigned integer value: >> In imputil.py, function _compile >> In py_compile.py, function compile >> >> So there's some kind of a potential discrepancy here, albeit a minor one. >> This discrepancy can be resolved in one of at least three ways: >> >> 1) The documentation for the time module is wrong, and the epoch for Python >> (at least versions 2.x) should be the Unix epoch. >> 2) These library functions are slightly wrong and should be modified by >> subtracing an "epoch offset" before doing other calculations. This offset >> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - >> time.timezone". >> 3) These library files should be considered part of the platform-specific >> implementation, and an alternate platform should provide its own version of >> these files if necessary. >> >> Any thoughts on this? >> >> From the perspective of implementing IronPython, I'd prefer that the answer >> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as >> possible, while respecting CLR-specific norms for functionality which is >> left up to individual implementations. >> >> -- >> Curt Hagenlocher >> curt at hagenlocher.org >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From guido at python.org Tue Jun 17 19:01:09 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jun 2008 10:01:09 -0700 Subject: [Python-Dev] Blow your mind In-Reply-To: <79990c6b0806170854n55cbb72bxe788d11c644c8a10@mail.gmail.com> References: <209A49DB-EAE8-4E28-A78A-8EADA5F9A6F7@python.org> <79990c6b0806170854n55cbb72bxe788d11c644c8a10@mail.gmail.com> Message-ID: On Tue, Jun 17, 2008 at 8:54 AM, Paul Moore wrote: > 2008/6/16 Barry Warsaw : >> A colleague just forward this to me and it blew my fscking mind to >> smithereens. It also brings back a lot of memories. Enjoy! > > Wow! > > One thing that surprised me was that I never saw Tim appear... > Paul. He wasn't able to contribute while we were at CNRI (I did the checkins for him) and once we were at SourceForge there were so many people that it was hard to find anyone in particular. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Jun 17 19:03:42 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jun 2008 10:03:42 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: Can you explain why you are so anxious to get this resolved (apart from the beer :-) ? On Tue, Jun 17, 2008 at 9:26 AM, Curt Hagenlocher wrote: > Any chance of an Official Pronouncement on this topic? It would help > us greatly -- even if only to figure out who'll be paying for the next > round of beer. > > On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum wrote: >> ISTR that we force the epoch to be 1970 on all major platforms -- or >> perhaps it happens to be 1970 even on Windows when using MS's C >> runtime. >> >> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher wrote: >>> The documentation for the time module says that "the epoch is the point >>> where the time starts. On January 1st of that year, at 0 hours, the ``time >>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the >>> epoch is, look at gmtime(0)." This confirms that the epoch is >>> platform-specific. As such, the only legal uses of the timestamp should be >>> >>> 1) comparing with another timestamp to determine elapsed time in seconds >>> 2) passing to another standard Python library function which expects a >>> timestamp >>> 3) as a source of randomness. >>> >>> However, the following files in the standard library have hardcoded the >>> assumption that the Python epoch will always be the same as the Unix epoch: >>> In gzip.py, method GzipFile._write_gzip_header >>> In tarfile.py, method _Stream._init_write_gz >>> In uuid.py, function uuid1 >>> >>> Additionally, the following files in the standard library have hardcoded the >>> assumption that the Python epoch will cause timestamps to fall within the >>> range of a 32-bit unsigned integer value: >>> In imputil.py, function _compile >>> In py_compile.py, function compile >>> >>> So there's some kind of a potential discrepancy here, albeit a minor one. >>> This discrepancy can be resolved in one of at least three ways: >>> >>> 1) The documentation for the time module is wrong, and the epoch for Python >>> (at least versions 2.x) should be the Unix epoch. >>> 2) These library functions are slightly wrong and should be modified by >>> subtracing an "epoch offset" before doing other calculations. This offset >>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - >>> time.timezone". >>> 3) These library files should be considered part of the platform-specific >>> implementation, and an alternate platform should provide its own version of >>> these files if necessary. >>> >>> Any thoughts on this? >>> >>> From the perspective of implementing IronPython, I'd prefer that the answer >>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as >>> possible, while respecting CLR-specific norms for functionality which is >>> left up to individual implementations. >>> >>> -- >>> Curt Hagenlocher >>> curt at hagenlocher.org >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/guido%40python.org >>> >>> >> >> >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) >> > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From agthorr at barsoom.org Tue Jun 17 19:24:57 2008 From: agthorr at barsoom.org (Daniel Stutzbach) Date: Tue, 17 Jun 2008 12:24:57 -0500 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: On Mon, Jun 16, 2008 at 6:38 PM, Guido van Rossum wrote: > ISTR that we force the epoch to be 1970 on all major platforms -- or > perhaps it happens to be 1970 even on Windows when using MS's C > runtime. I can confirm that Python under Windows does indeed use the Unix epoch: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import time >>> time.gmtime(0) (1970, 1, 1, 0, 0, 0, 3, 1, 0) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From list8a.forest at tibit.com Tue Jun 17 19:21:39 2008 From: list8a.forest at tibit.com (Forest) Date: Tue, 17 Jun 2008 10:21:39 -0700 (PDT) Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: <6289.75.55.199.5.1213723299.squirrel@webmail.sonic.net> ISTR using a Microsoft C compiler in the early/mid 1990s whose runtime library used an unusual epoch. I don't recall any others straying from the Unix way, but then again, I haven't been looking for such quirks. Guido wrote: > > ISTR that we force the epoch to be 1970 on all major platforms -- or > perhaps it happens to be 1970 even on Windows when using MS's C > runtime. > > On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher > wrote: >> The documentation for the time module says that "the epoch is the point >> where the time starts. On January 1st of that year, at 0 hours, the >> ``time >> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what >> the >> epoch is, look at gmtime(0)." This confirms that the epoch is >> platform-specific. As such, the only legal uses of the timestamp should >> be >> >> 1) comparing with another timestamp to determine elapsed time in seconds >> 2) passing to another standard Python library function which expects a >> timestamp >> 3) as a source of randomness. >> >> However, the following files in the standard library have hardcoded the >> assumption that the Python epoch will always be the same as the Unix >> epoch: >> In gzip.py, method GzipFile._write_gzip_header >> In tarfile.py, method _Stream._init_write_gz >> In uuid.py, function uuid1 >> >> Additionally, the following files in the standard library have hardcoded >> the >> assumption that the Python epoch will cause timestamps to fall within >> the >> range of a 32-bit unsigned integer value: >> In imputil.py, function _compile >> In py_compile.py, function compile >> >> So there's some kind of a potential discrepancy here, albeit a minor >> one. >> This discrepancy can be resolved in one of at least three ways: >> >> 1) The documentation for the time module is wrong, and the epoch for >> Python >> (at least versions 2.x) should be the Unix epoch. >> 2) These library functions are slightly wrong and should be modified by >> subtracing an "epoch offset" before doing other calculations. This >> offset >> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - >> time.timezone". >> 3) These library files should be considered part of the >> platform-specific >> implementation, and an alternate platform should provide its own version >> of >> these files if necessary. >> >> Any thoughts on this? >> >> From the perspective of implementing IronPython, I'd prefer that the >> answer >> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" >> as >> possible, while respecting CLR-specific norms for functionality which is >> left up to individual implementations. >> >> -- >> Curt Hagenlocher >> curt at hagenlocher.org From curt at hagenlocher.org Tue Jun 17 19:40:23 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 17 Jun 2008 10:40:23 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: I don't *feel* anxious, but my doctor *has* been trying to persuade me to switch to decaf... There's no real urgency. The reason this came up is because I just implemented zlib, which automatically enabled the gzip unit tests. The gzip tests are failing because the current timestamp can't be written as a 32-bit value. In order to checkin my changes, I can't have any failing tests -- so my choices are 1) change the IronPython epoch so that the timestamp works for gzip and tarlib 2) change gzip and tarlib to work with a "less standard" epoch, or 3) disable the failing test(s) ...and I'd rather not resort to #3, if possible. On Tue, Jun 17, 2008 at 10:03 AM, Guido van Rossum wrote: > Can you explain why you are so anxious to get this resolved (apart > from the beer :-) ? > > On Tue, Jun 17, 2008 at 9:26 AM, Curt Hagenlocher wrote: >> Any chance of an Official Pronouncement on this topic? It would help >> us greatly -- even if only to figure out who'll be paying for the next >> round of beer. >> >> On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum wrote: >>> ISTR that we force the epoch to be 1970 on all major platforms -- or >>> perhaps it happens to be 1970 even on Windows when using MS's C >>> runtime. >>> >>> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher wrote: >>>> The documentation for the time module says that "the epoch is the point >>>> where the time starts. On January 1st of that year, at 0 hours, the ``time >>>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the >>>> epoch is, look at gmtime(0)." This confirms that the epoch is >>>> platform-specific. As such, the only legal uses of the timestamp should be >>>> >>>> 1) comparing with another timestamp to determine elapsed time in seconds >>>> 2) passing to another standard Python library function which expects a >>>> timestamp >>>> 3) as a source of randomness. >>>> >>>> However, the following files in the standard library have hardcoded the >>>> assumption that the Python epoch will always be the same as the Unix epoch: >>>> In gzip.py, method GzipFile._write_gzip_header >>>> In tarfile.py, method _Stream._init_write_gz >>>> In uuid.py, function uuid1 >>>> >>>> Additionally, the following files in the standard library have hardcoded the >>>> assumption that the Python epoch will cause timestamps to fall within the >>>> range of a 32-bit unsigned integer value: >>>> In imputil.py, function _compile >>>> In py_compile.py, function compile >>>> >>>> So there's some kind of a potential discrepancy here, albeit a minor one. >>>> This discrepancy can be resolved in one of at least three ways: >>>> >>>> 1) The documentation for the time module is wrong, and the epoch for Python >>>> (at least versions 2.x) should be the Unix epoch. >>>> 2) These library functions are slightly wrong and should be modified by >>>> subtracing an "epoch offset" before doing other calculations. This offset >>>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - >>>> time.timezone". >>>> 3) These library files should be considered part of the platform-specific >>>> implementation, and an alternate platform should provide its own version of >>>> these files if necessary. >>>> >>>> Any thoughts on this? >>>> >>>> From the perspective of implementing IronPython, I'd prefer that the answer >>>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as >>>> possible, while respecting CLR-specific norms for functionality which is >>>> left up to individual implementations. >>>> >>>> -- >>>> Curt Hagenlocher >>>> curt at hagenlocher.org >>>> _______________________________________________ >>>> Python-Dev mailing list >>>> Python-Dev at python.org >>>> http://mail.python.org/mailman/listinfo/python-dev >>>> Unsubscribe: >>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org >>>> >>>> >>> >>> >>> >>> -- >>> --Guido van Rossum (home page: http://www.python.org/~guido/) >>> >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From millbr02 at luther.edu Tue Jun 17 19:49:20 2008 From: millbr02 at luther.edu (Brad Miller) Date: Tue, 17 Jun 2008 12:49:20 -0500 Subject: [Python-Dev] Error building python3.0 under OS X Message-ID: I have been updating the 3.0 source using bzr and after my latest bzr pull I get an error on make install. I'm running 10.5.3. ./configure --enable-framework make make install ... running install_egg_info Writing /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/lib-dynload/Python-3.0a5_-py3.0.egg-info ln -fs "../../../Python" "/Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/config/libpython3.0.a" cd Mac && make installmacsubtree DESTDIR="" DYLD_FRAMEWORK_PATH=/Users/bmiller/src/python/3.0: ../python.exe ./scripts/cachersrc.py -v /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/plat-mac /Library/Frameworks/Python.framework/Versions/3.0/Mac/Tools ../python.exe: can't open file './scripts/cachersrc.py': [Errno 2] No such file or directory make[1]: *** [installmacsubtree] Error 2 make: *** [frameworkinstallmaclib] Error 2 Any hints? Thanks, Brad From guido at python.org Tue Jun 17 19:56:13 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jun 2008 10:56:13 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher wrote: > I don't *feel* anxious, but my doctor *has* been trying to persuade me > to switch to decaf... > > There's no real urgency. The reason this came up is because I just > implemented zlib, which automatically enabled the gzip unit tests. > The gzip tests are failing because the current timestamp can't be > written as a 32-bit value. Why is that? Is it because your epoch is different? If so, I would much prefer the epoch to be 1970. (Maybe this is the resolution you're seeking?) > In order to checkin my changes, I can't > have any failing tests -- so my choices are > > 1) change the IronPython epoch so that the timestamp works for gzip and tarlib > 2) change gzip and tarlib to work with a "less standard" epoch, or > 3) disable the failing test(s) > > ...and I'd rather not resort to #3, if possible. > > On Tue, Jun 17, 2008 at 10:03 AM, Guido van Rossum wrote: >> Can you explain why you are so anxious to get this resolved (apart >> from the beer :-) ? >> >> On Tue, Jun 17, 2008 at 9:26 AM, Curt Hagenlocher wrote: >>> Any chance of an Official Pronouncement on this topic? It would help >>> us greatly -- even if only to figure out who'll be paying for the next >>> round of beer. >>> >>> On Mon, Jun 16, 2008 at 4:38 PM, Guido van Rossum wrote: >>>> ISTR that we force the epoch to be 1970 on all major platforms -- or >>>> perhaps it happens to be 1970 even on Windows when using MS's C >>>> runtime. >>>> >>>> On Mon, Jun 16, 2008 at 4:08 PM, Curt Hagenlocher wrote: >>>>> The documentation for the time module says that "the epoch is the point >>>>> where the time starts. On January 1st of that year, at 0 hours, the ``time >>>>> since the epoch'' is zero. For Unix, the epoch is 1970. To find out what the >>>>> epoch is, look at gmtime(0)." This confirms that the epoch is >>>>> platform-specific. As such, the only legal uses of the timestamp should be >>>>> >>>>> 1) comparing with another timestamp to determine elapsed time in seconds >>>>> 2) passing to another standard Python library function which expects a >>>>> timestamp >>>>> 3) as a source of randomness. >>>>> >>>>> However, the following files in the standard library have hardcoded the >>>>> assumption that the Python epoch will always be the same as the Unix epoch: >>>>> In gzip.py, method GzipFile._write_gzip_header >>>>> In tarfile.py, method _Stream._init_write_gz >>>>> In uuid.py, function uuid1 >>>>> >>>>> Additionally, the following files in the standard library have hardcoded the >>>>> assumption that the Python epoch will cause timestamps to fall within the >>>>> range of a 32-bit unsigned integer value: >>>>> In imputil.py, function _compile >>>>> In py_compile.py, function compile >>>>> >>>>> So there's some kind of a potential discrepancy here, albeit a minor one. >>>>> This discrepancy can be resolved in one of at least three ways: >>>>> >>>>> 1) The documentation for the time module is wrong, and the epoch for Python >>>>> (at least versions 2.x) should be the Unix epoch. >>>>> 2) These library functions are slightly wrong and should be modified by >>>>> subtracing an "epoch offset" before doing other calculations. This offset >>>>> can be calculated as "time.mktime((1970, 1, 1, 0, 0, 0, 3, 1, 0)) - >>>>> time.timezone". >>>>> 3) These library files should be considered part of the platform-specific >>>>> implementation, and an alternate platform should provide its own version of >>>>> these files if necessary. >>>>> >>>>> Any thoughts on this? >>>>> >>>>> From the perspective of implementing IronPython, I'd prefer that the answer >>>>> is 1 or 2 -- but mainly I just want to be as compatible with "the spec" as >>>>> possible, while respecting CLR-specific norms for functionality which is >>>>> left up to individual implementations. >>>>> >>>>> -- >>>>> Curt Hagenlocher >>>>> curt at hagenlocher.org >>>>> _______________________________________________ >>>>> Python-Dev mailing list >>>>> Python-Dev at python.org >>>>> http://mail.python.org/mailman/listinfo/python-dev >>>>> Unsubscribe: >>>>> http://mail.python.org/mailman/options/python-dev/guido%40python.org >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> --Guido van Rossum (home page: http://www.python.org/~guido/) >>>> >>> >> >> >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) >> > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From curt at hagenlocher.org Tue Jun 17 20:09:13 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Tue, 17 Jun 2008 11:09:13 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum wrote: > On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher wrote: >> >> There's no real urgency. The reason this came up is because I just >> implemented zlib, which automatically enabled the gzip unit tests. >> The gzip tests are failing because the current timestamp can't be >> written as a 32-bit value. > > Why is that? Is it because your epoch is different? If so, I would > much prefer the epoch to be 1970. (Maybe this is the resolution you're > seeking?) Yes! Except that I was hoping for something a little stronger, like "the epoch must be 1970" or "the epoch can be anything you want, but you're utterly retarded if you pick something other than 1970". But I'll definitely settle for "much prefer". :) Thanks, -Curt -- Curt Hagenlocher curt at hagenlocher.org From guido at python.org Tue Jun 17 20:34:45 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Jun 2008 11:34:45 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: On Tue, Jun 17, 2008 at 11:09 AM, Curt Hagenlocher wrote: > On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum wrote: >> On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher wrote: >>> >>> There's no real urgency. The reason this came up is because I just >>> implemented zlib, which automatically enabled the gzip unit tests. >>> The gzip tests are failing because the current timestamp can't be >>> written as a 32-bit value. >> >> Why is that? Is it because your epoch is different? If so, I would >> much prefer the epoch to be 1970. (Maybe this is the resolution you're >> seeking?) > > Yes! Except that I was hoping for something a little stronger, like > "the epoch must be 1970" or "the epoch can be anything you want, but > you're utterly retarded if you pick something other than 1970". But > I'll definitely settle for "much prefer". :) I could go with the "utterly retarded" wording. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amauryfa at gmail.com Tue Jun 17 21:44:45 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 17 Jun 2008 21:44:45 +0200 Subject: [Python-Dev] Merging changes before beta Message-ID: Hello, I am in the process of merging changes from trunk to the py3k branch. (If somebody else is already doing it, please tell me quickly!) IMO it would be better to have the beta versions in sync; the 2.6->3.0 migration should not be a regression! I will integrate the changes that are either simple, or that I can understand, or have unit tests. The changes I will skip are (for the moment): - r64114 (gregory.p.smith): a bunch of checks for integer overflows, contributed by Google. This change has many conflicts, I will do it later. - r63828 (mark.hammond): Fix bdist_wininst --user-access-control for win2k For this it is necessary to recompile the various wininst-*.exe, and not all my compilers have the necessary headers or libraries. - r63955 (ronald.oussoren): MacOS X: Enable 4-way universal builds I may try the merge, but without any way to test the result. OTOH, I could go and trust the buildbots. - 64062,64068-64069,64080 (josiah.carlson): Asynchat I may give a try, but it will be tricky. And I don't even know how asynchat works (I just remember it was hard to get it right for python 3.0) - 63207,63210,63218,63403,63469,63537,63597-63598,63601,63604,63617-63618, 63714,63718,63742,63745,63799 (jesus.cea, gregory.p.smith): bsddb module This is a important list of changes, and may require heavy adaptations to the code. issue2887 has been filed for this task. Do you think these tasks should be considered as Release Blockers? Beta1 is close... -- Amaury Forgeot d'Arc From barry at python.org Tue Jun 17 21:59:46 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Jun 2008 15:59:46 -0400 Subject: [Python-Dev] [Python-3000] Merging changes before beta In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 17, 2008, at 3:44 PM, Amaury Forgeot d'Arc wrote: > I am in the process of merging changes from trunk to the py3k branch. > (If somebody else is already doing it, please tell me quickly!) BTW, I will be hanging out on #python-dev @ freenode from now until tomorrow night, so if you have questions feel free to ping me. My userid is 'barry' (so creative ) and I keep generally UTC-0400 time. > The changes I will skip are (for the moment): > > - r64114 (gregory.p.smith): a bunch of checks for integer overflows, > contributed by Google. > This change has many conflicts, I will do it later. Not a blocker. > - r63828 (mark.hammond): Fix bdist_wininst --user-access-control for > win2k > For this it is necessary to recompile the various wininst-*.exe, > and not all my compilers have the necessary headers or libraries. Not a blocker. > - r63955 (ronald.oussoren): MacOS X: Enable 4-way universal builds > I may try the merge, but without any way to test the result. > OTOH, I could go and trust the buildbots. Only if you're prepared to back it out if they go red. Not a blocker. > - 64062,64068-64069,64080 (josiah.carlson): Asynchat > I may give a try, but it will be tricky. And I don't even know how > asynchat works > (I just remember it was hard to get it right for python 3.0) Hmm. I don't know the details of these. > - > 63207,63210,63218,63403,63469,63537,63597 > -63598,63601,63604,63617-63618, > 63714,63718,63742,63745,63799 (jesus.cea, gregory.p.smith): bsddb > module > This is a important list of changes, and may require heavy adaptations > to the code. > issue2887 has been filed for this task. Not a blocker. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFgXs3EjvBPtnXfVAQJdiQP8CIahHlmiwep5ljm0gakwYzEI7LJ1e7+0 5HXNqClrhntIP+wjR8BPR9oNTp04D0Y35v8ZZcZDh0t/2LtboP6fySO4eVmYcoma p5/Ld6xPoaSRfy15jVb9xKx4AsegKNZlOdJWpZs9+jkYK5x5mcflns0Nn4KoRKwS r3ExyMj5YIQ= =liS1 -----END PGP SIGNATURE----- From tnelson at onresolve.com Wed Jun 18 07:48:40 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 18 Jun 2008 06:48:40 +0100 Subject: [Python-Dev] Progress on switching Windows build to Berkeley DB 4.7.25... Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275B@exchange.onresolve.com> Hi all, Jesus, apologies that this has taken so long for me to get back to, I've been completely and utterly swamped with client work the past few weeks. However, thanks to a couple of hours spare at Detroit airport yesterday, I was finally able to make some progress on updating the Windows Berkeley DB build to 4.7.25. I've checked in the work I've done so far to branches/tnelson-trunk-bsddb-47-upgrade. One thing I wanted to double check with you is the following change: Modified: python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py ============================================================================== --- python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py (original) +++ python/branches/tnelson-trunk-bsddb-47-upgrade/Lib/bsddb/test/test_replication.py Wed Jun 18 06:13:44 2008 @@ -94,7 +94,7 @@ # The timeout is necessary in BDB 4.5, since DB_EVENT_REP_STARTUPDONE # is not generated if the master has no new transactions. # This is solved in BDB 4.6 (#15542). - timeout = time.time()+2 + timeout = time.time()+10 while (time.time()= (4,6) : Basically, when using +2, the test failed every so often when running the entire test_bsddb3 suite. I picked 10 arbitrarily; it improves things, but it's still not 100%, I still encounter the following failure every so often: ====================================================================== ERROR: test01_basic_replication (bsddb.test.test_replication.DBReplicationManager) ---------------------------------------------------------------------- Traceback (most recent call last): File "s:\src\svn+ssh\pythondev at svn.python.org\python\branches\tnelson-trunk-bsddb-47-upgrade\lib\bsddb\test\test_replication.py", line 101, in setUp self.assertTrue(time.time() References: <20080617230919.B2EA11E4004@bag.python.org> <48589636.202@v.loewis.de> <4858B166.5070300@gmail.com> Message-ID: <4858B829.2060002@v.loewis.de> >> Please never commit to python/trunk/Lib/lib2to3. >> Instead, commit to sandbox/trunk/2to3, and merge >> that using svnmerge. > > Could you explain that policy? Why isn't 2to3 maintained in the > trunk but in the sandbox? Or maybe more general: Where can I find > more information on the svn directory layout? It started off as a sandbox project, also with the intent of releasing it separately. At PyCon 2008, I restructured it introducing lib2to3, which I then copied into the trunk, and from there into the 3k branch. The official sources still remain in the sandbox: a) for historical reasons, because that's where the main 2to3 contributors commit to (who might not even have a trunk checkout, only a 2.5 installation and a 3.0 checkout) b) because the 2to3 sources contain a few files that don't really "belong" into the trunk (although a place for them could probably be found), such as find_pattern utility. c) to continue allowing stand-alone releases; IIUC, Collin Winter wants to change this package into a general Python rewriting library, which might be useful even in a 2.5 context. I don't think there can be much said about the layout of the repository: the top-level directory contains various projects (including python), and sandbox contains "stuff". Within the Python tree, there are really not that many exceptions to the rule "trunk is where all new features go", apart from the 3k branch, of course. One class of exceptions is libraries that need to stay backwards compatible, see PEP 291. The other class of exceptions is libraries whose official sources are outside trunk, namely (possibly incomplete): - lib2to3 - xml.etree, and Modules/_elementtree.c - bsddb (recently) - _multiprocessing ??? - Modules/zlib Those have special commit constraints, and the maintainer should be contacted if you don't know what the constraints are. A few other packages are also separately released but don't have such constraints: - ctypes - sqlite For these packages, the maintainers will pickup and changes themselves and merge them into whatever external repository they may use. Regards, Martin From pete.forman at westerngeco.com Wed Jun 18 11:50:59 2008 From: pete.forman at westerngeco.com (Pete Forman) Date: Wed, 18 Jun 2008 10:50:59 +0100 Subject: [Python-Dev] Epoch and Platform References: Message-ID: "Guido van Rossum" writes: > On Tue, Jun 17, 2008 at 11:09 AM, Curt Hagenlocher > wrote: >> On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum >> wrote: >>> On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher >>> wrote: >>>> >>>> There's no real urgency. The reason this came up is because I >>>> just implemented zlib, which automatically enabled the gzip unit >>>> tests. The gzip tests are failing because the current timestamp >>>> can't be written as a 32-bit value. >>> >>> Why is that? Is it because your epoch is different? If so, I would >>> much prefer the epoch to be 1970. (Maybe this is the resolution >>> you're seeking?) >> >> Yes! Except that I was hoping for something a little stronger, like >> "the epoch must be 1970" or "the epoch can be anything you want, but >> you're utterly retarded if you pick something other than 1970". But >> I'll definitely settle for "much prefer". :) > > I could go with the "utterly retarded" wording. :-) The POSIX (UNIX) rules for handling time are laid out clearly. Not everyone is happy with how leap seconds work but at least they can be dealt with by those who need to. Most people will use local time or UTC and other time scales are possible with a bit of effort. If you were to allow an epoch earlier than 1970 then ambiguity is more likely to sneak in. Another reason not to allow an epoch of 1900 is that Microsoft introduced a bug in its leap year algorithm in Excel. The better (less bad) way of using that epoch is to propagate that bug. According to the Pythonic way you should not have to look before you leap. ;-) +1 for strengthening the definition of epoch in the time module to 1970-01-01T00:00:00Z for all platforms. -- Pete Forman -./\.- Disclaimer: This post is originated WesternGeco -./\.- by myself and does not represent pete.forman at westerngeco.com -./\.- the opinion of Schlumberger or http://petef.22web.net -./\.- WesternGeco. From barry at python.org Wed Jun 18 13:45:47 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Jun 2008 07:45:47 -0400 Subject: [Python-Dev] Another run at beta one Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Well, I'm going to try again tonight to release beta one for Python 2.6 and 3.0. I'm planning on starting the release at 6pm US/Eastern or 2200 UTC. Right now the Python 2.6 buildbots look pretty good. We have /one/ green buildbot for Python 3.0. That's better than last week at least, but it would be great to get a few more. I know folks have been working hard at this, and I really appreciate it. I think we're looking fairly good on the release critical bugs. I'll clear 3089 and 3009 tonight, and check locally on 3088 over the day. If you're looking for a way to help out, see if you can turn any of the other 3.0 buildbots green. I will be hanging out on #python-dev all day, but ping my nick (barry) if you need my attention. Thanks, and let's get it done today! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFj1bHEjvBPtnXfVAQLrvAP/ftfpN50HbeimRoK2WhX4RcuN9G81IfbJ 4+CXdjH7WLC/wwKvNp1Q6qc3vHWTqPgA7jF7yowU/a/3FretkzUPGNmtjoWvfQzM BL1ktG6fh298+KRJTSTXGeNSEzjwBg2GryyQJV5xmwsUMiwvrI4OE3w8O6eE5AkS gFmU86oZSFw= =SiNl -----END PGP SIGNATURE----- From jnoller at gmail.com Wed Jun 18 13:54:17 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Jun 2008 07:54:17 -0400 Subject: [Python-Dev] [Python-3000] Another run at beta one In-Reply-To: References: Message-ID: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com> On Wed, Jun 18, 2008 at 7:45 AM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Well, I'm going to try again tonight to release beta one for Python 2.6 and > 3.0. I'm planning on starting the release at 6pm US/Eastern or 2200 UTC. > > Right now the Python 2.6 buildbots look pretty good. We have /one/ green > buildbot for Python 3.0. That's better than last week at least, but it > would be great to get a few more. I know folks have been working hard at > this, and I really appreciate it. > > I think we're looking fairly good on the release critical bugs. I'll clear > 3089 and 3009 tonight, and check locally on 3088 over the day. > > If you're looking for a way to help out, see if you can turn any of the > other 3.0 buildbots green. I will be hanging out on #python-dev all day, > but ping my nick (barry) if you need my attention. > > Thanks, and let's get it done today! > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iQCVAwUBSFj1bHEjvBPtnXfVAQLrvAP/ftfpN50HbeimRoK2WhX4RcuN9G81IfbJ > 4+CXdjH7WLC/wwKvNp1Q6qc3vHWTqPgA7jF7yowU/a/3FretkzUPGNmtjoWvfQzM > BL1ktG6fh298+KRJTSTXGeNSEzjwBg2GryyQJV5xmwsUMiwvrI4OE3w8O6eE5AkS > gFmU86oZSFw= > =SiNl > -----END PGP SIGNATURE----- > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/jnoller%40gmail.com > Please let me know about 3088 - I've disabled the more unreliable tests until I can give the entire suite some TLC. I'm still trying to work around/identify the "Function not Implemented" error in issue3111 - this happens to be failing the tests on a few of the linux boxes. I suspected it was due to a combination of chroot + lack of /dev/shm on some of the build bots, ergo a patch put in by Ben last night to skip the tests on machines lacking the device, but it doesn't seem to be skipping/the device exists. -jesse From jnoller at gmail.com Wed Jun 18 16:30:50 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Jun 2008 10:30:50 -0400 Subject: [Python-Dev] [Python-3000] Another run at beta one In-Reply-To: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com> References: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com> Message-ID: <4222a8490806180730i616ec5fdyfec8478a80ec6814@mail.gmail.com> On Wed, Jun 18, 2008 at 7:54 AM, Jesse Noller wrote: > On Wed, Jun 18, 2008 at 7:45 AM, Barry Warsaw wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Well, I'm going to try again tonight to release beta one for Python 2.6 and >> 3.0. I'm planning on starting the release at 6pm US/Eastern or 2200 UTC. >> >> Right now the Python 2.6 buildbots look pretty good. We have /one/ green >> buildbot for Python 3.0. That's better than last week at least, but it >> would be great to get a few more. I know folks have been working hard at >> this, and I really appreciate it. >> >> I think we're looking fairly good on the release critical bugs. I'll clear >> 3089 and 3009 tonight, and check locally on 3088 over the day. >> >> If you're looking for a way to help out, see if you can turn any of the >> other 3.0 buildbots green. I will be hanging out on #python-dev all day, >> but ping my nick (barry) if you need my attention. >> >> Thanks, and let's get it done today! >> - -Barry >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.9 (Darwin) >> >> iQCVAwUBSFj1bHEjvBPtnXfVAQLrvAP/ftfpN50HbeimRoK2WhX4RcuN9G81IfbJ >> 4+CXdjH7WLC/wwKvNp1Q6qc3vHWTqPgA7jF7yowU/a/3FretkzUPGNmtjoWvfQzM >> BL1ktG6fh298+KRJTSTXGeNSEzjwBg2GryyQJV5xmwsUMiwvrI4OE3w8O6eE5AkS >> gFmU86oZSFw= >> =SiNl >> -----END PGP SIGNATURE----- >> _______________________________________________ >> Python-3000 mailing list >> Python-3000 at python.org >> http://mail.python.org/mailman/listinfo/python-3000 >> Unsubscribe: >> http://mail.python.org/mailman/options/python-3000/jnoller%40gmail.com >> Update: Most of the trunk bots went green post r64375, so I merged the change over to py3k. The change aggressively skips the MP package tests if RLock instantiation throws an OSError. -jesse From guido at python.org Wed Jun 18 18:21:46 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Jun 2008 09:21:46 -0700 Subject: [Python-Dev] Epoch and Platform In-Reply-To: References: Message-ID: Leap seconds? Leap seconds aren't supposed to exist when looking at POSIX timestamps. They don't exist when I look at my watch either, and that's what I expect from any time-keeping device except those for use by astronomers. On Wed, Jun 18, 2008 at 2:50 AM, Pete Forman wrote: > "Guido van Rossum" writes: > > On Tue, Jun 17, 2008 at 11:09 AM, Curt Hagenlocher > > wrote: >>> On Tue, Jun 17, 2008 at 10:56 AM, Guido van Rossum >>> wrote: >>>> On Tue, Jun 17, 2008 at 10:40 AM, Curt Hagenlocher >>>> wrote: >>>>> >>>>> There's no real urgency. The reason this came up is because I >>>>> just implemented zlib, which automatically enabled the gzip unit >>>>> tests. The gzip tests are failing because the current timestamp >>>>> can't be written as a 32-bit value. >>>> >>>> Why is that? Is it because your epoch is different? If so, I would >>>> much prefer the epoch to be 1970. (Maybe this is the resolution >>>> you're seeking?) >>> >>> Yes! Except that I was hoping for something a little stronger, like >>> "the epoch must be 1970" or "the epoch can be anything you want, but >>> you're utterly retarded if you pick something other than 1970". But >>> I'll definitely settle for "much prefer". :) >> > > I could go with the "utterly retarded" wording. :-) > > The POSIX (UNIX) rules for handling time are laid out clearly. Not > everyone is happy with how leap seconds work but at least they can be > dealt with by those who need to. Most people will use local time or > UTC and other time scales are possible with a bit of effort. If you > were to allow an epoch earlier than 1970 then ambiguity is more likely > to sneak in. > > Another reason not to allow an epoch of 1900 is that Microsoft > introduced a bug in its leap year algorithm in Excel. The better > (less bad) way of using that epoch is to propagate that bug. > > According to the Pythonic way you should not have to look before you > leap. ;-) > > +1 for strengthening the definition of epoch in the time module to > 1970-01-01T00:00:00Z for all platforms. > -- > Pete Forman -./\.- Disclaimer: This post is originated > WesternGeco -./\.- by myself and does not represent > pete.forman at westerngeco.com -./\.- the opinion of Schlumberger or > http://petef.22web.net -./\.- WesternGeco. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tnelson at onresolve.com Wed Jun 18 18:20:14 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 18 Jun 2008 17:20:14 +0100 Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness Message-ID: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> I gave my Windows buildbots a little bit of TLC last night. This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not: def test_listener_client(self): for family in self.connection.families: l = self.connection.Listener(family=family) p = self.Process(target=self._test, args=(l.address,)) p.set_daemon(True) p.start() conn = l.accept() self.assertEqual(conn.recv(), 'hello') p.join() l.close() The wedging will be a result of that accept() call. Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered. (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.) Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows: test_multiprocessing command timed out: 1200 seconds without output SIGKILL failed to kill process using fake rc=-1 program finished with exit code -1 remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last): Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process ] (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).) Trent. From amauryfa at gmail.com Wed Jun 18 18:35:26 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 18 Jun 2008 18:35:26 +0200 Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness In-Reply-To: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> Message-ID: Hello, 2008/6/18 Trent Nelson : > I gave my Windows buildbots a little bit of TLC last night. This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not: > > def test_listener_client(self): > for family in self.connection.families: > l = self.connection.Listener(family=family) > p = self.Process(target=self._test, args=(l.address,)) > p.set_daemon(True) > p.start() > conn = l.accept() > self.assertEqual(conn.recv(), 'hello') > p.join() > l.close() > > The wedging will be a result of that accept() call. Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered. (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.) > > Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows: > > test_multiprocessing > > command timed out: 1200 seconds without output > SIGKILL failed to kill process > using fake rc=-1 > program finished with exit code -1 > remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last): > Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process > ] > > (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).) I just found the cause of the problem ten minutes ago: It seems that when a socket listens on the address "127.0.0.1" or "localhost", another process cannot connect to it using the machine's name (even from the same machine). The best seems to listen with the empty address "". Index: Lib/multiprocessing/connection.py =================================================================== --- Lib/multiprocessing/connection.py (revision 64374) +++ Lib/multiprocessing/connection.py (working copy) @@ -49,7 +49,7 @@ Return an arbitrary free address for the given family ''' if family == 'AF_INET': - return ('localhost', 0) + return ('', 0) elif family == 'AF_UNIX': return tempfile.mktemp(prefix='listener-', dir=get_temp_dir()) elif family == 'AF_PIPE': And the test started to pass for me. Can you please check this in if it works; I don't have svn access for the moment. -- Amaury Forgeot d'Arc From jnoller at gmail.com Wed Jun 18 18:39:30 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Jun 2008 12:39:30 -0400 Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness In-Reply-To: References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> Message-ID: <4222a8490806180939xc1e5340g560f35c2b639321e@mail.gmail.com> On Wed, Jun 18, 2008 at 12:35 PM, Amaury Forgeot d'Arc wrote: > Hello, > > 2008/6/18 Trent Nelson : >> I gave my Windows buildbots a little bit of TLC last night. This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not: >> >> def test_listener_client(self): >> for family in self.connection.families: >> l = self.connection.Listener(family=family) >> p = self.Process(target=self._test, args=(l.address,)) >> p.set_daemon(True) >> p.start() >> conn = l.accept() >> self.assertEqual(conn.recv(), 'hello') >> p.join() >> l.close() >> >> The wedging will be a result of that accept() call. Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered. (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.) >> >> Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows: >> >> test_multiprocessing >> >> command timed out: 1200 seconds without output >> SIGKILL failed to kill process >> using fake rc=-1 >> program finished with exit code -1 >> remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last): >> Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process >> ] >> >> (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).) > > I just found the cause of the problem ten minutes ago: > It seems that when a socket listens on the address "127.0.0.1" or > "localhost", another process cannot connect to it using the machine's > name (even from the same machine). > The best seems to listen with the empty address "". > > Index: Lib/multiprocessing/connection.py > =================================================================== > --- Lib/multiprocessing/connection.py (revision 64374) > +++ Lib/multiprocessing/connection.py (working copy) > @@ -49,7 +49,7 @@ > Return an arbitrary free address for the given family > ''' > if family == 'AF_INET': > - return ('localhost', 0) > + return ('', 0) > elif family == 'AF_UNIX': > return tempfile.mktemp(prefix='listener-', dir=get_temp_dir()) > elif family == 'AF_PIPE': > > And the test started to pass for me. > Can you please check this in if it works; I don't have svn access for > the moment. > > -- > Amaury Forgeot d'Arc I am testing the patch locally now. From solipsis at pitrou.net Wed Jun 18 18:40:02 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Jun 2008 16:40:02 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?test=5Fmultiprocessing=3Atest=5Flistener?= =?utf-8?q?=5Fclient=09flakiness?= References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> Message-ID: Amaury Forgeot d'Arc gmail.com> writes: > if family == 'AF_INET': > - return ('localhost', 0) > + return ('', 0) > elif family == 'AF_UNIX': > return tempfile.mktemp(prefix='listener-', dir=get_temp_dir()) > elif family == 'AF_PIPE': Doesn't it have important security implications? From exarkun at divmod.com Wed Jun 18 18:41:36 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 18 Jun 2008 12:41:36 -0400 Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness In-Reply-To: Message-ID: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> On Wed, 18 Jun 2008 18:35:26 +0200, Amaury Forgeot d'Arc wrote: > [snip] > >I just found the cause of the problem ten minutes ago: >It seems that when a socket listens on the address "127.0.0.1" or >"localhost", another process cannot connect to it using the machine's >name (even from the same machine). That's because when you try to connect to A:B you won't connect to a server listening on X:B - somewhat by design. Changing the test to listen on A:B and X:B might fix it, but so would changing it to connect to the same address it listens to. >The best seems to listen with the empty address "". This will cause it to listen on all available interfaces, including public ones. It's rather unlikely that someone from the internet will connect to the port while the test suite is running and use it to do terrible things to you, but it's not /impossible/. I'd suggest changing the client to connect to "127.0.0.1" or "localhost", instead. Jean-Paul From amauryfa at gmail.com Wed Jun 18 18:48:20 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 18 Jun 2008 18:48:20 +0200 Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness In-Reply-To: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> Message-ID: Hello, 2008/6/18 Jean-Paul Calderone : > On Wed, 18 Jun 2008 18:35:26 +0200, Amaury Forgeot d'Arc > wrote: >> >> [snip] >> >> I just found the cause of the problem ten minutes ago: >> It seems that when a socket listens on the address "127.0.0.1" or >> "localhost", another process cannot connect to it using the machine's >> name (even from the same machine). > > That's because when you try to connect to A:B you won't connect to a > server listening on X:B - somewhat by design. Changing the test to > listen on A:B and X:B might fix it, but so would changing it to connect > to the same address it listens to. > >> The best seems to listen with the empty address "". > > This will cause it to listen on all available interfaces, including > public ones. It's rather unlikely that someone from the internet > will connect to the port while the test suite is running and use it > to do terrible things to you, but it's not /impossible/. > > I'd suggest changing the client to connect to "127.0.0.1" or "localhost", > instead. OK, I am not a tcp expert. How about the following patch? It seems to me that it is wrong for the server to lie about the address it listens to. Index: ../../Lib/multiprocessing/connection.py =================================================================== --- ../../Lib/multiprocessing/connection.py (revision 64378) +++ ../../Lib/multiprocessing/connection.py (working copy) @@ -216,8 +216,6 @@ self._socket.bind(address) self._socket.listen(backlog) address = self._socket.getsockname() - if type(address) is tuple: - address = (socket.getfqdn(address[0]),) + address[1:] self._address = address self._family = family self._last_accepted = None -- Amaury Forgeot d'Arc From g.brandl at gmx.net Wed Jun 18 18:56:47 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 18 Jun 2008 18:56:47 +0200 Subject: [Python-Dev] Opcode frequency Message-ID: Hi, Maciej Fijalkowski did an opcode analysis for PyPy, it also shows the relative frequency of opcodes following a specifc one: http://codespeak.net/svn/user/fijal/opcodes.txt Might it make sense to add more PREDICT()ions based on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR? Georg From pete.forman at westerngeco.com Wed Jun 18 18:58:18 2008 From: pete.forman at westerngeco.com (Pete Forman) Date: Wed, 18 Jun 2008 17:58:18 +0100 Subject: [Python-Dev] Epoch and Platform References: Message-ID: "Guido van Rossum" writes: > Leap seconds? Leap seconds aren't supposed to exist when looking > at POSIX timestamps. They don't exist when I look at my watch > either, and that's what I expect from any time-keeping device > except those for use by astronomers. That's pretty much what I was saying. The POSIX rule for Seconds Since the Epoch explicitly excludes leap seconds. Astronomers and the like then know what further adjustments are required for the their needs. The rest of us get along with the assumption that there are 86400 seconds in every day. -- Pete Forman -./\.- Disclaimer: This post is originated WesternGeco -./\.- by myself and does not represent pete.forman at westerngeco.com -./\.- the opinion of Schlumberger or http://petef.22web.net -./\.- WesternGeco. From solipsis at pitrou.net Wed Jun 18 19:51:56 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Jun 2008 17:51:56 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?test=5Fmultiprocessing=3Atest=5Flistener?= =?utf-8?q?=5Fclient=09flakiness?= References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> Message-ID: Amaury Forgeot d'Arc gmail.com> writes: > > OK, I am not a tcp expert. > How about the following patch? > It seems to me that it is wrong for the server to lie about the > address it listens to. Maybe I'm missing something but... if the client part defaults to connecting to "localhost" in TCP mode, why doesn't the server part also default to listening on "localhost" in TCP mode? Having different address detection algorithms on both ends sounds like a recipe for bugs like this. In any case, perhaps one should ask guidance to the original author before making important changes like this one. From jnoller at gmail.com Wed Jun 18 19:55:12 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Jun 2008 13:55:12 -0400 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> Message-ID: <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> On Wed, Jun 18, 2008 at 1:51 PM, Antoine Pitrou wrote: > Amaury Forgeot d'Arc gmail.com> writes: >> >> OK, I am not a tcp expert. >> How about the following patch? >> It seems to me that it is wrong for the server to lie about the >> address it listens to. > > Maybe I'm missing something but... if the client part defaults to connecting > to "localhost" in TCP mode, why doesn't the server part also default to > listening on "localhost" in TCP mode? Having different address detection > algorithms on both ends sounds like a recipe for bugs like this. > > In any case, perhaps one should ask guidance to the original author before > making important changes like this one. The server component *is* listening to localhost by default. The client in the code I pasted has the server's .address attribute passed into it to tell the client where to connect to. -jesse From jnoller at gmail.com Wed Jun 18 19:59:08 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Jun 2008 13:59:08 -0400 Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness In-Reply-To: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> Message-ID: <4222a8490806181059m407ba440l3cfd0689d7e2d2c5@mail.gmail.com> On Wed, Jun 18, 2008 at 12:20 PM, Trent Nelson wrote: > I gave my Windows buildbots a little bit of TLC last night. This little chestnut in test_multiprocessing.py around line 1346 is causing my buildbots to wedge more often than not: > > def test_listener_client(self): > for family in self.connection.families: > l = self.connection.Listener(family=family) > p = self.Process(target=self._test, args=(l.address,)) > p.set_daemon(True) > p.start() > conn = l.accept() > self.assertEqual(conn.recv(), 'hello') > p.join() > l.close() > > The wedging will be a result of that accept() call. Not knowing anything about the module or the test suite, I can only assume that there's a race condition introduced between when the subprocess attempts to connect to the listener, versus when the l.accept() call is actually entered. (On the basis that a race condition would explain why sometimes it wedges and sometimes it doesn't.) > > Just FYI, the error in the buildbot log (http://www.python.org/dev/buildbot/all/x86%20W2k8%20trunk/builds/810/step-test/0) when this occurs is as follows: > > test_multiprocessing > > command timed out: 1200 seconds without output > SIGKILL failed to kill process > using fake rc=-1 > program finished with exit code -1 > remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last): > Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process > ] > > (The fact it can't be killed cleanly is a bug in Twisted's signalProcess('KILL') method, which doesn't work against Python processes that have entered accept() calls on Windows (which present the 'wedged' behaviour and have to be forcibly killed with OpenProcess/TerminateProcess).) > > Trent. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com > Trent, I commented out the test causing the issue for now (the entire suite is being revisited post beta-1) See r64378 -jesse From solipsis at pitrou.net Wed Jun 18 20:04:45 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Jun 2008 18:04:45 +0000 (UTC) Subject: [Python-Dev] Opcode frequency References: Message-ID: Hi, > Maciej Fijalkowski did an opcode analysis for PyPy, > it also shows the relative frequency of opcodes following a > specifc one: > > http://codespeak.net/svn/user/fijal/opcodes.txt Nice, but we have to be careful here: what is the tested workload? For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would always (99%) be followed by STORE_FAST. I'm not even sure we're talking about the same VM/compiler. What are CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-) > Might it make sense to add more PREDICT()ions based > on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR? This particular one sounds fine, yes. Another approach is to create opcodes which better reflect actual usage. For example part of the following patch (*) was to create opcodes fusing JUMP_IF_{TRUE,FALSE} and POP_TOP. This is because in most cases, you want to pop the comparison result regardless of its value (true or false), which nowadays produces the following kind of code: JUMP_IF_FALSE POP_TOP [... do stuff and then ...] JUMP_FORWARD branch_for_false: POP_TOP branch_for_false_after_pop_top: [etc.] With a JUMP_IF_FALSE which also pops the pop of the stack, this gives: JUMP_IF_FALSE [... do stuff and then ...] branch_for_false: [etc.] Prediction doesn't help you in this case. (*) http://bugs.python.org/issue2459 Regards Antoine. From solipsis at pitrou.net Wed Jun 18 20:09:51 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Jun 2008 18:09:51 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?test=5Fmultiprocessing=3Atest=5Flistener?= =?utf-8?q?=5Fclient_flakiness?= References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> Message-ID: Jesse Noller gmail.com> writes: > > The server component *is* listening to localhost by default. The > client in the code I pasted has the server's .address attribute passed > into it to tell the client where to connect to. So I take it that the .address attribute is different from "localhost", but why? Is there any point in trying to be clever here? From jnoller at gmail.com Wed Jun 18 20:32:35 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 18 Jun 2008 14:32:35 -0400 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> Message-ID: <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> On Wed, Jun 18, 2008 at 2:09 PM, Antoine Pitrou wrote: > Jesse Noller gmail.com> writes: >> >> The server component *is* listening to localhost by default. The >> client in the code I pasted has the server's .address attribute passed >> into it to tell the client where to connect to. > > So I take it that the .address attribute is different from "localhost", > but why? Is there any point in trying to be clever here? No, it is *not* different than localhost - it *is* localhost as that is the object.address of the server. From g.brandl at gmx.net Wed Jun 18 21:45:07 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 18 Jun 2008 21:45:07 +0200 Subject: [Python-Dev] Opcode frequency In-Reply-To: References: Message-ID: Antoine Pitrou schrieb: > Hi, > >> Maciej Fijalkowski did an opcode analysis for PyPy, >> it also shows the relative frequency of opcodes following a >> specifc one: >> >> http://codespeak.net/svn/user/fijal/opcodes.txt > > Nice, but we have to be careful here: what is the tested workload? > For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would > always (99%) be followed by STORE_FAST. I'm sorry, I should have given a reference to the original message: http://codespeak.net/pipermail/pypy-dev/2008q2/004666.html > I'm not even sure we're talking about the same VM/compiler. What are > CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-) He used the PyPy compiler, of course. >> Might it make sense to add more PREDICT()ions based >> on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR? > > This particular one sounds fine, yes. > > Another approach is to create opcodes which better reflect actual usage. > For example part of the following patch (*) was to create opcodes fusing > JUMP_IF_{TRUE,FALSE} and POP_TOP. This is because in most cases, you want > to pop the comparison result regardless of its value (true or false), which > nowadays produces the following kind of code: > > JUMP_IF_FALSE > POP_TOP > [... do stuff and then ...] > JUMP_FORWARD > branch_for_false: > POP_TOP > branch_for_false_after_pop_top: > [etc.] > > With a JUMP_IF_FALSE which also pops the pop of the stack, this gives: > > JUMP_IF_FALSE > [... do stuff and then ...] > branch_for_false: > [etc.] > > Prediction doesn't help you in this case. Agreed; is there a way for a JUMP to end up without popping the top after jumping? It would have to do a DUP first then. Georg From solipsis at pitrou.net Wed Jun 18 23:12:15 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Jun 2008 21:12:15 +0000 (UTC) Subject: [Python-Dev] Opcode frequency References: Message-ID: Georg Brandl gmx.net> writes: > Agreed; is there a way for a JUMP to end up without popping the top > after jumping? It would have to do a DUP first then. For example: >>> def f(x, y): ... return x and y ... >>> import dis >>> dis.dis(f) 2 0 LOAD_FAST 0 (x) 3 JUMP_IF_FALSE 4 (to 10) 6 POP_TOP 7 LOAD_FAST 1 (y) >> 10 RETURN_VALUE From cfbolz at gmx.de Wed Jun 18 23:20:50 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 18 Jun 2008 23:20:50 +0200 Subject: [Python-Dev] Opcode frequency In-Reply-To: References: Message-ID: <48597C32.6030607@gmx.de> Georg Brandl wrote: > Antoine Pitrou schrieb: >>> Maciej Fijalkowski did an opcode analysis for PyPy, >>> it also shows the relative frequency of opcodes following a >>> specifc one: >>> >>> http://codespeak.net/svn/user/fijal/opcodes.txt >> >> Nice, but we have to be careful here: what is the tested workload? >> For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would >> always (99%) be followed by STORE_FAST. > > I'm sorry, I should have given a reference to the original message: > http://codespeak.net/pipermail/pypy-dev/2008q2/004666.html > >> I'm not even sure we're talking about the same VM/compiler. What are >> CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-) > > He used the PyPy compiler, of course. Just for the reference, PyPy uses exactly CPython's opcodes. Then there are some completely optional optimizations which add opcodes (like CALL_LIKELY_BUILTIN and LOOKUP_METHOD) that can be enabled at compile time. Cheers, Carl Friedrich Bolz From cfbolz at gmx.de Wed Jun 18 23:20:50 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 18 Jun 2008 23:20:50 +0200 Subject: [Python-Dev] Opcode frequency In-Reply-To: References: Message-ID: <48597C32.6030607@gmx.de> Georg Brandl wrote: > Antoine Pitrou schrieb: >>> Maciej Fijalkowski did an opcode analysis for PyPy, >>> it also shows the relative frequency of opcodes following a >>> specifc one: >>> >>> http://codespeak.net/svn/user/fijal/opcodes.txt >> >> Nice, but we have to be careful here: what is the tested workload? >> For example, I find it hard to believe that CALL_FUNCTION_VAR_KW would >> always (99%) be followed by STORE_FAST. > > I'm sorry, I should have given a reference to the original message: > http://codespeak.net/pipermail/pypy-dev/2008q2/004666.html > >> I'm not even sure we're talking about the same VM/compiler. What are >> CALL_LIKELY_BUILTIN and LOOKUP_METHOD? :-) > > He used the PyPy compiler, of course. Just for the reference, PyPy uses exactly CPython's opcodes. Then there are some completely optional optimizations which add opcodes (like CALL_LIKELY_BUILTIN and LOOKUP_METHOD) that can be enabled at compile time. Cheers, Carl Friedrich Bolz From amauryfa at gmail.com Wed Jun 18 23:36:22 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 18 Jun 2008 23:36:22 +0200 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> Message-ID: 2008/6/18 Jesse Noller : > On Wed, Jun 18, 2008 at 2:09 PM, Antoine Pitrou wrote: >> Jesse Noller gmail.com> writes: >>> >>> The server component *is* listening to localhost by default. The >>> client in the code I pasted has the server's .address attribute passed >>> into it to tell the client where to connect to. >> >> So I take it that the .address attribute is different from "localhost", >> but why? Is there any point in trying to be clever here? > > No, it is *not* different than localhost - it *is* localhost as that > is the object.address of the server. Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name of the machine. This is the name that was stored in server.address. Hence my patch that simply remove the call to getfqdn. -- Amaury Forgeot d'Arc From ncoghlan at gmail.com Thu Jun 19 01:03:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Jun 2008 09:03:08 +1000 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> Message-ID: <4859942C.9010009@gmail.com> Amaury Forgeot d'Arc wrote: > 2008/6/18 Jesse Noller : >> On Wed, Jun 18, 2008 at 2:09 PM, Antoine Pitrou wrote: >>> Jesse Noller gmail.com> writes: >>>> The server component *is* listening to localhost by default. The >>>> client in the code I pasted has the server's .address attribute passed >>>> into it to tell the client where to connect to. >>> So I take it that the .address attribute is different from "localhost", >>> but why? Is there any point in trying to be clever here? >> No, it is *not* different than localhost - it *is* localhost as that >> is the object.address of the server. > > Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name > of the machine. > This is the name that was stored in server.address. > Hence my patch that simply remove the call to getfqdn. Point of interest: I was involved in developing an application where we gave up on trying to persuade getfqdn() to return reliable results on Windows machines - regardless of which local network interface you use to do the lookup, Windows 2k3 is likely to return an answer of 'computername.computerdomain' without bothering to consult the local DNS server (annoyingly, it used to do this right on previous Windows versions). So I'm inclined to consider any code which expects a useful answer out of getfqdn() to be non-portable at best and outright broken at worst. We worked around the problem by only doing forward DNS lookups and always working with addresses in the IP domain. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Thu Jun 19 01:03:19 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 18 Jun 2008 16:03:19 -0700 Subject: [Python-Dev] Opcode frequency References: Message-ID: > Maciej Fijalkowski did an opcode analysis for PyPy, > it also shows the relative frequency of opcodes following a > specifc one: > > http://codespeak.net/svn/user/fijal/opcodes.txt > > Might it make sense to add more PREDICT()ions based > on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR? This particular one might be okay. What code generates it? Are there other possible successors to BUILD_SLICE? What code generates those? There were lots of other opcode pairings that were previously studied and rejected. You're going over well traveled ground. Also, opcode analysis is a somewhat hazardous undertaking. Dynamic analysis takes into account which codes tend to occur more often inside loops but it is *very* sensitive to who wrote the app and their coding style. These results always have to be taken with a grain of salt. Raymond From barry at python.org Thu Jun 19 01:38:19 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Jun 2008 19:38:19 -0400 Subject: [Python-Dev] releasing betas tonight Message-ID: <8AE8BB5B-447B-4D31-854F-671AE902E4FF@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 There are enough green buildbots and I think all the showstoppers are resolvable, so I'm going ahead and releasing beta 1 tonight. Please refrain from any commits unless I ask you to check something in. We've got a major thunderstorm blowing through right now, but when it clears I'll be on irc.freenode.net #python-dev. I'll start making the release as soon as I get on, and feel free to ask any questions. - -B -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFmca3EjvBPtnXfVAQICfAP8DHTOoQap9MnXJrUY0a3rurgRerMDNxSe 1dmOlhSVGmbQncPSZWP5x0xf1gC5QOCUAofGYfh2xY5ng6BbCQlzMX3Iogy1KvHQ Fb1HQ4EPUO5PGEQipgvHlkN38OFDqevoyvaRBIOGOppIVjL0v8P66eXNkkLq4BvT RztyojPrhxE= =jZ8g -----END PGP SIGNATURE----- From barry at python.org Thu Jun 19 03:02:05 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Jun 2008 21:02:05 -0400 Subject: [Python-Dev] [Python-3000] Another run at beta one In-Reply-To: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com> References: <4222a8490806180454i5671c8f6sd9d2423961eec940@mail.gmail.com> Message-ID: <9E5AEF05-6169-42A5-8CC8-EEC8750EE00B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 18, 2008, at 7:54 AM, Jesse Noller wrote: > Please let me know about 3088 - I've disabled the more unreliable > tests until I can give the entire suite some TLC. I'm still trying to > work around/identify the "Function not Implemented" error in issue3111 > - this happens to be failing the tests on a few of the linux boxes. I > suspected it was due to a combination of chroot + lack of /dev/shm on > some of the build bots, ergo a patch put in by Ben last night to skip > the tests on machines lacking the device, but it doesn't seem to be > skipping/the device exists. We're going to have to deal with 3088 after the beta release. I'm not going to let it hold us up this time. I think we're going to have a tougher time than we think getting the 3.0 release stabilized for final though. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFmwDXEjvBPtnXfVAQIFjAP+LGwB1Z3eBLaF/ZEez5TgkpMWdhHZwmWA zefJnlORHRN4TiawGDNkGdbs+B3O0An8vuCkXTnM97sdKqXYIUZ+0C8qdU5g9OaA UlX2Wagl1UTC8HHMhmPVlo7AnIzcR0mUOlu7T9SgtjKeEbcrSB4vCT9Cqy7rH7r8 jsHzlmzjswI= =r2K6 -----END PGP SIGNATURE----- From barry at python.org Thu Jun 19 04:46:06 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Jun 2008 22:46:06 -0400 Subject: [Python-Dev] RELEASED Python 2.6b1 and 3.0b1 Message-ID: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I am happy to announce the first beta releases of Python 2.6 and Python 3.0. Please note that these are beta releases, and as such are not suitable for production environments. We continue to strive for a high degree of quality, and these releases are intended to freeze the feature set for Python 2.6 and 3.0. From now until the planned final releases in September 2008, we will be fixing known problems and stabilizing these new Python versions. You can help by downloading and testing them, providing feedback and hopefully helping to fix bugs. You can also use these releases to determine how changes in 2.6 and 3.0 might impact you. If you find things broken or incorrect, please submit bug reports at http://bugs.python.org For more information and downloadable distributions, see the Python 2.6 website: http://www.python.org/download/releases/2.6/ and the Python 3.0 web site: http://www.python.org/download/releases/3.0/ See PEP 361 for release schedule details: http://www.python.org/dev/peps/pep-0361/ Enjoy, - -Barry Barry Warsaw barry at python.org Python 2.6/3.0 Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFnIbnEjvBPtnXfVAQKCMgP+L9Vpk9vQgYw01KShtF9HyvCKE0guSy86 0Q/IRbkXGahXg910D4nVLY9NORaOSq/VP8Kx+fOU8egUDHvVF9MuKkY75ZHYoixR mieeV7BIpYHYKYhva48FVoF4p0+sUeNvfeOAP0nnV5Pi2icGqCpzizfSHZloXJjc Y8M4F5vVrOM= =Oo15 -----END PGP SIGNATURE----- From cesare at pronto.it Thu Jun 19 06:58:07 2008 From: cesare at pronto.it (Cesare Di Mauro) Date: Thu, 19 Jun 2008 06:58:07 +0200 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <48536B7E.30103@v.loewis.de> <4854614B.9070903@canterbury.ac.nz> Message-ID: Very very, interesting. Thanks. :) Somebody thinks that Python is unsuitable to implement a DSL: IMO your example prove the contrary. :D Cesare In data 16 giugno 2008 alle ore 01:12:46, Alex Martelli ha scritto: > +1 on updating the FAQ. Maybe we could even have it notice that a > read-only version of the desired semantic for 'with' is easily hacked > with the *current* semantic of 'with'...: > > @contextlib.contextmanager > def readonly(anobj): > caller_globals = sys._getframe(2).f_globals > saved_globals = caller_globals.copy() > caller_globals.update((n, getattr(anobj, n)) for n in dir(anobj)) > yield > caller_globals.clear() > caller_globals.update(saved_globals) > > and then, of course, > > with readonly(someobj): ... > > (local variables take precedence, and in particular all assignments > define local variables, as usual -- but you can say e.g. 'zz' to mean > 'someobj.zz', if you're sufficiently keen on giving up the advantage > of having many well-separated namespaces;-). > > > Alex From tnelson at onresolve.com Thu Jun 19 07:39:55 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 19 Jun 2008 06:39:55 +0100 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275C@exchange.onresolve.com> > The server component *is* listening to localhost by default. The > client in the code I pasted has the server's .address attribute passed > into it to tell the client where to connect to. Interestingly enough, I can't coerce test_multiprocessing.test_listener_client to wedge on my XP laptop. When I run it on my Windows Server 2008 box (same box the buildbots run on), it wedges every time. So, I suspect this isn't actually a race like I first thought, but something quirky with 2008. Will dig around for a bit longer and see what I can find... Trent. From aleaxit at gmail.com Thu Jun 19 07:55:32 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 18 Jun 2008 22:55:32 -0700 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4852F7EC.6040403@voidspace.org.uk> <4854614B.9070903@canterbury.ac.nz> Message-ID: On Wed, Jun 18, 2008 at 9:58 PM, Cesare Di Mauro wrote: > Very very, interesting. Thanks. :) > > Somebody thinks that Python is unsuitable to implement a DSL: IMO your example prove the contrary. :D As long as you're willing to do the "DSL" within the strictures of Python syntax, it's OK - not quite as flexible as LISP or Scheme or even Ruby, but better than most;-). I did design and implement "DSL"s in Python (mostly specialized class trees with suitable metaclasses, decorators &c) in many jobs in my recent past, I'll admit -- it never feels as free-flowing as Scheme did back when I used THAT, but, close enough to make my jobs successful!-) Alex From tnelson at onresolve.com Thu Jun 19 08:02:46 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 19 Jun 2008 07:02:46 +0100 Subject: [Python-Dev] test_multiprocessing: test_listener_client flakiness In-Reply-To: References: <6167796BFEB5D0438720AC212E89A6B007865825@exchange.onresolve.com> Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275D@exchange.onresolve.com> > I just found the cause of the problem ten minutes ago: > It seems that when a socket listens on the address "127.0.0.1" or > "localhost", another process cannot connect to it using the machine's > name (even from the same machine). > The best seems to listen with the empty address "". That doesn't seem right. If that were the case, none of the hundreds of network-oriented tests would work either. These all bind against ('localhost', 0) by default which seems to Do The Right Thing on all the platforms we have buildbots for. Trent. From tnelson at onresolve.com Thu Jun 19 08:07:01 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 19 Jun 2008 07:07:01 +0100 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> Message-ID: <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com> > Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name > of the machine. This is the name that was stored in server.address. > Hence my patch that simply remove the call to getfqdn. +1 to ditching getfqdn, following patch fixes the issue on my buildbot server: Index: connection.py =================================================================== --- connection.py (revision 64369) +++ connection.py (working copy) @@ -215,10 +215,7 @@ self._socket = socket.socket(getattr(socket, family)) self._socket.bind(address) self._socket.listen(backlog) - address = self._socket.getsockname() - if type(address) is tuple: - address = (socket.getfqdn(address[0]),) + address[1:] - self._address = address + self._address = self._socket.getsockname() self._family = family self._last_accepted = None Trent. From p.f.moore at gmail.com Thu Jun 19 10:43:25 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 19 Jun 2008 09:43:25 +0100 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1 In-Reply-To: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> Message-ID: <79990c6b0806190143n58e51a22ubafd3fe343291dfb@mail.gmail.com> On 19/06/2008, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On behalf of the Python development team and the Python community, I am > happy to announce the first beta releases of Python 2.6 and Python 3.0. Any ETA for Windows builds? The web pages still point to the alphas. (I'd like to see the Windows builds more closely integrated with the releases now we're in beta stage...) Paul. From barry at python.org Thu Jun 19 13:53:10 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 19 Jun 2008 07:53:10 -0400 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1 In-Reply-To: <79990c6b0806190143n58e51a22ubafd3fe343291dfb@mail.gmail.com> References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> <79990c6b0806190143n58e51a22ubafd3fe343291dfb@mail.gmail.com> Message-ID: <5A74EFE6-0915-4EF9-9096-3CE75F32F00E@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 19, 2008, at 4:43 AM, Paul Moore wrote: > On 19/06/2008, Barry Warsaw wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On behalf of the Python development team and the Python community, >> I am >> happy to announce the first beta releases of Python 2.6 and Python >> 3.0. > > Any ETA for Windows builds? The web pages still point to the alphas. > (I'd like to see the Windows builds more closely integrated with the > releases now we're in beta stage...) Martin usually fills these in pretty quickly. I think the current situation works fine for the betas but we'll make sure the final release (and candidates) are better coordinated. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSFpIpnEjvBPtnXfVAQJyWQP9FSH8Ipg93UDM3nmH3UtN+i61YGsQPd0O ypHlnz4yHpxeRkJm1zkppHHI0hKMou6JOeUf05QCnPzrAdsG/mkuv5aoBrBt3dDd UncHLoQOvXEhGrrPzexmHKv3ehxUXPQOzkiWBWVv9e69GYH4e4HcqV6s2Ya2733T zC/EyOgkyMg= =5wM5 -----END PGP SIGNATURE----- From ctb at msu.edu Thu Jun 19 13:54:51 2008 From: ctb at msu.edu (C. Titus Brown) Date: Thu, 19 Jun 2008 04:54:51 -0700 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: References: <4854614B.9070903@canterbury.ac.nz> Message-ID: <20080619115450.GC23599@caltech.edu> On Wed, Jun 18, 2008 at 10:55:32PM -0700, Alex Martelli wrote: -> On Wed, Jun 18, 2008 at 9:58 PM, Cesare Di Mauro wrote: -> > Very very, interesting. Thanks. :) -> > -> > Somebody thinks that Python is unsuitable to implement a DSL: IMO your example prove the contrary. :D -> -> As long as you're willing to do the "DSL" within the strictures of -> Python syntax, it's OK - not quite as flexible as LISP or Scheme or -> even Ruby, but better than most;-). I did design and implement "DSL"s -> in Python (mostly specialized class trees with suitable metaclasses, -> decorators &c) in many jobs in my recent past, I'll admit -- it never -> feels as free-flowing as Scheme did back when I used THAT, but, close -> enough to make my jobs successful!-) It's pretty easy to put a simple syntax remapper around Python, too; this is what twill does. So, for example, this: -- setlocal username setlocal password go http://www.slashdot.org/ formvalue 1 unickname $username formvalue 1 upasswd $password submit code 200 # make sure form submission is correct! -- translates very directly to --- setlocal('username', '...') setlocal('password', '...') go('http://www.slashdot.org/') formvalue('1', 'unickname', username) formvalue('1', 'upasswd', password) submit() code('200') --- which is DSL-ish enough for me... More generally, I've never understood why some people insist that certain features make Ruby better for DSLs -- are code blocks really that important to DSLs? Or is it just the lack of parens?? --titus -- C. Titus Brown, ctb at msu.edu From jnoller at gmail.com Thu Jun 19 14:57:17 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 19 Jun 2008 08:57:17 -0400 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com> References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com> Message-ID: <4222a8490806190557s66a519ccj7b7bd80e3db68c9b@mail.gmail.com> Thanks Trent, I'll apply the diff and run the tests in a tight loop after re-enabling the client listener tests. I appreciate you tracking this down On Thu, Jun 19, 2008 at 2:07 AM, Trent Nelson wrote: >> Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name >> of the machine. This is the name that was stored in server.address. >> Hence my patch that simply remove the call to getfqdn. > > +1 to ditching getfqdn, following patch fixes the issue on my buildbot server: > > Index: connection.py > =================================================================== > --- connection.py (revision 64369) > +++ connection.py (working copy) > @@ -215,10 +215,7 @@ > self._socket = socket.socket(getattr(socket, family)) > self._socket.bind(address) > self._socket.listen(backlog) > - address = self._socket.getsockname() > - if type(address) is tuple: > - address = (socket.getfqdn(address[0]),) + address[1:] > - self._address = address > + self._address = self._socket.getsockname() > self._family = family > self._last_accepted = None > > > > Trent. > From janssen at parc.com Thu Jun 19 19:06:22 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 19 Jun 2008 10:06:22 PDT Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com> References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com> Message-ID: <08Jun19.100622pdt."58698"@synergy1.parc.xerox.com> This is a common problem. Binding to '127.0.0.1' will bind to *only* that address; binding to "" will bind to *all* addresses the machine is known by. If the client uses a different way of getting the address than the literal '127.0.0.1' (like a call to getfqdn(), which has pretty indeterminate results), you should use the "" form when calling bind. If you know you want to talk to '127.0.0.1', just use the tuple ('127.0.0.1', ) as the address. Bill > > Well, on my win2k machine getfqdn('127.0.0.1') returns the actual name > > of the machine. This is the name that was stored in server.address. > > Hence my patch that simply remove the call to getfqdn. > > +1 to ditching getfqdn, following patch fixes the issue on my buildbot server: > > Index: connection.py > =================================================================== > --- connection.py (revision 64369) > +++ connection.py (working copy) > @@ -215,10 +215,7 @@ > self._socket = socket.socket(getattr(socket, family)) > self._socket.bind(address) > self._socket.listen(backlog) > - address = self._socket.getsockname() > - if type(address) is tuple: > - address = (socket.getfqdn(address[0]),) + address[1:] > - self._address = address > + self._address = self._socket.getsockname() > self._family = family > self._last_accepted = None > > > > Trent. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com From curt at hagenlocher.org Thu Jun 19 19:19:34 2008 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Thu, 19 Jun 2008 10:19:34 -0700 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <20080619115450.GC23599@caltech.edu> References: <4854614B.9070903@canterbury.ac.nz> <20080619115450.GC23599@caltech.edu> Message-ID: On Thu, Jun 19, 2008 at 4:54 AM, C. Titus Brown wrote: > > More generally, I've never understood why some people insist that > certain features make Ruby better for DSLs -- are code blocks really > that important to DSLs? Or is it just the lack of parens?? Code blocks let Ruby-based DSLs "do" flow control, while the lack of parens make ordinary method names look like keywords. These things are a mixed bag -- on the one hand, they can certainly make the resulting DSL a lot easier to read. On the other, they seem to encourage a lot of over-the-top cleverness in terms of creating "fluent interfaces". I fail to see much value in being able to write code that says "7.seconds.ago". -- Curt Hagenlocher curt at hagenlocher.org From pje at telecommunity.com Thu Jun 19 19:21:23 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 19 Jun 2008 13:21:23 -0400 Subject: [Python-Dev] Python FAQ: Why doesn't Python have a "with" statement? In-Reply-To: <20080619115450.GC23599@caltech.edu> References: <4854614B.9070903@canterbury.ac.nz> <20080619115450.GC23599@caltech.edu> Message-ID: <20080619172047.DE1143A4112@sparrow.telecommunity.com> At 04:54 AM 6/19/2008 -0700, C. Titus Brown wrote: >More generally, I've never understood why some people insist that >certain features make Ruby better for DSLs -- are code blocks really >that important to DSLs? Or is it just the lack of parens?? Comparison to JavaScript suggests that it's the blocks that make the difference. Even the fact that you have to spell your blocks with "function(){...}" doesn't stop you from making a usable DSL, it's just not always as pretty as you'd like. The lack of parens and simpler syntax for blocks in Ruby just makes it easier to make *nice-looking* DSLs. From fijall at gmail.com Thu Jun 19 20:54:26 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 19 Jun 2008 20:54:26 +0200 Subject: [Python-Dev] Opcode frequency In-Reply-To: References: Message-ID: <693bc9ab0806191154s60c84c43pfb7e18003b827501@mail.gmail.com> On Thu, Jun 19, 2008 at 1:03 AM, Raymond Hettinger wrote: >> Maciej Fijalkowski did an opcode analysis for PyPy, >> it also shows the relative frequency of opcodes following a >> specifc one: >> >> http://codespeak.net/svn/user/fijal/opcodes.txt >> >> Might it make sense to add more PREDICT()ions based >> on this, e.g. for BUILD_SLICE -> BINARY_SUBSCR? > > This particular one might be okay. What code generates it? > Are there other possible successors to BUILD_SLICE? > What code generates those? > > There were lots of other opcode pairings that were previously > studied and rejected. You're going over well traveled ground. > > Also, opcode analysis is a somewhat hazardous undertaking. > Dynamic analysis takes into account which codes tend to > occur more often inside loops but it is *very* sensitive > to who wrote the app and their coding style. These results > always have to be taken with a grain of salt. > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fijall%40gmail.com > I think I should mention the code it comes from - it's pypy's translation toolchain and no, it's not trying to provide any data on "standard" python usage. For example CALL_LIKELY_BUILTIN comes from isinstance calls, which is not something completely natural to be called that often. It of course depends on coding style, this is our coding style (ie pypy team). From amk at amk.ca Thu Jun 19 22:30:30 2008 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 19 Jun 2008 16:30:30 -0400 Subject: [Python-Dev] [Python-checkins] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> Message-ID: <20080619203030.GA10666@amk-desktop.matrixgroup.net> On Thu, Jun 19, 2008 at 03:04:11PM -0500, Benjamin Peterson wrote: > I don't think the whole introduction had to go. I think it helped give > some idea of how multiprocessing works before jumping straight to the > API reference. I don't think overview material like that should be buried inside the documentation for one particular module; it should go in a chapter introduction. My plan is to move multiprocessing from the 'Optional OS Services' chapter to the 'Interprocess Communication' chapter, and then expand the introduction of the chapter to discuss the GIL and related issues. Presumably it was put in the 'Optional OS Services' chapter because that's where the threading and thread modules are. --amk From musiccomposition at gmail.com Thu Jun 19 22:32:18 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 19 Jun 2008 15:32:18 -0500 Subject: [Python-Dev] [Python-checkins] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <20080619203030.GA10666@amk-desktop.matrixgroup.net> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> Message-ID: <1afaf6160806191332mcfa1a50v72f537e11e8369eb@mail.gmail.com> On Thu, Jun 19, 2008 at 3:30 PM, A.M. Kuchling wrote: > My plan is to move multiprocessing from the 'Optional OS Services' > chapter to the 'Interprocess Communication' chapter, and then expand > the introduction of the chapter to discuss the GIL and related issues. > Presumably it was put in the 'Optional OS Services' chapter because > that's where the threading and thread modules are. Ok. Fair enough. Thanks for explaining. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From jnoller at gmail.com Thu Jun 19 23:16:38 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 19 Jun 2008 17:16:38 -0400 Subject: [Python-Dev] [Python-checkins] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <20080619203030.GA10666@amk-desktop.matrixgroup.net> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> Message-ID: <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> Where would that chapter end up (source-wise) I think a few of us might have additional things to add ;) On Thu, Jun 19, 2008 at 4:30 PM, A.M. Kuchling wrote: > On Thu, Jun 19, 2008 at 03:04:11PM -0500, Benjamin Peterson wrote: >> I don't think the whole introduction had to go. I think it helped give >> some idea of how multiprocessing works before jumping straight to the >> API reference. > > I don't think overview material like that should be buried inside the > documentation for one particular module; it should go in a chapter > introduction. > > My plan is to move multiprocessing from the 'Optional OS Services' > chapter to the 'Interprocess Communication' chapter, and then expand > the introduction of the chapter to discuss the GIL and related issues. > Presumably it was put in the 'Optional OS Services' chapter because > that's where the threading and thread modules are. > > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com > From alexandre at peadrop.com Thu Jun 19 23:23:12 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Thu, 19 Jun 2008 17:23:12 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: Message-ID: On Sun, Jun 1, 2008 at 12:28 AM, Adam Olsen wrote: > On Sat, May 31, 2008 at 10:11 PM, Alexandre Vassalotti > wrote: >> Would anyone mind if I did add a public C API for gc.disable() and >> gc.enable()? I would like to use it as an optimization for the pickle >> module (I found out that I get a good 2x speedup just by disabling the >> GC while loading large pickles). Of course, I could simply import the >> gc module and call the functions there, but that seems overkill to me. >> I included the patch below for review. > > I'd rather see it fixed. It behaves quadratically if you load enough > to trigger full collection a few times. > Do you have any idea how this behavior could be fixed? I am not a GC expert, but I could try to fix this. -- Alexandre From rhamph at gmail.com Thu Jun 19 23:35:55 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 19 Jun 2008 15:35:55 -0600 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: Message-ID: On Thu, Jun 19, 2008 at 3:23 PM, Alexandre Vassalotti wrote: > On Sun, Jun 1, 2008 at 12:28 AM, Adam Olsen wrote: >> On Sat, May 31, 2008 at 10:11 PM, Alexandre Vassalotti >> wrote: >>> Would anyone mind if I did add a public C API for gc.disable() and >>> gc.enable()? I would like to use it as an optimization for the pickle >>> module (I found out that I get a good 2x speedup just by disabling the >>> GC while loading large pickles). Of course, I could simply import the >>> gc module and call the functions there, but that seems overkill to me. >>> I included the patch below for review. >> >> I'd rather see it fixed. It behaves quadratically if you load enough >> to trigger full collection a few times. >> > > Do you have any idea how this behavior could be fixed? I am not a GC > expert, but I could try to fix this. Not sure. For long-running programs we actually want a quadratic cost, but spread out over a long period of time. It's the bursts of allocation that shouldn't be quadratic. So maybe balance it by the total number of allocations that have happened. If the total number of allocations is less than 10 times the allocations that weren't promptly deleted, assume it's a burst. If it's more than 10 times, assume it's over a long period. -- Adam Olsen, aka Rhamphoryncus From dbpokorny at gmail.com Fri Jun 20 00:59:13 2008 From: dbpokorny at gmail.com (David Pokorny) Date: Thu, 19 Jun 2008 15:59:13 -0700 Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations Message-ID: Hi, I am curious if there is any news on this issue. My understanding is that since this is a new feature, nothing will be committed until after 2.6 comes out, but it would be *really nice* if I could use annotations in 2.x. [warning - mini rant] Reason is, I am using 3.0a for now for this project, but I am sorry to say that using parentheses for raising exceptions drives me up the wall, and I don't like using them for the former print statement either. I suspect this is partly due to the fact that I'm using 2.5 for work, so I don't have the option to mentally "switch over" to the Python 3 way. /rant Does anyone have an annotations patch floating around against the trunk / 2.6b, or failing that, does anyone have opinions on what my chances are / things to look out for if I fiddle around with the annotations patch from the py3k branch? (My understanding is that this is revision 53170). Cheers, David From guido at python.org Fri Jun 20 01:16:12 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Jun 2008 16:16:12 -0700 Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations In-Reply-To: References: Message-ID: On Thu, Jun 19, 2008 at 3:59 PM, David Pokorny wrote: > I am curious if there is any news on this issue. My understanding is > that since this is a new feature, nothing will be committed until > after 2.6 comes out, but it would be *really nice* if I could use > annotations in 2.x. > > [warning - mini rant] > Reason is, I am using 3.0a for now for this project, but I am sorry to > say that using parentheses for raising exceptions drives me up the > wall, and I don't like using them for the former print statement > either. I suspect this is partly due to the fact that I'm using 2.5 > for work, so I don't have the option to mentally "switch over" to the > Python 3 way. > /rant > > Does anyone have an annotations patch floating around against the > trunk / 2.6b, or failing that, does anyone have opinions on what my > chances are / things to look out for if I fiddle around with the > annotations patch from the py3k branch? (My understanding is that this > is revision 53170). I give this approximately 0% chance of happening. It's a really complex change to the syntax and code generator. Your motivation is also suspect: 2.6 is supposed to be a stepping stone towards 3.0, not a safe haven for people who don't like certain 3.0 features. Just get used to print(). Parentheses for exceptions has been allowed since approximately Python 2.0, so you have even less of an excuse there. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri Jun 20 02:36:59 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 20 Jun 2008 12:36:59 +1200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: Message-ID: <485AFBAB.2060404@canterbury.ac.nz> Alexandre Vassalotti wrote: > Do you have any idea how this behavior could be fixed? I am not a GC > expert, but I could try to fix this. Perhaps after making a GC pass you could look at the number of objects reclaimed during that pass, and if it's less than some fraction of the objects in existence, increase the threshold for performing GC by some factor. You would also want to do the opposite, so that a GC pass which reclaims a large proportion of objects would reduce the threshold back down again. -- Greg From dbpokorny at gmail.com Fri Jun 20 03:05:27 2008 From: dbpokorny at gmail.com (dbpokorny at gmail.com) Date: Thu, 19 Jun 2008 18:05:27 -0700 (PDT) Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations In-Reply-To: References: Message-ID: On Jun 19, 4:16 pm, "Guido van Rossum" wrote: > complex change to the syntax and code generator. Your motivation is > also suspect: 2.6 is supposed to be a stepping stone towards 3.0, not > a safe haven for people who don't like certain 3.0 features. Just get Guilty as charged :) I tried out 3.0, decided I didn't like certain features I found annoying, and now I want to get the features (well only one feature right now) I like into 2.x so I can go back...I have some energy to do this, and I'm willing to learn, and I asked this list because I hoped someone would take pity on me and lend some help. What is my sentence, exile to outer PHP-iberia? At any rate, I am still interested if anyone has a working patch for this against the trunk, or any pointers for adapting 53170, words of experience when changing the grammar, additions to PEP 306, etc... any help would be greatly appreciated! Cheers, David From tjreedy at udel.edu Fri Jun 20 04:11:48 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 19 Jun 2008 22:11:48 -0400 Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations In-Reply-To: References: Message-ID: David Pokorny wrote: > [warning - mini rant] > Reason is, I am using 3.0a for now for this project, but I am sorry to > say that using parentheses for raising exceptions drives me up the > wall, [warning - a bit of a counter-rant ;-] Does it bother you that you need ()s to make instances elsewhere? That you have to type int('123') instead of int, '123'? The exception exception is an anomaly that would be gone in 2.6 (or possibly not until 2.7), if it had not been put off, with other removals, to 3.0. [/rant] > and I don't like using them for the former print statement either. That I sympathize with, but I prefer file = xxx to >>xxx and otherwise consider it an acceptible price for consistency with input() and getting rid of lot of things I do not like, from old-style classes to the exception anomaly ;-) > I suspect this is partly due to the fact that I'm using 2.5 > for work, so I don't have the option to mentally "switch over" to the > Python 3 way. Whereas I do, which, I admit, makes it easier. > /rant From amk at amk.ca Fri Jun 20 04:16:34 2008 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 19 Jun 2008 22:16:34 -0400 Subject: [Python-Dev] sum() in math module not duplicated? Message-ID: <20080620021634.GA3480@amk.local> In the comments before the implementation of sum() in mathmodule.c, note 4 says: Note 4: A similar implementation is in Modules/cmathmodule.c. Be sure to update both when making changes. I can't find a sum implementation or similar code in cmathmodule.c. Did someone intend to port the sum function to cmath, or was it copied and then taken out? I'm wondering if this note should simply be removed. --amk From python at rcn.com Fri Jun 20 04:41:55 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 19 Jun 2008 19:41:55 -0700 Subject: [Python-Dev] sum() in math module not duplicated? References: <20080620021634.GA3480@amk.local> Message-ID: We're still working on the implementation details for math.sum(). When it's finished, the cmath equilvalent will be added. Raymond ----- Original Message ----- From: "A.M. Kuchling" To: Sent: Thursday, June 19, 2008 7:16 PM Subject: [Python-Dev] sum() in math module not duplicated? > In the comments before the implementation of sum() > in mathmodule.c, note 4 says: > > Note 4: A similar implementation is in Modules/cmathmodule.c. > Be sure to update both when making changes. > > I can't find a sum implementation or similar code in cmathmodule.c. > Did someone intend to port the sum function to cmath, or was it copied > and then taken out? I'm wondering if this note should simply be > removed. > > --amk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com From tony at PageDNA.com Fri Jun 20 04:38:52 2008 From: tony at PageDNA.com (Tony Lownds) Date: Thu, 19 Jun 2008 19:38:52 -0700 Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations In-Reply-To: References: Message-ID: > At any rate, I am still interested if anyone has a working patch for > this against the trunk, or any pointers for adapting 53170, words of > experience when changing the grammar, additions to PEP 306, etc... any > help would be greatly appreciated! David, I can help. I don't have a patch against the trunk but my first revisions of the patch for annotations did handle things like tuple parameters which are relevant to 2.6. -Tony From tnelson at onresolve.com Fri Jun 20 07:02:02 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Fri, 20 Jun 2008 06:02:02 +0100 Subject: [Python-Dev] test_multiprocessing:test_listener_client flakiness In-Reply-To: <08Jun19.100622pdt."58698"@synergy1.parc.xerox.com> References: <20080618164136.4714.382689990.divmod.quotient.10414@ohm> <4222a8490806181055g427ff954rf61c87830480af7e@mail.gmail.com> <4222a8490806181132w1ef364beu4673cabaf1ecd5a8@mail.gmail.com> <6167796BFEB5D0438720AC212E89A6B00787275E@exchange.onresolve.com> <08Jun19.100622pdt."58698"@synergy1.parc.xerox.com> Message-ID: <6167796BFEB5D0438720AC212E89A6B007872765@exchange.onresolve.com> > This is a common problem. Binding to '127.0.0.1' will bind to *only* > that address; Indeed. > binding to "" will bind to *all* addresses the machine > is known by. Agreed again. I believe what we're dealing with here though is a lack of clarity regarding what role the 'address' attribute exposed by multiprocess.connection.Listener should play. The way test_listener_client() is written, it effectively treats 'address' as an end-point that can be connected to directly (irrespective of the underlying family (i.e. AF_INET, AF_UNIX, AF_PIPE)). I believe the problems we've run into stem from the fact that the API doesn't provide any guarantees as to what 'address' represents. The test suite assumes it always reflects a connectable end-point, which I think is more than reasonable. Unfortunately, nothing stops us from breaking this invariant by constructing the object as Listener(family='AF_INET', address=('0.0.0.0', 0)). How do I connect to an AF_INET Listener (i.e. SocketListener) instance whose 'address' attribute reports '0.0.0.0' as the host? I can't. So, for now, I think we should enforce this invariant by raising an exception in Listener.__init__() if self._socket.getsockbyname()[0] returns '0.0.0.0'. In effect, tightening up the API such that we can guarantee Listener.address will always represent a connectable end-point. We can look at how to service 'listen on all available interfaces' semantics at a later date -- that adds far less value IMO than being able to depend on the said guarantee. Thoughts? Trent. From amk at amk.ca Fri Jun 20 15:09:49 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 20 Jun 2008 09:09:49 -0400 Subject: [Python-Dev] [Python-checkins] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> Message-ID: <20080620130949.GA6641@amk-desktop.matrixgroup.net> On Thu, Jun 19, 2008 at 05:16:38PM -0400, Jesse Noller wrote: > Where would that chapter end up (source-wise) I think a few of us > might have additional things to add ;) This would be Doc/library/ipc.rst. The chapter is 'Interprocess Communication and Networking'. Is anyone else finding it increasingly odd that subprocess, signal, socket/ssl, and syncore are in the same chapter? I'm tempted to move socket, ssl, asyncore+asynchat into a 'networking' chapter, and then also move SocketServer from the 'Internet Protocols' chapter into this new chapter. --amk From bioinformed at gmail.com Fri Jun 20 16:15:49 2008 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Fri, 20 Jun 2008 10:15:49 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485AFBAB.2060404@canterbury.ac.nz> References: <485AFBAB.2060404@canterbury.ac.nz> Message-ID: <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> +1 on a C API for enabling and disabling GC. I have several instances where I create a large number of objects non-cyclic objects where I see huge GC overhead (30+ seconds with gc enabled, 0.15 seconds when disabled). +1000 to fixing the garbage collector to be smart enough to self-regulate itself better. In the mean time, I use the following context manager to deal with the hotspots as I find them: class gcdisabled(object): ''' Conext manager to temporarily disable Python's cyclic garbage collector. The primary use is to avoid thrashing while allocating large numbers of non-cyclic objects due to an overly aggressive garbage collector behavior. Will disable GC if it is enabled upon entry and renable upon exit: >>> gc.isenabled() True >>> with gcdisabled(): ... print gc.isenabled() False >>> print gc.isenabled() True Will not reenable if GC was disabled upon entry: >>> gc.disable() >>> gc.isenabled() False >>> with gcdisabled(): ... gc.isenabled() False >>> gc.isenabled() False ''' def __init__(self): self.isenabled = gc.isenabled() def __enter__(self): gc.disable() def __exit__(self, type, value, traceback): if self.isenabled: gc.enable() -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Fri Jun 20 16:25:01 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 20 Jun 2008 14:25:01 +0000 (UTC) Subject: [Python-Dev] C API for gc.enable() and gc.disable() References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> Message-ID: Hi, Kevin Jacobs bioinformed.com> gmail.com> writes: > > +1 on a C API for enabling and disabling GC.? I have several instances where I create a large number of objects non-cyclic objects where I see huge GC overhead (30+ seconds with gc enabled, 0.15 seconds when disabled). Could you try to post a stripped-down, self-contained example of such behaviour? Antoine. From bioinformed at gmail.com Fri Jun 20 17:04:00 2008 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Fri, 20 Jun 2008 11:04:00 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> Message-ID: <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> On Fri, Jun 20, 2008 at 10:25 AM, Antoine Pitrou wrote: > > Kevin Jacobs bioinformed.com> gmail.com> > writes: > > > > +1 on a C API for enabling and disabling GC. I have several instances > where > I create a large number of objects non-cyclic objects where I see huge GC > overhead (30+ seconds with gc enabled, 0.15 seconds when disabled). > > Could you try to post a stripped-down, self-contained example of such > behaviour? $ python -m timeit 'zip(*[range(1000000)]*5)' 10 loops, best of 3: 496 msec per loop $ python -m timeit -s 'import gc; gc.enable()' 'zip(*[range(1000000)]*5)' 10 loops, best of 3: 2.93 sec per loop Note that timeit cheats and disables GC by default. Attached is a less stripped down script to demonstrate the super-linear behavior for somewhat naively coded transpose operators. The output is listed below: FUNCTION ROWS COLUMNS GC ENABLED GC DISABLED -------------- ----------- ---------- ------------ ------------ transpose_comp 5 0 0.0000 0.0000 transpose_comp 5 250000 0.5535 0.3537 transpose_comp 5 500000 1.4359 0.6868 transpose_comp 5 750000 2.7148 1.0760 transpose_comp 5 1000000 3.8070 1.3936 transpose_comp 5 1250000 5.5184 1.7617 transpose_comp 5 1500000 7.8828 2.1308 transpose_comp 5 1750000 9.3279 2.5364 transpose_comp 5 2000000 11.8248 2.7399 transpose_comp 5 2250000 14.7436 3.1585 transpose_comp 5 2500000 18.4452 3.5818 transpose_comp 5 2750000 21.4856 3.8988 transpose_comp 5 3000000 24.4110 4.3148 transpose_zip 5 0 0.0000 0.0000 transpose_zip 5 250000 0.2537 0.0658 transpose_zip 5 500000 0.8380 0.1324 transpose_zip 5 750000 1.7507 0.1989 transpose_zip 5 1000000 2.6169 0.2648 transpose_zip 5 1250000 4.0760 0.3317 transpose_zip 5 1500000 5.8852 0.4145 transpose_zip 5 1750000 7.3925 0.5161 transpose_zip 5 2000000 10.0755 0.6708 transpose_zip 5 2250000 14.2698 0.7760 transpose_zip 5 2500000 16.7291 0.9022 transpose_zip 5 2750000 20.3833 1.0179 transpose_zip 5 3000000 24.5515 1.0971 Hope this helps, -Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bench_transpose.py Type: text/x-python-script Size: 796 bytes Desc: not available URL: From amauryfa at gmail.com Fri Jun 20 17:44:40 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 20 Jun 2008 17:44:40 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> Message-ID: 2008/6/20 Kevin Jacobs : > On Fri, Jun 20, 2008 at 10:25 AM, Antoine Pitrou > wrote: >> >> Kevin Jacobs bioinformed.com> gmail.com> >> writes: >> > >> > +1 on a C API for enabling and disabling GC. I have several instances >> > where >> I create a large number of objects non-cyclic objects where I see huge GC >> overhead (30+ seconds with gc enabled, 0.15 seconds when disabled). >> >> Could you try to post a stripped-down, self-contained example of such >> behaviour? > > $ python -m timeit 'zip(*[range(1000000)]*5)' > 10 loops, best of 3: 496 msec per loop > > $ python -m timeit -s 'import gc; gc.enable()' 'zip(*[range(1000000)]*5)' > 10 loops, best of 3: 2.93 sec per loop I remember that a similar issue was discussed some months ago: http://bugs.python.org/issue2607 In short: the gc is tuned for typical usage. If your usage of python is specific, use gc.set_threshold and increase its values. -- Amaury Forgeot d'Arc From janssen at parc.com Fri Jun 20 17:55:13 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 20 Jun 2008 08:55:13 PDT Subject: [Python-Dev] [Python-checkins] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <20080620130949.GA6641@amk-desktop.matrixgroup.net> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> <20080620130949.GA6641@amk-desktop.matrixgroup.net> Message-ID: <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com> > Is anyone else finding it increasingly odd that subprocess, signal, > socket/ssl, and syncore are in the same chapter? I'm tempted to move > socket, ssl, asyncore+asynchat into a 'networking' chapter, and then > also move SocketServer from the 'Internet Protocols' chapter into this > new chapter. Sounds like you mean, 'Non-HTTP Networking'. Bill From status at bugs.python.org Fri Jun 20 18:06:51 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 20 Jun 2008 18:06:51 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080620160651.ADE90780F2@psf.upfronthosting.co.za> ACTIVITY SUMMARY (06/13/08 - 06/20/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1917 open (+28) / 13057 closed (+19) / 14974 total (+47) Open issues with patches: 581 Average duration of open issues: 712 days. Median duration of open issues: 1479 days. Open Issues Breakdown open 1892 (+28) pending 25 ( +0) Issues Created Or Reopened (50) _______________________________ Implement __format__ for Decimal 06/20/08 http://bugs.python.org/issue2110 reopened benjamin.peterson patch Create the urllib package 06/18/08 http://bugs.python.org/issue2885 reopened brett.cannon Python 2.5.2 Windows Source Distribution missing Visual Studio 2 06/13/08 http://bugs.python.org/issue3105 created MarkE speedup some comparisons 06/13/08 http://bugs.python.org/issue3106 created pitrou patch memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Linu 06/13/08 http://bugs.python.org/issue3107 created hushp1pt Implicit exception chaining (PEP 3134) 06/13/08 CLOSED http://bugs.python.org/issue3108 created pitrou patch test_multiprocessing seems fragile 06/14/08 http://bugs.python.org/issue3109 created pitrou Multiprocessing package build problem on Solaris 10 06/14/08 http://bugs.python.org/issue3110 created jnoller multiprocessing ppc Debian/ ia64 Ubuntu compilation error 06/14/08 http://bugs.python.org/issue3111 created jnoller implement PEP 3134 exception reporting 06/14/08 http://bugs.python.org/issue3112 created benjamin.peterson patch Document exception chaining 06/14/08 http://bugs.python.org/issue3113 created benjamin.peterson bus error on lib2to3 06/15/08 CLOSED http://bugs.python.org/issue3114 reopened gvanrossum patch os.listdir randomly fails on occasions when it shouldn't 06/14/08 http://bugs.python.org/issue3115 created philipspencer patch Fix quadratic behavior for marshal.dumps() when len>32Mb 06/15/08 CLOSED http://bugs.python.org/issue3116 created rhettinger patch, patch segfault with (None,) as argument in a def/lambda 06/15/08 CLOSED http://bugs.python.org/issue3117 created santoemma test_math fails on 64bit 06/16/08 CLOSED http://bugs.python.org/issue3118 created benjamin.peterson patch, 64bit pickle.py is limited by python's call stack 06/16/08 http://bugs.python.org/issue3119 created habnabit patch subprocess module truncates handles on AMD64 06/16/08 http://bugs.python.org/issue3120 created rupole test_urllibnet fails 06/16/08 CLOSED http://bugs.python.org/issue3121 created cartman sys.getsizeof() gives an AttributeError for _sre objects. 06/16/08 http://bugs.python.org/issue3122 created schuppenies patch, patch 2to3 fails 06/16/08 CLOSED http://bugs.python.org/issue3123 created djc test_multiprocessing segfaults on Windows 06/16/08 CLOSED http://bugs.python.org/issue3124 created theller test_multiprocessing causes test_ctypes to fail 06/16/08 http://bugs.python.org/issue3125 created theller patch Lib/logging/__init__.py assumes its stream has flush and close m 06/16/08 CLOSED http://bugs.python.org/issue3126 created bobf patch Segfault provoked by generators and exceptions 06/17/08 CLOSED http://bugs.python.org/issue3127 created amajorek Regex causes python to hang up? / loop infinite? 06/17/08 CLOSED http://bugs.python.org/issue3128 created computercrustie struct allows repeat spec. without a format specifier 06/17/08 http://bugs.python.org/issue3129 created carrus85 In some UCS4 builds, sizeof(Py_UNICODE) could end up being more 06/17/08 http://bugs.python.org/issue3130 created schuppenies 2to3 can't find fixes_dir 06/17/08 http://bugs.python.org/issue3131 created bhy implement PEP 3118 struct changes 06/17/08 http://bugs.python.org/issue3132 created benjamin.peterson CGIHTTPRequestHandler does not work on windows 06/18/08 CLOSED http://bugs.python.org/issue3133 created amaury.forgeotdarc patch shutil references undefined WindowsError symbol 06/18/08 http://bugs.python.org/issue3134 created dvitek patch inspect.getcallargs() 06/18/08 http://bugs.python.org/issue3135 created gsakkis [PATCH] logging.config.fileConfig() compulsivly disable all exis 06/19/08 CLOSED http://bugs.python.org/issue3136 created llucax patch Python doesn't handle SIGINT well if it arrives during interpret 06/19/08 http://bugs.python.org/issue3137 created gjb1002 Hang when calling get() on an empty queue in the queue module 06/19/08 CLOSED http://bugs.python.org/issue3138 created slash2314 print is not thread safe 06/19/08 http://bugs.python.org/issue3139 created amaury.forgeotdarc patch str.format("{0:n}") poss. bug with setlocale() 06/19/08 http://bugs.python.org/issue3140 created mark Linux build requires Mac module _gestalt 06/19/08 CLOSED http://bugs.python.org/issue3141 created mark urllib docs don't match the new modules 06/19/08 http://bugs.python.org/issue3142 created mark development docs waste a lot of horizontal space on left nav bar 06/19/08 http://bugs.python.org/issue3143 created forest popen / popen[234] inconsistent fd behavior 06/19/08 CLOSED http://bugs.python.org/issue3144 created justincappos help> modules os raises UnicodeDecodeError 06/19/08 CLOSED http://bugs.python.org/issue3145 created msyang Sphinx/LaTeX fails on Python 3.0b1 documentation 06/19/08 http://bugs.python.org/issue3146 created vmanis1 tests for sys.getsizeof fail on win64 06/19/08 http://bugs.python.org/issue3147 created amaury.forgeotdarc multiprocessing test hangs 06/20/08 CLOSED http://bugs.python.org/issue3148 created MrJean1 multiprocessing build fails on Solaris 10 06/20/08 http://bugs.python.org/issue3149 created MrJean1 multiprocessing module not being installed 06/20/08 CLOSED http://bugs.python.org/issue3150 created hdiogenes patch, easy elementtree serialization bug for weird namespace urls 06/20/08 http://bugs.python.org/issue3151 created xrotwang SimpleHTTPServer reports wrong content-length for text files 06/13/08 http://bugs.python.org/issue839496 reopened tjreedy patch, easy Issues Now Closed (34) ______________________ test_socket_ssl hanhs on Windows (deadlock) 209 days http://bugs.python.org/issue1489 amaury.forgeotdarc o(n*n) marshal.dumps performance for largish objects with patch 156 days http://bugs.python.org/issue1792 rhettinger patch, easy Add a factorial function 92 days http://bugs.python.org/issue2138 marketdickinson Make range __eq__ work 67 days http://bugs.python.org/issue2603 gvanrossum patch range: lean and mean 45 days http://bugs.python.org/issue2735 gvanrossum patch Add memory footprint query 32 days http://bugs.python.org/issue2898 schuppenies patch Bug in slice.indices() 22 days http://bugs.python.org/issue3004 marketdickinson patch 3.0a5 tarballs contain the code twice 20 days http://bugs.python.org/issue3009 barry getsizeof incorrect for Unicode strings 7 days http://bugs.python.org/issue3048 schuppenies tokenize.py improvements 6 days http://bugs.python.org/issue3078 rhettinger patch All 3.0 buildbots are red 7 days http://bugs.python.org/issue3089 barry sys.sizeof test fails with wide unicode 5 days http://bugs.python.org/issue3098 schuppenies patch Implicit exception chaining (PEP 3134) 1 days http://bugs.python.org/issue3108 benjamin.peterson patch bus error on lib2to3 1 days http://bugs.python.org/issue3114 benjamin.peterson patch Fix quadratic behavior for marshal.dumps() when len>32Mb 1 days http://bugs.python.org/issue3116 rhettinger patch, patch segfault with (None,) as argument in a def/lambda 0 days http://bugs.python.org/issue3117 georg.brandl test_math fails on 64bit 2 days http://bugs.python.org/issue3118 benjamin.peterson patch, 64bit test_urllibnet fails 3 days http://bugs.python.org/issue3121 amaury.forgeotdarc 2to3 fails 0 days http://bugs.python.org/issue3123 benjamin.peterson test_multiprocessing segfaults on Windows 0 days http://bugs.python.org/issue3124 amaury.forgeotdarc Lib/logging/__init__.py assumes its stream has flush and close m 1 days http://bugs.python.org/issue3126 vsajip patch Segfault provoked by generators and exceptions 0 days http://bugs.python.org/issue3127 amaury.forgeotdarc Regex causes python to hang up? / loop infinite? 0 days http://bugs.python.org/issue3128 computercrustie CGIHTTPRequestHandler does not work on windows 1 days http://bugs.python.org/issue3133 amaury.forgeotdarc patch [PATCH] logging.config.fileConfig() compulsivly disable all exis 1 days http://bugs.python.org/issue3136 llucax patch Hang when calling get() on an empty queue in the queue module 0 days http://bugs.python.org/issue3138 benjamin.peterson Linux build requires Mac module _gestalt 0 days http://bugs.python.org/issue3141 benjamin.peterson popen / popen[234] inconsistent fd behavior 0 days http://bugs.python.org/issue3144 amaury.forgeotdarc help> modules os raises UnicodeDecodeError 0 days http://bugs.python.org/issue3145 amaury.forgeotdarc multiprocessing test hangs 0 days http://bugs.python.org/issue3148 benjamin.peterson multiprocessing module not being installed 0 days http://bugs.python.org/issue3150 benjamin.peterson patch, easy Allow PyArg_ParseTupleAndKeywords to unpack tuples 786 days http://bugs.python.org/issue1474454 loewis patch Python 2.5b2 fails to build on Solaris 10 (Sun Compiler) 695 days http://bugs.python.org/issue1528620 akuchling Add Tkinter.Checkbutton get() and set(value) 307 days http://bugs.python.org/issue1774370 gpolo patch Top Issues Most Discussed (10) ______________________________ 17 test_multiprocessing causes test_ctypes to fail 4 days open http://bugs.python.org/issue3125 17 sys.sizeof test fails with wide unicode 5 days closed http://bugs.python.org/issue3098 13 multiprocessing ppc Debian/ ia64 Ubuntu compilation error 6 days open http://bugs.python.org/issue3111 12 bus error on lib2to3 1 days closed http://bugs.python.org/issue3114 11 memory leak in make test (in "test list"), 2.5.2 not 2.5.1, Lin 7 days open http://bugs.python.org/issue3107 8 test_multiprocessing hangs on OS X 10.5.3 8 days open http://bugs.python.org/issue3088 8 Bug in slice.indices() 22 days closed http://bugs.python.org/issue3004 7 Let bin() show floats 21 days open http://bugs.python.org/issue3008 7 robotparser.py fixes 304 days open http://bugs.python.org/issue1778443 6 Regex causes python to hang up? / loop infinite? 0 days closed http://bugs.python.org/issue3128 From rhamph at gmail.com Fri Jun 20 19:54:15 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 20 Jun 2008 11:54:15 -0600 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> Message-ID: On Fri, Jun 20, 2008 at 9:44 AM, Amaury Forgeot d'Arc wrote: > 2008/6/20 Kevin Jacobs : >> On Fri, Jun 20, 2008 at 10:25 AM, Antoine Pitrou >> wrote: >>> >>> Kevin Jacobs bioinformed.com> gmail.com> >>> writes: >>> > >>> > +1 on a C API for enabling and disabling GC. I have several instances >>> > where >>> I create a large number of objects non-cyclic objects where I see huge GC >>> overhead (30+ seconds with gc enabled, 0.15 seconds when disabled). >>> >>> Could you try to post a stripped-down, self-contained example of such >>> behaviour? >> >> $ python -m timeit 'zip(*[range(1000000)]*5)' >> 10 loops, best of 3: 496 msec per loop >> >> $ python -m timeit -s 'import gc; gc.enable()' 'zip(*[range(1000000)]*5)' >> 10 loops, best of 3: 2.93 sec per loop > > I remember that a similar issue was discussed some months ago: > http://bugs.python.org/issue2607 > > In short: the gc is tuned for typical usage. If your usage of python > is specific, > use gc.set_threshold and increase its values. For very large bursts of allocation, tuning is no different from disabling it outright, and disabling is simpler/more reliable. -- Adam Olsen, aka Rhamphoryncus From amk at amk.ca Fri Jun 20 19:58:18 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 20 Jun 2008 13:58:18 -0400 Subject: [Python-Dev] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> <20080620130949.GA6641@amk-desktop.matrixgroup.net> <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com> Message-ID: <20080620175818.GA7872@amk-desktop.matrixgroup.net> On Fri, Jun 20, 2008 at 08:55:13AM -0700, Bill Janssen wrote: > > Is anyone else finding it increasingly odd that subprocess, signal, > > socket/ssl, and syncore are in the same chapter? I'm tempted to move > > socket, ssl, asyncore+asynchat into a 'networking' chapter, and then > > also move SocketServer from the 'Internet Protocols' chapter into this > > new chapter. > > Sounds like you mean, 'Non-HTTP Networking'. I don't think so -- SMTP, FTP, NNTP, and telnet have nothing to do with HTTP, but they're certainly Internet protocols. --amk From janssen at parc.com Fri Jun 20 20:26:52 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 20 Jun 2008 11:26:52 PDT Subject: [Python-Dev] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <20080620175818.GA7872@amk-desktop.matrixgroup.net> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> <20080620130949.GA6641@amk-desktop.matrixgroup.net> <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com> <20080620175818.GA7872@amk-desktop.matrixgroup.net> Message-ID: <08Jun20.112653pdt."58698"@synergy1.parc.xerox.com> > On Fri, Jun 20, 2008 at 08:55:13AM -0700, Bill Janssen wrote: > > > Is anyone else finding it increasingly odd that subprocess, signal, > > > socket/ssl, and syncore are in the same chapter? I'm tempted to move > > > socket, ssl, asyncore+asynchat into a 'networking' chapter, and then > > > also move SocketServer from the 'Internet Protocols' chapter into this > > > new chapter. > > > > Sounds like you mean, 'Non-HTTP Networking'. > > I don't think so -- SMTP, FTP, NNTP, and telnet have nothing > to do with HTTP, but they're certainly Internet protocols. Sure, but there's typically only one protocol that's carried over any of them. Whereas with REST and XML-RPC and so forth, there's a whole cottage industry of using HTTP as a carrier, in much the same way that TCP is used at a lower level. But you're right: there's "Networking / Generic Support", "Networking / Specific Protocols", and "Networking / HTTP-based", or some such. Bill From eric+python-dev at trueblade.com Fri Jun 20 23:17:23 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 20 Jun 2008 17:17:23 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: <20080620041816.4D5E81E4002@bag.python.org> References: <20080620041816.4D5E81E4002@bag.python.org> Message-ID: <485C1E63.2020900@trueblade.com> I thought there was a discussion of this earlier, and the idea was to leave the prior implementation, because that's how it's implemented in 3.0. bin() is a new feature in 2.6, so there's no particular need to make it work like hex() and oct(). Recall that in 3.0, __bin__, __oct__, and __hex__ don't exist. Instead, you use __index__ for integer conversions. That's how bin() worked in 2.6 until this checkin. But now that I look for it, I can't find the original discussion. raymond.hettinger wrote: > Author: raymond.hettinger > Date: Fri Jun 20 06:18:15 2008 > New Revision: 64424 > > Log: > Make bin() implementation parallel oct() and hex() so that int/long subclasses can override or so that other classes can support. From steve at holdenweb.com Fri Jun 20 23:46:24 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 20 Jun 2008 17:46:24 -0400 Subject: [Python-Dev] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: <20080620175818.GA7872@amk-desktop.matrixgroup.net> References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> <20080620130949.GA6641@amk-desktop.matrixgroup.net> <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com> <20080620175818.GA7872@amk-desktop.matrixgroup.net> Message-ID: A.M. Kuchling wrote: > On Fri, Jun 20, 2008 at 08:55:13AM -0700, Bill Janssen wrote: >>> Is anyone else finding it increasingly odd that subprocess, signal, >>> socket/ssl, and syncore are in the same chapter? I'm tempted to move >>> socket, ssl, asyncore+asynchat into a 'networking' chapter, and then >>> also move SocketServer from the 'Internet Protocols' chapter into this >>> new chapter. >> Sounds like you mean, 'Non-HTTP Networking'. > > I don't think so -- SMTP, FTP, NNTP, and telnet have nothing > to do with HTTP, but they're certainly Internet protocols. > Perhaps we need a split between "networking technologies" and "network-based applications". regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From fdrake at acm.org Fri Jun 20 23:55:57 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 20 Jun 2008 17:55:57 -0400 Subject: [Python-Dev] r64407 - python/trunk/Doc/library/multiprocessing.rst In-Reply-To: References: <20080619194843.0A12B1E4002@bag.python.org> <1afaf6160806191304v1c6a610aj57c0e46378a4ff8@mail.gmail.com> <20080619203030.GA10666@amk-desktop.matrixgroup.net> <4222a8490806191416t72fd0cf4gb9253123b8d9eaea@mail.gmail.com> <20080620130949.GA6641@amk-desktop.matrixgroup.net> <08Jun20.085516pdt."58698"@synergy1.parc.xerox.com> <20080620175818.GA7872@amk-desktop.matrixgroup.net> Message-ID: <9438CBE3-FA32-451A-B756-DDE78C5CE060@acm.org> On Jun 20, 2008, at 5:46 PM, Steve Holden wrote: > Perhaps we need a split between "networking technologies" and > "network-based applications". Perhaps that would help. I certainly see HTTP as being on the same layer as SMTP and the like, but application protocols that ride on top of HTTP are a different beast. -Fred -- Fred Drake From dbpokorny at gmail.com Sat Jun 21 03:51:02 2008 From: dbpokorny at gmail.com (dbpokorny at gmail.com) Date: Fri, 20 Jun 2008 18:51:02 -0700 (PDT) Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations In-Reply-To: References: Message-ID: <2be1c9ce-41df-41e5-a50a-47de509cb3e9@z24g2000prf.googlegroups.com> > I can help. I don't have a patch against the trunk but my first > revisions of the patch > for annotations did handle things like tuple parameters which are > relevant to 2.6. Ah yes, I forgot about nested parameters. I see that 53170 still has nested parameters, but they were removed at some later point. Do you think this revision (and the patch it checked in) is the best starting point, or are my chances better with the patches at http://bugs.python.org/issue1607548 ? I just learned that default argument values are not allowed for nested parameters (in any version of Python AFAIK). Is there any reason for this other than lack of tuits? Cheers, David From solipsis at pitrou.net Sat Jun 21 03:39:46 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 21 Jun 2008 03:39:46 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> Message-ID: <1214012386.6402.4.camel@fsol> Le vendredi 20 juin 2008 ? 17:44 +0200, Amaury Forgeot d'Arc a ?crit : > In short: the gc is tuned for typical usage. If your usage of python > is specific, > use gc.set_threshold and increase its values. It's fine for people "in the know" who take the time to test their code using various gc parameters. But most people don't (I know I never did, and until recently I didn't even know the gc could have such a disastrous effect on performance). I don't think expecting people to tweak gc parameters when they witness performance problems is reasonable. From martin at v.loewis.de Sat Jun 21 10:33:30 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 21 Jun 2008 10:33:30 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <1214012386.6402.4.camel@fsol> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> Message-ID: <485CBCDA.4020601@v.loewis.de> > I don't think expecting people to tweak gc parameters when they witness > performance problems is reasonable. What follows from that? To me, the natural conclusion is "people who witness performance problems just need to despair, or accept them, as they can't do anything about it", however, I don't think this is the conclusion that you had in mind. Regards, Martin From dbpokorny at gmail.com Sat Jun 21 10:41:04 2008 From: dbpokorny at gmail.com (dbpokorny at gmail.com) Date: Sat, 21 Jun 2008 01:41:04 -0700 (PDT) Subject: [Python-Dev] Status of Issue 2331 - Backport parameter annotations In-Reply-To: References: Message-ID: > Does it bother you that you need ()s to make instances elsewhere? > That you have to type int('123') instead of int, '123'? Not at all...Python never supported this. Cheers, David From drnlmuller+python at gmail.com Sat Jun 21 12:16:24 2008 From: drnlmuller+python at gmail.com (Neil Muller) Date: Sat, 21 Jun 2008 12:16:24 +0200 Subject: [Python-Dev] Issue 2722 Message-ID: Could some one have a look at the suggested fix for issue 2722? While not a particularly common problem, it can be an annoyance, and the patch looks reasonable. -- Neil Muller From skip at pobox.com Sat Jun 21 13:06:24 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 21 Jun 2008 06:06:24 -0500 Subject: [Python-Dev] csv module and Unicode in 2.6 and 3.0 Message-ID: <18524.57520.222364.658608@montanaro-dyndns-org.local> If I understand things correctly, the csv module in 3.0 supports Unicode, kind of for free because all strings are Unicode. I suspect the Unicode examples at the bottom of the 3.0 version of the module doc need to be removed. Does the 2.6 version support Unicode out of the box as well or are the Unicode examples still required there? Skip From solipsis at pitrou.net Sat Jun 21 15:18:31 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 21 Jun 2008 15:18:31 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485CBCDA.4020601@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> Message-ID: <1214054311.5874.12.camel@fsol> Le samedi 21 juin 2008 ? 10:33 +0200, "Martin v. L?wis" a ?crit : > > I don't think expecting people to tweak gc parameters when they witness > > performance problems is reasonable. > > What follows from that? To me, the natural conclusion is "people who > witness performance problems just need to despair, or accept them, as > they can't do anything about it", To me, Amaury's answer implies that most people can't do anything about it, indeed. Well, they could hang themselves or switch to another language (which some people might view as equivalent :-)), but perhaps optimistically the various propositions that were sketched out in this thread (by Adam Olsen and Greg Ewing) could bring an improvement. I don't know how realistic they are, perhaps an expert would have an answer. Regards Antoine. From bioinformed at gmail.com Sat Jun 21 16:27:43 2008 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Sat, 21 Jun 2008 10:27:43 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485CBCDA.4020601@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> Message-ID: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> On Sat, Jun 21, 2008 at 4:33 AM, "Martin v. L?wis" wrote: > > I don't think expecting people to tweak gc parameters when they witness > > performance problems is reasonable. > > What follows from that? To me, the natural conclusion is "people who > witness performance problems just need to despair, or accept them, as > they can't do anything about it", however, I don't think this is the > conclusion that you had in mind. > I can say with complete certainty that of the 20+ programmers I've had working for me, many who have used Python for 3+ years, not a single one would think to question the garbage collector if they observed the kind of quadratic time complexity I've demonstrated. This is not because they are stupid, but because they have only a vague idea that Python even has a garbage collector, never mind that it could be behaving badly for such innocuous looking code. Maybe we should consider more carefully before declaring the status quo sufficient. Average developers do allocate millions of objects in bursts and super-linear time complexity for such operations is not acceptable. Thankfully I am around to help my programmers work around such issues or else they'd be pushing to switch to Java, Ruby, C#, or whatever since Python was inexplicably "too slow" for "real work". This being open source, I'm certainly willing to help in the effort to do so, but not if potential solutions will be ruled out as being unnecessary. -Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat Jun 21 17:11:16 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Jun 2008 17:11:16 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> Message-ID: <485D1A14.60005@v.loewis.de> > I can say with complete certainty that of the 20+ programmers I've had > working for me, many who have used Python for 3+ years, not a single one > would think to question the garbage collector if they observed the kind > of quadratic time complexity I've demonstrated. This is not because > they are stupid, but because they have only a vague idea that Python > even has a garbage collector, never mind that it could be behaving badly > for such innocuous looking code. > > Maybe we should consider more carefully before declaring the status quo > sufficient. This was precisely my question: What follows from the above observation? I personally didn't declare the status quo sufficient - I merely declared it as being the status quo. > Average developers do allocate millions of objects in > bursts and super-linear time complexity for such operations is not > acceptable. Thankfully I am around to help my programmers work around > such issues or else they'd be pushing to switch to Java, Ruby, C#, or > whatever since Python was inexplicably "too slow" for "real work". This > being open source, I'm certainly willing to help in the effort to do so, > but not if potential solutions will be ruled out as being unnecessary. I wouldn't rule out solutions as being unnecessary. I might rule out solutions that negatively impact existing software, for the sake of improving other existing software. Unfortunately, the only way to find out whether a solution will be ruled out is to propose it first. Only then you'll see what response you get. Regards, Martin From martin at v.loewis.de Sat Jun 21 17:20:07 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 21 Jun 2008 17:20:07 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <1214054311.5874.12.camel@fsol> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> Message-ID: <485D1C27.60408@v.loewis.de> > Well, they could hang themselves or switch to another language (which > some people might view as equivalent :-)), but perhaps optimistically > the various propositions that were sketched out in this thread (by Adam > Olsen and Greg Ewing) could bring an improvement. I don't know how > realistic they are, perhaps an expert would have an answer. In general, any solution of the "do GC less often" needs to deal with cases where lots of garbage gets produced in a short amount of time (e.g. in a tight loop), and which run out of memory when GC is done less often. Given the choice of "run slower" and "run out of memory", Python should always prefer the former. One approach could be to measure how successful a GC run was: if GC finds that more-and-more objects get allocated and very few (or none) are garbage, it might conclude that this is an allocation spike, and back off. The tricky question is how to find out that the spike is over. Regards, Martin From bioinformed at gmail.com Sat Jun 21 17:33:14 2008 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Sat, 21 Jun 2008 11:33:14 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485D1C27.60408@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> Message-ID: <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> On Sat, Jun 21, 2008 at 11:20 AM, "Martin v. L?wis" wrote: > In general, any solution of the "do GC less often" needs to deal with > cases where lots of garbage gets produced in a short amount of time > (e.g. in a tight loop), and which run out of memory when GC is done less > often. > Idea 1: Allow GC to run automatically no more often than n CPU seconds, n being perhaps 5 or 10. Idea 2: Allow GC to run no more often than f(n) CPU seconds, where n is the time taken by the last GC round. These limits could be reset or scaled by the GC collecting more than n% of the generation 0 objects or maybe the number of PyMalloc arenas increasing by a certain amount? -Kevin > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jacobs%40bioinformed.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat Jun 21 17:49:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Jun 2008 17:49:49 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> Message-ID: <485D231D.9030806@v.loewis.de> > Idea 1: Allow GC to run automatically no more often than n CPU seconds, > n being perhaps 5 or 10. I think it's very easy to exhaust the memory with such a policy, even though much memory would still be available. Worse, in a program producing a lot of garbage, performance will go significantly down as the python starts thrashing the swap space. > Idea 2: Allow GC to run no more often than f(n) CPU seconds, where n is > the time taken by the last GC round. How would that take the incremental GC into account? (i.e. what is "the time taken by the last GC round"?) Furthermore, the GC run time might well be under the resolution of the CPU seconds clock. > These limits could be reset or scaled by the GC collecting more than n% > of the generation 0 objects or maybe the number of PyMalloc arenas > increasing by a certain amount? I don't think any strategies based on timing will be successful. Instead, one should count and analyze objects (although I'm unsure how exactly that could work). Regards, Martin From aahz at pythoncraft.com Sat Jun 21 18:17:42 2008 From: aahz at pythoncraft.com (Aahz) Date: Sat, 21 Jun 2008 09:17:42 -0700 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485D1C27.60408@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> Message-ID: <20080621161742.GA15793@panix.com> On Sat, Jun 21, 2008, "Martin v. L??wis" wrote: > > In general, any solution of the "do GC less often" needs to deal with > cases where lots of garbage gets produced in a short amount of time > (e.g. in a tight loop), and which run out of memory when GC is done less > often. > > Given the choice of "run slower" and "run out of memory", Python should > always prefer the former. I'm not sure I agree with this. GC IIRC was introduced primarily to alleviate *long-term* memory starvation. You are now IMO adding a new goal for GC that has not been previously articulated. I believe this requires consensus rather than a simple declaration of principle. Guido's opinion if he has one obviously overrules. ;-) Guido? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha From martin at v.loewis.de Sat Jun 21 18:36:10 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Jun 2008 18:36:10 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <20080621161742.GA15793@panix.com> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <20080621161742.GA15793@panix.com> Message-ID: <485D2DFA.6070306@v.loewis.de> > I'm not sure I agree with this. GC IIRC was introduced primarily to > alleviate *long-term* memory starvation. I don't think that's historically the case. GC would not need to be generational if releasing short-lived objects shortly after they become garbage was irrelevant. Of course, it was always expected that much memory is released through mere reference counting, and that GC only kicks in "after some time". However "after some time" was changed from "after 5000 allocations" to "after 700 allocations" in ------------------------------------------------------------------------ r17274 | jhylton | 2000-09-05 17:44:50 +0200 (Di, 05 Sep 2000) | 2 lines Ge?nderte Pfade: M /python/trunk/Modules/gcmodule.c compromise value for threshold0: not too high, not too low ------------------------------------------------------------------------ Regards, Martin From janssen at parc.com Sat Jun 21 19:10:06 2008 From: janssen at parc.com (Bill Janssen) Date: Sat, 21 Jun 2008 10:10:06 PDT Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> Message-ID: <08Jun21.101016pdt."58698"@synergy1.parc.xerox.com> > > What follows from that? To me, the natural conclusion is "people who > > witness performance problems just need to despair, or accept them, as > > they can't do anything about it", however, I don't think this is the > > conclusion that you had in mind. > > > > I can say with complete certainty that of the 20+ programmers I've had > working for me, many who have used Python for 3+ years, not a single one > would think to question the garbage collector if they observed the kind of > quadratic time complexity I've demonstrated. This is not because they are > stupid, but because they have only a vague idea that Python even has a > garbage collector, never mind that it could be behaving badly for such > innocuous looking code. Perhaps this is something documentation could help. I'm thinking of a one-page checklist listing places they might look for performance problems, that your programmers could work through. Bill From tjreedy at udel.edu Sat Jun 21 20:40:16 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 21 Jun 2008 14:40:16 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <2e1434c10806210727n1fb136bdhada96c197102adb@mail.gmail.com> Message-ID: Kevin Jacobs wrote: > I can say with complete certainty that of the 20+ programmers I've had > working for me, many who have used Python for 3+ years, not a single one > would think to question the garbage collector if they observed the kind > of quadratic time complexity I've demonstrated. This is not because > they are stupid, but because they have only a vague idea that Python > even has a garbage collector, never mind that it could be behaving badly > for such innocuous looking code. As I understand it, gc is needed now more that ever because new style classes make reference cycles more common. On the other hand, greatly increased RAM size (from some years ago) makes megaobject bursts possible. Such large bursts move the hidden quadratic do-nothing drag out of the relatively flat part of the curve (total time just double or triple what it should be) to where it can really bite. Leaving aside what you do for your local group, can we better warn Python programmers now, for the upcoming 2.5, 2.6, and 3.0 releases? Paragraph 3 of the Reference Manual chapter on Data Model(3.0 version) says: "Objects are never explicitly destroyed; however, when they become unreachable they may be garbage-collected. An implementation is allowed to postpone garbage collection or omit it altogether ? it is a matter of implementation quality how garbage collection is implemented, as long as no objects are collected that are still reachable. (Implementation note: the current implementation uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable, but is not guaranteed to collect garbage containing circular references. See the documentation of the gc module for information on controlling the collection of cyclic garbage.)" I am not sure what to add here, (especially for those who do not read it;-). The Library Manual gc section says "Since the collector supplements the reference counting already used in Python, you can disable the collector if you are sure your program does not create reference cycles." Perhaps it should also say "You should disable when creating millions of objects without cycles". The installed documentation set (on Windows, at least) include some Python HOWTOs. If one were added on Space Management (implementations, problems, and solutions), would your developers read it? > Maybe we should consider more carefully before declaring the status quo > sufficient. Average developers do allocate millions of objects in > bursts and super-linear time complexity for such operations is not > acceptable. Thankfully I am around to help my programmers work around > such issues or else they'd be pushing to switch to Java, Ruby, C#, or > whatever since Python was inexplicably "too slow" for "real work". This > being open source, I'm certainly willing to help in the effort to do so, > but not if potential solutions will be ruled out as being unnecessary. To me, 'sufficient' (time-dependent) and 'necessary' are either too vague or too strict to being about what you want -- change. This is the third thread I have read (here + c.l.p) on default-mode gc problems (but all in the last couple of years or so). So, especially with the nice table someone posted recently, on time with and without gc, and considering that installed RAM continues to grow, I am persuaded that default behavior improvement that does not negatively impact the vast majority would be desirable. Terry Jan Reedy From solipsis at pitrou.net Sat Jun 21 21:10:10 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 21 Jun 2008 21:10:10 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485D231D.9030806@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> Message-ID: <1214075410.5874.26.camel@fsol> Le samedi 21 juin 2008 ? 17:49 +0200, "Martin v. L?wis" a ?crit : > I don't think any strategies based on timing will be successful. > Instead, one should count and analyze objects (although I'm unsure > how exactly that could work). Would it be helpful if the GC was informed of memory growth by the Python memory allocator (that is, each time it either asks or gives back a block of memory to the system allocator) ? From martin at v.loewis.de Sat Jun 21 21:56:44 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 21 Jun 2008 21:56:44 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <1214075410.5874.26.camel@fsol> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> Message-ID: <485D5CFC.5020705@v.loewis.de> Antoine Pitrou wrote: > Le samedi 21 juin 2008 ? 17:49 +0200, "Martin v. L?wis" a ?crit : >> I don't think any strategies based on timing will be successful. >> Instead, one should count and analyze objects (although I'm unsure >> how exactly that could work). > > Would it be helpful if the GC was informed of memory growth by the > Python memory allocator (that is, each time it either asks or gives back > a block of memory to the system allocator) ? I don't see how. The garbage collector is already informed about memory growth; it learns exactly when a container object is allocated or deallocated. That the allocator then requests memory from the system only confirms what the garbage collector already knew: that there are lots of allocated objects. From that, one could infer that it might be time to perform garbage collection - or one could infer that all the objects are really useful, and no garbage can be collected. Regards, Martin From martin at v.loewis.de Sat Jun 21 22:23:16 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Jun 2008 22:23:16 +0200 Subject: [Python-Dev] Proposal: Run GC less often Message-ID: <485D6334.6080102@v.loewis.de> Here is my proposal for making the GC run less often. The objective is to avoid the quadratic-time behavior if many objects are created and none of them is garbage. The youngest generation remains collected after 700 allocations, the middle generation after 10 young collections. Therefore, full GC remains considered every 7000 allocations. The two youngest generations don't cause quadratic behavior, as the number of objects in each generation is bounded. Currently, full GC is invoked every 10 middle collections. Under my proposal, 10 middle collections must have passed, PLUS the number of survivor objects from the middle generation must exceed 10% of the number of objects in the oldest generation. As a consequence, garbage collection becomes less frequent as the number of objects on the heap grows, resulting in an amortized O(N) overhead for garbage collection (under the access pattern stated above). To determine the number of survivor objects, counts are taken during the collection. Objects deallocated through reference counting still remain accounted for as survivors (likewise for determining the size of the oldest generation). I made a simulation assuming an initially-empty heap, and invocations of the collector every M=7000 objects. The table below is in units of M and shows the number of objects allocated, and the number of objects inspected since the start of the simulation, for the old and the new scheme (the asterisk indicates that a collection took place; lines with no collection are dropped): total old_inspected new_inspected 10 10* 10* 20 30* 30* 30 60* 60* 40 100* 100* 50 150* 150* 60 210* 210* 70 280* 280* 80 360* 360* 90 450* 450* 100 550* 450 102 550 552* 110 660* 552 115 660 667* 120 780* 667 129 780 796* 130 910* 796 140 1050* 796 ... 940 44650* 7724 942 44650 8666* 950 45600* 8666 960 46560* 8666 970 47530* 8666 980 48510* 8666 990 49500* 8666 1000 50500* 8666 ... 9990 4995000* 95887 As you can see, the old and the new scheme would start of equally until 100M objects have been allocated. The table shows how the old scheme grows quadratically, and the new scheme eventually approaches roughly a factor of ten between the number of objects and the number of times that GC considers an object. Applications with a small number of objects will see no change in behavior, also, applications that create lots of small cycles will continue to see them collected in the youngest or middle generations. Please let me know what you think. Regards, Martin P.S. I don't plan to implement this myself, so if you like the idea, go ahead. -------------- next part -------------- A non-text attachment was scrubbed... Name: comp.py Type: text/x-python Size: 719 bytes Desc: not available URL: From brett at python.org Sat Jun 21 22:54:51 2008 From: brett at python.org (Brett Cannon) Date: Sat, 21 Jun 2008 13:54:51 -0700 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: <485D6334.6080102@v.loewis.de> References: <485D6334.6080102@v.loewis.de> Message-ID: On Sat, Jun 21, 2008 at 1:23 PM, "Martin v. L?wis" wrote: > Here is my proposal for making the GC run less often. > The objective is to avoid the quadratic-time behavior > if many objects are created and none of them is garbage. > [SNIP] > Applications with a small number of objects will see no > change in behavior, also, applications that create > lots of small cycles will continue to see them collected > in the youngest or middle generations. > > Please let me know what you think. > Interesting. Seems reasonable to me if the impact really is as minimal as you say, Martin. -Brett From stephen at xemacs.org Sat Jun 21 23:36:00 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 22 Jun 2008 06:36:00 +0900 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485D1C27.60408@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> Message-ID: <87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp> "Martin v. L?wis" writes: > Given the choice of "run slower" and "run out of memory", Python should > always prefer the former. > > One approach could be to measure how successful a GC run was: if GC > finds that more-and-more objects get allocated and very few (or none) > are garbage, it might conclude that this is an allocation spike, and > back off. The tricky question is how to find out that the spike is > over. XEmacs implements this strategy in a way which is claimed to give constant amortized time (ie, averaged over memory allocated). I forget the exact parameters, but ISTR it's just period ("time" measured by bytes allocated) is proportional to currently allocated memory. Some people claim this is much more comfortable than the traditional "GC after N bytes are allocated" algorithm but I don't notice much difference. I don't know how well this intuition carries over to noninteractive applications. In XEmacs experimenting with such strategies is pretty easy, since the function that determines period is only a few lines long. I assume that would be true of Python, too. However, isn't the real question whether there is memory pressure or not? If you've got an unloaded machine with 2GB of memory, even a 1GB spike might have no observable consequences. How about a policy of GC-ing with decreasing period ("time" measured by bytes allocated or number of allocations) as the fraction of memory used increases, starting from a pretty large fraction (say 50% by default)? The total amount of memory could be a soft limit, defaulting to the amount of fast memory actually available. For interactive and maybe some batch applications, it might be appropriate to generate a runtime warning as memory use approches some limits, too. Nevertheless, I think the real solution has to be for Python programmers to be aware that there is GC, and that they can tune it. From ralfs72 at googlemail.com Sat Jun 21 23:28:22 2008 From: ralfs72 at googlemail.com (none) Date: Sat, 21 Jun 2008 23:28:22 +0200 Subject: [Python-Dev] Another Proposal: Run GC less often Message-ID: <485d7277.0b38560a.065a.ffffcada@mx.google.com> Instead of collecting objects after a fixed number of allocations (700) You could make it dynamic like this: # initial values min_allocated_memory = 0 max_allocated_memory = 0 next_gc_run = 1024 * 1024 def manage_memory(): allocated_memory = get_allocated_memory() min_allocated_memory = min(min_allocated_memory, allocated_memory) max_allocated_memory = max(max_allocated_memory, allocated_memory) if max_allocated_memory - min_allocated_memory > next_gc_run: # run the gc memory_freed, allocated_memory = run_gc() next_gc_run = max( allocated_memory * 1.5 - memory_freed, 1024 * 1024 ) min_allocated_memory = allocated_memory max_allocated_memory = allocated_memory manage_memory() should be called after every allocation and after a ref count of an object reaches 0 (memory is freed) Expected behaviours: => As less objects contain cyclic references as less often the GC will run (memory_freed is small) => As more objects contain cyclic references as more often the GC will run (memory_freed is large) => If memory utiliaziation grows fast (burst allocations) GC will run less often: next_gc_run = allocated_memory * 1.5 - memory_freed ... Of course the constants: 1.5 and 1024 * 1024 are only suggestions... - Ralf From adlaiff6 at gmail.com Sat Jun 21 23:33:15 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sat, 21 Jun 2008 14:33:15 -0700 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: <485D6334.6080102@v.loewis.de> References: <485D6334.6080102@v.loewis.de> Message-ID: If you can get me a version of the interpreter with this change made (I wouldn't know what to change), I can run a very allocation/deallocation-heavy application I have lying around, and get you some good benchmarks. On Sat, Jun 21, 2008 at 1:23 PM, "Martin v. L?wis" wrote: - Show quoted text - > Here is my proposal for making the GC run less often. > The objective is to avoid the quadratic-time behavior > if many objects are created and none of them is garbage. > > The youngest generation remains collected after 700 > allocations, the middle generation after 10 young collections. > Therefore, full GC remains considered every 7000 allocations. > The two youngest generations don't cause quadratic behavior, > as the number of objects in each generation is bounded. > > Currently, full GC is invoked every 10 middle collections. > Under my proposal, 10 middle collections must have passed, > PLUS the number of survivor objects from the middle generation > must exceed 10% of the number of objects in the oldest > generation. > > As a consequence, garbage collection becomes less frequent > as the number of objects on the heap grows, resulting in > an amortized O(N) overhead for garbage collection (under > the access pattern stated above). > > To determine the number of survivor objects, counts are > taken during the collection. Objects deallocated through > reference counting still remain accounted for as survivors > (likewise for determining the size of the oldest generation). > > I made a simulation assuming an initially-empty heap, > and invocations of the collector every M=7000 objects. > The table below is in units of M and shows the number > of objects allocated, and the number of objects inspected > since the start of the simulation, for the old and the > new scheme (the asterisk indicates that a collection > took place; lines with no collection are dropped): > > total old_inspected new_inspected > 10 10* 10* > 20 30* 30* > 30 60* 60* > 40 100* 100* > 50 150* 150* > 60 210* 210* > 70 280* 280* > 80 360* 360* > 90 450* 450* > 100 550* 450 > 102 550 552* > 110 660* 552 > 115 660 667* > 120 780* 667 > 129 780 796* > 130 910* 796 > 140 1050* 796 > ... > 940 44650* 7724 > 942 44650 8666* > 950 45600* 8666 > 960 46560* 8666 > 970 47530* 8666 > 980 48510* 8666 > 990 49500* 8666 > 1000 50500* 8666 > ... > 9990 4995000* 95887 > > As you can see, the old and the new scheme would start > of equally until 100M objects have been allocated. The > table shows how the old scheme grows quadratically, and > the new scheme eventually approaches roughly a factor > of ten between the number of objects and the number of > times that GC considers an object. > > Applications with a small number of objects will see no > change in behavior, also, applications that create > lots of small cycles will continue to see them collected > in the youngest or middle generations. > > Please let me know what you think. > > Regards, > Martin > > P.S. I don't plan to implement this myself, so if you like > the idea, go ahead. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/adlaiff6%40gmail.com > > -- Cheers, Leif From leif.walsh at gmail.com Sat Jun 21 23:45:55 2008 From: leif.walsh at gmail.com (Leif Walsh) Date: Sat, 21 Jun 2008 14:45:55 -0700 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1 In-Reply-To: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> Message-ID: There is a typo on the download page (http://python.org/download/releases/3.0/): """The release plan is to have a series of alpha releases in 2007, beta releases in 2008, and a final release in September 2008. The alpha releases are primarily aimed at developers who want a sneak peek at the new langauge, especially those folks who plan to port their code to Python 3000. The hope is that by the time of the final release, many 3rd party packages will already be available in a 3.0-compatible form. """ Should be """The release plan is to have a series of alpha releases in 2007, beta releases in 2008, and a final release in September 2008. The alpha releases are primarily aimed at developers who want a sneak peek at the new *language*, especially those folks who plan to port their code to Python 3000. The hope is that by the time of the final release, many 3rd party packages will already be available in a 3.0-compatible form. """ -- Cheers, Leif From adlaiff6 at gmail.com Sat Jun 21 23:48:05 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sat, 21 Jun 2008 14:48:05 -0700 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1 In-Reply-To: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> Message-ID: There is a typo on the download page (http://python.org/download/releases/3.0/): """The release plan is to have a series of alpha releases in 2007, beta releases in 2008, and a final release in September 2008. The alpha releases are primarily aimed at developers who want a sneak peek at the new langauge, especially those folks who plan to port their code to Python 3000. The hope is that by the time of the final release, many 3rd party packages will already be available in a 3.0-compatible form. """ Should be """The release plan is to have a series of alpha releases in 2007, beta releases in 2008, and a final release in September 2008. The alpha releases are primarily aimed at developers who want a sneak peek at the new *language*, especially those folks who plan to port their code to Python 3000. The hope is that by the time of the final release, many 3rd party packages will already be available in a 3.0-compatible form. """ (I really need to add my other e-mail address to these lists...sorry if my first message got through to one of them and this is a dupe.) -- Cheers, Leif From barry at python.org Sun Jun 22 00:55:33 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Jun 2008 18:55:33 -0400 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6b1 and 3.0b1 In-Reply-To: References: <1972109D-735D-4485-82F4-9BC8F2984967@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 21, 2008, at 5:45 PM, Leif Walsh wrote: > There is a typo on the download page (http://python.org/download/releases/3.0/ > ): Fixed, thanks! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSF2G5nEjvBPtnXfVAQJbqwQAkEMtWMYdsMFWnGmQCrpEV3BXZGipmZQs 7dnWoJIVsjN7pBY/5aVMiQEl5JM3n8IdVpwZMWM9Q+VHgkB04u8f7/wS2xmR7XsA dySDMrxUJXqXX1ze/vWKosp39Yvey6HeTsiOUzvnXjS6IKM757IhyB94o4KQF69K mOS8tjydwE8= =cY+d -----END PGP SIGNATURE----- From martin at v.loewis.de Sun Jun 22 01:00:41 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 22 Jun 2008 01:00:41 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <485D8819.1040708@v.loewis.de> > XEmacs implements this strategy in a way which is claimed to give > constant amortized time (ie, averaged over memory allocated). See my recent proposal. The old trick is to do reorganizations in a fixed fraction of the total size, resulting in a per-increase amortized-constant overhead (assuming each reorganization takes time linear with total size). > However, isn't the real question whether there is memory pressure or > not? If you've got an unloaded machine with 2GB of memory, even a 1GB > spike might have no observable consequences. How about a policy of > GC-ing with decreasing period ("time" measured by bytes allocated or > number of allocations) as the fraction of memory used increases, > starting from a pretty large fraction (say 50% by default)? The problem with such an approach is that it is very difficult to measure. What to do about virtual memory? What to do about other applications that also consume memory? On some systems (Windows in particular), the operating system indicates memory pressure through some IPC mechanism; on such systems, it might be reasonable to perform garbage collection (only?) when the system asks for it. However, the system might not ask for GC while the swap space is still not exhausted, meaning that the deferred GC would take a long time to complete (having to page in every object). > Nevertheless, I think the real solution has to be for Python > programmers to be aware that there is GC, and that they can tune it. I don't think there is a "real solution". I think programmers should abstain from complaining if they can do something about the problem in their own application (unless the complaint is formulated as a patch) - wait - I think programmers should abstain from complaining, period. Regards, Martin From greg.ewing at canterbury.ac.nz Sun Jun 22 03:16:59 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 22 Jun 2008 13:16:59 +1200 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: <485D6334.6080102@v.loewis.de> References: <485D6334.6080102@v.loewis.de> Message-ID: <485DA80B.2080404@canterbury.ac.nz> Martin v. L?wis wrote: > Under my proposal, 10 middle collections must have passed, > PLUS the number of survivor objects from the middle generation > must exceed 10% of the number of objects in the oldest > generation. What happens if the program enters a phase where it's not producing any new cyclic garbage, but is breaking references among the old objects in such a way that cycles of them are being left behind? Under this rule, the oldest generation would never be scanned, so those cycles would never be collected. > As a consequence, garbage collection becomes less frequent > as the number of objects on the heap grows Wouldn't it be simpler just to base the collection frequency directly on the total number of objects in the heap? From what another poster said, this seems to be what emacs does. -- Greg From ncoghlan at gmail.com Sun Jun 22 03:56:39 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Jun 2008 11:56:39 +1000 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485D5CFC.5020705@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> Message-ID: <485DB157.1080907@gmail.com> Martin v. L?wis wrote: > Antoine Pitrou wrote: >> Le samedi 21 juin 2008 ? 17:49 +0200, "Martin v. L?wis" a ?crit : >>> I don't think any strategies based on timing will be successful. >>> Instead, one should count and analyze objects (although I'm unsure >>> how exactly that could work). >> Would it be helpful if the GC was informed of memory growth by the >> Python memory allocator (that is, each time it either asks or gives back >> a block of memory to the system allocator) ? > > I don't see how. The garbage collector is already informed about memory > growth; it learns exactly when a container object is allocated or > deallocated. That the allocator then requests memory from the system > only confirms what the garbage collector already knew: that there are > lots of allocated objects. From that, one could infer that it might > be time to perform garbage collection - or one could infer that all > the objects are really useful, and no garbage can be collected. I was wondering whether it might be useful to detect the end of an allocation spike: if PyMalloc incremented an internal counter each time it requested more memory from the system, then the GC could check whether that number had changed on each collection cycle. If the GC goes a certain number of collection cycles without more memory being requested from the system, then it can assume the allocation spike is over and tighten up the default tuning again. Such a pymalloc counter would provide a slightly more holistic view of overall memory usage changes than the container-focused view of the cyclic GC. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tonynelson at georgeanelson.com Sun Jun 22 04:40:48 2008 From: tonynelson at georgeanelson.com (Tony Nelson) Date: Sat, 21 Jun 2008 22:40:48 -0400 Subject: [Python-Dev] Another Proposal: Run GC less often In-Reply-To: <485d7277.0b38560a.065a.ffffcada@mx.google.com> References: <485d7277.0b38560a.065a.ffffcada@mx.google.com> Message-ID: At 11:28 PM +0200 6/21/08, none wrote: >Instead of collecting objects after a fixed number of allocations (700) ... I've seen this asserted several times in this thread: that GC is done every fixed number of allocations. This is not correct. GC is done when the surplus of allocations less deallocations exceeds a threashold. See Modules/gcmodule.c and look for ".count++" and ".count--". In normal operation, allocations and deallocations stay somewhat balanced, but when creating a large data structure, it's allocations all the way and GC runs often. -- ____________________________________________________________________ TonyN.:' ' From stephen at xemacs.org Sun Jun 22 05:43:26 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 22 Jun 2008 12:43:26 +0900 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485D8819.1040708@v.loewis.de> References: <485AFBAB.2060404@canterbury.ac.nz> <2e1434c10806200715x42855c3cs213bf447e47de81c@mail.gmail.com> <2e1434c10806200804o4404cd9eo606c5d95e64a6171@mail.gmail.com> <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <87bq1umpxb.fsf@uwakimon.sk.tsukuba.ac.jp> <485D8819.1040708@v.loewis.de> Message-ID: <877icim8wx.fsf@uwakimon.sk.tsukuba.ac.jp> "Martin v. L?wis" writes: > > XEmacs implements this strategy in a way which is claimed to give > > constant amortized time (ie, averaged over memory allocated). > > See my recent proposal. I did, crossed in the mail. To the extent that I understand both systems, your proposal looks like an improvement over what we've got. > > However, isn't the real question whether there is memory pressure or > > not? If you've got an unloaded machine with 2GB of memory, even a 1GB > > spike might have no observable consequences. How about a policy of > > GC-ing with decreasing period ("time" measured by bytes allocated or > > number of allocations) as the fraction of memory used increases, > > starting from a pretty large fraction (say 50% by default)? > > The problem with such an approach is that it is very difficult to > measure. On some systems it should be possible to get information about how much paging is taking place, which would be an indicator of pressure. > What to do about virtual memory? If you're in virtual memory, you're already in trouble. Once you're paging, you need to increase time between GCs, so the rule would not be monotonic. From martin at v.loewis.de Sun Jun 22 07:13:15 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 22 Jun 2008 07:13:15 +0200 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: <485DA80B.2080404@canterbury.ac.nz> References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz> Message-ID: <485DDF6B.9050401@v.loewis.de> > What happens if the program enters a phase where it's not > producing any new cyclic garbage, but is breaking references > among the old objects in such a way that cycles of them > are being left behind? Under this rule, the oldest > generation would never be scanned, so those cycles would > never be collected. Correct. However, this is the case already today. > Wouldn't it be simpler just to base the collection frequency > directly on the total number of objects in the heap? Using what precise formula? Regards, Martin From facundobatista at gmail.com Sun Jun 22 15:05:47 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Sun, 22 Jun 2008 10:05:47 -0300 Subject: [Python-Dev] Issue 2722 In-Reply-To: References: Message-ID: 2008/6/21 Neil Muller : > Could some one have a look at the suggested fix for issue 2722? While > not a particularly common problem, it can be an annoyance, and the > patch looks reasonable. I'm on it... Python Bug Day! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From solipsis at pitrou.net Sun Jun 22 16:32:09 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 22 Jun 2008 14:32:09 +0000 (UTC) Subject: [Python-Dev] Proposal: Run GC less often References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > What happens if the program enters a phase where it's not > producing any new cyclic garbage, but is breaking references > among the old objects in such a way that cycles of them > are being left behind? Under this rule, the oldest > generation would never be scanned, so those cycles would > never be collected. We could introduce a kind of "timing rule" such that there is at least one full collection, say, every minute. While timing is not relevant to memory management, it is relevant to the user behind the keyboard. In any case, I think MvL's suggestion is worth trying. Regards Antoine. From nas at arctrix.com Sun Jun 22 21:27:27 2008 From: nas at arctrix.com (Neil Schemenauer) Date: Sun, 22 Jun 2008 19:27:27 +0000 (UTC) Subject: [Python-Dev] Proposal: Run GC less often References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Martin v. L?wis wrote: > >> Under my proposal, 10 middle collections must have passed, >> PLUS the number of survivor objects from the middle generation >> must exceed 10% of the number of objects in the oldest >> generation. > > What happens if the program enters a phase where it's not > producing any new cyclic garbage, but is breaking references > among the old objects in such a way that cycles of them > are being left behind? Under this rule, the oldest > generation would never be scanned, so those cycles would > never be collected. Another problem is that the program could be slowly leaking and a full collection will never happen. >> As a consequence, garbage collection becomes less frequent >> as the number of objects on the heap grows > > Wouldn't it be simpler just to base the collection frequency > directly on the total number of objects in the heap? > From what another poster said, this seems to be what > emacs does. I like simple. The whole generational collection scheme was dreamed up by me early in the GC's life. There was not a lot of thought or benchmarking put into it since at that time I was more focused on getting the basic GC working. At some later point some tuning was done on the collection frequencies but that 10 middle collections scheme was never deeply investigated, AFAIK. BTW, I suspect that documentation needs updating since I understand that the GC is no longer optional (the stdlib and/or the Python internals create reference cycles themselves). Neil From tomerfiliba at gmail.com Sun Jun 22 21:28:13 2008 From: tomerfiliba at gmail.com (tomer filiba) Date: Sun, 22 Jun 2008 12:28:13 -0700 (PDT) Subject: [Python-Dev] forceful exit Message-ID: <07e859dc-c9cf-476f-8a9a-85f4f990de79@2g2000hsn.googlegroups.com> hi i'm having trouble when forking child processes to serve sockets. the skeleton is something like the following: def work(): try: while True: s = listener.accept()[0] log("hello %s", s) if os.fork() == 0: try: serve_client(sock) finally: sys.exit() #### <<<< (1) else: log("forked child") sock.close() except KeyboardInterrupt: log("got ctrl+c") finally: log("server terminated") listener.close() the problem is that sys.exit() raises an exception, which propagates all the way up and is handled by the finallies... the real code does more than just printing logs, so i can't allow that. i'm forced to resort to os._exit, which does the trick but doesn't perform cleanup, which made me realize there's no clean way in python to *force* exit. i think there ought to be something like sys.forcedexit(), which collects all objects nicely and then exits immediately without letting anything propagate up. this will also solve another problem i came across, with threads. turns out only the main thread can kill the process -- if another thread issues sys.exit, it only kills itself. there's no clean way for a thread to terminate the process... i think there must be. i can contribute a patch in a matter of days... i think it's pretty straight forward (calling Py_Finalize and then exit). aye or nay? -tomer From martin at v.loewis.de Sun Jun 22 22:36:19 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 22 Jun 2008 22:36:19 +0200 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz> Message-ID: <485EB7C3.9060602@v.loewis.de> > Another problem is that the program could be slowly leaking and a > full collection will never happen. I don't think that will be possible. If the program slowly leaks, survivor objects leave the middle generation, and account towards the 10%. As the count of objects in the oldest generation doesn't change, collection will eventually occur. However, it may occur much later than it currently does, if you have many objects on the heap, and each middle collection only has few survivors. One may argue that if the machine had the space to keep N objects in memory, it probably can also keep 1.1N objects in memory. Regards, Martin From tim.peters at gmail.com Sun Jun 22 23:10:52 2008 From: tim.peters at gmail.com (Tim Peters) Date: Sun, 22 Jun 2008 17:10:52 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <485D5CFC.5020705@v.loewis.de> References: <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> Message-ID: <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> [Antoine Pitrou] >> Would it be helpful if the GC was informed of memory growth by the >> Python memory allocator (that is, each time it either asks or gives back >> a block of memory to the system allocator) ? [Martin v. L?wis] > I don't see how. The garbage collector is already informed about memory > growth; it learns exactly when a container object is allocated or > deallocated. That the allocator then requests memory from the system > only confirms what the garbage collector already knew: that there are > lots of allocated objects. From that, one could infer that it might > be time to perform garbage collection - or one could infer that all > the objects are really useful, and no garbage can be collected. Really the same conundrum we currently face: cyclic gc is currently triggered by reaching a certain /excess/ of allocations over deallocations. From that we /do/ infer it's time to perform garbage collection -- but, as some examples here showed, it's sometimes really the case that the true meaning of the excess is that "all the objects are really useful, and no garbage can be collected -- and I'm creating a lot of them". pymalloc needing to allocate a new arena would be a different way to track an excess of allocations over deallocations, and in some ways more sensible (since it would reflect an excess of /bytes/ allocated over bytes freed, rather than an excess in the counts of objects allocated-over-freed regardless of their sizes -- an implication is, e.g., that cyclic gc would be triggered much less frequently by mass creation of small tuples than of small dicts, since a small tuple consumes much less memory than a small dict). Etc. ;-) From greg.ewing at canterbury.ac.nz Mon Jun 23 03:10:51 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 23 Jun 2008 13:10:51 +1200 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: <485DDF6B.9050401@v.loewis.de> References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz> <485DDF6B.9050401@v.loewis.de> Message-ID: <485EF81B.9080901@canterbury.ac.nz> Martin v. L?wis wrote: >>Wouldn't it be simpler just to base the collection frequency >>directly on the total number of objects in the heap? > > Using what precise formula? The simplest thing to try would be middle_collections >= num_objects_in_heap * some_constant -- Greg From martin at v.loewis.de Mon Jun 23 05:32:01 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 23 Jun 2008 05:32:01 +0200 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: <485EF81B.9080901@canterbury.ac.nz> References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz> <485DDF6B.9050401@v.loewis.de> <485EF81B.9080901@canterbury.ac.nz> Message-ID: <485F1931.7000804@v.loewis.de> >>> Wouldn't it be simpler just to base the collection frequency >>> directly on the total number of objects in the heap? >> >> Using what precise formula? > > The simplest thing to try would be > > middle_collections >= num_objects_in_heap * some_constant So what value is some_constant? Regards, Martin From martin at v.loewis.de Mon Jun 23 05:39:06 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 23 Jun 2008 05:39:06 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> References: <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> Message-ID: <485F1ADA.6090107@v.loewis.de> > pymalloc needing to allocate a new arena would be a different way to > track an excess of allocations over deallocations, and in some ways > more sensible (since it would reflect an excess of /bytes/ allocated > over bytes freed, rather than an excess in the counts of objects > allocated-over-freed regardless of their sizes -- an implication is, > e.g., that cyclic gc would be triggered much less frequently by mass > creation of small tuples than of small dicts, since a small tuple > consumes much less memory than a small dict). > > Etc. ;-) :-) So my question still is: how exactly? Currently, only youngest collections are triggered by allocation rate; middle and old are triggered by frequency of youngest collection. So would you now specify that the youngest collection should occur if-and-only-if a new arena is allocated? Or discount arenas returned from arenas allocated? Or apply this to triggering other generation collections but youngest? How would that help the quadratic behavior (which really needs to apply a factor somewhere)? Regards, Martin From tjreedy at udel.edu Mon Jun 23 08:13:55 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Jun 2008 02:13:55 -0400 Subject: [Python-Dev] Proposal: Run GC less often In-Reply-To: References: <485D6334.6080102@v.loewis.de> <485DA80B.2080404@canterbury.ac.nz> Message-ID: Neil Schemenauer wrote: > > BTW, I suspect that documentation needs updating since I understand > that the GC is no longer optional (the stdlib and/or the Python > internals create reference cycles themselves). Is it possible and might it be useful for those internal cycle-creating operations to increment a counter that was part of the gc trigger? Doing millions of 'safe' operations would then leave the counter alone and could have less effect in triggering gc. tjr From michael at araneidae.co.uk Mon Jun 23 09:02:52 2008 From: michael at araneidae.co.uk (Michael Abbott) Date: Mon, 23 Jun 2008 07:02:52 +0000 (GMT) Subject: [Python-Dev] Suggested API change to PyOS_InputHook Message-ID: <20080623064601.M29284@saturn.araneidae.co.uk> I have a problem with the PyOS_InputHook() API as implemented by Modules/readline.c: there is no way to communicate any interruption seen while waiting at the keyboard back to the process actually reading the line. I have solved this in my own application by creating a new module which reimplements the call_readline function and which has the following patch to readline_until_enter_or_signal(): FD_SET(fileno(rl_instream), &selectset); /* select resets selectset if no input was available */ has_input = select(fileno(rl_instream) + 1, &selectset, NULL, NULL, &timeout); - if(PyOS_InputHook) PyOS_InputHook(); + if (call_readline_InputHook && + call_readline_InputHook(fileno(rl_instream))) + { + has_input = -1; + PyErr_SetInterrupt(); + break; + } } if(has_input > 0) { rl_callback_read_char(); } - else if (errno == EINTR) { + else if (errno == EINTR || has_input < 0) { int s; #ifdef WITH_THREAD PyEval_RestoreThread(_PyOS_ReadlineTState); (I'm sorry, this isn't a proper patch, but hopefully if you've read this far you can figure out where it applies.) Here I've replaced PyOS_InputHook with effective type signature None->None with my own call_readline_InputHook which takes a file descriptor and returns a boolean, returning True iff an interrupt occurred. The file descriptor argument is a triviality, but the boolean return code is clearly crucial. Now, I don't know how this change might fit into mainstream Python, but I think there is a case for making this change. If the incompatibility was acceptable it would be better to change the signature of PyOS_InputHook (I declared my own variable call_readline_InputHook just to avoid accidents). I've checked the current svn trunk, and this change remains applicable. Please CC me in any replies: I am quite literally unable to cope with the volume of traffic that a subscription to python-dev will give me. From solipsis at pitrou.net Mon Jun 23 12:40:26 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 23 Jun 2008 10:40:26 +0000 (UTC) Subject: [Python-Dev] C API for gc.enable() and gc.disable() References: <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > Currently, only youngest collections are triggered by allocation > rate; middle and old are triggered by frequency of youngest collection. > So would you now specify that the youngest collection should occur > if-and-only-if a new arena is allocated? Or discount arenas returned > from arenas allocated? The latter sounds reasonable. IIRC an arena is 256KB, which is less than an entry level L2 cache. Therefore waiting for an arena to be filled shouldn't deteriorate cache locality a lot. To avoid situations where the GC is never called we could combine that with an allocation counter, but with a much higher threshold than currently. > Or apply this to triggering other generation > collections but youngest? How would that help the quadratic behavior > (which really needs to apply a factor somewhere)? It would not help the quadratic behaviour - and is orthogonal to your proposal - , but at least avoid calling the GC too often when lots of small objects are allocated (as opposed to lots of large objects). From aahz at pythoncraft.com Mon Jun 23 16:54:52 2008 From: aahz at pythoncraft.com (Aahz) Date: Mon, 23 Jun 2008 07:54:52 -0700 Subject: [Python-Dev] Suggested API change to PyOS_InputHook In-Reply-To: <20080623064601.M29284@saturn.araneidae.co.uk> References: <20080623064601.M29284@saturn.araneidae.co.uk> Message-ID: <20080623145452.GB13722@panix.com> On Mon, Jun 23, 2008, Michael Abbott wrote: > > Please CC me in any replies: I am quite literally unable to cope with the > volume of traffic that a subscription to python-dev will give me. Then don't bother posting code to the list. Please post a patch; up to you whether you send a notice to python-dev. And expect that you'll probably miss at least some replies. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha From michael at araneidae.co.uk Mon Jun 23 18:34:02 2008 From: michael at araneidae.co.uk (Michael Abbott) Date: Mon, 23 Jun 2008 16:34:02 +0000 (GMT) Subject: [Python-Dev] Suggested API change to PyOS_InputHook In-Reply-To: <20080623145452.GB13722@panix.com> References: <20080623064601.M29284@saturn.araneidae.co.uk> <20080623145452.GB13722@panix.com> Message-ID: <20080623163312.S32297@saturn.araneidae.co.uk> On Mon, 23 Jun 2008, Aahz wrote: > Then don't bother posting code to the list. Please post a patch; up to > you whether you send a notice to python-dev. Very well. Issue 3180 created. I hope I'm doing it right... From tjreedy at udel.edu Mon Jun 23 20:52:44 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Jun 2008 14:52:44 -0400 Subject: [Python-Dev] Suggested API change to PyOS_InputHook In-Reply-To: <20080623064601.M29284@saturn.araneidae.co.uk> References: <20080623064601.M29284@saturn.araneidae.co.uk> Message-ID: Michael Abbott wrote: > Please CC me in any replies: I am quite literally unable to cope with the > volume of traffic that a subscription to python-dev will give me. You can read python-dev (and 1000s of other mail lists, with a newsreader, like I do, for the same reason, via the free list to news gateway news.gmane.com tjr From martin at v.loewis.de Mon Jun 23 23:08:29 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 23 Jun 2008 23:08:29 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> Message-ID: <486010CD.2000904@v.loewis.de> > It would not help the quadratic behaviour - and is orthogonal to your proposal - > , but at least avoid calling the GC too often when lots of small objects are > allocated (as opposed to lots of large objects). I'd like to see in an experiment whether this is really true. Current GC only takes into account container objects, which, most significantly, ignores string objects (of which most applications create plenty). If you make actual memory consumption a trigger, the string objects would count towards triggering GC. That may or may not be the better approach, anyway - but it's not certain (for me) that such a scheme would cause GC to be triggered less often, in a typical application. Regards, Martin From solipsis at pitrou.net Tue Jun 24 11:36:13 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 24 Jun 2008 09:36:13 +0000 (UTC) Subject: [Python-Dev] C API for gc.enable() and gc.disable() References: <1214012386.6402.4.camel@fsol> <485CBCDA.4020601@v.loewis.de> <1214054311.5874.12.camel@fsol> <485D1C27.60408@v.loewis.de> <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > I'd like to see in an experiment whether this is really true. Right, all those ideas should be implemented and tried out. I don't really have time to spend on it right now. Also, what's missing is a suite of performance/efficiency tests for the garbage collector. From hsoft at hardcoded.net Tue Jun 24 12:18:37 2008 From: hsoft at hardcoded.net (Virgil Dupras) Date: Tue, 24 Jun 2008 12:18:37 +0200 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib Message-ID: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> Hi Devs, Today, I saw the Py3k beta 1 announcement, so I thought "Well, I gotta do my duty and try to migrate my own code base to see if I can uncover any problems". I'm just at the step where I run my test suite with the "-3" flag, and I noticed something peculiar: I get DeprecationWarning from stdlib code (namely, difflib, logging and unittest). Is it something to be expected or should I create tickets for those? I'm sure that the stdlib on the 3k side is fixed, but shouldn't the 2.6 side be fixed as well? Sorry if this question is already answered somewhere else, I searched for an answer but I couldn't find it. Additionally, one thing that strikes me as strange is how hard it is to find a "Migration guide" on the website. I had to dig in the PEP 3000 to find the brief bullet list telling me to run py26 with the -3 flag. Virgil Dupras From ncoghlan at gmail.com Tue Jun 24 12:34:23 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Jun 2008 20:34:23 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> Message-ID: <4860CDAF.2020806@gmail.com> Virgil Dupras wrote: > Hi Devs, > > Today, I saw the Py3k beta 1 announcement, so I thought "Well, I gotta > do my duty and try to migrate my own code base to see if I can uncover > any problems". I'm just at the step where I run my test suite with the > "-3" flag, and I noticed something peculiar: I get DeprecationWarning > from stdlib code (namely, difflib, logging and unittest). Is it > something to be expected or should I create tickets for those? I'm > sure that the stdlib on the 3k side is fixed, but shouldn't the 2.6 > side be fixed as well? I'd say so, yes. If tracker issues don't exist yet, please create them. The focus for the first beta was really on getting all the syntax and API changes stable - the remaining time between the first beta release and the first release candidate will focus on cleaning up issues like this. > Sorry if this question is already answered somewhere else, I searched > for an answer but I couldn't find it. > > Additionally, one thing that strikes me as strange is how hard it is > to find a "Migration guide" on the website. I had to dig in the PEP > 3000 to find the brief bullet list telling me to run py26 with the -3 > flag. Probably because nobody has stepped up to write a migration HOWTO yet... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From hsoft at hardcoded.net Tue Jun 24 12:43:36 2008 From: hsoft at hardcoded.net (Virgil Dupras) Date: Tue, 24 Jun 2008 12:43:36 +0200 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <4860CDAF.2020806@gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> Message-ID: <2a70578d0806240343i5aa9ef53ye831e27d4e81baf2@mail.gmail.com> On Tue, Jun 24, 2008 at 12:34 PM, Nick Coghlan wrote: > Probably because nobody has stepped up to write a migration HOWTO yet... I'm not much of a writer, but once I'm finished with the migration, I'll do it (if it hasn't been done then). It's about time I try to learn reST... Virgil Dupras From fuzzyman at voidspace.org.uk Tue Jun 24 12:50:16 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 24 Jun 2008 11:50:16 +0100 Subject: [Python-Dev] Python Windows Installers Message-ID: <4860D168.9050305@voidspace.org.uk> Hello all, I'm just doing some housekeeping on a Windows install, and notice that the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin v. Lowis". Whilst I *personally* find this very reassuring I wonder if this is intended / ideal? All the best, Michael Foord From fuzzyman at voidspace.org.uk Tue Jun 24 12:52:19 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 24 Jun 2008 11:52:19 +0100 Subject: [Python-Dev] Python Windows Installers In-Reply-To: <4860D168.9050305@voidspace.org.uk> References: <4860D168.9050305@voidspace.org.uk> Message-ID: <4860D1E3.3090906@voidspace.org.uk> Michael Foord wrote: > Hello all, > > I'm just doing some housekeeping on a Windows install, and notice that > the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin > v. Lowis". Whilst I *personally* find this very reassuring I wonder if > this is intended / ideal? > Just to clarify, this is on Windows Vista and shown in the Control Panel "Programs and Features" (what used to be "Add/Remove Programs"). Installers were from Python.org. Michael Foord > All the best, > > Michael Foord > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From musiccomposition at gmail.com Tue Jun 24 14:58:05 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 24 Jun 2008 07:58:05 -0500 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <4860CDAF.2020806@gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> Message-ID: <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan wrote: > I'd say so, yes. If tracker issues don't exist yet, please create them. The > focus for the first beta was really on getting all the syntax and API > changes stable - the remaining time between the first beta release and the > first release candidate will focus on cleaning up issues like this. I wonder if we could just 2to3 (with fixers that create incompatibilities disabled) over the whole stdlib and see what happens. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From ncoghlan at gmail.com Tue Jun 24 15:08:47 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Jun 2008 23:08:47 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> Message-ID: <4860F1DF.5080204@gmail.com> Benjamin Peterson wrote: > On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan wrote: >> I'd say so, yes. If tracker issues don't exist yet, please create them. The >> focus for the first beta was really on getting all the syntax and API >> changes stable - the remaining time between the first beta release and the >> first release candidate will focus on cleaning up issues like this. > > I wonder if we could just 2to3 (with fixers that create > incompatibilities disabled) over the whole stdlib and see what > happens. Just running the unit test suite with -3 would probably be a better starting point. 2to3 makes a lot of extra changes that we don't particularly care about. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From musiccomposition at gmail.com Tue Jun 24 15:11:00 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 24 Jun 2008 08:11:00 -0500 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <4860F1DF.5080204@gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> <4860F1DF.5080204@gmail.com> Message-ID: <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com> On Tue, Jun 24, 2008 at 8:08 AM, Nick Coghlan wrote: > Benjamin Peterson wrote: >> >> On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan wrote: >>> >>> I'd say so, yes. If tracker issues don't exist yet, please create them. >>> The >>> focus for the first beta was really on getting all the syntax and API >>> changes stable - the remaining time between the first beta release and >>> the >>> first release candidate will focus on cleaning up issues like this. >> >> I wonder if we could just 2to3 (with fixers that create >> incompatibilities disabled) over the whole stdlib and see what >> happens. > > Just running the unit test suite with -3 would probably be a better starting > point. 2to3 makes a lot of extra changes that we don't particularly care > about. Right, which is why I suggest running a few fixers (ie fix_has_key) and then tearing the test suite apart. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From ncoghlan at gmail.com Tue Jun 24 15:17:50 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Jun 2008 23:17:50 +1000 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: <4860ED9F.8030305@trueblade.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> Message-ID: <4860F3FE.8090307@gmail.com> Eric Smith wrote: > Note that in > http://mail.python.org/pipermail/python-dev/2008-February/077062.html, > Guido advocates not adding the __bin__ machinery, which is what lead to > the simple implementation of bin() just calling PyNumber_ToBase and > relying on __index__. > > I don't think __bin__ should be added to 2.6. I don't see the point in > adding a feature in 2.6 that's implemented so differently in 3.0. It's > just asking for porting hassles. > > Instead, I think the approach used in 3.0 (r64451) should be used > instead. That is, if this feature exist at all. I'm -0 on adding > bin(), etc. to floats. The 3.0 approach means that non-float floating point types still can't be displayed properly by bin()/oct()/hex(). The current 2.6 approach means every such class has to implement its own equivalent of PyNumber_ToBase. Both are lousy solutions to a relative non-problem (IMO) that can easily be implemented using a separate display function for those applications which happen to need it. I'd prefer to see a proper PEP for this proposing a new slot that lets any class return an (integer_part, fraction_part) tuple of integers, and have PyNumber_ToBase take care of the actual string formatting. Introducing such a gratuitous incompatibility between 2.6 and 3.0 at this late stage of the process certainly seems highly undesirable to me. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Tue Jun 24 15:56:37 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 24 Jun 2008 06:56:37 -0700 Subject: [Python-Dev] Implmentation details for bin/oct/hex References: <20080620041816.4D5E81E4002@bag.python.org><4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> Message-ID: <8A1B7D10023C4F259E63D96451590855@RaymondLaptop1> From: "Nick Coghlan" > The 3.0 approach means that non-float floating point types still can't be displayed properly by bin()/oct()/hex(). The current 2.6 > approach means every such class has to implement its own equivalent of PyNumber_ToBase. Feel free to change the implementation details however you like as long as the basic functionality stays the same. To support non-float floating point types, replace the float check with isinstance(v, real). The rest of PyNumber_ToBase looks fine to me since it supports anything with an __index__ method. Raymond From leif.walsh at gmail.com Tue Jun 24 17:59:43 2008 From: leif.walsh at gmail.com (Leif Walsh) Date: Tue, 24 Jun 2008 08:59:43 -0700 (PDT) Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: <4860F3FE.8090307@gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> Message-ID: On Tue, 24 Jun 2008, Nick Coghlan wrote: > I'd prefer to see a proper PEP for this proposing a new slot that lets any > class return an (integer_part, fraction_part) tuple of integers, and > have PyNumber_ToBase take care of the actual string formatting. I take issue only with your notion of returning a 'fraction_part integer'. How do you propose we do this? I suggest we add a third value, exponent, to the suggested return value. -- Cheers, Leif From guido at python.org Tue Jun 24 18:42:29 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jun 2008 09:42:29 -0700 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> <4860F1DF.5080204@gmail.com> <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com> Message-ID: The same generic warning against sweeping changes applies here though. You have to manually review each change. The stdlib and especially the test suite is likely to break if you just let 2to3 run over it, even just a single fixer like fix_has_key. On Tue, Jun 24, 2008 at 6:11 AM, Benjamin Peterson wrote: > On Tue, Jun 24, 2008 at 8:08 AM, Nick Coghlan wrote: >> Benjamin Peterson wrote: >>> >>> On Tue, Jun 24, 2008 at 5:34 AM, Nick Coghlan wrote: >>>> >>>> I'd say so, yes. If tracker issues don't exist yet, please create them. >>>> The >>>> focus for the first beta was really on getting all the syntax and API >>>> changes stable - the remaining time between the first beta release and >>>> the >>>> first release candidate will focus on cleaning up issues like this. >>> >>> I wonder if we could just 2to3 (with fixers that create >>> incompatibilities disabled) over the whole stdlib and see what >>> happens. >> >> Just running the unit test suite with -3 would probably be a better starting >> point. 2to3 makes a lot of extra changes that we don't particularly care >> about. > > Right, which is why I suggest running a few fixers (ie fix_has_key) > and then tearing the test suite apart. > > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Jun 24 18:56:41 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jun 2008 09:56:41 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: <4860F3FE.8090307@gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> Message-ID: On Tue, Jun 24, 2008 at 6:17 AM, Nick Coghlan wrote: > Eric Smith wrote: >> >> Note that in >> http://mail.python.org/pipermail/python-dev/2008-February/077062.html, Guido >> advocates not adding the __bin__ machinery, which is what lead to the simple >> implementation of bin() just calling PyNumber_ToBase and relying on >> __index__. >> >> I don't think __bin__ should be added to 2.6. I don't see the point in >> adding a feature in 2.6 that's implemented so differently in 3.0. It's just >> asking for porting hassles. >> >> Instead, I think the approach used in 3.0 (r64451) should be used instead. >> That is, if this feature exist at all. I'm -0 on adding bin(), etc. to >> floats. I missed the decision, but it's been added, at least to 2.6. However, it was added after beta 1, so unless there was consensus about this, IMO it should be rolled back. > The 3.0 approach means that non-float floating point types still can't be > displayed properly by bin()/oct()/hex(). Nor can float, AFAICT from the current 3.0 tree. > The current 2.6 approach means > every such class has to implement its own equivalent of PyNumber_ToBase. But that's not supposed to be a burden -- any type purporting to implement some kind of integer should implement __index__ anyway. (Unless I misunderstand what you mean by "equivalent of"?) > Both are lousy solutions to a relative non-problem (IMO) that can easily be > implemented using a separate display function for those applications which > happen to need it. > > I'd prefer to see a proper PEP for this proposing a new slot that lets any > class return an (integer_part, fraction_part) tuple of integers, and have > PyNumber_ToBase take care of the actual string formatting. I'd prefer not to see new features added at all, except for things that were being planned before the beta and specifically got permission to be added post-beta-1. > Introducing such a gratuitous incompatibility between 2.6 and 3.0 at this > late stage of the process certainly seems highly undesirable to me. Same here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ggpolo at gmail.com Tue Jun 24 19:15:53 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Tue, 24 Jun 2008 14:15:53 -0300 Subject: [Python-Dev] setitimer, getitimer wrapper (issue2240) Message-ID: Hello, It has been some time since some discussion happened on http://bugs.python.org/issue2240, and I intended to compose this email several days ago but just did it now. This is an attempt to find someone who might have some information/idea that could help solving the current problem regarding the setitimer/getitimer wrapper under FreeBSD (and maybe somewhere else?). The current problem is that the virtual and prof itimers, in some occasions, may never finish, and as I remember from talking with Trent Nelson, the problem showed up when running all the tests but not when running the test_signal alone. So, that is it.. any solutions ? :) Regards, -- -- Guilherme H. Polo Goncalves From eric+python-dev at trueblade.com Tue Jun 24 19:07:16 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 24 Jun 2008 13:07:16 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> Message-ID: <486129C4.4010700@trueblade.com> Guido van Rossum wrote: >> The 3.0 approach means that non-float floating point types still can't be >> displayed properly by bin()/oct()/hex(). > > Nor can float, AFAICT from the current 3.0 tree. $ ./python Python 3.0b1+ (py3k:64491:64497M, Jun 24 2008, 07:14:03) [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> bin(1.0) '0b1 * 2.0 ** 0' >>> >> The current 2.6 approach means >> every such class has to implement its own equivalent of PyNumber_ToBase. > > But that's not supposed to be a burden -- any type purporting to > implement some kind of integer should implement __index__ anyway. > (Unless I misunderstand what you mean by "equivalent of"?) But apparently there's a desire to have bin(), oct(), and hex() support non-integer types. See above, and issue 3008. In 3.0, it's not possible to extend these to other types, because of the lack of __bin__, etc. The test for float is hard coded in PyNumber_ToBase(). Eric. From martin at v.loewis.de Tue Jun 24 20:21:46 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 24 Jun 2008 20:21:46 +0200 Subject: [Python-Dev] Python Windows Installers In-Reply-To: <4860D168.9050305@voidspace.org.uk> References: <4860D168.9050305@voidspace.org.uk> Message-ID: <48613B3A.9040805@v.loewis.de> > I'm just doing some housekeeping on a Windows install, and notice that > the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin v. > Lowis". Whilst I *personally* find this very reassuring I wonder if this > is intended / ideal? I certainly intended it: the field really is called the "author" file in MSI, and is (was) intended to indicate the "manufacturer" of the MSI, see http://msdn.microsoft.com/en-us/library/aa367808(VS.85).aspx That Vista calls it "publisher" is somewhat ambiguous. In any case, bug #1737210 complained about it, and I changed it to "Python Software Foundation". I can't retroactively change it for the releases you are looking at. Regards, Martin From fuzzyman at voidspace.org.uk Tue Jun 24 20:22:13 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 24 Jun 2008 19:22:13 +0100 Subject: [Python-Dev] Python Windows Installers In-Reply-To: <48613B3A.9040805@v.loewis.de> References: <4860D168.9050305@voidspace.org.uk> <48613B3A.9040805@v.loewis.de> Message-ID: <48613B55.4040505@voidspace.org.uk> Martin v. L?wis wrote: >> I'm just doing some housekeeping on a Windows install, and notice that >> the 'Publisher' of my Python 2.4 and 2.5 installs is shown as "Martin v. >> Lowis". Whilst I *personally* find this very reassuring I wonder if this >> is intended / ideal? >> > > I certainly intended it: the field really is called the "author" file in > MSI, and is (was) intended to indicate the "manufacturer" of the MSI, > see > > http://msdn.microsoft.com/en-us/library/aa367808(VS.85).aspx > > That Vista calls it "publisher" is somewhat ambiguous. > > In any case, bug #1737210 complained about it, and I changed it to > "Python Software Foundation". I can't retroactively change it for the > releases you are looking at. > No problem. I was more concerned about future releases. All the best, Michael Foord > Regards, > Martin > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/ http://www.trypython.org/ http://www.ironpython.info/ http://www.theotherdelia.co.uk/ http://www.resolverhacks.net/ From guido at python.org Tue Jun 24 20:23:57 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jun 2008 11:23:57 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: <486129C4.4010700@trueblade.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> <486129C4.4010700@trueblade.com> Message-ID: On Tue, Jun 24, 2008 at 10:07 AM, Eric Smith wrote: > Guido van Rossum wrote: >>> >>> The 3.0 approach means that non-float floating point types still can't be >>> displayed properly by bin()/oct()/hex(). >> >> Nor can float, AFAICT from the current 3.0 tree. > > $ ./python > Python 3.0b1+ (py3k:64491:64497M, Jun 24 2008, 07:14:03) > [GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> bin(1.0) > '0b1 * 2.0 ** 0' >>>> Oops. >>> The current 2.6 approach means >>> every such class has to implement its own equivalent of PyNumber_ToBase. >> >> But that's not supposed to be a burden -- any type purporting to >> implement some kind of integer should implement __index__ anyway. >> (Unless I misunderstand what you mean by "equivalent of"?) > > But apparently there's a desire to have bin(), oct(), and hex() support > non-integer types. See above, and issue 3008. In 3.0, it's not possible to > extend these to other types, because of the lack of __bin__, etc. The test > for float is hard coded in PyNumber_ToBase(). There are lots of desires. This particular one goes straight against an earlier decision to simplify bin(), oct() and hex() by removing the __oct__ and __hex__ special methods and not adding __bin__. Certainly not something to be added without a PEP, certainly not after beta1 was released already. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Jun 24 21:05:12 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jun 2008 12:05:12 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> <486129C4.4010700@trueblade.com> Message-ID: I'm curious why the addition of a new feature, past beta 1, was constrained entirely to a single tracker issue. Clearly not enough people were aware of it (or there wouldn't be the discussion about it here). Following the discussion in the issue tracker is really hard (since most of the discussion apparently refers to earlier versions). I also don't see any doc changes. If you want to keep the feature, at least write some documentation for it so everyone knows what is actually implemented, and why, without having to go to the source. --Guido On Tue, Jun 24, 2008 at 11:44 AM, Raymond Hettinger wrote: >>>>>> bin(1.0) >>> >>> '0b1 * 2.0 ** 0' >>>>>> >> >> Oops. > > This was checked-in for Issue 3008 which has been under discussion and > development for a while. It was intentionally held until after the beta was > released so that the beta itself would be as stable as possible (in the > final week and a half most of the bots were red and it made no sense to > commit a feature request at that time). > > The patch allows a eval-able representation of a float that works across > platforms and is not dependent of %.17g, atof() or ftoa() routines. The > tool makes it much easier to examine or explain what floating point ops > really do. It was invaluable in the development of math.sum(). I've spent > several days of development time on this and would be really bummed if it > got reversed-out at this point. > > The use of __bin__ is not essential to its implementation so that > implementation detail can be changed if needed. > > > Raymond > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Tue Jun 24 21:43:54 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 24 Jun 2008 20:43:54 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> <486129C4.4010700@trueblade.com> Message-ID: <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> On Tue, Jun 24, 2008 at 8:05 PM, Guido van Rossum wrote: > > Following the discussion in the issue tracker is really hard (since > most of the discussion apparently refers to earlier versions). I also > don't see any doc changes. I think there may also still be room for some additional discussion on the output format; while being eval-able is nice, one not-so-nice aspect of the representation is that nearby floats can have entirely distinct-looking representations; as in, for example: >>> hex(126.38) '0x3f30a3d70a3d7 * 2.0 ** -43' >>> hex(126.39) '0x1f98f5c28f5c29 * 2.0 ** -46' bin() suffers much less from this, but on balance I think I'd still prefer to see a non eval-able output that's normalized with respect to the most-significant bit instead of the least. I realize that (a) this has little to do with implementation details and use or not of __bin__, and (b) I should have said this in the issue tracker a few days ago. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From musiccomposition at gmail.com Tue Jun 24 21:45:49 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 24 Jun 2008 14:45:49 -0500 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> <4860F1DF.5080204@gmail.com> <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com> Message-ID: <1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com> On Tue, Jun 24, 2008 at 11:42 AM, Guido van Rossum wrote: > The same generic warning against sweeping changes applies here though. > You have to manually review each change. The stdlib and especially the > test suite is likely to break if you just let 2to3 run over it, even > just a single fixer like fix_has_key. So be it. We'll go over each module individually as with #3189. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From martin at v.loewis.de Tue Jun 24 21:51:39 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 24 Jun 2008 21:51:39 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> <486129C4.4010700@trueblade.com> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> Message-ID: <4861504B.209@v.loewis.de> > I think there may also still be room for some additional discussion > on the output format; If so, I think the change should be reverted, and the feature deferred to 2.7/3.1. Regards, Martin From guido at python.org Tue Jun 24 22:13:55 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jun 2008 13:13:55 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: <4861504B.209@v.loewis.de> References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> <486129C4.4010700@trueblade.com> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> Message-ID: [Mark Dickinson] >> I think there may also still be room for some additional discussion >> on the output format; [Martin v. L?wis] > If so, I think the change should be reverted, and the feature deferred > to 2.7/3.1. It looks like pretty much every aspect of this change was discussed / reviewed insufficiently. +1 on rolling back and taking the time to get it right. -1 on rushing it in. Since it's (potentially) a pretty small feature I might be convinced to accept it in beta2, but I don't want the fact that it was committed to force our hand. Raymond, if you care about this feature, please roll it back so we can have a proper discussion without pressure. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Jun 24 23:48:04 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 24 Jun 2008 17:48:04 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk: Include/object.h Lib/test/test_sys.py Misc/NEWS Objects/intobject.c Objects/longobject.c Objects/typeobject.c Python/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com> <486129C4.4010700@trueblade.com> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> Message-ID: <5E3B218C-F52E-410F-B5C8-AE64174BE416@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 24, 2008, at 4:13 PM, Guido van Rossum wrote: > [Mark Dickinson] >>> I think there may also still be room for some additional discussion >>> on the output format; > > [Martin v. L?wis] >> If so, I think the change should be reverted, and the feature >> deferred >> to 2.7/3.1. > > It looks like pretty much every aspect of this change was discussed / > reviewed insufficiently. +1 on rolling back and taking the time to get > it right. -1 on rushing it in. > > Since it's (potentially) a pretty small feature I might be convinced > to accept it in beta2, but I don't want the fact that it was committed > to force our hand. Raymond, if you care about this feature, please > roll it back so we can have a proper discussion without pressure. FWIW, I concur. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGFrlnEjvBPtnXfVAQITxwP/T16pGX+aTV5O/lhmwYWDtv8bA0S2l+Kl 6paSWTz8HHD0cpAhMAKKN7XqnJ6A0uIebCEJSPFdir8Jl0AMSbBZq88l+D93eoC0 eiqryogRTUzwrs2zC3A/nMjpNe21MErQNUEqmpU7XgdOpBRFQF8UMOnRIoSV8ggx SKwy+cRHnEk= =olIq -----END PGP SIGNATURE----- From ncoghlan at gmail.com Wed Jun 25 00:05:21 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Jun 2008 08:05:21 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> <4860F1DF.5080204@gmail.com> <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com> <1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com> Message-ID: <48616FA1.8060500@gmail.com> Benjamin Peterson wrote: > On Tue, Jun 24, 2008 at 11:42 AM, Guido van Rossum wrote: >> The same generic warning against sweeping changes applies here though. >> You have to manually review each change. The stdlib and especially the >> test suite is likely to break if you just let 2to3 run over it, even >> just a single fixer like fix_has_key. > > So be it. We'll go over each module individually as with #3189. We need to be especially careful with the unit test suite itself - changing the test code to avoid the warning will normally be the right answer, but when the code is actually setting out to test the deprecated feature we need to suppress the warning in the test suite instead. I should have a decent amount of Python time available later this week/weekend, so I should be able to help with this once I get the updated docs for special method lookup knocked over as well as checking in the somewhat related fix for the following 2.5 behaviour: >>> print unicode(Exception) >>> print unicode(u"\xa3") ? >>> print unicode(Exception(u"\xa3")) Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 0: ordinal not in range(128) (The idea being to get that last line printing the same thing as the second line, just as unicode(Exception("A")) produces the answer u"A", but without breaking the current behaviour of the first line) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From glyph at divmod.com Wed Jun 25 01:03:33 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Tue, 24 Jun 2008 23:03:33 -0000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <48616FA1.8060500@gmail.com> References: <2a70578d0806240318u1e0116f6m9d5dfe159f4fbda8@mail.gmail.com> <4860CDAF.2020806@gmail.com> <1afaf6160806240558l40d6569ck4e08f0357864a587@mail.gmail.com> <4860F1DF.5080204@gmail.com> <1afaf6160806240611g6ebb745an9af3b3bf1650308@mail.gmail.com> <1afaf6160806241245x3ed56863kc05030f1c6950ab8@mail.gmail.com> <48616FA1.8060500@gmail.com> Message-ID: <20080624230333.25821.838405170.divmod.xquotient.10140@joule.divmod.com> On 10:05 pm, ncoghlan at gmail.com wrote: >We need to be especially careful with the unit test suite itself - >changing the test code to avoid the warning will normally be the right >answer, but when the code is actually setting out to test the >deprecated feature we need to suppress the warning in the test suite >instead. This is a dangerous road to go down. If you suppress warnings in the test suite, you might suppress additional warnings which should actually be reported. Or, if the API gets modified in some way that the warning is supposed to be emitted but isn't any longer, it will be silent. It's easy to accidentally suppress too much or not enough. The way we've dealt with this in Twisted is adding an 'assertWarns' method so that we can invoke an API that is supposed to generate a warning, and (A) that warning and only that *specific* warning will not be emitted, and (B) if the API stops emitting the warning in the future, the test will fail and we will notice. It's also nice to have this facility in the test harness itself, so that you don't run the additional risk of accidentally (and silently) leaving warning suppression in place for subsequent tests. From exarkun at divmod.com Wed Jun 25 04:24:55 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 24 Jun 2008 22:24:55 -0400 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <20080624230333.25821.838405170.divmod.xquotient.10140@joule.divmod.com> Message-ID: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> On Tue, 24 Jun 2008 23:03:33 -0000, glyph at divmod.com wrote: >On 10:05 pm, ncoghlan at gmail.com wrote: >>We need to be especially careful with the unit test suite itself - changing >>the test code to avoid the warning will normally be the right answer, but >>when the code is actually setting out to test the deprecated feature we >>need to suppress the warning in the test suite instead. > >This is a dangerous road to go down. If you suppress warnings in the test >suite, you might suppress additional warnings which should actually be >reported. Or, if the API gets modified in some way that the warning is >supposed to be emitted but isn't any longer, it will be silent. It's easy >to accidentally suppress too much or not enough. > >The way we've dealt with this in Twisted is adding an 'assertWarns' method >so that we can invoke an API that is supposed to generate a warning, and (A) >that warning and only that *specific* warning will not be emitted, and (B) >if the API stops emitting the warning in the future, the test will fail and >we will notice. > >It's also nice to have this facility in the test harness itself, so that you >don't run the additional risk of accidentally (and silently) leaving warning >suppression in place for subsequent tests. It would be *extra* nice to have this facility added to the standard library, since assertWarns in Twisted is broken by changes in Python 2.6 (ie, our tests for warnings all fail with trunk at HEAD). For now, we will probably address this by switching to a different warning API. In the long term, it'd be better for us, other Python developers, and the standard library if there were an API in the standard library which facilitated testing of warnings. Jean-Paul From python at rcn.com Wed Jun 25 05:20:05 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 24 Jun 2008 20:20:05 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org><4860ED9F.8030305@trueblade.com> <4860F3FE.8090307@gmail.com><486129C4.4010700@trueblade.com><5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com><4861504B.209@v.loewis.de> Message-ID: Issue 3008 has been re-opened for more commentary. Raymond ----- Original Message ----- From: "Guido van Rossum" Since it's (potentially) a pretty small feature I might be convinced to accept it in beta2, but I don't want the fact that it was committed to force our hand. Raymond, if you care about this feature, please roll it back so we can have a proper discussion without pressure. From guido at python.org Wed Jun 25 06:05:56 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Jun 2008 21:05:56 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <486129C4.4010700@trueblade.com> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> Message-ID: I don't want the discussion in the issue. I want to have it right here. While I think it's fine to have some function that reveals the binary representation of floats, I don't think that overlaying this on hex/oct/bin is worth the problems it causes. This API appears to be purely for educational purposes; why not implement something in pure Python using the struct module that reveals the lay-out of the floating-point value? (There are also several things wrong with the specific patch, apart from its lack of docs; #1 is the introduction of an externaly visible symbol that doesn't start with _Py.) --Guido On Tue, Jun 24, 2008 at 8:20 PM, Raymond Hettinger wrote: > Issue 3008 has been re-opened for more commentary. > > Raymond > > ----- Original Message ----- From: "Guido van Rossum" > Since it's (potentially) a pretty small feature I might be convinced > to accept it in beta2, but I don't want the fact that it was committed > to force our hand. Raymond, if you care about this feature, please > roll it back so we can have a proper discussion without pressure. > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Wed Jun 25 12:24:06 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Jun 2008 20:24:06 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> Message-ID: <48621CC6.2030801@gmail.com> Jean-Paul Calderone wrote: > On Tue, 24 Jun 2008 23:03:33 -0000, glyph at divmod.com wrote: >> It's also nice to have this facility in the test harness itself, so >> that you don't run the additional risk of accidentally (and silently) >> leaving warning suppression in place for subsequent tests. > > It would be *extra* nice to have this facility added to the standard > library, since assertWarns in Twisted is broken by changes in Python > 2.6 (ie, our tests for warnings all fail with trunk at HEAD). For now, > we will probably address this by switching to a different warning > API. In the long term, it'd be better for us, other Python developers, > and the standard library if there were an API in the standard library > which facilitated testing of warnings. I forgot this had already been added to the Python regression test machinery, so it will just be a matter of updating the relevant tests to use it: http://docs.python.org/dev/library/test.html#module-test.test_support test.test_support.catch_warning(record=True)? Return a context manager that guards the warnings filter from being permanently changed and records the data of the last warning that has been issued. The record argument specifies whether any raised warnings are captured by the object returned by warnings.catch_warning() or allowed to propagate as normal. The context manager is typically used like this: with catch_warning() as w: warnings.warn("foo") assert str(w.message) == "foo" New in version 2.6. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From andrew-pythondev at puzzling.org Wed Jun 25 15:08:57 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Wed, 25 Jun 2008 23:08:57 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <48621CC6.2030801@gmail.com> References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> Message-ID: <20080625130857.GA13663@steerpike.home.puzzling.org> Nick Coghlan wrote: [...] > > I forgot this had already been added to the Python regression test > machinery, so it will just be a matter of updating the relevant tests to > use it: That's a nice surprise! I'm glad the standard library is growing facilities like this. I think it could be improved a little, though: > http://docs.python.org/dev/library/test.html#module-test.test_support > > test.test_support.catch_warning(record=True)? > > Return a context manager that guards the warnings filter from being > permanently changed and records the data of the last warning that has > been issued. The record argument specifies whether any raised warnings > are captured by the object returned by warnings.catch_warning() or > allowed to propagate as normal. The description doesn't really make the record=False case clear. This context manager is doing two different jobs: 1) restore the filters list and showwarning function to their original state when done, and 2) optionally (if record=True) record the last warning in the "as" target. That feels a bit weird. I think a clearer way to provide that functionality would be with two separate context managers: one that copies and finally restores the filters list and showwarnning function called protect_warnings_module, and then catch_warnings to record the warnings. The catch_warnings context manager could reuse the protect_warnings_module one. "with protect_warnings_module:" seems easier to understand (and document) than "with catch_warning(record=False)". Should I file a bug for this? > The context manager is typically used like this: > > with catch_warning() as w: > warnings.warn("foo") > assert str(w.message) == "foo" > Maybe this is a YAGNI, but what if an expression triggers multiple warnings? Perhaps the WarningMessage object ought to keep a list rather than just the last warning. Another thought: if the warnings module had a public, documented way to manipulate the filter list (e.g. warnings.get_all_filters() and warnings.set_all_filters(...)), then people could build these sorts of test facilities for themselves if the test_support one turns out to be too limiting or otherwise a poor fit. -Andrew. From brett at python.org Wed Jun 25 19:18:55 2008 From: brett at python.org (Brett Cannon) Date: Wed, 25 Jun 2008 10:18:55 -0700 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <20080625130857.GA13663@steerpike.home.puzzling.org> References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> <20080625130857.GA13663@steerpike.home.puzzling.org> Message-ID: On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts wrote: > Nick Coghlan wrote: > [...] >> >> I forgot this had already been added to the Python regression test >> machinery, so it will just be a matter of updating the relevant tests to >> use it: > > That's a nice surprise! I'm glad the standard library is growing facilities > like this. > > I think it could be improved a little, though: > >> http://docs.python.org/dev/library/test.html#module-test.test_support >> >> test.test_support.catch_warning(record=True)? >> >> Return a context manager that guards the warnings filter from being >> permanently changed and records the data of the last warning that has >> been issued. The record argument specifies whether any raised warnings >> are captured by the object returned by warnings.catch_warning() or >> allowed to propagate as normal. > > The description doesn't really make the record=False case clear. This context > manager is doing two different jobs: 1) restore the filters list and showwarning > function to their original state when done, and 2) optionally (if record=True) > record the last warning in the "as" target. That feels a bit weird. > The 'record=False' functionality was only added a couple of months ago. The context manager was originally written to help test the warnings module itself, so it was not meant to allow warnings to propagate. I tossed in the 'record' argument so that I could mutate the warnings filter. > I think a clearer way to provide that functionality would be with two separate > context managers: one that copies and finally restores the filters list and > showwarnning function called protect_warnings_module, and then catch_warnings to > record the warnings. The catch_warnings context manager could reuse the > protect_warnings_module one. "with protect_warnings_module:" seems easier to > understand (and document) than "with catch_warning(record=False)". > > Should I file a bug for this? > If you want, but Benjamin plans to undocument this for users along with all other test.support stuff (which I agree with). Most of the APIs in test.support were just quickly written and have not necessarily been thought through in order to make sure that the APIs makes sense (like in this case). -Brett From faassen at startifact.com Wed Jun 25 20:45:11 2008 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 25 Jun 2008 20:45:11 +0200 Subject: [Python-Dev] doctest, exec and __module__ Message-ID: Hi there, I've just witnessed an interesting consequence of the way doctest works. I ran into an issue when doctesting an aspect of SQLAlchemy, where the following guard clause tripped me up: # In the normal call flow, a request for any of the 3 basic collection # types is transformed into one of our trivial subclasses # (e.g. InstrumentedList). Catch anything else that sneaks in here... if cls.__module__ == '__builtin__': raise sa_exc.ArgumentError( "Can not instrument a built-in type. Use a " "subclass, even a trivial one.") My class, coming in as cls here, was defined in a doctest, like this: >>> class Foo(object): ... pass It turns out that doctest compiles and executes this bit of code using a line like this: # Don't blink! This is where the user's code gets run. exec compile(example.source, filename, "single", compileflags, 1) in test.globs This places new key/value pairs into a dictionary, in this case test.globs. Unfortunately when the execution results in a class definition, it'll have its __module__ attribute set to '__builtin__'. Try as I might, I couldn't convince exec to do it any differently. I can't think of an nice way to work around this problem either. The ugly workaround in the doctest itself works: >>> Foo.__module__ = 'whatever' That isn't very nice though. You could also iterate through all the values in the dictionary after each exec, and then check whether it's a class, and if so, set its __module__ to something else than __builtin__, but that doesn't feel very pretty (or efficient) either. Any ideas? Am I missing something? Is there really no way to control this behavior with exec? I'd like to somehow fix doctest.py so it doesn't set the __module__ to '__builtin__' for everything. '__main__' would be nicer, as that's what the interpreter shell does, and a doctest example already looks like the interpreter shell. While the above SQLAlchemy code is hardly pretty, I can't think of any better way to put in a safeguard like that either. Regards, Martijn From fdrake at acm.org Wed Jun 25 21:37:28 2008 From: fdrake at acm.org (Fred Drake) Date: Wed, 25 Jun 2008 15:37:28 -0400 Subject: [Python-Dev] doctest, exec and __module__ In-Reply-To: References: Message-ID: On Jun 25, 2008, at 2:45 PM, Martijn Faassen wrote: > This places new key/value pairs into a dictionary, in this case > test.globs. Unfortunately when the execution results in a class > definition, it'll have its __module__ attribute set to > '__builtin__'. Try as I might, I couldn't convince exec to do it any > differently. The zope.testing package has a way to work around this from setup/ teardown functions passed to the DocFileSuite (or whatever wrapper you're using). See the module zope.testing.module for the functions. Essentially, the __name__ needs to be set in test.globs; class construction uses the global value for __name__ to provide __module__; doctests normally don't have one, so it acquires the value from __builtin__.__name__ instead. Which is, of course, horribly wrong. The solution in zope.testing.module allows the class to be used with pickle within the test as well. -Fred -- Fred Drake From hall.jeff at gmail.com Wed Jun 25 21:56:11 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Wed, 25 Jun 2008 15:56:11 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> Message-ID: <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> It seems to me that the root problem is allocation spikes of legitimate, useful data. Perhaps then we need some sort of "test" to determine if those are legitimate. Perhaps checking every nth (with n decreasing as allocation bytes increases) object allocated during a "spike" could be useful. Then delay garbage collection until x consecutive objects are found to be garbage? It seems like we should be attacking the root cause rather than finding some convoluted math that attempts to work for all scenarios. On a side note, the information about not GCing on string objects is interesting? Is there a way to override this behavior? I've found that re.py chokes on large text files (4MB plus) without line ends (don't ask, they're not our files but we have to parse them). I wonder if this isn't the reason... If the answer to that is, "no, strings are always ignored" I'd recommend rethinking this (or providing an option to override somehow. -- Haikus are easy Most make very little sense Refrigerator -------------- next part -------------- An HTML attachment was scrubbed... URL: From musiccomposition at gmail.com Wed Jun 25 22:00:17 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 25 Jun 2008 15:00:17 -0500 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) Message-ID: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> On Wed, Jun 25, 2008 at 12:18 PM, Brett Cannon wrote: > If you want, but Benjamin plans to undocument this for users along > with all other test.support stuff (which I agree with). Most of the > APIs in test.support were just quickly written and have not > necessarily been thought through in order to make sure that the APIs > makes sense (like in this case). Thanks for bringing that up, Brett. Is making test_support non-public a API in 3.0 fine with everybody? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From facundobatista at gmail.com Wed Jun 25 22:16:43 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 25 Jun 2008 17:16:43 -0300 Subject: [Python-Dev] Last "bug weekend" Message-ID: Hi all! In Python Argentina we get together in two places, Buenos Aires and Santa Fe, and worked out around 10-12 bugs, which is around the half of the total for both days, so I'm very happy, :) We even had a Python logo shaped cake, see, see [1]. Did you participate? What do you think it's the outcome for both days? Regards, [1] http://www.taniquetil.com.ar/facundo/imgs/tortalogopy.jpeg -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From faassen at startifact.com Wed Jun 25 22:26:27 2008 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 25 Jun 2008 22:26:27 +0200 Subject: [Python-Dev] doctest, exec and __module__ In-Reply-To: References: Message-ID: Fred Drake wrote: > On Jun 25, 2008, at 2:45 PM, Martijn Faassen wrote: >> This places new key/value pairs into a dictionary, in this case >> test.globs. Unfortunately when the execution results in a class >> definition, it'll have its __module__ attribute set to '__builtin__'. >> Try as I might, I couldn't convince exec to do it any differently. > > The zope.testing package has a way to work around this from > setup/teardown functions passed to the DocFileSuite (or whatever wrapper > you're using). See the module zope.testing.module for the functions. Thanks, I should've checked that! > Essentially, the __name__ needs to be set in test.globs; class > construction uses the global value for __name__ to provide __module__; > doctests normally don't have one, so it acquires the value from > __builtin__.__name__ instead. Which is, of course, horribly wrong. Okay, so is this mystery mechanism that I was missing. Sorry to bother people here on the list. Regards, Martijn From guido at python.org Wed Jun 25 22:49:24 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Jun 2008 13:49:24 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> Message-ID: [Adding back the list.] On Tue, Jun 24, 2008 at 9:53 PM, Raymond Hettinger wrote: >> While I think it's fine to have some function that reveals the binary >> representation of floats, I don't think that overlaying this on >> hex/oct/bin is worth the problems it causes. > > What problems? The patch is clean. Problems like no two people on python-dev agreeing on how exactly the feature should be implemented. Problems like whether this goes way beyond the philosophical underpinnings of bin/oct/hex. Problems like what to do about other types that might want to overload hex/oct/bin. See Kevin Jacobs' response. >> This API appears to be >> purely for educational purposes; why not implement something in pure >> Python using the struct module that reveals the lay-out of the >> floating-point value? > > This is not the internal floating point layout. It is the real value > expressed in exponential form. It is more than educational -- it is a > platform independent representation (look at Terry's reference -- it is the > usual way to precisely specify a float value and it does not depend on > atof() or vice versa). Possibly, but it is only readable by a Python expression parser. For all practical purposes "%.17g" % x works just as well. And this bypasses the question "why overload this functionality on bin/hex/oct rather than adding e.g. a new function to math or a new method to float." >> (There are also several things wrong with the specific patch, apart >> from its lack of docs; #1 is the introduction of an externaly visible >> symbol that doesn't start with _Py.) > > Will change the global symbol to _Py. I already added docs to the patch. > Did you see the one that was uploaded a few hours ago (float6.diff)? I don't care about the details of the patch until we have agreement about which form the feature should take. We don't have that agreement yet. I mentioned the flaws in the patch to point out that it was apparently a rush job. > I re-opened the discussion at your behest. [...] I'm very glad you're giving the discussion a second chance. Please give it a few days at least. My expectation is that the outcome will be not to overload bin/hex/oct but to add a custom function to math or a custom method to float, whose output can be further massaged to create the platform-independent representation you're after. (I doubt that it's worth changing pickle or marshal though, they are doing fine with their respective current approaches.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Jun 25 22:55:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 25 Jun 2008 22:55:32 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> References: <2e1434c10806210833t3dd3e121v98b29e74f24698cc@mail.gmail.com> <485D231D.9030806@v.loewis.de> <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> Message-ID: <4862B0C4.2080901@v.loewis.de> > It seems to me that the root problem is allocation spikes of legitimate, > useful data. Perhaps then we need some sort of "test" to determine if > those are legitimate. Perhaps checking every nth (with n decreasing as > allocation bytes increases) object allocated during a "spike" could be > useful. Then delay garbage collection until x consecutive objects are > found to be garbage? > > It seems like we should be attacking the root cause rather than finding > some convoluted math that attempts to work for all scenarios. I think exactly the other way 'round. The timing of thing should not matter at all, only the exact sequence of allocations and deallocations. I trust provable maths much more than I trust ad-hoc heuristics, even if you think the math is convoluted. > On a side note, the information about not GCing on string objects is > interesting? Is there a way to override this behavior? I think you misunderstand. Python releases unused string objects just fine, and automatically. It doesn't even need GC for that. Regards, Martin From guido at python.org Wed Jun 25 23:00:44 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Jun 2008 14:00:44 -0700 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> Message-ID: I'm a little worried about making stuff undocumented that every core developer needs to use -- everyone writing tests needs to continue to use test_support (now test.support?). I imagine people writing unit test suites for 3rd party libraries might want to use its services too. In general I'm not a big fan of having undocumented APIs; 9 out of 10 times someone finds a genuine use case for such an API, and then you're worse off than if it was documented to begin with, since if people start using undocumented APIs, they necessarily reverse-engineer how it works, and then you can never change it. If it was documented, at least you may be able to get away with modifying the undocumented parts, assuming people actually read the docs. (Though we've had cases where the docs and implementation were inconsistent for years, and eventually we ended up fixing the docs...) --Guido On Wed, Jun 25, 2008 at 1:00 PM, Benjamin Peterson wrote: > On Wed, Jun 25, 2008 at 12:18 PM, Brett Cannon wrote: >> If you want, but Benjamin plans to undocument this for users along >> with all other test.support stuff (which I agree with). Most of the >> APIs in test.support were just quickly written and have not >> necessarily been thought through in order to make sure that the APIs >> makes sense (like in this case). > > Thanks for bringing that up, Brett. Is making test_support non-public > a API in 3.0 fine with everybody? > > > -- > Cheers, > Benjamin Peterson > "There's no place like 127.0.0.1." > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From hall.jeff at gmail.com Wed Jun 25 23:08:58 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Wed, 25 Jun 2008 17:08:58 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <4862B0C4.2080901@v.loewis.de> References: <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> <4862B0C4.2080901@v.loewis.de> Message-ID: <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com> On Wed, Jun 25, 2008 at 4:55 PM, "Martin v. L?wis" wrote: > > It seems to me that the root problem is allocation spikes of legitimate, > > useful data. Perhaps then we need some sort of "test" to determine if > > those are legitimate. Perhaps checking every nth (with n decreasing as > > allocation bytes increases) object allocated during a "spike" could be > > useful. Then delay garbage collection until x consecutive objects are > > found to be garbage? > > > > It seems like we should be attacking the root cause rather than finding > > some convoluted math that attempts to work for all scenarios. > > I think exactly the other way 'round. The timing of thing should not > matter at all, only the exact sequence of allocations and deallocations. > > I trust provable maths much more than I trust ad-hoc heuristics, even > if you think the math is convoluted. > I probably chose my wording poorly (particularly for a newcomer/outsider). What I meant was that the numbers used in GC currently appear arbitrary. The idea of three "groups" (youngest, oldest and middle) is also arbitrary. Would it not be better to tear that system apart and create a "sliding" scale. If the timing method is undesirable then make it slide based on the allocation/deallocation difference. In this way, the current breakpoints and number of groups (all of which are arbitrary and fixed) could be replaced by one coefficient (and yes, I recognize that it would also be arbitrary but it would be one, tweakable number rather than several). My gut tells me that your current fix is going to work just fine for now but we're going to end up tweaking it (or at least discussing tweaking it) every 6-12 months. > > On a side note, the information about not GCing on string objects is > > interesting? Is there a way to override this behavior? > > I think you misunderstand. Python releases unused string objects just > fine, and automatically. It doesn't even need GC for that. > I took the statement, "Current GC only takes into account container objects, which, most significantly, ignores string objects (of which most applications create plenty)" to mean that strings were ignored for deciding when to do garbage collection. I mistakenly thought that was because they were assumed to be small. It sounds like they're ignored because they're automatically collected and so they SHOULD be ignored for object garbage collection. Thanks for the clarification... Back to the drawing board on my other problem ;) -- Haikus are easy Most make very little sense Refrigerator -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Thu Jun 26 00:27:33 2008 From: brett at python.org (Brett Cannon) Date: Wed, 25 Jun 2008 15:27:33 -0700 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> Message-ID: On Wed, Jun 25, 2008 at 2:00 PM, Guido van Rossum wrote: > I'm a little worried about making stuff undocumented that every core > developer needs to use -- everyone writing tests needs to continue to > use test_support Right, but I would think all core developers know about test.support and are capable of reading the code and docstring. > (now test.support?). In 3.0, yes. > I imagine people writing unit > test suites for 3rd party libraries might want to use its services > too. > > In general I'm not a big fan of having undocumented APIs; 9 out of 10 > times someone finds a genuine use case for such an API, and then > you're worse off than if it was documented to begin with, since if > people start using undocumented APIs, they necessarily > reverse-engineer how it works, and then you can never change it. If it > was documented, at least you may be able to get away with modifying > the undocumented parts, assuming people actually read the docs. > (Though we've had cases where the docs and implementation were > inconsistent for years, and eventually we ended up fixing the docs...) > My worry is that the module has had stuff tossed in ad-hoc in such a way that maintaining the code for public consumption is a pain. If we took the time to clean up the APIs then I would be fine with documenting the module. -Brett From martin at v.loewis.de Thu Jun 26 00:39:37 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 00:39:37 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com> References: <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> <4862B0C4.2080901@v.loewis.de> <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com> Message-ID: <4862C929.6070909@v.loewis.de> > I took the statement, "Current GC only takes into account container > objects, which, most significantly, ignores string objects (of which > most applications create plenty)" to mean that strings were ignored for > deciding when to do garbage collection. I mistakenly thought that was > because they were assumed to be small. It sounds like they're ignored > because they're automatically collected and so they SHOULD be ignored > for object garbage collection. More precisely, a string object can never participate in a cycle (it can be referenced from a cycle, but not be *in* the cycle, as it has no references to other objects). GC in Python is only about container objects (which potentially can be cyclic); non-container objects are released when the refcount says they are no longer referenced. Whether or not allocation of definitely-non-cyclic objects should still trigger cyclic GC (in the hope that some objects hang on a garbage cycle) is a question that is open to debate; I'd prefer an analysis of existing applications before making decisions. Regards, Martin From musiccomposition at gmail.com Thu Jun 26 00:42:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 25 Jun 2008 17:42:37 -0500 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> Message-ID: <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> On Wed, Jun 25, 2008 at 4:00 PM, Guido van Rossum wrote: > I'm a little worried about making stuff undocumented that every core > developer needs to use -- everyone writing tests needs to continue to > use test_support (now test.support?). I imagine people writing unit > test suites for 3rd party libraries might want to use its services > too. I think undocumented is a little unspecific here. What I mean is "reserved for core Python tests and no promise is made to retain compatibility." Of course, we would keep docs for them (perhaps in Lib/test/README), so new core developers could write their tests. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From alexandre at peadrop.com Thu Jun 26 01:35:15 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Wed, 25 Jun 2008 19:35:15 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <4862B0C4.2080901@v.loewis.de> References: <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> <4862B0C4.2080901@v.loewis.de> Message-ID: On Wed, Jun 25, 2008 at 4:55 PM, "Martin v. L?wis" wrote: > I think exactly the other way 'round. The timing of thing should not > matter at all, only the exact sequence of allocations and deallocations. I would it be possible, if not a good idea, to only track object deallocations as the GC traversal trigger? As far as I know, dangling cyclic references cannot be formed when allocating objects. So, this could potentially mitigate the quadratic behavior during allocation bursts. -- Alexandre From jmp at MIT.EDU Wed Jun 25 23:13:49 2008 From: jmp at MIT.EDU (Justin Mazzola Paluska) Date: Wed, 25 Jun 2008 17:13:49 -0400 Subject: [Python-Dev] Disable tests in unittest (issue3202) Message-ID: <20080625211349.GA471@mit.edu> Hi, I just reported issue3202 in the bugracker (http://bugs.python.org/issue3202) with a patch that implements a way to disable unittest.TestCases using a decorator. Benjamin Peterson suggested that I bring up the issue on python-dev. The idea behind the patch is that it?s sometimes useful to disable tests without removing them from the TestCase. For example, a co-worker and I have a module with a few tests that will fail for the forseeable future because we haven?t had a chance to implement the features the tests are testing. The tracebacks for these ?not implemented yet? tests obscure real failures in other tests. Normally, we?d just rename the failing methods from something starting with ?test? to something like ?_test?, so unittest ignores them. However, doing so removes all traces of the tests when you re-run the test suite, so the disabled tests are easy to forget. Instead, issue3202 allows you to write something like: from unittest import main, TestCase, disabled class SampleTest(TestCase): def test_addition(self): self.assertEqual(2, 1+1) def test_broken(self): # this is a real failure in our code self.assertEqual(5, 2+2) @disabled def test_not_implemented(self): # Test of a feature not implemented yet. doit() if __name__ == '__main__': main() which has one test disabled. Running the test suite outputs %python sample_tests2.py .FD ====================================================================== FAIL: test_broken (__main__.SampleTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "sample_tests2.py", line 12, in test_broken self.assertEqual(5, 2+2) AssertionError: 5 != 4 ---------------------------------------------------------------------- Ran 3 tests in 0.001s FAILED (failures=1) showing only the real failure and a simple ?D? for the disabled test. The disabled test is still there, but doesn?t get in your way. JUnit4 has a similar decorator for its tests, @Ignore. The patch in issue3202 implements the decorator @disabled and a few modifications to the classes in the unittest module. What does the Python hivemind think? I?m also open to enhancing it if the list has any ideas. ?Justin From jml at mumak.net Thu Jun 26 02:51:49 2008 From: jml at mumak.net (Jonathan Lange) Date: Thu, 26 Jun 2008 10:51:49 +1000 Subject: [Python-Dev] Disable tests in unittest (issue3202) In-Reply-To: <20080625211349.GA471@mit.edu> References: <20080625211349.GA471@mit.edu> Message-ID: On Thu, Jun 26, 2008 at 7:13 AM, Justin Mazzola Paluska wrote: > Hi, > ... > The idea behind the patch is that it's sometimes useful to disable > tests without removing them from the TestCase. For example, a > co-worker and I have a module with a few tests that will fail for the > forseeable future because we haven't had a chance to implement the > features the tests are testing. The tracebacks for these "not > implemented yet" tests obscure real failures in other tests. ... > The patch in issue3202 implements the decorator @disabled and a few > modifications to the classes in the unittest module. > > What does the Python hivemind think? I'm also open to enhancing it if > the list has any ideas. I think it's really worth looking at the approach that bzrlib's tests take here (see bzrlib.tests.ExtendedTestResult and the out-of-date-but-helpful http://bazaar-vcs.org/BzrExtendTestSuite) Instead of disabling the tests, their practice is to run the tests but have them raise KnownFailure. When they write a test that they know will fail, they have it raise this exception. The extended TestResult object catches this exception in addFailure and reports it appropriately.[1] So, when there's a test that is expected to fail, they do something like this: def test_foo(self): x = get_some_value() self.expectFailure('get_some_value is bogus', self.assertEqual, x, 42) Using KnownFailure is better than disabling because you'll be able to tell if the test suddenly and mysteriously passes. I can see that disabling still has some value, but I'd rather have KnownFailure first. jml [1] They use a similar strategy for skipping tests based on platform. (You can't test Bazaar's symlink support on Windows, for example). From facundobatista at gmail.com Thu Jun 26 02:52:11 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 25 Jun 2008 21:52:11 -0300 Subject: [Python-Dev] Disable tests in unittest (issue3202) In-Reply-To: <20080625211349.GA471@mit.edu> References: <20080625211349.GA471@mit.edu> Message-ID: 2008/6/25 Justin Mazzola Paluska : > The idea behind the patch is that it's sometimes useful to disable > tests without removing them from the TestCase. For example, a > co-worker and I have a module with a few tests that will fail for the > forseeable future because we haven't had a chance to implement the > features the tests are testing. The tracebacks for these "not > implemented yet" tests obscure real failures in other tests. What about TestSkipped? I thought that raising test_support.TestSkipped should behave like this: you're saying that you're actually NOT executing the tests, but you say that they are there. But a few days ago I needed to comment out a test that previously was raising this, because it made a buildbot to go red. How this should behave? Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From musiccomposition at gmail.com Thu Jun 26 02:55:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Wed, 25 Jun 2008 19:55:37 -0500 Subject: [Python-Dev] Disable tests in unittest (issue3202) In-Reply-To: References: <20080625211349.GA471@mit.edu> Message-ID: <1afaf6160806251755y1979651l29b6d7ca4d0d9e@mail.gmail.com> On Wed, Jun 25, 2008 at 7:52 PM, Facundo Batista wrote: > > What about TestSkipped? I thought that raising > test_support.TestSkipped should behave like this: you're saying that > you're actually NOT executing the tests, but you say that they are > there. I think he's talking about outside core Python development (putting it into unittest). -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From martin at v.loewis.de Thu Jun 26 06:01:14 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 26 Jun 2008 06:01:14 +0200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: References: <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> <4862B0C4.2080901@v.loewis.de> Message-ID: <4863148A.3050304@v.loewis.de> > I would it be possible, if not a good idea, to only track object > deallocations as the GC traversal trigger? As far as I know, dangling > cyclic references cannot be formed when allocating objects. Not sure what you mean by that. x = [] x.append(x) del x creates a cycle with no deallocation occurring. Regards, Martin From alexandre at peadrop.com Thu Jun 26 07:31:53 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Thu, 26 Jun 2008 01:31:53 -0400 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <4863148A.3050304@v.loewis.de> References: <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> <4862B0C4.2080901@v.loewis.de> <4863148A.3050304@v.loewis.de> Message-ID: On Thu, Jun 26, 2008 at 12:01 AM, "Martin v. L?wis" wrote: >> I would it be possible, if not a good idea, to only track object >> deallocations as the GC traversal trigger? As far as I know, dangling >> cyclic references cannot be formed when allocating objects. > > Not sure what you mean by that. > > x = [] > x.append(x) > del x > > creates a cycle with no deallocation occurring. > Oh... never mind then. -- Alexandre From andrew-pythondev at puzzling.org Thu Jun 26 07:49:16 2008 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Thu, 26 Jun 2008 15:49:16 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> <20080625130857.GA13663@steerpike.home.puzzling.org> Message-ID: <20080626054916.GA24366@steerpike.home.puzzling.org> Brett Cannon wrote: > On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts [...] > > > > Should I file a bug for this? > > > > If you want, but Benjamin plans to undocument this for users along > with all other test.support stuff (which I agree with). Most of the > APIs in test.support were just quickly written and have not > necessarily been thought through in order to make sure that the APIs > makes sense (like in this case). Ok, then we're back to there being no supported way to write tests that need to intercept warnings. Twisted has already suffered from this (JP reports that Twisted's assertWarns is broken in 2.6), and I doubt it's alone. So I guess I am filing a bug after all... :) -Andrew. From hodgestar+pythondev at gmail.com Thu Jun 26 10:58:03 2008 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Thu, 26 Jun 2008 10:58:03 +0200 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> Message-ID: On Thu, Jun 26, 2008 at 12:27 AM, Brett Cannon wrote: > Right, but I would think all core developers know about test.support > and are capable of reading the code and docstring. Docstrings don't help for things other than Modules, Classes and Functions. So, for example, are constants like FUZZ intended to be part of the external API provided by the module or not? Also, there are already a fair number of functions in test_support that don't have docstrings. Are these intended to be part of the API? Obviously one can grep around the existing tests to see what they do, but then one ends up perpetuating any errors others have made. While one could possibly attempt to make a case for "all core developers know" [1] it certainly isn't true that "all future core developers know" [2] or that "all people who could put patches into the issue tracker know" [3]. [1] Although personally I would bet against it until proven -- especially given a sensible definition of "core developer" to mean, in this case, anyone with the right to commit tests to the repository. [2] Since many of these future developers probably don't even use Python yet. [3] And we do want patches to issues to include tests, right? From python at rcn.com Thu Jun 26 13:24:43 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 26 Jun 2008 04:24:43 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> Message-ID: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> From: "Guido van Rossum" > I don't care about the details of the patch until we have agreement > about which form the feature should take. We don't have that agreement > yet. Updated to the patch to address everyone's review comments: http://bugs.python.org/file10742/float8.diff * Alexander Belopolsky requested exponential notation instead of potentially very long strings of bits. Done * Alexander Belopolsky requested true mathematical radix 2 representation of a float rather than its 64-bit memory layout. Done * Antoine Pitrou requested that hex() and oct() be supported as well as bin(). Terry J. Reedy also requested support for hex(). Done. * Alexander Belopolsky and Alexandre Vassalotti requested that the output be a two-way street -- something that can be round-tripped through eval(). Done. * Amaury Forgeot d'Arc requested that the implementation not manipulate C strings inplace. Fixed -- used PyUnicode_FromFormat() instead. * Amaury Forgeot d'Arc requested that tests should check if negative numbers have the same representation as their absolute value. Done. * Mark Dickinson requested sign preserving output for bin(-0.0). We couldn't find a clean way to do this without a special cased output format. * Mark Dickinson reviewed the NaN/Inf handling. Done. * Eric Smith requested that the routine be attached to _PyFloat_to_base() instead of attaching to __bin__, __oct__, and __hex__. Done. * Guido requested that the docs be updated. Done. * Guido requested that the globally visible C API function name be prefixed with _Py. Done. * Mark Dickinson requested normalizing output to start with a 1 so that nearby values have similar reprs. Done. Raymond From p.f.moore at gmail.com Thu Jun 26 14:16:33 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 26 Jun 2008 13:16:33 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> Message-ID: <79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com> On 26/06/2008, Raymond Hettinger wrote: > From: "Guido van Rossum" > > I don't care about the details of the patch until we have agreement > > about which form the feature should take. We don't have that agreement > > yet. > > > > Updated to the patch to address everyone's review comments: > http://bugs.python.org/file10742/float8.diff Just as a contrary point, I'm not particularly keen on the output format (which takes the form '0b1 * 2.0 ** 0' as far as I can see), and I'm definitely not keen on the fact that it's overloaded on the hex/bin/oct builtins. Can't it be a separate function? If it is, I don't much care about the output format (as I have no particular need for the feature) but would it not be better if it were machine-parseable, rather than executable? Paul From eric+python-dev at trueblade.com Thu Jun 26 14:23:49 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 26 Jun 2008 08:23:49 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> Message-ID: <48638A55.1030104@trueblade.com> Actually, after saying I was opposed to __bin__ in 2.6, I said: "Instead, I think the approach used in 3.0 (r64451) should be used instead. That is, if this feature exist at all. I'm -0 on adding bin(), etc. to floats." My last sentence is a little unclear. I meant I'm -0 on adding floats as arguments to bin(), oct(), and hex(). Primarily because a) it's not extensible in 3.0, and b) I find it surprising, in that I'd expect those functions to throw an error for non-integral types (that is, those not having __index__). I think adding a "float_as_binary_expression()" (with a better name) in some module would get the functionality you seek. What is gained by adding this to bin() and friends? Raymond Hettinger wrote: > From: "Guido van Rossum" >> I don't care about the details of the patch until we have agreement >> about which form the feature should take. We don't have that agreement >> yet. > > Updated to the patch to address everyone's review comments: > http://bugs.python.org/file10742/float8.diff > > * Alexander Belopolsky requested exponential notation instead of > potentially very long strings of bits. Done > > * Alexander Belopolsky requested true mathematical radix 2 > representation of a float rather than its 64-bit memory layout. Done > > * Antoine Pitrou requested that hex() and oct() be supported as well as > bin(). Terry J. Reedy also requested support for hex(). Done. > > * Alexander Belopolsky and Alexandre Vassalotti requested that the > output be a two-way street -- something that can be round-tripped > through eval(). Done. > > * Amaury Forgeot d'Arc requested that the implementation not manipulate > C strings inplace. Fixed -- used PyUnicode_FromFormat() instead. > > * Amaury Forgeot d'Arc requested that tests should check if negative > numbers have the same representation as their absolute value. Done. > > * Mark Dickinson requested sign preserving output for bin(-0.0). We > couldn't find a clean way to do this without a special cased output format. > > * Mark Dickinson reviewed the NaN/Inf handling. Done. > > * Eric Smith requested that the routine be attached to > _PyFloat_to_base() instead of attaching to __bin__, __oct__, and > __hex__. Done. > > * Guido requested that the docs be updated. Done. > > * Guido requested that the globally visible C API function name be > prefixed with _Py. Done. > > * Mark Dickinson requested normalizing output to start with a 1 so that > nearby values have similar reprs. Done. > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/eric%2Bpython-dev%40trueblade.com > > From musiccomposition at gmail.com Thu Jun 26 14:45:44 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 26 Jun 2008 07:45:44 -0500 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> Message-ID: <1afaf6160806260545o60fe1f0cl996ad482c352d0b@mail.gmail.com> On Thu, Jun 26, 2008 at 3:58 AM, Simon Cross wrote: > > [1] Although personally I would bet against it until proven -- > especially given a sensible definition of "core developer" to mean, in > this case, anyone with the right to commit tests to the repository. > [2] Since many of these future developers probably don't even use Python yet. > [3] And we do want patches to issues to include tests, right? See my above message. I don't think we need to worry about the docs part of "undocumented." All I mean by undocumented is that the API is not guaranteed throughout versions, and test.support really should only be used for core tests. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From solipsis at pitrou.net Thu Jun 26 14:49:44 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 26 Jun 2008 12:49:44 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?=5BPython-checkins=5D_r64424_-_in=09python?= =?utf-8?q?/trunk=3AInclude/object=2EhLib/test/test=5Fsys=2Epy=09Mi?= =?utf-8?q?sc/NEWSObjects/intobject=2EcObjects/longobject=2Ec=09Obj?= =?utf-8?q?ects/typeobject=2EcPython/bltinmodule=2Ec?= References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> Message-ID: Raymond Hettinger rcn.com> writes: > > * Antoine Pitrou requested that hex() and oct() be supported as well as bin(). Just to qualify this, I said that if bin() were to gain float support, the same should probably be done for hex() and oct(). That doesn't mean I'm in favor of bin() support for floats. Regards Antoine. From guido at python.org Thu Jun 26 15:52:32 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 06:52:32 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> Message-ID: Would you mind reading the rest of *this* thread on python-dev and respond to the discussion about the design of the feature? On Thu, Jun 26, 2008 at 4:24 AM, Raymond Hettinger wrote: > From: "Guido van Rossum" >> >> I don't care about the details of the patch until we have agreement >> about which form the feature should take. We don't have that agreement >> yet. > > Updated to the patch to address everyone's review comments: > http://bugs.python.org/file10742/float8.diff > > * Alexander Belopolsky requested exponential notation instead of potentially > very long strings of bits. Done > > * Alexander Belopolsky requested true mathematical radix 2 representation > of a float rather than its 64-bit memory layout. Done > > * Antoine Pitrou requested that hex() and oct() be supported as well as > bin(). Terry J. Reedy also requested support for hex(). Done. > > * Alexander Belopolsky and Alexandre Vassalotti requested that the output be > a two-way street -- something that can be round-tripped through eval(). > Done. > > * Amaury Forgeot d'Arc requested that the implementation not manipulate C > strings inplace. Fixed -- used PyUnicode_FromFormat() instead. > > * Amaury Forgeot d'Arc requested that tests should check if negative numbers > have the same representation as their absolute value. Done. > > * Mark Dickinson requested sign preserving output for bin(-0.0). We > couldn't find a clean way to do this without a special cased output format. > > * Mark Dickinson reviewed the NaN/Inf handling. Done. > > * Eric Smith requested that the routine be attached to _PyFloat_to_base() > instead of attaching to __bin__, __oct__, and __hex__. Done. > > * Guido requested that the docs be updated. Done. > > * Guido requested that the globally visible C API function name be prefixed > with _Py. Done. > > * Mark Dickinson requested normalizing output to start with a 1 so that > nearby values have similar reprs. Done. > > > Raymond > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Jun 26 14:50:31 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 26 Jun 2008 05:50:31 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com> Message-ID: > Just as a contrary point, I'm not particularly keen on the output > format (which takes the form '0b1 * 2.0 ** 0' as far as I can see), That format was requested by everyone else on the tracker discussion. What I originally wanted was something like 0b11.0101. But that didn't round-trip through eval, it didn't match the style used in the numerical papers referenced by Terry Reedy, and it didn't scale well with inputs like 1.1E+100. > and I'm definitely not keen on the fact that it's overloaded on the > hex/bin/oct builtins. > > Can't it be a separate function? Simplicity. bin/oct/hex have the job of giving alternate base representations for numbers. Nothing is gained by adding a duplicate set of functions in the math module for float inputs. > would > it not be better if it were machine-parseable, rather than executable? We already have struct.pack for machine-parseable output. This is supposed to be human readable as well as providing an exact, platform indepent way of specifying a particular float value (something that's useful in testing algorithms like that in math.sum()). Raymond From ncoghlan at gmail.com Thu Jun 26 15:56:23 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Jun 2008 23:56:23 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <20080626054916.GA24366@steerpike.home.puzzling.org> References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> <20080625130857.GA13663@steerpike.home.puzzling.org> <20080626054916.GA24366@steerpike.home.puzzling.org> Message-ID: <4863A007.60804@gmail.com> Andrew Bennetts wrote: > Brett Cannon wrote: >> On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts > [...] >>> Should I file a bug for this? >>> >> If you want, but Benjamin plans to undocument this for users along >> with all other test.support stuff (which I agree with). Most of the >> APIs in test.support were just quickly written and have not >> necessarily been thought through in order to make sure that the APIs >> makes sense (like in this case). > > Ok, then we're back to there being no supported way to write tests that need to > intercept warnings. Twisted has already suffered from this (JP reports that > Twisted's assertWarns is broken in 2.6), and I doubt it's alone. > > So I guess I am filing a bug after all... :) Yeah - Brett's correct that everything under "test.test_support" should really be formally undocumented. It's mostly a place for code that reflects "things we do a lot in our unit tests and are tired of repeating" rather than "this is a good API that we want to support forever and encourage other people to use". However, if other folks turn out to have similar needs, then it may be possible to add something to unittest to support it. However, given that the beta deadline has already passed, you may need to use similar hackery to that used by catch_warning and replace warnings.showwarning with a test function that saves the raised exception (it also wouldn't be hard to enhance it a bit to handle more than a single warning). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 26 16:08:02 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 00:08:02 +1000 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> Message-ID: <4863A2C2.8000202@gmail.com> Benjamin Peterson wrote: > On Wed, Jun 25, 2008 at 4:00 PM, Guido van Rossum wrote: >> I'm a little worried about making stuff undocumented that every core >> developer needs to use -- everyone writing tests needs to continue to >> use test_support (now test.support?). I imagine people writing unit >> test suites for 3rd party libraries might want to use its services >> too. > > I think undocumented is a little unspecific here. What I mean is > "reserved for core Python tests and no promise is made to retain > compatibility." Of course, we would keep docs for them (perhaps in > Lib/test/README), so new core developers could write their tests. Alternatively, we could just leave them out of the docs table of contents, and stick a big warning at the top saying that these are internal APIs and subject to change without warning between releases. (This discussion actually argues somewhat in favour of having _test as the top-level package name for the regression test suite - after all, we make *zero* promises about keeping the API of any of the test modules the same from release to release, so we should really be using the standard leading underscore convention to flag internal implementation details that are not really intended for use by third parties) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Thu Jun 26 16:08:24 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 07:08:24 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com> Message-ID: On Thu, Jun 26, 2008 at 5:50 AM, Raymond Hettinger wrote: >> Just as a contrary point, I'm not particularly keen on the output >> format (which takes the form '0b1 * 2.0 ** 0' as far as I can see), > > That format was requested by everyone else on the tracker > discussion. What I originally wanted was something like 0b11.0101. > But that didn't round-trip through eval, it didn't match the style used in > the numerical papers referenced by Terry Reedy, and it didn't scale > well with inputs like 1.1E+100. > >> and I'm definitely not keen on the fact that it's overloaded on the >> hex/bin/oct builtins. >> >> Can't it be a separate function? > > Simplicity. bin/oct/hex have the job of giving alternate base > representations for numbers. > Nothing is gained by adding a duplicate set of functions in the math module > for float inputs. I disagree, and others here have disagreed too. We made a consicous decision to *remove* the overloading of hex/oct/bin via __hex__/__oct__/__bin__ in 3.0, in order to simplify these functions, which only work for integers, not for any other style of numbers. If bin(3.4) works, why not bin() of a Fraction, or of a complex number? Or for that matter, why not hex() of a string? All these have some use case. But is that use case important enough to put it in the bin()/hex()/oct() built-in functions? Why not hex() of a dictionary? Where does it end? We drew a line in the sand -- these are only for ints. >> would >> it not be better if it were machine-parseable, rather than executable? > > We already have struct.pack for machine-parseable output. > This is supposed to be human readable as well as providing > an exact, platform indepent way of specifying a particular > float value (something that's useful in testing algorithms like that in > math.sum()). The only use cases you bring up appear to be in testing and education. This is not a strong enough motivation for adding a wart to the bin/oct/hex builtins. I'm sure you can write the same thing in pure Python -- wouldn't that be good enough for testing? And if you then put it somewhere in a module in the stdlib, wouldn't that be good enough for education? There's a strong movement for keeping the language small and clean. Adding more overloaded functionality to built-in functions goes counter to that ideal. A new stdlib function causes no overhead to the *language* definition (and built-ins *are* part of the language). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Jun 26 16:12:17 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 26 Jun 2008 07:12:17 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> Message-ID: > Would you mind reading the rest of *this* thread on python-dev and > respond to the discussion about the design of the feature? The last four entries were from this thread. I don't know what else you want me to do. I can update the patch as people make suggestions. That's pretty much it. I recapped the earlier discussion from the tracker so the participants in this thread would be aware of the requests that were made there and why. I originally wanted a different output format but it evolved to the one that's there now to meet the various needs of the posters. This is important background for someone just joining the thread and thinking a different output format would be better. There's a part of this thread that says basically, "fine, but stick it somewhere else." To me, it doesn't make any sense at all to create a parallel set of functions in the math module. To convert a number to binary, it makes sense to use the bin() function. I don't understand this notion that bin() is a sacred cow of intergerdom and would philosophically corrupt if it handled floats also. Raymond From ncoghlan at gmail.com Thu Jun 26 16:19:10 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 00:19:10 +1000 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com> Message-ID: <4863A55E.1050102@gmail.com> Raymond Hettinger wrote: >> and I'm definitely not keen on the fact that it's overloaded on the >> hex/bin/oct builtins. >> >> Can't it be a separate function? > > Simplicity. bin/oct/hex have the job of giving alternate base > representations for numbers. > Nothing is gained by adding a duplicate set of functions in the math > module for float inputs. I'd place additional requirements on using bin/oct/hex for this: 1. The new feature must be available to floating point types other than float (such as Decimal) in both 2.6 and 3.0 (keeping in mind that 3.0 does not support __bin__, __hex__, or __oct__ methods - it uses only __index__ to implement bin(), hex() and oct() 2. Other classes (such as Decimal) should be able to leverage the formatting functionality provided for floats. If it was just a new method on float objects or a new function in the math module, neither of those additional requirements would apply - I would be completely fine with the function only working for actual float objects. However, in either case, I think this also runs afoul of the "we're in beta" argument - yes, it's a nice feature, but I don't think it's one that will cause any great dramas if users don't get their hands on it until 2.7/3.1. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 26 16:40:49 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 00:40:49 +1000 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> Message-ID: <4863AA71.5020901@gmail.com> Raymond Hettinger wrote: > There's a part of this thread that says basically, "fine, but stick it > somewhere else." To me, it doesn't make any sense at all to create a > parallel set of functions in the math module. To convert a number to > binary, it makes sense to use the bin() function. I don't understand > this notion that bin() is a sacred cow of intergerdom and would > philosophically corrupt if it handled floats also. It isn't the extension to something other than integers that bothers me, it's the extension to floats in a way that can't be easily supported for other types such as decimal.Decimal and fractions.Fraction. Well, that and the beta deadline (we have to draw the line somewhere, or we'll be stuck in an eternal spiral of "just one more feature") Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From exarkun at divmod.com Thu Jun 26 16:46:46 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 26 Jun 2008 10:46:46 -0400 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <4863A007.60804@gmail.com> Message-ID: <20080626144646.4714.191332201.divmod.quotient.13422@ohm> On Thu, 26 Jun 2008 23:56:23 +1000, Nick Coghlan wrote: > > [snip] >> >>Ok, then we're back to there being no supported way to write tests that >>need to >>intercept warnings. Twisted has already suffered from this (JP reports >>that >>Twisted's assertWarns is broken in 2.6), and I doubt it's alone. >> >>So I guess I am filing a bug after all... :) > >Yeah - Brett's correct that everything under "test.test_support" should >really be formally undocumented. It's mostly a place for code that reflects >"things we do a lot in our unit tests and are tired of repeating" rather >than "this is a good API that we want to support forever and encourage other >people to use". > >However, if other folks turn out to have similar needs, then it may be >possible to add something to unittest to support it. However, given that the >beta deadline has already passed, you may need to use similar hackery to >that used by catch_warning and replace warnings.showwarning with a test >function that saves the raised exception (it also wouldn't be hard to >enhance it a bit to handle more than a single warning). We don't use showwarning because in order to reliably catch warnings that way, it's necessary to rely on even more private implementation details of the warning system. Consider this: from warnings import warn from test.test_support import catch_warning def f(): warn("foo") def test(): with catch_warning() as w: f() assert str(w.message) == "foo", "%r != %r" % (w.message, "foo") test() test() The first assertion passes (by the way, I don't understand why w.message isn't the message passed to warn, but is instead an instance of UserWarning) but the second assertion fails. A more subtle example might include two functions, the first of which is deprecated and called by the second, and one test for each of them. Now the test for the warning will only pass if it runs before the other test; if they accidentally run in the other order, you won't see the warning, so as far as I can tell, you can't reliably write a unit test for warnings using catch_warning. The real problem with testing many uses of the warning system is that it doesn't expose enough public APIs for this to be possible. You *have* to use APIs which are, apparently, private (such as warn_explicit). Jean-Paul From ncoghlan at gmail.com Thu Jun 26 16:52:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 00:52:57 +1000 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <35D8067A7D90402BB2A4DF125ADD5036@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <79990c6b0806260516n5dd10c5bmde03f35fcb8e2156@mail.gmail.com> <4863A55E.1050102@gmail.com> <35D8067A7D90402BB2A4DF125ADD5036@RaymondLaptop1> Message-ID: <4863AD49.4050202@gmail.com> Raymond Hettinger wrote: > FYI, I had already updated the patch to incorporate your suggestion. > It checks __index__ first and then __float__ if index doesn't exist. > The result makes it work with Decimal inputs. Only sort of - the domain of the output is then constrained by what the builtin float type can represent, which strikes me as fairly inappropriate for what should be a lossless display operation (both fractions.Fraction and decimal.Decimal can represent a lot of numbers that float can't deal with properly). To use the old faithful example of this, given: >>> float(Decimal("1.1")) 1.1000000000000001 what happens for bin(Decimal("1.1")) with the patch version that uses __float__? As I've said elsewhere in this thread, I'm certainly not averse to the basic idea of letting bin/oct/hex handle rational numbers as well as integers, I just don't see it as something we have to have in the current release cycle, and nor do I see it really working properly without adding a __rational__ special method that returns a numerator/denominator 2-tuple (which would obviously require a PEP). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From eric+python-dev at trueblade.com Thu Jun 26 16:59:34 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 26 Jun 2008 10:59:34 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <48638A55.1030104@trueblade.com> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <48638A55.1030104@trueblade.com> Message-ID: <4863AED6.2020702@trueblade.com> Eric Smith wrote: > Actually, after saying I was opposed to __bin__ in 2.6, I said: > "Instead, I think the approach used in 3.0 (r64451) should be used > instead. That is, if this feature exist at all. I'm -0 on adding > bin(), etc. to floats." > > My last sentence is a little unclear. I meant I'm -0 on adding floats > as arguments to bin(), oct(), and hex(). Primarily because a) it's not > extensible in 3.0, and b) I find it surprising, in that I'd expect those > functions to throw an error for non-integral types (that is, those not > having __index__). I think adding a "float_as_binary_expression()" > (with a better name) in some module would get the functionality you > seek. What is gained by adding this to bin() and friends? And to clarify myself, yet again (from private email with Raymond): I actually think it's a useful feature, just not in bin(). I'm sure it will land somewhere, and I'm also sure I'll use it, at least from the interactive prompt. And if bin() were generally extensible for all types, I wouldn't really even care all that much if this feature landed in bin(). But 3.0 is going in the opposite direction, which is what much of my concern is based on, and why I commented at the outset on the 2.6 approach differing from the 3.0 approach. Eric. From guido at python.org Thu Jun 26 17:02:09 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 08:02:09 -0700 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <4863A007.60804@gmail.com> References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> <20080625130857.GA13663@steerpike.home.puzzling.org> <20080626054916.GA24366@steerpike.home.puzzling.org> <4863A007.60804@gmail.com> Message-ID: On Thu, Jun 26, 2008 at 6:56 AM, Nick Coghlan wrote: > Andrew Bennetts wrote: >> >> Brett Cannon wrote: >>> >>> On Wed, Jun 25, 2008 at 6:08 AM, Andrew Bennetts >> >> [...] >>>> >>>> Should I file a bug for this? >>>> >>> If you want, but Benjamin plans to undocument this for users along >>> with all other test.support stuff (which I agree with). Most of the >>> APIs in test.support were just quickly written and have not >>> necessarily been thought through in order to make sure that the APIs >>> makes sense (like in this case). >> >> Ok, then we're back to there being no supported way to write tests that >> need to >> intercept warnings. Twisted has already suffered from this (JP reports >> that >> Twisted's assertWarns is broken in 2.6), and I doubt it's alone. >> >> So I guess I am filing a bug after all... :) > > Yeah - Brett's correct that everything under "test.test_support" should > really be formally undocumented. It's mostly a place for code that reflects > "things we do a lot in our unit tests and are tired of repeating" rather > than "this is a good API that we want to support forever and encourage other > people to use". I still have a big problem with this attitude. If it's valuable enough to share between our tests, it should be properly written for reuse and documented. A collection of hacks remains a collection of hacks. We can do better. > However, if other folks turn out to have similar needs, then it may be > possible to add something to unittest to support it. However, given that the > beta deadline has already passed, you may need to use similar hackery to > that used by catch_warning and replace warnings.showwarning with a test > function that saves the raised exception (it also wouldn't be hard to > enhance it a bit to handle more than a single warning). I don't think the beta should be used as an argument to prevent refactoring and documenting code. I think Andrew's complaint is justified and should be addressed. I don't expect the test suite to remain unchanged while we're in beta anyways... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Thu Jun 26 17:12:40 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 26 Jun 2008 11:12:40 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <4863AA71.5020901@gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> Message-ID: <328D5F1E-E082-4002-9A65-38A31625C161@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 26, 2008, at 10:40 AM, Nick Coghlan wrote: > Well, that and the beta deadline (we have to draw the line > somewhere, or we'll be stuck in an eternal spiral of "just one more > feature") Guido wanted to get the beta out when we did exactly so we could draw this line in the sand. I'd much rather people be spending time getting what features we do have tested, stabilized, bug fixed, and turning the buildbots green across the board. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGOx6HEjvBPtnXfVAQK/IwP+MM75Kw0pMtzQ0wND51D6Gkyx/anr6xOf uLQ41AhcPd//XDTfMQOO9DYsks1sxt8UnDtajyAFGKK9tZmymhwttplJpBdFDSM7 4km2tV9TZZJQUI/VUwV/d6IT80xfdct6c+lkOjeOl2KMnkPEuu+AoLnG0Me8FKiX heC8FnyXut4= =pG60 -----END PGP SIGNATURE----- From ncoghlan at gmail.com Thu Jun 26 17:14:36 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 01:14:36 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <20080626144646.4714.191332201.divmod.quotient.13422@ohm> References: <20080626144646.4714.191332201.divmod.quotient.13422@ohm> Message-ID: <4863B25C.5070706@gmail.com> Jean-Paul Calderone wrote: > The first assertion passes (by the way, I don't understand why w.message > isn't the message passed to warn, but is instead an instance of > UserWarning) > but the second assertion fails. See Brett's comment elsewhere in this thread about the level of thought (or lack thereof) that went into the design of some of the test_support APIs (including catch_warning). > A more subtle example might include two > functions, the first of which is deprecated and called by the second, and > one test for each of them. Now the test for the warning will only pass if > it runs before the other test; if they accidentally run in the other order, > you won't see the warning, so as far as I can tell, you can't reliably > write > a unit test for warnings using catch_warning. Yeah, the Python regression tests actually cheat quite a bit when it comes to testing warnings - several of the tests clear the warning cache just in case the warning has already been triggered by other code. I suspect what some of those tests *should* be doing (aside from those testing the warning module itself) is configuring the warning system to convert the relevant warnings to exceptions, then catching the resulting exception. > The real problem with testing many uses of the warning system is that it > doesn't expose enough public APIs for this to be possible. You *have* > to use APIs which are, apparently, private (such as warn_explicit). Hmm, I think the bigger problem is that there is no documented way to save the warnings filter and restore it to a previous state - the 'filters' attribute (which holds the actual list of filters) isn't documented and isn't included in __all__. This makes it hard to write an officially supported test that fiddles with the warning settings then puts them back the way they were. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From glyph at divmod.com Thu Jun 26 17:18:55 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 26 Jun 2008 15:18:55 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics Message-ID: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Today on planetpython.org, Doug Hellman announced the June issue of Python magazine. The cover story this month is about Pybots, "the fantastic automation system that has been put in place to make sure new releases of Python software are as robust and stable as possible". Last week, there was a "beta" release of Python which, according to the community buildbots, cannot run any existing python software. Normally I'd be complaining here about Twisted, but in fact Twisted is doing relatively well right now; only 80 failing tests. Django apparently cannot even be imported. The community buildbots have been in a broken state for months now[1]. I've been restraining myself from whinging about this, but now that it's getting close to release, it's time to get these into shape, or it's time to get rid of them. There are a few separate issues here. One is, to what extent should Python actually be compatible between releases? There's not much point in saying "this release is compatible with python 2.5" if all python 2.5 programs need to be modified in order to run on python 2.6. Of course, it's quite likely that there are some 3rd-party Python programs that continue to work, but python.org provides links to 2 projects that don't and none that do. Another way to phrase this question is, "whose responsibility is it to make Python 2.5 programs run on Python 2.6"? Or, "what happens when the core team finds out that a change they have made has broken some python software 'in the wild'"? Here are a couple of ways to answer these questions: 1) It's the python core team's responsibility. There should be Python buildbots for lots of different projects and they should all be green before the code can be considered [one of: allowed in trunk, alpha, beta, RC]. 2) It's the python core team's responsibility as long as the major projects are using public APIs. The code should not be considered [alpha, beta, RC] if there are any known breakages in public APIs. 3) It's only the python core team's responsibility to notify projects of incompatible changes of which they are aware, which may occur in public or private APIs. Major projects with breakages will be given a grace period before a [beta, RC, final] to release a forward-compatible version. 4) The status quo: every new Python release can (and will, at least speaking in terms of 2.6) break lots of popular software, with no warning aside from the beta period. There are no guarantees. For obvious reasons, I'd prefer #1, but any of the above is better than #4. I don't think #4 is an intentional choice, either; I think it's the result of there being no clear consequence for community buildbots failing. Or, for that matter, of core buildbots failing. Investigating the history is tricky (I see a bunch of emails from Barry here, but I'm not sure which is the final state) but I believe the beta went out with a bunch of buildbots offline or failing. A related but independent issue is: should the community buildbots continue to exist? An automated test is only as good as the consequences of its failure. As it stands, the community buildbots seem to be nothing but an embarrassment. I don't mean this to slam Grig, or the Python team - I mean, literally, if there is no consequence of their failure then the only use I can see for the data the buildbots currently provide (i.e. months and months of failures) is to embarrass the Python core developers. python.org's purpose should not be to provide negative press for Python ;), so if this continues to be the case, they probably shouldn't be linked. This isn't just an issue with changes to Python breaking stuff; the pybots' configuration is apparently not well maintained, since the buildbots which say "2.5" aren't passing either, and that's not a problem with the code, it's just that the slaves are actually running against trunk at HEAD. Ultimately these tools only exist to ensure the quality of Python releases. The really critical question here is, what does it mean to have a Python release that is high-quality? There are some obvious things: it shouldn't crash, it should conform to its own documentation. Personally I think "it passes all of its own tests" and "it runs existing Python code" are important metrics too, but the most important thing I'm trying to get across here is that there should be a clear understanding of which goals the release / QA / buildbot / test process is trying to accomplish. For me, personally, it really needs to be clear when I can say "you guys screwed up and broke stuff", and when I just have to suck it up and deal with the consequences of a new version of Python in Twisted. It's definitely bad for all of us if the result is that new releases of Python just break everything. Users don't care where the responsibility lies, they just know that stuff doesn't work, so we should decide on a process which allows Twisted (and other popular projects, like Django, Plone, pytz, PyGTK, pylons, ...) to (mostly) run when new versions of Python are released. [1]: http://mail.python.org/pipermail/python-dev/2008-May/079212.html From guido at python.org Thu Jun 26 17:19:31 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 08:19:31 -0700 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: <4863A2C2.8000202@gmail.com> References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> <4863A2C2.8000202@gmail.com> Message-ID: On Thu, Jun 26, 2008 at 7:08 AM, Nick Coghlan wrote: > Benjamin Peterson wrote: >> >> On Wed, Jun 25, 2008 at 4:00 PM, Guido van Rossum >> wrote: >>> >>> I'm a little worried about making stuff undocumented that every core >>> developer needs to use -- everyone writing tests needs to continue to >>> use test_support (now test.support?). I imagine people writing unit >>> test suites for 3rd party libraries might want to use its services >>> too. >> >> I think undocumented is a little unspecific here. What I mean is >> "reserved for core Python tests and no promise is made to retain >> compatibility." Of course, we would keep docs for them (perhaps in >> Lib/test/README), so new core developers could write their tests. > > Alternatively, we could just leave them out of the docs table of contents, > and stick a big warning at the top saying that these are internal APIs and > subject to change without warning between releases. > > (This discussion actually argues somewhat in favour of having _test as the > top-level package name for the regression test suite - after all, we make > *zero* promises about keeping the API of any of the test modules the same > from release to release, so we should really be using the standard leading > underscore convention to flag internal implementation details that are not > really intended for use by third parties) That would also remove the problem users encounter from time to time with module files named "test.py". But I think we should think about this more. I don't think anyone expects the code inside any particular test_foo.py to have a stable public interface. But quite a bit of the test support infrastructure is reused by third party test frameworks. I think we should acknowledge and support such reuse. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Jun 26 17:21:36 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 01:21:36 +1000 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <328D5F1E-E082-4002-9A65-38A31625C161@python.org> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806241243w2aa65d82h8db272defcb96ea4@mail.gmail.com> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> Message-ID: <4863B400.4090401@gmail.com> Barry Warsaw wrote: > On Jun 26, 2008, at 10:40 AM, Nick Coghlan wrote: > >> Well, that and the beta deadline (we have to draw the line somewhere, >> or we'll be stuck in an eternal spiral of "just one more feature") > > Guido wanted to get the beta out when we did exactly so we could draw > this line in the sand. I'd much rather people be spending time getting > what features we do have tested, stabilized, bug fixed, and turning the > buildbots green across the board. Having been caught by the 2.5 beta deadline with the changes that eventually became PEP 361 (and I think were significantly improved by the additional attention that was available due to the delay) I understand completely. (And to everyone with features that get bumped to 2.7/3.1 because of this... while a number of you no doubt know this already, it really is astonishing how soon the next release seems to roll around, even with our fairly leisurely release schedules!) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From p.f.moore at gmail.com Thu Jun 26 17:27:01 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 26 Jun 2008 16:27:01 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <4863AED6.2020702@trueblade.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4861504B.209@v.loewis.de> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <48638A55.1030104@trueblade.com> <4863AED6.2020702@trueblade.com> Message-ID: <79990c6b0806260827g39b10270g4ccfb06ba4640cbf@mail.gmail.com> On 26/06/2008, Eric Smith wrote: > I actually think it's a useful feature, just not in bin(). I'm sure it will > land somewhere, and I'm also sure I'll use it, at least from the interactive > prompt. Can you give an example of its use? Maybe there are such examples in the tracker discussion, but I'm not following the tracker, and I think it would add some weight to the discussion here if use cases were noted. At the moment, the impression I get is that most of the python-dev comments are negative, but (without looking to confirm) most of the tracker comments are positive. If so, getting everything in one place will make it easier to see a balanced discussion. Having said all this, I agree with the point that this should be deferred to 2.7/3.1 now we're in beta. So there's hardly a rush. Paul. From exarkun at divmod.com Thu Jun 26 17:28:01 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 26 Jun 2008 11:28:01 -0400 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <4863B25C.5070706@gmail.com> Message-ID: <20080626152801.4714.909118230.divmod.quotient.13430@ohm> On Fri, 27 Jun 2008 01:14:36 +1000, Nick Coghlan wrote: >Jean-Paul Calderone wrote: > [snip] >>The real problem with testing many uses of the warning system is that it >>doesn't expose enough public APIs for this to be possible. You *have* >>to use APIs which are, apparently, private (such as warn_explicit). > >Hmm, I think the bigger problem is that there is no documented way to save >the warnings filter and restore it to a previous state - the 'filters' >attribute (which holds the actual list of filters) isn't documented and >isn't included in __all__. This makes it hard to write an officially >supported test that fiddles with the warning settings then puts them back >the way they were. It sounds like you're agreeing that there aren't enough public APIs. Making warn_explicit public addresses this particular problem in one way - by letting applications hook into the warning system before filters are applied. Making the filter list public is another way, since it would let applications clear and then restore the filters. I don't particularly care about the details, I just want some public API for this. Making warn_explicit public seems better to me, since it was already there in previous versions of Python, and it lets you completely ignore both the filters list and the global registry, but if others would rather make the filters and global registry a part of the public API, that's fine by me as well. Jean-Paul From guido at python.org Thu Jun 26 17:28:36 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 08:28:36 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <4863B400.4090401@gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> Message-ID: On Thu, Jun 26, 2008 at 8:21 AM, Nick Coghlan wrote: > Barry Warsaw wrote: >> >> On Jun 26, 2008, at 10:40 AM, Nick Coghlan wrote: >> >>> Well, that and the beta deadline (we have to draw the line somewhere, or >>> we'll be stuck in an eternal spiral of "just one more feature") >> >> Guido wanted to get the beta out when we did exactly so we could draw this >> line in the sand. I'd much rather people be spending time getting what >> features we do have tested, stabilized, bug fixed, and turning the buildbots >> green across the board. > > Having been caught by the 2.5 beta deadline with the changes that eventually > became PEP 361 (and I think were significantly improved by the additional > attention that was available due to the delay) I understand completely. > > (And to everyone with features that get bumped to 2.7/3.1 because of this... > while a number of you no doubt know this already, it really is astonishing > how soon the next release seems to roll around, even with our fairly > leisurely release schedules!) I'd like to separate the concerns though. I personally don't want to see the feature in its current form added to 2.7/3.1 either. As others pointed out, it's a wart on the bin/oct/hex functions. So as far as the feature design goes, I offer some suggestions: a new module; or a new function in math; or a new method on float. Since Raymond is the champion for the feature let him choose the API from those alternatives. Regarding the addition to 2.6/3.0 post beta 1, I think a new module has the most chance of success, especially if it's written in Python in such a way as to need minimal changes between 2.6 and 3.0. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Jun 26 17:32:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 01:32:08 +1000 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> <4863A2C2.8000202@gmail.com> Message-ID: <4863B678.5040205@gmail.com> Guido van Rossum wrote: > But I think we should think about this more. I don't think anyone > expects the code inside any particular test_foo.py to have a stable > public interface. But quite a bit of the test support infrastructure > is reused by third party test frameworks. I think we should > acknowledge and support such reuse. I'm personally fine with that approach, but some of the new items in there for 2.6 could probably use a bit of cleaning up to improve the APIs and/or the way they work. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Thu Jun 26 17:33:23 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 08:33:23 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: Too verbose, Glyph. :-) It needs to be decided case-by-case. The beta is called beta because, well, it may break stuff and we may want to fix it. But there are also likely many frameworks that use undefined behavior. I'm not particularly impressed by statistics like "all tests are red" -- this may all be caused by a single issue. For example, I'd much rather read an explanation about *why* Django cannot even be imported than a blanket complaint that this is a disgrace. So why is it? --Guido On Thu, Jun 26, 2008 at 8:18 AM, wrote: > Today on planetpython.org, Doug Hellman announced the June issue of Python > magazine. The cover story this month is about Pybots, "the fantastic > automation system that has been put in place to make sure new releases of > Python software are as robust and stable as possible". > > Last week, there was a "beta" release of Python which, according to the > community buildbots, cannot run any existing python software. Normally I'd > be complaining here about Twisted, but in fact Twisted is doing relatively > well right now; only 80 failing tests. Django apparently cannot even be > imported. > > The community buildbots have been in a broken state for months now[1]. I've > been restraining myself from whinging about this, but now that it's getting > close to release, it's time to get these into shape, or it's time to get rid > of them. > > There are a few separate issues here. One is, to what extent should Python > actually be compatible between releases? There's not much point in saying > "this release is compatible with python 2.5" if all python 2.5 programs need > to be modified in order to run on python 2.6. Of course, it's quite likely > that there are some 3rd-party Python programs that continue to work, but > python.org provides links to 2 projects that don't and none that do. > > Another way to phrase this question is, "whose responsibility is it to make > Python 2.5 programs run on Python 2.6"? Or, "what happens when the core > team finds out that a change they have made has broken some python software > 'in the wild'"? > > Here are a couple of ways to answer these questions: > > 1) It's the python core team's responsibility. There should be Python > buildbots for lots of different projects and they should all be green before > the code can be considered [one of: allowed in trunk, alpha, beta, RC]. > 2) It's the python core team's responsibility as long as the major projects > are using public APIs. The code should not be considered [alpha, beta, RC] > if there are any known breakages in public APIs. > 3) It's only the python core team's responsibility to notify projects of > incompatible changes of which they are aware, which may occur in public or > private APIs. Major projects with breakages will be given a grace period > before a [beta, RC, final] to release a forward-compatible version. > 4) The status quo: every new Python release can (and will, at least > speaking in terms of 2.6) break lots of popular software, with no warning > aside from the beta period. There are no guarantees. > > For obvious reasons, I'd prefer #1, but any of the above is better than #4. > I don't think #4 is an intentional choice, either; I think it's the result > of there being no clear consequence for community buildbots failing. Or, > for that matter, of core buildbots failing. Investigating the history is > tricky (I see a bunch of emails from Barry here, but I'm not sure which is > the final state) but I believe the beta went out with a bunch of buildbots > offline or failing. > > A related but independent issue is: should the community buildbots continue > to exist? An automated test is only as good as the consequences of its > failure. As it stands, the community buildbots seem to be nothing but an > embarrassment. I don't mean this to slam Grig, or the Python team - I mean, > literally, if there is no consequence of their failure then the only use I > can see for the data the buildbots currently provide (i.e. months and months > of failures) is to embarrass the Python core developers. python.org's > purpose should not be to provide negative press for Python ;), so if this > continues to be the case, they probably shouldn't be linked. This isn't > just an issue with changes to Python breaking stuff; the pybots' > configuration is apparently not well maintained, since the buildbots which > say "2.5" aren't passing either, and that's not a problem with the code, > it's just that the slaves are actually running against trunk at HEAD. > > Ultimately these tools only exist to ensure the quality of Python releases. > The really critical question here is, what does it mean to have a Python > release that is high-quality? There are some obvious things: it shouldn't > crash, it should conform to its own documentation. Personally I think "it > passes all of its own tests" and "it runs existing Python code" are > important metrics too, but the most important thing I'm trying to get across > here is that there should be a clear understanding of which goals the > release / QA / buildbot / test process is trying to accomplish. For me, > personally, it really needs to be clear when I can say "you guys screwed up > and broke stuff", and when I just have to suck it up and deal with the > consequences of a new version of Python in Twisted. > > It's definitely bad for all of us if the result is that new releases of > Python just break everything. Users don't care where the responsibility > lies, they just know that stuff doesn't work, so we should decide on a > process which allows Twisted (and other popular projects, like Django, > Plone, pytz, PyGTK, pylons, ...) to (mostly) run when new versions of Python > are released. > > [1]: http://mail.python.org/pipermail/python-dev/2008-May/079212.html > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jun 26 17:34:25 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 08:34:25 -0700 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: <4863B678.5040205@gmail.com> References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> <4863A2C2.8000202@gmail.com> <4863B678.5040205@gmail.com> Message-ID: On Thu, Jun 26, 2008 at 8:32 AM, Nick Coghlan wrote: > Guido van Rossum wrote: >> >> But I think we should think about this more. I don't think anyone >> expects the code inside any particular test_foo.py to have a stable >> public interface. But quite a bit of the test support infrastructure >> is reused by third party test frameworks. I think we should >> acknowledge and support such reuse. > > I'm personally fine with that approach, but some of the new items in there > for 2.6 could probably use a bit of cleaning up to improve the APIs and/or > the way they work. So get crackin'! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Jun 26 17:42:56 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 01:42:56 +1000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: <4863B900.1040808@gmail.com> glyph at divmod.com wrote: > Today on planetpython.org, Doug Hellman announced the June issue of > Python magazine. The cover story this month is about Pybots, "the > fantastic automation system that has been put in place to make sure new > releases of Python software are as robust and stable as possible". > > Last week, there was a "beta" release of Python which, according to the > community buildbots, cannot run any existing python software. Normally > I'd be complaining here about Twisted, but in fact Twisted is doing > relatively well right now; only 80 failing tests. Django apparently > cannot even be imported. beta 1 has some trouble running *our* test suite - I'd be fairly surprised if the community buildbots were in significantly better shape. > The community buildbots have been in a broken state for months now[1]. Continuously running community buildbots on the maintenance trees makes sense, since those trees should always be in a releasable state. For the trunk, they're really only interesting when the Python core buildbots are reporting all green, but some of the community buildbots are reporting red. One of the problems is what the term "beta" means to different groups - for us, this first beta was really about saying "zero new features from here on, focus on making what we have now work properly". The relatively late landing of a couple of major PEPs (371, 3108) also didn't do any favours for trunk stability. If the community buildbots aren't largely green by the time beta 2 comes out, that's when I'll agree we have a problem - they should definitely be green by the time first release candidate comes out. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 26 17:47:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 01:47:57 +1000 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> Message-ID: <4863BA2D.8030805@gmail.com> Guido van Rossum wrote: > I personally don't want to see the feature in its current form added > to 2.7/3.1 either. As others pointed out, it's a wart on the > bin/oct/hex functions. > > So as far as the feature design goes, I offer some suggestions: a new > module; or a new function in math; or a new method on float. Since > Raymond is the champion for the feature let him choose the API from > those alternatives. > > Regarding the addition to 2.6/3.0 post beta 1, I think a new module > has the most chance of success, especially if it's written in Python > in such a way as to need minimal changes between 2.6 and 3.0. One of the other reasons I'd like to postpone the feature is that I think with a clean design behind it it could be an elegant addition to those builtins rather than a wart. But helping Raymond to convince you or anyone else of that is well down my to-do list at the moment (which I think just got longer with the test_support discussion...) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Jun 26 17:52:18 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 01:52:18 +1000 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <20080626152801.4714.909118230.divmod.quotient.13430@ohm> References: <20080626152801.4714.909118230.divmod.quotient.13430@ohm> Message-ID: <4863BB32.3040101@gmail.com> Jean-Paul Calderone wrote: > I don't particularly care about the details, I just want some public > API for this. Making warn_explicit public seems better to me, since > it was already there in previous versions of Python, and it lets you > completely ignore both the filters list and the global registry, but > if others would rather make the filters and global registry a part of > the public API, that's fine by me as well. Why do you say warn_explicit isn't public? It's in both the 2.5 and 2.6 API docs for the warnings module. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From exarkun at divmod.com Thu Jun 26 18:01:18 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 26 Jun 2008 12:01:18 -0400 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <4863BB32.3040101@gmail.com> Message-ID: <20080626160118.4714.38891548.divmod.quotient.13435@ohm> On Fri, 27 Jun 2008 01:52:18 +1000, Nick Coghlan wrote: >Jean-Paul Calderone wrote: >>I don't particularly care about the details, I just want some public >>API for this. Making warn_explicit public seems better to me, since >>it was already there in previous versions of Python, and it lets you >>completely ignore both the filters list and the global registry, but >>if others would rather make the filters and global registry a part of >>the public API, that's fine by me as well. > >Why do you say warn_explicit isn't public? It's in both the 2.5 and 2.6 API >docs for the warnings module. Brett told me it was private (on this list several weeks or a month or so ago). It's also no longer called in 2.6 by the C implementation of the warning system. Jean-Paul From glyph at divmod.com Thu Jun 26 18:21:08 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 26 Jun 2008 16:21:08 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> On 03:33 pm, guido at python.org wrote: >Too verbose, Glyph. :-) Mea culpa. "Glyph" might be a less appropriate moniker than "Altogether too many glyphs". >It needs to be decided case-by-case. If certain tests are to be ignored on a case-by-case basis, why not record that decision by disabling the test in the code? Otherwise, the decision inevitably gets boiled down to "it's okay to release the code with a bunch of tests failing, but I don't know which ones". As it was on Twisted when we used to make case-by-case decisions about failures, and as it is here now. >The beta is called beta because, well, it may break stuff and we may >want to fix it. That's also why the alpha is called an alpha. My informal understanding is that a beta should have no (or at least very few) known issues, with a medium level of confidence that no further ones will be found. An RC would have absolutely no known issues with a fairly high level of confidence that no further ones will be found. This would include the community buildbots basically working for the most part; I would not be surprised if there were a few tests that still had issues. But clearly the reality does not meet my informal expectations, so it would be nice to have something written down to check against. Still, I'm bringing this up now because it _is_ a beta, and I think it will be too late to talk about dealing with persistent test failures after the RCs start coming out. (Of course, I'm just being sneaky. I don't actually care if it's clearly documented, I just care that I stop having problems with incompatibility. But I believe having it clearly spelled out would actually prevent a lot of the problems that I've been having, since I don't think anyone would *intentionally* select a policy where every release breaks at least one major dependent project.) >I'm not particularly impressed by statistics like "all tests are red" >-- this >may all be caused by a single issue. The issue, for me, is not specifically that tests are red. It's that there's no clear policy about what to do about that. Can a release go out with some of the tests being red? If so, what are the extenuating circumstances? Does this have to be fixed? If not, why not? Why are we talking about this now? Shouldn't the change which caused Django to become unimportable have been examined at the time, rather than months later? (In other words, if it *is* a single issue, great, it's easy to fix: revert that single issue.) If not, then shouldn't someone in Django-land have been notified so they could cope with the change? Sorry that there are so many questions here; if I had fewer, I'd use fewer words to ask them. >For example, I'd much rather read an explanation about *why* Django >cannot even be imported than a blanket complaint that this is a >disgrace. So why is it? I don't know. JP is already addressing the issues affecting Twisted in another thread (incompatible changes in the private-but-necessary-to- get-any-testing-done API of the warnings module). But I really think that whoever made the change which broke it should be the one investigating it, not me. From glyph at divmod.com Thu Jun 26 18:33:46 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 26 Jun 2008 16:33:46 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <4863B900.1040808@gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <4863B900.1040808@gmail.com> Message-ID: <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com> On 03:42 pm, ncoghlan at gmail.com wrote: >glyph at divmod.com wrote: >beta 1 has some trouble running *our* test suite - I'd be fairly >surprised if the community buildbots were in significantly better >shape. That's another problem, yes :) >>The community buildbots have been in a broken state for months now[1]. >If the community buildbots aren't largely green by the time beta 2 >comes out, that's when I'll agree we have a problem - they should >definitely be green by the time first release candidate comes out. I have mixed feelings about this: on the one hand, if this were a matter of public record, I would have known that this was too early to start complaining, and we could have saved everyone a lot of time reading these messages. That would be good; I would prefer to just let everyone work without bothering about process. On the other hand, it's much easier to deal with test failures as they arise than in one giant chunk of work at the end of the development process. I spoke to a few core developers at PyCon who thought that the buildbots should always be green and any change that broke them should be reverted. They may remain nameless unless they wish to raise their hands :-) but that's definitely how I feel about it. >Continuously running community buildbots on the maintenance trees makes >sense, since those trees should always be in a releasable state. For >the trunk, they're really only interesting when the Python core >buildbots are reporting all green, but some of the community buildbots >are reporting red. (... which should be all the time ...) >One of the problems is what the term "beta" means to different groups - >for us, this first beta was really about saying "zero new features from >here on, focus on making what we have now work properly". The >relatively late landing of a couple of major PEPs (371, 3108) also >didn't do any favours for trunk stability. A big part of why I wrote this message is that I wanted a clear understanding of the relationship between the definition of "alpha", "beta" and "RC" and the state of various buildbots. If that relationship exists already, just linking to it from http://python.org/download/releases/2.6/ would be good. By the way, that page still says these are "alpha" releases. From schmir at gmail.com Thu Jun 26 18:42:33 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Thu, 26 Jun 2008 18:42:33 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum wrote: > an explanation about *why* Django cannot even be imported than a > blanket complaint that this is a disgrace. So why is it? > File "/home/ralf/pediapress/appserver/django/db/models/options.py", line 198, in _many_to_many self._fill_m2m_cache() File "/home/ralf/pediapress/appserver/django/db/models/options.py", line 221, in _fill_m2m_cache cache[field] = None File "/home/ralf/pediapress/appserver/django/utils/datastructures.py", line 75, in __setitem__ super(SortedDict, self).__setitem__(key, value) TypeError: unhashable type: 'ManyToManyField' same for sqlalchemy: egg/sqlalchemy/schema.py", line 113, in __call__ return type.__call__(self, name, metadata, *args, **kwargs) File "/home/ralf/py26/lib/python2.6/site-packages/SQLAlchemy-0.5.0beta1-py2.6.egg/sqlalchemy/schema.py", line 207, in __init__ self.primary_key = PrimaryKeyConstraint() File "/home/ralf/py26/lib/python2.6/site-packages/SQLAlchemy-0.5.0beta1-py2.6.egg/sqlalchemy/schema.py", line 294, in _set_primary_key self.constraints.add(pk) TypeError: unhashable type: 'PrimaryKeyConstraint' and already discussed: http://mail.python.org/pipermail/python-dev/2008-April/078421.html - Ralf From scott+python-dev at scottdial.com Thu Jun 26 18:52:58 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Thu, 26 Jun 2008 12:52:58 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> Message-ID: <4863C96A.9030802@scottdial.com> Ralf Schmitt wrote: > TypeError: unhashable type: 'ManyToManyField' > TypeError: unhashable type: 'PrimaryKeyConstraint' > > and already discussed: > http://mail.python.org/pipermail/python-dev/2008-April/078421.html Following the discussion to it's conclusions leads one to a tracker issue [1] that was supposed to be a blocker for the beta but (I assume) was forgotten about. Clearly this should be promoted to being a blocker again and some decision made. [1] http://bugs.python.org/issue2235 -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From glyph at divmod.com Thu Jun 26 18:54:49 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 26 Jun 2008 16:54:49 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> Message-ID: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> On 04:42 pm, schmir at gmail.com wrote: >On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum >wrote: >>an explanation about *why* Django cannot even be imported than a >>blanket complaint that this is a disgrace. So why is it? >and already discussed: >http://mail.python.org/pipermail/python-dev/2008-April/078421.html Following that trail of breadcrumbs, I ended up here: http://bugs.python.org/issue2235 with a message from some guy named "Barry Warsaw" (anyone know him?) that says: """ Guido, can you comment on Amaury's latest patch? I'm going to bump this back to critical so as not to hold up 2.6 alpha, but it should be marked as a release blocker for the first beta. """ I don't know if this "Barry" guy has the appropriate permissions on the bugtracker to increase priorities, so I've taken the liberty of upgrading it as a release blocker for the _second_ beta ... ;-). So, at least there's been one productive consequence of this discussion. From guido at python.org Thu Jun 26 19:03:59 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 10:03:59 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> Message-ID: On Thu, Jun 26, 2008 at 9:21 AM, wrote: > On 03:33 pm, guido at python.org wrote: >> It needs to be decided case-by-case. > > If certain tests are to be ignored on a case-by-case basis, why not record > that decision by disabling the test in the code? Otherwise, the decision > inevitably gets boiled down to "it's okay to release the code with a bunch > of tests failing, but I don't know which ones". As it was on Twisted when > we used to make case-by-case decisions about failures, and as it is here > now. No, I just meant that we need to figure out for each 3rd party test that fails whether the failure is our fault (too incompatibile) or theirs (relied on undefined behavior) and what the best fix is (change our code or theirs -- note that even if it's there fault there are cases where the best fix is to change our code. >> The beta is called beta because, well, it may break stuff and we may want >> to fix it. > > That's also why the alpha is called an alpha. My informal understanding is > that a beta should have no (or at least very few) known issues, with a > medium level of confidence that no further ones will be found. An RC would > have absolutely no known issues with a fairly high level of confidence that > no further ones will be found. This would include the community buildbots > basically working for the most part; I would not be surprised if there were > a few tests that still had issues. I guess we differ in interpretation. I see alphas as early outreach to the community. I see betas as a line in the sand where we stop adding/changing functionality -- but certainly not the moment where we expect everything to be perfect. During the beta period we fix bugs only -- those could be problems with new features, but those could also be problems where we accidentally introduced a backwards incompatibility. I'm sorry if your interpretation of the terminology is different, but this is mine and this is what we've always used, and it's not likely to change. (At least not for the 2.6/3.0 release.) > But clearly the reality does not meet my informal expectations, so it would > be nice to have something written down to check against. Still, I'm > bringing this up now because it _is_ a beta, and I think it will be too late > to talk about dealing with persistent test failures after the RCs start > coming out. Absolutely correct. The RCs are hoped to be as good as the final release. *Now* is the time to bring up issue. But please bring up specific issues -- I don't want to have an extended discussion about process or quality or expectations. I just want the code to be fixed. > (Of course, I'm just being sneaky. I don't actually care if it's clearly > documented, I just care that I stop having problems with incompatibility. > But I believe having it clearly spelled out would actually prevent a lot of > the problems that I've been having, since I don't think anyone would > *intentionally* select a policy where every release breaks at least one > major dependent project.) And here we seem to be parting our ways. We have a large amount of process already. I don't want more. >> I'm not particularly impressed by statistics like "all tests are red" -- >> this >> may all be caused by a single issue. > > The issue, for me, is not specifically that tests are red. It's that > there's no clear policy about what to do about that. Can a release go out > with some of the tests being red? If so, what are the extenuating > circumstances? Does this have to be fixed? If not, why not? Why are we > talking about this now? Shouldn't the change which caused Django to become > unimportable have been examined at the time, rather than months later? (In > other words, if it *is* a single issue, great, it's easy to fix: revert that > single issue.) If not, then shouldn't someone in Django-land have been > notified so they could cope with the change? > > Sorry that there are so many questions here; if I had fewer, I'd use fewer > words to ask them. If you're talking about community buildbots (which I presume are testing 3rd party packages against the core release) being red, that's out of scope for the core developers. We have enough trouble keeping our own (core) buildbots green, as you may have noticed. Some of the core buildbots are red because, well, frankly, they run on a cranky old version of an OS that few people care about. And then sometimes we leave them red, or turn them off. I see the core buildbots as a tool for developers to have their changes tested on a wide variety of platforms. They are an internal tool that gives us guidance about whether we're ready. I hope the community buildbots can be used the same way: a red bot means someone needs to look into an issue. The issue could be with the core or with the 3rd party package being tested. I don't think a policy like "no community buildbots should be red" makes any sense. >> For example, I'd much rather read an explanation about *why* Django cannot >> even be imported than a blanket complaint that this is a disgrace. So why is >> it? > > I don't know. JP is already addressing the issues affecting Twisted in > another thread (incompatible changes in the private-but-necessary-to- > get-any-testing-done API of the warnings module). But I really think that > whoever made the change which broke it should be the one investigating it, > not me. Whoever made what change? You can't seriously expect core developers investigating issues with 3rd party packages, no matter what the cause. The correct process is that someone who cares about the 3rd party package (could be an end user, a developer of that package, or a core developer who happens to care) looks into the issue enough to diagnose it, and then either proposes a fix or files a bug with the likely culprit, which could be the core or the 3rd party package. If nobody cares, well, that's open source too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jun 26 19:07:12 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 10:07:12 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> Message-ID: And just to make my position perfectly clear, I've unassigned it, since I don't foresee to be able to give this issue the quality time it clearly needs. Mind you, I agree it's a release blocker. But I don't have time to personally investigate it. Sorry. On Thu, Jun 26, 2008 at 9:54 AM, wrote: > On 04:42 pm, schmir at gmail.com wrote: >> >> On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum >> wrote: >>> >>> an explanation about *why* Django cannot even be imported than a >>> blanket complaint that this is a disgrace. So why is it? > >> and already discussed: >> http://mail.python.org/pipermail/python-dev/2008-April/078421.html > > Following that trail of breadcrumbs, I ended up here: > > http://bugs.python.org/issue2235 > > with a message from some guy named "Barry Warsaw" (anyone know him?) that > says: > > """ > Guido, can you comment on Amaury's latest patch? I'm going to bump this > back to critical so as not to hold up 2.6 alpha, but it should be marked > as a release blocker for the first beta. > """ > > I don't know if this "Barry" guy has the appropriate permissions on the > bugtracker to increase priorities, so I've taken the liberty of upgrading it > as a release blocker for the _second_ beta ... ;-). So, at least there's > been one productive consequence of this discussion. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Jun 26 19:14:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 19:14:32 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> Message-ID: <4863CE78.7010108@v.loewis.de> > That's also why the alpha is called an alpha. My informal understanding > is that a beta should have no (or at least very few) known issues No, that's not the purpose. Instead, it is a promise that no further features will be added, i.e. that the code is stable from a feature point of view. It certainly will contain bugs. The final release will certainly contain bugs, just look at the long list of open bug reports. > The issue, for me, is not specifically that tests are red. It's that > there's no clear policy about what to do about that. Can a release go > out with some of the tests being red? If so, what are the extenuating > circumstances? The final release, no. The beta release: sure (in particular if it's the first beta release). Also make a difference between the Python buildbots, and the community buildbots. I think few if any core committers ever look at the status of the community buildbots - the community has to bring breakage to our attention (as you just did). > Does this have to be fixed? If not, why not? Here, I'm confused what specifically you talk about. That Django doesn't import? It is certainly not part of the release procedure to verify that Django can still be imported. > Why are > we talking about this now? Shouldn't the change which caused Django to > become unimportable have been examined at the time, rather than months > later? Somebody should have pointed it out when it happened. Unfortunately, nobody did, so apparently, the community doesn't really care. > I don't know. JP is already addressing the issues affecting Twisted in > another thread (incompatible changes in the private-but-necessary-to- > get-any-testing-done API of the warnings module). But I really think > that whoever made the change which broke it should be the one > investigating it, not me. How could they have known that they broke it? Regards, Martin From martin at v.loewis.de Thu Jun 26 19:23:47 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 19:23:47 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <4863B900.1040808@gmail.com> <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com> Message-ID: <4863D0A3.50600@v.loewis.de> > A big part of why I wrote this message is that I wanted a clear > understanding of the relationship between the definition of "alpha", > "beta" and "RC" and the state of various buildbots. If that > relationship exists already, just linking to it from > http://python.org/download/releases/2.6/ would be good. By the way, > that page still says these are "alpha" releases. I think its significantly simpler to answer specific questions on the mailing list, to educate the specific set of people who have this question, than to put an official answer on the web page. So I'm skeptical that anybody will change the web page - just continue asking questions as you encounter them. Regards, Martin From schmir at gmail.com Thu Jun 26 19:40:25 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Thu, 26 Jun 2008 19:40:25 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> Message-ID: <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> On Thu, Jun 26, 2008 at 6:54 PM, wrote: > On 04:42 pm, schmir at gmail.com wrote: >> >> On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum >> wrote: >>> >>> an explanation about *why* Django cannot even be imported than a >>> blanket complaint that this is a disgrace. So why is it? > >> and already discussed: >> http://mail.python.org/pipermail/python-dev/2008-April/078421.html > > Following that trail of breadcrumbs, I ended up here: > > http://bugs.python.org/issue2235 > > with a message from some guy named "Barry Warsaw" (anyone know him?) that > says: > > """ > Guido, can you comment on Amaury's latest patch? I'm going to bump this > back to critical so as not to hold up 2.6 alpha, but it should be marked > as a release blocker for the first beta. > """ > > I don't know if this "Barry" guy has the appropriate permissions on the > bugtracker to increase priorities, so I've taken the liberty of upgrading it > as a release blocker for the _second_ beta ... ;-). So, at least there's > been one productive consequence of this discussion. this patch even make things worse for me. now py.test also dies. From guido at python.org Thu Jun 26 19:55:30 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 10:55:30 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> Message-ID: On Thu, Jun 26, 2008 at 10:40 AM, Ralf Schmitt wrote: > this patch even make things worse for me. now py.test also dies. Please add details to the tracker. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From schmir at gmail.com Thu Jun 26 20:07:41 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Thu, 26 Jun 2008 20:07:41 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> Message-ID: <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com> On Thu, Jun 26, 2008 at 7:55 PM, Guido van Rossum wrote: > On Thu, Jun 26, 2008 at 10:40 AM, Ralf Schmitt wrote: >> this patch even make things worse for me. now py.test also dies. > > Please add details to the tracker. > Well, I think most probably the patch is correct and it just catches another class which defines __eq__ but does not define __hash__. So nothing to add there. The question is if this behaviour should be kept or not. I guess there's no bug tracking that... From python at rcn.com Thu Jun 26 20:07:55 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 26 Jun 2008 11:07:55 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> Message-ID: From: "Guido van Rossum" > So as far as the feature design goes, I offer some suggestions: a new > module; or a new function in math; or a new method on float. Since > Raymond is the champion for the feature let him choose the API from > those alternatives. I choose bin/hex/oct methods on floatobjects. Will work-up a patch. Raymond From guido at python.org Thu Jun 26 20:45:06 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 11:45:06 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com> Message-ID: On Thu, Jun 26, 2008 at 11:07 AM, Ralf Schmitt wrote: > On Thu, Jun 26, 2008 at 7:55 PM, Guido van Rossum wrote: >> On Thu, Jun 26, 2008 at 10:40 AM, Ralf Schmitt wrote: >>> this patch even make things worse for me. now py.test also dies. >> >> Please add details to the tracker. > Well, I think most probably the patch is correct and it just catches > another class which defines __eq__ but does not define __hash__. So > nothing to add there. So you're saying py.test needs to be fixed? Fine with me, but then I don't understand why you bothered bringing it up here instead of fixing it (or reporting to the py.test maintainers, I don't know if you're one or not). > The question is if this behaviour should be kept or not. I guess > there's no bug tracking that... The best place is that very same issue. If this feature is really too annoying something needs to be done. Detailed reports about real code that gets broken without a good cause help making that case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Jun 26 20:50:08 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 11:50:08 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> Message-ID: On Thu, Jun 26, 2008 at 11:07 AM, Raymond Hettinger wrote: > From: "Guido van Rossum" >> >> So as far as the feature design goes, I offer some suggestions: a new >> module; or a new function in math; or a new method on float. Since >> Raymond is the champion for the feature let him choose the API from >> those alternatives. > > I choose bin/hex/oct methods on floatobjects. > Will work-up a patch. Let's step back and discuss the API some more. - Do we need all three? - If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.) - What should the output format be? I know you originally favored 0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex builtins, the constraint that it needs to be an eval() able expression may be dropped (unless you see a use case for that too). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jmp at MIT.EDU Thu Jun 26 20:57:27 2008 From: jmp at MIT.EDU (Justin Mazzola Paluska) Date: Thu, 26 Jun 2008 14:57:27 -0400 Subject: [Python-Dev] Disable tests in unittest (issue3202) In-Reply-To: References: <20080625211349.GA471@mit.edu> Message-ID: <20080626185727.GA29900@mit.edu> On Thu, Jun 26, 2008 at 10:51:49AM +1000, Jonathan Lange wrote: > On Thu, Jun 26, 2008 at 7:13 AM, Justin Mazzola Paluska wrote: > I think it's really worth looking at the approach that bzrlib's tests > take here (see bzrlib.tests.ExtendedTestResult and the > out-of-date-but-helpful http://bazaar-vcs.org/BzrExtendTestSuite) > > Instead of disabling the tests, their practice is to run the tests but > have them raise KnownFailure. When they write a test that they know > will fail, they have it raise this exception. The extended TestResult > object catches this exception in addFailure and reports it > appropriately.[1] I wasn?t aware of bzrlib?s extentions to unittest ? many of them, including KnownFailure, TestSkipped, bzrlib.test.ExtendedTestResult.expectFailure, and the sundry list of bzrlib.test.TestCase.assert* look useful enough to belong in the standard library. > I can see that disabling still has some value, but I'd rather have > KnownFailure first. I think disabling and the bzr extensions have different use cases. KnownFailure lets someone with an intimate knowledge of the the failing test case write down that they expect the test to fail. The disabled decorator lets someone less familiar with the test cases to easily disable tests that are getting in their way, without needing to understand the body of the test function (or worse, mis-understanding it and breaking it in a hard-to-understand way). I agree that in the ideal world, we?d like to have both. In fact, if we had the bzr extensions, it?d be easy to write the disabled decorator in terms of their KnownFailure: def disabled(func): @wraps(func): def wrapper(*args, **kwargs): raise KnownFailure("%s disabled" % func.__name__) return wrapper However, unless Python?s unittest is going to incorporate the useful features of bzrlib?s TestCase object[*], it probably makes sense to implement the disabled feature separately. ?Justin [*] Perhaps in the long term, this is the right thing to do as unittest seems to have fallen behind in useful feature count as its ?conceptual? parent JUnit. From g.brandl at gmx.net Thu Jun 26 21:33:58 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Jun 2008 21:33:58 +0200 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> <20080625130857.GA13663@steerpike.home.puzzling.org> <20080626054916.GA24366@steerpike.home.puzzling.org> <4863A007.60804@gmail.com> Message-ID: Guido van Rossum schrieb: >>> Ok, then we're back to there being no supported way to write tests that >>> need to >>> intercept warnings. Twisted has already suffered from this (JP reports >>> that >>> Twisted's assertWarns is broken in 2.6), and I doubt it's alone. >>> >>> So I guess I am filing a bug after all... :) >> >> Yeah - Brett's correct that everything under "test.test_support" should >> really be formally undocumented. It's mostly a place for code that reflects >> "things we do a lot in our unit tests and are tired of repeating" rather >> than "this is a good API that we want to support forever and encourage other >> people to use". > > I still have a big problem with this attitude. If it's valuable enough > to share between our tests, it should be properly written for reuse > and documented. A collection of hacks remains a collection of hacks. > We can do better. Since this is part of Benjamin's project, we will make sure that the test_support that emerges from it will be properly documented, stable and usable. Georg From glyph at divmod.com Thu Jun 26 21:32:10 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 26 Jun 2008 19:32:10 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> Message-ID: <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com> I do tend to ramble on, so here's an executive summary of my response: I want python developers to pay attention to the community buildbots and to treat breakages of existing projects as a serious issue. However, I don't think that maintaining those projects is the core team's job, so all I'm asking for is for core developers to: * treat breakages of 3rd party packages as a potentially serious issue, * if possible (i.e. if they find out about the breakage soon enough, which should be the case in any pybots failure) revert the change that caused the problem until the problem can be fixed, and * notify 3rd party maintainers when it's decided that the breakage will not be fixed. This only applies to breakages that the core developers find out about, which for all practical purposes means the ones on the community builders page. Those of you looking for point-by-point responses and some repetition of the above points, enjoy :). On 05:03 pm, guido at python.org wrote: >On Thu, Jun 26, 2008 at 9:21 AM, wrote: >>On 03:33 pm, guido at python.org wrote: >>>It needs to be decided case-by-case. >>(misunderstanding) >No, I just meant that we need to figure out for each 3rd party test >that fails whether the failure is our fault (too incompatibile) or >theirs (relied on undefined behavior) and what the best fix is (change >our code or theirs -- note that even if it's there fault there are >cases where the best fix is to change our code. This is basically fine, as far as I'm concerned. I would like to suggest, however, that these issues be dealt with as soon as possible, rather than waiting for the release process to begin. A lot of decisions are made on this mailing list about the supposed properties of "average" python code, without any actual survey of said code. Sometimes the results of that survey can be really surprising. The end goal of any particular compatibility policy, of a distinction between "public" and "private" APIs, and so on, is to keep code working. >I'm sorry if your interpretation of the terminology is different, but >this is mine and this is what we've always used, and it's not likely >to change. (At least not for the 2.6/3.0 release.) I have no problem with your definitions of these terms. I think that they should probably be in PEP 101 though. Would you accept a patch that added an edited / expanded version of this paragraph? >>Still, I'm bringing this up now because it _is_ a beta, >Absolutely correct. The RCs are hoped to be as good as the final >release. *Now* is the time to bring up issue. Well, that's good, at least :) >But please bring up specific issues -- I don't want to have an >extended discussion about process or quality or expectations. I just >want the code to be fixed. Well, one specific issue has been bumped in priority as a result of this thread, and others are under discussion. The code is getting fixed. >>(I just care that I stop having problems with incompatibility.) > >And here we seem to be parting our ways. We have a large amount of >process already. I don't want more. Looking at it from my perspective, I'm proposing a reduction in process. Under the current process, if a buildbot goes red, the developer makes a judgment call, the release manager makes a judgment call, there's discussion on a ticket, a ticket gets filed, it gets promoted, it gets demoted, the RM forgets to re-promote it... My suggestion is that the process be, simply: if a buildbot (community or otherwise) goes red, the change that broke it gets reverted. No questions asked! It's still there in the revision history, ready to be re-applied once the issues get worked out. Discussion can then take place and case-by-case judgments can be applied. >If you're talking about community buildbots (which I presume are >testing 3rd party packages against the core release) being red, that's >out of scope for the core developers. I don't necessarily think that keeping the community buildbots red is the core developers' responsibility, but I don't think it should be entirely out of scope, either. The python test suite is, frankly, poor - and I hope I'm not surprising you by saying that. It's full of race conditions, tends to fail intermittently, and is frequently ignored. Not only that, but it is quite often changed, so tests for issues that affect real code are quite often removed. So, the community buildbots are not just making sure that 3rd-party code still works, they are an expanded, independently developed test suite to make sure that *python itself* still works. Sometimes they will not fill that role particularly well, but they are worth paying attention to. If python had a good, exhaustive regression test suite that was immutable between major versions, I'd probably feel differently. But that's not the world we live in. Right now, apparently, the *default* policy is that if the community buildbots go red, later, before a release, someone will maybe take a look at it. I'd suggest that the *default* policy ought to be that if a particular changeset breaks a community buildbot, it needs further examination before being merged to trunk. However, this is just the way I prefer to do development; if you think that would slow things down too much, the only thing I'm _really_ asking for is a clear statement that says "there should be no test failures on community buildbots that have not been explicitly accepted before a final release". I'm not even sure what "explicitly accepted" means - you have to sign off? the release manager, maybe? A discussion on this list? I don't really care, as long as _somebody_ does. Right now, my impression of the process is this: * The community buildbot goes red; no core developer looks at it. * If the project is Twisted, JP fixes the bug on Twisted's end. * If the project is Django, nobody notices. * Months later, a beta goes out. A few people try it out and report some bugs, but don't really understand the output. A good number go un- triaged. * A little while later, a final release comes out. Many projects are broken as a result. This is not a hypothetical concern. This is what happened with 2.5; Twisted was broken for months, and Zope *to this day* does not support Python 2.5. 2.6 looks like it's headed for the same trajectory. To be clear: this is with all of Python's _own_ tests passing, so it is specific to paying attention to community buildbots. (And the community buildbots only build django and twisted right now. I'm not talking about a massive pan-galactic survey of all possible python projects. I'm only talking about those popular enough to make this select list. Which should still be a slightly longer list, but I digress...) >Some of the >core buildbots are red because, well, frankly, they run on a cranky >old version of an OS that few people care about. On Twisted, we have a distinction between "supported" and "unsupported" platforms, to provide the ability to run on platforms which aren't really supported and don't really run the whole suite, but we are nevertheless interested in. I don't believe the setup is too hard and we'll definitely help out with that if you want to do it. (I believe Thomas Herve volunteered to do this at PyCon...) >I hope the community buildbots can be used the same way: a red bot >means someone needs to look into an issue. The issue could be with the >core or with the 3rd party package being tested. I don't think a >policy like "no community buildbots should be red" makes any sense. These bots have been red for months. The issues exist, but have not been looked into. As a result, Barry made a specific commitment on a ticket (i.e. "this should block beta 1") which was not met. I think _something_ has to be changed to encourage people to do this more immediately or more seriously. >Whoever made what change? You can't seriously expect core developers >investigating issues with 3rd party packages, no matter what the >cause. The correct process is that someone who cares about the 3rd >party package (could be an end user, a developer of that package, or a >core developer who happens to care) looks into the issue enough to >diagnose it, and then either proposes a fix or files a bug with the >likely culprit, which could be the core or the 3rd party package. If >nobody cares, well, that's open source too. If the breakage is calculated and expected, and the benefits clearly oughtweigh the costs... oh well, too bad for the 3rd party people. It would be nice if the core developers would notify the third party if they find out about it so that it can be verified that the change in question wasn't obscuring some *other* problem, but from what I've seen, the breakages that I have been concerned about have not been intentional, calculated changes, but side-effects of other things. I'm talking about the case where the breakage reveals either a bug in Python, or an unintentional / side-effect change in behavior, which is surprisingly frequent. From glyph at divmod.com Thu Jun 26 21:35:58 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 26 Jun 2008 19:35:58 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <4863CE78.7010108@v.loewis.de> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> <4863CE78.7010108@v.loewis.de> Message-ID: <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com> On 05:14 pm, martin at v.loewis.de wrote: >>I don't know. JP is already addressing the issues affecting Twisted >>in >>another thread (incompatible changes in the private-but-necessary-to- >>get-any-testing-done API of the warnings module). But I really think >>that whoever made the change which broke it should be the one >>investigating it, not me. > >How could they have known that they broke it? Because the relevant community buildbot turned red with that revision of trunk. Keep in mind I'm not talking about every piece of Python code in the universe; just the ones selected for the community buildbots. It would be nice if there were a dozen or so projects on there, but paying attention to the two builders that do actually run right now would be a good start. From g.brandl at gmx.net Thu Jun 26 21:44:22 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Jun 2008 21:44:22 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: glyph at divmod.com schrieb: > Another way to phrase this question is, "whose responsibility is it to > make Python 2.5 programs run on Python 2.6"? Or, "what happens when the > core team finds out that a change they have made has broken some python > software 'in the wild'"? > > Here are a couple of ways to answer these questions: > > 1) It's the python core team's responsibility. There should be Python > buildbots for lots of different projects and they should all be green > before the code can be considered [one of: allowed in trunk, alpha, > beta, RC]. > 2) It's the python core team's responsibility as long as the major > projects are using public APIs. The code should not be considered > [alpha, beta, RC] if there are any known breakages in public APIs. > > 3) It's only the python core team's responsibility to notify projects > of incompatible changes of which they are aware, which may occur in > public or private APIs. Major projects with breakages will be given a > grace period before a [beta, RC, final] to release a forward-compatible > version. > 4) The status quo: every new Python release can (and will, at least > speaking in terms of 2.6) break lots of popular software, with no > warning aside from the beta period. There are no guarantees. Python's backwards compatibility has never extended to non-public APIs. That rules out 1). 2) would be nice, but to be honest, 2) and 3) are unrealistic given the number of Python core developers and the tasks already at hand. 4) is not acceptable either. However, you seem to be overlooking that there's a 3.5) in there: it's the Python core team's responsibility as long as the major projects are using public APIs; but to reduce the workload every project should watch its own buildbots and notify the core team using the issue tracker in order to get the issue resolved. At no time will a policy "the community buildbots must be green" be useful: the tests that run on these buildbots are not under our control, so if the tests test things we deem non-public we can't do anything about it. (And we may have a hard time convincing project authors to change the tests if we promised to make them run anyway.) Georg From g.brandl at gmx.net Thu Jun 26 21:46:48 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Jun 2008 21:46:48 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <4863B900.1040808@gmail.com> <20080626163346.25821.1622910462.divmod.xquotient.10493@joule.divmod.com> Message-ID: glyph at divmod.com schrieb: > On 03:42 pm, ncoghlan at gmail.com wrote: >>glyph at divmod.com wrote: > >>beta 1 has some trouble running *our* test suite - I'd be fairly >>surprised if the community buildbots were in significantly better >>shape. > > That's another problem, yes :) >>>The community buildbots have been in a broken state for months now[1]. > >>If the community buildbots aren't largely green by the time beta 2 >>comes out, that's when I'll agree we have a problem - they should >>definitely be green by the time first release candidate comes out. > > I have mixed feelings about this: on the one hand, if this were a matter > of public record, I would have known that this was too early to start > complaining, and we could have saved everyone a lot of time reading > these messages. That would be good; I would prefer to just let everyone > work without bothering about process. The problem is that not enough people care about the community buildbots, and this isn't likely to improve since we have not enough manpower to do it. > On the other hand, it's much easier to deal with test failures as they > arise than in one giant chunk of work at the end of the development > process. I spoke to a few core developers at PyCon who thought that the > buildbots should always be green and any change that broke them should > be reverted. They may remain nameless unless they wish to raise their > hands :-) but that's definitely how I feel about it. I think no core developer will not tell you that the (core) buildbots should be green all the time. As for reverting changes that break, I'd support this only for changes that break *all* of them. For example, I only use one platform to develop on (and I guess it's the same for many others), having the buildbots go red on another platform means I can try to fix the issue. Georg From hall.jeff at gmail.com Thu Jun 26 21:48:10 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Thu, 26 Jun 2008 15:48:10 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> <4863CE78.7010108@v.loewis.de> <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com> Message-ID: <1bc395c10806261248k42f49d43va4d09a7251ee523a@mail.gmail.com> To me (and I'm an outsider not a direct developer), it's Python developers responsibility to handle the Python problems and the Python build bots. The community build bots are the responsibility of their authors. If JP is handling the Twisted one then great. It's got a gatekeeper. If no one is handling the Django build bot that's not the Python Dev Team's problem but is a problem on the Django side and someone involved in that should be tasked with monitoring. Having said all that, I agree that the community bots ought to at least be looked at. If I check in a patch in and I notice that a community bot went from green to red then I probably should double check my code. The problem is that, as you said, the community bots haven't been well maintained... They've been red for awhile. So asking the developers to then go do the leg work to find the original error, fix it and then back check everything between then and now that might have ALSO caused a problem is alot of effort. It's not the Bank's responsibility to balance my check book. They give me the tools to do it and then I have to check. A similar principle applies here, I believe. -------------- next part -------------- An HTML attachment was scrubbed... URL: From exarkun at divmod.com Thu Jun 26 21:48:41 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 26 Jun 2008 15:48:41 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: Message-ID: <20080626194841.4714.1180228744.divmod.quotient.13515@ohm> On Thu, 26 Jun 2008 21:46:48 +0200, Georg Brandl wrote: > [snip] > >As for reverting changes that break, I'd support this >only for changes that break *all* of them. For example, I only use one >platform to develop on (and I guess it's the same for many others), having >the buildbots go red on another platform means I can try to fix the issue. BuildBot has two ways to let you run your code on all builders before you commit it to trunk. You can force a build on a branch or you can try a build with a patch. I don't know if these options are enabled on Python's buildmaster. If they are, then if you want, you can use them to make sure your code works on all platforms before you put it into trunk, where it may cause problems for someone else. Jean-Paul From python at rcn.com Thu Jun 26 21:52:25 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 26 Jun 2008 12:52:25 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> Message-ID: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> From: "Guido van Rossum" > Let's step back and discuss the API some more. > > - Do we need all three? I think so -- see the the reasons below. Of course, my first choice was not on your list. To me, the one obvious way to convert a number to a eval-able string in a different base is to use bin(), oct(), or hex(). But that appears to be off the table for reasons that I've read but don't make any sense to me. It seems simple enough, extendable enough, and clean enough for bin/oct/hex to use __index__ if present and __float__ if not. > - If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.) I don't think it's user-friendly to have the float-to-bin API fail to parallel the int-to-bin API. IMO, it should be done the same way in both places. I don't find it attractive in appearance. Any use case I can imagine involves multiple calls using the same base and I would likely end-up using functools.partial or somesuch to factor-out the repeated use of the same variable. In particular, it's less usable with a series of numbers at the interactive prompt. That is one of the primary use cases since it allows you to see exactly what is happening with float arithmetic: >>> .6 + .7 1.2999999999999998 >>> bin(.6) '0b10011001100110011001100110011001100110011001100110011 * 2.0 ** -53' >>> bin(.7) '0b1011001100110011001100110011001100110011001100110011 * 2.0 ** -52' >>> bin(.6 + .7) '0b101001100110011001100110011001100110011001100110011 * 2.0 ** -50' >>> bin(1.3) '0b10100110011001100110011001100110011001100110011001101 * 2.0 ** -52' Or checking whether a number is exactly representable: >>> bin(3.375) '0b11011 * 2.0 ** -3' Both of those bits of analysis become awkward with the tobase() method: >>> (.6).tobase(2) ... > - What should the output format be? I know you originally favored > 0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex > builtins, the constraint that it needs to be an eval() able expression > may be dropped (unless you see a use case for that too). The other guys convinced me that round tripping was important and that there is a good use case for being able to read/write precisely specified floats in a platform independent manner. Also, my original idea didn't scale well without exponential notation -- i.e. bin(125E-100) would have a heckofa lot of leading zeroes. Terry and Mark also pointed-out that the hex with exponential notation was the normal notation used in papers on floating point arithmetic. Lastly, once I changed over to the new way, it dramatically simplified the implementation. Raymond From martin at v.loewis.de Thu Jun 26 21:55:40 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 21:55:40 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <85CA47A2BC4A41668B88C68AD3C86062@RaymondLaptop1> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> Message-ID: <4863F43C.2080904@v.loewis.de> >> So as far as the feature design goes, I offer some suggestions: a new >> module; or a new function in math; or a new method on float. Since >> Raymond is the champion for the feature let him choose the API from >> those alternatives. > > I choose bin/hex/oct methods on floatobjects. > Will work-up a patch. I think the feature is misguided in the first place. Why do you want a hex representation of floating point numbers? Can't you use struct.pack for that? And, if bin/hex/oct are useful, why not base 6 (say)? Regards, Martin From martin at v.loewis.de Thu Jun 26 22:00:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 22:00:30 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com> Message-ID: <4863F55E.4010200@v.loewis.de> > I want python developers to pay attention to the community buildbots I don't think that goal is realistic. Instead, somebody who has actual interest in this matter should pay this attention, and then bring it up on python-dev when something breaks. Regards, Martin From tjreedy at udel.edu Thu Jun 26 22:02:19 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Jun 2008 16:02:19 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: glyph at divmod.com wrote: > to what extent should Python actually be compatible between releases? As I understand things from years of observation, the following are fair game to changed in ways possibly backward-incompatible for specific code: bugs, detailed float behavior (which may be system-specific anyway), error messages, private interfaces, non-enforcement of documented rules, and defined implementation-dependent behavior. But Guido has been somewhat conservative about this (least so for bug fixes, I think) and sometimes put off changes even when the fault lies with questionable usages. One fly in the ointment is that bugs (a discrepancy between doc and code) may be fixed either by changing the doc or the code, according to which embodies the design intention. So neither can be taken as gospel. From steve at holdenweb.com Thu Jun 26 22:04:27 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 26 Jun 2008 16:04:27 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: <4863F64B.9000609@holdenweb.com> Georg Brandl wrote: > glyph at divmod.com schrieb: > >> Another way to phrase this question is, "whose responsibility is it to >> make Python 2.5 programs run on Python 2.6"? Or, "what happens when >> the core team finds out that a change they have made has broken some >> python software 'in the wild'"? >> >> Here are a couple of ways to answer these questions: >> >> 1) It's the python core team's responsibility. There should be >> Python buildbots for lots of different projects and they should all be >> green before the code can be considered [one of: allowed in trunk, >> alpha, beta, RC]. >> 2) It's the python core team's responsibility as long as the major >> projects are using public APIs. The code should not be considered >> [alpha, beta, RC] if there are any known breakages in public APIs. > > >> 3) It's only the python core team's responsibility to notify >> projects of incompatible changes of which they are aware, which may >> occur in public or private APIs. Major projects with breakages will >> be given a grace period before a [beta, RC, final] to release a >> forward-compatible version. >> 4) The status quo: every new Python release can (and will, at least >> speaking in terms of 2.6) break lots of popular software, with no >> warning aside from the beta period. There are no guarantees. > > Python's backwards compatibility has never extended to non-public APIs. > That rules out 1). 2) would be nice, but to be honest, 2) and 3) are > unrealistic given the number of Python core developers and the tasks > already at hand. 4) is not acceptable either. > > However, you seem to be overlooking that there's a 3.5) in there: it's > the Python core team's responsibility as long as the major projects are > using public APIs; but to reduce the workload every project should watch > its own buildbots and notify the core team using the issue tracker in > order to get the issue resolved. > > At no time will a policy "the community buildbots must be green" be > useful: the tests that run on these buildbots are not under our control, > so if the tests test things we deem non-public we can't do anything > about it. (And we may have a hard time convincing project authors to > change the tests if we promised to make them run anyway.) > If, however, the community buildbot goes red because of a regression or backwards incompatibility the developers should, when notified, at least undertake to verify that the breakage is intentional. Unfortunately having a buildbot is like installing a firewall. Some people assume that the very presence of the buildbot is a guard against version incompatibilities, whereas in truth there is always the messy business of communication required to ensure the information gets to where it is useful. The bottom line, of course, is that if project X can't be bothered to tell the core development team when changes break their build it's nobody's fault but their own. If the report it and the core developers ignore the reports, that's something that should be rectified. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu Jun 26 22:04:27 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 26 Jun 2008 16:04:27 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: <4863F64B.9000609@holdenweb.com> Georg Brandl wrote: > glyph at divmod.com schrieb: > >> Another way to phrase this question is, "whose responsibility is it to >> make Python 2.5 programs run on Python 2.6"? Or, "what happens when >> the core team finds out that a change they have made has broken some >> python software 'in the wild'"? >> >> Here are a couple of ways to answer these questions: >> >> 1) It's the python core team's responsibility. There should be >> Python buildbots for lots of different projects and they should all be >> green before the code can be considered [one of: allowed in trunk, >> alpha, beta, RC]. >> 2) It's the python core team's responsibility as long as the major >> projects are using public APIs. The code should not be considered >> [alpha, beta, RC] if there are any known breakages in public APIs. > > >> 3) It's only the python core team's responsibility to notify >> projects of incompatible changes of which they are aware, which may >> occur in public or private APIs. Major projects with breakages will >> be given a grace period before a [beta, RC, final] to release a >> forward-compatible version. >> 4) The status quo: every new Python release can (and will, at least >> speaking in terms of 2.6) break lots of popular software, with no >> warning aside from the beta period. There are no guarantees. > > Python's backwards compatibility has never extended to non-public APIs. > That rules out 1). 2) would be nice, but to be honest, 2) and 3) are > unrealistic given the number of Python core developers and the tasks > already at hand. 4) is not acceptable either. > > However, you seem to be overlooking that there's a 3.5) in there: it's > the Python core team's responsibility as long as the major projects are > using public APIs; but to reduce the workload every project should watch > its own buildbots and notify the core team using the issue tracker in > order to get the issue resolved. > > At no time will a policy "the community buildbots must be green" be > useful: the tests that run on these buildbots are not under our control, > so if the tests test things we deem non-public we can't do anything > about it. (And we may have a hard time convincing project authors to > change the tests if we promised to make them run anyway.) > If, however, the community buildbot goes red because of a regression or backwards incompatibility the developers should, when notified, at least undertake to verify that the breakage is intentional. Unfortunately having a buildbot is like installing a firewall. Some people assume that the very presence of the buildbot is a guard against version incompatibilities, whereas in truth there is always the messy business of communication required to ensure the information gets to where it is useful. The bottom line, of course, is that if project X can't be bothered to tell the core development team when changes break their build it's nobody's fault but their own. If the report it and the core developers ignore the reports, that's something that should be rectified. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From martin at v.loewis.de Thu Jun 26 22:05:24 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 22:05:24 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626194841.4714.1180228744.divmod.quotient.13515@ohm> References: <20080626194841.4714.1180228744.divmod.quotient.13515@ohm> Message-ID: <4863F684.9050505@v.loewis.de> > BuildBot has two ways to let you run your code on all builders before you > commit it to trunk. You can force a build on a branch or you can try a > build with a patch. I don't know if these options are enabled on Python's > buildmaster. If they are, then if you want, you can use them to make sure > your code works on all platforms before you put it into trunk, where it may > cause problems for someone else. Even if that's possible, it adds a burden. As likely not all people agree that such a procedure would be a good thing, it's necessary that somebody establishes a policy. Guido has already indicated that he is not interested in further policies; Barry (the canonical other person to set policies) has indicated in the past that he would like such a policy, but deems it unrealistic at this point. He gives other procedural changes higher priority, so that realistically, such a policy might not be established within the next two years. Regards, Martin From glyph at divmod.com Thu Jun 26 22:06:38 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 26 Jun 2008 20:06:38 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> On 07:44 pm, g.brandl at gmx.net wrote: >At no time will a policy "the community buildbots must be green" be >useful: the tests that run on these buildbots are not under our >control, >so if the tests test things we deem non-public we can't do anything >about it. (And we may have a hard time convincing project authors to >change the tests if we promised to make them run anyway.) That's not what I'm suggesting. If there is a legitimate disagreement between Python developers and developers of a project about whether an API should continue to be supported, then clearly, the Python developers get to win. Welcome to open source. However, I believe that the core team is not paying attention to other projects breaking. I'm not saying that you want to make breaking changes, or that bug reports are not dealt with, but the problem is that dealing with these problems after the fact makes it _much_ more painful. When you get to the release, and you have 30 bug reports due to other projects breaking, they get triaged, some get left in, and some features of lots of different projects are left broken. And many projects do not bother to test with the beta. If developers were simply presented with the results of their changes immediately ("you broke twisted, django doesn't import, zope segfaults and mercurial destroys your repository") then perhaps they would weigh the benefits of compatibility differently, and do the trivial, obvious fix immediately, rather than waiting 6 months and having to figure out what the heck change caused the bug. I accept the fact that python core development is done differently than i.e. Java development, and some backward compatibility may simply be broken. Case in point: changes to the warnings module being discussed in a separate thread break a significant number of Twisted's tests, because they remove functionality which is not public but is required to test code that uses the warnings module. Would Brett have taken this into account if he knew about it immediately when revision 62303 was committed? Maybe not, but now that the code is written and done, there's significantly less motivation to go back and fix it. At the time it might have only been a few minutes' work to continue supporting the use-case which Twisted needs. Brett wouldn't even have necessarily needed to do the work: it would have been easier to convince a Twisted developer to do the work it to keep the community buildbot green rather than to make it a bit less red. Now, though, it's much easier to say "oh well, that's private, you lose". I don't ascribe this to malice - it really *would* be much harder to fix it now, for us as well as for him. From musiccomposition at gmail.com Thu Jun 26 22:09:11 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 26 Jun 2008 15:09:11 -0500 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> <4863A2C2.8000202@gmail.com> <4863B678.5040205@gmail.com> Message-ID: <1afaf6160806261309o3d346901jfc38fbda223ee9cb@mail.gmail.com> On Thu, Jun 26, 2008 at 10:34 AM, Guido van Rossum wrote: > On Thu, Jun 26, 2008 at 8:32 AM, Nick Coghlan wrote: >> I'm personally fine with that approach, but some of the new items in there >> for 2.6 could probably use a bit of cleaning up to improve the APIs and/or >> the way they work. > > So get crackin'! I'll add to my list. :) Anyway, if we are going to make test.support public for real why don't we take Nick's suggestion to rename the test package to _test, but move test.support to a toplevel position like testutils or such. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From dickinsm at gmail.com Thu Jun 26 22:17:48 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 21:17:48 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <4863F43C.2080904@v.loewis.de> References: <20080620041816.4D5E81E4002@bag.python.org> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> Message-ID: <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> On Thu, Jun 26, 2008 at 8:55 PM, "Martin v. L?wis" wrote: > > I think the feature is misguided in the first place. Why do you want > a hex representation of floating point numbers? Answering for myself: because it gives an exact representation of a floating-point number in a fairly human-readable format. > Can't you use struct.pack for that? struct.pack would only show the bit layout, leaving the user to manually extract the sign bit, exponent, and fraction, and then make sense of the whole thing. The proposed feature works at a higher level of abstraction, directly describing the *value* of the float rather than its bit layout. In particular, this means it'll make sense across platforms, regardless of variations in bit layout. And, if bin/hex/oct are useful, why not base > 6 (say)? > I'd say that bin and hex are special: bin is natural because floats are usually thought of, and stored as, binary numbers. hex is special because it gives a compact way of representing a float, and because there's already a history of using hex floats in numerical analysis literature and in programming languages (C99, Java, ...) I have to admit that I can't see much use for octal floats. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Jun 26 22:21:49 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 13:21:49 -0700 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> <20080625130857.GA13663@steerpike.home.puzzling.org> <20080626054916.GA24366@steerpike.home.puzzling.org> <4863A007.60804@gmail.com> Message-ID: On Thu, Jun 26, 2008 at 12:33 PM, Georg Brandl wrote: > Guido van Rossum schrieb: > >>>> Ok, then we're back to there being no supported way to write tests that >>>> need to >>>> intercept warnings. Twisted has already suffered from this (JP reports >>>> that >>>> Twisted's assertWarns is broken in 2.6), and I doubt it's alone. >>>> >>>> So I guess I am filing a bug after all... :) >>> >>> Yeah - Brett's correct that everything under "test.test_support" should >>> really be formally undocumented. It's mostly a place for code that >>> reflects >>> "things we do a lot in our unit tests and are tired of repeating" rather >>> than "this is a good API that we want to support forever and encourage >>> other >>> people to use". >> >> I still have a big problem with this attitude. If it's valuable enough >> to share between our tests, it should be properly written for reuse >> and documented. A collection of hacks remains a collection of hacks. >> We can do better. > > Since this is part of Benjamin's project, we will make sure that the > test_support that emerges from it will be properly documented, stable > and usable. I'm assuming however that Benjamin's project will land in 2.7 or 3.1, right? If he's going to be refactoring tests all summer there's no way that can be merged into the trunk and the py3k branch right before the final release. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Jun 26 22:23:19 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 22:23:19 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> Message-ID: <4863FAB7.6040200@v.loewis.de> > I don't ascribe this to malice - > it really *would* be much harder to fix it now, for us as well as for him. I think I disagree. It's easier to fix it now than it was to fix it back then. Fixing it back then would have meant to constantly observe the buildbots, and keep them running, so it would be a huge effort to maintain the infrastructure just to find a the few changes that unintentially broke something. Looking at them now is a lot of effort, also. But this effort is feasible, once the root cause is identified, the patch causing it might just get backed out. Maintaining the community buildbots has proven infeasible. It's unfortunate that many package authors don't understand that not all breakage is deliberate, and that their only chance to get undesired breakage reverted is to report bugs NOW. Regards, Martin From guido at python.org Thu Jun 26 22:28:20 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 13:28:20 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> Message-ID: On Thu, Jun 26, 2008 at 1:17 PM, Mark Dickinson wrote: > I'd say that bin and hex are special: bin is natural because > floats are usually thought of, and stored as, binary numbers. > hex is special because it gives a compact way of representing > a float, and because there's already a history of using hex > floats in numerical analysis literature and in programming > languages (C99, Java, ...) Can you show us what APIs and output formats C99 and Java support? Maybe we can borrow something from there rather than reinventing the wheel? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Jun 26 22:30:51 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 26 Jun 2008 22:30:51 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> Message-ID: <4863FC7B.6070903@v.loewis.de> > > I think the feature is misguided in the first place. Why do you want > a hex representation of floating point numbers? > > > Answering for myself: because it gives an exact representation of a > floating-point number in a fairly human-readable format. Ok. But py> binascii.hexlify(struct.pack("d", 3.14)) '1f85eb51b81e0940' does that already, no? You won't know the precise value, but you won't know that with hex support, either. > struct.pack would only show the bit layout, leaving the user to > manually extract the sign bit, exponent, and fraction, and then make > sense of the whole thing. I'd question that the user is able to make sense of a number when mantissa and exponent is represented in hex. > I'd say that bin and hex are special: bin is natural because > floats are usually thought of, and stored as, binary numbers. > hex is special because it gives a compact way of representing > a float, and because there's already a history of using hex > floats in numerical analysis literature and in programming > languages (C99, Java, ...) Then I'd argue that the feature should be symmetric: If there is support for printing floating point numbers as hex, there should also be support for hex floating point literals. Also, to follow C's tradition, it would be better if that was *not* integrated into the hex function (or a hex method), but if there was support for %a in string formatting. Regards, Martin From python at rcn.com Thu Jun 26 22:31:13 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 26 Jun 2008 13:31:13 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> Message-ID: [Mark Dickinson] > I have to admit that I can't see much use for octal floats. Neither do I. They look weird to me. Raymond From g.brandl at gmx.net Thu Jun 26 22:36:12 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Jun 2008 22:36:12 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> Message-ID: glyph at divmod.com schrieb: > On 07:44 pm, g.brandl at gmx.net wrote: >>At no time will a policy "the community buildbots must be green" be >>useful: the tests that run on these buildbots are not under our >>control, >>so if the tests test things we deem non-public we can't do anything >>about it. (And we may have a hard time convincing project authors to >>change the tests if we promised to make them run anyway.) > > That's not what I'm suggesting. Sure, but I wanted to say that nevertheless :) > If there is a legitimate disagreement between Python developers and > developers of a project about whether an API should continue to be > supported, then clearly, the Python developers get to win. Welcome to > open source. > > However, I believe that the core team is not paying attention to other > projects breaking. Quite true. It is their duty to bring the breakage to our attention, if they want to see results. > I'm not saying that you want to make breaking > changes, or that bug reports are not dealt with, but the problem is that > dealing with these problems after the fact makes it _much_ more painful. > When you get to the release, and you have 30 bug reports due to other > projects breaking, they get triaged, some get left in, and some features > of lots of different projects are left broken. And many projects do not > bother to test with the beta. I thought we're talking about the projects that *do* use the buildbots? > If developers were simply presented with the results of their changes > immediately ("you broke twisted, django doesn't import, zope segfaults > and mercurial destroys your repository") then perhaps they would weigh > the benefits of compatibility differently, and do the trivial, obvious > fix immediately, rather than waiting 6 months and having to figure out > what the heck change caused the bug. This sounds very nice in theory, but it must be implemented in a way that does not add a burden to the developer, as Martin said. > Case in point: changes to the warnings module being discussed in a > separate thread break a significant number of Twisted's tests, because > they remove functionality which is not public but is required to test > code that uses the warnings module. Would Brett have taken this into > account if he knew about it immediately when revision 62303 was > committed? Maybe not, but now that the code is written and done, > there's significantly less motivation to go back and fix it. At the > time it might have only been a few minutes' work to continue supporting > the use-case which Twisted needs. Brett wouldn't even have necessarily > needed to do the work: it would have been easier to convince a Twisted > developer to do the work it to keep the community buildbot green rather > than to make it a bit less red. Now, though, it's much easier to say > "oh well, that's private, you lose". I don't ascribe this to malice - > it really *would* be much harder to fix it now, for us as well as for > him. I must say that I'm increasingly surprised by this discussion since it seems that not only the Python core devs neglect the community buildbots. Shouldn't the project authors have at least the same interest that their code runs on the next major Python version? And while they don't necessarily have more time to watch the buildbots than we have, the amount of what they need to keep an eye on. Georg From musiccomposition at gmail.com Thu Jun 26 22:38:22 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 26 Jun 2008 15:38:22 -0500 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: References: <20080625022455.4714.1019244056.divmod.quotient.12903@ohm> <48621CC6.2030801@gmail.com> <20080625130857.GA13663@steerpike.home.puzzling.org> <20080626054916.GA24366@steerpike.home.puzzling.org> <4863A007.60804@gmail.com> Message-ID: <1afaf6160806261338o769ace1ag6530632bf3e61b94@mail.gmail.com> On Thu, Jun 26, 2008 at 3:21 PM, Guido van Rossum wrote: > On Thu, Jun 26, 2008 at 12:33 PM, Georg Brandl wrote: >> >> Since this is part of Benjamin's project, we will make sure that the >> test_support that emerges from it will be properly documented, stable >> and usable. > > I'm assuming however that Benjamin's project will land in 2.7 or 3.1, > right? If he's going to be refactoring tests all summer there's no way > that can be merged into the trunk and the py3k branch right before the > final release. Good timing! Georg and I were just talking about this. You're correct; I've never imagined having it in before 2.6 or 3.0 final. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From g.brandl at gmx.net Thu Jun 26 22:39:00 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Jun 2008 22:39:00 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> Message-ID: Terry Reedy schrieb: > > glyph at divmod.com wrote: > >> to what extent should Python actually be compatible between releases? > > As I understand things from years of observation, the following are fair > game to changed in ways possibly backward-incompatible for specific > code: bugs, detailed float behavior (which may be system-specific > anyway), error messages, private interfaces, non-enforcement of > documented rules, and defined implementation-dependent behavior. But > Guido has been somewhat conservative about this (least so for bug fixes, > I think) and sometimes put off changes even when the fault lies with > questionable usages. Which is the source of many unresolved open issues in the tracker -- everyone agrees that "it" is a bug and should be fixed, perhaps there's even a patch, but people may be using it this way, so nothing is done forever. (And these bugs usually are the kind of things we don't want to change in 3.0 either since they are subtle things.) However, I don't have a solution here, so I won't complain about it anymore. Georg From guido at python.org Thu Jun 26 22:41:50 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 13:41:50 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> Message-ID: On Thu, Jun 26, 2008 at 12:52 PM, Raymond Hettinger wrote: > From: "Guido van Rossum" >> >> Let's step back and discuss the API some more. >> >> - Do we need all three? > > I think so -- see the the reasons below. Sounds like Mark Dickinson only cares about bin and hex. > Of course, my first choice was not > on your list. To me, the one obvious way to convert a number to a eval-able > string in a different base is to use bin(), oct(), or hex(). But that > appears to be off the table for reasons that I've read but don't make any > sense to me. > It seems simple enough, extendable enough, and clean enough > for bin/oct/hex to use __index__ if present and __float__ if not. That's not extendable to types that aren't int or float though. And it would accept Decimal instances which seems a really odd thing to do. >> - If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.) > > I don't think it's user-friendly to have the float-to-bin API > fail to parallel the int-to-bin API. IMO, it should be done > the same way in both places. Consistency only goes so far. We have 0b, 0o and 0x notations for integers, and the bin/oct/hex builtins are meant to invert those. We don't have base-{2,8,16} literals for floats. > I don't find it attractive in appearance. Any use case I can > imagine involves multiple calls using the same base and I would likely > end-up using functools.partial or somesuch > to factor-out the repeated use of the same variable. In particular, > it's less usable with a series of numbers at the interactive prompt. That is > one of the primary use cases since it allows you to see > exactly what is happening with float arithmetic: > >>>> .6 + .7 > > 1.2999999999999998 >>>> >>>> bin(.6) > > '0b10011001100110011001100110011001100110011001100110011 * 2.0 ** -53' >>>> >>>> bin(.7) > > '0b1011001100110011001100110011001100110011001100110011 * 2.0 ** -52' >>>> >>>> bin(.6 + .7) > > '0b101001100110011001100110011001100110011001100110011 * 2.0 ** -50' >>>> >>>> bin(1.3) > > '0b10100110011001100110011001100110011001100110011001101 * 2.0 ** -52' > > Or checking whether a number is exactly representable: > >>>> bin(3.375) > > '0b11011 * 2.0 ** -3' > > Both of those bits of analysis become awkward with the tobase() method: > >>>> (.6).tobase(2) You don't need the parentheses around .6. I think much fewer than 0.01% of Python users will ever need this. It's a one-liner helper function if you prefer to say bin(x) instead of x.bin(). >> - What should the output format be? I know you originally favored >> 0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex >> builtins, the constraint that it needs to be an eval() able expression >> may be dropped (unless you see a use case for that too). > > The other guys convinced me that round tripping was important > and that there is a good use case for being able to read/write > precisely specified floats in a platform independent manner. Can you summarize those reasons? Who are the users of that feature? I'm still baffled why a feature whose only users are extreme experts needs to have such a prominent treatment. Surely there are a lot more Python users who call urlopen() or urlparse() all day long. Should these be built-in functions then? > Also, my original idea didn't scale well without exponential > notation -- i.e. bin(125E-100) would have a heckofa lot > of leading zeroes. Terry and Mark also pointed-out that > the hex with exponential notation was the normal notation > used in papers on floating point arithmetic. Lastly, once I > changed over to the new way, it dramatically simplified the > implementation. I agree that you need to have a notation using an exponent. If it weren't for the roundtripping, I'd probably have preferred something which simply showed me the bits of the IEEE floating point number broken out into mantissa and exponent -- that seems more educational to me than normalizing things so that the last bit is nonzero. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Thu Jun 26 22:55:54 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 21:55:54 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> Message-ID: <5c6f2a5d0806261355j71041a0dvca7672605c66f74d@mail.gmail.com> On Thu, Jun 26, 2008 at 9:28 PM, Guido van Rossum wrote: > Can you show us what APIs and output formats C99 and Java support? > Maybe we can borrow something from there rather than reinventing the > wheel? > Java's toHexString method is documented at: http://java.sun.com/javase/6/docs/api/java/lang/Double.html#toHexString(double) It's disadvantage from Python's point of view is that some features are IEEE 754 specific (e.g. treatment of subnormals, which don't exist for most other floating point types). C99s support for hex literals uses a similar format; the standard is less specific about the precise output format, but it's still of the form 0x1.p Incidentally, the funny 'p' for the exponent instead of 'e' is apparently there to avoid ambiguity in something like: 0x1e+3 Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Thu Jun 26 22:46:05 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 21:46:05 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <4863FC7B.6070903@v.loewis.de> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> Message-ID: <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> On Thu, Jun 26, 2008 at 9:30 PM, "Martin v. L?wis" wrote: > > Answering for myself: because it gives an exact representation of a > > floating-point number in a fairly human-readable format. > > Ok. But > > py> binascii.hexlify(struct.pack("d", 3.14)) > '1f85eb51b81e0940' > > does that already, no? You won't know the precise value, but you won't > know that with hex support, either. > The output from hex_float(3.14) would be something like: '0x1.91eb851eb851fp+1' The exponent is still usually given in decimal; there's no need for it to be hexadecimal for exactness. I'd question that the user is able to make sense of a number when > mantissa and exponent is represented in hex. > I think the above it still a bit easier to understand than if one has to figure out where the sign/exponent and exponent/fraction bit boundaries are, unbias the exponent, and add the extra hidden '1' bit into the mantissa. That's a lot of mental work. > Then I'd argue that the feature should be symmetric: If there is > support for printing floating point numbers as hex, there should > also be support for hex floating point literals. > I agree with this. Or at least support for hex floating point strings, if not literals. > > Also, to follow C's tradition, it would be better if that was > *not* integrated into the hex function (or a hex method), but > if there was support for %a in string formatting. > I'd be delighted with '%a' support. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Thu Jun 26 23:06:03 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 26 Jun 2008 17:06:03 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <4863B900.1040808@gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <4863B900.1040808@gmail.com> Message-ID: <3B30AF7E-42D9-4D90-AB99-5A675094135F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 26, 2008, at 11:42 AM, Nick Coghlan wrote: > If the community buildbots aren't largely green by the time beta 2 > comes out, that's when I'll agree we have a problem - they should > definitely be green by the time first release candidate comes out. Just FTR, I'll be looking at the Python buildbots and Roundup tracker to decide whether I think beta2 is in shape to be released or not. I definitely won't be tracking the non-core buildbots, so if something troublesome comes up there, you will have to submit a bug report to the Python tracker. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGQEu3EjvBPtnXfVAQIp6QP/bOU+8vImlRkfKCXazYhi3yEXcNwCPpWI e3AoiOlgKbOhSPraNTyUBjC+OjuqHQttyMJz8TTpexSwhpG/erDIqUjRbkW0Yaas attvFZBYbJhF3qvHyLzmSp9U3oXY6VyWayL8ZdzIGcukEW7g8DgQNQyoGOE4kEeX i6+4RIRf+7A= =X1lh -----END PGP SIGNATURE----- From barry at python.org Thu Jun 26 23:10:41 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 26 Jun 2008 17:10:41 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> Message-ID: <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 26, 2008, at 12:54 PM, glyph at divmod.com wrote: > On 04:42 pm, schmir at gmail.com wrote: >> On Thu, Jun 26, 2008 at 5:33 PM, Guido van Rossum >> wrote: >>> an explanation about *why* Django cannot even be imported than a >>> blanket complaint that this is a disgrace. So why is it? > >> and already discussed: >> http://mail.python.org/pipermail/python-dev/2008-April/078421.html > > Following that trail of breadcrumbs, I ended up here: > > http://bugs.python.org/issue2235 > > with a message from some guy named "Barry Warsaw" (anyone know him?) > that says: > > """ > Guido, can you comment on Amaury's latest patch? I'm going to bump > this > back to critical so as not to hold up 2.6 alpha, but it should be > marked > as a release blocker for the first beta. > """ > > I don't know if this "Barry" guy has the appropriate permissions on > the bugtracker to increase priorities, so I've taken the liberty of > upgrading it as a release blocker for the _second_ beta ... ;-). > So, at least there's been one productive consequence of this > discussion. Glyph, you did the right thing. I wish there was a way of marking a bug "not-release-blocker-for-now" and have it automatically update back to blocker at a certain time or after a certain event. I think it's valid for this issue to block beta2. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGQF0XEjvBPtnXfVAQJwiQP/T9f33lNtAQ9/eooKEyVCDms7NXJE7mIf QPOQtXuTZdsfUl51OxkxewVj6ERZasHPwIB2c13HYuHRrMrmrE9EYStdbmMxh0BI 7EhsO4SfDI3ZnYFJvAEMrTvgY8Vz4v817LhzWNZ7RWxq/yOHG8C/ZgubPJIa8mnd EGWUVoLg77E= =OJHh -----END PGP SIGNATURE----- From guido at python.org Thu Jun 26 23:17:02 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 14:17:02 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> <4863CE78.7010108@v.loewis.de> <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com> Message-ID: On Thu, Jun 26, 2008 at 12:35 PM, wrote: > > On 05:14 pm, martin at v.loewis.de wrote: >>> >>> I don't know. JP is already addressing the issues affecting Twisted in >>> another thread (incompatible changes in the private-but-necessary-to- >>> get-any-testing-done API of the warnings module). But I really think >>> that whoever made the change which broke it should be the one >>> investigating it, not me. >> >> How could they have known that they broke it? > > Because the relevant community buildbot turned red with that revision of > trunk. Keep in mind I'm not talking about every piece of Python code in the > universe; just the ones selected for the community buildbots. It would be > nice if there were a dozen or so projects on there, but paying attention to > the two builders that do actually run right now would be a good start. Sorry, this is an unacceptable burden on the core developers. The core developers aren't going to look at the community buildbots (unless voluntarily) and they aren't going to roll back changes just because some community buildbot goes red. In general someone outside the core developer group needs to bring the issue to the core developers' attention and then a fix will be created if appropriate. Rollbacks are generally reserved for accidental checkins or checkins against the process rules (e.g. during a code freeze). Heck, we don't even roll back if one of our own buildbots goes red. I'm fine with requesting that the core developers pay serious attention to reports about 3rd party code being broken. The community buildbots are a fine tool to find out about this. But any policy that requires an automatic rollback because a buildbot (community or core) goes red is unacceptable. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Thu Jun 26 23:20:49 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 26 Jun 2008 23:20:49 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org> Message-ID: Barry Warsaw schrieb: >> I don't know if this "Barry" guy has the appropriate permissions on >> the bugtracker to increase priorities, so I've taken the liberty of >> upgrading it as a release blocker for the _second_ beta ... ;-). >> So, at least there's been one productive consequence of this >> discussion. > > Glyph, you did the right thing. I wish there was a way of marking a > bug "not-release-blocker-for-now" and have it automatically update > back to blocker at a certain time or after a certain event. > > I think it's valid for this issue to block beta2. I just added a "deferred blocker" priority -- that implements one half of your wish. Mass issue updating must be done by someone who knows Roundup better than me, I'm afraid. Georg From guido at python.org Thu Jun 26 23:24:52 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 14:24:52 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> Message-ID: On Thu, Jun 26, 2008 at 1:06 PM, wrote: > On 07:44 pm, g.brandl at gmx.net wrote: >> >> At no time will a policy "the community buildbots must be green" be >> useful: the tests that run on these buildbots are not under our control, >> so if the tests test things we deem non-public we can't do anything >> about it. (And we may have a hard time convincing project authors to >> change the tests if we promised to make them run anyway.) > > That's not what I'm suggesting. > > If there is a legitimate disagreement between Python developers and > developers of a project about whether an API should continue to be > supported, then clearly, the Python developers get to win. Welcome to open > source. > > However, I believe that the core team is not paying attention to other > projects breaking. I'm not saying that you want to make breaking changes, > or that bug reports are not dealt with, but the problem is that dealing with > these problems after the fact makes it _much_ more painful. When you get to > the release, and you have 30 bug reports due to other projects breaking, > they get triaged, some get left in, and some features of lots of different > projects are left broken. And many projects do not bother to test with the > beta. Well, sorry, that's life. We're not going to deal with breakage in 3rd party code on a "drop all other work" basis. You seem to have a different idea about how to do development than the core developers. Tough luck. You can't tell us how to do our work. We're doing the best we can, and we're happy to listen to suggestions you're making, but the set of suggestions you're making in this thread are an unacceptable burden, and it's not going to happen. > If developers were simply presented with the results of their changes > immediately ("you broke twisted, django doesn't import, zope segfaults and > mercurial destroys your repository") then perhaps they would weigh the > benefits of compatibility differently, and do the trivial, obvious fix > immediately, rather than waiting 6 months and having to figure out what the > heck change caused the bug. I accept the fact that python core development > is done differently than i.e. Java development, and some backward > compatibility may simply be broken. > > Case in point: changes to the warnings module being discussed in a separate > thread break a significant number of Twisted's tests, because they remove > functionality which is not public but is required to test code that uses the > warnings module. Would Brett have taken this into account if he knew about > it immediately when revision 62303 was committed? Maybe not, but now that > the code is written and done, there's significantly less motivation to go > back and fix it. I disagree. It's broken and should be fixed. Beta 1 just came out so this is the perfect time to file a bug. > At the time it might have only been a few minutes' work to > continue supporting the use-case which Twisted needs. Brett wouldn't even > have necessarily needed to do the work: it would have been easier to > convince a Twisted developer to do the work it to keep the community > buildbot green rather than to make it a bit less red. Why? That sounds like it's a problem with the psychology of the Twisted developers. > Now, though, it's > much easier to say "oh well, that's private, you lose". I don't ascribe > this to malice - it really *would* be much harder to fix it now, for us as > well as for him. I don't believe this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Thu Jun 26 23:26:42 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 22:26:42 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261355j71041a0dvca7672605c66f74d@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <5c6f2a5d0806261355j71041a0dvca7672605c66f74d@mail.gmail.com> Message-ID: <5c6f2a5d0806261426x71cd35ffm8a07574d8f09be00@mail.gmail.com> On Thu, Jun 26, 2008 at 9:55 PM, Mark Dickinson wrote: > > It's disadvantage from Python's point of view is that some features are IEEE 754 Aargh! I can't believe I wrote that. Its. Its. Its. Anyway; some more detail: Both C99 and Java 1.5/1.6 support hex floating-point literals; both in exactly the same format, as far as I can tell. Here are the relevant productions from the Java grammar: HexDigit: one of 0 1 2 3 4 5 6 7 8 9 a b c d e f A B C D E F HexNumeral: 0 x HexDigits 0 X HexDigits HexDigits: HexDigit HexDigit HexDigits HexadecimalFloatingPointLiteral: HexSignificand BinaryExponent FloatTypeSuffix_opt HexSignificand: HexNumeral HexNumeral . 0x HexDigits_opt . HexDigits 0X HexDigits_opt . HexDigits BinaryExponent: BinaryExponentIndicator SignedInteger BinaryExponentIndicator:one of p P Java's 'Double' class has a 'toHexString' method that outputs a valid hex floating point string, and the Double() constructor also accepts such strings. C99 also appears to have full support for input/output of hex floats; e.g. using strtod and printf('%a', ...). Not sure how helpful this is. Mark From guido at python.org Thu Jun 26 23:28:56 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 26 Jun 2008 14:28:56 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> Message-ID: On Thu, Jun 26, 2008 at 1:46 PM, Mark Dickinson wrote: > I'd be delighted with '%a' support. Remind me what %a does? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Jun 26 23:30:43 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 26 Jun 2008 14:30:43 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org><4863AA71.5020901@gmail.com><328D5F1E-E082-4002-9A65-38A31625C161@python.org><4863B400.4090401@gmail.com><4863F43C.2080904@v.loewis.de><5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com><4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> Message-ID: <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> [MvL] >> Then I'd argue that the feature should be symmetric: >> If there is support for printing floating point numbers >> as hex, there should also be support for hex floating >> point literals. [Mark] > I agree with this. Or at least support for hex floating point > strings, if not literals. ISTM, that the currently proposed output format gives us this benefit for free (no changes to the parser). The format is already close to the C99 notation but replaces the 'p' with '* 2.0 **' which I find to be both readable and self-explanatory. Raymond From martin at v.loewis.de Thu Jun 26 23:35:44 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Thu, 26 Jun 2008 23:35:44 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> Message-ID: <48640BB0.9080304@v.loewis.de> > I think the above it still a bit easier to understand than > if one has to figure out where the sign/exponent and > exponent/fraction bit boundaries are, unbias the exponent, > and add the extra hidden '1' bit into the mantissa. That's > a lot of mental work. Sure. However, I'd argue that most people are unable to even remotely guess the number in decimal when presented with the number in hexadecimal - at a minimum, they'll fail to recognize that the exponent is not to the power of 10, or when they realize that, might guess that it is to the power of 16 (I find it fairly confusing, but consequential, that it is to the power of 2, even when the digits are hex - and then, the exponent is decimal :-). So I'd like to dismiss any objective of "the output must be human-understandable" as unrealistic. That an unambiguous representation is desired, I can understand - but only if there also is a way to enter the same representation elsewhere. In addition, I fail to see the point in binary representation. For unambiguous representation, it's sufficient to use hex. I can't accept claims that people will be actually able to understand what number is represented when given the bit string. For educational purposes, decoding mantissa and biased exponent directly out of the IEEE representation is better than having binary output builtin. > Also, to follow C's tradition, it would be better if that was > *not* integrated into the hex function (or a hex method), but > if there was support for %a in string formatting. > > > I'd be delighted with '%a' support. I personally find that much less problematic than extending the hex, and wouldn't object to a patch providing such formatting (assuming it does the same thing that C99 printf does). Regards, Martin From barry at python.org Thu Jun 26 23:43:55 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 26 Jun 2008 17:43:55 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 26, 2008, at 5:20 PM, Georg Brandl wrote: > Barry Warsaw schrieb: > >>> I don't know if this "Barry" guy has the appropriate permissions >>> on the bugtracker to increase priorities, so I've taken the >>> liberty of upgrading it as a release blocker for the _second_ >>> beta ... ;-). So, at least there's been one productive >>> consequence of this discussion. >> Glyph, you did the right thing. I wish there was a way of marking >> a bug "not-release-blocker-for-now" and have it automatically >> update back to blocker at a certain time or after a certain event. >> I think it's valid for this issue to block beta2. > > I just added a "deferred blocker" priority -- that implements one half > of your wish. Mass issue updating must be done by someone who knows > Roundup better than me, I'm afraid. Cool, thanks! I should have thought of that before. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGQNm3EjvBPtnXfVAQIy1wP/QPFwklmnay8VPPovHA6H4XA6tW+ziWPV hbeyAZX/MMxEYv/98Z4nOQU7M4AczlXHZks80UvDdKLwPe2wTwfOIgmCTeb+vciI 20GpxrHpWQNHy/XKeawEBxyLHJZ4qNGIAFmHwoiUusM6erTeVb9kWa0czG31En6r Z4MV0YKfu+A= =Zxh4 -----END PGP SIGNATURE----- From martin at v.loewis.de Thu Jun 26 23:47:31 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 23:47:31 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> Message-ID: <48640E73.5070109@v.loewis.de> Guido van Rossum wrote: > On Thu, Jun 26, 2008 at 1:46 PM, Mark Dickinson wrote: >> I'd be delighted with '%a' support. > > Remind me what %a does? It's a C99 feature. From the spec (7.19.6.1p8) a,A A double argument representing a floating-point number is converted in the style [-]0xh.hhhhp?d, where there is one hexadecimal digit (which is nonzero if the argument is a normalized floating- point number and is otherwise unspecified) before the decimal-point character235) and the number of hexadecimal digits after it is equal to the precision; if the precision is missing and FLT_RADIX is a power of 2, then the precision is sufficient for an exact representation of the value; if the precision is missing and FLT_RADIX is not a power of 2, then the precision is sufficient to distinguish236) values of type double, except that trailing zeros may be omitted; if the precision is zero and the # flag is not specified, no decimal- point character appears. The letters abcdef are used for a conversion and the letters ABCDEF for A conversion. The A conversion specifier produces a number with X and P instead of x and p. The exponent always contains at least one digit, and only as many more digits as necessary to represent the decimal exponent of 2. If the value is zero, the exponent is zero. A double argument representing an infinity or NaN is converted in the style of an f or F conversion specifier. Footnotes 235) Binary implementations can choose the hexadecimal digit to the left of the decimal-point character so that subsequent digits align to nibble (4-bit) boundaries. 236) The precision p is sufficient to distinguish values of the source type if 16p-1>bn where b is FLT_RADIX and n is the number of base-b digits in the significand of the source type. A smaller p might suffice depending on the implementation's scheme for determining the digit to the left of the decimal-point character. This is symmetric with C99's hexadecimal floating point literals: hexadecimal-floating-constant: hexadecimal-prefix hexadecimal-fractional-constant binary-exponent-part floating-suffix-opt hexadecimal-prefix hexadecimal-digit-sequence binary-exponent-part floating-suffix-opt hexadecimal-fractional-constant: hexadecimal-digit-sequence-opt . hexadecimal-digit-sequence hexadecimal-digit-sequence . binary-exponent-part: p sign-opt digit-sequence P sign-opt digit-sequence hexadecimal-digit-sequence: hexadecimal-digit hexadecimal-digit-sequence hexadecimal-digit scanf and strtod support the same format. Regards, Martin From dickinsm at gmail.com Thu Jun 26 23:50:10 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 22:50:10 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> Message-ID: <5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com> On Thu, Jun 26, 2008 at 10:28 PM, Guido van Rossum wrote: > Remind me what %a does? From the C99 standard (section 7.19.6.1): A double argument representing a ?oating-point number is converted in the style [?]0xh.hhhhp?d, [...] From dickinsm at gmail.com Thu Jun 26 23:53:18 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 22:53:18 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> Message-ID: <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> On Thu, Jun 26, 2008 at 10:30 PM, Raymond Hettinger wrote: > ISTM, that the currently proposed output format gives > us this benefit for free (no changes to the parser). The format is already > close to the C99 notation but replaces the 'p' with '* 2.0 **' which I find > to > be both readable and self-explanatory. There's one other major difference between the C99 notation and the current patch: the C99 notation includes a (hexa)decimal point. The advantages of this include: - the exponent gives a rough idea of the magnitude of the number, and - the exponent doesn't vary with changes to the least significant bits of the float. The disadvantage is the loss of evalability. (Is that a word?) Mark From martin at v.loewis.de Thu Jun 26 23:54:17 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 23:54:17 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org> Message-ID: <48641009.4080105@v.loewis.de> > I just added a "deferred blocker" priority -- that implements one half > of your wish. Mass issue updating must be done by someone who knows > Roundup better than me, I'm afraid. I doubt true mass update will be necessary. If you remind me that I should convert all "deferred blocker" issues to some other priority, I'll do that manually in a matter of minutes (using a query to find all issues with that priority). Regards, Martin From g.brandl at gmx.net Fri Jun 27 00:00:13 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 27 Jun 2008 00:00:13 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com> Message-ID: Mark Dickinson schrieb: > On Thu, Jun 26, 2008 at 10:28 PM, Guido van Rossum wrote: >> Remind me what %a does? > > From the C99 standard (section 7.19.6.1): > > A double argument representing a ?oating-point number is converted in the > style [?]0xh.hhhhp?d, [...] Let me remind you that %a currently means "call ascii()" in 3.0. Georg From martin at v.loewis.de Thu Jun 26 23:57:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 26 Jun 2008 23:57:32 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> Message-ID: <486410CC.2030906@v.loewis.de> > The disadvantage is the loss of evalability. (Is that a word?) Until the parser has support for it, having a float class method, or even the float callable itself for conversion seems reasonable. If repr() didn't produce it, eval() doesn't need to understand it. Regards, Martin From tjreedy at udel.edu Fri Jun 27 00:00:23 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 26 Jun 2008 18:00:23 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > From: "Guido van Rossum" >> Let's step back and discuss the API some more. >> >> - Do we need all three? > > I think so -- see the the reasons below. I would prefer 1, see below. > Of course, my first choice was > not on your list. To me, the one obvious way to convert a number to a > eval-able string in a different base is to use bin(), oct(), or hex(). > But that appears to be off the table for reasons that I've read but > don't make any sense to me. Let me try. I am one of those who prefer smaller to bigger for the core language to make it easier to learn and teach. But, to me, there deeper consideration that applies here. A Python interpreter, human or mechanical, must do exact integer arithmetic. But a Python interpreter does not have to convert float literals to fixed size binary and does *not* have to do float arithmetic with binary presentations that are usually approximations. (Indeed, human interpreters do neither, which is why they are often surprised at CPython's float output, and which is why this function will be useful.) If built-in functions are part of the language definition, as Guido just clarified, their definition and justification should not depend on the float implementation. > It seems simple enough, extendable enough, and clean enough > for bin/oct/hex to use __index__ if present and __float__ if not. To me, a binary representation, in whatever base, of a Decimal is senseless. The point of this issue is to reveal the exact binary bit pattern of float instances. >> - If so, why not .tobase(N)? (Even if N is restricted to 2, 8 and 16.) > > I don't think it's user-friendly to have the float-to-bin API > fail to parallel the int-to-bin API. IMO, it should be done > the same way in both places. I would like to turn this around. I think that 3 nearly identical built-ins is 2 too many. I am going to propose on the Py3 list that bin, oct, and hex be condensed to one function, bin(integer, base=2,8,or16), for 3.1 if not 3.0. Base 8 and 16 are, to me, compressed binary. Three methods is definitely too many for a somewhat subsidiary function. So, I would like to see float.bin([base=2]) > I don't find it attractive in appearance. Any use case I can > imagine involves multiple calls using the same base and I would likely > end-up using functools.partial or somesuch > to factor-out the repeated use of the same variable. Make the base that naive users want to see the default. I believe this to be 2. Numerical analysts who want base 16 can deal with partial if they really have scattered calls (as opposes to a few within loops) and cannot deal with typing '16' over and over. >>>> bin(.6) > '0b10011001100110011001100110011001100110011001100110011 * 2.0**-53' ... > Both of those bits of analysis become awkward with the tobase() method: >>>> (.6).tobase(2) Eliminate the unneeded parentheses and default value, and this is >>> .6.bin() which is just one extra char. >> - What should the output format be? I know you originally favored >> 0b10101.010101 etc. Now that it's not overloaded on the bin/oct/hex >> builtins, the constraint that it needs to be an eval() able expression >> may be dropped (unless you see a use case for that too). > > The other guys convinced me that round tripping was important > and that there is a good use case for being able to read/write > precisely specified floats in a platform independent manner. Definitely. The paper I referenced in the issue discussion, http://bugs.python.org/issue3008 mentioned a few times here, is http://hal.archives-ouvertes.fr/docs/00/28/14/29/PDF/floating-point-article.pdf > Also, my original idea didn't scale well without exponential > notation -- i.e. bin(125E-100) would have a heckofa lot > of leading zeroes. Terry and Mark also pointed-out that > the hex with exponential notation was the normal notation > used in papers on floating point arithmetic. Lastly, once I > changed over to the new way, it dramatically simplified the > implementation. I originally thought I preferred the 'hexponential' notation that uses P for power instead of E for exponential. But with multiple bases, the redundancy of repeating the bases is ok, and being able to eval() without changing the parser is a plus. But I would prefer losing the spaces around the ** operator. Terry Jan Reedy From dickinsm at gmail.com Fri Jun 27 00:01:47 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 23:01:47 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com> Message-ID: <5c6f2a5d0806261501j4ff3c5eeqcf5d4197ebe26a44@mail.gmail.com> On Thu, Jun 26, 2008 at 11:00 PM, Georg Brandl wrote: > Let me remind you that %a currently means "call ascii()" in 3.0. Oh well. That's out then. I'll rephrase to "I'd be delighted with something similar in spirit to '%a' support." :-) Mark From brett at python.org Fri Jun 27 00:11:25 2008 From: brett at python.org (Brett Cannon) Date: Thu, 26 Jun 2008 15:11:25 -0700 Subject: [Python-Dev] Py3k DeprecationWarning in stdlib In-Reply-To: <20080626160118.4714.38891548.divmod.quotient.13435@ohm> References: <4863BB32.3040101@gmail.com> <20080626160118.4714.38891548.divmod.quotient.13435@ohm> Message-ID: On Thu, Jun 26, 2008 at 9:01 AM, Jean-Paul Calderone wrote: > On Fri, 27 Jun 2008 01:52:18 +1000, Nick Coghlan wrote: >> >> Jean-Paul Calderone wrote: >>> >>> I don't particularly care about the details, I just want some public >>> API for this. Making warn_explicit public seems better to me, since >>> it was already there in previous versions of Python, and it lets you >>> completely ignore both the filters list and the global registry, but >>> if others would rather make the filters and global registry a part of >>> the public API, that's fine by me as well. >> >> Why do you say warn_explicit isn't public? It's in both the 2.5 and 2.6 >> API docs for the warnings module. > > Brett told me it was private (on this list several weeks or a month or so > ago). It's also no longer called in 2.6 by the C implementation of the > warning system. > I don't remember saying it is private, just that you can't replace the function in the module and expect everything to continue to work properly like you can with showwarning(). -Brett From dickinsm at gmail.com Fri Jun 27 00:11:35 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 26 Jun 2008 23:11:35 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> Message-ID: <5c6f2a5d0806261511x11c1398cp4d3067950139430e@mail.gmail.com> On Thu, Jun 26, 2008 at 11:00 PM, Terry Reedy wrote: > Definitely. The paper I referenced in the issue discussion, > http://bugs.python.org/issue3008 mentioned a few times here, is > http://hal.archives-ouvertes.fr/docs/00/28/14/29/PDF/floating-point-article.pdf Perhaps it's worth reproducing the most relevant paragraph of that paper (the end of section 2.1) here: """Conversion to and from decimal representation is delicate; special care must be taken in order not to introduce inaccuracies or discrepan- cies. [Steele and White, 1990, Clinger, 1990]. Because of this, C99 introduces hexadecimal floating-point literals in source code. [ISO, 1999, ?6.4.4.2] Their syntax is as follows: 0xmmmmmm.mmmmp?ee where mmmmmm.mmmm is a mantissa in hexadecimal, possibly containing a point, and ee is an exponent, written in deci- mal, possibly preceded by a sign. They are interpreted as [mmmmmm.mmmm]16?2ee . Hexadecimal floating-point representations are especially important when val- ues must be represented exactly, for reproducible results ? for instance, for testing "borderline cases" in algorithms. For this reason, we shall use them in this paper wherever it is important to specify exact values. See also Section 4.4 for more information on inputting and outputting floating-point values.""" From schmir at gmail.com Fri Jun 27 00:22:41 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Fri, 27 Jun 2008 00:22:41 +0200 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com> Message-ID: <932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com> On Thu, Jun 26, 2008 at 8:45 PM, Guido van Rossum wrote: > > So you're saying py.test needs to be fixed? Fine with me, but then I > don't understand why you bothered bringing it up here instead of > fixing it (or reporting to the py.test maintainers, I don't know if > you're one or not). > no, I'm not. Just wanted to point out, that this ticket/patch does not solve the django issue. (this is what I first thought it would do). - Ralf From eric+python-dev at trueblade.com Fri Jun 27 00:37:01 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 26 Jun 2008 18:37:01 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806261501j4ff3c5eeqcf5d4197ebe26a44@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <5c6f2a5d0806261450v711dcc52m6346b42d060707ba@mail.gmail.com> <5c6f2a5d0806261501j4ff3c5eeqcf5d4197ebe26a44@mail.gmail.com> Message-ID: <48641A0D.6020203@trueblade.com> Mark Dickinson wrote: > On Thu, Jun 26, 2008 at 11:00 PM, Georg Brandl wrote: >> Let me remind you that %a currently means "call ascii()" in 3.0. > > Oh well. That's out then. I'll rephrase to "I'd be delighted with something > similar in spirit to '%a' support." :-) It could be added to str.format(). Well, actually float.__format__(). Not that I'm advocating it, but it's a place it would fit, and since it's restricted to the float format specifier, it would at least be well contained. And other types are free to implement it, or not. From barry at python.org Fri Jun 27 00:34:20 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 26 Jun 2008 18:34:20 -0400 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <48641009.4080105@v.loewis.de> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <9CE97649-0FD1-4EC1-B681-CF3B32A79BCB@python.org> <48641009.4080105@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jun 26, 2008, at 5:54 PM, Martin v. L?wis wrote: >> I just added a "deferred blocker" priority -- that implements one >> half >> of your wish. Mass issue updating must be done by someone who knows >> Roundup better than me, I'm afraid. > > I doubt true mass update will be necessary. If you remind me that I > should convert all "deferred blocker" issues to some other priority, > I'll do that manually in a matter of minutes (using a query to find > all issues with that priority). Martin, I think that will work great. The idea being that after each release, we always bump deferred blockers back to release blocker. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSGQZbHEjvBPtnXfVAQIwWQQAqWoIoqV0S0O8ZpkXpWd3LO4Fo7MBfjRT LL5QgIPBWdQdZ4RR68LSfdAhiHGnZCZorpNksGaeIxP55qgS1z31fy3JF39UUbVR 3ojymtF6Is0qCB5lmkJIx9Na/2RUEOUWTQoQP3dG851oRRuSVTXb4uRaBlNlQErY oN9JxG5xQR0= =z9nN -----END PGP SIGNATURE----- From steve at pearwood.info Fri Jun 27 01:34:57 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 27 Jun 2008 09:34:57 +1000 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> Message-ID: <200806270934.57783.steve@pearwood.info> On Fri, 27 Jun 2008 07:30:43 am Raymond Hettinger wrote: > The format is already close to the C99 notation > but replaces the 'p' with '* 2.0 **' which I find to > be both readable and self-explanatory. Since we're talking about what's "readable and self-explanatory", I find that jarring, unexpected, unintuitive, and mathematically bizarre (even if it is the convention in some areas). It's like writing '123 * A.0 ** -2' for 1.23. And putting spaces around the operators is ugly. I'd like to mention that what bin() et al is actually doing is not so much returning a binary number string but returning a hybrid binary/decimal arithmetic expression. So bin() returns a binary number string for int arguments, and an expression for float arguments: these are conceptually different kinds of things, even if they're both strings. Frankly, I'd be much happier if the API (whatever it is) returned a tuple of (binary string, base int, exponent int), and let users write their own helper function to format it any way they like. Or failing that, the p notation used by Java and C99. (And yes, mixing decimal exponents with binary mantissas upsets me too, but somehow it's less upsetting.) -- Steven From greg.ewing at canterbury.ac.nz Fri Jun 27 02:11:00 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Jun 2008 12:11:00 +1200 Subject: [Python-Dev] C API for gc.enable() and gc.disable() In-Reply-To: <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com> References: <1214075410.5874.26.camel@fsol> <485D5CFC.5020705@v.loewis.de> <1f7befae0806221410h41d61f0fm74e00451f929d125@mail.gmail.com> <485F1ADA.6090107@v.loewis.de> <486010CD.2000904@v.loewis.de> <1bc395c10806251256sdda632eg6c7d87fd237812b@mail.gmail.com> <4862B0C4.2080901@v.loewis.de> <1bc395c10806251408w700f0791y3474fcb1f639ee6e@mail.gmail.com> Message-ID: <48643014.5020008@canterbury.ac.nz> Jeff Hall wrote: > I mistakenly thought that was > because they were assumed to be small. It sounds like they're ignored > because they're automatically collected and so they SHOULD be ignored > for object garbage collection. Strings aren't tracked by the cyclic garbage collector because they don't contain object references and therefore can't form part of a cycle. However, unless I'm mistaken, allocations and deallocations of them are still counted for the purpose of determining when to perform a cyclic GC pass. So if you allocate lots of strings and they aren't getting deallocated, a cyclic GC pass will eventually occur, in case the strings are being referenced from a cycle that can be cleaned up. I don't know whether/how re uses string objects internally while it's matching, so I can't say what its garbage collection characteristics might be when matching against a huge string. The behaviour you observed might have been due to the nature of the re being matched -- some res can have quadratic or exponential behaviour all by themselves. -- Greg From glyph at divmod.com Fri Jun 27 02:33:56 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 27 Jun 2008 00:33:56 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> <4863CE78.7010108@v.loewis.de> <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com> Message-ID: <20080627003356.25821.685560775.divmod.xquotient.10844@joule.divmod.com> On 09:17 pm, guido at python.org wrote: >On Thu, Jun 26, 2008 at 12:35 PM, wrote: >>Because the relevant community buildbot turned red with that revision >>of >>trunk. Keep in mind I'm not talking about every piece of Python code >>in the >>universe; just the ones selected for the community buildbots. It >>would be >>nice if there were a dozen or so projects on there, but paying >>attention to >>the two builders that do actually run right now would be a good start. > >Sorry, this is an unacceptable burden on the core developers. The core >developers aren't going to look at the community buildbots (unless >voluntarily) and they aren't going to roll back changes just because >some community buildbot goes red. OK, fair enough. Taking a step back, I was pushing this really hard because to *me*, it seems like dealing with the influx of bug reports after the fact is an unreasonable amount of additional work, whereas immediate reverts are just an occasional, temporary inconvenience. Based on my experience, "We'll deal with the reports as they come in" sounds like "no". >In general someone outside the core developer group needs to bring the >issue to the core developers' attention and then a fix will be created >if appropriate. Rollbacks are generally reserved for accidental >checkins or checkins against the process rules (e.g. during a code >freeze). Heck, we don't even roll back if one of our own buildbots >goes red. I think since the last time we had a similar conversation, other Twisted developers have been fairly diligent in reporting bugs. (In the time they've been reporting Python bugs, I've mostly been planning a wedding. Others who have survived it tell me that eventually, this process ends... and then I should be participating more directly.) I'll try to step up that feeback loop and get other projects involved. I hope I am wrong about generating an unreasonable amount of work. >I'm fine with requesting that the core developers pay serious >attention to reports about 3rd party code being broken. The community >buildbots are a fine tool to find out about this. But any policy that >requires an automatic rollback because a buildbot (community or core) >goes red is unacceptable. Thanks. I appreciate it; an increased emphasis on 3rd party code being broken is really what I was looking for. This is fine with me. I mean, whatever your decision is, I'm going to have to live with it :), but in either case, we have to be looking for bugs and helping to investigate them. On my end of things I guess it's not going to make much difference. From rhamph at gmail.com Fri Jun 27 02:53:15 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 26 Jun 2008 18:53:15 -0600 Subject: [Python-Dev] GC Proposal Message-ID: We need two counters: one is the total number of traceable objects (those we would inspect if we did a full collection) and a number of "pending" trace operations. Every time an object is moved into the last generation, we increase "pending" by two - once for itself and once for an older object. Once pending equals the total number of traceable objects we do a full collection (and reset "pending" to 0). This behaves well under both extremes. First, if allocating in a burst, it waits until the total number of objects has doubled before doing a collection, which works out to about 2 passes per object. Second, a long running program may allocate an object, get it into the oldest generation (adding to "pending"), but then delete it through normal means. The pending trace is effectively borrowed by the remaining older objects, which ensures they will get scanned again eventually. However, we already payed for it when the young object was allocated, so the net cost is still a constant factor. Various refinements are possible, such as only adding 1.5 to "pending", or taking off some of the 2 if a young object is deleted without getting traced. We could also tweak when in the object life-cycle "pending" is increased. The first two generations would remain as they are today. -- Adam Olsen, aka Rhamphoryncus From greg.ewing at canterbury.ac.nz Fri Jun 27 03:11:27 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 27 Jun 2008 13:11:27 +1200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <2F11C59B1E294F87ADA5F36A70143739@RaymondLaptop1> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> Message-ID: <48643E3F.3010500@canterbury.ac.nz> Raymond Hettinger wrote: > To me, the one obvious way to convert a number to a > eval-able string in a different base is to use bin(), oct(), or hex(). What use cases are there for an eval-able representation of a float in those bases, as opposed to a human-readable one? -- Greg From glyph at divmod.com Fri Jun 27 03:13:15 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 27 Jun 2008 01:13:15 -0000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626200638.25821.1050616648.divmod.xquotient.10680@joule.divmod.com> Message-ID: <20080627011315.25821.242932439.divmod.xquotient.10918@joule.divmod.com> On 26 Jun, 09:24 pm, guido at python.org wrote: >On Thu, Jun 26, 2008 at 1:06 PM, wrote: >>On 07:44 pm, g.brandl at gmx.net wrote: >Well, sorry, that's life. We're not going to deal with breakage in 3rd >party code on a "drop all other work" basis. For the record, "automatic revert" does not mean "drop all other work". The changeset's still there. It's still in the revision history. It can easily be applied to anybody's working copy. It can easily be put into a branch. It can be automatically re-merged later. I do all of these things all the time, and I was *not* intending to suggest that any 3rd-party breakage should cause a code freeze or anything even remotely like that. >>Case in point: changes to the warnings module >I disagree. It's broken and should be fixed. Beta 1 just came out so >this is the perfect time to file a bug. I'll go back over the recent conversation and work out the specifics of the bug (if JP doesn't, or hasn't already beaten me to it). >>(...) it would have been easier to >>convince a Twisted developer to do the work it to keep the community >>buildbot green rather than to make it a bit less red. > >Why? That sounds like it's a problem with the psychology of the >Twisted developers. I hardly think it's unique to us. TDD test runners typically only know 2 colors and 2 states: "passed" and "fails". Once you're in the "fail" state, you tend to accumulate more failures; there's a much bigger bang for your buck. Better tools with nicer interfaces would let you easily mark individual tests as "usually intermittent" or something and make it a "slightly dirty green" or something, but we don't have those. The move from "red" to "green" is much more psychologically significant to just about anyone I know than the move from "red but 14 failures" to "red but 12 failures". Despite the idea's origins in a now-highly-controversial book on criminology, this is often referred to in XP discussion circles (wikis and the like) as the "fix broken windows" collaboration pattern. I notice this in lots of other areas of my life; a little pile of papers tends to become a big pile of papers, the first dent in a car makes the driver a little less careful, and so on. From bender at physik.rwth-aachen.de Fri Jun 27 12:14:03 2008 From: bender at physik.rwth-aachen.de (Walter Bender) Date: Fri, 27 Jun 2008 12:14:03 +0200 Subject: [Python-Dev] deepcopy Message-ID: <4864BD6B.1030203@physik.rwth-aachen.de> Hello togehter, I get an error while doing a deep copy of an "event". An event is an object, with has a lot of __dict__["attr"] = Event("SAMPLE") OTHER Events added to the object (no self refence) and store additional __dict__["attr2"] = 2 informations. deep copy is failing with no proper error message. nevent = copy.deepcopy(event); File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 174, in deepcopy y = copier(x, memo) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 305, in _deepcopy_inst state = deepcopy(state, memo) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 174, in deepcopy y = copier(x, memo) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 268, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 174, in deepcopy y = copier(x, memo) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 241, in _deepcopy_list y.append(deepcopy(a, memo)) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 174, in deepcopy y = copier(x, memo) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 241, in _deepcopy_list y.append(deepcopy(a, memo)) File "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", line 185, in deepcopy y = copier(x, memo) TypeError: __deepcopy__() takes no arguments (1 given cheers, Walter -- Walter Bender bender at physik.rwth-aachen.de RWTH Aachen III. Physikalisches Institut A Tel. +49-241-8027286 From ncoghlan at gmail.com Fri Jun 27 13:00:35 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 21:00:35 +1000 Subject: [Python-Dev] Undocumenting test.support in 3.x (was Py3k DeprecationWarning in stdlib) In-Reply-To: <1afaf6160806261309o3d346901jfc38fbda223ee9cb@mail.gmail.com> References: <1afaf6160806251300h1b686c2ar13a9cca8016653e1@mail.gmail.com> <1afaf6160806251542h451baee3ja3a1ef535d8c027c@mail.gmail.com> <4863A2C2.8000202@gmail.com> <4863B678.5040205@gmail.com> <1afaf6160806261309o3d346901jfc38fbda223ee9cb@mail.gmail.com> Message-ID: <4864C853.9040305@gmail.com> Benjamin Peterson wrote: > On Thu, Jun 26, 2008 at 10:34 AM, Guido van Rossum wrote: >> On Thu, Jun 26, 2008 at 8:32 AM, Nick Coghlan wrote: >>> I'm personally fine with that approach, but some of the new items in there >>> for 2.6 could probably use a bit of cleaning up to improve the APIs and/or >>> the way they work. >> So get crackin'! > > I'll add to my list. :) > > Anyway, if we are going to make test.support public for real why don't > we take Nick's suggestion to rename the test package to _test, but > move test.support to a toplevel position like testutils or such. Leaving the actual test modules undocumented (which is the situation now) is fine in my opinion. The _test idea was just for if we wanted to document test_support for our own benefit without promising to keep it stable for later releases. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri Jun 27 13:18:42 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Jun 2008 21:18:42 +1000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> <20080626193210.25821.328171947.divmod.xquotient.10633@joule.divmod.com> Message-ID: <4864CC92.2000405@gmail.com> glyph at divmod.com wrote: > > I do tend to ramble on, so here's an executive summary of my response: > > I want python developers to pay attention to the community buildbots and > to treat breakages of existing projects as a serious issue. Counter-proposal: * Interested developers or users of the major third party projects tested by the community buildbots should monitor the community buildbots, and start filing bug reports with the appropriate party as soon as the upcoming Python release hits 2.Xa1 (i.e. first alpha). * If the failure is due to an incompatibility between Python 2.X and 2.X-1, then the bug report should be filed against Python. While these issues may or may not be addressed before the first beta, they *must* be addressed before the first release candidate. * If the failure is due to an incompatibility between Python 2.X and 2.X-2 that was properly deprecated in 2.X-1, then the bug report should be filed against the third party project. Prioritising these is a question for the developers of that project. * Before filing a bug report against Python for a community buildbot failure, check if the relevant regression is also causing failures of the core buildbots. If it is, skip the bug report until the core buildbots are passing again. It's currently a fact of life that we do NOT keep the trunk in an always-releasable state. We just don't. It might be nice if we did, there are lots of reasons why that's a good way to run a project, but at this point in time it isn't the case with Python. Reacting every time a community buildbot goes red would be a serious waste of effort. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From amauryfa at gmail.com Fri Jun 27 13:56:54 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 27 Jun 2008 13:56:54 +0200 Subject: [Python-Dev] deepcopy In-Reply-To: <4864BD6B.1030203@physik.rwth-aachen.de> References: <4864BD6B.1030203@physik.rwth-aachen.de> Message-ID: Walter Bender wrote: Hello togehter, > > I get an error while doing a deep copy of an "event". An event is an object, > with has a lot of __dict__["attr"] = Event("SAMPLE") OTHER Events added to > the object (no self refence) and store additional __dict__["attr2"] = 2 > informations. > > deep copy is failing with no proper error message. > > > nevent = copy.deepcopy(event); > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 174, in deepcopy > y = copier(x, memo) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 305, in _deepcopy_inst > state = deepcopy(state, memo) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 174, in deepcopy > y = copier(x, memo) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 268, in _deepcopy_dict > y[deepcopy(key, memo)] = deepcopy(value, memo) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 174, in deepcopy > y = copier(x, memo) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 241, in _deepcopy_list > y.append(deepcopy(a, memo)) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 174, in deepcopy > y = copier(x, memo) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 241, in _deepcopy_list > y.append(deepcopy(a, memo)) > File > "/net/software_cms/slc4_ia32_gcc345/external/python/2.4.2-CMS3q/lib/python2.4/copy.py", > line 185, in deepcopy > y = copier(x, memo) > TypeError: __deepcopy__() takes no arguments (1 given It's possible that "event" contains an object that cannot be copied. >From the traceback, one can see that your object has an attribute that is a list of lists, and the failing object is there. Googling for the error message suggests that the object could be an array.array. The corresponding problem was corrected with python 2.4.4. -- Amaury Forgeot d'Arc From theller at ctypes.org Fri Jun 27 14:45:31 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 27 Jun 2008 14:45:31 +0200 Subject: [Python-Dev] Vacation Message-ID: FYI: I'm going to a vacation for the next two or three weeks. I will shutdown my buildbots, and restart them when I'm back: x86 osx.5, x86 XP-3, amd64 XP. Sorry for the inconvenience, Thomas From chris at atlee.ca Fri Jun 27 15:42:35 2008 From: chris at atlee.ca (Chris AtLee) Date: Fri, 27 Jun 2008 09:42:35 -0400 Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads Message-ID: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> Hello, I notice that there is some work being done on urllib / urllib2 for python 2.6/3.0. One thing I've always missed in urllib/urllib2 is the facility to encode POST data as multipart/form-data. I think it would also be useful to be able to stream a POST request to the remote server rather than having requiring the user to create the entire POST body in memory before starting the request. This would be extremely useful when writing any kind of code that does file uploads. I didn't see any recent discussion about this so I thought I'd ask here: do you think this would make a good addition to the new urllib package? Cheers, Chris From janssen at parc.com Fri Jun 27 17:40:40 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 27 Jun 2008 08:40:40 PDT Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> Message-ID: <08Jun27.084042pdt."58698"@synergy1.parc.xerox.com> > I notice that there is some work being done on urllib / urllib2 for > python 2.6/3.0. One thing I've always missed in urllib/urllib2 is the > facility to encode POST data as multipart/form-data. I think it would > also be useful to be able to stream a POST request to the remote > server rather than having requiring the user to create the entire POST > body in memory before starting the request. This would be extremely > useful when writing any kind of code that does file uploads. > > I didn't see any recent discussion about this so I thought I'd ask > here: do you think this would make a good addition to the new urllib > package? I think it would be very helpful. I'd separate the two things, though; you want to be able to format a set of values as "multipart/form-data", and do various things with that resulting "document", and you want to be able to stream a POST (or PUT) request. Bill From status at bugs.python.org Fri Jun 27 18:06:27 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 27 Jun 2008 18:06:27 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080627160627.863E678116@psf.upfronthosting.co.za> ACTIVITY SUMMARY (06/20/08 - 06/27/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1933 open (+40) / 13106 closed (+25) / 15039 total (+65) Open issues with patches: 600 Average duration of open issues: 707 days. Median duration of open issues: 1532 days. Open Issues Breakdown open 1908 (+39) pending 25 ( +1) Issues Created Or Reopened (69) _______________________________ Move Demo/classes/Rat.py to Lib/fractions.py and fix it up. 06/25/08 http://bugs.python.org/issue1682 reopened marketdickinson patch pickling of large recursive structures crashes cPickle 06/25/08 http://bugs.python.org/issue2702 reopened facundobatista patch Let bin/oct/hex show floats 06/25/08 http://bugs.python.org/issue3008 reopened rhettinger patch ast module docs document private attribute 06/20/08 CLOSED http://bugs.python.org/issue3152 created brett.cannon easy, 26backport sqlite leaks on error 06/20/08 http://bugs.python.org/issue3153 created Rhamphoryncus "Quick search" box renders too long on FireFox 3 06/20/08 http://bugs.python.org/issue3154 created tebeka Python should expose a pthread_cond_timedwait API for threading 06/20/08 http://bugs.python.org/issue3155 created gregory.p.smith bytes type has inconsistent methods (append, insert) 06/20/08 http://bugs.python.org/issue3156 created akuchling patch sqlite3 minor documentation issues 06/21/08 CLOSED http://bugs.python.org/issue3157 created slewis Doctest fails to find doctests in extension modules 06/21/08 http://bugs.python.org/issue3158 created fer_perez glob.py improvements 06/21/08 http://bugs.python.org/issue3159 created Indy patch Building a Win32 binary installer crashes 06/21/08 http://bugs.python.org/issue3160 created complex Missing import for sys in _abcoll 06/21/08 CLOSED http://bugs.python.org/issue3161 created hodgestar patch ssl.SSLSocket implements methods that are overriden by socket.so 06/21/08 http://bugs.python.org/issue3162 created hodgestar module struct support for ssize_t and size_t 06/21/08 http://bugs.python.org/issue3163 created MrJean1 patch cPickle calls to save_string and save_unicode with unicode objec 06/21/08 CLOSED http://bugs.python.org/issue3164 created cuerty patch cPickle recursion problem 06/21/08 CLOSED http://bugs.python.org/issue3165 created cuerty patch Make conversions from long to float correctly rounded. 06/21/08 http://bugs.python.org/issue3166 created marketdickinson patch math test fails on Solaris 10 06/21/08 http://bugs.python.org/issue3167 created MrJean1 cmath test fails on Solaris 10 06/21/08 http://bugs.python.org/issue3168 created MrJean1 email/header.py doesn't handle Base64 headers that have been ins 06/21/08 http://bugs.python.org/issue3169 created jasonjwwilliams test_pydoc has no way to regenerate pristine data 06/22/08 http://bugs.python.org/issue3170 created brett.cannon operator.*slice() should be deprecated in 2.6 06/22/08 http://bugs.python.org/issue3171 created scoder test macos skipped "unexpected" on linux2 06/22/08 CLOSED http://bugs.python.org/issue3172 created MrJean1 external strftime for Python? 06/23/08 http://bugs.python.org/issue3173 created skip.montanaro patch, patch 3.0b1 doesn't seem to install on macs 06/23/08 http://bugs.python.org/issue3174 created erickt multiprocessing build fails on Fedora 8 and Xubuntu 8 + solution 06/23/08 CLOSED http://bugs.python.org/issue3175 created mark result error for 2/5 and 4/5 06/23/08 CLOSED http://bugs.python.org/issue3176 created hiweed implement os.startfile on posix and MacOSX 06/23/08 http://bugs.python.org/issue3177 created ganadist patch __radd__(self,other) isn't called if self and other are of the s 06/23/08 CLOSED http://bugs.python.org/issue3178 created Frosburn cPickle is seriously broken 06/23/08 CLOSED http://bugs.python.org/issue3179 created schmir patch Interrupts are lost during readline PyOS_InputHook processing 06/23/08 http://bugs.python.org/issue3180 created Araneidae patch ConfigParsers are classic classes 06/23/08 http://bugs.python.org/issue3181 created hauva patch 2to3 Slight Patch 06/23/08 http://bugs.python.org/issue3182 created nedds sha modules & Modules/Setup.dist 06/23/08 http://bugs.python.org/issue3183 created rwgk The elementtree.ElementTree Module doc is missing 06/23/08 CLOSED http://bugs.python.org/issue3184 created gladstein Documentation of time.time() differs from source documentation 06/24/08 CLOSED http://bugs.python.org/issue3185 created cgrohmann bin(long) doesn't have a trailing 'L' 06/24/08 CLOSED http://bugs.python.org/issue3186 created marketdickinson os.listdir can return byte strings 06/24/08 http://bugs.python.org/issue3187 reopened amaury.forgeotdarc patch float('infinity') should be valid 06/24/08 http://bugs.python.org/issue3188 created marketdickinson patch Py3k DeprecationWarning in difflib 06/24/08 CLOSED http://bugs.python.org/issue3189 created vdupras patch Pydoc should ignore __package__ attributes 06/24/08 http://bugs.python.org/issue3190 created ncoghlan round docstring is inaccurate 06/24/08 http://bugs.python.org/issue3191 created marketdickinson exec(open(filename)) doesn't work 06/24/08 CLOSED http://bugs.python.org/issue3192 created ms Python 2.5.2 - Python 06/24/08 CLOSED http://bugs.python.org/issue3193 created peterdilley Demo/loop.c passing "char *" instead of "wchar_t *" 06/25/08 http://bugs.python.org/issue3194 created zanella patch invalid conversion xml.etree.ElementTree.Element object to boole 06/25/08 CLOSED http://bugs.python.org/issue3195 created xaka Option in pydoc to show docs from private methods 06/25/08 http://bugs.python.org/issue3196 created mortenlj patch Documentation for fractions module needs work 06/25/08 http://bugs.python.org/issue3197 created marketdickinson patch strings don't seem to roundtrip with repr() 06/25/08 CLOSED http://bugs.python.org/issue3198 created mark 2.6b1 Build Fails On OSX 10.5 06/25/08 CLOSED http://bugs.python.org/issue3199 created johna Python's crash in heavy multithreading IO operations 06/25/08 CLOSED http://bugs.python.org/issue3200 created sadit struct.Struct size attribute undocumented 06/25/08 CLOSED http://bugs.python.org/issue3201 created ericvw Wish: disable tests in unittest 06/25/08 http://bugs.python.org/issue3202 created jmp patch sphinx - table of contents doesn't render correctly (html) 06/25/08 http://bugs.python.org/issue3203 created grflanagan operator module docs are not updated to 3.0 06/26/08 CLOSED http://bugs.python.org/issue3204 created mishok13 patch bz2 iterator fails silently on MemoryError 06/26/08 http://bugs.python.org/issue3205 created mdehoon patch Multiprocessing Array and sharedctypes.Array error in docs/imple 06/26/08 http://bugs.python.org/issue3206 created mishok13 patch file.write() after file.readline() in mode "r+" 06/26/08 CLOSED http://bugs.python.org/issue3207 created peterdemin function annotation for builtin and C function 06/26/08 http://bugs.python.org/issue3208 created bhy Grammar error in UserDict module docs 06/26/08 CLOSED http://bugs.python.org/issue3209 created technel subprocess.Popen does not release process handles if process can 06/26/08 http://bugs.python.org/issue3210 created gjb1002 warnings.warn_explicit raises SystemError 06/26/08 CLOSED http://bugs.python.org/issue3211 created exarkun ssl module - should test for a wrong cert 06/26/08 http://bugs.python.org/issue3212 created jonas.wagner patch "pydoc -p" should listen to [::] if IPv6 is supported 06/27/08 http://bugs.python.org/issue3213 created ndim patch Suggest change to glossary explanation: "Duck Typing" 06/27/08 http://bugs.python.org/issue3214 created paddy3118 Can't import sqlite3 in Python 2.6b1 06/27/08 http://bugs.python.org/issue3215 created craigneuro errors in msilib documentation 06/27/08 http://bugs.python.org/issue3216 created sjoerd IndexError: Add bad index to msg 06/23/08 http://bugs.python.org/issue1534607 reopened benjamin.peterson Issues Now Closed (52) ______________________ simple patch, improving unreachable bytecode removing 232 days http://bugs.python.org/issue1394 gvanrossum patch os.system() fails for commands with multiple quoted file names 208 days http://bugs.python.org/issue1524 d4rk1 Failure when calling __str__ for MIMEBase(message, rfc822) objec 199 days http://bugs.python.org/issue1556 facundobatista module-cgi: handling GET and POST together 161 days http://bugs.python.org/issue1817 facundobatista patch, easy shutil: NameError (WindowsError) when moving from ext3 to fat32 80 days http://bugs.python.org/issue2549 draghuram easy default_scheme in urlparse.urlparse() useless 75 days http://bugs.python.org/issue2569 facundobatista csv.reader accepts string instead of file object (duck typing go 56 days http://bugs.python.org/issue2701 facundobatista os.getcwd fails for long path names on linux 54 days http://bugs.python.org/issue2722 facundobatista patch, easy curses.textpad loses characters at the end of lines 52 days http://bugs.python.org/issue2732 akuchling pprint produces different output in 2.6 and 3.0 36 days http://bugs.python.org/issue2888 facundobatista patch Verify doc updates for the decimal module 21 days http://bugs.python.org/issue3017 facundobatista Recursion bug in deepcopy 16 days http://bugs.python.org/issue3043 facundobatista heapq change breaking compatibility 13 days http://bugs.python.org/issue3051 rhettinger patch Let set.union and set.intersection accept multiple arguments 13 days http://bugs.python.org/issue3069 rhettinger patch chapter 17.1.3.5 'Replacing os.popen*' in the Python library ref 11 days http://bugs.python.org/issue3085 benjamin.peterson patch str.format("{0:n}") poss. bug with setlocale() 5 days http://bugs.python.org/issue3140 eric.smith urllib docs don't match the new modules 4 days http://bugs.python.org/issue3142 georg.brandl Sphinx/LaTeX fails on Python 3.0b1 documentation 3 days http://bugs.python.org/issue3146 georg.brandl ast module docs document private attribute 0 days http://bugs.python.org/issue3152 loewis easy, 26backport sqlite3 minor documentation issues 2 days http://bugs.python.org/issue3157 georg.brandl Missing import for sys in _abcoll 1 days http://bugs.python.org/issue3161 rhettinger patch cPickle calls to save_string and save_unicode with unicode objec 1 days http://bugs.python.org/issue3164 facundobatista patch cPickle recursion problem 4 days http://bugs.python.org/issue3165 facundobatista patch test macos skipped "unexpected" on linux2 0 days http://bugs.python.org/issue3172 benjamin.peterson multiprocessing build fails on Fedora 8 and Xubuntu 8 + solution 0 days http://bugs.python.org/issue3175 benjamin.peterson result error for 2/5 and 4/5 0 days http://bugs.python.org/issue3176 amaury.forgeotdarc __radd__(self,other) isn't called if self and other are of the s 0 days http://bugs.python.org/issue3178 amaury.forgeotdarc cPickle is seriously broken 3 days http://bugs.python.org/issue3179 jcea patch The elementtree.ElementTree Module doc is missing 0 days http://bugs.python.org/issue3184 gladstein Documentation of time.time() differs from source documentation 0 days http://bugs.python.org/issue3185 loewis bin(long) doesn't have a trailing 'L' 0 days http://bugs.python.org/issue3186 marketdickinson Py3k DeprecationWarning in difflib 0 days http://bugs.python.org/issue3189 rhettinger patch exec(open(filename)) doesn't work 2 days http://bugs.python.org/issue3192 georg.brandl Python 2.5.2 - Python 0 days http://bugs.python.org/issue3193 peterdilley invalid conversion xml.etree.ElementTree.Element object to boole 0 days http://bugs.python.org/issue3195 benjamin.peterson strings don't seem to roundtrip with repr() 0 days http://bugs.python.org/issue3198 amaury.forgeotdarc 2.6b1 Build Fails On OSX 10.5 0 days http://bugs.python.org/issue3199 marketdickinson Python's crash in heavy multithreading IO operations 0 days http://bugs.python.org/issue3200 amaury.forgeotdarc struct.Struct size attribute undocumented 0 days http://bugs.python.org/issue3201 benjamin.peterson operator module docs are not updated to 3.0 0 days http://bugs.python.org/issue3204 georg.brandl patch file.write() after file.readline() in mode "r+" 0 days http://bugs.python.org/issue3207 amaury.forgeotdarc Grammar error in UserDict module docs 0 days http://bugs.python.org/issue3209 georg.brandl warnings.warn_explicit raises SystemError 0 days http://bugs.python.org/issue3211 brett.cannon POP3 lib timeout 1602 days http://bugs.python.org/issue888830 facundobatista xrange overflows 1417 days http://bugs.python.org/issue1003935 rhettinger docs for Py_UNICODE are wrong 1349 days http://bugs.python.org/issue1044479 facundobatista easy functions replaced by subprocess should point to its docs 1287 days http://bugs.python.org/issue1083895 facundobatista BaseHTTPServer uses deprecated mimetools.Message 1169 days http://bugs.python.org/issue1180470 benjamin.peterson installation problem with python 2.4.1 on Win2k system 1137 days http://bugs.python.org/issue1199808 loewis poplib.py list interface 633 days http://bugs.python.org/issue1567948 facundobatista File Read/Write Flushing Patch 527 days http://bugs.python.org/issue1636874 amaury.forgeotdarc patch Add Tkinter.Checkbutton get() and set(value) 317 days http://bugs.python.org/issue1774370 gpolo patch Top Issues Most Discussed (10) ______________________________ 12 math test fails on Solaris 10 6 days open http://bugs.python.org/issue3167 12 implement PEP 3134 exception reporting 13 days open http://bugs.python.org/issue3112 11 sys.getsizeof() gives an AttributeError for _sre objects. 11 days open http://bugs.python.org/issue3122 9 pickling of large recursive structures crashes cPickle 2 days open http://bugs.python.org/issue2702 8 Let bin/oct/hex show floats 3 days open http://bugs.python.org/issue3008 8 No float formatting in PyString_FromFormat 48 days open http://bugs.python.org/issue2813 8 email.MIMEText.MIMEText.as_string incorrectly folding long subj 149 days open http://bugs.python.org/issue1974 7 exec(open(filename)) doesn't work 2 days closed http://bugs.python.org/issue3192 7 cPickle is seriously broken 3 days closed http://bugs.python.org/issue3179 6 rlcompleter add "(" to callables feature 2515 days open http://bugs.python.org/issue449227 From guido at python.org Fri Jun 27 20:10:11 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 11:10:11 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com> <932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com> Message-ID: On Thu, Jun 26, 2008 at 3:22 PM, Ralf Schmitt wrote: > On Thu, Jun 26, 2008 at 8:45 PM, Guido van Rossum wrote: >> >> So you're saying py.test needs to be fixed? Fine with me, but then I >> don't understand why you bothered bringing it up here instead of >> fixing it (or reporting to the py.test maintainers, I don't know if >> you're one or not). >> > > no, I'm not. Just wanted to point out, that this ticket/patch does not > solve the django issue. > (this is what I first thought it would do). I'm still missing details. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 27 20:42:47 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 11:42:47 -0700 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: <20080627003356.25821.685560775.divmod.xquotient.10844@joule.divmod.com> References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <20080626162108.25821.214433897.divmod.xquotient.10474@joule.divmod.com> <4863CE78.7010108@v.loewis.de> <20080626193558.25821.770000589.divmod.xquotient.10637@joule.divmod.com> <20080627003356.25821.685560775.divmod.xquotient.10844@joule.divmod.com> Message-ID: On Thu, Jun 26, 2008 at 5:33 PM, wrote: > OK, fair enough. Taking a step back, I was pushing this really hard because > to *me*, it seems like dealing with the influx of bug reports after the fact > is an unreasonable amount of additional work, whereas immediate reverts are > just an occasional, temporary inconvenience. Based on my experience, "We'll > deal with the reports as they come in" sounds like "no". No, that's not how it was intended. I'd rather hear you tell us about your pain than telling us how to fix it. > I think since the last time we had a similar conversation, other Twisted > developers have been fairly diligent in reporting bugs. (In the time > they've been reporting Python bugs, I've mostly been planning a wedding. > Others who have survived it tell me that eventually, this process ends... > and then I should be participating more directly.) I'll try to step up that > feeback loop and get other projects involved. I hope I am wrong about > generating an unreasonable amount of work. Again, I'd much rather see you generate a huge pile of work for us in the form of specific bug reports than trying to tell us how to change our process. > Thanks. I appreciate it; an increased emphasis on 3rd party code being > broken is really what I was looking for. This is fine with me. I mean, > whatever your decision is, I'm going to have to live with it :), but in > either case, we have to be looking for bugs and helping to investigate them. > On my end of things I guess it's not going to make much difference. Right. The beta period is the time to pay attention to 3rd party breakage. While we won't make any specific promises, we're taking 3rd party code breakage very seriously and, depending on the specifics, it can hold up a release. It just won't be automatic; many 3rd party breakages are most easily fixed by making small adjustments to the 3rd party code involved. (Imagine a 3rd party package still using string exceptions.) . . . On Thu, Jun 26, 2008 at 6:13 PM, wrote: > For the record, "automatic revert" does not mean "drop all other work". The > changeset's still there. It's still in the revision history. It can easily > be applied to anybody's working copy. It can easily be put into a branch. > It can be automatically re-merged later. I do all of these things all the > time, and I was *not* intending to suggest that any 3rd-party breakage > should cause a code freeze or anything even remotely like that. Still, a revert often holds up other work that depends on the reverted change. I expect that in most cases, finding the problem and applying a fix is much less of a burden for the core developer team as a whole than rolling back, working on a fix, and re-applying. I also expect that the policy of semi-automated merges from 2.6 into 3.0 would make a revert even more work, and more likely to cause second-order problem in the 3.0 branch. >>> (...) it would have been easier to >>> convince a Twisted developer to do the work it to keep the community >>> buildbot green rather than to make it a bit less red. >> >> Why? That sounds like it's a problem with the psychology of the >> Twisted developers. > > I hardly think it's unique to us. TDD test runners typically only know 2 > colors and 2 states: "passed" and "fails". Once you're in the "fail" state, > you tend to accumulate more failures; there's a much bigger bang for your > buck. Better tools with nicer interfaces would let you easily mark > individual tests as "usually intermittent" or something and make it a > "slightly dirty green" or something, but we don't have those. The move from > "red" to "green" is much more psychologically significant to just about > anyone I know than the move from "red but 14 failures" to "red but 12 > failures". Still sounds like perhaps you're too much focused on the "green bar". I know this is encouraged by XP, but I don't want to have a religious debates about whether XP is the right development model for Python. (The time would be better spent on improving the buildbot reporting!) > Despite the idea's origins in a now-highly-controversial book on > criminology, this is often referred to in XP discussion circles (wikis and > the like) as the "fix broken windows" collaboration pattern. I notice this > in lots of other areas of my life; a little pile of papers tends to become a > big pile of papers, the first dent in a car makes the driver a little less > careful, and so on. For me, with that first dent comes the realization that it's really not worth it to obsess over scratches, and that makes me a more relaxed and hence *better* (because less stressed) driver. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 27 21:02:32 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 12:02:32 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <48643E3F.3010500@canterbury.ac.nz> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> Message-ID: Now that I've learned about the hex float format supported by C++ and Java, I wonder if it wouldn't be better to support conversion to and from that format and nothing else. E.g. >>> math.tohex(3.14) '0x1.91eb851eb851fp+1' >>> math.fromhex('0x1.91eb851eb851fp+1') 3.1400000000000001 >>> BTW I am still hoping to be able the output of the second command to just "3.14" but the tracker issue for that (http://bugs.python.org/issue1580) is stuck on trying to decide whether it's okay to have repr() occasionally return a string that doesn't convert to the exact same value on another platform. (My preferred approach would ensure that it does convert to the same value on the same platform, because that's how I'd compute it.) Perhaps the existance of an unambiguous hex format that is also interchangeable with Java and C (and presumably C++) would alleviate that concern. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jun 27 21:04:34 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 12:04:34 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> Message-ID: PS. I can't get excited about having support for this in %-style format strings (and even less so now %a already means "call ascii()"). It would be easy enough to add support for it to float.__format__() though. On Fri, Jun 27, 2008 at 12:02 PM, Guido van Rossum wrote: > Now that I've learned about the hex float format supported by C++ and > Java, I wonder if it wouldn't be better to support conversion to and > from that format and nothing else. > > E.g. > >>>> math.tohex(3.14) > '0x1.91eb851eb851fp+1' >>>> math.fromhex('0x1.91eb851eb851fp+1') > 3.1400000000000001 >>>> > > BTW I am still hoping to be able the output of the second command to > just "3.14" but the tracker issue for that > (http://bugs.python.org/issue1580) is stuck on trying to decide > whether it's okay to have repr() occasionally return a string > that doesn't convert to the exact same value on another platform. (My > preferred approach would ensure that it does convert to the same value > on the same platform, because that's how I'd compute it.) Perhaps the > existance of an unambiguous hex format that is also interchangeable > with Java and C (and presumably C++) would alleviate that concern. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tomerfiliba at gmail.com Fri Jun 27 21:07:30 2008 From: tomerfiliba at gmail.com (tomer filiba) Date: Fri, 27 Jun 2008 21:07:30 +0200 Subject: [Python-Dev] repeated keyword arguments Message-ID: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> the following code works on python 2.5: >>> def f(**kwargs): ... print kwargs ... >>> f(a=5,b=7,a=8) {'a': 8, 'b': 7} >>> but fails on python2.4, saying that "a" is given twice. is this a bug or a feature? -tomer -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Jun 27 21:11:07 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 12:11:07 -0700 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> Message-ID: Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due to the switch to the new AST-based compiler. Can you file a bug? I think we should leave 2.5 alone (too much risk of breaking code) but fix it in 2.6 and 3.0 if we can. --Guido On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba wrote: > the following code works on python 2.5: > >>>> def f(**kwargs): > ... print kwargs > ... >>>> f(a=5,b=7,a=8) > {'a': 8, 'b': 7} >>>> > > but fails on python2.4, saying that "a" is given twice. > is this a bug or a feature? > > > -tomer > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From chris at atlee.ca Fri Jun 27 22:20:59 2008 From: chris at atlee.ca (Chris AtLee) Date: Fri, 27 Jun 2008 16:20:59 -0400 Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <4610679334594632865@unknownmsgid> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> <4610679334594632865@unknownmsgid> Message-ID: <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com> On Fri, Jun 27, 2008 at 11:40 AM, Bill Janssen wrote: >> I notice that there is some work being done on urllib / urllib2 for >> python 2.6/3.0. One thing I've always missed in urllib/urllib2 is the >> facility to encode POST data as multipart/form-data. I think it would >> also be useful to be able to stream a POST request to the remote >> server rather than having requiring the user to create the entire POST >> body in memory before starting the request. This would be extremely >> useful when writing any kind of code that does file uploads. >> >> I didn't see any recent discussion about this so I thought I'd ask >> here: do you think this would make a good addition to the new urllib >> package? > > I think it would be very helpful. I'd separate the two things, > though; you want to be able to format a set of values as > "multipart/form-data", and do various things with that resulting > "document", and you want to be able to stream a POST (or PUT) request. How about if the function that encoded the values as "multipart/form-data" was able to stream data to a POST (or PUT) request via an iterator that yielded chunks of data? def multipart_encode(params, boundary=None): """Encode ``params`` as multipart/form-data. ``params`` should be a dictionary where the keys represent parameter names, and the values are either parameter values, or file-like objects to use as the parameter value. The file-like object must support the .read(), .seek(), and .tell() methods. If ``boundary`` is set, then it as used as the MIME boundary. Otherwise a randomly generated boundary will be used. In either case, if the boundary string appears in the parameter values a ValueError will be raised. Returns an iterable object that will yield blocks of data representing the encoded parameters.""" The file objects need to support .seek() and .tell() so we can determine how large they are before including them in the output. I've been trying to come up with a good way to specify the size separately so you could use unseekable objects, but no good ideas have come to mind. Maybe it could look for a 'size' attribute or callable on the object? That seems a bit hacky... A couple helper functions would be necessary as well, one to generate random boundary strings that are guaranteed not to collide with file data, and another function to calculate the total size of the encoding to be used in the 'Content-Length' header in the main HTTP request. Then we'd need to change either urllib or httplib to support iterable objects in addition to the regular strings that it currently uses. Cheers, Chris From steve at holdenweb.com Fri Jun 27 23:18:42 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 27 Jun 2008 17:18:42 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> Message-ID: <48655932.2010203@holdenweb.com> So we wait until they port their code to 2.6 to break it? regards Steve Guido van Rossum wrote: > Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due > to the switch to the new AST-based compiler. Can you file a bug? I > think we should leave 2.5 alone (too much risk of breaking code) but > fix it in 2.6 and 3.0 if we can. > > --Guido > > On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba wrote: >> the following code works on python 2.5: >> >>>>> def f(**kwargs): >> ... print kwargs >> ... >>>>> f(a=5,b=7,a=8) >> {'a': 8, 'b': 7} >> but fails on python2.4, saying that "a" is given twice. >> is this a bug or a feature? >> >> >> -tomer >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> >> > > > -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri Jun 27 23:18:42 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 27 Jun 2008 17:18:42 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> Message-ID: <48655932.2010203@holdenweb.com> So we wait until they port their code to 2.6 to break it? regards Steve Guido van Rossum wrote: > Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due > to the switch to the new AST-based compiler. Can you file a bug? I > think we should leave 2.5 alone (too much risk of breaking code) but > fix it in 2.6 and 3.0 if we can. > > --Guido > > On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba wrote: >> the following code works on python 2.5: >> >>>>> def f(**kwargs): >> ... print kwargs >> ... >>>>> f(a=5,b=7,a=8) >> {'a': 8, 'b': 7} >> but fails on python2.4, saying that "a" is given twice. >> is this a bug or a feature? >> >> >> -tomer >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/guido%40python.org >> >> > > > -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From musiccomposition at gmail.com Fri Jun 27 23:23:30 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 27 Jun 2008 16:23:30 -0500 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> Message-ID: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> On Fri, Jun 27, 2008 at 2:11 PM, Guido van Rossum wrote: > Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due > to the switch to the new AST-based compiler. Can you file a bug? I > think we should leave 2.5 alone (too much risk of breaking code) but > fix it in 2.6 and 3.0 if we can. I think code that uses this is probably already quite broken in some fundamental way and putting the fix in 2.5 isn't much of a risk. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From wolever at cs.toronto.edu Fri Jun 27 23:30:20 2008 From: wolever at cs.toronto.edu (David Wolever) Date: Fri, 27 Jun 2008 18:30:20 -0300 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: <7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu> On 27-Jun-08, at 6:23 PM, Benjamin Peterson wrote: > On Fri, Jun 27, 2008 at 2:11 PM, Guido van Rossum > wrote: >> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably >> due >> to the switch to the new AST-based compiler. Can you file a bug? I >> think we should leave 2.5 alone (too much risk of breaking code) but >> fix it in 2.6 and 3.0 if we can. > > I think code that uses this is probably already quite broken in some > fundamental way and putting the fix in 2.5 isn't much of a risk. I don't have 2.4 handy to test it, but it is more likely that a keyword and dictionary are passed, both containing the same item: >>> f(a=3, **{'a': 4}) Would that be a potential risk? From scott+python-dev at scottdial.com Fri Jun 27 23:40:21 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 27 Jun 2008 17:40:21 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> <7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu> Message-ID: <48655E45.50503@scottdial.com> David Wolever wrote: > I don't have 2.4 handy to test it, but it is more likely that a keyword > and dictionary are passed, both containing the same item: > >>> f(a=3, **{'a': 4}) > > Would that be a potential risk? Python 2.4.3 >>> f(a=3, **{'a': 4}) Traceback (most recent call last): File "", line 1, in ? TypeError: f() got multiple values for keyword argument 'a' Python 2.5 >>> f(a=3, **{'a': 4}) TypeError: f() got multiple values for keyword argument 'a' The regression is purely in the way an argument list is reduced to a dictionary. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From fdrake at acm.org Fri Jun 27 23:54:46 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 27 Jun 2008 17:54:46 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: On Jun 27, 2008, at 5:23 PM, Benjamin Peterson wrote: > I think code that uses this is probably already quite broken in some > fundamental way and putting the fix in 2.5 isn't much of a risk. I suspect the risk has more to do with breaking something else in Python than in breaking 3rd-party code in this case. I think it should be fixed for 2.5 as well, myself. -Fred -- Fred Drake From dickinsm at gmail.com Fri Jun 27 23:54:52 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 27 Jun 2008 22:54:52 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> Message-ID: <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com> On Fri, Jun 27, 2008 at 8:02 PM, Guido van Rossum wrote: > Now that I've learned about the hex float format supported by C++ and > Java, I wonder if it wouldn't be better to support conversion to and > from that format and nothing else. > > E.g. > >>>> math.tohex(3.14) > '0x1.91eb851eb851fp+1' >>>> math.fromhex('0x1.91eb851eb851fp+1') > 3.1400000000000001 This would certainly be enough for me, though I think there's still some educational value in having binary output available. But that's just a matter of substituting a four-bit binary string for each hexadecimal digit (or learning to read hexadecimal as though it were binary). In fromhex, what would be done with a string that gives more hex digits than the machine precision can support? An obvious answer is just to round to the nearest float, but since part of the point of hex floats is having a way to specify a given value *exactly*, it might make more sense to raise an exception rather than changing the value by rounding it. Mark From scott+python-dev at scottdial.com Sat Jun 28 00:00:23 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 27 Jun 2008 18:00:23 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <48655E45.50503@scottdial.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> <7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu> <48655E45.50503@scottdial.com> Message-ID: <486562F7.80608@scottdial.com> Scott Dial wrote: > The regression is purely in the way an argument list is reduced to a > dictionary. To further elaborate: Python 2.4: >>> import dis >>> dis.dis(compile('f(a=3, b=1, a=4)', '', 'eval')) Traceback (most recent call last): File "", line 1, in ? SyntaxError: duplicate keyword argument Python 2.5: >>> import dis >>> dis.dis(compile('f(a=3, b=1, a=4)', '', 'eval')) 1 0 LOAD_NAME 0 (f) 3 LOAD_CONST 0 ('a') 6 LOAD_CONST 1 (3) 9 LOAD_CONST 2 ('b') 12 LOAD_CONST 3 (1) 15 LOAD_CONST 0 ('a') 18 LOAD_CONST 4 (4) 21 CALL_FUNCTION 768 24 RETURN_VALUE The old compiler checked for this, but the AST-based compiler just blindly compiles the entire keyword argument sequence. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From musiccomposition at gmail.com Sat Jun 28 00:03:36 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 27 Jun 2008 17:03:36 -0500 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <486562F7.80608@scottdial.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> <7AA4A53F-3A92-49D6-ABDE-EEEFC684F2B3@cs.toronto.edu> <48655E45.50503@scottdial.com> <486562F7.80608@scottdial.com> Message-ID: <1afaf6160806271503p7157c91x1aeb82a5224c7589@mail.gmail.com> On Fri, Jun 27, 2008 at 5:00 PM, Scott Dial wrote: > > The old compiler checked for this, but the AST-based compiler just blindly > compiles the entire keyword argument sequence. The new compiler will check for it when my patch on the issue 3219 is applied. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From dickinsm at gmail.com Sat Jun 28 00:50:35 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 27 Jun 2008 23:50:35 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> Message-ID: <5c6f2a5d0806271550yc0623cbxe0c6b2bc677b57d4@mail.gmail.com> On Fri, Jun 27, 2008 at 8:02 PM, Guido van Rossum wrote: > Now that I've learned about the hex float format supported by C++ and > Java, I wonder if it wouldn't be better to support conversion to and > from that format and nothing else. By the way, this particular format is also recommended by the draft versions of IEEE 754r that I've seen: section 7.12.2 of draft version 1.2.5 (this is publicly available---there's a link from the wikipedia 754r page) says: """Implementations supporting binary formats shall provide conversions between all supported internal binary formats and external hexadecimal character sequences. External hexadecimal character sequences for finite numbers are of the form specified by C99 subclauses: 6.4.4.2 floating constants, 20.1.3 strtod, 7.19.6.2 fscanf (a, e, f, g), and 7.19.6.1 fprintf (a, A).""" More recent 754r drafts spell the grammar out explicitly instead of referring to C99, and weaken the 'shall' (i.e., 'is required to') to a 'should' ('is recommended to'). Mark From guido at python.org Sat Jun 28 00:56:18 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 15:56:18 -0700 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: On Fri, Jun 27, 2008 at 2:23 PM, Benjamin Peterson wrote: > On Fri, Jun 27, 2008 at 2:11 PM, Guido van Rossum wrote: >> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due >> to the switch to the new AST-based compiler. Can you file a bug? I >> think we should leave 2.5 alone (too much risk of breaking code) but >> fix it in 2.6 and 3.0 if we can. > > I think code that uses this is probably already quite broken in some > fundamental way and putting the fix in 2.5 isn't much of a risk. No, it could just be a harmless typo in a long argument list. I really don't want to break code that is apparently working as part of an upgrade from 2.5.2 to 2.5.3. I want people to be completely fearless when it comes to such an upgrade: they should be able to just install it without having to think about testing anything, just like most people routinely install a new minor Linux upgrade pushed by their distribution. 2.6 is a different story, everyone knows they have to do testing before deciding it's safe to upgrade from 2.x to 2.(x+1). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jun 28 00:56:46 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 15:56:46 -0700 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <48655932.2010203@holdenweb.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <48655932.2010203@holdenweb.com> Message-ID: Yes. On Fri, Jun 27, 2008 at 2:18 PM, Steve Holden wrote: > So we wait until they port their code to 2.6 to break it? > > regards > Steve > > Guido van Rossum wrote: >> >> Sounds like a regression in 2.5 (and in 2.6, and in 3.0). Probably due >> to the switch to the new AST-based compiler. Can you file a bug? I >> think we should leave 2.5 alone (too much risk of breaking code) but >> fix it in 2.6 and 3.0 if we can. >> >> --Guido >> >> On Fri, Jun 27, 2008 at 12:07 PM, tomer filiba >> wrote: >>> >>> the following code works on python 2.5: >>> >>>>>> def f(**kwargs): >>> >>> ... print kwargs >>> ... >>>>>> >>>>>> f(a=5,b=7,a=8) >>> >>> {'a': 8, 'b': 7} >>> but fails on python2.4, saying that "a" is given twice. >>> is this a bug or a feature? >>> >>> >>> -tomer >>> >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/guido%40python.org >>> >>> >> >> >> > > > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > Holden Web LLC http://www.holdenweb.com/ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Sat Jun 28 00:59:08 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 27 Jun 2008 18:59:08 -0400 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com> Message-ID: Mark Dickinson wrote: >> >>>>> math.tohex(3.14) >> '0x1.91eb851eb851fp+1' >>>>> math.fromhex('0x1.91eb851eb851fp+1') >> 3.1400000000000001 How about just one self-inverse method .hex? .hex(float/hexstring) returns corresponding hexstring/float From guido at python.org Sat Jun 28 01:03:45 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 16:03:45 -0700 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: On Fri, Jun 27, 2008 at 2:54 PM, Fred Drake wrote: > On Jun 27, 2008, at 5:23 PM, Benjamin Peterson wrote: >> >> I think code that uses this is probably already quite broken in some >> fundamental way and putting the fix in 2.5 isn't much of a risk. > > I suspect the risk has more to do with breaking something else in Python > than in breaking 3rd-party code in this case. > > I think it should be fixed for 2.5 as well, myself. Let me clarify why I want to be so careful with this. If there is code that was expected to work but due to a bug in our code raises an exception, it's generally safe to fix this: people who ran into the issue found that their code didn't work, used a work-around, and that's the end of the story. But if there is code that was expected to *break* but due to a bug in our code *doesn't* raise an exception, people can very well have harmless occurrences of such code, and never noticed. Maybe their code is "broken" in the sense that it doesn't produce the correct result, but it may well be in a "don't care" way -- but if an upgrade suddenly starts raising the exception, they are likely to get unhandled exceptions where before they had none. This is particularly annoying when the author of the program that breaks is not the user of the program, to whose machine the upgrade was applied. In such cases I think it's better not to introduce new exceptions in point-point releases. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jun 28 01:04:44 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 16:04:44 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com> Message-ID: On Fri, Jun 27, 2008 at 2:54 PM, Mark Dickinson wrote: > On Fri, Jun 27, 2008 at 8:02 PM, Guido van Rossum wrote: >> Now that I've learned about the hex float format supported by C++ and >> Java, I wonder if it wouldn't be better to support conversion to and >> from that format and nothing else. >> >> E.g. >> >>>>> math.tohex(3.14) >> '0x1.91eb851eb851fp+1' >>>>> math.fromhex('0x1.91eb851eb851fp+1') >> 3.1400000000000001 > > This would certainly be enough for me, though I think there's still > some educational value in having binary output available. But > that's just a matter of substituting a four-bit binary string for > each hexadecimal digit (or learning to read hexadecimal as > though it were binary). If it's educational it can be left as an exercise for the reader. :-) > In fromhex, what would be done with a string that gives more > hex digits than the machine precision can support? An obvious > answer is just to round to the nearest float, but since part of the > point of hex floats is having a way to specify a given value > *exactly*, it might make more sense to raise an exception > rather than changing the value by rounding it. Whatever Java and C99 do. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Jun 28 01:06:01 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Jun 2008 16:06:01 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> <5c6f2a5d0806271454x370b5aeay5e1ea873542d04f9@mail.gmail.com> Message-ID: On Fri, Jun 27, 2008 at 3:59 PM, Terry Reedy wrote: > > > Mark Dickinson wrote: > >>> >>>>>> math.tohex(3.14) >>> >>> '0x1.91eb851eb851fp+1' >>>>>> >>>>>> math.fromhex('0x1.91eb851eb851fp+1') >>> >>> 3.1400000000000001 > > How about just one self-inverse method .hex? > .hex(float/hexstring) returns corresponding hexstring/float That seems to be a misplaced attempt at economy, obscuring the intent from the reader. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Jun 28 01:08:13 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 01:08:13 +0200 Subject: [Python-Dev] [Python-checkins] r64424 - in python/trunk:Include/object.h Lib/test/test_sys.py Misc/NEWSObjects/intobject.c Objects/longobject.c Objects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863AA71.5020901@gmail.com> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <6F8B0560F400452A934A174137F0F227@RaymondLaptop1> <48643E3F.3010500@canterbury.ac.nz> Message-ID: <486572DD.9080609@v.loewis.de> > Now that I've learned about the hex float format supported by C++ and > Java, I wonder if it wouldn't be better to support conversion to and > from that format and nothing else. I would be fine with that, and prefer it over the original change. Regards, Martin From janssen at parc.com Sat Jun 28 01:21:03 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 27 Jun 2008 16:21:03 PDT Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> <4610679334594632865@unknownmsgid> <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com> Message-ID: <08Jun27.162111pdt."58698"@synergy1.parc.xerox.com> All sounds reasonable to me. Bill > On Fri, Jun 27, 2008 at 11:40 AM, Bill Janssen wrote: > >> I notice that there is some work being done on urllib / urllib2 for > >> python 2.6/3.0. One thing I've always missed in urllib/urllib2 is the > >> facility to encode POST data as multipart/form-data. I think it would > >> also be useful to be able to stream a POST request to the remote > >> server rather than having requiring the user to create the entire POST > >> body in memory before starting the request. This would be extremely > >> useful when writing any kind of code that does file uploads. > >> > >> I didn't see any recent discussion about this so I thought I'd ask > >> here: do you think this would make a good addition to the new urllib > >> package? > > > > I think it would be very helpful. I'd separate the two things, > > though; you want to be able to format a set of values as > > "multipart/form-data", and do various things with that resulting > > "document", and you want to be able to stream a POST (or PUT) request. > > How about if the function that encoded the values as "multipart/form-data" > was able to stream data to a POST (or PUT) request via an iterator that > yielded chunks of data? > > def multipart_encode(params, boundary=None): > """Encode ``params`` as multipart/form-data. > > ``params`` should be a dictionary where the keys represent parameter names, > and the values are either parameter values, or file-like objects to > use as the parameter value. The file-like object must support the .read(), > .seek(), and .tell() methods. > > If ``boundary`` is set, then it as used as the MIME boundary. Otherwise > a randomly generated boundary will be used. In either case, if the > boundary string appears in the parameter values a ValueError will be > raised. > > Returns an iterable object that will yield blocks of data representing > the encoded parameters.""" > > The file objects need to support .seek() and .tell() so we can determine > how large they are before including them in the output. I've been trying > to come up with a good way to specify the size separately so you could use > unseekable objects, but no good ideas have come to mind. Maybe it could > look for a 'size' attribute or callable on the object? That seems a bit > hacky... > > A couple helper functions would be necessary as well, one to generate > random boundary strings that are guaranteed not to collide with file data, > and another function to calculate the total size of the encoding to be used > in the 'Content-Length' header in the main HTTP request. > > Then we'd need to change either urllib or httplib to support iterable > objects in addition to the regular strings that it currently uses. > > Cheers, > Chris > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com From jml at mumak.net Sat Jun 28 02:19:25 2008 From: jml at mumak.net (Jonathan Lange) Date: Sat, 28 Jun 2008 10:19:25 +1000 Subject: [Python-Dev] Disable tests in unittest (issue3202) In-Reply-To: <20080626185727.GA29900@mit.edu> References: <20080625211349.GA471@mit.edu> <20080626185727.GA29900@mit.edu> Message-ID: On Fri, Jun 27, 2008 at 4:57 AM, Justin Mazzola Paluska wrote: > I wasn't aware of bzrlib's extentions to unittest ? many of them, > including KnownFailure, TestSkipped, > bzrlib.test.ExtendedTestResult.expectFailure, and the sundry list of > bzrlib.test.TestCase.assert* look useful enough to belong in the > standard library. > I agree 100%. Twisted came up with some of the same things independently, and for a lot of things, we've stolen bzrlib's ideas. I think I've filed a couple of tickets with patches along these lines. I'll try to get my act in gear and finish the rest of the patches to unittest and get them landed. jml From hall.jeff at gmail.com Sat Jun 28 02:25:19 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Fri, 27 Jun 2008 20:25:19 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com> That's all fine and good but in this case there may be "stealth errors". If the user/programmer is expecting the first value to hold but instead On Fri, Jun 27, 2008 at 7:03 PM, Guido van Rossum wrote: > On Fri, Jun 27, 2008 at 2:54 PM, Fred Drake wrote: > > On Jun 27, 2008, at 5:23 PM, Benjamin Peterson wrote: > >> > >> I think code that uses this is probably already quite broken in some > >> fundamental way and putting the fix in 2.5 isn't much of a risk. > > > > I suspect the risk has more to do with breaking something else in Python > > than in breaking 3rd-party code in this case. > > > > I think it should be fixed for 2.5 as well, myself. > > Let me clarify why I want to be so careful with this. > > If there is code that was expected to work but due to a bug in our > code raises an exception, it's generally safe to fix this: people who > ran into the issue found that their code didn't work, used a > work-around, and that's the end of the story. > > But if there is code that was expected to *break* but due to a bug in > our code *doesn't* raise an exception, people can very well have > harmless occurrences of such code, and never noticed. Maybe their code > is "broken" in the sense that it doesn't produce the correct result, > but it may well be in a "don't care" way -- but if an upgrade suddenly > starts raising the exception, they are likely to get unhandled > exceptions where before they had none. This is particularly annoying > when the author of the program that breaks is not the user of the > program, to whose machine the upgrade was applied. > > In such cases I think it's better not to introduce new exceptions in > point-point releases. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/hall.jeff%40gmail.com > -- Haikus are easy Most make very little sense Refrigerator -------------- next part -------------- An HTML attachment was scrubbed... URL: From hall.jeff at gmail.com Sat Jun 28 02:26:52 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Fri, 27 Jun 2008 20:26:52 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com> Message-ID: <1bc395c10806271726t7fda334l4e2285c1d32a8b7b@mail.gmail.com> oops... baby jumped in my lap... i pretty much said it all though... I think the error of the software functioning incorrectly may necessitate the patch... I certainly understand Guido's concern, however. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Jun 28 03:04:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Jun 2008 11:04:57 +1000 Subject: [Python-Dev] Community buildbots and Python release quality metrics In-Reply-To: References: <20080626151855.25821.815972320.divmod.xquotient.10384@joule.divmod.com> <932f8baf0806260942h1f270c67x15f0b5c1c3c74ac5@mail.gmail.com> <20080626165449.25821.931459094.divmod.xquotient.10501@joule.divmod.com> <932f8baf0806261040q5cc37f6ct783742ee00d4f9b5@mail.gmail.com> <932f8baf0806261107k9bc92b7w14869e2be34c2c24@mail.gmail.com> <932f8baf0806261522m7b5eb75aiac986ea029cdd6ff@mail.gmail.com> Message-ID: <48658E39.9040700@gmail.com> Guido van Rossum wrote: > On Thu, Jun 26, 2008 at 3:22 PM, Ralf Schmitt wrote: >> On Thu, Jun 26, 2008 at 8:45 PM, Guido van Rossum wrote: >>> So you're saying py.test needs to be fixed? Fine with me, but then I >>> don't understand why you bothered bringing it up here instead of >>> fixing it (or reporting to the py.test maintainers, I don't know if >>> you're one or not). >>> >> no, I'm not. Just wanted to point out, that this ticket/patch does not >> solve the django issue. >> (this is what I first thought it would do). > > I'm still missing details. > I'm going to spend some time looking into this this weekend (issue assigned appropriately. Getting it working again shouldn't be too bad (just revert the relevants bits to their 2.5 equivalents) - the tricky part is going to be keeping the warning of the associated change to the semantics in 3.0. (I'll also take a look at 3.0 to make sure the relevant test case from the tracker issue works properly there) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Jun 28 03:06:45 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Jun 2008 11:06:45 +1000 Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> <4610679334594632865@unknownmsgid> <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com> Message-ID: <48658EA5.4000803@gmail.com> Chris AtLee wrote: > Then we'd need to change either urllib or httplib to support iterable > objects in addition to the regular strings that it currently uses. Chris, To avoid losing these ideas, could you add them to the issue tracker as feature requests? It's too late to get them into 2.6/3.0 but they may make good additions for the next release cycle. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Sat Jun 28 03:17:10 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 28 Jun 2008 13:17:10 +1200 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> Message-ID: <48659116.10302@canterbury.ac.nz> tomer filiba wrote: > >>> def f(**kwargs): > ... print kwargs > ... > >>> f(a=5,b=7,a=8) > {'a': 8, 'b': 7} > >>> I can't think of any reason why one would need to be able to write such code, or even want to. -- Greg From greg.ewing at canterbury.ac.nz Sat Jun 28 03:36:05 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 28 Jun 2008 13:36:05 +1200 Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> Message-ID: <48659585.8030100@canterbury.ac.nz> Chris AtLee wrote: > One thing I've always missed in urllib/urllib2 is the > facility to encode POST data as multipart/form-data. I second that, having had to reinvent it a couple of times recently. It seems like an obvious thing to want to do, and it's surprising to find it's not supported. -- Greg From steve at pearwood.info Sat Jun 28 03:58:41 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 28 Jun 2008 11:58:41 +1000 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <48659116.10302@canterbury.ac.nz> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <48659116.10302@canterbury.ac.nz> Message-ID: <200806281158.41558.steve@pearwood.info> On Sat, 28 Jun 2008 11:17:10 am Greg Ewing wrote: > tomer filiba wrote: > > >>> def f(**kwargs): > > > > ... print kwargs > > ... > > > > >>> f(a=5,b=7,a=8) > > > > {'a': 8, 'b': 7} > > I can't think of any reason why one would need to be > able to write such code, or even want to. It would be nice to be able to do this: defaults = dict(a=5, b=7) f(**defaults, a=8) # override the value of a in defaults but unfortunately that gives a syntax error. Reversing the order would override the wrong value. So as Python exists now, no, it's not terribly useful. But it's not inherently a stupid idea. -- Steven From scott+python-dev at scottdial.com Sat Jun 28 04:11:03 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 27 Jun 2008 22:11:03 -0400 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <200806281158.41558.steve@pearwood.info> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <48659116.10302@canterbury.ac.nz> <200806281158.41558.steve@pearwood.info> Message-ID: <48659DB7.4090008@scottdial.com> Steven D'Aprano wrote: > It would be nice to be able to do this: > > defaults = dict(a=5, b=7) > f(**defaults, a=8) # override the value of a in defaults I can't help but think that would be difficult coding convention to use. However, I'm considerably less bothered by: def f_with_defaults(**kw): defaults = dict(a=5, b=7) defaults.update(kw) return f(**defaults) f_with_default(a=8) The only way f(**defaults, a=8) would be useful is if it happens a lot, and in that case, it's just as good of a candidate for being made into a function itself. I see this pattern all the time, and given that "f_with_defaults" probably has some semantic meaning, it probably would be nice to give it it's own name ("g"). Just my 2 cents. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From martin at v.loewis.de Sat Jun 28 09:47:16 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 09:47:16 +0200 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> <1bc395c10806271725q6c766eabjeba27dcdc1660100@mail.gmail.com> Message-ID: <4865EC84.5050706@v.loewis.de> Jeff Hall wrote: > That's all fine and good but in this case there may be "stealth errors". That is fully understood, in all of its consequences. Regards, Martin From martin at v.loewis.de Sat Jun 28 10:14:29 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 10:14:29 +0200 Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> Message-ID: <4865F2E5.4000506@v.loewis.de> > I didn't see any recent discussion about this so I thought I'd ask > here: do you think this would make a good addition to the new urllib > package? Just in case that isn't clear: any such change must be delayed for 2.7/3.1. That is not to say that you couldn't start implementing it now, of course. Regards, Martin From tomerfiliba at gmail.com Sat Jun 28 10:30:38 2008 From: tomerfiliba at gmail.com (tomer filiba) Date: Sat, 28 Jun 2008 01:30:38 -0700 (PDT) Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: On Jun 28, 12:56?am, "Guido van Rossum" wrote: > No, it could just be a harmless typo in a long argument list. to elaborate on this point a little, i came across this error when i ported my code to 2.4. i used the optparse class which takes 10's of kwargs, and it turned out i'd given the same parameter twice (the infamous copy-paste problem). so on the one hand, it was a harmless typo (because the latest instance was taken). on the other hand, it's a good thing i tested it on 2.4, or i'd never notice the repeated argument, which may have led to weird runtime errors (if the order of the params was changed). i'd be in favor of fixing this in 2.5, just to eliminate possibly hard- to-debug runtime errors. since it's a syntax error, it would be early- noticed when the code is first run/imported, and it wouldn't require the original author of the code to fix. -tomer From ncoghlan at gmail.com Sat Jun 28 10:32:09 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Jun 2008 18:32:09 +1000 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: <4865F709.4010901@gmail.com> Guido van Rossum wrote: > In such cases I think it's better not to introduce new exceptions in > point-point releases. Perhaps they should be backported to the maintenance as warnings? Then users can decide on a case-by-case basis if they want to make that particularly warning trigger an exception. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Sat Jun 28 11:23:28 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 11:23:28 +0200 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: <48660310.1080307@v.loewis.de> > i'd be in favor of fixing this in 2.5, just to eliminate possibly hard- > to-debug runtime errors. since it's a syntax error, it would be early- > noticed when the code is first run/imported, and it wouldn't require > the original author of the code to fix. As release manager for Python 2.5, I'd like to support Guido's position: the risk of breaking existing code is just not worth it. Developers who made such a mistake will find out when they port the code to 2.6; there is no value whatsoever in end-users finding out minor bugs in software they didn't even know was written in Python. Regards, Martin From martin at v.loewis.de Sat Jun 28 11:24:43 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 11:24:43 +0200 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: <4865F709.4010901@gmail.com> References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> <4865F709.4010901@gmail.com> Message-ID: <4866035B.4030706@v.loewis.de> > Perhaps they should be backported to the maintenance as warnings? Then > users can decide on a case-by-case basis if they want to make that > particularly warning trigger an exception. No. There will likely be one more 2.5 release. This entire issue never came up in the lifetime of 2.5, so it can't be so serious to annoy end-users with a warning they don't know how to deal with. Regards, Martin From Roger.Wenham at gmx.net Sat Jun 28 14:59:50 2008 From: Roger.Wenham at gmx.net (Roger wenham) Date: Sat, 28 Jun 2008 14:59:50 +0200 Subject: [Python-Dev] Python 2.6 SSL Message-ID: <20080628125950.189860@gmx.net> HI, I have used the SSL lib to wrap sockets in existing client server code, but I find that the max send size is 16K. In other words I send 20K, the write call returns 20K but the receiving end only gets 16K. I remove the wraper and everything worksagain. I even modified the unit test to send >16K, but got the same result. Is therea patch for this ? I hope this the correct place to report this... Regards Roger -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal f?r Modem und ISDN: http://www.gmx.net/de/go/smartsurfer From eyal.lotem at gmail.com Sat Jun 28 15:40:04 2008 From: eyal.lotem at gmail.com (Eyal Lotem) Date: Sat, 28 Jun 2008 16:40:04 +0300 Subject: [Python-Dev] Cycle collection enhancement idea Message-ID: I see why a cycle that has multiple objects with a __del__ method is a problem. Once you call __del__ on one of the objects, its no longer usable by the others, and its not clear which order is correct. My question regards the case where a cycle of objects only has 1 object which has a __del__. I think a correct strategy to collect the entire cycle is the same one used on a single object. On a single object Python uses: 1. Temporarily revive object 2. Call __del__ 3. Unrevive object (if(--refcount == 0) then we're done), otherwise it was resurrected). We can apply this to the whole cycle: 1. Temporarily revive entire cycle (each of its objects) 2. Call __del__ 3. Unrevive the objects of the entire cycle (each of its objects). Step 1 will allow __del__ to run safely. Since there is only one __del__ in the cycle, it is not dangerous that its references will disappear from "under its feet". (Some code restructuring will probably be necessary, because of assumptions that are hard-coded into slot_tp_del and subtype_dealloc). I believe this enhancement is important, because: A. When using existing code -- you do not control whether its objects have a __del__. In my experience, a majority of these cases only have a single __del__-containing object in their cycles. B. Python's exit cleanup calls __del__ in the wrong order, and Python's runtime is full of cycles (Each global is a cycle, including the class objects themselves: class->dict->function->func_globals)). These cycles very often have only 1 __del__ method. Some examples of the problem posed by B: http://www.google.com/search?q=ignored+%22%27NoneType%27+object+has+no+attribute%22+%22__del__+of%22&btnG=Search Ugly workarounds exist even in the standard library [subprocess]: "def __del__(self, sys=sys):"). Example: import os class RunningFile(object): filename = '/tmp/running' def __init__(self): open(self.filename, 'wb') def __del__(self): os.unlink(self.filename) running_file = RunningFile() The deller object is in a cycle as described above [as well as the Deller class itself]. When Python exits, it could call deller.__del__() and then collect the cycle. But Python does the wrong thing here, and gets rid of the globals before calling __del__: Exception exceptions.AttributeError: "'NoneType' object has no attribute 'unlink'" in > ignored I believe applying the above enhancement would solve these problems. From ncoghlan at gmail.com Sat Jun 28 17:21:19 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 29 Jun 2008 01:21:19 +1000 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: Message-ID: <486656EF.1080209@gmail.com> Eyal Lotem wrote: > Example: > > import os > class RunningFile(object): > filename = '/tmp/running' > def __init__(self): > open(self.filename, 'wb') > def __del__(self): > os.unlink(self.filename) > running_file = RunningFile() > > The deller object is in a cycle as described above [as well as the > Deller class itself]. When Python exits, it could call > deller.__del__() and then collect the cycle. But Python does the wrong > thing here, and gets rid of the globals before calling __del__: > Exception exceptions.AttributeError: "'NoneType' object has no > attribute 'unlink'" in <__main__.RunningFile object at 0x7f9655eb92d0>> ignored I don't know what you're trying to get at with this example. There isn't any cyclic GC involved at all, just referencing counting. And before the module globals are cleared, running_file is still referenced, so calling its __del__ method early would be an outright error in the interpreter (as far as I know, getting __del__ methods to run is one of the *reasons* for clearing the module globals). It's a fact of Python development: __del__ methods cannot safely reference module globals, because those globals may be gone by the time that method is invoked. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Sat Jun 28 17:31:08 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Jun 2008 08:31:08 -0700 Subject: [Python-Dev] Python 2.6 SSL In-Reply-To: <20080628125950.189860@gmx.net> References: <20080628125950.189860@gmx.net> Message-ID: Roger, can you show us the relevant code? On Sat, Jun 28, 2008 at 5:59 AM, Roger wenham wrote: > HI, > > I have used the SSL lib to wrap sockets in existing client server code, but I find that > the max send size is 16K. In other words I send 20K, the write call returns 20K > but the receiving end only gets 16K. > > I remove the wraper and everything worksagain. > > I even modified the unit test to send >16K, but got the same result. > > Is therea patch for this ? > > I hope this the correct place to report this... > > Regards > > Roger > > > -- > Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! > Ideal f?r Modem und ISDN: http://www.gmx.net/de/go/smartsurfer > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Jun 28 17:32:08 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 17:32:08 +0200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: Message-ID: <48665978.3020605@v.loewis.de> > Example: > > import os > class RunningFile(object): > filename = '/tmp/running' > def __init__(self): > open(self.filename, 'wb') > def __del__(self): > os.unlink(self.filename) > running_file = RunningFile() > > The deller object is in a cycle as described above [as well as the > Deller class itself]. I think you are mistaken here. The RunningFile instance in above code is *not* part of a cycle. It doesn't have any instance variables (i.e. its __dict__ is empty), and it only refers to its class, which (AFAICT) doesn't refer back to the instance. > When Python exits, it could call > deller.__del__() and then collect the cycle. But Python does the wrong > thing here, and gets rid of the globals before calling __del__: > Exception exceptions.AttributeError: "'NoneType' object has no > attribute 'unlink'" in <__main__.RunningFile object at 0x7f9655eb92d0>> ignored This is a different issue. For shutdown, Python doesn't rely on cyclic garbage collection (only). Instead, all modules get forcefully cleared, causing this problem. > I believe applying the above enhancement would solve these problems. No, they wouldn't. To work around the real problem in your case, put everything that the destructor uses into an instance or class attribute: class RunningFile(object): filename = '/tmp/running' _unlink = os.unlink def __init__(self): open(self.filename, 'wb') def __del__(self): self._unlink(self.filename) Regards, Martin From guido at python.org Sat Jun 28 17:38:41 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Jun 2008 08:38:41 -0700 Subject: [Python-Dev] repeated keyword arguments In-Reply-To: References: <1d85506f0806271207n49542b91x35b5c565378c4124@mail.gmail.com> <1afaf6160806271423u2095d1c3rac55f8bd3fdd1e54@mail.gmail.com> Message-ID: On Sat, Jun 28, 2008 at 1:30 AM, tomer filiba wrote: > On Jun 28, 12:56 am, "Guido van Rossum" wrote: >> No, it could just be a harmless typo in a long argument list. > > to elaborate on this point a little, i came across this error when i > ported my code to 2.4. i used the optparse class which takes 10's of > kwargs, and it turned out i'd given the same parameter twice (the > infamous copy-paste problem). > > so on the one hand, it was a harmless typo (because the latest > instance was taken). on the other hand, it's a good thing i tested it > on 2.4, or i'd never notice the repeated argument, which may have led > to weird runtime errors (if the order of the params was changed). > > i'd be in favor of fixing this in 2.5, just to eliminate possibly hard- > to-debug runtime errors. since it's a syntax error, it would be early- > noticed when the code is first run/imported, and it wouldn't require > the original author of the code to fix. But your anecdote is exactly why I don't want it fixed in 2.5. Your code was working, the typo was harmless. I don't want upgrades from 2.5.2 to 2.5.3 (or any x.y.z to x.y.(z+1)) to break code that was working before the upgrade! This is a principle we've adopted for such point-point upgrades for a long time. Also note that it took a long time before this was first reported, so it's not exactly like it's an important or frequent problem. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Sat Jun 28 18:57:04 2008 From: janssen at parc.com (Bill Janssen) Date: Sat, 28 Jun 2008 09:57:04 PDT Subject: [Python-Dev] Python 2.6 SSL In-Reply-To: <20080628125950.189860@gmx.net> References: <20080628125950.189860@gmx.net> Message-ID: <08Jun28.095712pdt."58698"@synergy1.parc.xerox.com> > I hope this the correct place to report this... Hi, Roger. Please file a bug report at http://bugs.python.org/, and assign it to me. Please attach a patch for the change you made to the unit test suite to send >16K. Thanks! Bill From solipsis at pitrou.net Sat Jun 28 20:47:54 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 28 Jun 2008 18:47:54 +0000 (UTC) Subject: [Python-Dev] GC Proposal References: Message-ID: Adam Olsen gmail.com> writes: > > We need two counters: one is the total number of traceable objects > (those we would inspect if we did a full collection) and a number of > "pending" trace operations. Every time an object is moved into the > last generation, we increase "pending" by two - once for itself and > once for an older object. Once pending equals the total number of > traceable objects we do a full collection (and reset "pending" to 0). It sounds rather similar to Martin's proposal, except with different coefficients and slightly different definitions (but the "total number of traceable objects" should be roughly equal to the number of objects in the oldest generation, and the "number of pending trace operations" roughly equal to the number of survivor objects after a collection of the middle generation). Am I missing something? From martin at v.loewis.de Sat Jun 28 21:50:20 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 21:50:20 +0200 Subject: [Python-Dev] GC Proposal In-Reply-To: References: Message-ID: <486695FC.2090407@v.loewis.de> > It sounds rather similar to Martin's proposal, except with different > coefficients and slightly different definitions (but the "total number > of traceable objects" should be roughly equal to the number of objects > in the oldest generation, and the "number of pending trace operations" > roughly equal to the number of survivor objects after a collection of > the middle generation). > > Am I missing something? I think that's an accurate description. I think the major difference is the factor, and I think making it 100% (in my terminology) might hold up collections for too long, in some cases. Whether or not 10% is the "best" factor, I don't know (it most likely is not). Regards, Martin From rhamph at gmail.com Sat Jun 28 21:59:17 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 28 Jun 2008 13:59:17 -0600 Subject: [Python-Dev] GC Proposal In-Reply-To: References: Message-ID: On Sat, Jun 28, 2008 at 12:47 PM, Antoine Pitrou wrote: > Adam Olsen gmail.com> writes: >> >> We need two counters: one is the total number of traceable objects >> (those we would inspect if we did a full collection) and a number of >> "pending" trace operations. Every time an object is moved into the >> last generation, we increase "pending" by two - once for itself and >> once for an older object. Once pending equals the total number of >> traceable objects we do a full collection (and reset "pending" to 0). > > It sounds rather similar to Martin's proposal, except with different > coefficients and slightly different definitions (but the "total number > of traceable objects" should be roughly equal to the number of objects > in the oldest generation, and the "number of pending trace operations" > roughly equal to the number of survivor objects after a collection of > the middle generation). The effect is similar for the "batch allocation" case, but opposite for the "long-running program" case. Which is preferred is debatable.. If we had an incremental GC mine wouldn't have any bad cases, just the constant overhead. However, lacking an incremental GC, and since refcounting GC is sufficient for most cases, we might prefer to save overhead and avoid the pauses than to handle the "long-running program" case. My proposal can be made equivalent to Martin's proposal by removing all of its pending traces when an untraced object is deleted. We could even change this at runtime, by adding a counter for pending objects. Come to think of it, I think Martin's proposal needs to be implemented as mine. He wants the middle generation to be 10% larger than the oldest generation, but to find out the size you need to either iterate it (reintroducing the original problem), or keep some counters. With counters, his middle generation size is my "pending traces". > Am I missing something? Actually, I was. I lost track of Martin's thread when preparing my idea. Doh! -- Adam Olsen, aka Rhamphoryncus From martin at v.loewis.de Sat Jun 28 22:42:11 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Jun 2008 22:42:11 +0200 Subject: [Python-Dev] GC Proposal In-Reply-To: References: Message-ID: <4866A223.8040902@v.loewis.de> > The effect is similar for the "batch allocation" case, but opposite > for the "long-running program" case. I don't understand. Where is the difference? > My proposal can be made equivalent to Martin's proposal by removing > all of its pending traces when an untraced object is deleted. We > could even change this at runtime, by adding a counter for pending > objects. What is a "pending trace"? > Come to think of it, I think Martin's proposal needs to be implemented > as mine. He wants the middle generation to be 10% larger than the > oldest generation Not exactly: 10% of the oldest generation, not 10% larger. So if the oldest generation has 1,000,000 objects, I want to collect when the survivors from the middle generation reach 100,000, not when they reach 1,100,000. > but to find out the size you need to either iterate > it (reintroducing the original problem), or keep some counters. With > counters, his middle generation size is my "pending traces". Yes, I have two counters: one for the number of objects in the oldest generation (established at the last collection, and unchanged afterwards), and one for the number of survivors from the middle collection (increased every time objects move to the oldest generation). So it seems there are minor difference (such as whether a counter for the total number of traceable objects is maintained, which you seem to be suggesting), but otherwise, I still think the proposals are essentially the same. Regards, Martin From python at rcn.com Sun Jun 29 01:46:46 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 28 Jun 2008 16:46:46 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c References: <20080620041816.4D5E81E4002@bag.python.org> <328D5F1E-E082-4002-9A65-38A31625C161@python.org> <4863B400.4090401@gmail.com> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> Message-ID: <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> From: "Mark Dickinson" > There's one other major difference between the C99 notation and the > current patch: the C99 notation includes a (hexa)decimal point. The > advantages of this include: > > - the exponent gives a rough idea of the magnitude of the number, and > - the exponent doesn't vary with changes to the least significant bits > of the float. Is everyone agreed on a tohex/fromhex pair using the C99 notation as recommended in 754R? Are you thinking of math module functions or as a method and classmethod on floats? Raymond From greg.ewing at canterbury.ac.nz Sun Jun 29 02:39:11 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 29 Jun 2008 12:39:11 +1200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <486656EF.1080209@gmail.com> References: <486656EF.1080209@gmail.com> Message-ID: <4866D9AF.1050508@canterbury.ac.nz> Nick Coghlan wrote: > It's a fact of Python development: __del__ methods cannot safely > reference module globals, because those globals may be gone by the time > that method is invoked. Speaking of this, has there been any more thought given to the idea of dropping the module clearing and just relying on cyclic GC? -- Greg From aleaxit at gmail.com Sun Jun 29 04:12:49 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 28 Jun 2008 19:12:49 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> Message-ID: On Sat, Jun 28, 2008 at 4:46 PM, Raymond Hettinger wrote: > From: "Mark Dickinson" >> >> There's one other major difference between the C99 notation and the >> current patch: the C99 notation includes a (hexa)decimal point. The >> advantages of this include: >> >> - the exponent gives a rough idea of the magnitude of the number, and >> - the exponent doesn't vary with changes to the least significant bits >> of the float. > > Is everyone agreed on a tohex/fromhex pair using the C99 notation as > recommended in 754R? Dunno about everyone, but I'm +1 on that. > Are you thinking of math module functions or as a method and classmethod on > floats? I'd prefer math modules functions. Alex From rhamph at gmail.com Sun Jun 29 04:31:37 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sat, 28 Jun 2008 20:31:37 -0600 Subject: [Python-Dev] GC Proposal In-Reply-To: <4866A223.8040902@v.loewis.de> References: <4866A223.8040902@v.loewis.de> Message-ID: On Sat, Jun 28, 2008 at 2:42 PM, "Martin v. L?wis" wrote: >> The effect is similar for the "batch allocation" case, but opposite >> for the "long-running program" case. > > I don't understand. Where is the difference? > >> My proposal can be made equivalent to Martin's proposal by removing >> all of its pending traces when an untraced object is deleted. We >> could even change this at runtime, by adding a counter for pending >> objects. > > What is a "pending trace"? > >> Come to think of it, I think Martin's proposal needs to be implemented >> as mine. He wants the middle generation to be 10% larger than the >> oldest generation > > Not exactly: 10% of the oldest generation, not 10% larger. So if the > oldest generation has 1,000,000 objects, I want to collect when the > survivors from the middle generation reach 100,000, not when they reach > 1,100,000. > >> but to find out the size you need to either iterate >> it (reintroducing the original problem), or keep some counters. With >> counters, his middle generation size is my "pending traces". > > Yes, I have two counters: one for the number of objects in the oldest > generation (established at the last collection, and unchanged > afterwards), and one for the number of survivors from the middle > collection (increased every time objects move to the oldest > generation). > > So it seems there are minor difference (such as whether a counter > for the total number of traceable objects is maintained, which you > seem to be suggesting), but otherwise, I still think the proposals > are essentially the same. They are definitely quite close to equivalent. The terminology doesn't quite match up, so let me rearrange things and compare: old * 0.1 <= survivors # Martin old <= survivors * 2.0 # Adam Looks about equivalent, but "survivors" may mean two different things depending on if it removes deleted survivors or not. Splitting that up, we get this form: old <= survivors * 2.0 + deleted * 1.0 The deleted count ensures stable memory loads will still eventually cause full collections. Since our GC isn't incremental/concurrent/realtime, we'd probably don't want the full collection pauses except from big bursts, which is trivially done by making the deleted factor 0.0. My original proposal was assuming a non-zero deleted factor, while yours (and the existing codebase) assumed it'd be zero - this is how our two proposals differed. (My "pending traces" is merely a running total of survivors * 2.0 + deleted * 1.0. It looks much easier to keep separate counts though.) -- Adam Olsen, aka Rhamphoryncus From guido at python.org Sun Jun 29 06:52:22 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 28 Jun 2008 21:52:22 -0700 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <4866D9AF.1050508@canterbury.ac.nz> References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: On Sat, Jun 28, 2008 at 5:39 PM, Greg Ewing wrote: > Nick Coghlan wrote: > >> It's a fact of Python development: __del__ methods cannot safely reference >> module globals, because those globals may be gone by the time that method is >> invoked. > > Speaking of this, has there been any more thought given > to the idea of dropping the module clearing and just > relying on cyclic GC? No, but it is an intriguing thought nevertheless. The module clearing causes nothing but trouble... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sun Jun 29 08:59:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jun 2008 08:59:32 +0200 Subject: [Python-Dev] GC Proposal In-Reply-To: References: <4866A223.8040902@v.loewis.de> Message-ID: <486732D4.3020004@v.loewis.de> > Looks about equivalent, but "survivors" may mean two different things > depending on if it removes deleted survivors or not. Splitting that > up, we get this form: > > old <= survivors * 2.0 + deleted * 1.0 What precisely would be the "deleted" count? If it counts deallocations, is it relevant what generation the deallocated object was from? If so, how do you determine the generation? If not, wouldn't while 1: x=[] trigger a full garbage collection fairly quickly? Regards, Martin From eyal.lotem at gmail.com Sun Jun 29 13:26:08 2008 From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com) Date: Sun, 29 Jun 2008 04:26:08 -0700 (PDT) Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <486656EF.1080209@gmail.com> References: <486656EF.1080209@gmail.com> Message-ID: <6a7bef08-bf66-4e19-959b-625ab1bcb013@z66g2000hsc.googlegroups.com> On Jun 28, 6:21?pm, Nick Coghlan wrote: > Eyal Lotem wrote: > > Example: > > > import os > > class RunningFile(object): > > ? ? filename = '/tmp/running' > > ? ? def __init__(self): > > ? ? ? ? open(self.filename, 'wb') > > ? ? def __del__(self): > > ? ? ? ? os.unlink(self.filename) > > running_file = RunningFile() > > > The deller object is in a cycle as described above [as well as the > > Deller class itself]. ?When Python exits, it could call > > deller.__del__() and then collect the cycle. But Python does the wrong > > thing here, and gets rid of the globals before calling __del__: > > Exception exceptions.AttributeError: "'NoneType' object has no > > attribute 'unlink'" in > <__main__.RunningFile object at 0x7f9655eb92d0>> ignored > > I don't know what you're trying to get at with this example. There isn't > any cyclic GC involved at all, just referencing counting. And before the > module globals are cleared, running_file is still referenced, so calling > its __del__ method early would be an outright error in the interpreter > (as far as I know, getting __del__ methods to run is one of the > *reasons* for clearing the module globals). > > It's a fact of Python development: __del__ methods cannot safely > reference module globals, because those globals may be gone by the time > that method is invoked. That's because globals are being cleaned up in an ad-hoc manner, instead of just using cyclic GC that supports cycles. > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncogh... at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > ? ? ? ? ? ? ?http://www.boredomandlaziness.org > _______________________________________________ > Python-Dev mailing list > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... From eyal.lotem at gmail.com Sun Jun 29 13:30:09 2008 From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com) Date: Sun, 29 Jun 2008 04:30:09 -0700 (PDT) Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <48665978.3020605@v.loewis.de> References: <48665978.3020605@v.loewis.de> Message-ID: <11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com> On Jun 28, 6:32?pm, "Martin v. L?wis" wrote: > > Example: > > > import os > > class RunningFile(object): > > ? ? filename = '/tmp/running' > > ? ? def __init__(self): > > ? ? ? ? open(self.filename, 'wb') > > ? ? def __del__(self): > > ? ? ? ? os.unlink(self.filename) > > running_file = RunningFile() > > > The deller object is in a cycle as described above [as well as the > > Deller class itself]. > > I think you are mistaken here. The RunningFile instance in above code > is *not* part of a cycle. It doesn't have any instance variables (i.e. > its __dict__ is empty), and it only refers to its class, which (AFAICT) > doesn't refer back to the instance. As I explained above, it *is* part of a cycle: """including the class objects themselves: class->dict->function->func_globals""". Note: running_file.__class.__.__dict__['__init__'].func_globals['running_file'] is running_file. > > > When Python exits, it could call > > deller.__del__() and then collect the cycle. But Python does the wrong > > thing here, and gets rid of the globals before calling __del__: > > Exception exceptions.AttributeError: "'NoneType' object has no > > attribute 'unlink'" in > <__main__.RunningFile object at 0x7f9655eb92d0>> ignored > > This is a different issue. For shutdown, Python doesn't rely on > cyclic garbage collection (only). Instead, all modules get forcefully > cleared, causing this problem. I know. I assumed Python does not rely on cyclic garbage collectioin for shutdown, because it wouldn't work, as _all_ globals that have *any* instance method will be part of a cycle, and any of them which have a __del__ will not be collected. > > I believe applying the above enhancement would solve these problems. > > No, they wouldn't. > > To work around the real problem in your case, put everything that the > destructor uses into an instance or class attribute: > > class RunningFile(object): > ? ? filename = '/tmp/running' > ? ? _unlink = os.unlink > ? ? def __init__(self): > ? ? ? ? open(self.filename, 'wb') > ? ? def __del__(self): > ? ? ? ? self._unlink(self.filename) I *mentioned* this workaround. What I propose is not a workaround but a solution. You wouldn't need to clean up module globals ad-hoc'ishly, because the cyclic collection would collect your object, even with its __del__. > Regards, > Martin Please read my entire mail before replying to it. Thanks! Eyal > _______________________________________________ > Python-Dev mailing list > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... From eyal.lotem at gmail.com Sun Jun 29 13:33:28 2008 From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com) Date: Sun, 29 Jun 2008 04:33:28 -0700 (PDT) Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: On Jun 29, 7:52?am, "Guido van Rossum" wrote: > On Sat, Jun 28, 2008 at 5:39 PM, Greg Ewing wrote: > > Nick Coghlan wrote: > > >> It's a fact of Python development: __del__ methods cannot safely reference > >> module globals, because those globals may be gone by the time that method is > >> invoked. > > > Speaking of this, has there been any more thought given > > to the idea of dropping the module clearing and just > > relying on cyclic GC? > > No, but it is an intriguing thought nevertheless. The module clearing > causes nothing but trouble... This is exactly what my post tried to address. I assumed it was clear that module clearing is the wrong solution, and that it was also clear that due to the cycles I mentioned (global.__class__.__dict__['any_method'].func_globals['global'] is global), all globals that have a __del__ will not be collectible. Therefore, I proposed a solution to cycles with a __del__ in them. Only with this solution it is possible to replace module clearing with normal garbage collection. Eyal > > -- > --Guido van Rossum (home page:http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... From solipsis at pitrou.net Sun Jun 29 14:04:28 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 29 Jun 2008 12:04:28 +0000 (UTC) Subject: [Python-Dev] Cycle collection enhancement idea References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: eyal.lotem+pyutils gmail.com gmail.com> writes: > This is exactly what my post tried to address. > I assumed it was clear that module clearing is the wrong solution, and > that it was also clear that due to the cycles I mentioned > (global.__class__.__dict__['any_method'].func_globals['global'] is > global), all globals that have a __del__ will not be collectible. > Therefore, I proposed a solution to cycles with a __del__ in them. > Only with this solution it is possible to replace module clearing with > normal garbage collection. A more generic solution would be to pass a special flag to the GC when it is called as part of interpreter shutdown. This flag would instruct it to not put aside objects with finalizers, but first call all their __del__ methods and then collect them all at once regardless of whether they are part of a cycle or not. From eyal.lotem at gmail.com Sun Jun 29 14:18:46 2008 From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com) Date: Sun, 29 Jun 2008 05:18:46 -0700 (PDT) Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: On Jun 29, 3:04?pm, Antoine Pitrou wrote: > eyal.lotem+pyutils gmail.com gmail.com> writes: > > > This is exactly what my post tried to address. > > I assumed it was clear that module clearing is the wrong solution, and > > that it was also clear that due to the cycles I mentioned > > (global.__class__.__dict__['any_method'].func_globals['global'] is > > global), all globals that have a __del__ will not be collectible. > > Therefore, I proposed a solution to cycles with a __del__ in them. > > Only with this solution it is possible to replace module clearing with > > normal garbage collection. > > A more generic solution would be to pass a special flag to the GC when it is > called as part of interpreter shutdown. This flag would instruct it to not put > aside objects with finalizers, but first call all their __del__ methods and then > collect them all at once regardless of whether they are part of a cycle or not. That would be no worse than what happens now - but its still not perfect (__del__ ordering issues). Also, you would need to temporarily revive the cycles as mentioned above (to avoid accessibility of partially destructed objects). > > _______________________________________________ > Python-Dev mailing list > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... From solipsis at pitrou.net Sun Jun 29 14:36:46 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 29 Jun 2008 12:36:46 +0000 (UTC) Subject: [Python-Dev] Cycle collection enhancement idea References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: eyal.lotem+pyutils gmail.com gmail.com> writes: > > That would be no worse than what happens now - but its still not > perfect (__del__ ordering issues). Also, you would need to temporarily > revive the cycles as mentioned above (to avoid accessibility of > partially destructed objects). The idea is to call all __del__'s *before* any object in the cycle is deallocated (that is, call them manually rather than as part of deallocating them). That way you shouldn't have the issues mentioned above. From eyal.lotem at gmail.com Sun Jun 29 16:12:40 2008 From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com) Date: Sun, 29 Jun 2008 07:12:40 -0700 (PDT) Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> On Jun 29, 3:36?pm, Antoine Pitrou wrote: > eyal.lotem+pyutils gmail.com gmail.com> writes: > > > > > That would be no worse than what happens now - but its still not > > perfect (__del__ ordering issues). Also, you would need to temporarily > > revive the cycles as mentioned above (to avoid accessibility of > > partially destructed objects). > > The idea is to call all __del__'s *before* any object in the cycle is > deallocated (that is, call them manually rather than as part of deallocating > them). That way you shouldn't have the issues mentioned above. Firstly, as I said above: you will still have __del__ ordering issues. Secondly, the destructor itself currently calls __del__, so if you call __del__ before any deallocation, it will get called again as part of the deallocation. Might be a technicality but it will still probably require some code restructuring to work around (or making that code even more hairy). > _______________________________________________ > Python-Dev mailing list > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... From eyal.lotem at gmail.com Sun Jun 29 16:21:38 2008 From: eyal.lotem at gmail.com (eyal.lotem+pyutils@gmail.com) Date: Sun, 29 Jun 2008 07:21:38 -0700 (PDT) Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> Message-ID: <0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com> On Jun 29, 5:12?pm, "eyal.lotem+pyut... at gmail.com" wrote: > On Jun 29, 3:36?pm, Antoine Pitrou wrote: > > > eyal.lotem+pyutils gmail.com gmail.com> writes: > > > > That would be no worse than what happens now - but its still not > > > perfect (__del__ ordering issues). Also, you would need to temporarily > > > revive the cycles as mentioned above (to avoid accessibility of > > > partially destructed objects). > > > The idea is to call all __del__'s *before* any object in the cycle is > > deallocated (that is, call them manually rather than as part of deallocating > > them). That way you shouldn't have the issues mentioned above. > > Firstly, as I said above: you will still have __del__ ordering issues. > Secondly, the destructor itself currently calls __del__, so if you > call __del__ before any deallocation, it will get called again as part > of the deallocation. ?Might be a technicality but it will still > probably require some code restructuring to work around (or making > that code even more hairy). Additionally, there is another problem: If the cycle is not temporarily revived, and you call __del__ manually, it may break the cycle by removing the references. Thus, objects in the cycle will go down to refcount=0 during your attempt to call __del__'s on the objects in the cycle. The only sane thing is to temporarily revive the entire cycle so you can safely call __del__'s on it. Then, you might want to disable the normal __del__ calling that occurs as part of the later destruction of the cycle. > > > _______________________________________________ > > Python-Dev mailing list > > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... > > _______________________________________________ > Python-Dev mailing list > Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev > Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv... From solipsis at pitrou.net Sun Jun 29 16:40:54 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 29 Jun 2008 14:40:54 +0000 (UTC) Subject: [Python-Dev] Cycle collection enhancement idea References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> <0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com> Message-ID: eyal.lotem+pyutils gmail.com gmail.com> writes: > > Additionally, there is another problem: If the cycle is not > temporarily revived, and you call __del__ manually, it may break the > cycle by removing the references. > Thus, objects in the cycle will go down to refcount=0 during your > attempt to call __del__'s on the objects in the cycle. But if we use gcmodule's current linked list mechanisme, wouldn't this situation be correctly handled by _PyObject_GC_UNTRACK? That is, before the object is destroyed, the gc module already automatically removes it from its current "collection". Therefore, walking the collection (in this case, the list of unreachable objects) still does the right thing. (ISTM the gc relies heavily on this property) > Then, you might want to disable the > normal __del__ calling that occurs as part of the later destruction of > the cycle. I was thinking a flag in the PyObject header would do the trick but there aren't any flags in the PyObject header... *gasp*. From solipsis at pitrou.net Sun Jun 29 16:53:45 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 29 Jun 2008 14:53:45 +0000 (UTC) Subject: [Python-Dev] Cycle collection enhancement idea References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> <0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com> Message-ID: Antoine Pitrou pitrou.net> writes: > > I was thinking a flag in the PyObject header would do the trick but there > aren't any flags in the PyObject header... *gasp*. Actually, we only care about GC-tracked objects (the others are deallocated simply when their refcount falls to zero), so this could be another special value in the gc_refs field, together with a specific macro. From rhamph at gmail.com Sun Jun 29 19:12:14 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 29 Jun 2008 11:12:14 -0600 Subject: [Python-Dev] GC Proposal In-Reply-To: <486732D4.3020004@v.loewis.de> References: <4866A223.8040902@v.loewis.de> <486732D4.3020004@v.loewis.de> Message-ID: On Sun, Jun 29, 2008 at 12:59 AM, "Martin v. L?wis" wrote: >> Looks about equivalent, but "survivors" may mean two different things >> depending on if it removes deleted survivors or not. Splitting that >> up, we get this form: >> >> old <= survivors * 2.0 + deleted * 1.0 > > What precisely would be the "deleted" count? If it counts deallocations, > is it relevant what generation the deallocated object was from? > If so, how do you determine the generation? If not, wouldn't > > while 1: > x=[] > > trigger a full garbage collection fairly quickly? "deleted" means deallocated survivors. I could imagine wanting to move survivors and deleted up to the first generation, to give a stronger guarantee of how often it'd run, but that'd require a much smaller deleted factor. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Sun Jun 29 19:22:05 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 29 Jun 2008 11:22:05 -0600 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: On Sat, Jun 28, 2008 at 10:52 PM, Guido van Rossum wrote: > On Sat, Jun 28, 2008 at 5:39 PM, Greg Ewing wrote: >> Nick Coghlan wrote: >> >>> It's a fact of Python development: __del__ methods cannot safely reference >>> module globals, because those globals may be gone by the time that method is >>> invoked. >> >> Speaking of this, has there been any more thought given >> to the idea of dropping the module clearing and just >> relying on cyclic GC? > > No, but it is an intriguing thought nevertheless. The module clearing > causes nothing but trouble... Already gone in python-safethread. Then again, so is __del__. ;) (Replaced with __finalize__) -- Adam Olsen, aka Rhamphoryncus From martin at v.loewis.de Sun Jun 29 19:49:44 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jun 2008 19:49:44 +0200 Subject: [Python-Dev] GC Proposal In-Reply-To: References: <4866A223.8040902@v.loewis.de> <486732D4.3020004@v.loewis.de> Message-ID: <4867CB38.70006@v.loewis.de> >> What precisely would be the "deleted" count? If it counts deallocations, >> is it relevant what generation the deallocated object was from? >> If so, how do you determine the generation? If not, wouldn't >> >> while 1: >> x=[] >> >> trigger a full garbage collection fairly quickly? > > "deleted" means deallocated survivors. So when an object gets deallocated, how do you find out whether it survived a middle collection? > I could imagine wanting to move survivors and deleted up to the first > generation, to give a stronger guarantee of how often it'd run, but > that'd require a much smaller deleted factor. This I don't understand. What is the first generation? Generation 0 (i.e. the youngest objects)? Why would you want to move objects surviving a garbage collection back to the youngest generation? That totally defeats the purpose of generational garbage collection. Also, why would you want to move deleted objects anywhere? Their storage is reclaimed, and they don't occur in any list of objects. Regards, Martin From martin at v.loewis.de Sun Jun 29 20:00:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jun 2008 20:00:49 +0200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com> References: <48665978.3020605@v.loewis.de> <11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com> Message-ID: <4867CDD1.1040006@v.loewis.de> > As I explained above, it *is* part of a cycle: """including > the class objects themselves: class->dict->function->func_globals""". Ah, right. I must have missed that explanation. > I know. I assumed Python does not rely on cyclic garbage collectioin > for shutdown, because it wouldn't work, as _all_ globals that have > *any* instance method will be part of a cycle, and any of them which > have a __del__ will not be collected. No. The mechanism for cleaning modules at shutdown time predates cyclic GC, and was not removed because "it wouldn't work". This specific issue certainly contributes to the fact that it doesn't work, but there might be other problems as well (such as extension modules holding onto objects participating in cycles). > I *mentioned* this workaround. What I propose is not a workaround but > a solution. You wouldn't need to clean up module globals ad-hoc'ishly, > because the cyclic collection would collect your object, even with its > __del__. I don't think it solves the problem. You seem to be assuming that any such cycle will contain only a single global object with an __del__. However, as modules refer to each other, I very much doubt that is the case. > Please read my entire mail before replying to it. Thanks! I really, really tried. I read it three times before replying. However, I found it really, really difficult to follow your writing, as it was mixing problem statement and solution, so that I couldn't tell what paragraph was about what. English is not my native language, complicating communication further. Please accept my apologies. Regards, Martin From martin at v.loewis.de Sun Jun 29 20:04:06 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jun 2008 20:04:06 +0200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: <4867CE96.5060707@v.loewis.de> > That would be no worse than what happens now - but its still not > perfect (__del__ ordering issues). Also, you would need to temporarily > revive the cycles as mentioned above (to avoid accessibility of > partially destructed objects). I don't quite understand what you mean by "revive cycles". There is not need to revive any object in a cycle, as all objects are still alive (or else they wouldn't be garbage). Regards, Martin From martin at v.loewis.de Sun Jun 29 20:13:27 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jun 2008 20:13:27 +0200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> Message-ID: <4867D0C7.2060202@v.loewis.de> > Firstly, as I said above: you will still have __del__ ordering issues. Can you please elaborate? What would such __del__ ordering issues be? > Secondly, the destructor itself currently calls __del__, so if you > call __del__ before any deallocation, it will get called again as part > of the deallocation. Might be a technicality but it will still > probably require some code restructuring to work around (or making > that code even more hairy). There could be a global barricade for calling __del__: you first call all __del__s of existing objects, then set the barricade, and then start breaking cycles. This could even be done with the current approach to module clearing. Regards, Martin From martin at v.loewis.de Sun Jun 29 20:15:07 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jun 2008 20:15:07 +0200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com> References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> <8aea13f3-6621-4b0e-9565-1a103a31be4a@8g2000hse.googlegroups.com> <0d7ccb29-8e2e-44e8-8955-990517c6f8d8@x41g2000hsb.googlegroups.com> Message-ID: <4867D12B.6030506@v.loewis.de> > Additionally, there is another problem: If the cycle is not > temporarily revived, and you call __del__ manually, it may break the > cycle by removing the references. > Thus, objects in the cycle will go down to refcount=0 during your > attempt to call __del__'s on the objects in the cycle. > The only sane thing is to temporarily revive the entire cycle so you > can safely call __del__'s on it. Then, you might want to disable the > normal __del__ calling that occurs as part of the later destruction of > the cycle. I still don't understand what "revive the cycle" means. You will need to incref the object for which you call __del__, that's all. Regards, Martin From eyal.lotem at gmail.com Sun Jun 29 20:23:34 2008 From: eyal.lotem at gmail.com (Eyal Lotem) Date: Sun, 29 Jun 2008 21:23:34 +0300 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: <4867CDD1.1040006@v.loewis.de> References: <48665978.3020605@v.loewis.de> <11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com> <4867CDD1.1040006@v.loewis.de> Message-ID: On Sun, Jun 29, 2008 at 9:00 PM, "Martin v. L?wis" wrote: >> As I explained above, it *is* part of a cycle: """including >> the class objects themselves: class->dict->function->func_globals""". > > Ah, right. I must have missed that explanation. > >> I know. I assumed Python does not rely on cyclic garbage collectioin >> for shutdown, because it wouldn't work, as _all_ globals that have >> *any* instance method will be part of a cycle, and any of them which >> have a __del__ will not be collected. > > No. The mechanism for cleaning modules at shutdown time predates cyclic > GC, and was not removed because "it wouldn't work". > > This specific issue certainly contributes to the fact that it doesn't > work, but there might be other problems as well (such as extension > modules holding onto objects participating in cycles). > >> I *mentioned* this workaround. What I propose is not a workaround but >> a solution. You wouldn't need to clean up module globals ad-hoc'ishly, >> because the cyclic collection would collect your object, even with its >> __del__. > > I don't think it solves the problem. You seem to be assuming that any > such cycle will contain only a single global object with an __del__. > However, as modules refer to each other, I very much doubt that is the > case. > >> Please read my entire mail before replying to it. Thanks! > > I really, really tried. I read it three times before replying. > > However, I found it really, really difficult to follow your writing, > as it was mixing problem statement and solution, so that I couldn't > tell what paragraph was about what. English is not my native language, > complicating communication further. Please accept my apologies. I apologize for my tone as well, it just seemed frustrating that all the replies seemed to completely ignore the core point I was trying to make and claim that the problem I was trying to solve was not a problem at all, but a behavior I was unaware of... Mixing another quote from another mail: > > That would be no worse than what happens now - but its still not > > perfect (__del__ ordering issues). Also, you would need to temporarily > > revive the cycles as mentioned above (to avoid accessibility of > > partially destructed objects). > > I don't quite understand what you mean by "revive cycles". There is > not need to revive any object in a cycle, as all objects are still alive > (or else they wouldn't be garbage). By "revive cycles", I mean make sure that they are referenced by an independent referrer (one that won't go away as part of the __del__ calling process). This is similar to how the tp_dealloc code increases the refcount (actually sets it to 1, because it was certainly 0 when entering the destructor) before calling the __del__ slot. Without reviving the object before calling its __del__ in the destructor, and without reviving the objects of a cycle before calling its __del__'s, the __del__ Pythonic code may be exposed to "dead objects" (refcount==0). Consider the cycle: a.x = b b.x = a Lets suppose the a object has a __del__. Lets assume each object in the cycle has a refcount of 1 (and the cycle should die). Now lets say this is a's __del__ code: def __del__(self): self.x = None Running it will set 'b's refcount to 0 and call its destructor, which will set 'a's refcount to 0 and also call its destructor. But its __del__ is currently running - so "self" must not have a refcount of 0. If you only incref on 'a' before calling __del__, then you are probably alright, as long as there is only one __del__. > Can you please elaborate? What would such __del__ ordering issues be? class A(object): def __del__(self): print self.x.attribute class B(object): def __del__(self): print "B is going down!" del self.attribute a = A() b = B() a.x = b b.attribute = 1 If you call b's __del__ first then a's __del__ will fail. If you call a's __del__ first, then all is well. Ofcourse you can create true cyclic dependencies that no order will work, and its pretty clear there is no way to deduce the right order anyway. This is what I mean by "ordering issues". > There could be a global barricade for calling __del__: you first call > all __del__s of existing objects, then set the barricade, and then > start breaking cycles. > This could even be done with the current approach to module clearing. Note that the __del__'s themselves may be breaking cycles and refcounts will go to 0 - unless you temporarily revive (incref) the entire cycle first. > I still don't understand what "revive the cycle" means. You will need to > incref the object for which you call __del__, that's all. Unless there are multiple __del__'s in the cycle. > > Regards, > Martin > From rhamph at gmail.com Sun Jun 29 20:49:41 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 29 Jun 2008 12:49:41 -0600 Subject: [Python-Dev] GC Proposal In-Reply-To: <4867CB38.70006@v.loewis.de> References: <4866A223.8040902@v.loewis.de> <486732D4.3020004@v.loewis.de> <4867CB38.70006@v.loewis.de> Message-ID: On Sun, Jun 29, 2008 at 11:49 AM, "Martin v. L?wis" wrote: > >>> What precisely would be the "deleted" count? If it counts deallocations, >>> is it relevant what generation the deallocated object was from? >>> If so, how do you determine the generation? If not, wouldn't >>> >>> while 1: >>> x=[] >>> >>> trigger a full garbage collection fairly quickly? >> >> "deleted" means deallocated survivors. > > So when an object gets deallocated, how do you find out whether it > survived a middle collection? Reusing gc_refs for another flag. >> I could imagine wanting to move survivors and deleted up to the first >> generation, to give a stronger guarantee of how often it'd run, but >> that'd require a much smaller deleted factor. > > This I don't understand. What is the first generation? Generation 0 > (i.e. the youngest objects)? Why would you want to move objects > surviving a garbage collection back to the youngest generation? That > totally defeats the purpose of generational garbage collection. > > Also, why would you want to move deleted objects anywhere? Their > storage is reclaimed, and they don't occur in any list of objects. Sorry, really poor wording on my part. I meant to move the *counters*, so that even the first generation qualifies as a "survivor", and thus contributes to the "deleted" count. -- Adam Olsen, aka Rhamphoryncus From martin at v.loewis.de Sun Jun 29 20:57:43 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 29 Jun 2008 20:57:43 +0200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: <48665978.3020605@v.loewis.de> <11765c26-7b07-4387-9a40-bf11aa7b89d2@a70g2000hsh.googlegroups.com> <4867CDD1.1040006@v.loewis.de> Message-ID: <4867DB27.3010302@v.loewis.de> > By "revive cycles", I mean make sure that they are referenced by an > independent referrer (one that won't go away as part of the __del__ > calling process). I think this is a) unfortunate terminology (as the cycle is not dead, so no need to revive), and b) unnecessary, as calling __del__ will add a reference, anyway (in the implicit self parameter). So the object won't go away as long as __del__ runs. It might go away immediately after __del__ returns, which may or may not be a problem. > This is similar to how the tp_dealloc code > increases the refcount (actually sets it to 1, because it was > certainly 0 when entering the destructor) before calling the __del__ > slot. Without reviving the object before calling its __del__ in the > destructor, and without reviving the objects of a cycle before calling > its __del__'s, the __del__ Pythonic code may be exposed to "dead > objects" (refcount==0). No, that can't happen, and, AFAICT, is *not* the reason why the tp_dealloc resurrects the object. Instead, if it wouldn't resurrect it, tp_dealloc might become recursive, deallocating the object twice. > Consider the cycle: > a.x = b > b.x = a > > Lets suppose the a object has a __del__. Lets assume each object in > the cycle has a refcount of 1 (and the cycle should die). Now lets say > this is a's __del__ code: > def __del__(self): > self.x = None > > Running it will set 'b's refcount to 0 and call its destructor, which > will set 'a's refcount to 0 and also call its destructor. But its > __del__ is currently running - so "self" must not have a refcount of > 0. And it won't, because (say) PyObject_CallMethod (to call __del__) calls PyObject_GetAttrString, which returns a bound method which refers to im_self for the entire life of > If you only incref on 'a' before calling __del__, then you are > probably alright, as long as there is only one __del__. Why would you think so? We explicitly call one __del__. Assume that breaks the cycle, causing another object with __del__ to go to refcount zero. Now, tp_dealloc is called, raises the refcount, calls __del__ of the other object, and releases its storage. >> Can you please elaborate? What would such __del__ ordering issues be? > > If you call b's __del__ first then a's __del__ will fail. If you call > a's __del__ first, then all is well. Ofcourse you can create true > cyclic dependencies that no order will work, and its pretty clear > there is no way to deduce the right order anyway. This is what I mean > by "ordering issues". I see. As we are in interpreter shutdown, any such exceptions should be ignored (as exceptions in __del__ are, anyway). Programs involving such cycles should be considered broken, and be rewritten to avoid them (which I claim is always possible, and straight-forward) > Note that the __del__'s themselves may be breaking cycles and > refcounts will go to 0 - unless you temporarily revive (incref) the > entire cycle first. See above - you shouldn't need to. >> I still don't understand what "revive the cycle" means. You will need to >> incref the object for which you call __del__, that's all. > > Unless there are multiple __del__'s in the cycle. Not even then. Regards, Martin From dickinsm at gmail.com Mon Jun 30 02:26:11 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 30 Jun 2008 01:26:11 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> References: <20080620041816.4D5E81E4002@bag.python.org> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> Message-ID: <5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com> On Sun, Jun 29, 2008 at 12:46 AM, Raymond Hettinger wrote: > Is everyone agreed on a tohex/fromhex pair using the C99 notation as > recommended in 754R? Sounds good to me. I've attached a Python version of a possible implementation to the issue. See: http://bugs.python.org/file10780/hex_float.py It might be useful for testing. Mark From greg.ewing at canterbury.ac.nz Mon Jun 30 02:43:28 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 30 Jun 2008 12:43:28 +1200 Subject: [Python-Dev] Cycle collection enhancement idea In-Reply-To: References: <486656EF.1080209@gmail.com> <4866D9AF.1050508@canterbury.ac.nz> Message-ID: <48682C30.3060106@canterbury.ac.nz> Since it's already possible for __del__-containing cycles to survive interpreter shutdown, I don't see that this issue ought to be a showstopper for elimination of module clearing. Also, it seems to me that the kind of cycles module clearing is designed to break, i.e. those between classes, functions and the module dict, are unlikely to contain objects with ___del__ methods, so there wouldn't be much of a problem in practice. -- Greg From guido at python.org Mon Jun 30 17:53:14 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jun 2008 08:53:14 -0700 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: <5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com> References: <20080620041816.4D5E81E4002@bag.python.org> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> <5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com> Message-ID: FWIW, I'm fine with making these methods on float -- a class method float.fromhex(...) echoes e.g. dict.fromkeys(...) and datetime.fromordinal(...). The to-hex conversion could be x.hex() -- we don't tend to use ".toxyz()" as a naming convention much in Python. On Sun, Jun 29, 2008 at 5:26 PM, Mark Dickinson wrote: > On Sun, Jun 29, 2008 at 12:46 AM, Raymond Hettinger wrote: >> Is everyone agreed on a tohex/fromhex pair using the C99 notation as >> recommended in 754R? > > Sounds good to me. > > I've attached a Python version of a possible implementation to the issue. See: > > http://bugs.python.org/file10780/hex_float.py > > It might be useful for testing. > > Mark > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Mon Jun 30 18:31:06 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 30 Jun 2008 17:31:06 +0100 Subject: [Python-Dev] [Python-checkins] r64424 - inpython/trunk:Include/object.h Lib/test/test_sys.pyMisc/NEWSObjects/intobject.c Objects/longobject.cObjects/typeobject.cPython/bltinmodule.c In-Reply-To: References: <20080620041816.4D5E81E4002@bag.python.org> <4863F43C.2080904@v.loewis.de> <5c6f2a5d0806261317m3c8b848dm6e8071d8b841fa59@mail.gmail.com> <4863FC7B.6070903@v.loewis.de> <5c6f2a5d0806261346o7af44dc6g449d6bece2d75842@mail.gmail.com> <04BCC25BF0EC4DFBB06FEEA568199FB1@RaymondLaptop1> <5c6f2a5d0806261453n6ebe20b7yb26ca69c27f75517@mail.gmail.com> <58A6CFFCB6AC4A84A6C86809A50295FF@RaymondLaptop1> <5c6f2a5d0806291726o77bcd4ffvcf08c4ab2be539a4@mail.gmail.com> Message-ID: <5c6f2a5d0806300931x7635cee5t597c09ff7b06fc6f@mail.gmail.com> On Mon, Jun 30, 2008 at 4:53 PM, Guido van Rossum wrote: > FWIW, I'm fine with making these methods on float -- a class method > float.fromhex(...) echoes e.g. dict.fromkeys(...) and > datetime.fromordinal(...). The to-hex conversion could be x.hex() -- > we don't tend to use ".toxyz()" as a naming convention much in Python. Would it be totally outrageous for the float constructor to accept hex strings directly? Mark From chris at atlee.ca Mon Jun 30 19:18:17 2008 From: chris at atlee.ca (Chris AtLee) Date: Mon, 30 Jun 2008 13:18:17 -0400 Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <4865F2E5.4000506@v.loewis.de> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> <4865F2E5.4000506@v.loewis.de> Message-ID: <7790b6530806301018v733955a4mfa050381a73d922a@mail.gmail.com> On Sat, Jun 28, 2008 at 4:14 AM, "Martin v. L?wis" wrote: >> I didn't see any recent discussion about this so I thought I'd ask >> here: do you think this would make a good addition to the new urllib >> package? > > Just in case that isn't clear: any such change must be delayed for > 2.7/3.1. That is not to say that you couldn't start implementing it > now, of course. I like a challenge :) As discussed previously, there are two parts to this: handling streaming HTTP requests, and multipart/form-data encoding. I notice that support for file objects has already been added to 2.6's httplib. The changes required to support iterable objects are very minimal: Index: Lib/httplib.py =================================================================== --- Lib/httplib.py (revision 64600) +++ Lib/httplib.py (working copy) @@ -688,7 +688,12 @@ self.__state = _CS_IDLE def send(self, str): - """Send `str' to the server.""" + """Send `str` to the server. + + ``str`` can be a string object, a file-like object that supports + a .read() method, or an iterable object that supports a .next() + method. + """ if self.sock is None: if self.auto_open: self.connect() @@ -710,6 +715,10 @@ while data: self.sock.sendall(data) data=str.read(blocksize) + elif hasattr(str,'next'): + if self.debuglevel > 0: print "sendIng an iterable" + for data in str: + self.sock.sendall(data) else: self.sock.sendall(str) except socket.error, v: (small aside, should the parameter called 'str' be renamed to something else to avoid conflicts with the 'str' builtin?) All regression tests continue to pass with this change applied. If this change is not applied, then we have to jump through a couple of hoops to support iterable HTTP request bodies: - Provide our own httplib.HTTP(S)Connection classes that override send() to do exactly what the patch above accomplishes - Provide our own urllib2.HTTP(S)Handler classes that will use the new HTTP(S)Connection classes - Register the new HTTP(S)Handler classes with urllib2 so they take priority over the standard handlers I've created the necessary sub-classes, as well as several classes and functions to do multipart/form-data encoding of strings and files. My current work is available online here: http://atlee.ca/software/poster (tarball here: http://atlee.ca/software/poster/dist/0.1/poster-0.1dev.tar.gz) Cheers, Chris From chris at atlee.ca Mon Jun 30 20:05:25 2008 From: chris at atlee.ca (Chris AtLee) Date: Mon, 30 Jun 2008 14:05:25 -0400 Subject: [Python-Dev] urllib, multipart/form-data encoding and file uploads In-Reply-To: <48658EA5.4000803@gmail.com> References: <7790b6530806270642v2ceac78cj37a94719215484d4@mail.gmail.com> <4610679334594632865@unknownmsgid> <7790b6530806271320q79971756x85e6c84b7120ab03@mail.gmail.com> <48658EA5.4000803@gmail.com> Message-ID: <7790b6530806301105t106d8d91j822c327f3df0ae06@mail.gmail.com> On Fri, Jun 27, 2008 at 9:06 PM, Nick Coghlan wrote: > Chris, > > To avoid losing these ideas, could you add them to the issue tracker as > feature requests? It's too late to get them into 2.6/3.0 but they may make > good additions for the next release cycle. > > Cheers, > Nick. Issues #3243 and #3244 created. Cheers, Chris