From zooko at zooko.com Thu May 1 00:21:13 2008 From: zooko at zooko.com (zooko) Date: Wed, 30 Apr 2008 16:21:13 -0600 Subject: [Python-Dev] shal we redefine "module" and "package"? Message-ID: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> Folks: Here's an experiment you can perform. Round up a Python programmer and ask him the following three questions: Q1. You type "import foo" and it works. What kind of thing is foo? Q2. You go to the Python package index and download something named "bar-1.0.0.tar.gz". What kind of thing is bar? Q3. What is a "distribution"? I'm willing to bet that you will get the following answers: A1. foo is a module. A2. bar is a package. A3. A distribution is a version of Linux that comes with a lot of Free Software. Unfortunately these answers aren't quite right. A "package" is actually a directory containing an __init__.py file, and a distribution is actually what you think of when you say "package" -- a reusable package of Python code that you can, for example, get from the Python package index. Educational efforts such as the Python tutorial and the distutils docs have not succeeded in training Python programmers to understand the terminology for these things as used by the Python implementors, so perhaps instead the implementors should start using the terminology understood by the programmers: 1. A "module" shall henceforth be the name for either a foo.py file (a single-file module), or a directory with an __init__.py in it (a directory module). 2. A "package" shall henceforth be the name of the thing that is currently called a "distribution". Regards, Zooko who doesn't mind stirring up trouble on occasion... From phd at phd.pp.ru Thu May 1 00:36:28 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu, 1 May 2008 02:36:28 +0400 Subject: [Python-Dev] shal we redefine "module" and "package"? In-Reply-To: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> References: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> Message-ID: <20080430223628.GA6645@phd.pp.ru> On Wed, Apr 30, 2008 at 04:21:13PM -0600, zooko wrote: > so perhaps instead the implementors should start using the > terminology understood by the programmers: > > 1. A "module" shall henceforth be the name for either a foo.py file > (a single-file module), or a directory with an __init__.py in it (a > directory module). > > 2. A "package" shall henceforth be the name of the thing that is > currently called a "distribution". And while we are at it, I'd like to propose to rename the very language because everybody knows that python is a huge four-legged poisonous lizard. (Sorry for being so poisonously [pythonic?] sarcastic. Just cannot resist.) Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From steve at holdenweb.com Thu May 1 00:53:45 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 30 Apr 2008 18:53:45 -0400 Subject: [Python-Dev] shal we redefine "module" and "package"? In-Reply-To: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> References: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> Message-ID: zooko wrote: > Folks: > > Here's an experiment you can perform. Round up a Python programmer and > ask him the following three questions: > > Q1. You type "import foo" and it works. What kind of thing is foo? > > Q2. You go to the Python package index and download something named > "bar-1.0.0.tar.gz". What kind of thing is bar? > > Q3. What is a "distribution"? > > I'm willing to bet that you will get the following answers: > > A1. foo is a module. > > A2. bar is a package. > > A3. A distribution is a version of Linux that comes with a lot of Free > Software. > > > Unfortunately these answers aren't quite right. A "package" is actually > a directory containing an __init__.py file, and a distribution is > actually what you think of when you say "package" -- a reusable package > of Python code that you can, for example, get from the Python package > index. > > Educational efforts such as the Python tutorial and the distutils docs > have not succeeded in training Python programmers to understand the > terminology for these things as used by the Python implementors, so > perhaps instead the implementors should start using the terminology > understood by the programmers: > > 1. A "module" shall henceforth be the name for either a foo.py file (a > single-file module), or a directory with an __init__.py in it (a > directory module). > > 2. A "package" shall henceforth be the name of the thing that is > currently called a "distribution". > -1 > > Zooko > > who doesn't mind stirring up trouble on occasion... > regards Steve who doesn't mind trouble but would rather see communications improve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Thu May 1 01:08:25 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 01 May 2008 09:08:25 +1000 Subject: [Python-Dev] shal we redefine "module" and "package"? In-Reply-To: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> References: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> Message-ID: <4818FBE9.3040805@gmail.com> zooko wrote: > Folks: > > Here's an experiment you can perform. Round up a Python programmer and > ask him the following three questions: > > Q1. You type "import foo" and it works. What kind of thing is foo? foo is a package or a module. Not enough information is provide here to say which. > Q2. You go to the Python package index and download something named > "bar-1.0.0.tar.gz". What kind of thing is bar? bar is a package, a module, or a collection of a number of these things. Again, not enough information is provided in the question to say which is actually the case. Cheers, Nick. P.S. People, including programmers, are often sloppy with terminology. This is not a new problem. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From glyph at divmod.com Thu May 1 01:11:46 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 30 Apr 2008 23:11:46 -0000 Subject: [Python-Dev] shal we redefine "module" and "package"? In-Reply-To: References: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> Message-ID: <20080430231146.25821.1286760271.divmod.xquotient.4498@joule.divmod.com> On 10:53 pm, steve at holdenweb.com wrote: >zooko wrote: >>Unfortunately these answers aren't quite right. A "package" is >>actually a directory containing an __init__.py file, and a >>distribution is actually what you think of when you say "package" -- a >>reusable package of Python code that you can, for example, get from >>the Python package index. The fact that the Python "package" index is a place where you get "distributions" does seem a bit weird to me, but this is not necessarily a problem that can be fixed. Ambiguity is part of human language. Perhaps suggest transliterations of these terms for talking about python in lojban? :) >>1. A "module" shall henceforth be the name for either a foo.py file >>(a single-file module), or a directory with an __init__.py in it (a >>directory module). I have a less disruptive counterproposal. How about just starting to refer to directories (or "folders", or zip entries) with '__init__.py' in them as "package modules"? A package is-a module anyway. >>2. A "package" shall henceforth be the name of the thing that is >>currently called a "distribution". I belive a multi-word term here would be similarly more memorable and precise. A "package distribution" would include the more familiar term while still being specific, consistent with the old terminology, and correct. Using a qualifying word is probably a good idea in this context anyway. I usually say "debian package", "RPM", "MSI", or "tarball" unless I'm specifically talking about "packages for your platform", almost always in the phrase, "please do not use distutils to do a system install of Twisted, use the specific package for your platform". >-1 I do, however, agree with Steve emphatically on your original proposal. Changing the terminology now will make billions upon billions of Python web pages, modules (c.f. twisted.python.modules.PythonModule.isPackage()) documents, and searchable message archives obsolete, not to mention that 90% of the community will probably ignore you and use the old terminology anyway, creating more confusion than it eliminates. From tjreedy at udel.edu Thu May 1 03:07:26 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 30 Apr 2008 21:07:26 -0400 Subject: [Python-Dev] shal we redefine "module" and "package"? References: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> Message-ID: "zooko" wrote in message news:50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7 at zooko.com... | Folks: | Unfortunately these answers aren't quite right. A "package" is | actually a directory containing an __init__.py file, and a | distribution is actually what you think of when you say "package" -- | a reusable package of Python code that you can, for example, get from | the Python package index. If I go to a market, I may buy an individual food item (an apple, for instance), but most foods come in packages. The items and packages are them put in distribution carriers (bags) for transport home, where they are unbagged. A .tgz file is a bag, not a package. Since I am buying food, not bags, I would not want the store called a distribution/bag store. Indeed, other types of stores use the same or similar distribution carriers, just as .tgz is used for things others than Python modules and packages. From bignose+hates-spam at benfinney.id.au Thu May 1 04:37:58 2008 From: bignose+hates-spam at benfinney.id.au (Ben Finney) Date: Thu, 01 May 2008 12:37:58 +1000 Subject: [Python-Dev] shal we redefine "module" and "package"? References: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> Message-ID: <87zlra21h5.fsf@benfinney.id.au> zooko writes: > I'm willing to bet that you will get the following answers: > > A1. foo [from 'import foo'] is a module. > > A2. bar [of 'bar-1.2.3.tar.gz'] is a package. > > A3. A distribution is a version of Linux that comes with a lot of > Free Software. > > > Unfortunately these answers aren't quite right. More accurately, they're right in the context from which the speaker learned them. > A "package" is actually a directory containing an __init__.py file, > and a distribution is actually what you think of when you say > "package" -- a reusable package of Python code that you can, for > example, get from the Python package index. Only within Python's terminology. This, of course, conflicts with the meanings that adhere to those words just about everywhere else on the user's operating system. > Educational efforts such as the Python tutorial and the distutils > docs have not succeeded in training Python programmers to understand > the terminology for these things as used by the Python implementors, More accurately, the meanings you list in the hypothetical responses above are entrenched and more useful for general use. > so perhaps instead the implementors should start using the > terminology understood by the programmers: In principle this would be good. However: > 1. A "module" shall henceforth be the name for either a foo.py file > (a single-file module), or a directory with an __init__.py in it (a > directory module). How then to distinguish between these? They're clearly separate concepts, and I think they need distinct terms. > 2. A "package" shall henceforth be the name of the thing that is > currently [in Python terminology] called a "distribution". That one would be good. You then have the converse problem of changing the use of terminology that is currently entrenched in existing Python documentation and minds :-) -- \ "Always code as if the guy who ends up maintaining your code | `\ will be a violent psychopath who knows where you live." ?John | _o__) F. Woods | Ben Finney From armin.ronacher at active-4.com Thu May 1 10:42:19 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Thu, 1 May 2008 08:42:19 +0000 (UTC) Subject: [Python-Dev] Problems with the new super() References: Message-ID: Hi, Guido van Rossum python.org> writes: > The staticmethod thing isn't new; that's also the case in 2.x. staticmethod hasn't changed, method has. In the past Class.method gave you a unbound method, now you get a function back as if it was a static method. > The super() thing is a case of practicality beats purity. Note that > you pay a small but measurable cost for the implicit __class__ (it's > implemented as a "cell variable", the same mechanism used for nested > scopes) so we wouldn't want to introduce it unless it is used. I do agree that super() is a lot easier to work with than regular way to call it. But the fact that it breaks if i do `_super = super` or that it's impossible to emulate it from within Python. Regards, Armin From guido at python.org Thu May 1 18:41:07 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 09:41:07 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool Message-ID: Some of you may have seen a video recorded in November 2006 where I showed off Mondrian, a code review tool that I was developing for Google (http://www.youtube.com/watch?v=sMql3Di4Kgc). I've always hoped that I could release Mondrian as open source, but it was not to be: due to its popularity inside Google, it became more and more tied to proprietary Google infrastructure like Bigtable, and it remained limited to Perforce, the commercial revision control system most used at Google. What I'm announcing now is the next best thing: an code review tool for use with Subversion, inspired by Mondrian and (soon to be) released as open source. Some of the code is even directly derived from Mondrian. Most of the code is new though, written using Django and running on Google App Engine. I'm inviting the Python developer community to try out the tool on the web for code reviews. I've added a few code reviews already, but I'm hoping that more developers will upload at least one patch for review and invite a reviewer to try it out. To try it out, go here: http://codereview.appspot.com Please use the Help link in the top right to read more on how to use the app. Please sign in using your Google Account (either a Gmail address or a non-Gmail address registered with Google) to interact more with the app (you need to be signed in to create new issues and to add comments to existing issues). Don't hesitate to drop me a note with feedback -- note though that there are a few known issues listed at the end of the Help page. The Help page is really a wiki, so feel free to improve it! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Thu May 1 19:57:18 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 01 May 2008 19:57:18 +0200 Subject: [Python-Dev] Module Suggestion: ast In-Reply-To: References: Message-ID: Armin Ronacher schrieb: > Hi all, > > I would like to propose a new module for the stdlib for Python 2.6 > and higher: "ast". The motivation for this module is the pending > deprecation for compiler.ast which is widely used (debugging, > template engines, code coverage etc.). _ast is a very solid module > and is without a doubt easier to maintain then compiler.ast which > was written in Python, it's lacking some features such as pretty > printing the AST or traversing it. If there are no further objections, I'll add this to PEP 361 so that the proposal doesn't get lost. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Thu May 1 20:20:41 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 01 May 2008 20:20:41 +0200 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: Armin Ronacher schrieb: >> The super() thing is a case of practicality beats purity. Note that >> you pay a small but measurable cost for the implicit __class__ (it's >> implemented as a "cell variable", the same mechanism used for nested >> scopes) so we wouldn't want to introduce it unless it is used. > I do agree that super() is a lot easier to work with than regular way to > call it. But the fact that it breaks if i do `_super = super` or that > it's impossible to emulate it from within Python. That it isn't emulatable from Python doesn't bother me -- several functions have that property. But the other two magical things about super() really bother me too. I haven't looked at the new super in detail so far (and I don't know how many others have), and two things are really strikingly unpythonic in my view: * super() only works when named "super" [1]. It shouldn't be a function if it has that property; no other Python function has that. * "__class__" is magical in classes. If you define a local called "__class__" super won't work anymore in that function. Also, you can access "__class__" from any method, without the "self." qualifier -- another magical name. There may be more implications and surprising behavior surrounding this. I know that the implementation is a compromise, but I'd rather see a super() whose full semantics can be explained to programmers without using to "cell variable", "f_localsplus" and "symtable". cheers, Georg [1] Actually, it only works if a name "super" is accessed somewhere in the function, but this is what someone trying to alias "super" will perceive. -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From lists at cheimes.de Thu May 1 21:32:19 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 01 May 2008 21:32:19 +0200 Subject: [Python-Dev] Module properties for C modules In-Reply-To: References: <4818E1CF.7030903@cheimes.de> Message-ID: <481A1AC3.9070506@cheimes.de> Guido van Rossum schrieb: > But wouldn't this mean that those properties would no longer be > available in the module's __dict__? Correct. Module properties would behave exactly like instance properties. They don't appear on the instance's __dict__ attribute, too. By the way I was astonished that the vars() function dones't show properties but dir() does list them. >>> class Example(object): ... @property ... def x(self): ... return 42 ... >>> example = Example() >>> example.__dict__ {} >>> vars(example) {} >>> dir(example) ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'x'] Christian From facundobatista at gmail.com Thu May 1 21:38:56 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 1 May 2008 16:38:56 -0300 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: 2008/5/1, Georg Brandl : > There may be more implications and surprising behavior surrounding this. > > I know that the implementation is a compromise, but I'd rather see a > super() > whose full semantics can be explained to programmers without using to > "cell variable", "f_localsplus" and "symtable". In consideration of what's been said about super() in the past, and what is handled here regarding its Py3 implementation, I want to make a step back, and ask: Has super() proved more useful than harmful? Which is the value for Py3 to keep it? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From guido at python.org Thu May 1 21:55:22 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 12:55:22 -0700 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: On Thu, May 1, 2008 at 11:20 AM, Georg Brandl wrote: > But the other two magical things about super() really bother me too. I > haven't looked at the new super in detail so far (and I don't know how > many others have), and two things are really strikingly unpythonic in > my view: > > * super() only works when named "super" [1]. It shouldn't be a function if > it has that property; no other Python function has that. Actually, I believe IronPython and/or Jython have to use this trick in certain cases -- at least I recall Jim Hugunin talking about generating different code when the use of locals() was detected. I'm not proud of this, but I don't see a way around it. The alternative would be to make it a keyword, which seemed excessive (plus, it would be odd if super() were a keyword when self is not). There were long discussions about various possible ways to implement something like this, and they all had their downsides. (The PEP still isn't fixed to describe the status quo.) > * "__class__" is magical in classes. If you define a local called > "__class__" > super won't work anymore in that function. > > Also, you can access "__class__" from any method, without the "self." > qualifier -- another magical name. I don't mind this at all -- it's a name starting and ending with double underscores, so you shouldn't use it except for its defined semantics, which happen to be exactly what super needs. (I would have proposed __super__() to get an exception for that too, except that it's excessively ugly.) > There may be more implications and surprising behavior surrounding this. > > I know that the implementation is a compromise, but I'd rather see a > super() > whose full semantics can be explained to programmers without using to > "cell variable", "f_localsplus" and "symtable". You don't have to explain it that way at all. First you explain how the 2.x super(C, self) works. Then you explain that if you call it with no arguments, the arguments default to the current class object and the first argument of the current function. Only people wanting to write their own interpreter need to know more. > cheers, > Georg > > [1] Actually, it only works if a name "super" is accessed somewhere in the > function, but this is what someone trying to alias "super" will > perceive. To Facundo, who asks if we need super() at all: yes, we need it. You can't write decent multiple-inheritance code without it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Thu May 1 22:00:02 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 01 May 2008 16:00:02 -0400 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: <20080501195946.539BC3A4080@sparrow.telecommunity.com> At 04:38 PM 5/1/2008 -0300, Facundo Batista wrote: >Has super() proved more useful than harmful? For me, yes. I use it all the time. The only time I use explicit-target upcalls is in __init__ methods, and there usually only to skip a subclass' init or to explicitly manage a tricky bit of multiple inheritance. (Note, by the way, that you cannot safely write an upcall in a mixin class without super, so it can't safely be done away with, anyway.) From gjcarneiro at gmail.com Thu May 1 22:00:16 2008 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Thu, 1 May 2008 21:00:16 +0100 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: 2008/5/1 Facundo Batista : > 2008/5/1, Georg Brandl : > > > There may be more implications and surprising behavior surrounding > this. > > > > I know that the implementation is a compromise, but I'd rather see a > > super() > > whose full semantics can be explained to programmers without using to > > "cell variable", "f_localsplus" and "symtable". > > In consideration of what's been said about super() in the past, and > what is handled here regarding its Py3 implementation, I want to make > a step back, and ask: > > Has super() proved more useful than harmful? Which is the value for > Py3 to keep it? Since Python supports multiple inheritance, you can't get away from something like super. Alternatives for methods chaining to parent classes are 1. implicit/automatic (like C++, I think), 2. explicit/semi-automatic (python's super, on one form or another, 3. explicit manual (programmer has to manually keep track of everything). Of all these alternatives I prefer Python's super (in one form or another). 1 is too "magic", and 3 is too error prone and cumbersome. A better question would be, is multiple inheritance good or bad for programs? :-) I think many people's answer to the above would be, generally bad, but there are exceptions where it helps and can be justified. > > > Regards, > > -- > . Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu May 1 22:02:19 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 13:02:19 -0700 Subject: [Python-Dev] Module properties for C modules In-Reply-To: <481A1AC3.9070506@cheimes.de> References: <4818E1CF.7030903@cheimes.de> <481A1AC3.9070506@cheimes.de> Message-ID: On Thu, May 1, 2008 at 12:32 PM, Christian Heimes wrote: > Guido van Rossum schrieb: > > > But wouldn't this mean that those properties would no longer be > > available in the module's __dict__? > > Correct. Module properties would behave exactly like instance > properties. They don't appear on the instance's __dict__ attribute, too. > > By the way I was astonished that the vars() function dones't show > properties but dir() does list them. "Astonished" sounds stronger than you probably meant it. :-) > >>> class Example(object): > ... @property > ... def x(self): > ... return 42 > ... > >>> example = Example() > >>> example.__dict__ > {} > >>> vars(example) > {} > >>> dir(example) > ['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', > '__hash__', '__init__', '__module__', '__new__', '__reduce__', > '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'x'] They are intentionally different though -- dir() tries to give all the attributes, while vars() only accesses __dict__. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Thu May 1 22:20:56 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 01 May 2008 22:20:56 +0200 Subject: [Python-Dev] Module properties for C modules In-Reply-To: References: <4818E1CF.7030903@cheimes.de> <481A1AC3.9070506@cheimes.de> Message-ID: <481A2628.2060307@cheimes.de> Guido van Rossum schrieb: > On Thu, May 1, 2008 at 12:32 PM, Christian Heimes wrote: >> Guido van Rossum schrieb: >> >>> But wouldn't this mean that those properties would no longer be >> > available in the module's __dict__? >> >> Correct. Module properties would behave exactly like instance >> properties. They don't appear on the instance's __dict__ attribute, too. >> >> By the way I was astonished that the vars() function dones't show >> properties but dir() does list them. > > "Astonished" sounds stronger than you probably meant it. :-) Slightly :) What's your opinion on the module properties idea? Do you still like it although the property values won't show up in __dict__? Christian From zooko at zooko.com Thu May 1 22:25:27 2008 From: zooko at zooko.com (zooko) Date: Thu, 1 May 2008 14:25:27 -0600 Subject: [Python-Dev] shal we redefine "module" and "package"? In-Reply-To: <20080430231146.25821.1286760271.divmod.xquotient.4498@joule.divmod.com> References: <50D35C20-ED3D-4A7D-965D-D5BBD76EDDC7@zooko.com> <20080430231146.25821.1286760271.divmod.xquotient.4498@joule.divmod.com> Message-ID: <1AAD84FC-602B-42E5-84E7-31045DE78AAF@zooko.com> On Apr 30, 2008, at 5:11 PM, glyph at divmod.com wrote: > I have a less disruptive counterproposal. How about just starting > to refer to directories (or "folders", or zip entries) with > '__init__.py' in them as "package modules"? A package is-a module > anyway. That's a good idea. > I belive a multi-word term here would be similarly more memorable > and precise. A "package distribution" would include the more > familiar term while still being specific, consistent with the old > terminology, and correct. Using a qualifying word is probably a > good idea in this context anyway. I usually say "debian package", > "RPM", "MSI", or "tarball" unless I'm specifically talking about > "packages for your platform", That's a good one too. > almost always in the phrase, "please do not use distutils to do a > system install of Twisted, use the specific package for your > platform". This is a tangent, but why do you give that advice? I typically give people the opposite advice on how to install Twisted. > I do, however, agree with Steve emphatically on your original > proposal. Changing the terminology now will make billions upon > billions of Python web pages, modules (c.f. > twisted.python.modules.PythonModule.isPackage()) documents, and > searchable message archives obsolete, not to mention that 90% of > the community will probably ignore you and use the old terminology > anyway, creating more confusion than it eliminates. I suspect 90% of the community already uses my proposed terminology -- that was my original challenge to round up a Python programmer and find out. But I agree that my proposal would contribute to confusion and disruption, and I like your counterproposals better, at least for now. Directories, folders, or zip entries with __init__.py in them are "package modules", and Python packages are "package distributions". Regards, Zooko From barry at python.org Thu May 1 22:26:34 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 1 May 2008 16:26:34 -0400 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 Message-ID: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 This is a reminder that the LAST planned alpha releases of Python 2.6 and 3.0 are scheduled for next Wednesday, 07-May-2008. Please be diligent over the next week so that none of your changes break Python. The stable buildbots look moderately okay, let's see what we can do about getting them all green: http://www.python.org/dev/buildbot/stable/ We have a few showstopper bugs, and I will be looking at these more carefully starting next week. http://bugs.python.org/issue?@columns=title,id,activity,versions,status&@sort=activity&@filter=priority,status&@pagesize=50&@startwith=0&priority=1&status=1&@dispname=Showstoppers Time is running short to get any new features into Python 2.6 and 3.0. The release after this one is scheduled to be the first beta release, at which time we will institute a feature freeze. If your feature doesn't make it in by then, you'll have to wait until 2.7/3.1. If there is something that absolutely must go into 2.6/3.0 be sure that there is a bug issue open for it and that the Priority is set to 'release blocker'. I may reduce it to critical for the next alpha, but we'll review all the release blocker and critical issues for the first 2.6 and 3.0 beta releases. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSBonfHEjvBPtnXfVAQLaSwP+IMjYbLryACRColvgTU4ezPHhbBpdDaRA I2k15cLsqmkFwHitt9TaTlLklnZuETiEfl7pVzow20KW18Z2tWP5U5KVMrVVbrJM 9pMS/vC102FVD88ukyQcPP5q+pw2+r2qTLr3q/205zdELQlWo+Ny6ir6dAgTKOd4 /OZqgCMBHS4= =MhWr -----END PGP SIGNATURE----- From guido at python.org Thu May 1 22:27:14 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 13:27:14 -0700 Subject: [Python-Dev] Module properties for C modules In-Reply-To: <481A2628.2060307@cheimes.de> References: <4818E1CF.7030903@cheimes.de> <481A1AC3.9070506@cheimes.de> <481A2628.2060307@cheimes.de> Message-ID: On Thu, May 1, 2008 at 1:20 PM, Christian Heimes wrote: > What's your opinion on the module properties idea? Do you still like it > although the property values won't show up in __dict__? I don't see how it could work if any Python code is executed in the module, since code execution uses a dict for globals. Supporting it only for built-in modules seems too big an exception. So I'm -0. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Thu May 1 23:00:35 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 01 May 2008 23:00:35 +0200 Subject: [Python-Dev] Module properties for C modules In-Reply-To: References: <4818E1CF.7030903@cheimes.de> <481A1AC3.9070506@cheimes.de> <481A2628.2060307@cheimes.de> Message-ID: <481A2F73.5050109@cheimes.de> Guido van Rossum schrieb: > I don't see how it could work if any Python code is executed in the > module, since code execution uses a dict for globals. > > Supporting it only for built-in modules seems too big an exception. I came up with the idea in order to replace the setters and getters of builtin modules such as the sys module. I deliberately excluded Python modules from the idea. It's going to be too tricky to expose the API to Python. Christian From lists at cheimes.de Thu May 1 23:27:52 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 01 May 2008 23:27:52 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> Message-ID: <481A35D8.60604@cheimes.de> Barry Warsaw schrieb: > This is a reminder that the LAST planned alpha releases of Python 2.6 > and 3.0 are scheduled for next Wednesday, 07-May-2008. Please be > diligent over the next week so that none of your changes break Python. > The stable buildbots look moderately okay, let's see what we can do > about getting them all green: I like to draw some attention to two features for the last alpha: PEP 370: Per user site-packages directory http://www.python.org/dev/peps/pep-0370/ Alternative memory allocation for ints, floats and longs using PyMalloc instead of the current block allocation. The issue has been discussed in great length a few months ago but without a final decision. http://bugs.python.org/issue2039 Christian From p.f.moore at gmail.com Thu May 1 23:37:30 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 1 May 2008 22:37:30 +0100 Subject: [Python-Dev] Module Suggestion: ast In-Reply-To: References: Message-ID: <79990c6b0805011437k312aa0dcha245fab2ef64f8ba@mail.gmail.com> 2008/5/1 Georg Brandl : > Armin Ronacher schrieb: > > I would like to propose a new module for the stdlib for Python 2.6 > > and higher: "ast". > > If there are no further objections, I'll add this to PEP 361 so that the > proposal doesn't get lost. Excuse my confusion over process, but if this is to go into 2.6, does that mean it needs to be ready before the first beta? Or is there a more relaxed schedule for the stdlib (and if so, what is the deadline for the stdlib)? The same question probably applies to the stdlib reorg... Paul. From guido at python.org Fri May 2 00:41:14 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 15:41:14 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: On Thu, May 1, 2008 at 2:42 PM, Terry Reedy wrote: > As I understood this,one needs a diff to comment on. > I can imagine wanting, or wanting others, to be able to comment on a file > or lines of files without making a fake diff (of the file versus itself or > a blank file). Then only one column would be needed. Yeah, this use case is not well supported. In my experience with the internal tool at Google, I don't think that anybody has ever requested that feature, so perhaps in practice it's not so common. I mean, who wants to review a 5000-line file once it's checked in? :-) The right point for such a review (certainly this is the case at Google) is when it goes in. > I presume the current site is for trial purposes. Actually I'm hoping to keep it alive forever, just evolving the functionality based on feedback. > You obviously don't want > hundreds of repositories listed. Repository management is a bit of an open problem. Fortunately, when you use upload.py, you don't need to have a repository listed -- upload.py will specify the correct base URL, especially for repositories hosted at Google. (I should probably figure out how to support SourceForge as well...) > Are you planning, for instance, to > suggest that Google project hosting add a Review tab or link to the project > pages? They've been following my release with interest... > And I followed the link to pages about Rietveld ;-) Thanks. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amcnabb at mcnabbs.org Fri May 2 00:33:54 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Thu, 1 May 2008 16:33:54 -0600 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: <20080501223354.GA16902@mcnabbs.org> On Thu, May 01, 2008 at 12:55:22PM -0700, Guido van Rossum wrote: > > I'm not proud of this, but I don't see a way around it. The > alternative would be to make it a keyword, which seemed excessive > (plus, it would be odd if super() were a keyword when self is not). > There were long discussions about various possible ways to implement > something like this, and they all had their downsides. (The PEP still > isn't fixed to describe the status quo.) I remember some brainstorms about treating more like self. I'm not sure if these were thought through all the way, but I remember seeing something like: class MyClass(Super1, Super2): # This method requires super: @requires_super def __init__(self, super, **kwds): super(**kwds) # This method doesn't require super: def some_method(self): pass I'm sure there are drawbacks, but it fits in my head. Using super in Python 2.0 is verbose but simple. However, I'm a little scared of super in Python 3.0. I guess I'm probably just a wimp. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From guido at python.org Fri May 2 01:29:13 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 16:29:13 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: On Thu, May 1, 2008 at 4:24 PM, Terry Reedy wrote: > > "Guido van Rossum" wrote in message > news:ca471dc20805011541y63dd132eo6e67310eaeea3ffa at mail.gmail.com... > > | On Thu, May 1, 2008 at 2:42 PM, Terry Reedy wrote: > | > As I understood this,one needs a diff to comment on. > | > I can imagine wanting, or wanting others, to be able to comment on a > file > | > or lines of files without making a fake diff (of the file versus > itself or > | > a blank file). Then only one column would be needed. > | > | Yeah, this use case is not well supported. In my experience with the > | internal tool at Google, I don't think that anybody has ever requested > | that feature, so perhaps in practice it's not so common. I mean, who > | wants to review a 5000-line file once it's checked in? :-) The right > | point for such a review (certainly this is the case at Google) is when > | it goes in. > > I am thinking of an entirely different scenario: a package of modules that > are maybe a few hundred lines each and that accompany a book and are meant > for human reading as much or more than for machine execution. > > Or this: 15 minutes ago I was reading a PEP and discovered that a link did > not work. So I find the non-clickable author email at the top and notify > the author with my email program. But how much nicer to double click an > adjacent line and stick the comment in place (and let your system do the > emailing). (I presume the sponsor of an item in your system can remove > no-longer-needed comments.) So I guess I am thinking of your system as one > for collaborative online editing rather than just patch review. I agree that those are all great use cases. Eventually we'll be able to support these; right now though, I'd like to focus on the more immediate need (IMO) of patch reviews. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ndbecker2 at gmail.com Fri May 2 01:37:55 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 01 May 2008 19:37:55 -0400 Subject: [Python-Dev] Invitation to try out open source code review tool References: Message-ID: It would be really nice to see support for some other backends, such as Hg or bzr (which are both written in python), in addition to svn. From guido at python.org Fri May 2 01:45:01 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 16:45:01 -0700 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481A35D8.60604@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> Message-ID: On Thu, May 1, 2008 at 2:27 PM, Christian Heimes wrote: > Barry Warsaw schrieb: > > > This is a reminder that the LAST planned alpha releases of Python 2.6 > > and 3.0 are scheduled for next Wednesday, 07-May-2008. Please be > > diligent over the next week so that none of your changes break Python. > > The stable buildbots look moderately okay, let's see what we can do > > about getting them all green: > > I like to draw some attention to two features for the last alpha: > > PEP 370: Per user site-packages directory > http://www.python.org/dev/peps/pep-0370/ I like this, except one issue: I really don't like the .local directory. I don't see any compelling reason why this needs to be ~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide it from view, especially since the user is expected to manage this explicitly. > Alternative memory allocation for ints, floats and longs using PyMalloc > instead of the current block allocation. The issue has been discussed in > great length a few months ago but without a final decision. > http://bugs.python.org/issue2039 I might look at this later; but it seems to me to be a pure optimization and thus not required to be in before the first beta. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri May 2 01:45:33 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 16:45:33 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: On Thu, May 1, 2008 at 4:37 PM, Neal Becker wrote: > It would be really nice to see support for some other backends, such as Hg > or bzr (which are both written in python), in addition to svn. Once it's open source feel free to add those! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri May 2 01:45:46 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 1 May 2008 19:45:46 -0400 Subject: [Python-Dev] Problems with the new super() In-Reply-To: <20080501223354.GA16902@mcnabbs.org> References: <20080501223354.GA16902@mcnabbs.org> Message-ID: <2339C004-BEC7-4969-AC43-4D99E97085BB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 1, 2008, at 6:33 PM, Andrew McNabb wrote: > On Thu, May 01, 2008 at 12:55:22PM -0700, Guido van Rossum wrote: >> >> I'm not proud of this, but I don't see a way around it. The >> alternative would be to make it a keyword, which seemed excessive >> (plus, it would be odd if super() were a keyword when self is not). >> There were long discussions about various possible ways to implement >> something like this, and they all had their downsides. (The PEP still >> isn't fixed to describe the status quo.) > > I remember some brainstorms about treating more like self. I'm not > sure > if these were thought through all the way, but I remember seeing > something like: > > class MyClass(Super1, Super2): > # This method requires super: > @requires_super > def __init__(self, super, **kwds): > super(**kwds) > > # This method doesn't require super: > def some_method(self): > pass > > I'm sure there are drawbacks, but it fits in my head. Using super in > Python 2.0 is verbose but simple. However, I'm a little scared of > super > in Python 3.0. I guess I'm probably just a wimp. It certainly makes me uncomfortable too. I think of all the alternatives in PEP 3135, I'd probably prefer self.__super__.foo(), except that I'd call it self.super.foo(). Although I don't mind reserving a non-underscore-adorned name for Python 3.0, I could see adopting self.__super__ and using super(self).foo() as a shortcut. To me, that addresses the main rationale of the PEP without the magic (i.e. no need to repeat the class). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSBpWLHEjvBPtnXfVAQJmOAP+NW1tj67Ls+m6PCbF9wYpPRQhT2RJ1210 0QdYxyYz8akY5+I1QJTp3BN5erDLw1sAWGcKVP2phw7Rvb3pXf8FGh/Yg8du7KAg ZAm96xdaNLPiATVDaZZHuoWZ3+S6zUbmx6QtpjU//EAOXhwQCoTdhDme9QyPDI/2 kA+oldSXr+M= =bBRP -----END PGP SIGNATURE----- From guido at python.org Fri May 2 01:46:12 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 16:46:12 -0700 Subject: [Python-Dev] Module properties for C modules In-Reply-To: <481A2F73.5050109@cheimes.de> References: <4818E1CF.7030903@cheimes.de> <481A1AC3.9070506@cheimes.de> <481A2628.2060307@cheimes.de> <481A2F73.5050109@cheimes.de> Message-ID: On Thu, May 1, 2008 at 2:00 PM, Christian Heimes wrote: > Guido van Rossum schrieb: > > > I don't see how it could work if any Python code is executed in the > > module, since code execution uses a dict for globals. > > > > Supporting it only for built-in modules seems too big an exception. > > I came up with the idea in order to replace the setters and getters of > builtin modules such as the sys module. I deliberately excluded Python > modules from the idea. It's going to be too tricky to expose the API to > Python. Then I propose to drop the idea. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From adlaiff6 at gmail.com Fri May 2 01:47:51 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 1 May 2008 19:47:51 -0400 (EDT) Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: On Thu, 1 May 2008, Neal Becker wrote: > It would be really nice to see support for some other backends, such as Hg > or bzr (which are both written in python), in addition to svn. /me starts the clamour for git -- Cheers, Leif From barry at python.org Fri May 2 01:54:40 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 1 May 2008 19:54:40 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> Message-ID: <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 1, 2008, at 7:45 PM, Guido van Rossum wrote: > On Thu, May 1, 2008 at 2:27 PM, Christian Heimes > wrote: >> Barry Warsaw schrieb: >> >>> This is a reminder that the LAST planned alpha releases of Python >>> 2.6 >>> and 3.0 are scheduled for next Wednesday, 07-May-2008. Please be >>> diligent over the next week so that none of your changes break >>> Python. >>> The stable buildbots look moderately okay, let's see what we can do >>> about getting them all green: >> >> I like to draw some attention to two features for the last alpha: >> >> PEP 370: Per user site-packages directory >> http://www.python.org/dev/peps/pep-0370/ > > I like this, except one issue: I really don't like the .local > directory. I don't see any compelling reason why this needs to be > ~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide > it from view, especially since the user is expected to manage this > explicitly. Interesting. I'm of the opposite opinion. I really don't want Python dictating to me what my home directory should look like (a dot file doesn't count because so many tools conspire to hide it from me). I guess there's always $PYTHONUSERBASE, but I think I will not be alone. ;) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSBpYQ3EjvBPtnXfVAQLY+AP/dy7qoQKNEJiKtlwqCtw7LUCMLMQylBX8 DfbIonOnAaKHzjveyswuxVeAEq/C/fxssOGMhyd++H/1koJHjBdIHp47+RgohbHQ 1xCyA6Qj8f6xM3xdCR7lRuIDdjb6Tb/iCIQT/dHLrYxEf+VGUC+xVa3JIXdfJu4s kUYg7tU8SQ8= =xJWG -----END PGP SIGNATURE----- From glyph at divmod.com Fri May 2 02:03:24 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 02 May 2008 00:03:24 -0000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> Message-ID: <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> On 11:45 pm, guido at python.org wrote: >I like this, except one issue: I really don't like the .local >directory. I don't see any compelling reason why this needs to be >~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide >it from view, especially since the user is expected to manage this >explicitly. I've previously given a spirited defense of ~/.local on this list ( http://mail.python.org/pipermail/python-dev/2008-January/076173.html ) among other places. Briefly, "lib" is not the only directory participating in this convention; you've also got the full complement of other stuff that might go into an installation like /usr/local. So, while "lib" might annoy me a little, "bin etc games include lib lib32 man sbin share src" is going to get ugly pretty fast, especially if this is what comes up in Finder or Nautilus or Explorer every time I open a window. If it's going to be a visible directory on the grounds that this is a Python- specific thing that is explicitly *not* participating in a convention with other software, then please call it "~/Python" or something. Am I the only guy who finds software that insists on visible, fixed files in my home directory rude? vmware, for example, wants a "~/vmware" directory, but pretty much every other application I use is nice enough to use dotfiles (even cedega, with a roughly-comparable-to- lib "applications I've installed for you" folder). Put another way - it's trivial to make ~/.local/lib show up by symlinking ~/lib, but you can't make ~/lib disappear, and lots of software ends up looking at ~. From guido at python.org Fri May 2 03:55:56 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 18:55:56 -0700 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> Message-ID: On Thu, May 1, 2008 at 5:03 PM, wrote: > On 11:45 pm, guido at python.org wrote: > > > I like this, except one issue: I really don't like the .local > > directory. I don't see any compelling reason why this needs to be > > ~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide > > it from view, especially since the user is expected to manage this > > explicitly. > > > > I've previously given a spirited defense of ~/.local on this list ( > http://mail.python.org/pipermail/python-dev/2008-January/076173.html ) among > other places. > > Briefly, "lib" is not the only directory participating in this convention; > you've also got the full complement of other stuff that might go into an > installation like /usr/local. So, while "lib" might annoy me a little, "bin > etc games include lib lib32 man sbin share src" is going to get ugly pretty > fast, especially if this is what comes up in Finder or Nautilus or Explorer > every time I open a window. Unless I misread the PEP, there's only going to be a lib subdirectory. Python packages don't put stuff in other places AFAIK. On the Mac, the default Finder window is not your home directory but your Desktop, which is a subdirectory thereof with a markedly public name. In fact, OS X has a whole bunch of reserved names in your home directory, and none of them start with a dot. The rule seems to be that if it contains stuff that the user cares about, it doesn't start with a dot. > If it's going to be a visible directory on the > grounds that this is a Python- specific thing that is explicitly *not* > participating in a convention with other software, then please call it > "~/Python" or something. Much better than ~/.local/ IMO. > Am I the only guy who finds software that insists on visible, fixed files > in my home directory rude? vmware, for example, wants a "~/vmware" > directory, but pretty much every other application I use is nice enough to > use dotfiles (even cedega, with a roughly-comparable-to- lib "applications > I've installed for you" folder). The distinction to my mind is that most dot files (with the exception of a few like .profile or .bashrc) are not managed by most users -- the apps that manage them provide an APIs for manipulating their contents. (Sort of like thw Windows registry.) Non-dot files are for stuff that the user needs to be aware of. I'm not sure where Python packages fall, but ISTM that this is something a user must explicitly choose as the target of an installer. The user is also likely to have to dig through there to remove stuff, as Python package management doesn't have a way to remove packages. > Put another way - it's trivial to make ~/.local/lib show up by symlinking > ~/lib, That's not the same thing at all. > but you can't make ~/lib disappear, and lots of software ends up > looking at ~. But what software cares about another file there? My home directory is mostly a switching point where I have quick access to everything I access regularly. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri May 2 03:59:26 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 02 May 2008 13:59:26 +1200 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: <481A757E.2050401@canterbury.ac.nz> Facundo Batista wrote: > Has super() proved more useful than harmful? Which is the value for > Py3 to keep it? Personally I've found exactly zero use cases for super() so far in my own code. A couple of times I thought I'd found one, but it turned out not to do quite what I wanted, and I ended up finding better solutions. So if it were up to me, I wouldn't be putting any effort into making it easier to use. I'm actually worried that making it too easy to use could lead people into using it when it's not appropriate. -- Greg From greg.ewing at canterbury.ac.nz Fri May 2 04:07:42 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 02 May 2008 14:07:42 +1200 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: <481A776E.9080908@canterbury.ac.nz> Guido van Rossum wrote: > The > alternative would be to make it a keyword, which seemed excessive > (plus, it would be odd if super() were a keyword when self is not). If it's really such a useful thing as to warrant so much magic to support it, then I think it deserves to have a keyword. Conversely, I would say that if it doesn't deserve a keyword, it also doesn't deserve that much magic. -- Greg From greg.ewing at canterbury.ac.nz Fri May 2 04:12:06 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 02 May 2008 14:12:06 +1200 Subject: [Python-Dev] Problems with the new super() In-Reply-To: <20080501195946.539BC3A4080@sparrow.telecommunity.com> References: <20080501195946.539BC3A4080@sparrow.telecommunity.com> Message-ID: <481A7876.2050606@canterbury.ac.nz> Phillip J. Eby wrote: > (Note, by the way, that you cannot safely write an upcall in a mixin > class without super, so it can't safely be done away with, anyway.) It seems to me you can't safely write one in a mixin class *with* super either. I know that's what it's supposed to be for, but I can't see it working properly except under very special circumstances -- so special that they have never turned up in any code I've written so far. -- Greg From greg.ewing at canterbury.ac.nz Fri May 2 04:21:28 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 02 May 2008 14:21:28 +1200 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: <481A7AA8.8010004@canterbury.ac.nz> Gustavo Carneiro wrote: > A better question would be, is multiple inheritance good or bad for > programs? :-) I would say there are good ways of using it, and bad ways of using it. In my experience, the good ways occur when the classes being mixed together are completely independent -- there is no overlap in method or instance variable names, and each class brings its own independent bundle of functionality to the party. If the classes being mixed clash or overlap in functionality somehow, the inheriting class needs to override all of the clashing methods and properties and resolve matters by delegating to one or another of the inherited classes (using explicit inherited method calls, not super!). If it's not feasible to do that for some reason, then you're better off forgetting about multiple inheritance and finding some other solution to the problem. -- Greg From brett at python.org Fri May 2 04:31:20 2008 From: brett at python.org (Brett Cannon) Date: Thu, 1 May 2008 19:31:20 -0700 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> Message-ID: On Thu, May 1, 2008 at 1:26 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > This is a reminder that the LAST planned alpha releases of Python 2.6 and > 3.0 are scheduled for next Wednesday, 07-May-2008. Please be diligent over > the next week so that none of your changes break Python. The stable > buildbots look moderately okay, let's see what we can do about getting them > all green: > > http://www.python.org/dev/buildbot/stable/ > > We have a few showstopper bugs, and I will be looking at these more > carefully starting next week. > > > http://bugs.python.org/issue?@columns=title,id,activity,versions,status&@sort=activity&@filter=priority,status&@pagesize=50&@startwith=0&priority=1&status=1&@dispname=Showstoppers > > Time is running short to get any new features into Python 2.6 and 3.0. The > release after this one is scheduled to be the first beta release, at which > time we will institute a feature freeze. If your feature doesn't make it in > by then, you'll have to wait until 2.7/3.1. If there is something that > absolutely must go into 2.6/3.0 be sure that there is a bug issue open for > it and that the Priority is set to 'release blocker'. I may reduce it to > critical for the next alpha, but we'll review all the release blocker and > critical issues for the first 2.6 and 3.0 beta releases. I just closed the release blocker I created (the backwards-compatibility issue with warnings.showwarning() ). I would like to add a PendingDeprecationWarning (or stronger) to 2.6 for showwarning() implementations that don't support the optional 'line' argument. I guess the best way to do it in C code would be to see if PyFunction_GetDefaults() returns a tuple of length two (since showwarning() already has a single optional argument as it is). Anyone have an issue with me doing this? Is PendingDeprecationWarning safe enough for 2.6? Or should this be a 3.0-only thing with a DeprecationWarning? -Brett From musiccomposition at gmail.com Fri May 2 04:35:12 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 1 May 2008 21:35:12 -0500 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> Message-ID: <1afaf6160805011935k6dc045bexd6fb54f112f2307@mail.gmail.com> On Thu, May 1, 2008 at 9:31 PM, Brett Cannon wrote: > > I just closed the release blocker I created (the > backwards-compatibility issue with warnings.showwarning() ). I would > like to add a PendingDeprecationWarning (or stronger) to 2.6 for > showwarning() implementations that don't support the optional 'line' > argument. I guess the best way to do it in C code would be to see if > PyFunction_GetDefaults() returns a tuple of length two (since > showwarning() already has a single optional argument as it is). > > Anyone have an issue with me doing this? Is PendingDeprecationWarning > safe enough for 2.6? Or should this be a 3.0-only thing with a > DeprecationWarning? I vote for a full DeprecationWarning. > > -Brett -- Cheers, Benjamin Peterson From g.brandl at gmx.net Fri May 2 05:18:03 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 May 2008 05:18:03 +0200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> Message-ID: Guido van Rossum schrieb: > On Thu, May 1, 2008 at 5:03 PM, wrote: >> On 11:45 pm, guido at python.org wrote: >> >> > I like this, except one issue: I really don't like the .local >> > directory. I don't see any compelling reason why this needs to be >> > ~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide >> > it from view, especially since the user is expected to manage this >> > explicitly. >> > >> >> I've previously given a spirited defense of ~/.local on this list ( >> http://mail.python.org/pipermail/python-dev/2008-January/076173.html ) among >> other places. >> >> Briefly, "lib" is not the only directory participating in this convention; >> you've also got the full complement of other stuff that might go into an >> installation like /usr/local. So, while "lib" might annoy me a little, "bin >> etc games include lib lib32 man sbin share src" is going to get ugly pretty >> fast, especially if this is what comes up in Finder or Nautilus or Explorer >> every time I open a window. > > Unless I misread the PEP, there's only going to be a lib subdirectory. > Python packages don't put stuff in other places AFAIK. Maybe. But when I install other software in my homedir, I install it to ~/.local, precisely to avoid what Glyph said about getting the full set of subdirs, and it would be nice for Python to fit into this scheme. > On the Mac, the default Finder window is not your home directory but > your Desktop, which is a subdirectory thereof with a markedly public > name. In fact, OS X has a whole bunch of reserved names in your home > directory, and none of them start with a dot. The rule seems to be > that if it contains stuff that the user cares about, it doesn't start > with a dot. That's not my rule, and it seems that at least Barry and Glyph agree. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri May 2 05:23:14 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 May 2008 05:23:14 +0200 Subject: [Python-Dev] Module Suggestion: ast In-Reply-To: <79990c6b0805011437k312aa0dcha245fab2ef64f8ba@mail.gmail.com> References: <79990c6b0805011437k312aa0dcha245fab2ef64f8ba@mail.gmail.com> Message-ID: Paul Moore schrieb: > 2008/5/1 Georg Brandl : >> Armin Ronacher schrieb: >> > I would like to propose a new module for the stdlib for Python 2.6 >> > and higher: "ast". >> >> If there are no further objections, I'll add this to PEP 361 so that the >> proposal doesn't get lost. > > Excuse my confusion over process, but if this is to go into 2.6, does > that mean it needs to be ready before the first beta? Or is there a > more relaxed schedule for the stdlib (and if so, what is the deadline > for the stdlib)? > > The same question probably applies to the stdlib reorg... There is no feature freeze before the first beta. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From glyph at divmod.com Fri May 2 05:25:49 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 02 May 2008 03:25:49 -0000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> Message-ID: <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> On 01:55 am, guido at python.org wrote: >On Thu, May 1, 2008 at 5:03 PM, wrote: Hi everybody. I apologize for writing yet another lengthy screed about a simple directory naming issue. I feel strongly about it but I encourate anyone who doesn't to simply skip it. First, some background: my strong feelings here are actually based on an experience I had a long time ago when helping someone with some C++ programming homework. They were baffled because when I helped them the programs compiled, but then as soon as they tried it on their own it didn't. The issue was that I had replicated my own autotools-friendly directory structure for them (at the time, "~/bin", "~/include", "~/lib", "~/etc", and so on managed with GNU stow) onto their machine and edited their shell setup to include them appropriately. But, as soon as I was finished, they "cleaned up" the "mess" I had left behind, and thereby removed all of their build dependencies. This was on a shared university build server, before the days of linux as a friendly, graphical operating system which encouraged you to look even more frequently at your home directory, so if anything I suspect the likelihood that this is a problem would be worse now. Since cleaning up my own home directory, of course, I find that I appreciate the lack of visual noise in Nautilus et. al. as well. Also, while I obviously think all tools should work this way, I think that Python in particular will attract an audience who is learning to program but not necessarily savvy with arcane nuances of filesystem layout, and it would be best if those details were abstracted. My concern here is for the naive python developer reading installation instructions off of a wiki and trying to get started with Twisted development. Seeing a directory created in your home directory (or, as the case may be, 3 directories, "bin", "lib", and "include") is a bit of a surprise. They don't actually care where the files in their installed library are, as long as they're "installed", and they can import them. However, they may care that clicking on the little house icon now shows not "Pictures", "Movies", etc, but "lib" (what's a 'lib'?) "bin" (what's a bin? is that like a box where I throw my stuff?) "share" (I put my stuff in "share", but it's not shared. Wait, I'm supposed to put it in "Public"?). >> Briefly, "lib" is not the only directory participating in this >>convention; >>you've also got the full complement of other stuff that might go into >>an >>installation like /usr/local. So, while "lib" might annoy me a >>little, "bin >>etc games include lib lib32 man sbin share src" is going to get ugly >>pretty >>fast, especially if this is what comes up in Finder or Nautilus or >>Explorer >>every time I open a window. > >Unless I misread the PEP, there's only going to be a lib subdirectory. >Python packages don't put stuff in other places AFAIK. Python packages, at the very least, frequently put stuff in "bin" (or "scripts", I think, on Windows). Not all Python packages are pure- Python packages either; setup.py boasts --install-platlib, --install- headers, --install-data, and --exec-prefix options, which suggests an "include", "bin", and "share" directory, at least. I'm sure if I had more time to grovel around I'd find one that installed manpages. Twisted has some, but apparently setup.py doesn't do anything with them, we leave that to the OS packages... Of course, very little of this is handled by the PEP. But even the usage of the name "lib" implies that the PEP is taking some care to be compatible with an idiom that goes beyond Python itself here, or at least beyond simple Python packages. Even assuming that no Python library ever wanted to install any of these things, there are many Python libraries which are simply wrappers around lower-level libraries, and if I want to perform a per-user install of one of those, I am going to ./configure --prefix=~/something (and by "something", I mean ".local" ;)) and it would be nice to have Python living in the same space. For that matter it'd be nice to get autotools and Ruby and PHP and Perl and Emacs (ad nauseum) all looking at ~/.local as a mirror of /usr, so that I didn't have to write a bunch of shell bootstrap glue to get everything to behave consistently, or learn the new, special names for bits of configuration under "~" that are different from the ones under /usr/local or /etc. I replicate a consistent Python development environment with a ton of bizarre dependencies across something like 15 different OS installations (not to mention a bevy of virtual machines I keep around just for fun), so I think about these issues a lot. Most of these machines are macs and linux boxes, but I do my best on Windows too. FWIW I don't have any idea what the right thing to do is on Windows; ".local" doesn't particularly make sense, but neither does "lib" in that context. There's no reasonable guess as to where to put scripts, or dependent shared libraries... but then, per-user installation is less of an issue on Windows. >On the Mac, the default Finder window is not your home directory but >your Desktop, which is a subdirectory thereof with a markedly public >name. In fact, OS X has a whole bunch of reserved names in your home >directory, and none of them start with a dot. The rule seems to be >that if it contains stuff that the user cares about, it doesn't start >with a dot. Hmm. On my Mac laptop, the default Finder window is definitely my home directory; this may be an artifact of many OS upgrades or some tweak that I performed a long time ago and forgot about, though. Apologies if that is not the average user experience. For what it's worth, Ubuntu also has some directories that it creates: Desktop, Pictures, Documents, Examples, Templates, Videos. These are empty, and I typically delete the ones I don't use. >>If it's going to be a visible directory on the >>grounds that this is a Python- specific thing that is explicitly *not* >>participating in a convention with other software, then please call it >>"~/Python" or something. > >Much better than ~/.local/ IMO. It depends how this is being perceived. If this is Python mirroring the /usr/local layout convention for users, as the name "lib" implies, then this is worse. However, if Python is just trying to select a location for its own library bookkeeping and not allow the installation of platform libraries or scripts using this mechanism... well, ~/.python.d would still be my preference ;-) but I could at least understand "Python" as mirroring the Mac, GNOME and KDE convention for a few very special directories. >> Am I the only guy who finds software that insists on visible, fixed >>files >>in my home directory rude? vmware, for example, wants a "~/vmware" >>directory, but pretty much every other application I use is nice >>enough to >>use dotfiles (even cedega, with a roughly-comparable-to- lib >>"applications >>I've installed for you" folder). > >The distinction to my mind is that most dot files (with the exception >of a few like .profile or .bashrc) are not managed by most users -- >the apps that manage them provide an APIs for manipulating their >contents. (Sort of like thw Windows registry.) Non-dot files are for >stuff that the user needs to be aware of. My experience of modern Linux suggests that the usage you're describing is gradually being phased out - applications that want to manage some non-user-visible storage in something like the registry increasingly use gconf (or a database, in server-land). Granted, gconf itself is stored in dotfiles, but it's just a few. In my home directory I have, in version control, variously written by hand or databases maintained from externally downloaded stuff: ~/.asoundrc ~/.emacs ~/.vimrc ~/.vim ~/.Xresources ~/.fonts ~/.gnomerc ~/.inputrc ~/.bashrc ~/.bash_profile ~/.profile ~/.screenrc ~/.Xresources ~/.ssh/config ~/.ssh/authorized_keys ~/.ssh/known_hosts I know about these dot files and I care about them and I maintain them, but they're there for the benefit of particular pieces of software, not me. There are a lot of other dotfiles there, but I don't think that this set is "a few"; I am quite happy that I don't have to see every one of them every time I am looking at my home directory in a "save as" dialog. >I'm not sure where Python packages fall, but ISTM that this is >something a user must explicitly choose as the target of an installer. >The user is also likely to have to dig through there to remove stuff, >as Python package management doesn't have a way to remove packages. I hope that users never have to explicitly choose this as the target of the installer; I was under the impression that the point of adding this feature was to allow the default behavior of distutils to work simply and automatically on UNIX-y platforms rather than puking about permissions, or requiring arcana like "sudo" access or editing your shell's startup. I am quietly agitating elsewhere to get ~/.local/bin added to $PATH by default, by the way ;-). (~/.local/lib on $LD_LIBRARY_PATH is a hard sell, but that too...) Once you have to know about it and explicitly choose it it's not much more work to set all the appropriate shell environment variables yourself. And, for that matter, *I* already have, so I suppose regardless of the outcome of this discussion I'll still have a ~/.local :-). >> Put another way - it's trivial to make ~/.local/lib show up by >>symlinking >>~/lib, > >That's not the same thing at all. I'm not sure what you're saying it's not the same as. All I'm saying is that if advanced users want to show it, they'll symlink it; if naive users want to hide it, they'll delete it and break python, possibly without knowing why ;). >>but you can't make ~/lib disappear, and lots of software ends up >>looking at ~. > >But what software cares about another file there? My home directory is >mostly a switching point where I have quick access to everything I >access regularly. Nothing's going to break, if that's what you mean. No software processes the list of ~ and does anything with it; but lots of stuff shows me that list. In GNOME, on Ubuntu, when a "choose file" dialog comes up, 80% of the time it comes up by default in my home directory. When I open a terminal it opens in my home directory. The default location for Emacs is my home directory. I can quickly measure my cognitive load by looking at the contents of that directory. Since my shell starts there, autocomplete starts there, and so common-letter real estate is scarce. I have a directory called "Projects" that I currently autocomplete with 'p' and a directory called 'Linux' that I autocomplete with 'l'; either public-name proposal will have me typing an additional letter on these every day ;-). In other words, I care about another file there. I use my home directory as a sort of to-do list; it's mostly empty unless I have a lot going on, in which case it fills up with various objects I'm working on, and then I empty it out again. There are a few exceptions to this rule; on every platform there are a few things the OS puts there, but they are generally things like "Pictures", "Desktop", and "Music"... where I put pictures, downloaded files, and music. The Mac's "Library" directory has never bothered me, since it's OS-provided and basically an alternate location for dotfiles. ("Application Data" and friends are another story.) In a way, I agree with you. "everything I access regularly" is a good description of my home directory. Except, this "lib" directory is not something I want to access regularly; very occasionally, maybe once every few weeks, I want to chuck some dependency in there and then forget about it for a year. From g.brandl at gmx.net Fri May 2 05:28:19 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 02 May 2008 05:28:19 +0200 Subject: [Python-Dev] Problems with the new super() In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > On Thu, May 1, 2008 at 11:20 AM, Georg Brandl wrote: >> But the other two magical things about super() really bother me too. I >> haven't looked at the new super in detail so far (and I don't know how >> many others have), and two things are really strikingly unpythonic in >> my view: >> >> * super() only works when named "super" [1]. It shouldn't be a function if >> it has that property; no other Python function has that. > > Actually, I believe IronPython and/or Jython have to use this trick in > certain cases -- at least I recall Jim Hugunin talking about > generating different code when the use of locals() was detected. I don't know if it's possible in Jython to have "locals" referring to something else. For CPython, the name "super" in a function can refer to anything -- local, global or builtin -- and it just feels wrong for the compiler to make assumptions based on the mere mention of a non-reserved name. > I'm not proud of this, but I don't see a way around it. The > alternative would be to make it a keyword, which seemed excessive > (plus, it would be odd if super() were a keyword when self is not). I don't find it odd. In fact, IMO the whole magic needed for the runtime implementation of "super()" justifies super becoming a keyword. Georg [Moving this to the Python-3000 list] -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Fri May 2 05:49:18 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 1 May 2008 20:49:18 -0700 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> Message-ID: I stand corrected on a few points. You've convinced me that ~/lib/ is wrong. But I still don't like ~/.local/; not in the last place because it's not any more local than any other dot files or directories. The "symmetry" with /usr/local/ is pretty weak, and certainly won't help beginning users. As a compromise, I'm okay with ~/Python/. I would like to be able to say that the user explicitly has to set an environment variable in order to benefit from this feature, just like with $PYTHONPATH and $PYTHONSTARTUP. But that might defeat the point of making this easy to use for noobs. On OS X I think we should put this somewhere under ~/Library/. Just put it in a different place than where the Python framework puts its stuff. On Thu, May 1, 2008 at 8:25 PM, wrote: > On 01:55 am, guido at python.org wrote: > > > On Thu, May 1, 2008 at 5:03 PM, wrote: > > > > Hi everybody. I apologize for writing yet another lengthy screed about a > simple directory naming issue. I feel strongly about it but I encourate > anyone who doesn't to simply skip it. > > First, some background: my strong feelings here are actually based on an > experience I had a long time ago when helping someone with some C++ > programming homework. They were baffled because when I helped them the > programs compiled, but then as soon as they tried it on their own it didn't. > The issue was that I had replicated my own autotools-friendly directory > structure for them (at the time, "~/bin", "~/include", "~/lib", "~/etc", and > so on managed with GNU stow) onto their machine and edited their shell setup > to include them appropriately. But, as soon as I was finished, they > "cleaned up" the "mess" I had left behind, and thereby removed all of their > build dependencies. This was on a shared university build server, before > the days of linux as a friendly, graphical operating system which encouraged > you to look even more frequently at your home directory, so if anything I > suspect the likelihood that this is a problem would be worse now. Since > cleaning up my own home directory, of course, I find that I appreciate the > lack of visual noise in Nautilus et. al. as well. > > Also, while I obviously think all tools should work this way, I think that > Python in particular will attract an audience who is learning to program but > not necessarily savvy with arcane nuances of filesystem layout, and it would > be best if those details were abstracted. > > My concern here is for the naive python developer reading installation > instructions off of a wiki and trying to get started with Twisted > development. Seeing a directory created in your home directory (or, as the > case may be, 3 directories, "bin", "lib", and "include") is a bit of a > surprise. They don't actually care where the files in their installed > library are, as long as they're "installed", and they can import them. > However, they may care that clicking on the little house icon now shows not > "Pictures", "Movies", etc, but "lib" (what's a 'lib'?) "bin" (what's a bin? > is that like a box where I throw my stuff?) "share" (I put my stuff in > "share", but it's not shared. Wait, I'm supposed to put it in "Public"?). > > > > > > > Briefly, "lib" is not the only directory participating in this > convention; > > > you've also got the full complement of other stuff that might go into an > > > installation like /usr/local. So, while "lib" might annoy me a little, > "bin > > > etc games include lib lib32 man sbin share src" is going to get ugly > pretty > > > fast, especially if this is what comes up in Finder or Nautilus or > Explorer > > > every time I open a window. > > > > > > > Unless I misread the PEP, there's only going to be a lib subdirectory. > > Python packages don't put stuff in other places AFAIK. > > > > Python packages, at the very least, frequently put stuff in "bin" (or > "scripts", I think, on Windows). Not all Python packages are pure- Python > packages either; setup.py boasts --install-platlib, --install- headers, > --install-data, and --exec-prefix options, which suggests an "include", > "bin", and "share" directory, at least. I'm sure if I had more time to > grovel around I'd find one that installed manpages. Twisted has some, but > apparently setup.py doesn't do anything with them, we leave that to the OS > packages... > > Of course, very little of this is handled by the PEP. But even the usage > of the name "lib" implies that the PEP is taking some care to be compatible > with an idiom that goes beyond Python itself here, or at least beyond simple > Python packages. > > Even assuming that no Python library ever wanted to install any of these > things, there are many Python libraries which are simply wrappers around > lower-level libraries, and if I want to perform a per-user install of one of > those, I am going to ./configure --prefix=~/something (and by "something", I > mean ".local" ;)) and it would be nice to have Python living in the same > space. For that matter it'd be nice to get autotools and Ruby and PHP and > Perl and Emacs (ad nauseum) all looking at ~/.local as a mirror of /usr, so > that I didn't have to write a bunch of shell bootstrap glue to get > everything to behave consistently, or learn the new, special names for bits > of configuration under "~" that are different from the ones under /usr/local > or /etc. > > I replicate a consistent Python development environment with a ton of > bizarre dependencies across something like 15 different OS installations > (not to mention a bevy of virtual machines I keep around just for fun), so I > think about these issues a lot. Most of these machines are macs and linux > boxes, but I do my best on Windows too. FWIW I don't have any idea what the > right thing to do is on Windows; ".local" doesn't particularly make sense, > but neither does "lib" in that context. There's no reasonable guess as to > where to put scripts, or dependent shared libraries... but then, per-user > installation is less of an issue on Windows. > > > > On the Mac, the default Finder window is not your home directory but > > your Desktop, which is a subdirectory thereof with a markedly public > > name. In fact, OS X has a whole bunch of reserved names in your home > > directory, and none of them start with a dot. The rule seems to be > > that if it contains stuff that the user cares about, it doesn't start > > with a dot. > > > > Hmm. On my Mac laptop, the default Finder window is definitely my home > directory; this may be an artifact of many OS upgrades or some tweak that I > performed a long time ago and forgot about, though. Apologies if that is > not the average user experience. > > For what it's worth, Ubuntu also has some directories that it creates: > Desktop, Pictures, Documents, Examples, Templates, Videos. These are empty, > and I typically delete the ones I don't use. > > > > > > > If it's going to be a visible directory on the > > > grounds that this is a Python- specific thing that is explicitly *not* > > > participating in a convention with other software, then please call it > > > "~/Python" or something. > > > > > > > Much better than ~/.local/ IMO. > > > > It depends how this is being perceived. If this is Python mirroring the > /usr/local layout convention for users, as the name "lib" implies, then this > is worse. However, if Python is just trying to select a location for its > own library bookkeeping and not allow the installation of platform libraries > or scripts using this mechanism... well, ~/.python.d would still be my > preference ;-) but I could at least understand "Python" as mirroring the > Mac, GNOME and KDE convention for a few very special directories. > > > > > > > Am I the only guy who finds software that insists on visible, fixed > files > > > in my home directory rude? vmware, for example, wants a "~/vmware" > > > directory, but pretty much every other application I use is nice enough > to > > > use dotfiles (even cedega, with a roughly-comparable-to- lib > "applications > > > I've installed for you" folder). > > > > > > > The distinction to my mind is that most dot files (with the exception > > of a few like .profile or .bashrc) are not managed by most users -- > > the apps that manage them provide an APIs for manipulating their > > contents. (Sort of like thw Windows registry.) Non-dot files are for > > stuff that the user needs to be aware of. > > > > My experience of modern Linux suggests that the usage you're describing is > gradually being phased out - applications that want to manage some > non-user-visible storage in something like the registry increasingly use > gconf (or a database, in server-land). Granted, gconf itself is stored in > dotfiles, but it's just a few. > > In my home directory I have, in version control, variously written by hand > or databases maintained from externally downloaded stuff: > > ~/.asoundrc > ~/.emacs > ~/.vimrc > ~/.vim > ~/.Xresources > ~/.fonts > ~/.gnomerc > ~/.inputrc > ~/.bashrc > ~/.bash_profile > ~/.profile > ~/.screenrc > ~/.Xresources > ~/.ssh/config > ~/.ssh/authorized_keys > ~/.ssh/known_hosts > > I know about these dot files and I care about them and I maintain them, but > they're there for the benefit of particular pieces of software, not me. > There are a lot of other dotfiles there, but I don't think that this set is > "a few"; I am quite happy that I don't have to see every one of them every > time I am looking at my home directory in a "save as" dialog. > > > > I'm not sure where Python packages fall, but ISTM that this is > > something a user must explicitly choose as the target of an installer. > > The user is also likely to have to dig through there to remove stuff, > > as Python package management doesn't have a way to remove packages. > > > > I hope that users never have to explicitly choose this as the target of the > installer; I was under the impression that the point of adding this feature > was to allow the default behavior of distutils to work simply and > automatically on UNIX-y platforms rather than puking about permissions, or > requiring arcana like "sudo" access or editing your shell's startup. I am > quietly agitating elsewhere to get ~/.local/bin added to $PATH by default, > by the way ;-). (~/.local/lib on $LD_LIBRARY_PATH is a hard sell, but that > too...) > > Once you have to know about it and explicitly choose it it's not much more > work to set all the appropriate shell environment variables yourself. And, > for that matter, *I* already have, so I suppose regardless of the outcome of > this discussion I'll still have a ~/.local :-). > > > > > > > Put another way - it's trivial to make ~/.local/lib show up by > symlinking > > > ~/lib, > > > > > > > That's not the same thing at all. > > > > I'm not sure what you're saying it's not the same as. All I'm saying is > that if advanced users want to show it, they'll symlink it; if naive users > want to hide it, they'll delete it and break python, possibly without > knowing why ;). > > > > > > > but you can't make ~/lib disappear, and lots of software ends up > > > looking at ~. > > > > > > > But what software cares about another file there? My home directory is > > mostly a switching point where I have quick access to everything I > > access regularly. > > > > Nothing's going to break, if that's what you mean. No software processes > the list of ~ and does anything with it; but lots of stuff shows me that > list. In GNOME, on Ubuntu, when a "choose file" dialog comes up, 80% of the > time it comes up by default in my home directory. When I open a terminal it > opens in my home directory. The default location for Emacs is my home > directory. I can quickly measure my cognitive load by looking at the > contents of that directory. Since my shell starts there, autocomplete > starts there, and so common-letter real estate is scarce. I have a > directory called "Projects" that I currently autocomplete with 'p' and > a directory called 'Linux' that I autocomplete with 'l'; either > public-name proposal will have me typing an additional letter on these every > day ;-). > > In other words, I care about another file there. I use my home directory > as a sort of to-do list; it's mostly empty unless I have a lot going on, in > which case it fills up with various objects I'm working on, and then I empty > it out again. There are a few exceptions to this rule; on every platform > there are a few things the OS puts there, but they are generally things like > "Pictures", "Desktop", and "Music"... where I put pictures, downloaded > files, and music. The Mac's "Library" directory has never bothered me, since > it's OS-provided and basically an alternate location for dotfiles. > ("Application Data" and friends are another story.) > > In a way, I agree with you. "everything I access regularly" is a good > description of my home directory. Except, this "lib" directory is not > something I want to access regularly; very occasionally, maybe once every > few weeks, I want to chuck some dependency in there and then forget about it > for a year. > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jflatow at northwestern.edu Fri May 2 05:46:51 2008 From: jflatow at northwestern.edu (Jared Flatow) Date: Thu, 1 May 2008 22:46:51 -0500 Subject: [Python-Dev] Problems with the new super() In-Reply-To: <481A7AA8.8010004@canterbury.ac.nz> References: <481A7AA8.8010004@canterbury.ac.nz> Message-ID: <2CE5930E-A585-4D82-B70E-3F043B7EB151@northwestern.edu> On May 1, 2008, at 9:21 PM, Greg Ewing wrote: > If the classes being mixed clash or overlap in functionality > somehow, the inheriting class needs to override all of the > clashing methods and properties and resolve matters by > delegating to one or another of the inherited classes > (using explicit inherited method calls, not super!). Sorry but thats just not fair. It's not about convenience for the classes you're inheriting from, its so the inheriting class and superclasses can cooperate without requiring implementation details of one another. I agree that if your methods are 'clashing' then you are probably misinheriting, but cooperative methods can be the most natural way to model certain situations. Not to be dull but when building up a method from smaller reusable bits of functionality this is the case. Once you have explicitly stated the classes you want to inherit from, it isn't magic to expect some of their methods to chain without explicitly restating how exactly they should do that. In fact, doing so may make you more likely to make an error when you change which bits you want to include. If the method resolution order is there (and explicit somewhere) you might as well make use of it. > If it's not feasible to do that for some reason, then > you're better off forgetting about multiple inheritance > and finding some other solution to the problem. Is it an issue of feasibility, or of what is the 'most obvious' solution? jared From asmodai at in-nomine.org Fri May 2 07:07:20 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 2 May 2008 07:07:20 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> Message-ID: <20080502050720.GO78165@nexus.in-nomine.org> -On [20080501 22:27], Barry Warsaw (barry at python.org) wrote: >Time is running short to get any new features into Python 2.6 and >3.0. Is there a reliable way to identify 32-bits and 64-bits Windows from within Python? I have not found any yet, but it might be a mere oversight on my behalf. The reason I ask is that both return win32, which is most likely a reference to the API, even when having installed the 64 bits Python version. This, of course, by using win32 causes some issues with, for example, setuptools since it generate an egg with a win32 identifier. Now if you have Python C extension code it will be 64-bit compiled, thus not working on 32-bits Windows. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B All are lunatics, but he who can analyze his delusions is called a philosopher. From glyph at divmod.com Fri May 2 07:48:17 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 02 May 2008 05:48:17 -0000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> Message-ID: <20080502054817.25821.967243921.divmod.xquotient.7459@joule.divmod.com> On 03:49 am, guido at python.org wrote: >I stand corrected on a few points. You've convinced me that ~/lib/ is >wrong. But I still don't like ~/.local/; not in the last place because >it's not any more local than any other dot files or directories. The >"symmetry" with /usr/local/ is pretty weak, and certainly won't help >beginning users. Why do you say the symmetry is weak? The name might not be that evocative, but the main thrust of what I'm saying is that "~/." should be an autotools-style directory layout. The symmetry I suggest is in exactly that sense; that's what /usr/local is. I don't actually care what "" is, except I (and many others) already use "local" for that value, and the more software that honors it, the better. GNU stow (arguably the king of per-user installation management) suggests ~/local as an autotools --prefix target; the free desktop project implicitly suggests ~/.local (by suggesting ~/.local/share is a place to put the same files that would normally be searched for in /usr/share and /usr/local/share). So the word "local" is just floating around in this meme space; I don't like the word that much, but I don't see that there's a different one which more clearly evokes the concept either. I originally used "~/UNIX" and then ~/.unix, but switched to .local when I noticed other folks doing it. One I've actually seen mentioned a few times is "~/.nix-config", which I certainly don't think is any better. It would help beginning users if ~/.local/bin and ~/.local/lib were honored by the system. I, and other adherents of this idea that it would be nice if users could install source without admin privs, have been suggesting that to distro guys when I (we) can, and I figure in a few years, somebody might bite. If that happens, it will start being *easier* to build stuff from source into a separated location than to need root, stomp on the system, and inevitably break some stuff. Agitating for ~/Python/Platform/Libraries on $LD_LIBRARY_PATH (or equivalent) is a lot harder to do with a straight face. This is the reason I'm bothering to spill so many pixels on this topic; I think it would be great if Python were the first real adopter of this convention, and once *one* project has really gone full bore, each subsequent one is progressively easier to convince. However, if you've made up your mind on ~/Python, I think I've more than made my case at this point, so I'll stop cluttering up the lists :). (By the way, for what it's worth: I _hate_ the bin/lib/etc/man/src/include naming convention mess, but it's a mess which is programmatically honored in like a hundred billion lines of code. This is why I want it supported, but hidden ;).) >As a compromise, I'm okay with ~/Python/. I would like to be able to >say that the user explicitly has to set an environment variable in >order to benefit from this feature, just like with $PYTHONPATH and >$PYTHONSTARTUP. But that might defeat the point of making this easy to >use for noobs. Is there another point? It seems to me that this change is entirely about shared conventions and "works by default" behavior. If you are going to set an environment variable, set PYTHONPATH; it's already much more flexible. ~/Python opens up some new problems though, although perhaps they are trivially resolved: how should this interoperate with distutils? 'Just make "python setup.py --user" do what "python setup.py --prefix ~/.local" would do' is pretty straightforward, but "~/Python" would need a new convention. Should "~/Python" have a "~/Python/Scripts" directory that one could add to $PATH? A "~/Python/Platform" directory, for includes, libraries, other random junk like manpages or HTML docs? ~/Python/2.6/lib, or ~/Python2.6/lib? To be fair, a separate, and purpose-designed Python directory layout might also make certain things neater. For example one could support parallel installation with Python2.6 (or Python/2.6) by giving each a 'lib' and 'bin' directory, and always having the scripts in the 2.6/bin dir invoke the 2.6 interpreter, rather than having separated space for libraries but having to mangle the names of scripts ("twistd8.0-py2.6"). I'd still prefer compatibility-by-convention with other tools, languages, etc, though. In the long term, if everyone followed suit on ~/.local, that would be great. But I don't want a ~/Python, ~/Java, ~/Ruby, ~/PHP, ~/Perl, ~/OCaml and ~/Erlang and a $PATH as long as my arm just so I can run a few applications without system-installing them. >On OS X I think we should put this somewhere under ~/Library/. Just >put it in a different place than where the Python framework puts its >stuff. Isn't the whole point that it should be the same place? Under current Python releases, OS X already has this functionality via ~/Library/Python/2.5/site-packages. Also, I'd strongly suggest supporting both ~/Library (although the existing location seems fine to me) *and* whatever the default is on other platforms; there are already enough points of pain where OS X behaves "kind of like a UNIX, but not really", and every project needs to add these little workarounds and caveats in the documentation. Is there a benefit to be derived from making this situation worse by introducing another such subtlety? From mhammond at skippinet.com.au Fri May 2 07:57:31 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri, 2 May 2008 15:57:31 +1000 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502050720.GO78165@nexus.in-nomine.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> Message-ID: <05bb01c8ac19$6db816c0$49284440$@com.au> > Is there a reliable way to identify 32-bits and 64-bits Windows from > within Python? Not that I'm aware of. 'sys.platform=="win32" and "64 bits" in sys.version' will be reliable when it returns True, but it might be wrong when it returns False (although when it returns False, things will look a lot like a 32bit OS). The best way I can find for the win32 API to tell you this is a combination of the above and the IsWow64Process() (which returns True if you are a 32bit process on a 64bit platform) I'd be interested to know why you care though - ie, how will the behavior of your programs depend on that? The virtualization compatibility hacks which, best I can tell are currently enabled for Python mean that the answer to the question might not be as useful as people might think. But I'm sure valid reasons for wanting to know this exist, so I'd be happy to create a patch which add a new sys.iswow64process() process if desired. Cheers, Mark From greg.ewing at canterbury.ac.nz Fri May 2 07:54:14 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 02 May 2008 17:54:14 +1200 Subject: [Python-Dev] Problems with the new super() In-Reply-To: <2CE5930E-A585-4D82-B70E-3F043B7EB151@northwestern.edu> References: <481A7AA8.8010004@canterbury.ac.nz> <2CE5930E-A585-4D82-B70E-3F043B7EB151@northwestern.edu> Message-ID: <481AAC86.3010308@canterbury.ac.nz> Jared Flatow wrote: > > I agree that if your methods are 'clashing' then you are > probably misinheriting, but cooperative methods can be the most natural > way to model certain situations. I'm not saying that nobody should ever use super, only that it's not the right thing for the situation I was talking about there. What it essentially comes down to is that classes mix well if they were designed to mix well. Keeping their features independent is one way to achieve that. Designing their methods to fit together in a super call chain is another, if you can manage to pull it off. All I was really trying to say is that stating that "multiple inheritance is bad" is far too simplistic. > Is it an issue of feasibility, or of what is the 'most obvious' solution? I can only speak from my own experience, which is that whenever I've had a problem involving multiple inheritance, super() didn't solve it. What did solve it was either refactoring so that the classes being mixed were more independent, or finding another solution that didn't require multiple inheritance. Usually the new solution turned out to be better in other ways as well, so I've come to regard multiple inheritance issues as a code smell suggesting that I need to rethink something. -- Greg From martin at v.loewis.de Fri May 2 08:26:20 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 02 May 2008 08:26:20 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502050720.GO78165@nexus.in-nomine.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> Message-ID: <481AB40C.60104@v.loewis.de> > Is there a reliable way to identify 32-bits and 64-bits Windows from within > Python? I have not found any yet, but it might be a mere oversight on my > behalf. > > The reason I ask is that both return win32, which is most likely a reference > to the API, even when having installed the 64 bits Python version. This, of > course, by using win32 causes some issues with, for example, setuptools > since it generate an egg with a win32 identifier. Now if you have Python C > extension code it will be 64-bit compiled, thus not working on 32-bits > Windows. It seems you don't want to identify whether the Windows installation is a Win64 one, but whether the Python installation is, right? For this, you should use platform.architecture()[0] Regards, Martin From greg at krypto.org Fri May 2 08:50:33 2008 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 1 May 2008 23:50:33 -0700 Subject: [Python-Dev] socket.try_reuse_address() In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E248257AA@EXMBX04.exchhosting.com> References: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2477ED1F@EXMBX04.exchhosting.com> <87D3F9C72FBF214DB39FA4E3FE618CDC6E248257AA@EXMBX04.exchhosting.com> Message-ID: <52dc1c820805012350p3fc773b8q8aaba379500933b4@mail.gmail.com> On Wed, Apr 30, 2008 at 10:42 AM, Trent Nelson wrote: > > if os.name == "nt": > > _socketmethods = _socketmethods + ('ioctl',) > > + _is_windows = True > > +elif os.name == 'java': > > + from java.lang import System > > + _is_windows = 'windows' in System.getProperty('os.name').lower() > > +elif os.name == 'posix' and sys.platform == 'cygwin': > > + _is_windows = True > > Oops, that last line should have been: > > elif os.name == 'ce' or (os.name == 'posix' and sys.platform == 'cygwin'): > _is_windows = True > Why even bother to check os.name == 'posix' doesn't sys.platform == 'cygwin' mean its windows regardless of os.name? -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Fri May 2 09:17:46 2008 From: greg at krypto.org (Gregory P. Smith) Date: Fri, 2 May 2008 00:17:46 -0700 Subject: [Python-Dev] tracker issues? server error Message-ID: <52dc1c820805020017j3d94fee3y3f891cb0789e25de@mail.gmail.com> trying to update a bug I get: Fri May 2 07:17:17 2008: An error occurred. Please check the server log for more infomation. -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmodai at in-nomine.org Fri May 2 09:21:44 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 2 May 2008 09:21:44 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481AB40C.60104@v.loewis.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> <481AB40C.60104@v.loewis.de> Message-ID: <20080502072144.GQ78165@nexus.in-nomine.org> -On [20080502 08:26], "Martin v. L?wis" (martin at v.loewis.de) wrote: >It seems you don't want to identify whether the Windows installation is >a Win64 one, but whether the Python installation is, right? I think we can say with a reasonably certainty that if the Python installation is 64-bits the OS is 64-bits too. So yes, if we can identify the Python installation is 64-bits that would be helpful. >For this, you should use platform.architecture()[0] OK, thanks, that's useful, but please see my response to Mark about the why and why platform.architecture()[0] helps, but not fully though. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Tonight your soul sleeps, but one day you will feel real pain, maybe then you will see me as I am: a fragile wreck on a storm of emotion... From asmodai at in-nomine.org Fri May 2 09:46:01 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 2 May 2008 09:46:01 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <05bb01c8ac19$6db816c0$49284440$@com.au> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> <05bb01c8ac19$6db816c0$49284440$@com.au> Message-ID: <20080502074601.GR78165@nexus.in-nomine.org> -On [20080502 07:57], Mark Hammond (mhammond at skippinet.com.au) wrote: >The best way I can find for the win32 API to tell you this is a combination >of the above and the IsWow64Process() (which returns True if you are a >32bit process on a 64bit platform) Support for IsWow64Process() is quite interesting in itself, so if you can add support for that, cool. :) >I'd be interested to know why you care though - ie, how will the behavior >of your programs depend on that? For Genshi I was mucking around with setuptools to create Windows eggs. My machine at home is an AMD64 box. I have Python 2.5 installed, which is 64 bits on Windows x64. On a normal Unix box an egg for a 32 bits system will result in something like Genshi-0.5dev_r847-py2.5-linux-i686.egg. On an 64-bits system we get Genshi-0.5dev_r847-py2.5-linux-x86_64.egg. This distinction is important since Genshi includes a _speedups.so which is compiled to either 32 or 64-bits. On a 64-bits Windows it will always create an egg with win32 in the name. So both a 32-bits and 64-bits Python will in the end create an egg with win32. So imagine creating a 64-bits egg with compiled _speedups and putting it on PyPI. Any 32-bits Python requesting Genshi will get the egg with 64-bits compiled binaries in it. I think it's obvious this is Not Good(tm). (I already raised the issue on distutils-sig, but it has remained unanswered thus far.) So from what I gather we have no unique way to refer to 32-bits or 64-bits Python aside from the semi-adequate platform.architecture()[0]. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Be yourself is all that you can do. From lists at cheimes.de Fri May 2 10:30:20 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 02 May 2008 10:30:20 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> Message-ID: <481AD11C.4020806@cheimes.de> Guido van Rossum schrieb: > I like this, except one issue: I really don't like the .local > directory. I don't see any compelling reason why this needs to be > ~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide > it from view, especially since the user is expected to manage this > explicitly. The directory name has been commented on by glyph in great length (again). Thanks glyph! I'm all on his side. The base directory for Python related files should be a dot directory in the root directory of the users home dir. I slightly prefer ~/.local/ over other suggestions but I'm also open to ~/.python.d/ Should I wait with the commit until we have agreed on a directory name or do you want me to commit the code now? > I might look at this later; but it seems to me to be a pure > optimization and thus not required to be in before the first beta. Correct, it's an optimization to enhance the memory utilization. Christian From mhammond at skippinet.com.au Fri May 2 10:46:27 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri, 2 May 2008 18:46:27 +1000 Subject: [Python-Dev] Detecting 64bit Windows (was RE: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <20080502074601.GR78165@nexus.in-nomine.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> <05bb01c8ac19$6db816c0$49284440$@com.au> <20080502074601.GR78165@nexus.in-nomine.org> Message-ID: <05f201c8ac31$069c4a20$13d4de60$@com.au> > -On [20080502 07:57], Mark Hammond (mhammond at skippinet.com.au) wrote: > >The best way I can find for the win32 API to tell you this is a > combination > >of the above and the IsWow64Process() (which returns True if you are a > >32bit process on a 64bit platform) > > Support for IsWow64Process() is quite interesting in itself, so if you > can > add support for that, cool. :) OK - I'll make a patch for this tomorrow (unless someone pipes up suggesting it's not a great idea, or that ctypes should be used, etc) > >I'd be interested to know why you care though - ie, how will the > behavior > >of your programs depend on that? > > For Genshi I was mucking around with setuptools to create Windows eggs. > My machine at home is an AMD64 box. I have Python 2.5 installed, which is > 64 bits on Windows x64. > > On a normal Unix box an egg for a 32 bits system will result in > something like Genshi-0.5dev_r847-py2.5-linux-i686.egg. On an 64-bits > system we get > Genshi-0.5dev_r847-py2.5-linux-x86_64.egg. This distinction is > important since Genshi includes a _speedups.so which is compiled > to either 32 or 64-bits. Yeah - see recent changes to support cross-compilation in distutils - particularly http://bugs.python.org/issue2513, and a grep for 'plat_name' in Distutils/command might also help. > So from what I gather we have no unique way to refer to 32-bits or 64- > bits Python aside from the semi-adequate platform.architecture()[0]. >From the above, it's not clear what problem you have other than arranging for setuptools to select the correct binaries and generate an egg with a distinction in the filename. Ie, why isn't platform.architecture() (or possibly distutils.util.get_platform) adequate to have setuptools "do the right thing" in your examples? Unfortunately though, I don't know enough about setuptools to suggest how this might happen. If you go this route, please followup on the distutils list (I must have missed your message there while I was on vacation) Cheers, Mark From steve at holdenweb.com Fri May 2 10:49:17 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 May 2008 04:49:17 -0400 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> Message-ID: <481AD58D.2010201@holdenweb.com> Guido van Rossum wrote: > I stand corrected on a few points. You've convinced me that ~/lib/ is > wrong. But I still don't like ~/.local/; not in the last place because > it's not any more local than any other dot files or directories. The > "symmetry" with /usr/local/ is pretty weak, and certainly won't help > beginning users. > So it's the *name* you don't like rather than the invisibility? > As a compromise, I'm okay with ~/Python/. I would like to be able to > say that the user explicitly has to set an environment variable in > order to benefit from this feature, just like with $PYTHONPATH and > $PYTHONSTARTUP. But that might defeat the point of making this easy to > use for noobs. > Groan. Then everyone else realizes what a "great idea" this is, and we see ~/Perl/, ~/Ruby/, ~/C# (that'll screw the Microsoft users, a directory with a comment market in its name), ~/Lisp/ and the rest? I don't think people would thank us for that in the long term. I'm about +10 on invisibility, for the simple reason that "hiding the mechanism" is the right thing to do for naive users, who are the most likely to screw things up if given the chance and the most likely to be unaware of dot-name directories. If you don't like ~/.local/ then please consider ~/.private/ or ~/.personal/ or something else, but don't gratuitously add a visible subdirectory. > On OS X I think we should put this somewhere under ~/Library/. Just > put it in a different place than where the Python framework puts its > stuff. > Nothing to say about OS X. One day Windows might start to respect the "hidden dot" convention, but perhaps in the interim we could create a (Windows-hidden) ~/.private/? Assuming we could work out where to put it ;-) > On Thu, May 1, 2008 at 8:25 PM, wrote: [much good sense] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Fri May 2 10:49:17 2008 From: steve at holdenweb.com (Steve Holden) Date: Fri, 02 May 2008 04:49:17 -0400 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> Message-ID: <481AD58D.2010201@holdenweb.com> Guido van Rossum wrote: > I stand corrected on a few points. You've convinced me that ~/lib/ is > wrong. But I still don't like ~/.local/; not in the last place because > it's not any more local than any other dot files or directories. The > "symmetry" with /usr/local/ is pretty weak, and certainly won't help > beginning users. > So it's the *name* you don't like rather than the invisibility? > As a compromise, I'm okay with ~/Python/. I would like to be able to > say that the user explicitly has to set an environment variable in > order to benefit from this feature, just like with $PYTHONPATH and > $PYTHONSTARTUP. But that might defeat the point of making this easy to > use for noobs. > Groan. Then everyone else realizes what a "great idea" this is, and we see ~/Perl/, ~/Ruby/, ~/C# (that'll screw the Microsoft users, a directory with a comment market in its name), ~/Lisp/ and the rest? I don't think people would thank us for that in the long term. I'm about +10 on invisibility, for the simple reason that "hiding the mechanism" is the right thing to do for naive users, who are the most likely to screw things up if given the chance and the most likely to be unaware of dot-name directories. If you don't like ~/.local/ then please consider ~/.private/ or ~/.personal/ or something else, but don't gratuitously add a visible subdirectory. > On OS X I think we should put this somewhere under ~/Library/. Just > put it in a different place than where the Python framework puts its > stuff. > Nothing to say about OS X. One day Windows might start to respect the "hidden dot" convention, but perhaps in the interim we could create a (Windows-hidden) ~/.private/? Assuming we could work out where to put it ;-) > On Thu, May 1, 2008 at 8:25 PM, wrote: [much good sense] regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From lists at cheimes.de Fri May 2 10:57:21 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 02 May 2008 10:57:21 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481AD58D.2010201@holdenweb.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> Message-ID: <481AD771.6040802@cheimes.de> Steve Holden schrieb: > Nothing to say about OS X. > > One day Windows might start to respect the "hidden dot" convention, but > perhaps in the interim we could create a (Windows-hidden) ~/.private/? > Assuming we could work out where to put it ;-) Windows and Mac OS X have dedicated directories for application specific libraries. That is ~/Library on Mac and Application Data on Windows. The latter is i18n-ed and called "Anwendungsdaten" in German. Fortunately Windows sets an environment var to the application data directory. Christian From asmodai at in-nomine.org Fri May 2 11:06:22 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 2 May 2008 11:06:22 +0200 Subject: [Python-Dev] Detecting 64bit Windows (was RE: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <05f201c8ac31$069c4a20$13d4de60$@com.au> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> <05bb01c8ac19$6db816c0$49284440$@com.au> <20080502074601.GR78165@nexus.in-nomine.org> <05f201c8ac31$069c4a20$13d4de60$@com.au> Message-ID: <20080502090622.GU78165@nexus.in-nomine.org> -On [20080502 10:46], Mark Hammond (mhammond at skippinet.com.au) wrote: >OK - I'll make a patch for this tomorrow (unless someone pipes up >suggesting it's not a great idea, or that ctypes should be used, etc) Thanks. I am sure something like that might come in handy for our Python-using games developers. And other developers too. ;) >From the above, it's not clear what problem you have other than arranging >for setuptools to select the correct binaries and generate an egg with a >distinction in the filename. Ie, why isn't platform.architecture() (or >possibly distutils.util.get_platform) adequate to have setuptools "do the >right thing" in your examples? Unfortunately though, I don't know enough >about setuptools to suggest how this might happen. Sorry, my bad, yes, the platform.architecture() is helpful, the only problem is that I'd rather settle for a sort of standardized nomenclature for this. Both platforms use the win32 API, so would Windows XP be win32-32 and x64 be win32-64 or something like that? >If you go this route, please followup on the distutils list (I must have >missed your message there while I was on vacation) Will do, since there is another issue that makes using setuptools on 64-bits Windows non-working, at least where it concerns easy_install. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B The taller the bamboo grows, the lower it bends. - Chinese saying From asmodai at in-nomine.org Fri May 2 11:16:33 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 2 May 2008 11:16:33 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481AD771.6040802@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> <481AD771.6040802@cheimes.de> Message-ID: <20080502091633.GV78165@nexus.in-nomine.org> -On [20080502 11:00], Christian Heimes (lists at cheimes.de) wrote: >Windows and Mac OS X have dedicated directories for application specific >libraries. That is ~/Library on Mac and Application Data on Windows. The >latter is i18n-ed and called "Anwendungsdaten" in German. Fortunately >Windows sets an environment var to the application data directory. And Vista has C:\ProgramData\{vendor}\{application}, which is *not* $APPDATA, but $ProgramData. $APPDATA points to C:\Users\{user}\AppData\Roaming on Vista -- which is very different. "Windows uses the Roaming folder for application specific data, such as custom dictionaries, which are machine independent and should roam with the user profile. The AppData\Roaming folder in Windows Vista is the same as the Documents and Settings\username\Application Data folder in Windows XP." I think that's different from what you meant above though, since I doubt you'd want this (the libraries) to roam with the user. See http://download.microsoft.com/download/3/b/a/3ba6d659-6e39-4cd7-b3a2-9c96482f5353/Managing%20Roaming%20User%20Data%20Deployment%20Guide.doc for more background. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Seek not death in the error of your life: and pull not upon yourselves destruction with the works of your hands... From asmodai at in-nomine.org Fri May 2 11:20:08 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 2 May 2008 11:20:08 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481AD58D.2010201@holdenweb.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> Message-ID: <20080502092008.GW78165@nexus.in-nomine.org> -On [20080502 10:50], Steve Holden (steve at holdenweb.com) wrote: >Groan. Then everyone else realizes what a "great idea" this is, and we see >~/Perl/, ~/Ruby/, ~/C# (that'll screw the Microsoft users, a directory with >a comment market in its name), ~/Lisp/ and the rest? I don't think people >would thank us for that in the long term. I'm +1 on just using $HOME/.local, but otherwise $HOME/.python makes sense too. $HOME/.python.d doesn't do it for me, too clunky (and hardly used if I look at my .files in $HOME). But I agree with Steve that it should be a hidden directory. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Cum angelis et pueris, fideles inveniamur. Quis est iste Rex gloriae..? From lists at cheimes.de Fri May 2 11:44:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 02 May 2008 11:44:26 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502091633.GV78165@nexus.in-nomine.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> <481AD771.6040802@cheimes.de> <20080502091633.GV78165@nexus.in-nomine.org> Message-ID: <481AE27A.10906@cheimes.de> Jeroen Ruigrok van der Werven schrieb: > "Windows uses the Roaming folder for application specific data, such as > custom dictionaries, which are machine independent and should roam with the > user profile. The AppData\Roaming folder in Windows Vista is the same as the > Documents and Settings\username\Application Data folder in Windows XP." > > I think that's different from what you meant above though, since I doubt > you'd want this (the libraries) to roam with the user. In a matter of fact I *want* to roam the libraries. On the other hand this might become an issue if a user roams between a 32bit and 64bit system ... From ocean at m2.ccsnet.ne.jp Fri May 2 11:47:41 2008 From: ocean at m2.ccsnet.ne.jp (ocean) Date: Fri, 2 May 2008 18:47:41 +0900 Subject: [Python-Dev] Should bytearray(b'a')[0] should 1-char or number? Message-ID: <002201c8ac39$908f4180$0200a8c0@whiterabc2znlh> # I tried to post this to bug tracker, but couldn't, so posted here... r62095 says >Fix and enable a skipped test: >with python 2.6, enumerating bytes yields 1-char strings, not numbers. > >Don't merge this into the py3k branch. This is true for bytes, but not for bytearray. >>> bytearray(b'a')[0] 97 [28493 refs] >>> b'a'[0] 'a' And this causes error on my environment like this. ====================================================================== FAIL: testDecoder (__main__.StatefulIncrementalDecoderTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_io.py", line 629, in testDecoder self.assertEquals(d.decode(input, eof), output) AssertionError: u'o.i.a.b.c.d.' != u'abcd.' But strange, I cannot see this error on python.org buildbot. ??? From ncoghlan at gmail.com Fri May 2 12:43:19 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 02 May 2008 20:43:19 +1000 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502092008.GW78165@nexus.in-nomine.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> <20080502092008.GW78165@nexus.in-nomine.org> Message-ID: <481AF047.5050109@gmail.com> Jeroen Ruigrok van der Werven wrote: > -On [20080502 10:50], Steve Holden (steve at holdenweb.com) wrote: >> Groan. Then everyone else realizes what a "great idea" this is, and we see >> ~/Perl/, ~/Ruby/, ~/C# (that'll screw the Microsoft users, a directory with >> a comment market in its name), ~/Lisp/ and the rest? I don't think people >> would thank us for that in the long term. > > I'm +1 on just using $HOME/.local, but otherwise $HOME/.python makes sense > too. $HOME/.python.d doesn't do it for me, too clunky (and hardly used if I > look at my .files in $HOME). > > But I agree with Steve that it should be a hidden directory. This sums up my opinion pretty well. Hidden by default, but easy to expose (e.g. via a local -> .local symlink) for the more experienced users that want it more easily accessible. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From phil at riverbankcomputing.com Fri May 2 13:01:05 2008 From: phil at riverbankcomputing.com (Phil Thompson) Date: Fri, 2 May 2008 12:01:05 +0100 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481AF047.5050109@gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502092008.GW78165@nexus.in-nomine.org> <481AF047.5050109@gmail.com> Message-ID: <200805021201.05545.phil@riverbankcomputing.com> On Friday 02 May 2008, Nick Coghlan wrote: > Jeroen Ruigrok van der Werven wrote: > > -On [20080502 10:50], Steve Holden (steve at holdenweb.com) wrote: > >> Groan. Then everyone else realizes what a "great idea" this is, and we > >> see ~/Perl/, ~/Ruby/, ~/C# (that'll screw the Microsoft users, a > >> directory with a comment market in its name), ~/Lisp/ and the rest? I > >> don't think people would thank us for that in the long term. > > > > I'm +1 on just using $HOME/.local, but otherwise $HOME/.python makes > > sense too. $HOME/.python.d doesn't do it for me, too clunky (and hardly > > used if I look at my .files in $HOME). > > > > But I agree with Steve that it should be a hidden directory. > > This sums up my opinion pretty well. Hidden by default, but easy to > expose (e.g. via a local -> .local symlink) for the more experienced > users that want it more easily accessible. But you can't be serious about using such a generic word as "local" as the name??? At least include the letters "p" and "y" somewhere. Phil From barry at python.org Fri May 2 13:32:52 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 2 May 2008 07:32:52 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502054817.25821.967243921.divmod.xquotient.7459@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <20080502054817.25821.967243921.divmod.xquotient.7459@joule.divmod.com> Message-ID: <68FCCCBB-7DFF-4157-BE40-F816CBA7AA57@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 2, 2008, at 1:48 AM, glyph at divmod.com wrote: > etc, though. In the long term, if everyone followed suit on > ~/.local, that would be great. But I don't want a ~/Python, ~/Java, > ~/Ruby, ~/PHP, ~/Perl, ~/OCaml and ~/Erlang and a $PATH as long as > my arm just so I can run a few applications without system- > installing them. I hate to send a "me too" messages, but I have to say Glyph is exactly right here. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSBr75XEjvBPtnXfVAQIHAgP+JDpOymVEKfFvzZQZd8WtTpY6jsjvntAA 2J38LslMAXJSs3BcRBU/ELcbvTpr/JoEButktAQJCJpIhsmRTV0y3KcS/d/d+Sao 9V3ME2/yZ94qeQheB7jJIhfihNlC7VhG+CjSOMZrRZwm3k2drGGDdfdgGeSGZJOl B6uCEB0i0iI= =gup1 -----END PGP SIGNATURE----- From amauryfa at gmail.com Fri May 2 13:48:48 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Fri, 2 May 2008 13:48:48 +0200 Subject: [Python-Dev] Should bytearray(b'a')[0] should 1-char or number? In-Reply-To: <002201c8ac39$908f4180$0200a8c0@whiterabc2znlh> References: <002201c8ac39$908f4180$0200a8c0@whiterabc2znlh> Message-ID: On Fri, May 2, 2008 at 11:47 AM, ocean wrote: > # I tried to post this to bug tracker, but couldn't, so posted here... > > r62095 says > > >Fix and enable a skipped test: > >with python 2.6, enumerating bytes yields 1-char strings, not numbers. > > > >Don't merge this into the py3k branch. > > This is true for bytes, but not for bytearray. > > >>> bytearray(b'a')[0] > 97 > [28493 refs] > >>> b'a'[0] > 'a' > > And this causes error on my environment like this. > > ====================================================================== > FAIL: testDecoder (__main__.StatefulIncrementalDecoderTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "test_io.py", line 629, in testDecoder > self.assertEquals(d.decode(input, eof), output) > AssertionError: u'o.i.a.b.c.d.' != u'abcd.' > > But strange, I cannot see this error on python.org buildbot. The buildbots don't fail, because StatefulIncrementalDecoderTest is not listed in the run_unittest() call :-(. I found the cause of the problem: in io.py, TextIOWrapper.tell() calls decode() with a bytearray, without any reason it seems. I will take care of this tonight, if bugs.python.org is available. -- Amaury Forgeot d'Arc From glyph at divmod.com Fri May 2 14:43:19 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 02 May 2008 12:43:19 -0000 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <200805021201.05545.phil@riverbankcomputing.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502092008.GW78165@nexus.in-nomine.org> <481AF047.5050109@gmail.com> <200805021201.05545.phil@riverbankcomputing.com> Message-ID: <20080502124319.25821.896278060.divmod.xquotient.7463@joule.divmod.com> On 11:01 am, phil at riverbankcomputing.com wrote: >On Friday 02 May 2008, Nick Coghlan wrote: >>This sums up my opinion pretty well. Hidden by default, but easy to >>expose (e.g. via a local -> .local symlink) for the more experienced >>users that want it more easily accessible. > >But you can't be serious about using such a generic word as "local" as >the >name??? At least include the letters "p" and "y" somewhere. To be clear, the directory being proposed for Python's use here is "~/.local/lib/pythonX.Y/site-packages", not just "~/.local". There's 1 "y" and 2 "p"s in there. From asmodai at in-nomine.org Fri May 2 15:11:31 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Fri, 2 May 2008 15:11:31 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481B0DE6.30406@lemurconsulting.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> <20080502092008.GW78165@nexus.in-nomine.org> <481B0DE6.30406@lemurconsulting.com> Message-ID: <20080502131131.GD78165@nexus.in-nomine.org> -On [20080502 14:49], Richard Boulton (richard at lemurconsulting.com) wrote: >So, on Ubuntu computers at least, it seems likely that a $HOME/.local/ >directory will already exist, with the beginnings of a unix style layout >inside it. On my Ubuntu 8 box: [15:11] [ruigrok at akuma] (0) {0} % ls ~/.local share -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B The only source of knowledge is experience... From ncoghlan at gmail.com Fri May 2 15:19:41 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 02 May 2008 23:19:41 +1000 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <200805021201.05545.phil@riverbankcomputing.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502092008.GW78165@nexus.in-nomine.org> <481AF047.5050109@gmail.com> <200805021201.05545.phil@riverbankcomputing.com> Message-ID: <481B14ED.1000405@gmail.com> Phil Thompson wrote: > On Friday 02 May 2008, Nick Coghlan wrote: >> Jeroen Ruigrok van der Werven wrote: >>> -On [20080502 10:50], Steve Holden (steve at holdenweb.com) wrote: >>>> Groan. Then everyone else realizes what a "great idea" this is, and we >>>> see ~/Perl/, ~/Ruby/, ~/C# (that'll screw the Microsoft users, a >>>> directory with a comment market in its name), ~/Lisp/ and the rest? I >>>> don't think people would thank us for that in the long term. >>> I'm +1 on just using $HOME/.local, but otherwise $HOME/.python makes >>> sense too. $HOME/.python.d doesn't do it for me, too clunky (and hardly >>> used if I look at my .files in $HOME). >>> >>> But I agree with Steve that it should be a hidden directory. >> This sums up my opinion pretty well. Hidden by default, but easy to >> expose (e.g. via a local -> .local symlink) for the more experienced >> users that want it more easily accessible. > > But you can't be serious about using such a generic word as "local" as the > name??? At least include the letters "p" and "y" somewhere. As glyph noted, the idea is that the layout inside ~/.local would be modeled on that used inside /usr/local. A lot of that was thrashed out during the original PEP 370 discussions though - the current discussion relates purely to what would be visible to the user at the home directory level, so the internal layout of the .local directory hadn't really come up. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From exarkun at divmod.com Fri May 2 15:32:49 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 2 May 2008 09:32:49 -0400 Subject: [Python-Dev] warnings.showwarning (was Re: [Python-3000] Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: Message-ID: <20080502133249.6859.2057657874.divmod.quotient.58104@ohm> On Thu, 1 May 2008 19:31:20 -0700, Brett Cannon wrote: > > [snip] > >I just closed the release blocker I created (the >backwards-compatibility issue with warnings.showwarning() ). I would >like to add a PendingDeprecationWarning (or stronger) to 2.6 for >showwarning() implementations that don't support the optional 'line' >argument. I guess the best way to do it in C code would be to see if >PyFunction_GetDefaults() returns a tuple of length two (since >showwarning() already has a single optional argument as it is). Hi Brett, I'm still seeing some strange behavior from the warnings module, This can be observed on the community buildbot for Twisted, for example: http://python.org/dev/buildbot/community/trunk/x86%20Ubuntu%20Hardy%20trunk/builds/171/step-Twisted.zope.stable/0 The log ends with basically all of the warning-related tests in Twisted failing, reporting that no warnings happened. There is also some strange behavior that can be easily observed in the REPL: exarkun at boson:~/Projects/python/trunk$ ./python /home/exarkun/Projects/Divmod/trunk/Combinator/combinator/xsite.py:7: DeprecationWarning: the sets module is deprecated from sets import Set Python 2.6a2+ (trunk:62636M, May 2 2008, 09:19:41) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import warnings >>> warnings.warn("foo") :1: UserWarning: foo # Where'd the module name go? >>> def f(*a): ... print a ... >>> warnings.showwarning = f >>> warnings.warn("foo") >>> # Where'd the warning go? Any ideas on this? Jean-Paul From guido at python.org Fri May 2 15:56:33 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 2 May 2008 06:56:33 -0700 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481AD11C.4020806@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <481AD11C.4020806@cheimes.de> Message-ID: I'm withdrawing my opposition in the light of the sheer number of words that have already been written with this. On Fri, May 2, 2008 at 1:30 AM, Christian Heimes wrote: > Guido van Rossum schrieb: > > > I like this, except one issue: I really don't like the .local > > directory. I don't see any compelling reason why this needs to be > > ~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide > > it from view, especially since the user is expected to manage this > > explicitly. > > The directory name has been commented on by glyph in great length > (again). Thanks glyph! I'm all on his side. The base directory for > Python related files should be a dot directory in the root directory of > the users home dir. I slightly prefer ~/.local/ over other suggestions > but I'm also open to ~/.python.d/ > > Should I wait with the commit until we have agreed on a directory name > or do you want me to commit the code now? > > > > I might look at this later; but it seems to me to be a pure > > optimization and thus not required to be in before the first beta. > > Correct, it's an optimization to enhance the memory utilization. > > Christian > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri May 2 15:59:50 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 2 May 2008 06:59:50 -0700 Subject: [Python-Dev] Should bytearray(b'a')[0] should 1-char or number? In-Reply-To: <002201c8ac39$908f4180$0200a8c0@whiterabc2znlh> References: <002201c8ac39$908f4180$0200a8c0@whiterabc2znlh> Message-ID: On Fri, May 2, 2008 at 2:47 AM, ocean wrote: > # I tried to post this to bug tracker, but couldn't, so posted here... Was this because you can't figure out how to get a username, or because it was down? It's up now; to create a username, use this form: http://bugs.python.org/user?@template=register (most fields are optional :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Fri May 2 15:59:46 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 02 May 2008 23:59:46 +1000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> Message-ID: <481B1E52.908@gmail.com> Barry Warsaw wrote: > Time is running short to get any new features into Python 2.6 and 3.0. > The release after this one is scheduled to be the first beta release, at > which time we will institute a feature freeze. If your feature doesn't > make it in by then, you'll have to wait until 2.7/3.1. If there is > something that absolutely must go into 2.6/3.0 be sure that there is a > bug issue open for it and that the Priority is set to 'release > blocker'. I may reduce it to critical for the next alpha, but we'll > review all the release blocker and critical issues for the first 2.6 and > 3.0 beta releases. I tried to bump http://bugs.python.org/issue643841 ("New class special method lookup change") up to release blocker, but the bug tracker still appears to be a bit flaky (it keeps giving me an error when I try to submit the change - unfortunately I can't submit anything about it to the metatracker, because I've forgotten my password for it and the metatracker is getting a connection refused when it tries to send the reminder email :P). Here's the comment I was trying to submit along with the bug priority change: """Bumping the priority on this to release blocker for 3.0 - I think we need to have a good answer for the folks who've written old-style __getattr__ based auto-delegating classes before removing old-style classes entirely in 3.0. We could get away with ignoring the issue in the past because people had the option of just using an old-style class rather than having to deal with the difficulties of doing this with a new-style class. With 3.0, that approach is being eliminated. A ProxyMixin class written in Python would address that need (and shouldn't be particularly hard to write), but I'm not sure where it would go in the standard library.""" Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tom at vector-seven.com Fri May 2 16:21:33 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sat, 03 May 2008 00:21:33 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <4818D854.1060906@v.loewis.de> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> Message-ID: <481B236D.80807@vector-seven.com> Martin v. L?wis wrote: >> This leaves us with a few options: >> > > 5. Reuse/Abuse Num(object) for arbitrary constants. > AFAICT, this should work out of the box. > > Eek. It *does* seem like Num would work out of the box, but would this be a good idea? What about *replacing* Num with Const? Might make optimizations specifically for numeric values slightly hairier, and semantically I think they might be different enough to warrant separate AST nodes despite the similarity in implementation at the compiler level. FWIW, I read Num as "numeric literal" and Const as "arbitrary constant", but that's probably only because I've seen the immediate need for constants with non-Num values in the process of writing the AST optimizer. >> 1. A new AST expr node for constant values for types other than Str/Num >> >> I imagine this to be something like Const(PyObject* v), which is >> effectively translated to a "LOAD_CONST v" by the compiler. This trades >> the purity of the AST for a little practicality. A "Const" node has no >> real source representation, it would exist solely for the purpose of >> injecting PyObject constants into the AST. >> > > I think this is the way to go. It doesn't violate purity: it is an > *abstract* syntax, meaning that there doesn't need to be a 1:1 > relationship to source syntax. However, it is still possible to > reproduce source code from this Const node. > > I'm leaning toward this, too. It's dirt simple and quite clean to implement. > I also don't worry about Jython conflicts. The grammar has a version > number precisely so that you can refer to a specific version if you > need to. > > Any Jython folk care to weigh in on this? If there are no major objections I think I'm going to forge ahead with an independant Const() node. Cheers, T From status at bugs.python.org Fri May 2 18:06:49 2008 From: status at bugs.python.org (Tracker) Date: Fri, 2 May 2008 18:06:49 +0200 (CEST) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080502160649.7C5D57821F@psf.upfronthosting.co.za> ACTIVITY SUMMARY (04/25/08 - 05/02/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1850 open (+29) / 12712 closed (+16) / 14562 total (+45) Open issues with patches: 560 Average duration of open issues: 711 days. Median duration of open issues: 1347 days. Open Issues Breakdown open 1827 (+29) pending 23 ( +0) Issues Created Or Reopened (45) _______________________________ bytes object wrongly exposes writeable buffer interface 04/25/08 CLOSED http://bugs.python.org/issue2692 created theller IDLE doesn't work with Tk 8.5 04/25/08 http://bugs.python.org/issue2693 created gregc patch msilib file names check too strict ? 04/26/08 http://bugs.python.org/issue2694 created cdavid Ignore case when checking algorithm in urllib2 04/26/08 http://bugs.python.org/issue2695 created zathras unicode string does not get freed --> memory leak? 04/26/08 CLOSED http://bugs.python.org/issue2696 created ThurnerRupert Logging ancestors ignored after configuration 04/26/08 CLOSED http://bugs.python.org/issue2697 created acooke Extension module build fails for MinGW: missing vcvarsall.bat 04/26/08 http://bugs.python.org/issue2698 created kermode Exception name improperly indented 04/26/08 CLOSED http://bugs.python.org/issue2699 created brett.cannon document PyNumber_ToBase 04/27/08 CLOSED http://bugs.python.org/issue2700 created benjamin.peterson patch csv.reader accepts string instead of file object (duck typing go 04/27/08 http://bugs.python.org/issue2701 created roysmith pickling of large recursive structures crashes cPickle 04/27/08 http://bugs.python.org/issue2702 created cyhawk SimpleXMLRPCDispatcher.__init__ is not python2.4-backward-compat 04/27/08 http://bugs.python.org/issue2703 created doko patch, patch IDLE: Patch to make PyShell behave more like a Terminal interfac 04/27/08 http://bugs.python.org/issue2704 created serwy incompatible change to warnings.showwarning 04/27/08 CLOSED http://bugs.python.org/issue2705 created exarkun datetime: define division timedelta/timedelta 04/27/08 http://bugs.python.org/issue2706 created webograph patch Tiny fix for os.walk docstring 04/28/08 http://bugs.python.org/issue2707 created Yinon IDLE subprocess error 04/28/08 http://bugs.python.org/issue2708 created Kris tk.rst possibly wrong ? 04/28/08 http://bugs.python.org/issue2709 created gpolo error: (10035, 'The socket operation could not complete without 04/28/08 http://bugs.python.org/issue2710 created dhanlen error: (10035, 'The socket operation could not complete without 04/28/08 CLOSED http://bugs.python.org/issue2711 created dhanlen enhanced ioctl 04/28/08 CLOSED http://bugs.python.org/issue2712 created ndbecker Py3k warn on unicode escapes in raw strings 04/28/08 http://bugs.python.org/issue2713 created benjamin.peterson patch Unable to start IDLE on Windows Server 2003 x64 Edition w/ SP2 04/28/08 CLOSED http://bugs.python.org/issue2714 created qgd000 Remove carbon-specific code from binhex 04/29/08 http://bugs.python.org/issue2715 created brett.cannon Reimplement audioop because of copyright issues 04/29/08 http://bugs.python.org/issue2716 created brett.cannon tempfile.mkstempf 04/29/08 http://bugs.python.org/issue2717 created skip.montanaro patch, patch, easy Logging.basicConfig ignores level=0 04/29/08 CLOSED http://bugs.python.org/issue2718 created vdham Backport next() 04/29/08 CLOSED http://bugs.python.org/issue2719 created georg.brandl patch, 26backport make compiling struct be passed around to all ast helpers 04/29/08 http://bugs.python.org/issue2720 created benjamin.peterson patch unittest.makeSuite undocumented and "obsolete" - but what to use 04/29/08 http://bugs.python.org/issue2721 created slinkp os.getcwd fails for long path names on linux 04/30/08 http://bugs.python.org/issue2722 created exarkun Truncate __len__() at sys.maxsize 04/30/08 http://bugs.python.org/issue2723 created belopolsky patch Error in Python tutorial (http://docs.python.org/tut/node6.html) 04/30/08 CLOSED http://bugs.python.org/issue2724 created sandeepsabnani Handle ASDLSyntaxErrors gracefully 04/30/08 http://bugs.python.org/issue2725 created thomas.lee patch Autodoc's autoclass directive should include constructor docstri 04/30/08 http://bugs.python.org/issue2726 created abhik clarify tp_nextiter behavior in the C API documentation 04/30/08 CLOSED http://bugs.python.org/issue2727 created belopolsky patch Failing decimal doctest 04/30/08 http://bugs.python.org/issue2728 created benjamin.peterson patch Link error in What's New in Python 2.6 04/30/08 CLOSED http://bugs.python.org/issue2729 created humitos patch file readline w+ memory dumps 05/01/08 CLOSED http://bugs.python.org/issue2730 created tempname1234 Documentation error on prime example in tutorial 05/01/08 CLOSED http://bugs.python.org/issue2731 created gls curses.textpad loses characters at the end of lines 05/01/08 http://bugs.python.org/issue2732 created shish mmap resize fails on anonymous memory (Windows) 05/01/08 http://bugs.python.org/issue2733 created kmk 2to3 converts long(itude) argument to int 05/01/08 http://bugs.python.org/issue2734 created crmccreary range: lean and mean 05/01/08 http://bugs.python.org/issue2735 created benjamin.peterson patch datetime needs and "epoch" method 05/01/08 http://bugs.python.org/issue2736 created tebeka Issues Now Closed (47) ______________________ pickling bytes? 183 days http://bugs.python.org/issue1338 alexandre.vassalotti Add VS CRT redist to the MSI installer 146 days http://bugs.python.org/issue1569 loewis Enhancements for mathmodule 136 days http://bugs.python.org/issue1640 tiran patch contextlib.contextmanager does not use functools.wraps 115 days http://bugs.python.org/issue1748 georg.brandl ctypes exposing the pep 3118 buffer interface 91 days http://bugs.python.org/issue1971 theller patch IDLE - autocompletion to support alternate path separators 77 days http://bugs.python.org/issue2061 kbk patch IDLE - autocompletion logic optimization 77 days http://bugs.python.org/issue2062 kbk patch branches/trunk-math patch 59 days http://bugs.python.org/issue2224 marketdickinson patch Update command line docs for issue 1739468 52 days http://bugs.python.org/issue2258 georg.brandl easy Raise a Py3K warning when using a float where an int is expected 42 days http://bugs.python.org/issue2343 brett.cannon 26backport Use of __oct__/__hex__ should raise a Py3K warning 41 days http://bugs.python.org/issue2352 benjamin.peterson 26backport str.format() :n format does not appear to work for int and float 29 days http://bugs.python.org/issue2526 eric.smith Unicode escape sequences not parsed in raw strings. 23 days http://bugs.python.org/issue2541 benjamin.peterson patch Document pickle protocol 3 20 days http://bugs.python.org/issue2558 jcea patch [regression] reading from a urllib2 file descriptor happens byte 21 days http://bugs.python.org/issue2601 mhammond textwrap: bug in 'fix_sentence_endings' option 10 days http://bugs.python.org/issue2635 marketdickinson easy apply() documentation is slightly incorrect 4 days http://bugs.python.org/issue2668 georg.brandl PyErr_WarnPy3k 4 days http://bugs.python.org/issue2671 benjamin.peterson patch Argument rules for callables do not apply when function implemen 3 days http://bugs.python.org/issue2677 georg.brandl cyclic reference in ctypes CFunctionType objects 1 days http://bugs.python.org/issue2682 theller patch Logging Module still failing for %(filename)s, __init__ 2 days http://bugs.python.org/issue2684 vsajip Add -mieee to compile flags, when available 5 days http://bugs.python.org/issue2685 marketdickinson patch SSL example script fails mysteriously on MacOS 1 days http://bugs.python.org/issue2687 janssen Fix indentation in range_item 0 days http://bugs.python.org/issue2689 benjamin.peterson patch Document size_t related long object APIs 1 days http://bugs.python.org/issue2691 georg.brandl patch bytes object wrongly exposes writeable buffer interface 5 days http://bugs.python.org/issue2692 theller unicode string does not get freed --> memory leak? 0 days http://bugs.python.org/issue2696 loewis Logging ancestors ignored after configuration 2 days http://bugs.python.org/issue2697 vsajip Exception name improperly indented 1 days http://bugs.python.org/issue2699 brett.cannon document PyNumber_ToBase 1 days http://bugs.python.org/issue2700 benjamin.peterson patch incompatible change to warnings.showwarning 4 days http://bugs.python.org/issue2705 brett.cannon error: (10035, 'The socket operation could not complete without 0 days http://bugs.python.org/issue2711 draghuram enhanced ioctl 1 days http://bugs.python.org/issue2712 ndbecker Unable to start IDLE on Windows Server 2003 x64 Edition w/ SP2 2 days http://bugs.python.org/issue2714 facundobatista Logging.basicConfig ignores level=0 0 days http://bugs.python.org/issue2718 vsajip Backport next() 1 days http://bugs.python.org/issue2719 georg.brandl patch, 26backport Error in Python tutorial (http://docs.python.org/tut/node6.html) 0 days http://bugs.python.org/issue2724 sandeepsabnani clarify tp_nextiter behavior in the C API documentation 0 days http://bugs.python.org/issue2727 georg.brandl patch Link error in What's New in Python 2.6 0 days http://bugs.python.org/issue2729 benjamin.peterson patch file readline w+ memory dumps 0 days http://bugs.python.org/issue2730 amaury.forgeotdarc Documentation error on prime example in tutorial 0 days http://bugs.python.org/issue2731 georg.brandl urllib and cookie module improvements 1628 days http://bugs.python.org/issue841728 georg.brandl Memory leak in socket.py on Mac OS X 1218 days http://bugs.python.org/issue1092502 mhammond unpack error in getdefaultlocale() on certain locales 1144 days http://bugs.python.org/issue1158909 benjamin.peterson smarter behaviour for home key in IDLE 1087 days http://bugs.python.org/issue1196903 kbk patch httplib reads one byte per system call 620 days http://bugs.python.org/issue1542407 georg.brandl Empty class 'Object' 465 days http://bugs.python.org/issue1637926 alexandre.vassalotti Top Issues Most Discussed (10) ______________________________ 15 Precompute range length 7 days open http://bugs.python.org/issue2690 14 Make range __eq__ work 23 days open http://bugs.python.org/issue2603 12 Backport next() 1 days closed http://bugs.python.org/issue2719 9 incompatible change to warnings.showwarning 4 days closed http://bugs.python.org/issue2705 8 Option to ignore or substitute ~/.pydistutils.cfg 226 days open http://bugs.python.org/issue1180 7 Unable to start IDLE on Windows Server 2003 x64 Edition w/ SP2 2 days closed http://bugs.python.org/issue2714 7 enhanced ioctl 1 days closed http://bugs.python.org/issue2712 7 shutil.copytree glob-style filtering [patch] 12 days open http://bugs.python.org/issue2663 6 Exception name improperly indented 1 days closed http://bugs.python.org/issue2699 5 range: lean and mean 1 days open http://bugs.python.org/issue2735 From martin at v.loewis.de Fri May 2 19:38:48 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 02 May 2008 19:38:48 +0200 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <481B236D.80807@vector-seven.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> Message-ID: <481B51A8.5050609@v.loewis.de> Thomas Lee wrote: > Martin v. L?wis wrote: >>> This leaves us with a few options: >>> >> >> 5. Reuse/Abuse Num(object) for arbitrary constants. >> AFAICT, this should work out of the box. >> >> > Eek. It *does* seem like Num would work out of the box, but would this > be a good idea? No. I suggested it just for completeness. > What about *replacing* Num with Const? Might make optimizations > specifically for numeric values slightly hairier, and semantically I > think they might be different enough to warrant separate AST nodes > despite the similarity in implementation at the compiler level. I think they should be separate. Const would be a mere addition; for compatibility with other uses of the AST, that's actually better than simultaneous removal of Num. Regards, Martin From martin at v.loewis.de Fri May 2 19:45:48 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 02 May 2008 19:45:48 +0200 Subject: [Python-Dev] Detecting 64bit Windows (was RE: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <05f201c8ac31$069c4a20$13d4de60$@com.au> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> <05bb01c8ac19$6db816c0$49284440$@com.au> <20080502074601.GR78165@nexus.in-nomine.org> <05f201c8ac31$069c4a20$13d4de60$@com.au> Message-ID: <481B534C.1040005@v.loewis.de> > OK - I'll make a patch for this tomorrow (unless someone pipes up > suggesting it's not a great idea, or that ctypes should be used, etc) I'd like to pipe that up. It sounds like clutter of the sys module to me. Isn't there some environment variable you can look at? Some registry key? Regards, Martin From martin at v.loewis.de Fri May 2 19:47:21 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 02 May 2008 19:47:21 +0200 Subject: [Python-Dev] Detecting 64bit Windows (was RE: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <20080502090622.GU78165@nexus.in-nomine.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> <05bb01c8ac19$6db816c0$49284440$@com.au> <20080502074601.GR78165@nexus.in-nomine.org> <05f201c8ac31$069c4a20$13d4de60$@com.au> <20080502090622.GU78165@nexus.in-nomine.org> Message-ID: <481B53A9.2080700@v.loewis.de> > Sorry, my bad, yes, the platform.architecture() is helpful, the only problem > is that I'd rather settle for a sort of standardized nomenclature for this. > Both platforms use the win32 API, so would Windows XP be win32-32 and x64 be > win32-64 or something like that? We settled that the architecture name is amd64. "64" is not enough, because it can't tell Itanium and AMD64 apart (which are both Win64). Regards, Martin From fdrake at acm.org Fri May 2 19:53:54 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 2 May 2008 13:53:54 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> Message-ID: On May 1, 2008, at 7:54 PM, Barry Warsaw wrote: > Interesting. I'm of the opposite opinion. I really don't want > Python dictating to me what my home directory should look like (a > dot file doesn't count because so many tools conspire to hide it > from me). I guess there's always $PYTHONUSERBASE, but I think I > will not be alone. ;) Using ~/.local/ for user-managed content doesn't seem right to me at all, because it's hidden by default. If user-local package installs went to ~/ by default (~/bin/ for scripts, ~/lib/python/ or ~/lib/pythonX.Y/ for modules and packages), with a way to set an alternate "prefix" instead of ~/ using a distutils configuration setting, I'd be happy enough. I'd be even happier if there were no default per-user location, but a required configuration setting (in the existing distutils config locations) in order to enable per-user installation. -Fred -- Fred Drake From janssen at parc.com Fri May 2 20:10:27 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 2 May 2008 11:10:27 PDT Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481AD771.6040802@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> <481AD771.6040802@cheimes.de> Message-ID: <08May2.111033pdt."58696"@synergy1.parc.xerox.com> > Windows and Mac OS X have dedicated directories for application specific > libraries. That is ~/Library on Mac and Application Data on Windows. In fact, I had to write code for this, and had to read the specs for each. Here's the code (I've substituted Python for UpLib): if sys.platform == 'darwin': listdir = os.path.expanduser(os.path.join("~", "Library", "Application Support", "org.python")) elif sys.platform == 'win32': if os.environ.has_key('APPDATA'): listdir = os.path.join(os.environ['APPDATA'], 'Python') elif os.environ.has_key('USERPROFILE'): listdir = os.path.join(os.environ['USERPROFILE'], 'Application Data', 'Python') elif os.environ.has_key('HOMEDIR') and os.environ.has_key('HOMEPATH'): listdir = os.path.join(os.environ['HOMEDIR'], os.environ['HOMEPATH'], 'Python') else: listdir = os.path.join(os.path.expanduser("~"), 'Python') else: # pretty much has to be unix listdir = os.path.expanduser(os.path.join("~", ".python")) From glyph at divmod.com Fri May 2 20:34:35 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 02 May 2008 18:34:35 -0000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> Message-ID: <20080502183435.25821.905798949.divmod.xquotient.7501@joule.divmod.com> On 05:53 pm, fdrake at acm.org wrote: >On May 1, 2008, at 7:54 PM, Barry Warsaw wrote: >>Interesting. I'm of the opposite opinion. I really don't want >>Python dictating to me what my home directory should look like (a dot >>file doesn't count because so many tools conspire to hide it from >>me). I guess there's always $PYTHONUSERBASE, but I think I will not >>be alone. ;) >Using ~/.local/ for user-managed content doesn't seem right to me at >all, because it's hidden by default. I don't understand your reason for saying this. Terms like "user" and "manage" are somewhat vague. What sort of experience are you hoping to provide what sort of user with this convention? I hope my earlier explanations were clear as far as the types of users. I believe that the management of ~/.local/ is a subtle question. It will largely be "managed" by simply telling distutils to put files there; I hope, implicitly. In my mind there are 2 types of users who will be "managing" it - newbies, who don't really know what's going on but want "cd mypackage-0.0.1; python setup.py install; python -c 'import mypackage'" (or perhaps even "easy_install mypackage") to work, and advanced users who want to be able to mix-and-match different versions of different packages. Advanced users might already have a PYTHONPATH management (virtual python, virtualenv, combinator, ~/.bashrc hacks, a directory full of symlinks) that already works for them, or be comfortable with inspecting a hidden directory, so ~/.local isn't a problem for them (i.e. us); newbies don't want to see the directory until they already know what's going on. >I'd be even happier if there were no default per-user location, but a >required configuration setting (in the existing distutils config >locations) in order to enable per-user installation. If you're happier without this feature, then perhaps your tastes run counter to a useful implementation of it :). Why wouldn't you want it, though? PYTHONPATH still exists; you don't have to use it, personally. From larry at hastings.org Fri May 2 21:32:37 2008 From: larry at hastings.org (Larry Hastings) Date: Fri, 02 May 2008 12:32:37 -0700 Subject: [Python-Dev] Problems with the new super() In-Reply-To: <2339C004-BEC7-4969-AC43-4D99E97085BB@python.org> References: <20080501223354.GA16902@mcnabbs.org> <2339C004-BEC7-4969-AC43-4D99E97085BB@python.org> Message-ID: <481B6C55.6090107@hastings.org> Barry Warsaw wrote: > I think of all the alternatives in PEP 3135, I'd probably prefer > self.__super__.foo(), except that I'd call it self.super.foo(). I'm not sure that's sufficient. You need to be able to specify a class when using MI. I'd prefer self.super().foo(). self.super() would take one argument, the base class, optional iff self's class has only one base class. Not that it makes a lick of difference, /larry/ From guido at python.org Fri May 2 21:56:35 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 2 May 2008 12:56:35 -0700 Subject: [Python-Dev] Special offer! Ten code reviews Message-ID: I'd like to get some more people trying out codereview.appspot.com, so I'm offering the first 10 people to submit a new patch there for my review to do the review by Monday. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From p.f.moore at gmail.com Fri May 2 22:23:11 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 2 May 2008 21:23:11 +0100 Subject: [Python-Dev] Module Suggestion: ast In-Reply-To: References: <79990c6b0805011437k312aa0dcha245fab2ef64f8ba@mail.gmail.com> Message-ID: <79990c6b0805021323o4e96b60ek7ca033970fb4707b@mail.gmail.com> 2008/5/2 Georg Brandl : > Paul Moore schrieb: > > Excuse my confusion over process, but if this is to go into 2.6, does > > that mean it needs to be ready before the first beta? Or is there a > > more relaxed schedule for the stdlib (and if so, what is the deadline > > for the stdlib)? > > > > The same question probably applies to the stdlib reorg... > > There is no feature freeze before the first beta. No, but new features aren't allowed *after* the first beta. That's what I'm asking, must the proposed ast module and the stdlib reorg be complete before the first beta (currently Jun 04, or about a month away)? Paul. From musiccomposition at gmail.com Fri May 2 23:09:05 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 2 May 2008 16:09:05 -0500 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: <1afaf6160805021409l120fc02ag482af942eead0842@mail.gmail.com> On Thu, May 1, 2008 at 11:41 AM, Guido van Rossum wrote: > Some of you may have seen a video recorded in November 2006 where I > showed off Mondrian, a code review tool that I was developing for > Google (http://www.youtube.com/watch?v=sMql3Di4Kgc). I've always hoped > that I could release Mondrian as open source, but it was not to be: > due to its popularity inside Google, it became more and more tied to > proprietary Google infrastructure like Bigtable, and it remained > limited to Perforce, the commercial revision control system most used > at Google. I was salivating over that video, so I'm really excited be able to try out something like it now. > Don't hesitate to drop me a note with feedback -- note though that > there are a few known issues listed at the end of the Help page. The > Help page is really a wiki, so feel free to improve it! My request at the moment is to let people use their real names for display; my email address does not at all resemble my name. -- Cheers, Benjamin Peterson From guido at python.org Fri May 2 23:25:52 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 2 May 2008 14:25:52 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: <1afaf6160805021409l120fc02ag482af942eead0842@mail.gmail.com> References: <1afaf6160805021409l120fc02ag482af942eead0842@mail.gmail.com> Message-ID: On Fri, May 2, 2008 at 2:09 PM, Benjamin Peterson wrote: > On Thu, May 1, 2008 at 11:41 AM, Guido van Rossum wrote: > > Some of you may have seen a video recorded in November 2006 where I > > showed off Mondrian, a code review tool that I was developing for > > Google (http://www.youtube.com/watch?v=sMql3Di4Kgc). I've always hoped > > that I could release Mondrian as open source, but it was not to be: > > due to its popularity inside Google, it became more and more tied to > > proprietary Google infrastructure like Bigtable, and it remained > > limited to Perforce, the commercial revision control system most used > > at Google. > > I was salivating over that video, so I'm really excited be able to try > out something like it now. > > > > Don't hesitate to drop me a note with feedback -- note though that > > there are a few known issues listed at the end of the Help page. The > > Help page is really a wiki, so feel free to improve it! > > My request at the moment is to let people use their real names for > display; my email address does not at all resemble my name. I've noticed. Surely there's an interesting story there. :-) The feature request is on my TODO list. The design is a bit involved, since I'd have to ask people to register and maintain a userid -> nickname mapping; the Google Account API we're piggybacking on only gives you the email address. Once it's open sourced (Monday?) I'd love to see contributions like this! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From musiccomposition at gmail.com Fri May 2 23:28:55 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Fri, 2 May 2008 16:28:55 -0500 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: <1afaf6160805021409l120fc02ag482af942eead0842@mail.gmail.com> Message-ID: <1afaf6160805021428s397cd8eer425fe9712e21ede2@mail.gmail.com> On Fri, May 2, 2008 at 4:25 PM, Guido van Rossum wrote: > > My request at the moment is to let people use their real names for > > display; my email address does not at all resemble my name. > > I've noticed. Surely there's an interesting story there. :-) Maybe I tell you why next PyCon... One more question: What's the number on the upper right hand corner by my username? -- Cheers, Benjamin Peterson From brett at python.org Fri May 2 23:30:26 2008 From: brett at python.org (Brett Cannon) Date: Fri, 2 May 2008 14:30:26 -0700 Subject: [Python-Dev] tracker issues? server error In-Reply-To: <52dc1c820805020017j3d94fee3y3f891cb0789e25de@mail.gmail.com> References: <52dc1c820805020017j3d94fee3y3f891cb0789e25de@mail.gmail.com> Message-ID: On Fri, May 2, 2008 at 12:17 AM, Gregory P. Smith wrote: > trying to update a bug I get: > > Fri May 2 07:17:17 2008: An error occurred. Please check the server log for > more infomation. > Issues like this should go to the meta-tracker. Fail that, mail tracker-discuss. But I just created an issue without problems so that is at least working. -Brett From guido at python.org Fri May 2 23:39:54 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 2 May 2008 14:39:54 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: <1afaf6160805021428s397cd8eer425fe9712e21ede2@mail.gmail.com> References: <1afaf6160805021409l120fc02ag482af942eead0842@mail.gmail.com> <1afaf6160805021428s397cd8eer425fe9712e21ede2@mail.gmail.com> Message-ID: On Fri, May 2, 2008 at 2:28 PM, Benjamin Peterson wrote: > One more question: What's the number on the upper right hand corner by > my username? It's a debugging counter. It gets reset each time a new service instance is created. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Fri May 2 23:54:55 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 02 May 2008 23:54:55 +0200 Subject: [Python-Dev] Special offer! Ten code reviews In-Reply-To: References: Message-ID: <481B8DAF.3020808@cheimes.de> Guido van Rossum schrieb: > I'd like to get some more people trying out codereview.appspot.com, so > I'm offering the first 10 people to submit a new patch there for my > review to do the review by Monday. Oh gosh! As I read the title I was hoping for a free beer or a signed Guido photography for my heroes collection ... :) Christian From tjreedy at udel.edu Sat May 3 01:03:55 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 2 May 2008 19:03:55 -0400 Subject: [Python-Dev] PEP 8: Discourage named lambdas? Message-ID: Some people write somename = lambda args: expression instead of the more obvious (to most people) and, dare I say, standard def somename(args): return expression The difference in the result (the only one I know of) is that the code and function objects get the generic name '' instead of the more informative (in repr() output or tracebacks) 'somename'. I consider this a disadvantage. In the absence of any compensating advantages (other than the trivial saving of 3 chars), I consider the def form to be the proper Python style to the point I think it should be at least recommended for the stdlib in the Programming Recommendations section of PEP 8. There are currently uses of named lambdas at least in urllib2. This to me is a bad example for new Python programmers. What do our style mavens think? Terry Jan Reedy From guido at python.org Sat May 3 01:08:39 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 2 May 2008 16:08:39 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: Agreed, I tend to pick these out of style reviews at Google. On Fri, May 2, 2008 at 4:03 PM, Terry Reedy wrote: > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression > > The difference in the result (the only one I know of) is that the code and > function objects get the generic name '' instead of the more > informative (in repr() output or tracebacks) 'somename'. I consider this a > disadvantage. > > In the absence of any compensating advantages (other than the trivial > saving of 3 chars), I consider the def form to be the proper Python style > to the point I think it should be at least recommended for the stdlib in > the Programming Recommendations section of PEP 8. > > There are currently uses of named lambdas at least in urllib2. This to me > is a bad example for new Python programmers. > > What do our style mavens think? > > Terry Jan Reedy > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jnoller at gmail.com Sat May 3 01:11:36 2008 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 2 May 2008 19:11:36 -0400 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: +1 from me On May 2, 2008, at 7:03 PM, "Terry Reedy" wrote: > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression > > The difference in the result (the only one I know of) is that the > code and > function objects get the generic name '' instead of the more > informative (in repr() output or tracebacks) 'somename'. I consider > this a > disadvantage. > > In the absence of any compensating advantages (other than the trivial > saving of 3 chars), I consider the def form to be the proper Python > style > to the point I think it should be at least recommended for the > stdlib in > the Programming Recommendations section of PEP 8. > > There are currently uses of named lambdas at least in urllib2. This > to me > is a bad example for new Python programmers. > > What do our style mavens think? > > Terry Jan Reedy > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com From mike.klaas at gmail.com Sat May 3 02:23:27 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Fri, 2 May 2008 17:23:27 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: On 2-May-08, at 4:03 PM, Terry Reedy wrote: > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression > > The difference in the result (the only one I know of) is that the > code and > function objects get the generic name '' instead of the more > informative (in repr() output or tracebacks) 'somename'. I consider > this a > disadvantage. > > In the absence of any compensating advantages (other than the trivial > saving of 3 chars), I consider the def form to be the proper Python > style > to the point I think it should be at least recommended for the > stdlib in > the Programming Recommendations section of PEP 8. > > There are currently uses of named lambdas at least in urllib2. This > to me > is a bad example for new Python programmers. > > What do our style mavens think? I'm not a style maven, but I'll put forward why I don't think this is bad style. Most importantly, these statements can result from sensible changes from what is (I believe) considered good style. For example, consider: registerCallback(lambda: frobnicate(7)) what if there are too places that the callback needs to be registered? registerCallback(lambda: frobnicate(7)) registerCallback2(lambda: frobnicate(7)) DRY leads to factoring this out into a variable in a straightforward manner: callback = lambda: frobnicate(7) registerCallback(callback) registerCallback2(callback) Another thing to consider is that the def() pattern is only possible when the bound variable has no dots. A common pattern for me is to replace an instances method with a lambda to add monitoring hooks or disable certain functionality: inst.get_foo = lambda: FakeFoo() This is not replacable in one line with a def (or without locals() detritius). Assuming this is good style, it seems odd that inst.get_foo = lambda: FakeFoo() is acceptible style, but get_foo = lambda: FakeFoo() isn't. (I also happen to think that the def pattern is less clear in some situations, but that speaks more to personal taste so isn't particularly relevant) -Mike From skip at pobox.com Sat May 3 02:03:20 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 2 May 2008 19:03:20 -0500 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> Message-ID: <18459.43976.85481.758104@montanaro-dyndns-org.local> Fred> If user-local package installs went to ~/ by default ... with a Fred> way to set an alternate "prefix" instead of ~/ using a distutils Fred> configuration setting, I'd be happy enough. +1 from me. Skip From tom at vector-seven.com Sat May 3 06:51:47 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sat, 03 May 2008 14:51:47 +1000 Subject: [Python-Dev] AST Optimization: Branch Elimination in Generator Functions Message-ID: <481BEF63.2030506@vector-seven.com> The next problem that cropped up during the implementation of the AST code optimizer is related to branch elimination and the elimination of any code after a return. Within a FunctionDef node, we would (ideally) like to blow away If nodes with a constant - but false - test expression. e.g.: def foo(): if False: # ... stuff ... For most functions, this will cause no problems and the code will behave as expected. However, if the eliminated branch contains a "yield" expression, the function is actually a generator function - even if the yield expression can never be reached: def foo(): if False: yield 5 In addition to this, the following should also be treated as a generator even though we'd like to be able to get rid of all the code following the "return" statement: def foo(): return yield 5 Again, blowing away the yield results in a normal function instead of a generator. Not what we want: we need to preserve the generator semantics. Upon revisiting this, it's actually made me reconsider the use of a Const node for the earlier problem relating to arbitrary constants. We may be better off with annotations after all ... then we could mark FunctionDef nodes as being generators at the AST level to force the compiler to produce code for a generator, but eliminate the branches anyway. The other alternative I can think of is injecting a yield node somewhere unreachable and ensuring it doesn't get optimized away, but this seems pretty hacky in comparison. Any other ideas? Cheers, Tom From hall.jeff at gmail.com Sat May 3 02:42:29 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Fri, 2 May 2008 20:42:29 -0400 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: <1bc395c10805021742l25931ckd01b81c7a15a2cff@mail.gmail.com> +1 For what it's worth from a newbie -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhammond at skippinet.com.au Sat May 3 02:27:42 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Sat, 3 May 2008 10:27:42 +1000 Subject: [Python-Dev] Detecting 64bit Windows (was RE: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <481B534C.1040005@v.loewis.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <20080502050720.GO78165@nexus.in-nomine.org> <05bb01c8ac19$6db816c0$49284440$@com.au> <20080502074601.GR78165@nexus.in-nomine.org> <05f201c8ac31$069c4a20$13d4de60$@com.au> <481B534C.1040005@v.loewis.de> Message-ID: <068d01c8acb4$86f05fd0$94d11f70$@com.au> > > OK - I'll make a patch for this tomorrow (unless someone pipes up > > suggesting it's not a great idea, or that ctypes should be used, etc) > > I'd like to pipe that up. I thought you might. > It sounds like clutter of the sys module to > me. Fair enough - but no more than getwindowsversion IMO (and it seems it would be used in similar situations to getwindowsversion, in terms of wanting to know fundamental characteristics of the OS.) But I don't personally care - although I added this to pywin32 some time ago, I decided to wait for someone else to request it before submitting it for inclusion in the core. > Isn't there some environment variable you can look at? > Some registry key? Not that I'm aware of. I've uploaded my patch as http://bugs.python.org/issue2745, and I've taken the liberty of adding you to the nosy list, incase you wish to reject it outright (but I'm likely to assume silence for a few weeks on a tracker issue means no serious objections - at least none so serious they can't be rectified by reverting the change) Cheers, Mark From tom at vector-seven.com Sat May 3 04:22:34 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sat, 03 May 2008 12:22:34 +1000 Subject: [Python-Dev] AST Optimization: Branch Elimination in Generator Functions Message-ID: <481BCC6A.7040505@vector-seven.com> The next problem that cropped up during the implementation of the AST code optimizer is related to branch elimination and the elimination of any code after a return. Within a FunctionDef node, we would (ideally) like to blow away If nodes with a constant - but false - test expression. e.g.: def foo(): if False: # ... stuff ... For most functions, this will cause no problems and the code will behave as expected. However, if the eliminated branch contains a "yield" expression, the function is actually a generator function - even if the yield expression can never be reached: def foo(): if False: yield 5 In addition to this, the following should also be treated as a generator even though we'd like to be able to get rid of all the code following the "return" statement: def foo(): return yield 5 Again, blowing away the yield results in a normal function instead of a generator. Not what we want: we need to preserve the generator semantics. Upon revisiting this, it's actually made me reconsider the use of a Const node for the earlier problem relating to arbitrary constants. We may be better off with annotations after all ... then we could mark FunctionDef nodes as being generators at the AST level to force the compiler to produce code for a generator, but eliminate the branches anyway. The other alternative I can think of is injecting a yield node somewhere unreachable and ensuring it doesn't get optimized away, but this seems pretty hacky in comparison. Any other ideas? Cheers, Tom From aleaxit at gmail.com Sat May 3 02:09:10 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Fri, 2 May 2008 17:09:10 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: On Fri, May 2, 2008 at 4:11 PM, Jesse Noller wrote: > +1 from me +2 from me -- of all abuses of lambdas, this one's the worst. Alex From brett at python.org Sat May 3 02:24:27 2008 From: brett at python.org (Brett Cannon) Date: Sat, 3 May 2008 02:24:27 +0200 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: On Sat, May 3, 2008 at 1:03 AM, Terry Reedy wrote: > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression > > The difference in the result (the only one I know of) is that the code and > function objects get the generic name '' instead of the more > informative (in repr() output or tracebacks) 'somename'. I consider this a > disadvantage. > > In the absence of any compensating advantages (other than the trivial > saving of 3 chars), I consider the def form to be the proper Python style > to the point I think it should be at least recommended for the stdlib in > the Programming Recommendations section of PEP 8. > > There are currently uses of named lambdas at least in urllib2. This to me > is a bad example for new Python programmers. > > What do our style mavens think? +1. -Brett From Scott.Daniels at Acm.Org Sat May 3 08:23:59 2008 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri, 02 May 2008 23:23:59 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: Mike Klaas wrote: > ... A common pattern for me is to replace an instances method with a > lambda to add monitoring hooks or disable certain functionality: > inst.get_foo = lambda: FakeFoo() > This is not replacable in one line with a def (or without locals() > detritius). Assuming this is good style, it seems odd that > inst.get_foo = lambda: FakeFoo() > is acceptible style, but > get_foo = lambda: FakeFoo() But surely, none of these are great style, and in fact the lambda lures you into using it. I'd propose a far better use is: inst.get_foo = FakeFoo or get_foo = FakeFoo --Scott David Daniels Scott.Daniels at Acm.Org From mike.klaas at gmail.com Sat May 3 08:32:40 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Fri, 2 May 2008 23:32:40 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: <316E2C74-2941-45FE-9315-D573B524E833@gmail.com> On 2-May-08, at 11:23 PM, Scott David Daniels wrote: > Mike Klaas wrote: >> ... A common pattern for me is to replace an instances method with a >> lambda to add monitoring hooks or disable certain functionality: >> inst.get_foo = lambda: FakeFoo() >> This is not replacable in one line with a def (or without locals() >> detritius). Assuming this is good style, it seems odd that >> inst.get_foo = lambda: FakeFoo() >> is acceptible style, but >> get_foo = lambda: FakeFoo() > But surely, none of these are great style, and in fact the lambda > lures you into using it. > > I'd propose a far better use is: > inst.get_foo = FakeFoo > or > get_foo = FakeFoo Sorry, that was a bad example. It is obviously silly if the return value of the function is callable. -Mike From steve at pearwood.info Sat May 3 02:04:37 2008 From: steve at pearwood.info (Steven) Date: Sat, 3 May 2008 10:04:37 +1000 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: <20080503100437.75928d69.steve@pearwood.info> On Fri, 2 May 2008 19:03:55 -0400 "Terry Reedy" wrote: > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression [...] > There are currently uses of named lambdas at least in urllib2. This to me > is a bad example for new Python programmers. > > What do our style mavens think? Speaking as one of those "some people", my position is that functions created with lambda are first-class objects the same as everything else in Python, and a rule that says "You must not assign a lambda to a name, ever" would be a terrible rule. (And I don't do it to save three characters. I don't do it often, and I can't exactly articulate why I do it, only that I do it when it feels right. It's a style thing.) However, I'm happy for "no named lambdas" to be a guideline or recommendation. I'm even happy for a stronger prohibition to apply to the standard library. I don't dislike named lambdas, but I don't expect others to like them. -- Steven D'Aprano From ncoghlan at gmail.com Sat May 3 11:05:43 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 May 2008 19:05:43 +1000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <18459.43976.85481.758104@montanaro-dyndns-org.local> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> Message-ID: <481C2AE7.9010805@gmail.com> skip at pobox.com wrote: > Fred> If user-local package installs went to ~/ by default ... with a > Fred> way to set an alternate "prefix" instead of ~/ using a distutils > Fred> configuration setting, I'd be happy enough. > > +1 from me. But then we clutter up people's (read *my*) home directory with no way for them to do anything about it. We should stay out of people's way by default, while making it easy for them to poke around if they want to. The ~/.local convention does that, but using ~/ directly does not. The major reasons why I think staying out of people's way by default is important: - for people like me (glyph, Georg, etc), it allows us to keep our home directory organised the way we like it. As far as I am concered, applications can store whatever user-specific configuration and data files they like inside hidden files or directories, but they shouldn't be inflicting any visible files on me that aren't related to things I am working on. - for novice users, the fact that it's hidden helps keep them from deleting it by accident - for experienced users (Barry, skip, etc) that want ~/.local to be more easily accessible, creating a visible ~/local symlink is an utterly trivial exercise. Switching the default to use public directories instead of hidden ones helps the third group at the expense of the first two groups. Given that the third group already has an easy workaround to get the behaviour they want, that seems like a bad trade-off to me. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat May 3 11:11:29 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 May 2008 19:11:29 +1000 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: <481C2C41.8040505@gmail.com> Alex Martelli wrote: > On Fri, May 2, 2008 at 4:11 PM, Jesse Noller wrote: >> +1 from me > > +2 from me -- of all abuses of lambdas, this one's the worst. What Alex said :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pedronis at openend.se Sat May 3 11:32:38 2008 From: pedronis at openend.se (Samuele Pedroni) Date: Sat, 03 May 2008 11:32:38 +0200 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: <481C3136.4080902@openend.se> Terry Reedy wrote: > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression > > The difference in the result (the only one I know of) is that the code and > function objects get the generic name '' instead of the more > informative (in repr() output or tracebacks) 'somename'. I consider this a > disadvantage. > > In the absence of any compensating advantages (other than the trivial > saving of 3 chars), I consider the def form to be the proper Python style > to the point I think it should be at least recommended for the stdlib in > the Programming Recommendations section of PEP 8. > > There are currently uses of named lambdas at least in urllib2. This to me > is a bad example for new Python programmers. > > What do our style mavens think? > I found only an example in my personal recent code: START = "" END = "" TITLEPATTERN = lambda s: "%s" % s this three are later used in a very few .find() and .replace() expressions in the same module. I suppose my point is that while I agree it should be discouraged and is really silly to do it for the few chars gain, it can be used to some effect in very rare cases. From fumanchu at aminus.org Sat May 3 17:01:40 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Sat, 3 May 2008 08:01:40 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <481C3136.4080902@openend.se> References: <481C3136.4080902@openend.se> Message-ID: Samuele Pedroni wrote: > I found only an example in my personal recent code: > > START = "" > END = "" > TITLEPATTERN = lambda s: "%s" % s > > this three are later used in a very few .find() and .replace() > expressions in the same module. I suppose my point is that while > I agree it should be discouraged and is really silly to do it > for the few chars gain, it can be used to some effect in very > rare cases. i.e. anywhere you need a portable expression (as opposed to a portable set of statements). I have a feeling that if it were named 'expr' instead of 'lambda' we wouldn't be having this discussion. Robert Brewer fumanchu at aminus.org From pedronis at openend.se Sat May 3 12:35:09 2008 From: pedronis at openend.se (Samuele Pedroni) Date: Sat, 03 May 2008 12:35:09 +0200 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <481C3E0F.4080308@gmail.com> References: <481C3136.4080902@openend.se> <481C3E0F.4080308@gmail.com> Message-ID: <481C3FDD.8050409@openend.se> Nick Coghlan wrote: > Samuele Pedroni wrote: >> I found only an example in my personal recent code: >> >> START = "" >> END = "" >> TITLEPATTERN = lambda s: "%s" % s >> >> this three are later used in a very few .find() and .replace() >> expressions in the same module. I suppose my point is that while I >> agree it should be discouraged and is really silly to do it for the >> few chars gain, it can be used to some effect in very rare cases. > > The way that's written currently, I think I could *very* easily miss > the fact that TITLEPATTERN is a callable while glancing over the code > (and be very confused if I later saw it being called or passed as a > callback). confused? maybe, "very" seems an overstatement From lists at cheimes.de Sat May 3 13:19:34 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 03 May 2008 13:19:34 +0200 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <20080503100437.75928d69.steve@pearwood.info> References: <20080503100437.75928d69.steve@pearwood.info> Message-ID: <481C4A46.7040708@cheimes.de> Steven schrieb: > Speaking as one of those "some people", my position is that functions > created with lambda are first-class objects the same as everything else > in Python, and a rule that says "You must not assign a lambda to a > name, ever" would be a terrible rule. PEP 8 is for the standard library and Python core only. You can write your own code like you prefer. Nobody is going to enforce the PEP 8 style guide unless you write code for the stdlib. (By the way +1 from me, too). Christian From skip at pobox.com Sat May 3 13:51:40 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 3 May 2008 06:51:40 -0500 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481C2AE7.9010805@gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> Message-ID: <18460.20940.882777.235301@montanaro-dyndns-org.local> Nick> skip at pobox.com wrote: Fred> If user-local package installs went to ~/ by default ... with a Fred> way to set an alternate "prefix" instead of ~/ using a distutils Fred> configuration setting, I'd be happy enough. Skip> +1 from me. Nick> But then we clutter up people's (read *my*) home directory with no Nick> way for them to do anything about it. Fred asked for a --prefix flag (which is what I was voting on). I don't really care what you do by default as long as you give me a way to do it differently. Skip From skip at pobox.com Sat May 3 17:08:32 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 3 May 2008 10:08:32 -0500 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> Message-ID: <18460.32752.462983.25145@montanaro-dyndns-org.local> >> - for experienced users (Barry, skip, etc) that want ~/.local to be >> more easily accessible, creating a visible ~/local symlink is an >> utterly trivial exercise. Barry> Hey Nick, I agree with everything above, except that I'd probably Barry> put myself more in Glyph's camp :). Can't speak for Skip Barry> though... I already install everything in ~/local and just have ~/local/bin in my PATH. If I lived in a truly platform-dependent world I'd add platform-dependent ~/local-plat1, ~/local/plat2, etc directories and extend PATH a bit more. Skip From g.brandl at gmx.net Sat May 3 18:46:11 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 03 May 2008 18:46:11 +0200 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <481C4A46.7040708@cheimes.de> References: <20080503100437.75928d69.steve@pearwood.info> <481C4A46.7040708@cheimes.de> Message-ID: Christian Heimes schrieb: > Steven schrieb: >> Speaking as one of those "some people", my position is that functions >> created with lambda are first-class objects the same as everything else >> in Python, and a rule that says "You must not assign a lambda to a >> name, ever" would be a terrible rule. > > PEP 8 is for the standard library and Python core only. You can write > your own code like you prefer. Nobody is going to enforce the PEP 8 > style guide unless you write code for the stdlib. Well, most sane people I know use it at least as a strong guideline for their own code as well, so we should always consider the impact when changing PEP 8. This specific case though is a good guideline to have. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From barry at python.org Sat May 3 15:10:30 2008 From: barry at python.org (Barry Warsaw) Date: Sat, 3 May 2008 09:10:30 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481C2AE7.9010805@gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 3, 2008, at 5:05 AM, Nick Coghlan wrote: > > The major reasons why I think staying out of people's way by default > is important: > - for people like me (glyph, Georg, etc), it allows us to keep our > home directory organised the way we like it. As far as I am > concered, applications can store whatever user-specific > configuration and data files they like inside hidden files or > directories, but they shouldn't be inflicting any visible files on > me that aren't related to things I am working on. > - for novice users, the fact that it's hidden helps keep them from > deleting it by accident > - for experienced users (Barry, skip, etc) that want ~/.local to be > more easily accessible, creating a visible ~/local symlink is an > utterly trivial exercise. Hey Nick, I agree with everything above, except that I'd probably put myself more in Glyph's camp :). Can't speak for Skip though... - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSBxkSXEjvBPtnXfVAQKSigP/d6HIeQ5QLZR4QZ7GAIttb0d+8JI6PM0e 3E2+br0jZ9IeDwjjCLIAx1kbfgIX56++NGoU7tQqiQtbcapI3H3Vb+X+VSAcs30L ORj709MDtF2oqXSzEHww5HHeKoZiQ8/FfiaZoXrXzqPVP5k9MSZu1zLrT3rpWAUP 8YLFekz/LUA= =l5be -----END PGP SIGNATURE----- From stefan_ml at behnel.de Sat May 3 13:17:57 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 03 May 2008 13:17:57 +0200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: <481C49E5.6080403@behnel.de> Guido van Rossum wrote: > What I'm announcing now is the next best thing: an code review tool > for use with Subversion, inspired by Mondrian and (soon to be) > released as open source. Some of the code is even directly derived > from Mondrian. Most of the code is new though, written using Django > and running on Google App Engine. > > To try it out, go here: > > http://codereview.appspot.com Any chance to try it without signing contracts with Google? Stefan From aleaxit at gmail.com Sat May 3 17:24:54 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 3 May 2008 08:24:54 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <316E2C74-2941-45FE-9315-D573B524E833@gmail.com> References: <316E2C74-2941-45FE-9315-D573B524E833@gmail.com> Message-ID: On Fri, May 2, 2008 at 11:32 PM, Mike Klaas wrote: ... > Sorry, that was a bad example. It is obviously silly if the return value > of the function is callable. ...and yet it's *exactly* what keeps happening to lambda-happy programmers -- in production code as well as examples, and in major/famous projects too. E.g., a simple google code search shows many Zope versions containing "Bucket=lambda:{}" instead of the obvious "Bucket=dict", Chandler with an intricate t = threading.Thread(target=lambda x=activePort:testcon(x),verbose=0) instead of t = threading.Thread(target=testcon, args=(activePort,), verbose=0) SQLAlchemy with "callable_=lambda i: self.setup_loader(i)" instead of "callable_=self.setup_loader" ... apparently the existence of lambda may easily blind one to the fact that one can simply pass a callable. I guess that's inevitable (given lambda's existence... and human nature;-) and about on the same plane as another hatefully redundant construct I find myself having to beat upon over and over in code reviews: if : result = True else: result = False return result vs the simple "return " [[or bool() if it's actually mandatory to return a bool and can't be relied upon to produce one]]. Alex From musiccomposition at gmail.com Sat May 3 19:25:39 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 3 May 2008 12:25:39 -0500 Subject: [Python-Dev] Duplicate tests Message-ID: <1afaf6160805031025m1494ab72h42ea0e90a548e470@mail.gmail.com> test_builtin tests most of the builtin types some how or another. All of these also have complete suites elsewhere in the tests. Shouldn't the tests in test_builtin be moved to the type specific tests? -- Cheers, Benjamin Peterson From ncoghlan at gmail.com Sat May 3 12:27:27 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 May 2008 20:27:27 +1000 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <481C3136.4080902@openend.se> References: <481C3136.4080902@openend.se> Message-ID: <481C3E0F.4080308@gmail.com> Samuele Pedroni wrote: > I found only an example in my personal recent code: > > START = "" > END = "" > TITLEPATTERN = lambda s: "%s" % s > > this three are later used in a very few .find() and .replace() > expressions in the same module. I suppose my point is that while I agree > it should be discouraged and is really silly to do it for the few chars > gain, it can be used to some effect in very rare cases. The way that's written currently, I think I could *very* easily miss the fact that TITLEPATTERN is a callable while glancing over the code (and be very confused if I later saw it being called or passed as a callback). Converting to a def makes it obvious that one of these lines is not like the others: START = "" END = "" def TITLEPATTERN(s): return "%s" % s Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat May 3 20:13:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 May 2008 04:13:08 +1000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> Message-ID: <481CAB34.6040809@gmail.com> Barry Warsaw wrote: > On May 3, 2008, at 5:05 AM, Nick Coghlan wrote: >> - for experienced users (Barry, skip, etc) that want ~/.local to be >> more easily accessible, creating a visible ~/local symlink is an >> utterly trivial exercise. > > Hey Nick, I agree with everything above, except that I'd probably put > myself more in Glyph's camp :). Can't speak for Skip though... I was actually looking at something Fred wrote and managed to misread it as something you had posted - and it turns out Skip was just agreeing with Fred about the 'provide an option to tell distutils to use a different user-specific directory name than the default one' idea, and isn't particularly worried about where the packages go by default. Sorry for the confusion. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tjreedy at udel.edu Sat May 3 20:46:07 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 3 May 2008 14:46:07 -0400 Subject: [Python-Dev] Invitation to try out open source code review tool References: <481C49E5.6080403@behnel.de> Message-ID: "Stefan Behnel" wrote in message news:481C49E5.6080403 at behnel.de... | Any chance to try it without signing contracts with Google? Depends: Yes. You may read anything without an account. No. You may not write anything to a Google hosted service without a Google account, agreeing to standard Terms of Service, and signing in. This will not change. Maybe. You might someday be able to write without a Google account IF Guido can release as open source and IF someone adapts it to work on a different platform (Django with regular DB backend) and then hosts it elsewhere. Of course, any such alternative would probably also require an account and Terms of Service agreement. tjr From adlaiff6 at gmail.com Sat May 3 20:47:27 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sat, 3 May 2008 14:47:27 -0400 (EDT) Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: On Sat, 3 May 2008, Brett Cannon wrote: > On Sat, May 3, 2008 at 1:03 AM, Terry Reedy wrote: > > Some people write > > somename = lambda args: expression > > instead of the more obvious (to most people) and, dare I say, standard > > def somename(args): return expression > > > > The difference in the result (the only one I know of) is that the code and > > function objects get the generic name '' instead of the more > > informative (in repr() output or tracebacks) 'somename'. I consider this a > > disadvantage. > > > > In the absence of any compensating advantages (other than the trivial > > saving of 3 chars), I consider the def form to be the proper Python style > > to the point I think it should be at least recommended for the stdlib in > > the Programming Recommendations section of PEP 8. > > > > There are currently uses of named lambdas at least in urllib2. This to me > > is a bad example for new Python programmers. > > > > What do our style mavens think? > > +1. A superfluous +1 from me too, although I will mention that lists of lambdas have saved my butt more than a few times. -- Cheers, Leif From okkotonushi at googlemail.com Sat May 3 21:02:08 2008 From: okkotonushi at googlemail.com (Robert Schuppenies) Date: Sat, 03 May 2008 21:02:08 +0200 Subject: [Python-Dev] Another GSoC Student Introduction In-Reply-To: <20080430205531.GA15767@vortex.local.net> References: <20080430205531.GA15767@vortex.local.net> Message-ID: <481CB6B0.1020008@googlemail.com> Hi Everybody. My name is Robert Schuppenies and I got accepted for this years Google Summer of Code. I will be working on the Python Core with the Memory Usage Profiler project and my mentor will be Martin von L?wis. I am very happy that I got selected and look forward to work on my project. Many thanks to Martin and everybody else making this possible. cheers, robert From guido at python.org Sat May 3 21:24:23 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 3 May 2008 12:24:23 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: On Sat, May 3, 2008 at 11:46 AM, Terry Reedy wrote: > > "Stefan Behnel" wrote in message > news:481C49E5.6080403 at behnel.de... > > | Any chance to try it without signing contracts with Google? > > Depends: > > Yes. You may read anything without an account. > > No. You may not write anything to a Google hosted service without a Google > account, agreeing to standard Terms of Service, and signing in. This will > not change. That's not true for apps hosted under AppEngine. Apps are not required to request users to log in before they upload things. I chose to do this for the review app because I don't like the idea of anonymous comments and uploads (and the spam danger). > Maybe. You might someday be able to write without a Google account IF Guido > can release as open source It will be released as open source (Apache 2) on Monday. I'm just cleaning up the code a bit. > and IF someone adapts it to work on a different > platform (Django with regular DB backend) and then hosts it elsewhere. No, that won't be necessary. If someone contributes an alternate authentication system I'd be happy to adopt it. > Of > course, any such alternative would probably also require an account and > Terms of Service agreement. You're seeing this to strictly. While developers hosting their app are held by an Google terms of service agreement, they are not required to hold their users to such an agreement. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Sat May 3 21:54:16 2008 From: brett at python.org (Brett Cannon) Date: Sat, 3 May 2008 12:54:16 -0700 Subject: [Python-Dev] Duplicate tests In-Reply-To: <1afaf6160805031025m1494ab72h42ea0e90a548e470@mail.gmail.com> References: <1afaf6160805031025m1494ab72h42ea0e90a548e470@mail.gmail.com> Message-ID: On Sat, May 3, 2008 at 10:25 AM, Benjamin Peterson wrote: > test_builtin tests most of the builtin types some how or another. All > of these also have complete suites elsewhere in the tests. Shouldn't > the tests in test_builtin be moved to the type specific tests? > The test duplication issue has been discussed and needs to be addressed as an overall problem. But in this specific case this sounds fine to me. -Brett From greg at krypto.org Sat May 3 22:20:44 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 3 May 2008 13:20:44 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: <52dc1c820805031320h20db5480g580bf58c570104e9@mail.gmail.com> > > To try it out, go here: > > http://codereview.appspot.com > > Please use the Help link in the top right to read more on how to use > the app. Please sign in using your Google Account (either a Gmail > address or a non-Gmail address registered with Google) to interact > more with the app (you need to be signed in to create new issues and > to add comments to existing issues). > > Don't hesitate to drop me a note with feedback -- note though that > there are a few known issues listed at the end of the Help page. The > Help page is really a wiki, so feel free to improve it! I'd be great to integrate this with the bug tracker so that all submitted patches automagically show up in codereview with links to one another. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Sat May 3 23:28:33 2008 From: brett at python.org (Brett Cannon) Date: Sat, 3 May 2008 14:28:33 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: <52dc1c820805031320h20db5480g580bf58c570104e9@mail.gmail.com> References: <52dc1c820805031320h20db5480g580bf58c570104e9@mail.gmail.com> Message-ID: On Sat, May 3, 2008 at 1:20 PM, Gregory P. Smith wrote: > > > > > > To try it out, go here: > > > > http://codereview.appspot.com > > > > Please use the Help link in the top right to read more on how to use > > the app. Please sign in using your Google Account (either a Gmail > > address or a non-Gmail address registered with Google) to interact > > more with the app (you need to be signed in to create new issues and > > to add comments to existing issues). > > > > Don't hesitate to drop me a note with feedback -- note though that > > there are a few known issues listed at the end of the Help page. The > > Help page is really a wiki, so feel free to improve it! > > I'd be great to integrate this with the bug tracker so that all submitted > patches automagically show up in codereview with links to one another. Yeah, or a simple button to move it over there. Either way some integration would be cool. Since this is being open-sourced we could probably have a python-dev instance installed that has all this bonus functionality for us if we can get an appspot account for the tracker team and there is some API we can tap into in order to create new issues. -Brett From aleaxit at gmail.com Sat May 3 23:35:36 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 3 May 2008 14:35:36 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: <52dc1c820805031320h20db5480g580bf58c570104e9@mail.gmail.com> Message-ID: On Sat, May 3, 2008 at 2:28 PM, Brett Cannon wrote: ... > > I'd be great to integrate this with the bug tracker so that all submitted > > patches automagically show up in codereview with links to one another. > > Yeah, or a simple button to move it over there. Either way some > integration would be cool. > > Since this is being open-sourced we could probably have a python-dev > instance installed that has all this bonus functionality for us if we > can get an appspot account for the tracker team and there is some API > we can tap into in order to create new issues. Given the app engine model, I guess any "API" to an app engine-hosted service would have to be of the RESTful kind, i.e., some URLs to which client-code can POST appropriately. Alex From fdrake at acm.org Sun May 4 01:34:03 2008 From: fdrake at acm.org (Fred Drake) Date: Sat, 3 May 2008 19:34:03 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <18460.20940.882777.235301@montanaro-dyndns-org.local> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> Message-ID: <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> On May 3, 2008, at 7:51 AM, skip at pobox.com wrote: > Fred asked for a --prefix flag (which is what I was voting on). I > don't > really care what you do by default as long as you give me a way to > do it > differently. What's most interesting (to me) is that no one's commented on my note that my preferred approach would be that there's no default at all; the location would have to be specified explicitly. Whether on the command line or in the distutils configuration doesn't matter, but explicitness should be required. -Fred -- Fred Drake From ncoghlan at gmail.com Sun May 4 06:50:45 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 May 2008 14:50:45 +1000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> Message-ID: <481D40A5.1050905@gmail.com> Fred Drake wrote: > On May 3, 2008, at 7:51 AM, skip at pobox.com wrote: >> Fred asked for a --prefix flag (which is what I was voting on). I don't >> really care what you do by default as long as you give me a way to do it >> differently. > > What's most interesting (to me) is that no one's commented on my note > that my preferred approach would be that there's no default at all; the > location would have to be specified explicitly. Whether on the command > line or in the distutils configuration doesn't matter, but explicitness > should be required. I thought Christian said something about that defeating one of the main points of the PEP - to allow per-user installation of modules to "just work" for non-administrators. (It may not have been Christian, and it may not have been directly in response to you, but I'm pretty sure I read it somewhere in this thread ;) Anyway, a per-user site-packages directly only "just works" if the standard behaviour of a Python installation is to provide access to the per-user packages without requiring any additional action on the part of the user. A couple of paragraphs in the PEP may also be of interest to you: """For security reasons the user site directory is not added to sys.path when the effective user id or group id is not equal to the process uid / gid [9]. It's an additional barrier against code injection into suid apps. However Python suid scripts must always use the -E and -s option or users can sneak in their own code. The user site directory can be suppressed with a new option -s or the environment variable PYTHONNOUSERSITE. The feature can be disabled globally by setting site.ENABLE_USER_SITE to the value False. It must be set by editing site.py. It can't be altered in sitecustomize.py or later.""" So Python itself turns the feature off automatically for invocation via sudo and the like, and the sysadmin can disable the feature completely through site.py. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From martin at v.loewis.de Sun May 4 08:13:08 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 04 May 2008 08:13:08 +0200 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: <316E2C74-2941-45FE-9315-D573B524E833@gmail.com> Message-ID: <481D53F4.5060206@v.loewis.de> > I guess that's inevitable (given lambda's existence... and human > nature;-) and about on the same plane as another hatefully redundant > construct I find myself having to beat upon over and over in code > reviews: > > if : > result = True > else: > result = False > return result > > vs the simple "return " [[or bool() if it's > actually mandatory to return a bool and can't be relied > upon to produce one]]. Indeed, I think these are the same phenomenons. If lambdas where not available, people would, instead of writing Thread(target = lambda: foo()) write def target(): foo() Thread(target = target) Your example above demonstrates that the boolean type is a concept that is *very* difficult to grasp (not the notion of boolean expressions, which are easy to understand - it's the boolean *type* that is difficult, i.e. that a boolean expression can be used not just in conditional statements, but also assigned to variables, returned, etc.). When I ask students in the exam to rewrite your code above (and assuming they use Java), so that "it is a single line", they typically write return ? true : false; Likewise, apparently, the notion of generic callables is difficult to understand. Unless you have written it yourself, you won't recognize an expression as callable. Regards, Martin From aahz at pythoncraft.com Sun May 4 01:25:51 2008 From: aahz at pythoncraft.com (Aahz) Date: Sat, 3 May 2008 16:25:51 -0700 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <68FCCCBB-7DFF-4157-BE40-F816CBA7AA57@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <20080502054817.25821.967243921.divmod.xquotient.7459@joule.divmod.com> <68FCCCBB-7DFF-4157-BE40-F816CBA7AA57@python.org> Message-ID: <20080503232550.GB28577@panix.com> On Fri, May 02, 2008, Barry Warsaw wrote: > On May 2, 2008, at 1:48 AM, glyph at divmod.com wrote: >> >>In the long term, if everyone followed suit on >>~/.local, that would be great. But I don't want a ~/Python, ~/Java, >>~/Ruby, ~/PHP, ~/Perl, ~/OCaml and ~/Erlang and a $PATH as long as >>my arm just so I can run a few applications without system- >>installing them. > > I hate to send a "me too" messages, but I have to say Glyph is exactly > right here. +1 -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Help a hearing-impaired person: http://rule6.info/hearing.html From steve at pearwood.info Sun May 4 11:36:50 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 4 May 2008 19:36:50 +1000 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <481C3E0F.4080308@gmail.com> References: <481C3136.4080902@openend.se> <481C3E0F.4080308@gmail.com> Message-ID: <200805041936.51136.steve@pearwood.info> On Sat, 3 May 2008 08:27:27 pm Nick Coghlan wrote: > Samuele Pedroni wrote: > > I found only an example in my personal recent code: > > > > START = "" > > END = "" > > TITLEPATTERN = lambda s: "%s" % s > > > > this three are later used in a very few .find() and .replace() > > expressions in the same module. I suppose my point is that while I > > agree it should be discouraged and is really silly to do it for the > > few chars gain, it can be used to some effect in very rare cases. > > The way that's written currently, I think I could *very* easily miss > the fact that TITLEPATTERN is a callable while glancing over the code > (and be very confused if I later saw it being called or passed as a > callback). I think you're exaggerating a tad here. Why would you be "very confused" when you see TITLEPATTERN() or foo(callback=TITLEPATTERN)? What else would TITLEPATTERN be but a callable when it's being used as a callable? > Converting to a def makes it obvious that one of these lines is not > like the others: Do you get confused by factory functions? (or for that matter callable class instances, types, etc.) TITLEPATTERN = factory(args) # note the lack of def ... lots of code ... foo(callback=TITLEPATTERN) # Note: TITLEPATTERN is a callable I think that if I argued that factory functions were bad because "I think I could *very* easily miss the fact that TITLEPATTERN is a callable while glancing over the code (and be very confused if I later saw it being called or passed as a callback)", most people would dismiss the concern and tell me it was my problem. It certainly isn't a good reason for discouraging factory functions, and I don't think it should be considered a good reason for discouraging lambdas. -- Steven D'Aprano From steve at pearwood.info Sun May 4 11:39:48 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 4 May 2008 19:39:48 +1000 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: <316E2C74-2941-45FE-9315-D573B524E833@gmail.com> Message-ID: <200805041939.48379.steve@pearwood.info> On Sun, 4 May 2008 01:24:54 am Alex Martelli wrote: > On Fri, May 2, 2008 at 11:32 PM, Mike Klaas > wrote: ... > > > Sorry, that was a bad example. It is obviously silly if the > > return value of the function is callable. > > ...and yet it's *exactly* what keeps happening to lambda-happy > programmers -- in production code as well as examples, and in > major/famous projects too. E.g., a simple google code search shows > many Zope versions containing "Bucket=lambda:{}" instead of the > obvious "Bucket=dict", In fairness, up until a few years ago, Bucket=dict wouldn't have worked. I'm sure there's a lot of Python programmers who learnt the language with version < 2.0 who still have blind-spots when it comes to types. I know I do (but I'm getting better with practice). Besides, would that be any better written as this? def Bucket(): return {} I think not. This is not a "named lambda" problem, this is a "developer doesn't know how to use types" problem. While we're discussing named lambdas, and at the risk of dragging this thread out even longer, here's one example where I would use one: def parrot(args, transformation=None): if transformation is None: # Use an identity function. transformation = lambda x: x for arg in args: do_something_with(transformation(arg)) Does anybody have any constructive comments to make about this usage? -- Steven D'Aprano From stefan_ml at behnel.de Sun May 4 11:58:24 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 04 May 2008 11:58:24 +0200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: Hi, Guido van Rossum wrote: > On Sat, May 3, 2008 at 11:46 AM, Terry Reedy wrote: >> and IF someone adapts it to work on a different >> platform (Django with regular DB backend) and then hosts it elsewhere. > > No, that won't be necessary. If someone contributes an alternate > authentication system I'd be happy to adopt it. Personally, I would consider the following sufficient: 1) people who have authenticated themselves against the underlying VCS (i.e. project members) may post public comments and comment on other comments 2) anonymous users can post comments that won't become publicly visible until an authenticated user acknowledges them or comments on them. That should be enough to keep the system open to everybody and to keep spam away. >> Of >> course, any such alternative would probably also require an account and >> Terms of Service agreement. > > You're seeing this to strictly. While developers hosting their app are > held by an Google terms of service agreement, they are not required to > hold their users to such an agreement. I agree with that, it's about commensurability. People who use my service shouldn't be bothered with the contracts I signed with my own service provider. Imagine you'd have to sign a contract with my web hoster to visit my web site. Stefan From python at rcn.com Sun May 4 12:31:12 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 4 May 2008 03:31:12 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? References: Message-ID: <017c01c8add1$fa62fae0$c600a8c0@RaymondLaptop1> [Terry Reedy] > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression Though def-statements are usually the best way to go, there are some reasonable cases where the first form reads better. I say leave it to the programmer to decide. Experience will quickly teach the def-statement is more flexible. Another thought is that I often give the advice that if a long-line becomes hard to read, then an easy solution is to pull-out an inner expression and give it a name. This is especially true when the intended purpose of the inner expression isn't obvious and giving it a name adds clarity. I would have to burden this advice with an admonition to go further and move the relevant code farther away from where it is used: for k,g in groupby(iterable, key=lambda r: (r[0].lower(), r[5].lower())): ... lastname_firstname = lambda r: (r[0].lower(), r[5].lower()) for k, g in groupby(iterable, key=lastname_firstname): ... That transformation adds clarity. Going further and creating a separate def-statement outside the current function would just move the relevant code farther away and impair readability. Also, I'm somewhat opposed to using PEP 8 in this way. Somehow, the PEPs recommendations seem to get occasionally interpreted as law. IMO, the PEP should be limited to things that have a real impact on code being maintained by multiple programmers. It should not grow into a general purpose set of situation independent prejudgments about which constructs should be favored over others (i.e. programmer x hates for-else so those should be avoided in favor of using flags or programmer y thinks it's uncool to have a return-statement anywhere but the end of a function). > What do our style mavens think? -1 Raymond From fuzzyman at voidspace.org.uk Sun May 4 01:56:05 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 04 May 2008 00:56:05 +0100 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: <481CFB95.2080003@voidspace.org.uk> Terry Reedy wrote: > Some people write > somename = lambda args: expression > instead of the more obvious (to most people) and, dare I say, standard > def somename(args): return expression > Visually I find the second form very ugly. The colon indicates to me that a new line is expected, and breaking that expectation breaks my flow of code reading. I very *rarely* use lambdas in the form you show, but where you do I prefer them to the single line function. Michael Foord > The difference in the result (the only one I know of) is that the code and > function objects get the generic name '' instead of the more > informative (in repr() output or tracebacks) 'somename'. I consider this a > disadvantage. > > In the absence of any compensating advantages (other than the trivial > saving of 3 chars), I consider the def form to be the proper Python style > to the point I think it should be at least recommended for the stdlib in > the Programming Recommendations section of PEP 8. > > There are currently uses of named lambdas at least in urllib2. This to me > is a bad example for new Python programmers. > > What do our style mavens think? > > Terry Jan Reedy > > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From solipsis at pitrou.net Sun May 4 12:55:38 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 4 May 2008 10:55:38 +0000 (UTC) Subject: [Python-Dev] PEP 8: Discourage named lambdas? References: <481C3136.4080902@openend.se> <481C3E0F.4080308@gmail.com> <200805041936.51136.steve@pearwood.info> Message-ID: Steven D'Aprano pearwood.info> writes: > > I think you're exaggerating a tad here. Why would you be "very confused" > when you see TITLEPATTERN() or foo(callback=TITLEPATTERN)? What else > would TITLEPATTERN be but a callable when it's being used as a > callable? The real problem is this example is not the fact that there is a lambda, but that the variable name doesn't reflect it is a callable. It is common practice to use verbs for callables, and nouns for non-callables. Thus I'd expect the variable to be called something like make_title_pattern rather than TITLEPATTERN (and what's with the ugly allcaps by the way?). Antoine. From stefan_ml at behnel.de Sun May 4 14:52:42 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 04 May 2008 14:52:42 +0200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: <481DB19A.20605@behnel.de> Guido van Rossum wrote: > On Thu, May 1, 2008 at 4:37 PM, Neal Becker wrote: >> It would be really nice to see support for some other backends, such as Hg >> or bzr (which are both written in python), in addition to svn. > > Once it's open source feel free to add those! trac supports a pretty wide set of VCSes. http://trac.edgewall.org/wiki/VersioningSystemBackend Maybe your tools could integrate these backends somehow instead of re-implementing yet another suite of VCS backend connectors. Stefan From glyph at divmod.com Sun May 4 15:58:03 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Sun, 04 May 2008 13:58:03 -0000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> Message-ID: <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> On 3 May, 11:34 pm, fdrake at acm.org wrote: >On May 3, 2008, at 7:51 AM, skip at pobox.com wrote: >>Fred asked for a --prefix flag (which is what I was voting on). I >>don't >>really care what you do by default as long as you give me a way to do >>it >>differently. > >What's most interesting (to me) is that no one's commented on my note >that my preferred approach would be that there's no default at all; >the location would have to be specified explicitly. Whether on the >command line or in the distutils configuration doesn't matter, but >explicitness should be required. I thought I responded to it in my initial response, but let me be clearer. First, Skip, I *only* care about the default behavior. There's already a way to do it differently: PYTHONPATH. So, Fred, I think what you're arguing for is to drop this feature entirely. Or is there some other use for a new way to allow users to explicitly add something to sys.path, aside from PYTHONPATH? It seems that it would add more complexity and I can't see what the value would be. As I've said a dozen times in this thread already, the feature I'd like to get from a per-user installation location is that 'setup.py install', or at least some completely canonical distutils incantation, should work, by default, for non-root users; ideally non-administrators on windows as well as non-root users on unixish platforms. From skip at pobox.com Sun May 4 16:14:59 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 4 May 2008 09:14:59 -0500 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> Message-ID: <18461.50403.928218.622685@montanaro-dyndns-org.local> glyph> As I've said a dozen times in this thread already, the feature glyph> I'd like to get from a per-user installation location is that glyph> 'setup.py install', or at least some completely canonical glyph> distutils incantation, should work, by default, for non-root glyph> users; ideally non-administrators on windows as well as non-root glyph> users on unixish platforms. I'm unclear why anything needs changing then. At work we have idiosyncratic central install locations for everything, not just Python. None of this stuff is installed by root. When I want to install some package to test without polluting the central waters I simply run setup.py install with a --prefix arg then set PYTHONPATH to pick up my stuff before the central stuff. I see no reason to change the behavior of setup.py's install command. It gives you the flexibility needed to handle a number of different scenarios. Skip From jnoller at gmail.com Sun May 4 16:17:36 2008 From: jnoller at gmail.com (Jesse Noller) Date: Sun, 4 May 2008 10:17:36 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> Message-ID: <4222a8490805040717u5de66d0cw7eadf471f19fe7b8@mail.gmail.com> On Sun, May 4, 2008 at 9:58 AM, wrote: ...snip... > As I've said a dozen times in this thread already, the feature I'd like to > get from a per-user installation location is that 'setup.py install', or at > least some completely canonical distutils incantation, should work, by > default, for non-root users; ideally non-administrators on windows as well > as non-root users on unixish platforms. > This is a big +1 from me. The way I currently work around the "must be root to install stuff" on both OS/X and other Lin/Uni(xes) is via virtualenv.py and a lot of bash environment trickery. If nothing else comes out of this, I think what glyph points out is the ideal, and simplest goal. Ignoring the directory name debate, I would like to see this local "user" dir mirror the normal directory tree that packages installed from distutils/setuptools typically use, namely it should have the: lib/site-packages/ and bin/ directories, and a known parent name. One thing that could be done is pick a default name for the parent, ala ~/Python - but let users override it with an environment variable if they so desire (PYTHON_USER_DIR?) so that those who want it hidden can have it hidden, and those of us who don't, don't. -jesse From ncoghlan at gmail.com Sun May 4 16:22:27 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 05 May 2008 00:22:27 +1000 Subject: [Python-Dev] PEP 370 (was Re: [Python-3000] Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> Message-ID: <481DC6A3.70104@gmail.com> glyph at divmod.com wrote: > As I've said a dozen times in this thread already, the feature I'd like > to get from a per-user installation location is that 'setup.py install', > or at least some completely canonical distutils incantation, should > work, by default, for non-root users; ideally non-administrators on > windows as well as non-root users on unixish platforms. This is what I see as the goal of PEP 370 as well. Perhaps the PEP could be more explicit in spelling that out? """The primary goal of this PEP is to provide a standard mechanism allowing Python users to install distutils packages for their own use without affecting other users of the same machine, and without requiring any change to the packages themselves.""" I think the current Rationale section kind of assumes that the reader already recognises the above paragraph as the reason for the PEP. In the UNIX Notes section, the PEP should probably also state that the reason for choosing a hidden dot-file directory is that users generally aren't going to have any interest in the source files for the Python packages that they install, and that users that would prefer for the files to be visible can easily make a symbolic link to the directory. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From arigo at tunes.org Sun May 4 17:39:27 2008 From: arigo at tunes.org (Armin Rigo) Date: Sun, 4 May 2008 17:39:27 +0200 Subject: [Python-Dev] Next PyPy sprint in Berlin, 17-22 May Message-ID: <20080504153926.GA23220@code0.codespeak.net> Hi all, Our next PyPy sprint will take place in Berlin, 17-22nd May 2008. More precisely, the sprint will be in the crashed c-base space station, Berlin, Germany, Earth, Solar System. This is a fully public sprint: newcomers (from all planets) and topics other than those proposed in the announcement are welcome. For more information: http://codespeak.net/pypy/extradoc/sprintinfo/berlin-2008/announcement.html A bientot, Armin. From lists at cheimes.de Sun May 4 18:14:06 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 04 May 2008 18:14:06 +0200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> Message-ID: <481DE0CE.8010306@cheimes.de> > First, Skip, I *only* care about the default behavior. There's already > a way to do it differently: PYTHONPATH. So, Fred, I think what you're > arguing for is to drop this feature entirely. Or is there some other > use for a new way to allow users to explicitly add something to > sys.path, aside from PYTHONPATH? It seems that it would add more > complexity and I can't see what the value would be. PYTHONPATH is lacking one feature which is important for lots of packages and setuptools. The directories in PYTHONPATH are just added to sys.path. But setuptools require a site package directory. Maybe a new env var PYTHONSITEPATH could solve the problem. > As I've said a dozen times in this thread already, the feature I'd like > to get from a per-user installation location is that 'setup.py install', > or at least some completely canonical distutils incantation, should > work, by default, for non-root users; ideally non-administrators on > windows as well as non-root users on unixish platforms. The implementation of my PEP provides a new option for install: $ python setup.py install --user Is it sufficient for you? Christian From lists at cheimes.de Sun May 4 18:19:17 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 04 May 2008 18:19:17 +0200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481C2AE7.9010805@gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> Message-ID: <481DE205.1060108@cheimes.de> Nick Coghlan schrieb: > - for experienced users (Barry, skip, etc) that want ~/.local to be more > easily accessible, creating a visible ~/local symlink is an utterly > trivial exercise. Our you can set the environment variable PYTHONUSERBASE to $HOME. PYTHONUSERBASE is the root directory for user specific data: def addusersitepackages(known_paths): """Add a per user site-package to sys.path Each user has its own python directory with site-packages in the home directory. USER_BASE is the root directory for all Python versions USER_SITE is the user specific site-packages directory USER_SITE/.. can be used for data. """ global USER_BASE, USER_SITE env_base = os.environ.get("PYTHONUSERBASE", None) def joinuser(*args): return os.path.expanduser(os.path.join(*args)) #if sys.platform in ('os2emx', 'riscos'): # # Don't know what to put here # USER_BASE = '' # USER_SITE = '' if os.name == "nt": base = os.environ.get("APPDATA") or "~" USER_BASE = env_base if env_base else joinuser(base, "Python") USER_SITE = os.path.join(USER_BASE, "Python" + sys.version[0] + sys.version[2], "site-packages") else: USER_BASE = env_base if env_base else joinuser("~", ".local") USER_SITE = os.path.join(USER_BASE, "lib", "python" + sys.version[:3], "site-packages") if os.path.isdir(USER_SITE): addsitedir(USER_SITE, known_paths) return known_paths Christian From lists at cheimes.de Sun May 4 18:24:38 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 04 May 2008 18:24:38 +0200 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <4222a8490805040717u5de66d0cw7eadf471f19fe7b8@mail.gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <4222a8490805040717u5de66d0cw7eadf471f19fe7b8@mail.gmail.com> Message-ID: <481DE346.2040608@cheimes.de> Jesse Noller schrieb: > One thing that could be done is pick a default name for the parent, > ala ~/Python - but let users override it with an environment variable > if they so desire (PYTHON_USER_DIR?) so that those who want it hidden > can have it hidden, and those of us who don't, don't. Has anybody read my PEP or do I need a Christian's English to real English converter? *g* >From my PEP 370: --- The path to the user base directory can be overwritten with the environment variable PYTHONUSERBASE. The default location is used when PYTHONUSERBASE is not set or empty. --- PYTHONUSERBASE defaults to ~/.local/ on Unix. In order to install packages in ~/lib, ~/bin etc directly you can do export PYTHONUSERBASE=$HOME in your .bashrc or .profile. Christian From mal at egenix.com Sun May 4 18:31:09 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Sun, 04 May 2008 18:31:09 +0200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481DE0CE.8010306@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DE0CE.8010306@cheimes.de> Message-ID: <481DE4CD.7070401@egenix.com> On 2008-05-04 18:14, Christian Heimes wrote: >> First, Skip, I *only* care about the default behavior. There's already >> a way to do it differently: PYTHONPATH. So, Fred, I think what you're >> arguing for is to drop this feature entirely. Or is there some other >> use for a new way to allow users to explicitly add something to >> sys.path, aside from PYTHONPATH? It seems that it would add more >> complexity and I can't see what the value would be. > > PYTHONPATH is lacking one feature which is important for lots of > packages and setuptools. The directories in PYTHONPATH are just added to > sys.path. But setuptools require a site package directory. Maybe a new > env var PYTHONSITEPATH could solve the problem. We don't need another setup variable for this. Just place a well-known module into the site-packages/ directory and then query it's __file__ attribute, e.g. site-packages/site_packages.py The module could even include a few helpers to query various settings which apply to the site packages directory, e.g. site_packages.get_dir() site_packages.list_packages() site_packages.list_modules() etc. >> As I've said a dozen times in this thread already, the feature I'd like >> to get from a per-user installation location is that 'setup.py install', >> or at least some completely canonical distutils incantation, should >> work, by default, for non-root users; ideally non-administrators on >> windows as well as non-root users on unixish platforms. > > The implementation of my PEP provides a new option for install: > > $ python setup.py install --user > > Is it sufficient for you? Just in case you don't know... python setup.py install --home=~ will install to ~/lib/python The problem is not getting the packages installed in a non-admin location. It's about Python looking in a non-admin location per default (as well as in the site-packages location). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 04 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From guido at python.org Sun May 4 18:36:35 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 4 May 2008 09:36:35 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: On Sun, May 4, 2008 at 2:58 AM, Stefan Behnel wrote: > Personally, I would consider the following sufficient: > > 1) people who have authenticated themselves against the underlying VCS (i.e. > project members) may post public comments and comment on other comments Tell me how to authenticate against a SVN project using HTTP only. > 2) anonymous users can post comments that won't become publicly visible until > an authenticated user acknowledges them or comments on them. Can you work out this design more? I don't understand how an authenticated user can acknowledge an anonymous comment if it isn't publicly visible. Also, AFAIK our bug tracker doesn't support anonymous comments either, so I don't think this is an important use case. > That should be enough to keep the system open to everybody and to keep spam away. It's also more work to code. Once I've released the code I'll let others decide whether they want to contribute such features. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun May 4 18:38:58 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 4 May 2008 09:38:58 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <481DB19A.20605@behnel.de> References: <481DB19A.20605@behnel.de> Message-ID: On Sun, May 4, 2008 at 5:52 AM, Stefan Behnel wrote: > Guido van Rossum wrote: > > On Thu, May 1, 2008 at 4:37 PM, Neal Becker wrote: > >> It would be really nice to see support for some other backends, such as Hg > >> or bzr (which are both written in python), in addition to svn. > > > > Once it's open source feel free to add those! > > trac supports a pretty wide set of VCSes. > > http://trac.edgewall.org/wiki/VersioningSystemBackend > > Maybe your tools could integrate these backends somehow instead of > re-implementing yet another suite of VCS backend connectors. That depends. If it requires more than plain HTTP requests it currently cannot be done on AppEngine. Anyway, I will continue to say, I'm making it open source Monday and then you can see for yourself how to add this. I don't want to be writing all the code for this project; I want it to grow with user contributions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From stefan_ml at behnel.de Sun May 4 18:52:07 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 04 May 2008 18:52:07 +0200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: <481DE9B7.2020104@behnel.de> Guido van Rossum wrote: > On Sun, May 4, 2008 at 2:58 AM, Stefan Behnel wrote: >> Personally, I would consider the following sufficient: >> >> 1) people who have authenticated themselves against the underlying VCS (i.e. >> project members) may post public comments and comment on other comments > > Tell me how to authenticate against a SVN project using HTTP only. Good question. :) >> 2) anonymous users can post comments that won't become publicly visible until >> an authenticated user acknowledges them or comments on them. > > Can you work out this design more? I don't understand how an > authenticated user can acknowledge an anonymous comment if it isn't > publicly visible. I was suggesting that you'd sign in as a project member and would then see all comments. They just wouldn't show up on the public web sites without further action. I think that matches the normal work flow. External users usually do some kind of upstream talk anyway. It would be rare that no project member is involved in a patch contribution. And new contributions would go through the bug tracker first anway, right? > Also, AFAIK our bug tracker doesn't support > anonymous comments either, so I don't think this is an important use > case. >> That should be enough to keep the system open to everybody and to keep spam away. > > It's also more work to code. Sure, it's also just an idea from my side. Stefan From guido at python.org Sun May 4 19:01:37 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 4 May 2008 10:01:37 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <481DE9B7.2020104@behnel.de> References: <481C49E5.6080403@behnel.de> <481DE9B7.2020104@behnel.de> Message-ID: [Stefan, can you please keep python-dev in the CC list?] On Sun, May 4, 2008 at 9:52 AM, Stefan Behnel wrote: > Guido van Rossum wrote: > > On Sun, May 4, 2008 at 2:58 AM, Stefan Behnel wrote: > >> 2) anonymous users can post comments that won't become publicly visible until > >> an authenticated user acknowledges them or comments on them. > > > > Can you work out this design more? I don't understand how an > > authenticated user can acknowledge an anonymous comment if it isn't > > publicly visible. > > I was suggesting that you'd sign in as a project member and would then see all > comments. Hm. Since most users of the site are project members this means most people would see the anonymous comments (= potential spam). However, the anonymous user couldn't see their *own* comments back later (or be allowed to edit them). Doesn't sound like a good idea to me. > They just wouldn't show up on the public web sites without further > action. I think that matches the normal work flow. External users usually do > some kind of upstream talk anyway. It would be rare that no project member is > involved in a patch contribution. And new contributions would go through the > bug tracker first anway, right? Not necessarily. Unless there's an uprising, I'll keep support for anonymous comments a non-goal for now. > > Also, AFAIK our bug tracker doesn't support > > anonymous comments either, so I don't think this is an important use > > case. > > > >> That should be enough to keep the system open to everybody and to keep spam away. > > > > It's also more work to code. > > Sure, it's also just an idea from my side. > > Stefan > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aleaxit at gmail.com Sun May 4 19:23:40 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Sun, 4 May 2008 10:23:40 -0700 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <017c01c8add1$fa62fae0$c600a8c0@RaymondLaptop1> References: <017c01c8add1$fa62fae0$c600a8c0@RaymondLaptop1> Message-ID: On Sun, May 4, 2008 at 3:31 AM, Raymond Hettinger wrote: ... > for k,g in groupby(iterable, key=lambda r: (r[0].lower(), r[5].lower())): > ... > lastname_firstname = lambda r: (r[0].lower(), r[5].lower()) > for k, g in groupby(iterable, key=lastname_firstname): ... > > That transformation adds clarity. Going further and creating a separate > def-statement outside the current function would just move the relevant > code farther away and impair readability. And that would be totally silly and uncalled for -- why ever would it be placed *outside the current function*?! What a straw-man...! Just do def last_first(r): return r[0].lower(), r[5].lower() for k, g in groupby(iterable, key=last_first): ... putting the def right where you now have the " = lambda ..." of course! Alex From josiah.carlson at gmail.com Sun May 4 20:13:53 2008 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Sun, 4 May 2008 11:13:53 -0700 Subject: [Python-Dev] Py3k and asyncore/asynchat In-Reply-To: References: Message-ID: On Mon, Mar 31, 2008 at 12:11 AM, Neal Norwitz wrote: > On Sun, Mar 30, 2008 at 7:44 PM, Josiah Carlson > > wrote: > > > > > I haven't really had time to update the tests/documentation, but > > again, I wasn't experiencing any strange test failures with > > asyncore/asynchat, nor have I been able to find the buildbot failures > > that you are referring to. Could someone please link the failures > > that are not related to being unable to discover a port number? > > 3 different platforms, 3 different problems: > > http://www.python.org/dev/buildbot/all/alpha%20Tru64%205.1%20trunk/builds/2790/step-test/0 > http://www.python.org/dev/buildbot/all/x86%20FreeBSD%203%203.0/builds/0/step-test/0 > http://www.python.org/dev/buildbot/all/x86%20XP-4%203.0/builds/643 > > > > According to the release schedule, we should have at least a couple > > more months for documentation and tests to be updated (I can get > > patches ready for alpha 3). > > When you get the patches with tests and doc, I'll be happy to check in. > > n I have updated the documentation, and as much of the tests as was required to pass on my Windows XP machine. In looking at the buildbots, I'm not seeing any more common issues. Unfortunately, I also hit the urls provided above too late, and I wasn't able to see the actual issues (if they still persist). If possible, the syntax in the documentation that I added should be checked, as I needed to convert from my older TeX docs to the new ReST doc format. I have attached the patch file, and am in the process of regaining access to the bug tracker. In the mean time, Giampaolo will be posting the patch to the tracker in issue 1736190. If anyone has any questions, please ask. - Josiah -------------- next part -------------- A non-text attachment was scrubbed... Name: full_async_patch.patch Type: application/octet-stream Size: 19144 bytes Desc: not available URL: From greg at krypto.org Sun May 4 20:24:24 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 4 May 2008 11:24:24 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: <52dc1c820805041124h6a3bee76o718ef7056b8236f5@mail.gmail.com> On Sun, May 4, 2008 at 9:36 AM, Guido van Rossum wrote: > On Sun, May 4, 2008 at 2:58 AM, Stefan Behnel wrote: > > Personally, I would consider the following sufficient: > > > > 1) people who have authenticated themselves against the underlying VCS > (i.e. > > project members) may post public comments and comment on other comments > > Tell me how to authenticate against a SVN project using HTTP only. > > > 2) anonymous users can post comments that won't become publicly visible > until > > an authenticated user acknowledges them or comments on them. > > Can you work out this design more? I don't understand how an > authenticated user can acknowledge an anonymous comment if it isn't > publicly visible. Also, AFAIK our bug tracker doesn't support > anonymous comments either, so I don't think this is an important use > case. > Rather than svn authentication i suggest just piggybacking on top of the bug trackers authentication. that is an integration i think we should aim for anyways and it should keep the no-google-account fear mongers happy. code comments (as bug comments are today) should not require svn commit access. anyways, i expect someone else may implement that once the code is out there. -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sun May 4 20:45:44 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 4 May 2008 11:45:44 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <52dc1c820805041124h6a3bee76o718ef7056b8236f5@mail.gmail.com> References: <481C49E5.6080403@behnel.de> <52dc1c820805041124h6a3bee76o718ef7056b8236f5@mail.gmail.com> Message-ID: On Sun, May 4, 2008 at 11:24 AM, Gregory P. Smith wrote: > Rather than svn authentication i suggest just piggybacking on top of the bug > trackers authentication. that is an integration i think we should aim for > anyways and it should keep the no-google-account fear mongers happy. code > comments (as bug comments are today) should not require svn commit access. The problem with this is that I'd like to have a single codereview webapp that serves many projects, as long as they use subversion. The scalability of AppEngine should make this a very attractive solution. Integrating with each separate bug tracker however would be a pain. > anyways, i expect someone else may implement that once the code is out > there. I sure hope so. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Sun May 4 21:57:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 04 May 2008 21:57:26 +0200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481DE4CD.7070401@egenix.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DE0CE.8010306@cheimes.de> <481DE4CD.7070401@egenix.com> Message-ID: <481E1526.6000903@cheimes.de> M.-A. Lemburg schrieb: >> PYTHONPATH is lacking one feature which is important for lots of >> packages and setuptools. The directories in PYTHONPATH are just added to >> sys.path. But setuptools require a site package directory. Maybe a new >> env var PYTHONSITEPATH could solve the problem. > > We don't need another setup variable for this. Just place a > well-known module into the site-packages/ directory and then > query it's __file__ attribute, e.g. > > site-packages/site_packages.py > > The module could even include a few helpers to query various > settings which apply to the site packages directory, e.g. > > site_packages.get_dir() > site_packages.list_packages() > site_packages.list_modules() > etc. I don't see how it is going to solve the use case "Add another site package directory when I don't have write access to the global site package directory and I don't want to modify my apps." > Just in case you don't know... > > python setup.py install --home=~ > > will install to ~/lib/python > > The problem is not getting the packages installed in a non-admin > location. It's about Python looking in a non-admin location per > default (as well as in the site-packages location). I know the --home option. For one the --home option is Unix only and not supported on Windows Also the --user option takes all options of my PEP 370 user site directory into account, includinge the PYTHONUSERBASE env var. Christian From lists at cheimes.de Sun May 4 21:59:51 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 04 May 2008 21:59:51 +0200 Subject: [Python-Dev] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <481DC6A3.70104@gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DC6A3.70104@gmail.com> Message-ID: <481E15B7.9060003@cheimes.de> Nick Coghlan schrieb: > This is what I see as the goal of PEP 370 as well. Perhaps the PEP could > be more explicit in spelling that out? > > """The primary goal of this PEP is to provide a standard mechanism > allowing Python users to install distutils packages for their own use > without affecting other users of the same machine, and without requiring > any change to the packages themselves.""" > > I think the current Rationale section kind of assumes that the reader > already recognises the above paragraph as the reason for the PEP. Good point ;) The author of the PEP was kinda sure all readers would recognize the ratio. Again explicit is better than implicit. I'll update the PEP later. > In the UNIX Notes section, the PEP should probably also state that the > reason for choosing a hidden dot-file directory is that users generally > aren't going to have any interest in the source files for the Python > packages that they install, and that users that would prefer for the > files to be visible can easily make a symbolic link to the directory. Good point, too. Thanks Nick! Christian From mal at egenix.com Sun May 4 22:56:34 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Sun, 04 May 2008 22:56:34 +0200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <481E1526.6000903@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DE0CE.8010306@cheimes.de> <481DE4CD.7070401@egenix.com> <481E1526.6000903@cheimes.de> Message-ID: <481E2302.8000509@egenix.com> On 2008-05-04 21:57, Christian Heimes wrote: > M.-A. Lemburg schrieb: >>> PYTHONPATH is lacking one feature which is important for lots of >>> packages and setuptools. The directories in PYTHONPATH are just added to >>> sys.path. But setuptools require a site package directory. Maybe a new >>> env var PYTHONSITEPATH could solve the problem. >> We don't need another setup variable for this. Just place a >> well-known module into the site-packages/ directory and then >> query it's __file__ attribute, e.g. >> >> site-packages/site_packages.py >> >> The module could even include a few helpers to query various >> settings which apply to the site packages directory, e.g. >> >> site_packages.get_dir() >> site_packages.list_packages() >> site_packages.list_modules() >> etc. > > I don't see how it is going to solve the use case "Add another site > package directory when I don't have write access to the global site > package directory and I don't want to modify my apps." No, but it's going to solve the issue "which of the sys.path directories is to be considered the site packages" directory. I was under the impression that this is what you were after. >> Just in case you don't know... >> >> python setup.py install --home=~ >> >> will install to ~/lib/python >> >> The problem is not getting the packages installed in a non-admin >> location. It's about Python looking in a non-admin location per >> default (as well as in the site-packages location). > > I know the --home option. For one the --home option is Unix only and not > supported on Windows Also the --user option takes all options of my PEP > 370 user site directory into account, includinge the PYTHONUSERBASE env var. Ok. Just wanted to mention that there is a precedent in distutils for doing user home directory installations. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 04 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From greg.ewing at canterbury.ac.nz Mon May 5 02:59:36 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 05 May 2008 12:59:36 +1200 Subject: [Python-Dev] PEP 8: Discourage named lambdas? In-Reply-To: <017c01c8add1$fa62fae0$c600a8c0@RaymondLaptop1> References: <017c01c8add1$fa62fae0$c600a8c0@RaymondLaptop1> Message-ID: <481E5BF8.5010903@canterbury.ac.nz> Raymond Hettinger wrote: > lastname_firstname = lambda r: (r[0].lower(), r[5].lower()) > for k, g in groupby(iterable, key=lastname_firstname): ... > > That transformation adds clarity. Going further and creating a separate > def-statement outside the current function would just move the relevant > code farther away and impair readability. It doesn't have to be outside the function -- it can be in exactly the same place as the lambda assignment above. def lastname_firstname(r): return (r[0].lower(), r[5].lower()) for k, g in groupby(iterable, key=lastname_firstname): ... Maybe "def is an executable statement" is another thing people have a blind spot about? -- Greg From greg.ewing at canterbury.ac.nz Mon May 5 03:25:11 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 05 May 2008 13:25:11 +1200 Subject: [Python-Dev] Next PyPy sprint in Berlin, 17-22 May In-Reply-To: <20080504153926.GA23220@code0.codespeak.net> References: <20080504153926.GA23220@code0.codespeak.net> Message-ID: <481E61F7.9020108@canterbury.ac.nz> Armin Rigo wrote: > More precisely, the sprint will be in the crashed c-base space station, > Berlin, Germany, Earth, Solar System. You have a crashed space station in Berlin? Wow! I hope it didn't come down on anything important... -- Greg From guido at python.org Mon May 5 05:38:04 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 4 May 2008 20:38:04 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: This code is now open source! Browse it here: http://code.google.com/p/rietveld/source/browse --Guido On Thu, May 1, 2008 at 9:41 AM, Guido van Rossum wrote: > Some of you may have seen a video recorded in November 2006 where I > showed off Mondrian, a code review tool that I was developing for > Google (http://www.youtube.com/watch?v=sMql3Di4Kgc). I've always hoped > that I could release Mondrian as open source, but it was not to be: > due to its popularity inside Google, it became more and more tied to > proprietary Google infrastructure like Bigtable, and it remained > limited to Perforce, the commercial revision control system most used > at Google. > > What I'm announcing now is the next best thing: an code review tool > for use with Subversion, inspired by Mondrian and (soon to be) > released as open source. Some of the code is even directly derived > from Mondrian. Most of the code is new though, written using Django > and running on Google App Engine. > > I'm inviting the Python developer community to try out the tool on the > web for code reviews. I've added a few code reviews already, but I'm > hoping that more developers will upload at least one patch for review > and invite a reviewer to try it out. > > To try it out, go here: > > http://codereview.appspot.com > > Please use the Help link in the top right to read more on how to use > the app. Please sign in using your Google Account (either a Gmail > address or a non-Gmail address registered with Google) to interact > more with the app (you need to be signed in to create new issues and > to add comments to existing issues). > > Don't hesitate to drop me a note with feedback -- note though that > there are a few known issues listed at the end of the Help page. The > Help page is really a wiki, so feel free to improve it! > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon May 5 05:42:15 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 4 May 2008 20:42:15 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: I forgot -- you need to link or copy the 'django' directory from Django 0.97.pre into the app directory. Otherwise you'll be using the Django 0.96.1 that's included with the AppEngine runtime, and the code is not compatible with that version. On Sun, May 4, 2008 at 8:38 PM, Guido van Rossum wrote: > This code is now open source! Browse it here: > > http://code.google.com/p/rietveld/source/browse > > --Guido > > > > On Thu, May 1, 2008 at 9:41 AM, Guido van Rossum wrote: > > Some of you may have seen a video recorded in November 2006 where I > > showed off Mondrian, a code review tool that I was developing for > > Google (http://www.youtube.com/watch?v=sMql3Di4Kgc). I've always hoped > > that I could release Mondrian as open source, but it was not to be: > > due to its popularity inside Google, it became more and more tied to > > proprietary Google infrastructure like Bigtable, and it remained > > limited to Perforce, the commercial revision control system most used > > at Google. > > > > What I'm announcing now is the next best thing: an code review tool > > for use with Subversion, inspired by Mondrian and (soon to be) > > released as open source. Some of the code is even directly derived > > from Mondrian. Most of the code is new though, written using Django > > and running on Google App Engine. > > > > I'm inviting the Python developer community to try out the tool on the > > web for code reviews. I've added a few code reviews already, but I'm > > hoping that more developers will upload at least one patch for review > > and invite a reviewer to try it out. > > > > To try it out, go here: > > > > http://codereview.appspot.com > > > > Please use the Help link in the top right to read more on how to use > > the app. Please sign in using your Google Account (either a Gmail > > address or a non-Gmail address registered with Google) to interact > > more with the app (you need to be signed in to create new issues and > > to add comments to existing issues). > > > > Don't hesitate to drop me a note with feedback -- note though that > > there are a few known issues listed at the end of the Help page. The > > Help page is really a wiki, so feel free to improve it! > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tom at vector-seven.com Mon May 5 05:45:58 2008 From: tom at vector-seven.com (Thomas Lee) Date: Mon, 05 May 2008 13:45:58 +1000 Subject: [Python-Dev] Module Suggestion: ast In-Reply-To: <79990c6b0805011437k312aa0dcha245fab2ef64f8ba@mail.gmail.com> References: <79990c6b0805011437k312aa0dcha245fab2ef64f8ba@mail.gmail.com> Message-ID: <481E82F6.1060103@vector-seven.com> I'm in the process of writing C code for the purposes of traversing AST nodes in the AST optimization branch. This seems to be an ideal case for code generation based on the ASDL representation of the AST as we're currently doing for Python-ast.[ch]. I'm already considering this approach for some code I need to visit the C AST representation. We could likely write a similar generator for the PyObject representation of the AST, which might be useful for the proposed "ast" module. This would ensure that both implementations of the AST visitation code are always kept in step with the ASDL. The only real problem I can foresee with this is that the code in asdl_c.py is already getting pretty hairy. Adding to the mess is going to make it worse still. Maybe this will serve as a good opportunity to clean it up a little? Any objections? Cheers, T Paul Moore wrote: > 2008/5/1 Georg Brandl : > >> Armin Ronacher schrieb: >> >>> I would like to propose a new module for the stdlib for Python 2.6 >>> and higher: "ast". >>> >> If there are no further objections, I'll add this to PEP 361 so that the >> proposal doesn't get lost. >> > > Excuse my confusion over process, but if this is to go into 2.6, does > that mean it needs to be ready before the first beta? Or is there a > more relaxed schedule for the stdlib (and if so, what is the deadline > for the stdlib)? > > The same question probably applies to the stdlib reorg... > > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/tom%40vector-seven.com > From bborcic at gmail.com Mon May 5 12:43:14 2008 From: bborcic at gmail.com (Boris Borcic) Date: Mon, 05 May 2008 12:43:14 +0200 Subject: [Python-Dev] Dotted/indexed defs ? Re: PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: Mike Klaas wrote: > Another thing to consider is that the def() pattern is only possible > when the bound variable has no dots. A common pattern for me is to > replace an instances method with a lambda to add monitoring hooks or > disable certain functionality: > > inst.get_foo = lambda: FakeFoo() Good point. That one may write eg for inst.bar in range(10) : do_something() makes one wonder whether def inst.bar(...) : ... or even def fn[k](...) : ... should be similarly permitted. Has this ever been discussed ? Regards, BB From schmir at gmail.com Mon May 5 13:28:06 2008 From: schmir at gmail.com (Ralf Schmitt) Date: Mon, 5 May 2008 13:28:06 +0200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> Message-ID: <932f8baf0805050428t2cd31b00x9c60d8ef43b5828c@mail.gmail.com> On Thu, May 1, 2008 at 10:26 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > This is a reminder that the LAST planned alpha releases of Python 2.6 and > 3.0 are scheduled for next Wednesday, 07-May-2008. Please be diligent over > the next week so that none of your changes break Python. The stable > buildbots look moderately okay, let's see what we can do about getting them > all green: > > http://www.python.org/dev/buildbot/stable/ > > We have a few showstopper bugs, and I will be looking at these more > carefully starting next week. > > > http://bugs.python.org/issue?@columns=title,id,activity,versions,status&@sort=activity&@filter=priority,status&@pagesize=50&@startwith=0&priority=1&status=1&@dispname=Showstoppers > running the testsuite segfaults on my 64 bit debian testing in test_pyexpat. This does not happen in a debug build: test_pyclbr test_pydoc test_pyexpat Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x2b573851a6e0 (LWP 19486)] 0x0000000000000000 in ?? () (gdb) bt #0 0x0000000000000000 in ?? () #1 0x00002aaaaf694f0a in doContent (parser=0x1b4bab0, startTagLevel=0, enc=0x1b4dba0, s=0x1b4cb3c "", ' ' , "frozenset([frozenset([2]),\n", ' ' , "frozenset([0,\n", ' ' ..., end=0x1b4cb40 ' ' , "frozenset([frozenset([2]),\n", ' ' , "frozenset([0,\n", ' ' ..., nextPtr=0x1b4bae0, haveMore=1 '\001') at extensions/expat/lib/xmlparse.c:2540 #2 0x00002aaaaf6972ee in contentProcessor (parser=0x1b4bab0, start=0x0, end=0x1b4c470 " q\001", endPtr=0x0) at extensions/expat/lib/xmlparse.c:2003 #3 0x00002aaaaf698662 in doProlog (parser=0x1b4bab0, enc=0x1b4dba0, s=0x1b4c738 "", 'a' ..., end=0x1b4cb40 ' ' , "frozenset([frozenset([2]),\n", ' ' , "frozenset([0,\n", ' ' ..., tok=29, next=0x1b4c738 "", 'a' ..., nextPtr=0x1b4bae0, haveMore=1 '\001') at extensions/expat/lib/xmlparse.c:3803 #4 0x00002aaaaf69adc3 in prologInitProcessor (parser=0x1b4bab0, s=0x1b4c710 "", 'a' ..., end=0x1b4cb40 ' ' , "frozenset([frozenset([2]),\n", ' ' , "frozenset([0,\n", ' ' ..., nextPtr=0x1b4bae0) at extensions/expat/lib/xmlparse.c:3551 #5 0x00002aaaaf68cc61 in XML_ParseBuffer (parser=0x1d20670, len=28625724, isFinal=0) at extensions/expat/lib/xmlparse.c:1562 #6 0x00002aaaaf689467 in xmlparse_Parse (self=0x1d20670, args=) at extensions/pyexpat.c:922 #7 0x0000000000419b9d in PyObject_Call (func=0x1a52b48, arg=0x2b7a710, kw=0x1b4c610) at Objects/abstract.c:2490 #8 0x00000000004902f8 in PyEval_EvalFrameEx (f=0x8cba40, throwflag=) at Python/ceval.c:3944 #9 0x0000000000494824 in PyEval_EvalCodeEx (co=0x2b5738764558, globals=, locals=, args=0x23aa130, argcount=4, kws=0x23aa150, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2908 -------------- next part -------------- An HTML attachment was scrubbed... URL: From unknown_kev_cat at hotmail.com Mon May 5 17:27:59 2008 From: unknown_kev_cat at hotmail.com (Joe Smith) Date: Mon, 5 May 2008 11:27:59 -0400 Subject: [Python-Dev] Invitation to try out open source code review tool References: <481C49E5.6080403@behnel.de> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20805040936nf06de9dk77f37433254b0495 at mail.gmail.com... > On Sun, May 4, 2008 at 2:58 AM, Stefan Behnel wrote: >> Personally, I would consider the following sufficient: >> >> 1) people who have authenticated themselves against the underlying VCS >> (i.e. >> project members) may post public comments and comment on other comments > > Tell me how to authenticate against a SVN project using HTTP only. > Hmm... Is there a good reason not to just migrate over to OpenID? For one thing, surely an OpenID provider could exist that just authenticates a username against an SVN account. Then many people already have a Yahoo account or AOL screenname. On the downside, the OpenID system will likely not be terribly effective at avoiding spam in the long run, as spammers register accounts at providers (especially those without or with weak Captchas). On the otherhand, there are already reports of Spammers mass signing-up for Google accounts, so it may not be that different in the long run anyway. From guido at python.org Mon May 5 18:28:01 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 5 May 2008 09:28:01 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: If you're asking for reasons not to do this: - AppEngine has a built-in API for dealing with Google Accounts, but not for OpenID - Apparently OpenID requires some crypto that would be slow without additional API support - I'm not personally familiar with OpenID None of these seem insurmountable, so now that it's open source, I'm looking forward to contributions. --Guido On Mon, May 5, 2008 at 8:27 AM, Joe Smith wrote: > > "Guido van Rossum" wrote in message > news:ca471dc20805040936nf06de9dk77f37433254b0495 at mail.gmail.com... > > > > On Sun, May 4, 2008 at 2:58 AM, Stefan Behnel wrote: > > > > > Personally, I would consider the following sufficient: > > > > > > 1) people who have authenticated themselves against the underlying VCS > (i.e. > > > project members) may post public comments and comment on other comments > > > > > > > Tell me how to authenticate against a SVN project using HTTP only. > > > > > > Hmm... Is there a good reason not to just migrate over to OpenID? > For one thing, surely an OpenID provider could exist that just > authenticates a username against an SVN account. Then many people already > have a Yahoo account or AOL screenname. On the downside, the OpenID system > will likely not be terribly effective at avoiding spam in the long run, as > spammers register accounts at providers (especially those without or with > weak Captchas). On the otherhand, there are already reports of Spammers mass > signing-up for Google accounts, so it may not be that different in the long > run anyway. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon May 5 18:46:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 05 May 2008 18:46:32 +0200 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: <481F39E8.2010904@v.loewis.de> > This code is now open source! Browse it here: > > http://code.google.com/p/rietveld/source/browse Are you also going to call it Rietveld then? Sounds better to me than "the open source code review tool". Regards, Martin From draghuram at gmail.com Mon May 5 18:49:26 2008 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Mon, 5 May 2008 12:49:26 -0400 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: <2c51ecee0805050949g2074035do59b2388236743435@mail.gmail.com> On Mon, May 5, 2008 at 12:28 PM, Guido van Rossum wrote: > None of these seem insurmountable, so now that it's open source, I'm > looking forward to contributions. Are there any mailing lists associated with this project? I looked at google code page but couldn't find any. Thanks, Raghu From guido at python.org Mon May 5 19:24:56 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 5 May 2008 10:24:56 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: <481F39E8.2010904@v.loewis.de> References: <481F39E8.2010904@v.loewis.de> Message-ID: On Mon, May 5, 2008 at 9:46 AM, "Martin v. L?wis" wrote: > > This code is now open source! Browse it here: > > > > http://code.google.com/p/rietveld/source/browse > > Are you also going to call it Rietveld then? Sounds better > to me than "the open source code review tool". I've been reluctant to use the Rietveld name too much since Americans can't spell it. :-) But the open source project *is* called Rietveld, so I suppose I should start using that name... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon May 5 19:25:24 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 5 May 2008 10:25:24 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <2c51ecee0805050949g2074035do59b2388236743435@mail.gmail.com> References: <481C49E5.6080403@behnel.de> <2c51ecee0805050949g2074035do59b2388236743435@mail.gmail.com> Message-ID: On Mon, May 5, 2008 at 9:49 AM, Raghuram Devarakonda wrote: > On Mon, May 5, 2008 at 12:28 PM, Guido van Rossum wrote: > > > None of these seem insurmountable, so now that it's open source, I'm > > looking forward to contributions. > > Are there any mailing lists associated with this project? I looked at > google code page but couldn't find any. Not yet. Do you want me to create a Google group for it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From draghuram at gmail.com Mon May 5 19:28:53 2008 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Mon, 5 May 2008 13:28:53 -0400 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> <2c51ecee0805050949g2074035do59b2388236743435@mail.gmail.com> Message-ID: <2c51ecee0805051028y397851a0l1b88de01dc5c1757@mail.gmail.com> > > Are there any mailing lists associated with this project? I looked at > > google code page but couldn't find any. > > Not yet. Do you want me to create a Google group for it? I think that will be very useful. From skip at pobox.com Mon May 5 19:32:43 2008 From: skip at pobox.com (skip at pobox.com) Date: Mon, 5 May 2008 12:32:43 -0500 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: <481F39E8.2010904@v.loewis.de> Message-ID: <18463.17595.324578.284849@montanaro-dyndns-org.local> Guido> I've been reluctant to use the Rietveld name too much since Guido> Americans can't spell it. :-) But the open source project *is* Guido> called Rietveld, so I suppose I should start using that name... Which reminds me... What's it mean? All I saw was a Dutch city and (maybe?) a Dutch architect by that name. Skip From guido at python.org Mon May 5 19:33:57 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 5 May 2008 10:33:57 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: <18463.17595.324578.284849@montanaro-dyndns-org.local> References: <481F39E8.2010904@v.loewis.de> <18463.17595.324578.284849@montanaro-dyndns-org.local> Message-ID: On Mon, May 5, 2008 at 10:32 AM, wrote: > > Guido> I've been reluctant to use the Rietveld name too much since > Guido> Americans can't spell it. :-) But the open source project *is* > Guido> called Rietveld, so I suppose I should start using that name... > > Which reminds me... What's it mean? All I saw was a Dutch city and > (maybe?) a Dutch architect by that name. > > Skip > http://code.google.com/p/rietveld/wiki/CodeReviewBackground -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon May 5 19:36:49 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 5 May 2008 10:36:49 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <2c51ecee0805051028y397851a0l1b88de01dc5c1757@mail.gmail.com> References: <2c51ecee0805050949g2074035do59b2388236743435@mail.gmail.com> <2c51ecee0805051028y397851a0l1b88de01dc5c1757@mail.gmail.com> Message-ID: Done. I've sent an invite to this list, but just in case it doesn't make it, here's the group home page: http://groups.google.com/group/codereview-discuss On Mon, May 5, 2008 at 10:28 AM, Raghuram Devarakonda wrote: > > > Are there any mailing lists associated with this project? I looked at > > > google code page but couldn't find any. > > > > Not yet. Do you want me to create a Google group for it? > > I think that will be very useful. > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Mon May 5 19:56:34 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 05 May 2008 19:56:34 +0200 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: <481F39E8.2010904@v.loewis.de> Message-ID: Guido van Rossum schrieb: > On Mon, May 5, 2008 at 9:46 AM, "Martin v. L?wis" wrote: >> > This code is now open source! Browse it here: >> > >> > http://code.google.com/p/rietveld/source/browse >> >> Are you also going to call it Rietveld then? Sounds better >> to me than "the open source code review tool". > > I've been reluctant to use the Rietveld name too much since Americans > can't spell it. :-) But the open source project *is* called Rietveld, > so I suppose I should start using that name... BTW, Google code's Python highlighting is broken, look at http://code.google.com/p/rietveld/source/browse/trunk/views.py line 205ff. Where can this be reported? Georg From tjreedy at udel.edu Mon May 5 20:43:16 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 5 May 2008 14:43:16 -0400 Subject: [Python-Dev] [Python-3000] Invitation to try out open sourcecode review tool References: <481F39E8.2010904@v.loewis.de> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20805051024v4b88f28fvc0b713529dd08e68 at mail.gmail.com... |I've been reluctant to use the Rietveld name too much since Americans can't spell it. :-) Well, it follows the i before e rule, so I think I will manage. And it is a nice name, unlike Unix 'creat'. In any case, American programmers are not 'average' Americans ;-). So I say use it! From guido at python.org Mon May 5 20:56:19 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 5 May 2008 11:56:19 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: <481F39E8.2010904@v.loewis.de> Message-ID: That's a good question. I'll find out. On Mon, May 5, 2008 at 10:56 AM, Georg Brandl wrote: > Guido van Rossum schrieb: > > > > On Mon, May 5, 2008 at 9:46 AM, "Martin v. L?wis" > wrote: > > > > > > This code is now open source! Browse it here: > > > > > > > > http://code.google.com/p/rietveld/source/browse > > > > > > Are you also going to call it Rietveld then? Sounds better > > > to me than "the open source code review tool". > > > > > > > I've been reluctant to use the Rietveld name too much since Americans > > can't spell it. :-) But the open source project *is* called Rietveld, > > so I suppose I should start using that name... > > > > BTW, Google code's Python highlighting is broken, look at > http://code.google.com/p/rietveld/source/browse/trunk/views.py line 205ff. > > Where can this be reported? > > Georg > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dberlin at dberlin.org Mon May 5 21:21:51 2008 From: dberlin at dberlin.org (Daniel Berlin) Date: Mon, 5 May 2008 15:21:51 -0400 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: <481F39E8.2010904@v.loewis.de> Message-ID: <4aca3dc20805051221t290caa56u3a49701e567163ee@mail.gmail.com> There is a google code support project where google code issues are tracked. http://code.google.com/p/support/ On Mon, May 5, 2008 at 2:56 PM, Guido van Rossum wrote: > That's a good question. I'll find out. > > > > On Mon, May 5, 2008 at 10:56 AM, Georg Brandl wrote: > > Guido van Rossum schrieb: > > > > > > > On Mon, May 5, 2008 at 9:46 AM, "Martin v. L?wis" > > wrote: > > > > > > > > This code is now open source! Browse it here: > > > > > > > > > > http://code.google.com/p/rietveld/source/browse > > > > > > > > Are you also going to call it Rietveld then? Sounds better > > > > to me than "the open source code review tool". > > > > > > > > > > I've been reluctant to use the Rietveld name too much since Americans > > > can't spell it. :-) But the open source project *is* called Rietveld, > > > so I suppose I should start using that name... > > > > > > > BTW, Google code's Python highlighting is broken, look at > > http://code.google.com/p/rietveld/source/browse/trunk/views.py line 205ff. > > > > Where can this be reported? > > > > Georg > > > > > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > > > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/dberlin%40dberlin.org > From guido at python.org Mon May 5 22:38:45 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 5 May 2008 13:38:45 -0700 Subject: [Python-Dev] [Python-3000] Invitation to try out open source code review tool In-Reply-To: References: <481F39E8.2010904@v.loewis.de> Message-ID: I reported this: http://code.google.com/p/support/issues/detail?id=1006 The tracker is here: http://code.google.com/p/support/issues Thanks! On Mon, May 5, 2008 at 10:56 AM, Georg Brandl wrote: > Guido van Rossum schrieb: > > > > On Mon, May 5, 2008 at 9:46 AM, "Martin v. L?wis" > wrote: > > > > > > This code is now open source! Browse it here: > > > > > > > > http://code.google.com/p/rietveld/source/browse > > > > > > Are you also going to call it Rietveld then? Sounds better > > > to me than "the open source code review tool". > > > > > > > I've been reluctant to use the Rietveld name too much since Americans > > can't spell it. :-) But the open source project *is* called Rietveld, > > so I suppose I should start using that name... > > > > BTW, Google code's Python highlighting is broken, look at > http://code.google.com/p/rietveld/source/browse/trunk/views.py line 205ff. > > Where can this be reported? > > Georg > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Tue May 6 02:36:55 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 May 2008 12:36:55 +1200 Subject: [Python-Dev] Dotted/indexed defs ? Re: PEP 8: Discourage named lambdas? In-Reply-To: References: Message-ID: <481FA827.1000501@canterbury.ac.nz> Boris Borcic wrote: > def inst.bar(...) : ... > def fn[k](...) : ... > > should be similarly permitted. Has this ever been discussed ? Yes, but there didn't seem to be much interest in the idea. Which is disappointing, because it would be ideal for setting up function tables and such like. One small problem is that there would be no sensible name to give the function object, but that's no worse than what you get when using a lambda. -- Greg From greg.ewing at canterbury.ac.nz Tue May 6 03:05:31 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 May 2008 13:05:31 +1200 Subject: [Python-Dev] [Python-3000] Invitation to try out open sourcecode review tool In-Reply-To: References: <481F39E8.2010904@v.loewis.de> Message-ID: <481FAEDB.5000108@canterbury.ac.nz> Terry Reedy wrote: > In any case, American programmers are not 'average' Americans ;-). > So I say use it! Plus, think of all the millions of people who will have their lives enriched by learning to spell 'Rietveld' properly! -- Greg From asmodai at in-nomine.org Tue May 6 09:48:54 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 6 May 2008 09:48:54 +0200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> Message-ID: <20080506074854.GO1016@nexus.in-nomine.org> -On [20080505 18:28], Guido van Rossum (guido at python.org) wrote: >- AppEngine has a built-in API for dealing with Google Accounts, but >not for OpenID > >- Apparently OpenID requires some crypto that would be slow without >additional API support > >- I'm not personally familiar with OpenID Check out http://openid-provider.appspot.com/ Apparently it was created by your colleague Ryan Barrett. :) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B If Winter comes, can Spring be far behind..? From asmodai at in-nomine.org Tue May 6 09:51:01 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 6 May 2008 09:51:01 +0200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: Message-ID: <20080506075101.GP1016@nexus.in-nomine.org> -On [20080505 05:38], Guido van Rossum (guido at python.org) wrote: > http://code.google.com/p/rietveld/source/browse Is it a reference to Gerrit Rietveld (Dutch architect and furniture designer)? I guess the architect part would make sense for a code review tool. :) -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B If Winter comes, can Spring be far behind..? From solipsis at pitrou.net Tue May 6 12:47:23 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 6 May 2008 10:47:23 +0000 (UTC) Subject: [Python-Dev] Warn about mktemp once again? References: <18452.65322.843226.299769@montanaro-dyndns-org.local> Message-ID: pobox.com> writes: > > Back in r29829, Guido commented out the security hole warning for > tempfile.mktemp: > [...] > > Comment out the warnings about mktemp(). These are too annoying, and > often unavoidable. > > Any thought about whether this warning should be restored? We're 5+ years > later. Hopefully many uses of mktemp have been removed. If we're not going > to restore the warning perhaps the commented code should just be deleted. Sorry to revive this thread, but mktemp() is very useful when the file is meant to be created by another application (e.g. launched by subprocess, but it could even be a daemon running under a different user). For example if I have a processing chain to converts a PDF to a temporary JPEG using an external tool and then does other things with the JPEG: I don't want Python to actually create the file, just to generate an unique filename. Of course one can use NamedTemporaryFile, retrieve the name, close the file handle and then pass the name to the other application. But it's an useless complication compared to the simplicity of writing "my_filename = tempfile.mktemp()" From barry at python.org Tue May 6 13:26:20 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 6 May 2008 07:26:20 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> Message-ID: <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 1, 2008, at 8:03 PM, glyph at divmod.com wrote: > > Am I the only guy who finds software that insists on visible, fixed > files in my home directory rude? vmware, for example, wants a "~/ > vmware" directory, but pretty much every other application I use is > nice enough to use dotfiles (even cedega, with a roughly-comparable- > to- lib "applications I've installed for you" folder). No Glyph, you are not alone! I don't even like the OS putting stuff like Pictures, Music, Movies, Videos and Desktop in my home directory, but I guess that's the price we pay for a modrin desktopy operatin' systum. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCBAXHEjvBPtnXfVAQJdrgP+Mw0qZebL+MqUk3wKMsRt5mHzT/uHhQ0Z NVwyooWKWnvLMMifCbaG3pjVs7MehfcbAK8uLTlF8Ss9/w1Q5SWJkdhLMWOvHdA6 CJMvGyuokElD5e2cKXiakUWUshN/CeGNElTpxHUBdwmkirfXLQzQll9jlYbnr0I8 du2+rTj/oAc= =015L -----END PGP SIGNATURE----- From mithrandi-python-dev at mithrandi.za.net Tue May 6 13:19:15 2008 From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann) Date: Tue, 6 May 2008 13:19:15 +0200 Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: References: <18452.65322.843226.299769@montanaro-dyndns-org.local> Message-ID: <20080506111915.GA4193@mithrandi.net> * Antoine Pitrou [2008-05-06 10:47:23 +0000]: > pobox.com> writes: > > > > Back in r29829, Guido commented out the security hole warning for > > tempfile.mktemp: > > > [...] > > > > Comment out the warnings about mktemp(). These are too annoying, and > > often unavoidable. > > > > Any thought about whether this warning should be restored? We're 5+ years > > later. Hopefully many uses of mktemp have been removed. If we're not going > > to restore the warning perhaps the commented code should just be deleted. > > Sorry to revive this thread, but mktemp() is very useful when the file is meant > to be created by another application (e.g. launched by subprocess, but it could > even be a daemon running under a different user). For example if I have a > processing chain to converts a PDF to a temporary JPEG using an external tool > and then does other things with the JPEG: I don't want Python to actually > create the file, just to generate an unique filename. The correct way to do this is to create a temporary directory, and then generate a filename underneath that directory to use. -- mithrandi, i Ainil en-Balandor, a faer Ambar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: Digital signature URL: From alec at swapoff.org Tue May 6 14:45:22 2008 From: alec at swapoff.org (Alec Thomas) Date: Tue, 6 May 2008 22:45:22 +1000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> Message-ID: <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> 2008/5/6 Barry Warsaw : > On May 1, 2008, at 8:03 PM, glyph at divmod.com wrote: > > Am I the only guy who finds software that insists on visible, fixed files > in my home directory rude? vmware, for example, wants a "~/vmware" > directory, but pretty much every other application I use is nice enough to > use dotfiles (even cedega, with a roughly-comparable-to- lib "applications > I've installed for you" folder). > > No Glyph, you are not alone! I don't even like the OS putting stuff like > Pictures, Music, Movies, Videos and Desktop in my home directory, but I > guess that's the price we pay for a modrin desktopy operatin' systum. I too find this irritating. FWIW my vote is for ~/.python. ~/.local comes in a distant second due to non-obviousness and ~/Python is several light years beyond that. -- Evolution: Taking care of those too stupid to take care of themselves. From ncoghlan at gmail.com Tue May 6 15:51:32 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 06 May 2008 23:51:32 +1000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> Message-ID: <48206264.1010507@gmail.com> Alec Thomas wrote: > FWIW my vote is for ~/.python. ~/.local comes in a distant second due > to non-obviousness and ~/Python is several light years beyond that. I think if the obviousness (or lack thereof) of the chosen directory name ever really matters to anyone, we did it wrong. After all, unless you're trying to use something other than distutils to get a package ready for installation, how often does it really matter that the site-packages directory for an installed python interpreter actually lives somewhere inside /usr/local? The main advantage I see to using the "~/.local" approach is that a lot of questions about file layout (e.g. where to put architecture specific code) are automatically (and fairly obviously) answered "Do whatever is done for the system-wide equivalent in /usr/local". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Tue May 6 16:21:35 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 6 May 2008 09:21:35 -0500 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> Message-ID: <18464.26991.824584.173058@montanaro-dyndns-org.local> Alec> FWIW my vote is for ~/.python. ~/.local comes in a distant second Alec> due to non-obviousness and ~/Python is several light years beyond Alec> that. I guess we're going to have to agree to disagree. I find hiding directories which contain executable code extremely non-obvious. Would you prefer /usr/.local to /usr/local? If not, then why prefer ~/.local to ~/local? Skip From alec at swapoff.org Tue May 6 17:11:57 2008 From: alec at swapoff.org (Alec Thomas) Date: Wed, 7 May 2008 01:11:57 +1000 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <18464.26991.824584.173058@montanaro-dyndns-org.local> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> Message-ID: <5b52d53b0805060811h34f6cb29sb8539a490b7be3d0@mail.gmail.com> 2008/5/7 : > > Alec> FWIW my vote is for ~/.python. ~/.local comes in a distant second > Alec> due to non-obviousness and ~/Python is several light years beyond > Alec> that. > > I guess we're going to have to agree to disagree. I find hiding directories > which contain executable code extremely non-obvious. Would you prefer Python would not be unique. Mozilla/Firefox does exactly this, putting per-user plugins in ~/.mozilla. > /usr/.local to /usr/local? If not, then why prefer ~/.local to ~/local? Because unlike a home directory, users don't frequently perform directory listings or tab completion of /usr/. For a frequently used personal directory one wants the minimum of noise. Also: 1. If every application followed the convention of creating non-hidden paths in home directories the directory listing would be *incredibly* noisy. To illustrate, I have 160 dotfiles, most of which were created by applications. I have only 8 non-hidden directories, all of which I have created myself. 2. Non-hidden directories interfere with tab completion muscle memory. 3. On a more subjective note, home directories are personal space. People shape them to their personality, and interfering with this is impolite. 4. Per-application dotfiles have 30 years of convention behind them. Conversely, only a few applications use ~/.local (for example, Openbox and Audacious both look for configuration here) and none that I'm aware of default to ~/local. 5. Applications that create non-hidden directories in user home directories are generally perceived as being obnoxious. -- Evolution: Taking care of those too stupid to take care of themselves. From janssen at parc.com Tue May 6 17:30:35 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 6 May 2008 08:30:35 PDT Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <5b52d53b0805060811h34f6cb29sb8539a490b7be3d0@mail.gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> <5b52d53b0805060811h34f6cb29sb8539a490b7be3d0@mail.gmail.com> Message-ID: <08May6.083045pdt."58696"@synergy1.parc.xerox.com> > > /usr/.local to /usr/local? If not, then why prefer ~/.local to ~/local? > > Because unlike a home directory, users don't frequently perform > directory listings or tab completion of /usr/. For a frequently used > personal directory one wants the minimum of noise. Glad someone around here knows actual facts about the statistics of using "ls" :-). Can you point to published user studies about this? If not, let me just say that I perform directory listings of /usr a whole lot *more* than my home directory. Um, isn't this all argument about what color to paint the shed? Bill From mwm at mired.org Tue May 6 17:29:25 2008 From: mwm at mired.org (Mike Meyer) Date: Tue, 6 May 2008 11:29:25 -0400 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> Message-ID: <20080506112925.023901a1@mbook-fbsd> On Fri, 02 May 2008 00:03:24 -0000 glyph at divmod.com wrote: > On 11:45 pm, guido at python.org wrote: > >I like this, except one issue: I really don't like the .local > >directory. I don't see any compelling reason why this needs to be > >~/.local/lib/ -- IMO it should just be ~/lib/. There's no need to hide > >it from view, especially since the user is expected to manage this > >explicitly. > > I've previously given a spirited defense of ~/.local on this list ( > http://mail.python.org/pipermail/python-dev/2008-January/076173.html ) > among other places. > > Briefly, "lib" is not the only directory participating in this > convention; you've also got the full complement of other stuff that > might go into an installation like /usr/local. So, while "lib" might > annoy me a little, "bin etc games include lib lib32 man sbin share src" > is going to get ugly pretty fast, especially if this is what comes up in > Finder or Nautilus or Explorer every time I open a window. You have a problem with 10 directories? Well, ok - if you have that on top of all the clutter that you normally get, yeah, I might object too. On the other hand, if *every* application used those 10 directories - and *only* those 10 directories - for all the files it needed that weren't for user-created data, that would be heaven. The fallacy you're falling into is that users never have to deal with those dot-files (or directories). They do. One of the most common operations when trying to diagnose a misbehaving application is "delete the configuration files" (my favorite is that I fix gnucash printing failures by deleting CUPS config files....), and the user has to figure out which, if any, of those magic files need to be deleted. If you're using Finder, you wind up turning on the preference that says "show me those", and suddenly your nice, clean directory explodes into ... Well, here's my home directory, shared between a Mac and a Unix box: mbook-fbsd% cd mbook-fbsd% ls | wc -l 42 mbook-fbsd% ls -d .* | wc -l 174 It's not very clean. Because it's a Mac, it's got some directories that the Mac felt I needed that I really have no use for. And there's maybe a dozen files there that are scratch files from various things I haven't cleaned up yet. Of course, the dot-files are much worse, because I normally don't see them, so there's not incentive to clean them up at all. But if i could trade those 172 (can't lose . and ..) "hidden" .files for 10 visible directories in ~, I'd do it in an instant - even if I didn't already have bin, etc, src & lib directories there. > Put another way - it's trivial to make ~/.local/lib show up by > symlinking ~/lib, but you can't make ~/lib disappear, and lots of > software ends up looking at ~. Just for the record, it's equally trivial - but better - to make ".local" disappear by symlinking '.local' to '.'. But providing an option is even cleaner, and then the fact that you can't use symlink to hide one is moot. As far as I'm concerned, .local is the worst possible choice for this choice for this name. Not only does it wind up in the more cluttered of the two name spaces, it doesn't tell me anything about the application(s) it belongs to, so I have to worry about it pretty much every time I'm mucking about with the config files. .python would be much better - at least I'd know what it was for by the name. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From jeremy at alum.mit.edu Tue May 6 17:39:21 2008 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 6 May 2008 11:39:21 -0400 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <481B51A8.5050609@v.loewis.de> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> Message-ID: On Fri, May 2, 2008 at 1:38 PM, "Martin v. L?wis" wrote: > Thomas Lee wrote: > > Martin v. L?wis wrote: > >>> This leaves us with a few options: > >>> > >> > >> 5. Reuse/Abuse Num(object) for arbitrary constants. > >> AFAICT, this should work out of the box. > >> > >> > > Eek. It *does* seem like Num would work out of the box, but would this > > be a good idea? > > No. I suggested it just for completeness. > > > > What about *replacing* Num with Const? Might make optimizations > > specifically for numeric values slightly hairier, and semantically I > > think they might be different enough to warrant separate AST nodes > > despite the similarity in implementation at the compiler level. > > I think they should be separate. Const would be a mere addition; > for compatibility with other uses of the AST, that's actually better > than simultaneous removal of Num. Adding Const sounds good to me. Jeremy > > Regards, > Martin > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From jeremy at alum.mit.edu Tue May 6 17:41:09 2008 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Tue, 6 May 2008 11:41:09 -0400 Subject: [Python-Dev] AST Optimization: Branch Elimination in Generator Functions In-Reply-To: <481BEF63.2030506@vector-seven.com> References: <481BEF63.2030506@vector-seven.com> Message-ID: I'm not sure I understand the problem exactly. If we have one pass converting the concrete syntax to the AST, we can mark functions as generators as part of that pass. In a later pass, you can remove the unreachable code. Jeremy On Sat, May 3, 2008 at 12:51 AM, Thomas Lee wrote: > The next problem that cropped up during the implementation of the AST code > optimizer is related to branch elimination and the elimination of any code > after a return. > > Within a FunctionDef node, we would (ideally) like to blow away If nodes > with a constant - but false - test expression. e.g.: > > def foo(): > if False: > # ... stuff ... > > For most functions, this will cause no problems and the code will behave as > expected. However, if the eliminated branch contains a "yield" expression, > the function is actually a generator function - even if the yield expression > can never be reached: > > def foo(): > if False: > yield 5 > > In addition to this, the following should also be treated as a generator > even though we'd like to be able to get rid of all the code following the > "return" statement: > > def foo(): > return > yield 5 > > Again, blowing away the yield results in a normal function instead of a > generator. Not what we want: we need to preserve the generator semantics. > > Upon revisiting this, it's actually made me reconsider the use of a Const > node for the earlier problem relating to arbitrary constants. We may be > better off with annotations after all ... then we could mark FunctionDef > nodes as being generators at the AST level to force the compiler to produce > code for a generator, but eliminate the branches anyway. > > The other alternative I can think of is injecting a yield node somewhere > unreachable and ensuring it doesn't get optimized away, but this seems > pretty hacky in comparison. > > Any other ideas? > > Cheers, > Tom > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From guido at python.org Tue May 6 18:37:00 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 6 May 2008 09:37:00 -0700 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <5b52d53b0805060811h34f6cb29sb8539a490b7be3d0@mail.gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> <5b52d53b0805060811h34f6cb29sb8539a490b7be3d0@mail.gmail.com> Message-ID: On Tue, May 6, 2008 at 8:11 AM, Alec Thomas wrote: > Python would not be unique. Mozilla/Firefox does exactly this, putting > per-user plugins in ~/.mozilla. Note that this is moot since I'm going to accept the PEP as it stands (i.e. ~/.local) but I want to point out something that seems to be lost occasionally. Hiding stuff in dot files is the right thing to do when there's a separate API (like Mozilla) to manage those files. It is IMO much more questionable when the user is expected to manage things directly using the standard filesystem API. That's why Pictures etc. are not dot files. Of course, there's a gray area -- grizzled Unix wizards manage dozens of dot files like .profile and .exrc -- but I still think this is a useful (partial) guiding principle. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 6 18:42:20 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 6 May 2008 09:42:20 -0700 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> Message-ID: On Tue, May 6, 2008 at 7:37 AM, Steve Holden wrote: > If you want it visible, make a visible symbolic link! Note that the point is moot, since I'm going to accept Christian's PEP, i.e. ~/.local, but this argument "you can make it visible yourself" is bogus. The point of visibility (when it's brought up) isn't that you *can* make it visible -- you can always do that with ls -a or whatever Finder option. The point is that (in some people's view) the results of an action should be left *in plain sight* so that the user has clear evidence of what happened. I'm fine in this case with the counterarguments though, so I'll be accepting the PEP in a minute. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Tue May 6 18:48:34 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 6 May 2008 11:48:34 -0500 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <5b52d53b0805060811h34f6cb29sb8539a490b7be3d0@mail.gmail.com> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> <5b52d53b0805060811h34f6cb29sb8539a490b7be3d0@mail.gmail.com> Message-ID: <18464.35810.137067.640251@montanaro-dyndns-org.local> >> /usr/.local to /usr/local? If not, then why prefer ~/.local to ~/local? Alec> Because unlike a home directory, users don't frequently perform Alec> directory listings or tab completion of /usr/. For a frequently Alec> used personal directory one wants the minimum of noise. I don't mind the system clearly telling me about code I've installed. That's a lot different than Mozilla hiding it's internal stuff in ~/.mozilla. Skip From janssen at parc.com Tue May 6 18:43:26 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 6 May 2008 09:43:26 PDT Subject: [Python-Dev] ctypes documentation Message-ID: <08May6.094335pdt."58696"@synergy1.parc.xerox.com> I just made another attempt to use ctypes to wrap a library, and am facing the same problem I had the last time: the documentation doesn't really work. I'm wondering if we have any projects underway to re-write it? Bill From guido at python.org Tue May 6 19:03:59 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 6 May 2008 10:03:59 -0700 Subject: [Python-Dev] [Python-3000] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <481E15B7.9060003@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DC6A3.70104@gmail.com> <481E15B7.9060003@cheimes.de> Message-ID: All, I've accepted PEP 370, Christian Heimes's proposal to add a per-user site-package directory. The location will be somewhere under ~/.local for Unix/Linux/OS X, and %APPDATA%/Python for Windows (per the original proposal in the PEP). Congratulations Christian, and thanks for championing this. Thanks also to everyone who contributed to the discussion and showed the error of my ways -- especially those who did so in under 100 words. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 6 19:16:10 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 6 May 2008 10:16:10 -0700 Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: <20080506111915.GA4193@mithrandi.net> References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <20080506111915.GA4193@mithrandi.net> Message-ID: On Tue, May 6, 2008 at 4:19 AM, Tristan Seligmann wrote: > * Antoine Pitrou [2008-05-06 10:47:23 +0000]: > > Sorry to revive this thread, but mktemp() is very useful when the file is meant > > to be created by another application (e.g. launched by subprocess, but it could > > even be a daemon running under a different user). For example if I have a > > processing chain to converts a PDF to a temporary JPEG using an external tool > > and then does other things with the JPEG: I don't want Python to actually > > create the file, just to generate an unique filename. > > The correct way to do this is to create a temporary directory, and then > generate a filename underneath that directory to use. Good catch. The problem with mktemp() is exactly its convenience, which opens it up for well-documented symlink attacks. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 6 19:23:05 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 6 May 2008 10:23:05 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <20080506074854.GO1016@nexus.in-nomine.org> References: <481C49E5.6080403@behnel.de> <20080506074854.GO1016@nexus.in-nomine.org> Message-ID: On Tue, May 6, 2008 at 12:48 AM, Jeroen Ruigrok van der Werven wrote: > -On [20080505 18:28], Guido van Rossum (guido at python.org) wrote: > >- AppEngine has a built-in API for dealing with Google Accounts, but > >not for OpenID > > > >- Apparently OpenID requires some crypto that would be slow without > >additional API support > > > >- I'm not personally familiar with OpenID > > Check out http://openid-provider.appspot.com/ > > Apparently it was created by your colleague Ryan Barrett. :) It solves the opposite problem -- using a Google Account to log in to an app that requires OpenID. People are clamoring for the opposite here. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 6 19:26:04 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 6 May 2008 10:26:04 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <20080506075101.GP1016@nexus.in-nomine.org> References: <20080506075101.GP1016@nexus.in-nomine.org> Message-ID: On Tue, May 6, 2008 at 12:51 AM, Jeroen Ruigrok van der Werven wrote: > -On [20080505 05:38], Guido van Rossum (guido at python.org) wrote: > > http://code.google.com/p/rietveld/source/browse > > Is it a reference to Gerrit Rietveld (Dutch architect and furniture > designer)? Yes, as I explain here: http://code.google.com/p/rietveld/wiki/CodeReviewBackground > I guess the architect part would make sense for a code review > tool. :) That's purely a coincidence. My dad was an architect and took me on trips of Rietveld's work. I also own some fairly original Zig Zag chairs (much to my family's despair, as they aren't too comfortable. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lxander.m at gmail.com Tue May 6 19:41:35 2008 From: lxander.m at gmail.com (Alexander Michael) Date: Tue, 6 May 2008 13:41:35 -0400 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <20080506075101.GP1016@nexus.in-nomine.org> Message-ID: <525f23e80805061041p367b066ft52ff8e42b514c30e@mail.gmail.com> On Tue, May 6, 2008 at 1:26 PM, Guido van Rossum wrote: > On Tue, May 6, 2008 at 12:51 AM, Jeroen Ruigrok van der Werven > wrote: > > -On [20080505 05:38], Guido van Rossum (guido at python.org) wrote: > > > http://code.google.com/p/rietveld/source/browse > > > > Is it a reference to Gerrit Rietveld (Dutch architect and furniture > > designer)? > > Yes, as I explain here: > http://code.google.com/p/rietveld/wiki/CodeReviewBackground > > > > I guess the architect part would make sense for a code review > > tool. :) > > That's purely a coincidence. My dad was an architect and took me on > trips of Rietveld's work. I also own some fairly original Zig Zag > chairs (much to my family's despair, as they aren't too comfortable. > :-) Was it also a coincidence that Mondrian and Rietveld were both members of the De Stijl movement? If so, a very nice one! From guido at python.org Tue May 6 19:50:46 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 6 May 2008 10:50:46 -0700 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <525f23e80805061041p367b066ft52ff8e42b514c30e@mail.gmail.com> References: <20080506075101.GP1016@nexus.in-nomine.org> <525f23e80805061041p367b066ft52ff8e42b514c30e@mail.gmail.com> Message-ID: On Tue, May 6, 2008 at 10:41 AM, Alexander Michael wrote: > Was it also a coincidence that Mondrian and Rietveld were both members > of the De Stijl movement? If so, a very nice one! No, that's my naming scheme. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Tue May 6 22:22:47 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 6 May 2008 16:22:47 -0400 Subject: [Python-Dev] Invitation to try out open source code review tool References: <20080506075101.GP1016@nexus.in-nomine.org><525f23e80805061041p367b066ft52ff8e42b514c30e@mail.gmail.com> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20805061050w55836effn4fefcca8a0a635ea at mail.gmail.com... | On Tue, May 6, 2008 at 10:41 AM, Alexander Michael wrote: | > Was it also a coincidence that Mondrian and Rietveld were both members | > of the De Stijl movement? If so, a very nice one! | | No, that's my naming scheme. :-) Neat. I think I see a few echoes of De Stijl esthetic in Python. "Proponents of De Stijl sought to express a new utopian ideal of spiritual harmony and order. They advocated pure abstraction and universality by a reduction to the essentials of form and colour" from http://en.wikipedia.org/wiki/De_stijl The group was one of the best things to come out of the Great Stupid War. I wonder who is next. Van Doesburg, the group's BDFL? (I know... wait ;-) tjr From musiccomposition at gmail.com Tue May 6 22:43:35 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 6 May 2008 15:43:35 -0500 Subject: [Python-Dev] ctypes documentation In-Reply-To: <-7981895834533490698@unknownmsgid> References: <-7981895834533490698@unknownmsgid> Message-ID: <1afaf6160805061343j413c686brd6a187715ee4294a@mail.gmail.com> On Tue, May 6, 2008 at 11:43 AM, Bill Janssen wrote: > I just made another attempt to use ctypes to wrap a library, and am > facing the same problem I had the last time: the documentation doesn't > really work. I'm wondering if we have any projects underway to > re-write it? So what exactly is eating you about it? -- Cheers, Benjamin Peterson From okkotonushi at googlemail.com Tue May 6 23:44:04 2008 From: okkotonushi at googlemail.com (Robert Schuppenies) Date: Tue, 06 May 2008 23:44:04 +0200 Subject: [Python-Dev] Another GSoC Student Introduction In-Reply-To: <20080430205531.GA15767@vortex.local.net> References: <20080430205531.GA15767@vortex.local.net> Message-ID: <4820D124.8000606@googlemail.com> Hi Everybody. My name is Robert Schuppenies and I also got accepted for this years Google Summer of Code. I will be working on the Python Core with the Memory Usage Profiler project and my mentor will be Martin von L?wis. I am very happy that I got selected and look forward to work on my project. Many thanks to Martin and everybody else making this possible. cheers, robert From greg.ewing at canterbury.ac.nz Wed May 7 00:16:05 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 May 2008 10:16:05 +1200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: <20080506075101.GP1016@nexus.in-nomine.org> References: <20080506075101.GP1016@nexus.in-nomine.org> Message-ID: <4820D8A5.8080405@canterbury.ac.nz> Jeroen Ruigrok van der Werven wrote: > Is it a reference to Gerrit Rietveld (Dutch architect and furniture > designer)? I guess the architect part would make sense for a code review > tool. :) According to Wikipedia, he was also a friend of Mondrian and incorporated some of Mondrian's ideas into his designs. -- Greg From greg.ewing at canterbury.ac.nz Wed May 7 00:23:07 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 May 2008 10:23:07 +1200 Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: <20080506111915.GA4193@mithrandi.net> References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <20080506111915.GA4193@mithrandi.net> Message-ID: <4820DA4B.5050503@canterbury.ac.nz> Tristan Seligmann wrote: > The correct way to do this is to create a temporary directory, and then > generate a filename underneath that directory to use. Yes, otherwise you can get bitten by the race condition that's the reason for mktemp being deprecated. -- Greg From lists at cheimes.de Wed May 7 00:38:16 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 07 May 2008 00:38:16 +0200 Subject: [Python-Dev] PEP 370 heads up Message-ID: <4820DDD8.2040600@cheimes.de> Dear fellow Pythonistas! Guido has accepted my user site directory PEP today. http://python.org/dev/peps/pep-0370/ I'm about the merge the code. But first I like to let you know some things and get your opinion. The location of the user site-package directory stays as written in my PEP. The master himself said "The location will be somewhere under ~/.local for Unix/Linux/OS X, and %APPDATA%/Python for Windows (per the original proposal in the PEP)." Please do NOT take my check-in message as an opportunity to start yet the third bike shed discussion. Thank you very much! :) The PEP adds two environment vars to adjust the location of the user site directory to your liking or to disable the feature entirely. If you are still not convinced you are better off by adding a single line to your .profile file. For now the unit tests are creating the user site directory tree. I'm going to chance that in a few days unless everybody is fine with it. The docs aren't finished yet. Unless somebody is faster than me (*wink*) I'm going to work on the docs after I've finished the rework of the json module. As Brett suggested I'm going to module its input and return values after the pickle module before I merge it into the py3k branch. It's currently mixing unicode and str in a way which is not compatible with py3k. I've added some additional code to the site module. It should become in handy for scripters, debugging and unit tests: $ ./python -m site sys.path = [ '/home/heimes/dev/python/trunk', '/usr/local/lib/python26.zip', '/home/heimes/dev/python/trunk/Lib', '/home/heimes/dev/python/trunk/Lib/plat-linux2', '/home/heimes/dev/python/trunk/Lib/lib-tk', '/home/heimes/dev/python/trunk/Modules', '/home/heimes/dev/python/trunk/build/lib.linux-i686-2.6-pydebug', '/home/heimes/.local/lib/python2.6/site-packages', ] USER_BASE: '/home/heimes/.local' (exists) USER_SITE: '/home/heimes/.local/lib/python2.6/site-packages' (exists) ENABLE_USER_SITE: True $ ./python -m site --help /home/heimes/dev/python/trunk/Lib/site.py [--user-base] [--user-site] Without arguments print some useful information With arguments print the value of USER_BASE and/or USER_SITE separated by ':'. Exit codes with --user-base or --user-site: 0 - user site directory is enabled 1 - user site diretory is disabled by user 2 - uses site directory is disabled by super user or for security reasons >2 - unknown error $ ./python -m site --user-base --user-site /home/heimes/.local:/home/heimes/.local/lib/python2.6/site-packages $ echo $? 0 $ ./python -s -m site --user-base --user-site /home/heimes/.local:/home/heimes/.local/lib/python2.6/site-packages $ echo $? 1 Christian From barry at python.org Wed May 7 00:43:05 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 6 May 2008 18:43:05 -0400 Subject: [Python-Dev] PEP 370 heads up In-Reply-To: <4820DDD8.2040600@cheimes.de> References: <4820DDD8.2040600@cheimes.de> Message-ID: <8F920497-9672-4AE4-9DD2-FC76F98EC568@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 6, 2008, at 6:38 PM, Christian Heimes wrote: > Guido has accepted my user site directory PEP today. > http://python.org/dev/peps/pep-0370/ > > I'm about the merge the code. But first I like to let you know some > things and get your opinion. Very awesome Christian! I'm psyched for this to get into the last alpha releases, which I remind everyone happens tomorrow. Plan on svn tree freeze at approximately 6pm EDT (2200 UTC). Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCDe+nEjvBPtnXfVAQJJMAP/XiZvXPptw8tZ4/01hD7r39/lWgoDUmjp gVzne4+XMfz8NcLQMP2+Y38cPrQziyG8BYDqN/vWT641bOwv20QHuZYFvI9Kr09q jTEC39DzNRfD6ThzD/na6M1M7glpXiWr3hj4Va56JEnn1ekj6Ejb7BoW1oyuyz6T gUuAgVT2lOw= =2IIq -----END PGP SIGNATURE----- From lists at cheimes.de Wed May 7 00:50:37 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 07 May 2008 00:50:37 +0200 Subject: [Python-Dev] PEP 370 heads up In-Reply-To: <8F920497-9672-4AE4-9DD2-FC76F98EC568@python.org> References: <4820DDD8.2040600@cheimes.de> <8F920497-9672-4AE4-9DD2-FC76F98EC568@python.org> Message-ID: <4820E0BD.9040405@cheimes.de> Barry Warsaw schrieb: > Very awesome Christian! I'm psyched for this to get into the last alpha > releases, which I remind everyone happens tomorrow. Plan on svn tree > freeze at approximately 6pm EDT (2200 UTC). Thanks Barry! Also thanks to Glyph, Nick and all the other people that stepped in during the discussion in favor of ~/.local! Christian PS: I'll try to get json into shape for Python 3.0. It's going to be tricky for various reasons For example the re module still doesn't support bytes. From lists at cheimes.de Wed May 7 00:52:09 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 07 May 2008 00:52:09 +0200 Subject: [Python-Dev] r62778 - in python/branches/py3k: Lib/io.py Lib/test/test_StringIO.py Lib/test/test_io.py Lib/test/test_largefile.py Lib/test/test_memoryio.py Lib/test/test_mimetools.py Modules/_bytesio.c Modules/_fileio.c Python/pythonrun.c setup.py In-Reply-To: <20080506194839.C70461E400A@bag.python.org> References: <20080506194839.C70461E400A@bag.python.org> Message-ID: <4820E119.6010209@cheimes.de> alexandre.vassalotti schrieb: > Author: alexandre.vassalotti > Date: Tue May 6 21:48:38 2008 > New Revision: 62778 > > Log: > Added fast alternate io.BytesIO implementation and its test suite. > Removed old test suite for StringIO. > Modified truncate() to imply a seek to given argument value. Thanks for your great work! But what about the trunk? :] Can you port your code to the trunk before the alpha gets out? Christian From greg.ewing at canterbury.ac.nz Wed May 7 01:11:51 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 May 2008 11:11:51 +1200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <18464.26991.824584.173058@montanaro-dyndns-org.local> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> Message-ID: <4820E5B7.7080208@canterbury.ac.nz> skip at pobox.com wrote: > I guess we're going to have to agree to disagree. I find hiding directories > which contain executable code extremely non-obvious. I'm worried by this too. I don't like the idea of putting large and important things in hidden directories automatically without being told about it. -- Greg From greg.ewing at canterbury.ac.nz Wed May 7 01:17:15 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 May 2008 11:17:15 +1200 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> Message-ID: <4820E6FB.5010108@canterbury.ac.nz> Steve Holden wrote: > If you want it visible, make a visible symbolic link! I have to know it's there first. The idea of an installer deliberately hiding stuff from me in a very unconventional and non-obvious way makes me uncomfortable. -- Greg From greg.ewing at canterbury.ac.nz Wed May 7 01:37:28 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 May 2008 11:37:28 +1200 Subject: [Python-Dev] [Python-3000] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DC6A3.70104@gmail.com> <481E15B7.9060003@cheimes.de> Message-ID: <4820EBB8.7070108@canterbury.ac.nz> Guido van Rossum wrote: > The location will be somewhere under ~/.local > for Unix/Linux/OS X, Can I just put in a plea for this to be somewhere under ~/Library on OSX, and not hidden? MacOSX already has clear conventions for this sort of thing -- there's no need to use a dot-name there. -- Greg From brett at python.org Wed May 7 01:54:28 2008 From: brett at python.org (Brett Cannon) Date: Tue, 6 May 2008 16:54:28 -0700 Subject: [Python-Dev] A Python-Dev bundle for TextMate Message-ID: I just checked in Misc/TextMate/Python-Dev.tmbundle for those of you use TextMate. I have been using this bundle for a while and I figured others could use it. Usage tends to expect that you have opened a project from the top-level directory of a Python checkout. There is support to run the Makefile, build the docs, open the documentation index file (which can be set to an environment variable when you are not in a checkout), open an issue in the browser, and open a PEP in a browser. There is even a command to generate the boilerplate for marking a file to be removed in 3.0, complete with automatic fill-in of the module name in the message string and set up to be triggered as a tab trigger. I am hoping to continue to tweak the bundle as time goes on (and permits) with more stuff that is useful. This is not meant to replace the Python bundle, though! This is merely to add stuff specific to hack on Python itself. -Brett From lists at cheimes.de Wed May 7 02:09:07 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 07 May 2008 02:09:07 +0200 Subject: [Python-Dev] [Python-3000] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <4820EBB8.7070108@canterbury.ac.nz> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DC6A3.70104@gmail.com> <481E15B7.9060003@cheimes.de> <4820EBB8.7070108@canterbury.ac.nz> Message-ID: <4820F323.3040903@cheimes.de> Greg Ewing schrieb: > Guido van Rossum wrote: >> The location will be somewhere under ~/.local >> for Unix/Linux/OS X, > > Can I just put in a plea for this to be somewhere > under ~/Library on OSX, and not hidden? > > MacOSX already has clear conventions for this sort > of thing -- there's no need to use a dot-name there. ~/Library is only used by framework builds of Python so far. Framework builds include ~/Library/Python/2.6/site-packages and ~/.local/.. and non-framework builds just the ~/.local directory. Can you come up with a path that fits the purpose of a base directory? Christian From thomaspinckney3 at gmail.com Wed May 7 04:06:01 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Tue, 6 May 2008 22:06:01 -0400 Subject: [Python-Dev] urllib unicode handling Message-ID: <101755A8-570C-449A-926F-A2827F563182@gmail.com> Hi, While trying to use urllib in python 2.5.1 to HTTP GET content from various web sites, I've run into a problem with urllib.quote (and .quote_plus): they don't accept unicode strings. I see that this is an issue that has been discussed before: see this thread: http://mail.python.org/pipermail/python-dev/2006-July/067248.html especially this post: http://mail.python.org/pipermail/python-dev/2006-July/067335.html While I don't really want to re-open a can of worms, it seems that the current implementation of urllib.quote and urllib.quote_plus is painfully incompatible with how the web (circa 2008) actually works. While the standards may say there is no official way to represent unicode strings in URLs, in practice the world uses UTF-8 quite heavily. For example, I found the following URLs in Google pretty quickly by looking for percent encoded utf-8 encoded accented e's. http://www.last.fm/music/Jos%C3%A9+Gonz%C3%A1lez http://en.wikipedia.org/wiki/Joseph_Fouch%C3%A9 http://apps.facebook.com/ilike/artist/Jos%C3%A9+Gonz%C3%A1lez/track/Stay+In+The+Shade?apv=1 While in theory UTF-8 is not a standard, sites like Last.fm, Facebook and Wikipedia seem to have embraced it (as have pretty much all other major web sites). As with HTML, there is what the standard says and what the actual browsers have to accept in order to work in the real world. urllib.urlencode already converts unicode characters to their UTF-8 representation before percent encoding them. Why not urllib.quote and urllib.quote_plus? Thanks for any thoughts on this, Tom From martin at v.loewis.de Wed May 7 07:10:48 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 07 May 2008 07:10:48 +0200 Subject: [Python-Dev] Invitation to try out open source code review tool In-Reply-To: References: <481C49E5.6080403@behnel.de> <20080506074854.GO1016@nexus.in-nomine.org> Message-ID: <482139D8.7050102@v.loewis.de> >> Check out http://openid-provider.appspot.com/ >> >> Apparently it was created by your colleague Ryan Barrett. :) I found that to be of limited use. If I use my proper gmail account (mvloewis at googlemail.com), I managed to log into the OpenID directory, but not, say, into SourceForge (probably because SF doesn't accept that specific provider - so it's not truly open, and probably shouldn't be). If I used it with the Google account associated with my regular email address, I couldn't login anywhere at all. In any case, this provider apparently didn't provide an email address to the site. That makes it useless for most applications I can think of, unless it's acceptable to go through a setup procedure again at the new site that asks for all the user details (real name, email address) again. > It solves the opposite problem -- using a Google Account to log in to > an app that requires OpenID. People are clamoring for the opposite > here. Apparently, this comes from the appengine samples, http://code.google.com/p/google-app-engine-samples/ which also includes an OpenID consumer. I'm fairly skeptical about this OpenID thing. I personally couldn't remember any of my OpenIDs, because they are just too long; plus I haven't actually seen this really work anywhere yet. Regards, Martin From martin at v.loewis.de Wed May 7 07:16:12 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 07 May 2008 07:16:12 +0200 Subject: [Python-Dev] urllib unicode handling In-Reply-To: <101755A8-570C-449A-926F-A2827F563182@gmail.com> References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> Message-ID: <48213B1C.4080205@v.loewis.de> > Thanks for any thoughts on this, The proper way to implement this would be IRIs (RFC 3987), in particular section 3.1. This is not as simple as just encoding it as UTF-8, as you might have to apply IDNA to the host part. Code doing so just hasn't been contributed yet. Regards, Martin From asmodai at in-nomine.org Wed May 7 07:20:16 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Wed, 7 May 2008 07:20:16 +0200 Subject: [Python-Dev] urllib unicode handling In-Reply-To: <101755A8-570C-449A-926F-A2827F563182@gmail.com> References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> Message-ID: <20080507052016.GV1016@nexus.in-nomine.org> -On [20080507 04:06], Tom Pinckney (thomaspinckney3 at gmail.com) wrote: >While in theory UTF-8 is not a standard, sites like Last.fm, Facebook and >Wikipedia seem to have embraced it (as have pretty much all other major web >sites). As with HTML, there is what the standard says and what the actual >browsers have to accept in order to work in the real world. I agree with you. The dictionary project I am working on (Dutch <> Japanese) uses in the URLs UTF-8 characters and things just worked with reasonably new browsers (at least no problems with Opera 9, Firefox 2 and 3, Internet Explorer 7 and Safari 3). Then later Armin Ronacher warned me that you still have to URL-escape these things in order to not be in lala-land. Would people object if such functionality got added to urllib? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B If Winter comes, can Spring be far behind..? From stephen at xemacs.org Wed May 7 08:23:11 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 07 May 2008 15:23:11 +0900 Subject: [Python-Dev] [Python-3000] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <4820E5B7.7080208@canterbury.ac.nz> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <61E8E3D5-B66F-4CA0-B5CF-3A7BB3CE12BD@python.org> <5b52d53b0805060545s513ce7e4m52f07275e450ba9a@mail.gmail.com> <18464.26991.824584.173058@montanaro-dyndns-org.local> <4820E5B7.7080208@canterbury.ac.nz> Message-ID: <87zlr2ab00.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > skip at pobox.com wrote: > > I guess we're going to have to agree to disagree. I find hiding directories > > which contain executable code extremely non-obvious. > > I'm worried by this too. I don't like the idea of putting > large and important things in hidden directories automatically > without being told about it. So what we need is a PEP specifying that installers have -q and -v switches, defaulting to -v, right? From martin at v.loewis.de Wed May 7 08:29:28 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 07 May 2008 08:29:28 +0200 Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: References: <18452.65322.843226.299769@montanaro-dyndns-org.local> Message-ID: <48214C48.2040005@v.loewis.de> > Sorry to revive this thread, but mktemp() is very useful when the file is meant > to be created by another application (e.g. launched by subprocess, but it could > even be a daemon running under a different user). For example if I have a > processing chain to converts a PDF to a temporary JPEG using an external tool > and then does other things with the JPEG: I don't want Python to actually > create the file, just to generate an unique filename. Are you (or are you not) aware that this strategy allows for malicious code to provide you with a fake JPEG file? If so, does it not concern you? As others have said: the reason the function is deprecated is that it is easy to run into security problems while using it, and users are often unaware of the security implications. Of course, for many applications, there is no real threat, and any risk might be acceptable. Unfortunately, if a security catastrophe results from the function, blame is (correctly) also upon Python for providing the function in the first place. Regards, Martin From ziade.tarek at gmail.com Wed May 7 09:44:54 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 7 May 2008 09:44:54 +0200 Subject: [Python-Dev] #1858, looking for a reviewer In-Reply-To: <94bdd2610805070025s5899ea26ke5fac80c5f70aab9@mail.gmail.com> References: <94bdd2610805070025s5899ea26ke5fac80c5f70aab9@mail.gmail.com> Message-ID: <94bdd2610805070044h93d521ev61f77cfa1e252c56@mail.gmail.com> Hi, The register and upload commands have a --repository option that allows to use them with any server. You can register and upload a package to any server that has the right set of web services. They also work with a configuration file called .pypirc Unfortunately there is no inline option to define the username/password. This has to be defined in .pypirc. But this file does not allow having different sets of username/password, so in reality it is not possible to play with multiple servers. #1858 provides a patch for this [0] I have submitted it in January, after some discussion in catalog-sig and distutils-sig, that led to a proposal [1] I am looking for a final reviewer, to decide if the new .pypirc is considered suitable, and maybe have it integrated in the 2.6. This new format will be used in the Plone community to interact with plone.org and the cheeseshop [2], (with a third party package at this time, that implements the patch) Regards Tarek [0] http://bugs.python.org/issue1858 [1] http://wiki.python.org/moin/EnhancedPyPI [2] http://tarekziade.wordpress.com/2008/05/06/plone-paris-sprint-wrapup-3-newploneorg-collectivedist-released/ -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From solipsis at pitrou.net Wed May 7 10:09:52 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 7 May 2008 08:09:52 +0000 (UTC) Subject: [Python-Dev] Warn about mktemp once again? References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <48214C48.2040005@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > Are you (or are you not) aware that this strategy allows for malicious > code to provide you with a fake JPEG file? If so, does it not concern > you? Yes, I'm aware of it and no, it doesn't concern me. When you write a processing script for internal use there's no reason to worry about security issues like that (if someone manages to enter your private machine he can do much more dangereous things anyway). > As others have said: the reason the function is deprecated is that it > is easy to run into security problems while using it, I know, I've read the doc :) I'm sure there are lots of other insecure things in Python's stdlib (e.g. HTTP Basic Authentication, or plaintext authentication in email clients, or algorithms known to be weak in hashlib) and they haven't been deprecated. Of course some of those may be necessary for interoperability, while mktemp() isn't. In any case what I wanted to point out is that the secure replacement for a one-liner mktemp() is not as concise and obvious as the insecure version, and that sometimes there are good reasons to not even care about the "security problems". Regards Antoine. From tdickenson at geminidataloggers.com Wed May 7 10:40:49 2008 From: tdickenson at geminidataloggers.com (Toby Dickenson) Date: Wed, 07 May 2008 09:40:49 +0100 Subject: [Python-Dev] Warn about mktemp once again? References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <20080506111915.GA4193@mithrandi.net> Message-ID: Tristan Seligmann wrote: > The correct way to do this is to create a temporary directory, and then > generate a filename underneath that directory to use. There is a platform difference here. On unix mktemp will usually provide a file name in a world-writeable directory (/tmp/) which is wide open to race condition attacks leading to privilege escalation. On win32 it will usually (but not always) provide a file name in a directory writeable only by the current user. The temporary directory step sometimes seems unnecessary to windows developers. From sjoerd at acm.org Wed May 7 12:49:50 2008 From: sjoerd at acm.org (Sjoerd Mullender) Date: Wed, 07 May 2008 12:49:50 +0200 Subject: [Python-Dev] sock.close() not closing? Message-ID: <4821894E.3000304@acm.org> Why does sock.close() not actually close sock? If I run the code import socket sock = socket.socket() ... sock.close() I would expect that a system call is done to actually close the socket and free the file descriptor. But that does not happen. Look at the code in socket.py. It merely replaces the socket instance with a dummy instance so that all subsequent calls on the sock object fail, but it does nothing else! -- Sjoerd Mullender From amauryfa at gmail.com Wed May 7 13:37:31 2008 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 7 May 2008 13:37:31 +0200 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: <4821894E.3000304@acm.org> References: <4821894E.3000304@acm.org> Message-ID: Hello, 2008/5/7 Sjoerd Mullender : > Why does sock.close() not actually close sock? > > If I run the code > > import socket > sock = socket.socket() > ... > sock.close() > > I would expect that a system call is done to actually close the socket and > free the file descriptor. But that does not happen. Look at the code in > socket.py. It merely replaces the socket instance with a dummy instance so > that all subsequent calls on the sock object fail, but it does nothing else! It does close the socket: In socket.py, when self._sock is replaced, its __del__ method will be called. This __del__ is implemented in C, in socketmodule.c: static void sock_dealloc(PySocketSockObject *s) { if (s->sock_fd != -1) (void) SOCKETCLOSE(s->sock_fd); Py_TYPE(s)->tp_free((PyObject *)s); } Of course, if you call sock.dup() or sock.makefile(), there is another reference to the underlying _sock, and you must close() all these objects. -- Amaury Forgeot d'Arc From armin.ronacher at active-4.com Wed May 7 14:12:05 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Wed, 7 May 2008 12:12:05 +0000 (UTC) Subject: [Python-Dev] urllib unicode handling References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <20080507052016.GV1016@nexus.in-nomine.org> Message-ID: Hi, Jeroen Ruigrok van der Werven in-nomine.org> writes: > Would people object if such functionality got added to urllib? I would ;-) There are IRIs, just that nobody wrote a useful module for that. There are algorithms in the RFC that can convert URIs to IRIs and the other way round. IMO that's the way to go. Regards, Armin From kristjan at ccpgames.com Wed May 7 14:00:51 2008 From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=) Date: Wed, 7 May 2008 12:00:51 +0000 Subject: [Python-Dev] urllib unicode handling In-Reply-To: <20080507052016.GV1016@nexus.in-nomine.org> References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <20080507052016.GV1016@nexus.in-nomine.org> Message-ID: <4E9372E6B2234D4F859320D896059A950FE1305D21@exchis.ccp.ad.local> > -----Original Message----- > From: python-dev-bounces+kristjan=ccpgames.com at python.org > [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf > Of Jeroen Ruigrok van der Werven > Sent: Wednesday, May 07, 2008 05:20 > To: Tom Pinckney > Cc: python-dev at python.org > Subject: Re: [Python-Dev] urllib unicode handling > > -On [20080507 04:06], Tom Pinckney (thomaspinckney3 at gmail.com) wrote: > >While in theory UTF-8 is not a standard, sites like Last.fm, Facebook > and > >Wikipedia seem to have embraced it (as have pretty much all other > major web > >sites). As with HTML, there is what the standard says and what the > actual > >browsers have to accept in order to work in the real world. > FYI, here is how we have patched urrlib2 for use in EVE: --- C:\p4\sdk\stackless25\Lib\urllib.py 2008-03-21 14:47:23.000000000 -0000 +++ C:\p4\eve\KALI\common\stdlib\urllib.py 2007-11-06 11:18:01.000000000 -0000 @@ -1158,12 +1158,29 @@ except KeyError: res[i] = '%' + item except UnicodeDecodeError: res[i] = unichr(int(item[:2], 16)) + item[2:] return "".join(res) +unquote_inner = unquote +def unquote(s): + """CCP attempt at making sensible choices in unicode quoteing / unquoting """ + s = unquote_inner(s) + try: + u = s.decode("utf-8") + try: + s2 = s.decode("ascii") + except UnicodeDecodeError: + s = u #yes, s was definitely utf8, which isn't pure ascii + else: + if u != s: + s = u + except UnicodeDecodeError: + pass #can't have been utf8 + return s + def unquote_plus(s): """unquote('%7e/abc+def') -> '~/abc def'""" s = s.replace('+', ' ') return unquote(s) always_safe = ('ABCDEFGHIJKLMNOPQRSTUVWXYZ' @@ -1201,12 +1218,20 @@ for i in range(256): c = chr(i) safe_map[c] = (c in safe) and c or ('%%%02X' % i) _safemaps[cachekey] = safe_map res = map(safe_map.__getitem__, s) return ''.join(res) + +quote_inner = quote +def quote(s, safe = '/'): + """CCP addition, to try to sensibly support / circumvent issues with unicode in urls""" + try: + return quote_inner(s, safe) + except KeyError: + return quote_inner(s.encode("utf-8", safe)) def quote_plus(s, safe = ''): """Quote the query fragment of a URL; replacing ' ' with '+'""" if ' ' in s: s = quote(s, safe + ' ') return s.replace(' ', '+') From thomaspinckney3 at gmail.com Wed May 7 15:19:41 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Wed, 7 May 2008 09:19:41 -0400 Subject: [Python-Dev] urllib unicode handling In-Reply-To: References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <20080507052016.GV1016@nexus.in-nomine.org> Message-ID: I may be missing something, but it seems that RFC 3987 (which is about IRIs) basically says: 1) IRIs are identical to URIs except they may have unicode characters in them 2) IRIs must be converted to URIs before being used in HTTP 3) The way to convert IRIs to URIs is to UTF-8 encode the unicode characters in the IRI and then percent encode the resulting octects that are unsafe to have in a URI 4) There's some ambiguity over what to do with the hostname portion of the URI if it hash one (IDN, replace non-ascii characters with dashes etc) If this is indeed the case, it sounds perfectly legal (according to the RFC) and perfectly practical (as required by numerous popular websites) to have urllib.quote and urllib.quote_plus do an automatic UTF-8 encoding of unicode strings before percent encoding them. It's not entirely clear to me if people should be calling urllib.quote on hostnames and expecting them to be encoded properly if the hostname contains non-ascii characters. Perhaps the docs should be clarified on this matter? Similarly, urllib.unquote should precent-decode characters and then attempt to convert the resulting octects from utf-8 to unicode. If that conversion fails, we can assume the octects should be returned as a byte string rather than a unicode string. On May 7, 2008, at 8:12 AM, Armin Ronacher wrote: > Hi, > > Jeroen Ruigrok van der Werven in-nomine.org> writes: > >> Would people object if such functionality got added to urllib? > I would ;-) There are IRIs, just that nobody wrote a useful module > for that. > There are algorithms in the RFC that can convert URIs to IRIs and > the other way > round. IMO that's the way to go. > > Regards, > Armin > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/thomaspinckney3%40gmail.com From exarkun at divmod.com Wed May 7 16:11:20 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 7 May 2008 10:11:20 -0400 Subject: [Python-Dev] Community buildbots In-Reply-To: 0 Message-ID: <20080507141120.6859.2120454231.divmod.quotient.60068@ohm> Hi, I just wanted to point out a few things: Community 2.5 bots, 6 out of 8 offline, of the remaining two (which are both red), one is actually using Python 2.6, not Python 2.5: http://python.org/dev/buildbot/community/2.5/ Community 2.6 bots, 6 out of 8 offline, but at least the remaining two (both of which are red) seem to be using the correct Python version: http://python.org/dev/buildbot/community/trunk/ Jean-Paul From sjoerd at acm.org Wed May 7 16:37:20 2008 From: sjoerd at acm.org (Sjoerd Mullender) Date: Wed, 07 May 2008 16:37:20 +0200 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: References: <4821894E.3000304@acm.org> Message-ID: <4821BEA0.3060402@acm.org> On 2008-05-07 13:37, Amaury Forgeot d'Arc wrote: > Hello, > > 2008/5/7 Sjoerd Mullender : >> Why does sock.close() not actually close sock? >> >> If I run the code >> >> import socket >> sock = socket.socket() >> ... >> sock.close() >> >> I would expect that a system call is done to actually close the socket and >> free the file descriptor. But that does not happen. Look at the code in >> socket.py. It merely replaces the socket instance with a dummy instance so >> that all subsequent calls on the sock object fail, but it does nothing else! > > It does close the socket: > > In socket.py, when self._sock is replaced, its __del__ method will be called. > This __del__ is implemented in C, in socketmodule.c: > > static void > sock_dealloc(PySocketSockObject *s) > { > if (s->sock_fd != -1) > (void) SOCKETCLOSE(s->sock_fd); > Py_TYPE(s)->tp_free((PyObject *)s); > } > > > Of course, if you call sock.dup() or sock.makefile(), > there is another reference to the underlying _sock, and you must > close() all these objects. > I have to question the design of this. When I close() an object I expect it to be closed there and then and not at some indeterminate later time (well, it is determinate when you're fully aware of all references, but often you aren't--trust me, I understand reference counting). Then there also seems to be a bug in imaplib.IMAP4_SSL since in its shutdown method it closes the socket but leaves the sslobj untouched. I assume that that object keeps a reference to the socket. But as I said, I expect the close() to actually close. -- Sjoerd Mullender From ncoghlan at gmail.com Wed May 7 16:56:29 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 08 May 2008 00:56:29 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> Message-ID: <4821C31D.9000506@gmail.com> Jeremy Hylton wrote: > On Fri, May 2, 2008 at 1:38 PM, "Martin v. L?wis" wrote: >> Thomas Lee wrote: >> > Martin v. L?wis wrote: >> >>> This leaves us with a few options: >> >>> >> >> >> >> 5. Reuse/Abuse Num(object) for arbitrary constants. >> >> AFAICT, this should work out of the box. >> >> >> >> >> > Eek. It *does* seem like Num would work out of the box, but would this >> > be a good idea? >> >> No. I suggested it just for completeness. >> >> >> > What about *replacing* Num with Const? Might make optimizations >> > specifically for numeric values slightly hairier, and semantically I >> > think they might be different enough to warrant separate AST nodes >> > despite the similarity in implementation at the compiler level. >> >> I think they should be separate. Const would be a mere addition; >> for compatibility with other uses of the AST, that's actually better >> than simultaneous removal of Num. > > Adding Const sounds good to me. As Thomas mentions in a later message, making it possible to annotate nodes would permit Functions to be annotated as being a generator at the AST stage (currently it is left to the bytecode compiler's symtable generation pass to make that determination). Although I guess an alternative solution to that would be to have separate AST nodes for Functions and Generators as well... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From janssen at parc.com Wed May 7 17:36:55 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 7 May 2008 08:36:55 PDT Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <20080506111915.GA4193@mithrandi.net> Message-ID: <08May7.083702pdt."58696"@synergy1.parc.xerox.com> > On win32 it will usually (but not always) provide a file name in a directory > writeable only by the current user. OS X 10.5 also does this. >>> import tempfile >>> tempfile.mktemp() '/var/folders/Ru/RuapMUan2RWWJ++1YwBnRU++0T6/-Tmp-/tmpjjkKha' >>> os.system("ls -dl /var/folders/Ru/RuapMUan2RWWJ++1YwBnRU++0T6/-Tmp-") drwx------ 2 wjanssen 100 68 May 7 08:34 /var/folders/Ru/RuapMUan2RWWJ++1YwBnRU++0T6/-Tmp- 0 >>> Bill From tom at vector-seven.com Wed May 7 17:43:45 2008 From: tom at vector-seven.com (Thomas Lee) Date: Thu, 08 May 2008 01:43:45 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <4821C31D.9000506@gmail.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> Message-ID: <4821CE31.40609@vector-seven.com> Nick Coghlan wrote: > > As Thomas mentions in a later message, making it possible to annotate > nodes would permit Functions to be annotated as being a generator at > the AST stage (currently it is left to the bytecode compiler's > symtable generation pass to make that determination). > > Although I guess an alternative solution to that would be to have > separate AST nodes for Functions and Generators as well... > I've actually backtracked a little and gone back down the Const path again. I know this is the third time I've changed my mind, but it's primarily because annotations tend to get a little clunky (or my implementation was, at least). Using Const nodes feels a lot more natural inside the optimizer. I think it's going to stick, at least in the short term. Rather than separate FunctionDef and GeneratorDef nodes, I think a new bool attribute (is_generator?) on FunctionDef should do the job nicely. Further, I'm thinking we can move the "generator detection code" from the symtable into Python/ast.c so the generator/function information is available to the optimizer. This is made a little tricky by the absence of the contextual information available that is normally available when flagging generators in the symtable. When generating AST nodes for a suite, we know nothing about the parent node in which the suite resides. Still, it might be doable. If this winds up being ugly, we might need to fall back to the original plan of a separate pass over function bodies to detect yield expressions. I'll look into all this tomorrow night, along with any other crazy suggestions. For now I need to sleep a few hours. :) Thanks for the feedback, it's much appreciated. Cheers, T From fumanchu at aminus.org Wed May 7 17:55:34 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Wed, 7 May 2008 08:55:34 -0700 Subject: [Python-Dev] urllib unicode handling In-Reply-To: <48213B1C.4080205@v.loewis.de> References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <48213B1C.4080205@v.loewis.de> Message-ID: "Martin v. L?wis" wrote: > The proper way to implement this would be IRIs (RFC 3987), > in particular section 3.1. This is not as simple as just > encoding it as UTF-8, as you might have to apply IDNA to > the host part. > > Code doing so just hasn't been contributed yet. But if someone wanted to do so, it's pretty simple: >>> u'www.\u212bngstr\xf6m.com'.encode("idna") 'www.xn--ngstrm-hua5l.com' Robert Brewer fumanchu at aminus.org From fumanchu at aminus.org Wed May 7 18:07:08 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Wed, 7 May 2008 09:07:08 -0700 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: <4821BEA0.3060402@acm.org> References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> Message-ID: Sjoerd Mullender wrote: > On 2008-05-07 13:37, Amaury Forgeot d'Arc wrote: > > 2008/5/7 Sjoerd Mullender : > >> I would expect that a system call is done to actually close the > >> socket and free the file descriptor. But that does not happen. > > > > It does close the socket: > > > > In socket.py, when self._sock is replaced, its __del__ method will be > > called. > > I have to question the design of this. When I close() an object I > expect it to be closed there and then and not at some indeterminate > later time (well, it is determinate when you're fully aware of all > references, but often you aren't--trust me, I understand reference > counting). Even if you're fully aware of all references, it's indeterminate in multithreaded apps. I've just taken to doing: self.socket._sock.close() self.socket.close() ...in order to send the FIN I wanted ASAP. Robert Brewer fumanchu at aminus.org From thomaspinckney3 at gmail.com Wed May 7 18:11:34 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Wed, 7 May 2008 12:11:34 -0400 Subject: [Python-Dev] urllib unicode handling In-Reply-To: References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <48213B1C.4080205@v.loewis.de> Message-ID: <7BF72683-AA90-44D3-A18F-8422831D799A@gmail.com> Maybe I didn't understand the RFC quite right, but it seemed like how to handle hostnames was left as a choice between IDNA encoding the hostname or replacing the non-ascii characters with dashes? I guess in practice IDNA is the right decision. Another part I wasn't clear on is whether urllib.quote() understands it's working on URIs, arbitrary strings, URLs or what. It seems that from the documentation it looks like it's expecting to just work on the path component of URLs. If this is so, then it doesn't need to understand what to do if the IRI contains a hostname. Seems like the other somewhat under-specified part of all of this is how urllib.unquote() should work. If after percent decoding it sees non-ascii octets, should it try to decode them as utf-8 and if that fails then leave them as is? On May 7, 2008, at 11:55 AM, Robert Brewer wrote: > "Martin v. L?wis" wrote: >> The proper way to implement this would be IRIs (RFC 3987), >> in particular section 3.1. This is not as simple as just >> encoding it as UTF-8, as you might have to apply IDNA to >> the host part. >> >> Code doing so just hasn't been contributed yet. > > But if someone wanted to do so, it's pretty simple: > >>>> u'www.\u212bngstr\xf6m.com'.encode("idna") > 'www.xn--ngstrm-hua5l.com' > > > Robert Brewer > fumanchu at aminus.org > From guido at python.org Wed May 7 18:42:10 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 7 May 2008 09:42:10 -0700 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: <4821BEA0.3060402@acm.org> References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> Message-ID: On Wed, May 7, 2008 at 7:37 AM, Sjoerd Mullender wrote: > On 2008-05-07 13:37, Amaury Forgeot d'Arc wrote: > > > Hello, > > > > 2008/5/7 Sjoerd Mullender : > > > > > Why does sock.close() not actually close sock? > > > > > > If I run the code > > > > > > import socket > > > sock = socket.socket() > > > ... > > > sock.close() > > > > > > I would expect that a system call is done to actually close the socket > and > > > free the file descriptor. But that does not happen. Look at the code > in > > > socket.py. It merely replaces the socket instance with a dummy instance > so > > > that all subsequent calls on the sock object fail, but it does nothing > else! > > > > > > > It does close the socket: > > > > In socket.py, when self._sock is replaced, its __del__ method will be > called. > > This __del__ is implemented in C, in socketmodule.c: > > > > static void > > sock_dealloc(PySocketSockObject *s) > > { > > if (s->sock_fd != -1) > > (void) SOCKETCLOSE(s->sock_fd); > > Py_TYPE(s)->tp_free((PyObject *)s); > > } > > > > > > Of course, if you call sock.dup() or sock.makefile(), > > there is another reference to the underlying _sock, and you must > > close() all these objects. > > > > > > I have to question the design of this. When I close() an object I > expect it to be closed there and then and not at some indeterminate > later time (well, it is determinate when you're fully aware of all > references, but often you aren't--trust me, I understand reference > counting). > > Then there also seems to be a bug in imaplib.IMAP4_SSL since in its > shutdown method it closes the socket but leaves the sslobj untouched. I > assume that that object keeps a reference to the socket. > > But as I said, I expect the close() to actually close. I agree that the design is incredibly fragile. It came about from a desire to remain compatible with the semantics enshrined in the first implementation supporting makefile() -- once you call makefile(), you have multiple references to the socket, and the choice was made to dup() the file descriptor for each reference. This meant that, while .close() on each object would close the file descriptor, the socket connection would not be closed until the last fd was closed, essentially letting the Unix kernel do our reference counting for us. Example: $ python2.2 Python 2.2.3+ (#1, Sep 6 2005, 04:14:07) [GCC 4.0.2 20050808 (prerelease) (Debian 4.0.1-4ubuntu6)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> s = socket.socket() >>> s.connect(('python.org', 80)) >>> f = s.makefile() >>> s.fileno() 3 >>> f.fileno() 4 >>> Unfortunately we couldn't use this architecture on Windows, which doesn't (or at the time didn't) have a dup() system call, and I believe that's where the code that doesn't close the socket but relies on refcounting comes from. The SSL architecture also caused additional problems. In the end a version of this approach was used on all platforms, so that we now have the following: python2.6 Python 2.6a1 (trunk:61189, Mar 2 2008, 17:07:23) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> s = socket.socket() >>> s.connect(('python.org', 80)) >>> f = s.makefile() >>> s.fileno() 3 >>> f.fileno() 3 >>> Note that s and f now share a file descriptor! Python 2.4 behaves the same way. I don't recall if we changed this in 2.3 or in 2.4. All this was because of the requirement that should be allowed to close (or simply delete) the original socket even when you continue to use the stream(s) derived from it using makefile(). This requirement existed out of the desire to be backwards compatible with the original Unix implementation, which had this property. I would be okay with changing the requirements in Py3k so that you are required to keep the socket object open at least as long as you plan on using the derived stream(s), but this will require some careful redesign, especially in the light of SSL support. (Read ssl.py and _ssl.c to understand the level of intertwinement.) In 2.6 I think we should leave the design alone (to the point that it hasn't already changed due to Bill Janssen's providing a new, enhanced SSL implementation) for best compatibility. I don't know anything about the IMAP issue, sorry. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy at alum.mit.edu Wed May 7 19:20:13 2008 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Wed, 7 May 2008 13:20:13 -0400 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <4821CE31.40609@vector-seven.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> Message-ID: On Wed, May 7, 2008 at 11:43 AM, Thomas Lee wrote: > Nick Coghlan wrote: > > > > > As Thomas mentions in a later message, making it possible to annotate > nodes would permit Functions to be annotated as being a generator at the AST > stage (currently it is left to the bytecode compiler's symtable generation > pass to make that determination). > > > > Although I guess an alternative solution to that would be to have separate > AST nodes for Functions and Generators as well... > > > > > I've actually backtracked a little and gone back down the Const path again. > I know this is the third time I've changed my mind, but it's primarily > because annotations tend to get a little clunky (or my implementation was, > at least). Using Const nodes feels a lot more natural inside the optimizer. > I think it's going to stick, at least in the short term. > > Rather than separate FunctionDef and GeneratorDef nodes, I think a new bool > attribute (is_generator?) on FunctionDef should do the job nicely. Further, > I'm thinking we can move the "generator detection code" from the symtable > into Python/ast.c so the generator/function information is available to the > optimizer. Does the optimizer run after the symtable? I'd expect the symbol table information to be essential for almost all analysis, so it should be fine to wait until that pass to mark the generators. Jeremy > This is made a little tricky by the absence of the contextual information > available that is normally available when flagging generators in the > symtable. When generating AST nodes for a suite, we know nothing about the > parent node in which the suite resides. Still, it might be doable. If this > winds up being ugly, we might need to fall back to the original plan of a > separate pass over function bodies to detect yield expressions. > > I'll look into all this tomorrow night, along with any other crazy > suggestions. For now I need to sleep a few hours. :) > > Thanks for the feedback, it's much appreciated. > > Cheers, > T > > From janssen at parc.com Wed May 7 19:29:02 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 7 May 2008 10:29:02 PDT Subject: [Python-Dev] sock.close() not closing? In-Reply-To: References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> Message-ID: <08May7.102903pdt."58696"@synergy1.parc.xerox.com> > I would be okay with changing the requirements in Py3k so that you are > required to keep the socket object open at least as long as you plan > on using the derived stream(s), but this will require some careful > redesign, especially in the light of SSL support. (Read ssl.py and > _ssl.c to understand the level of intertwinement.) In 2.6 I think we > should leave the design alone (to the point that it hasn't already > changed due to Bill Janssen's providing a new, enhanced SSL > implementation) for best compatibility. I'd be perfectly happy to get rid of this. It's not SSL in particular that requires it, if I'm remembering correctly, it's really the HTTP library -- HTTP over SSL was failing when we tried to change this. The HTTP client-side library calls "makefile" on the socket, then closes it, then passes the file returned from "makefile" on to application code to work with. That code merrily reads and writes from the (closed) socket to read responses from the remote Web server. This particular nasty pattern is deeply entwined in all the code that touches the HTTP library in any way, so it will be a big job to get rid of it -- basically re-writing HTTP support and all the services which use it. I didn't want to get into that :-). And I agree with Guido; it doesn't make sense to try to fix this in 2.6. And plan a few months of work to fix it in 3.x; even then, I think you'll break a lot of user code (that won't close the socket when they're finished with it). On the other hand, maybe it's not so bad; maybe you could just (in httplib) read the whole response into a StringIO, and pass that on to the client code. Bill From janssen at parc.com Wed May 7 19:39:47 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 7 May 2008 10:39:47 PDT Subject: [Python-Dev] sock.close() not closing? In-Reply-To: <08May7.102903pdt."58696"@synergy1.parc.xerox.com> References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> <08May7.102903pdt."58696"@synergy1.parc.xerox.com> Message-ID: <08May7.103948pdt."58696"@synergy1.parc.xerox.com> See http://bugs.python.org/issue1348. Bill From amk at amk.ca Wed May 7 20:22:12 2008 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 7 May 2008 14:22:12 -0400 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: <08May7.102903pdt."58696"@synergy1.parc.xerox.com> References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> <08May7.102903pdt."58696"@synergy1.parc.xerox.com> Message-ID: <20080507182212.GA22674@amk-desktop.matrixgroup.net> On Wed, May 07, 2008 at 10:29:02AM -0700, Bill Janssen wrote: > This particular nasty pattern is deeply entwined in all the code that > touches the HTTP library in any way, so it will be a big job to get > rid of it -- basically re-writing HTTP support and all the services > which use it. I didn't want to get into that :-). This would be a good chance for Py3K to dump httplib/urllib/urllib2 and use some more modern library. --amk From facundobatista at gmail.com Wed May 7 20:29:33 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Wed, 7 May 2008 15:29:33 -0300 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: <20080507182212.GA22674@amk-desktop.matrixgroup.net> References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> <20080507182212.GA22674@amk-desktop.matrixgroup.net> Message-ID: 2008/5/7, A.M. Kuchling : > This would be a good chance for Py3K to dump httplib/urllib/urllib2 > and use some more modern library. Which modern library do you propose? -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From amk at amk.ca Wed May 7 20:58:03 2008 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 7 May 2008 14:58:03 -0400 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> <20080507182212.GA22674@amk-desktop.matrixgroup.net> Message-ID: <20080507185803.GA23438@amk-desktop.matrixgroup.net> On Wed, May 07, 2008 at 03:29:33PM -0300, Facundo Batista wrote: > > This would be a good chance for Py3K to dump httplib/urllib/urllib2 > > and use some more modern library. > > Which modern library do you propose? I have no idea -- presumably we'd need to compare a bunch of them (curl, libwget, and whatever else is out there) and decide upon one. --amk From martin at v.loewis.de Wed May 7 21:06:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 07 May 2008 21:06:00 +0200 Subject: [Python-Dev] urllib unicode handling In-Reply-To: <7BF72683-AA90-44D3-A18F-8422831D799A@gmail.com> References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <48213B1C.4080205@v.loewis.de> <7BF72683-AA90-44D3-A18F-8422831D799A@gmail.com> Message-ID: <4821FD98.9030501@v.loewis.de> > Maybe I didn't understand the RFC quite right, but it seemed like how to > handle hostnames was left as a choice between IDNA encoding the hostname > or replacing the non-ascii characters with dashes? I guess in practice > IDNA is the right decision. I haven't fully understood it, either, but I think that's the right conclusion. People want to fetch the resource, then, and encoding the host name in UTF-8 won't do much good. > Seems like the other somewhat under-specified part of all of this is how > urllib.unquote() should work. If after percent decoding it sees > non-ascii octets, should it try to decode them as utf-8 and if that > fails then leave them as is? That's why I think that using IRIs should be a separate feature, perhaps a separate module entirely. Regards, Martin From martin at v.loewis.de Wed May 7 21:10:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 07 May 2008 21:10:00 +0200 Subject: [Python-Dev] urllib unicode handling In-Reply-To: References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <20080507052016.GV1016@nexus.in-nomine.org> Message-ID: <4821FE88.4000300@v.loewis.de> > If this is indeed the case, it sounds perfectly legal (according to the > RFC) and perfectly practical (as required by numerous popular websites) > to have urllib.quote and urllib.quote_plus do an automatic UTF-8 > encoding of unicode strings before percent encoding them. It's probably legal, but I don't understand why you think it's practical. The DNS lookup then will certainly fail, no? Regards, Martin From martin at v.loewis.de Wed May 7 21:34:43 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 07 May 2008 21:34:43 +0200 Subject: [Python-Dev] ctypes documentation In-Reply-To: <08May6.094335pdt."58696"@synergy1.parc.xerox.com> References: <08May6.094335pdt."58696"@synergy1.parc.xerox.com> Message-ID: <48220453.5020305@v.loewis.de> > I just made another attempt to use ctypes to wrap a library, and am > facing the same problem I had the last time: the documentation doesn't > really work. I'm wondering if we have any projects underway to > re-write it? If *we* had one, *we* would know, right ?-) So, no. The author of ctypes is not a native English speaker, so unless you want the details clarified in German, you need to find somebody who actually speaks the language in which you want to read the documentation. Regards, Martin From tnelson at onresolve.com Wed May 7 21:52:05 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 7 May 2008 12:52:05 -0700 Subject: [Python-Dev] REMINDER: Python Sprint Weekend This Weekend! Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E249D3D17@EXMBX04.exchhosting.com> Just a friendly reminder that this weekend is the Python sprint weekend! Look forward to seeing everyone on #python-dev irc.freenode.net over the course of the weekend! Trent. On 16 Apr, 18:52, Trent Nelson wrote: > > Following on from the success of previous sprint/bugfix weekends and > sprinting efforts at PyCon 2008, I'd like to propose the next two > Global Python Sprint Weekends, taking place on the following dates: > > * May 10th-11th (four days after 2.6a3 and 3.0a5 are released) > * June 21st-22nd (~week before 2.6b2 and 3.0b2 are released) > > It seems there are a few of the Python User Groups keen on meeting > up in person and sprinting collaboratively, akin to PyCon, which I > highly recommend. I'd like to nominate Saturday across the board > as the day for PUGs to meet up in person, with Sunday geared more > towards an online collaboration day via IRC, where we can take care > of all the little things that got in our way of coding on Saturday > (like finalising/preparing/reviewing patches, updating tracker and > documentation, writing tests ;-). > > For User Groups that are planning on meeting up to collaborate, > please reply to this thread on python-dev at python.org and let every- > one know your intentions! > > As is commonly the case, #python-dev on irc.freenode.net will be > the place to be over the course of each sprint weekend; a large > proportion of Python developers with commit access will be present, > increasing the amount of eyes available to review and apply patches. > > For those that have an idea on areas they'd like to sprint on and > want to look for other developers to rope in (or just to communicate > plans in advance), please also feel free to jump on this thread via > python-dev@ and indicate your intentions. > > For those that haven't the foggiest on what to work on, but would > like to contribute, the bugs tracker at http://bugs.python.org is > the best place to start. Register an account and start searching > for issues that you'd be able to lend a hand with. > > All contributors that submit code patches or documentation updates > will typically get listed in Misc/ACKS.txt; come September when the > final release of 2.6 and 3.0 come about, you'll be able to point at > the tarball or .msi and exclaim loudly ``I helped build that!'', > and actually back it up with hard evidence ;-) > > Bring on the pizza and Red Bull! > > Trent. From thomaspinckney3 at gmail.com Wed May 7 22:04:29 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Wed, 7 May 2008 16:04:29 -0400 Subject: [Python-Dev] urllib unicode handling In-Reply-To: <4821FE88.4000300@v.loewis.de> References: <101755A8-570C-449A-926F-A2827F563182@gmail.com> <20080507052016.GV1016@nexus.in-nomine.org> <4821FE88.4000300@v.loewis.de> Message-ID: <7CAF33E8-5C21-4F21-A9A0-E1E1C032D463@gmail.com> I was assuming urllib.quote/unquote would only be called on text intended to be used in non-hostname portions of the URIs. I'm not sure if this is the actual intent of urllib.quote and perhaps the documentation should be updated to specify what precisely it does and then peopel can decide what parts of URIs it is appropriate to quote/ unquote. I don't believe quote/unquote does anything sensical with hostnames today that contain non-printable ascii, so this is no loss of existing functionality. Re your suggestion that IRIs should be a separate module: I guess my thought is that urllib out of the box should just work with the way websites on the web today actually work. Thus, we should make urllib do the utf-8 encode / decode rather than make users switch to a different module for certain URLs and another library for other URLs. Re the specific issue of how urllib.unquote should work: Perhaps there could be an optional second argument that specified a content encoding to use when decoding escaped characters? I would propose that this parameter have a default value of utf-8 since that is what most websites seem to do, but if the author knew that the website they were using encoded URLs in iso-8559 then they could unquote using that scheme. On May 7, 2008, at 3:10 PM, Martin v. L?wis wrote: >> If this is indeed the case, it sounds perfectly legal (according to >> the >> RFC) and perfectly practical (as required by numerous popular >> websites) >> to have urllib.quote and urllib.quote_plus do an automatic UTF-8 >> encoding of unicode strings before percent encoding them. > > It's probably legal, but I don't understand why you think it's > practical. The DNS lookup then will certainly fail, no? > > Regards, > Martin From barry at python.org Thu May 8 01:24:32 2008 From: barry at python.org (Barry Warsaw) Date: Wed, 7 May 2008 19:24:32 -0400 Subject: [Python-Dev] Releasing alphas tonight Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, Just a reminder that I'm going to be cutting the releases tonight. Because of work, I didn't make the 6pm EDT goal, and now I have to run out for a few hours. I will send another message when I'm ready to start spinning the release, but figure it will be at about 10pm EDT. Please limit your commits between now and then to only things you absolutely know will improve stability and test passing. Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCI6MXEjvBPtnXfVAQIfgwP+P7XOTMwWex5+YwiOza0fEeUj5n8OJuxU ISK3p3Tas4tPM65eMCHk5vmIFOBfJDFyWBpNhGr+uKmaWMgiqtPX5fs6nMmkbkrY dWrfG5Mgth9U1hpR4/1y/p2W82DJX9exmnjYL2BxjZ/TGeZdbcpUcs6Cc/fpHKR/ wTQ3dagAPNA= =bDtn -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Thu May 8 02:15:52 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 08 May 2008 12:15:52 +1200 Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <48214C48.2040005@v.loewis.de> Message-ID: <48224638.7040803@canterbury.ac.nz> Antoine Pitrou wrote: > When you write a processing > script for internal use there's no reason to worry about security issues like > that Even without security issues, there's still the potential for two processes creating temp files at the same time to trip over each other. -- Greg From ggpolo at gmail.com Thu May 8 02:38:54 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Wed, 7 May 2008 21:38:54 -0300 Subject: [Python-Dev] GSoC Student Introduction, again Message-ID: Hi, I have been accepted as a student for GSoC and I'm already working in my project, "Bringing Ttk to Tkinter". This is my second GSoC, first time in PSF, and it has been awesome to have Fredrik Lundh as mentor, we have talked a lot and he is a very nice person. Regarding the project, I expect it to be integrated into Python's stdlib sometime in the future, I believe it will be in very good shape before binaries compiled against tcl/tk 8.5 start showing up. I've put a site that contains all the information available, which may be of interest, at http://gpolo.ath.cx:81/projects/ttk_to_tkinter/ and will be updating it as necessary. There is also a repo you may take a look: http://svn.python.org/view/sandbox/trunk/ttk-gsoc/ Thank you all, -- -- Guilherme H. Polo Goncalves From lists at cheimes.de Thu May 8 02:51:43 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 08 May 2008 02:51:43 +0200 Subject: [Python-Dev] Releasing alphas tonight In-Reply-To: References: Message-ID: <48224E9F.40407@cheimes.de> Barry Warsaw schrieb: > Hi all, > > Just a reminder that I'm going to be cutting the releases tonight. > Because of work, I didn't make the 6pm EDT goal, and now I have to run > out for a few hours. I will send another message when I'm ready to > start spinning the release, but figure it will be at about 10pm EDT. > Please limit your commits between now and then to only things you > absolutely know will improve stability and test passing. The py3k branch has a major show stopper, It's leaking references to the max. ... test_builtin leaked [14, 14, 14, 14] references, sum=56 test_exceptions beginning 9 repetitions 123456789 ......... test_exceptions leaked [40, 40, 40, 40] references, sum=160 test_types beginning 9 repetitions 123456789 ......... test_types leaked [2, 2, 2, 2] references, sum=8 test_unittest beginning 9 repetitions 123456789 ......... test_unittest leaked [23, 23, 23, 23] references, sum=92 ... I'm trying to find the issue. Christian From greg.ewing at canterbury.ac.nz Thu May 8 02:56:39 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 08 May 2008 12:56:39 +1200 Subject: [Python-Dev] sock.close() not closing? In-Reply-To: <08May7.102903pdt.58696@synergy1.parc.xerox.com> References: <4821894E.3000304@acm.org> <4821BEA0.3060402@acm.org> <08May7.102903pdt.58696@synergy1.parc.xerox.com> Message-ID: <48224FC7.5000802@canterbury.ac.nz> Bill Janssen wrote: > The HTTP client-side library calls "makefile" on the socket, then > closes it, then passes the file returned from "makefile" on to > application code to work with. Seems to me we really need two different APIs for doing a makefile()-like operation, depending on whether you're expecting to get a copy of the file descriptor or not. -- Greg From greg.ewing at canterbury.ac.nz Thu May 8 01:19:19 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 08 May 2008 11:19:19 +1200 Subject: [Python-Dev] [Python-3000] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <4820F323.3040903@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DC6A3.70104@gmail.com> <481E15B7.9060003@cheimes.de> <4820EBB8.7070108@canterbury.ac.nz> <4820F323.3040903@cheimes.de> Message-ID: <482238F7.2080303@canterbury.ac.nz> Christian Heimes wrote: > Can you come up with a path that fits the purpose of a base directory? "~/Library/Application Support/Python" would seem to be appropriate. -- Greg From greg.ewing at canterbury.ac.nz Thu May 8 01:24:02 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 08 May 2008 11:24:02 +1200 Subject: [Python-Dev] [Python-3000] PEP 370 (was Re: Reminder: last alphas next Wednesday 07-May-2008) In-Reply-To: <4820F323.3040903@cheimes.de> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <2B80BDE9-0CF4-4741-BF33-A6EC3BB0462A@python.org> <18459.43976.85481.758104@montanaro-dyndns-org.local> <481C2AE7.9010805@gmail.com> <18460.20940.882777.235301@montanaro-dyndns-org.local> <45D12407-959A-43C5-B9E4-3E1D9F2B6B29@acm.org> <20080504135803.25821.2140838939.divmod.xquotient.7545@joule.divmod.com> <481DC6A3.70104@gmail.com> <481E15B7.9060003@cheimes.de> <4820EBB8.7070108@canterbury.ac.nz> <4820F323.3040903@cheimes.de> Message-ID: <48223A12.5070708@canterbury.ac.nz> Christian Heimes wrote: > Can you come up with a path that fits the purpose of a base directory? Second thoughts, I think there's something better -- there's already a system-supplied directory on my system called /Library/Python/2.3 with a site-packages containing a README that says it's for installing 3rd party packages. So the obvious user-level analogy would be "~/Library/Python/x.y". -- Greg From alexandre at peadrop.com Thu May 8 03:33:00 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Wed, 7 May 2008 21:33:00 -0400 Subject: [Python-Dev] r62778 - in python/branches/py3k: Lib/io.py Lib/test/test_StringIO.py Lib/test/test_io.py Lib/test/test_largefile.py Lib/test/test_memoryio.py Lib/test/test_mimetools.py Modules/_bytesio.c Modules/_fileio.c Python/pythonrun.c setup.py In-Reply-To: <4820E119.6010209@cheimes.de> References: <20080506194839.C70461E400A@bag.python.org> <4820E119.6010209@cheimes.de> Message-ID: On Tue, May 6, 2008 at 6:52 PM, Christian Heimes wrote: > alexandre.vassalotti schrieb: > > > Author: alexandre.vassalotti > > Date: Tue May 6 21:48:38 2008 > > New Revision: 62778 > > > > Log: > > Added fast alternate io.BytesIO implementation and its test suite. > > Removed old test suite for StringIO. > > Modified truncate() to imply a seek to given argument value. > > Thanks for your great work! But what about the trunk? :] Can you port > your code to the trunk before the alpha gets out? > I have a backported version of my code for the trunk. Should I commit it or should I post it to issue tracker and wait for proper review? -- Alexandre From ocean at m2.ccsnet.ne.jp Thu May 8 03:41:16 2008 From: ocean at m2.ccsnet.ne.jp (Hirokazu Yamamoto) Date: Thu, 8 May 2008 10:41:16 +0900 Subject: [Python-Dev] Releasing alphas tonight References: <48224E9F.40407@cheimes.de> Message-ID: <001801c8b0ac$9b902dc0$0200a8c0@whiterabc2znlh> Hello. > The py3k branch has a major show stopper, It's leaking references to the > max. Is there any chance this leak also will be fixed? http://bugs.python.org/issue2222 Thank you. From skip at pobox.com Thu May 8 03:47:44 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 7 May 2008 20:47:44 -0500 Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: <48224638.7040803@canterbury.ac.nz> References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <48214C48.2040005@v.loewis.de> <48224638.7040803@canterbury.ac.nz> Message-ID: <18466.23488.560586.183048@montanaro-dyndns-org.local> This has been a fairly interesting discussion about tempfile.mktemp, but can we return to the original question? Should we begin warning about its use again? Should we ever warn about it? It appears that NamedTemporaryFile gives you a reasonable (secure) replacement. I'm happy to document mktemp->NamedTemporaryFile with an example in tempfile.rst if that's what it will take. Before knowing about the delete=False parameter I proposed a mkstempf() function, whose tracker issue I will close if that hasn't already happened. Skip From lists at cheimes.de Thu May 8 03:25:01 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 08 May 2008 03:25:01 +0200 Subject: [Python-Dev] Releasing alphas tonight In-Reply-To: <48224E9F.40407@cheimes.de> References: <48224E9F.40407@cheimes.de> Message-ID: <4822566D.7080207@cheimes.de> Christian Heimes schrieb: > The py3k branch has a major show stopper, It's leaking references to the > max. Fixed ;) Christian From barry at python.org Thu May 8 06:36:00 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 00:36:00 -0400 Subject: [Python-Dev] Releasing alphas tonight In-Reply-To: <4822566D.7080207@cheimes.de> References: <48224E9F.40407@cheimes.de> <4822566D.7080207@cheimes.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 7, 2008, at 9:25 PM, Christian Heimes wrote: > Christian Heimes schrieb: >> The py3k branch has a major show stopper, It's leaking references >> to the >> max. > > Fixed ;) Thanks! Folks, I apologize. I had some system problems tonight so I fell behind on the release. I just applied Antoine's patch for bug 2507 and I'd like to make sure the buildbots complete. Other than that, the only other release critical is one for the release process. I'll complete the releases tomorrow morning (EDT) so in the meantime, please refrain from committing anything. Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCKDMXEjvBPtnXfVAQL1CgP9Heg1XlNjpM3wgT4N8PK090HnaGIJ6MzH Fs3QtngvLB/YPf31VrkYILIIMG/YBs+yqCZFziuSR2alNYNBcvwNVfpIljMuq9AM qtj+cu2vbhkoh+gR8LjM1J8ZWAKhI5G6eAxHuGlTWykdumcSllkB6xW4uLQ2RolZ eOS5Avnc/Qs= =kJdD -----END PGP SIGNATURE----- From barry at python.org Thu May 8 06:36:22 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 00:36:22 -0400 Subject: [Python-Dev] Releasing alphas tonight In-Reply-To: <001801c8b0ac$9b902dc0$0200a8c0@whiterabc2znlh> References: <48224E9F.40407@cheimes.de> <001801c8b0ac$9b902dc0$0200a8c0@whiterabc2znlh> Message-ID: <6AB977B4-8300-4A07-B7CF-F2403C5D4156@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 7, 2008, at 9:41 PM, Hirokazu Yamamoto wrote: > Hello. > >> The py3k branch has a major show stopper, It's leaking references >> to the >> max. > > Is there any chance this leak also will be fixed? > http://bugs.python.org/issue2222 Not for the alphas, sorry. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCKDRnEjvBPtnXfVAQKjGAP8CFKRDMBYJC8dm+tR/nHucrRa/Nqfy977 I8rx/B5QWN+feBk6LhODaEQ2NPOQaF+iSTaDnOlF9f2+Z6m85b94zsLJPY9EoiAC qdNmYBmZWYtuzvLmCh5Ef2aCjtfbn4Ik8i3SR9amQJBhuq7ubbdYVUsbcy6HCUUV K2Xp8LV1HWM= =aYB/ -----END PGP SIGNATURE----- From martin at v.loewis.de Thu May 8 06:52:49 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 08 May 2008 06:52:49 +0200 Subject: [Python-Dev] GSoC Student Introduction, again In-Reply-To: References: Message-ID: <48228721.1030600@v.loewis.de> > Regarding the project, I expect it to be integrated into Python's > stdlib sometime in the future, I believe it will be in very good shape > before binaries compiled against tcl/tk 8.5 start showing up. Actually, I would like to release Python 2.6 and 3.0 on Windows with Tk 8.5 included, preferably starting at the next betas. Would you see a problem with that? Regards, Martin From ggpolo at gmail.com Thu May 8 13:27:13 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Thu, 8 May 2008 08:27:13 -0300 Subject: [Python-Dev] GSoC Student Introduction, again In-Reply-To: <48228721.1030600@v.loewis.de> References: <48228721.1030600@v.loewis.de> Message-ID: 2008/5/8 "Martin v. L?wis" : >> Regarding the project, I expect it to be integrated into Python's >> stdlib sometime in the future, I believe it will be in very good shape >> before binaries compiled against tcl/tk 8.5 start showing up. > > Actually, I would like to release Python 2.6 and 3.0 on Windows with Tk > 8.5 included, preferably starting at the next betas. Would you see a > problem with that? > I've already wrapped all the Ttk functionality by now, and I will complete the documentation till the first betas. But as you know there isn't much people using (I know myself and a Tcl guy) so I'm not sure if it would be acceptable to include this module right into these betas, or would it be ? Should I start doing something more/else ? > Regards, > Martin > > Thanks, -- -- Guilherme H. Polo Goncalves From barry at python.org Thu May 8 13:32:42 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 07:32:42 -0400 Subject: [Python-Dev] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <20080508043520.B60821E400E@bag.python.org> References: <20080508043520.B60821E400E@bag.python.org> Message-ID: <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 8, 2008, at 12:35 AM, raymond.hettinger wrote: > Author: raymond.hettinger > Date: Thu May 8 06:35:20 2008 > New Revision: 62848 > > Log: > Frozensets do not benefit from autoconversion. Since the trunk buildbots appear to be mostly happy (well those that are connected anyway), and because I couldn't get the releases out last night, I'll let this one slide. I'd like to find a way to more forcefully enforce commit freezes for the betas though. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCLk2nEjvBPtnXfVAQJxiwP/VPTmeKVLoKkc/xIF0tc/lb6pT7kZ0swL b1M2TUkl/+xOuKf3J2EIkHOiKdNNmivl80nG/wP9/VTa7lVJGnWgIeLi0yC20Q9n wvtHaXCrHDc4/ibiShjwYqD4YR0BGwJI7BrlyCYzohbjFK6QYsxd+5a96Cipb/cB +K/Akjqry4Q= =xQfb -----END PGP SIGNATURE----- From musiccomposition at gmail.com Thu May 8 13:54:35 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 8 May 2008 06:54:35 -0500 Subject: [Python-Dev] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> Message-ID: <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> On Thu, May 8, 2008 at 6:32 AM, Barry Warsaw wrote: > Since the trunk buildbots appear to be mostly happy (well those that are > connected anyway), and because I couldn't get the releases out last night, > I'll let this one slide. I'd like to find a way to more forcefully enforce > commit freezes for the betas though. I wonder if you couldn't alter the server side commit hook to reject everything with the message "Sorry, we're in a freeze." (You'd have to make an exception for yourself.) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From barry at python.org Thu May 8 13:59:50 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 07:59:50 -0400 Subject: [Python-Dev] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> Message-ID: <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 8, 2008, at 7:54 AM, Benjamin Peterson wrote: > On Thu, May 8, 2008 at 6:32 AM, Barry Warsaw wrote: >> Since the trunk buildbots appear to be mostly happy (well those >> that are >> connected anyway), and because I couldn't get the releases out last >> night, >> I'll let this one slide. I'd like to find a way to more forcefully >> enforce >> commit freezes for the betas though. > > I wonder if you couldn't alter the server side commit hook to reject > everything with the message "Sorry, we're in a freeze." (You'd have to > make an exception for yourself.) This is exactly what I'm thinking about! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCLrNnEjvBPtnXfVAQITDwP/WGqlRHSfvE668clPM3gshhYbAapZcF+e mNKGwu407/q03LYRqHr2QY0gBxsySJBWl5OsozmJUOTc7NEY/E/MtiauauzCJiyO 24sJ2V52aROwYBLG+4tLFcaGmWmnsWPg79Qj/yJQKMMiH5OznPfagLECOjlwDZZA ianWqOZxeYc= =xyD7 -----END PGP SIGNATURE----- From ncoghlan at gmail.com Thu May 8 14:00:35 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 08 May 2008 22:00:35 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> Message-ID: <4822EB63.1040608@gmail.com> Jeremy Hylton wrote: > On Wed, May 7, 2008 at 11:43 AM, Thomas Lee wrote: >> Nick Coghlan wrote: >> >>> As Thomas mentions in a later message, making it possible to annotate >> nodes would permit Functions to be annotated as being a generator at the AST >> stage (currently it is left to the bytecode compiler's symtable generation >> pass to make that determination). >>> Although I guess an alternative solution to that would be to have separate >> AST nodes for Functions and Generators as well... >>> >> I've actually backtracked a little and gone back down the Const path again. >> I know this is the third time I've changed my mind, but it's primarily >> because annotations tend to get a little clunky (or my implementation was, >> at least). Using Const nodes feels a lot more natural inside the optimizer. >> I think it's going to stick, at least in the short term. >> >> Rather than separate FunctionDef and GeneratorDef nodes, I think a new bool >> attribute (is_generator?) on FunctionDef should do the job nicely. Further, >> I'm thinking we can move the "generator detection code" from the symtable >> into Python/ast.c so the generator/function information is available to the >> optimizer. > > Does the optimizer run after the symtable? I'd expect the symbol > table information to be essential for almost all analysis, so it > should be fine to wait until that pass to mark the generators. There are a lot of micro-optimisations that are actually context independent, so moving them before the symtable pass should be quite feasible - e.g. replacing "return None" with "return", stripping dead code after a return statement, changing a "if not" statement into an "if" statement with the two suites reversed, changing "(1, 2, 3)" into a stored constant, folding "1 + 2" into the constant "3". I believe the goal is to see how many of the current bytecode optimisations can actually be brought forward to the AST generation stage, rather than waiting until after the bytecode symtable calculation and compilation passes. The current structure goes: tokenisation->AST construction->symtable construction->bytecode compilation->bytecode optimisation My understanding of what Thomas is trying to do is to make it look more like this: tokenisation->AST construction->AST optimisation->symtable construction->bytecode compilation There may still be a direct bytecode optimisation step left at the end if it turns out something relies on information that isn't worked out until the symtable construction pass (and that can't easily be worked out earlier, such as adding an 'is_generator' flag to the FunctionDef AST node). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From lists at cheimes.de Thu May 8 14:21:54 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 08 May 2008 14:21:54 +0200 Subject: [Python-Dev] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> Message-ID: <4822F062.7090305@cheimes.de> Barry Warsaw schrieb: > This is exactly what I'm thinking about! -1 A technical solution never solves a social problem. It's just going to cause more social and technical problems. All community members with svn write privileges must subscribe to the Python developer list. Committers must check the lists prior to a check in if a release is immanent. Releases are announced at least four days prior to svn freeze so it's not going to be a problem. The problem often lies with occasional committers and maintainers of stdlib packages. People need to show more discipline or eventually we have to (temporarily) revoke their privileges. Christian From jeremy at alum.mit.edu Thu May 8 15:18:33 2008 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Thu, 8 May 2008 09:18:33 -0400 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <4822EB63.1040608@gmail.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> Message-ID: On Thu, May 8, 2008 at 8:00 AM, Nick Coghlan wrote: > Jeremy Hylton wrote: > > > On Wed, May 7, 2008 at 11:43 AM, Thomas Lee wrote: > > > > > Nick Coghlan wrote: > > > > > > > > > > As Thomas mentions in a later message, making it possible to annotate > > > > > > > nodes would permit Functions to be annotated as being a generator at the > AST > > > stage (currently it is left to the bytecode compiler's symtable > generation > > > pass to make that determination). > > > > > > > Although I guess an alternative solution to that would be to have > separate > > > > > > > AST nodes for Functions and Generators as well... > > > > > > > > > > > > > > I've actually backtracked a little and gone back down the Const path > again. > > > I know this is the third time I've changed my mind, but it's primarily > > > because annotations tend to get a little clunky (or my implementation > was, > > > at least). Using Const nodes feels a lot more natural inside the > optimizer. > > > I think it's going to stick, at least in the short term. > > > > > > Rather than separate FunctionDef and GeneratorDef nodes, I think a new > bool > > > attribute (is_generator?) on FunctionDef should do the job nicely. > Further, > > > I'm thinking we can move the "generator detection code" from the > symtable > > > into Python/ast.c so the generator/function information is available to > the > > > optimizer. > > > > > > > Does the optimizer run after the symtable? I'd expect the symbol > > table information to be essential for almost all analysis, so it > > should be fine to wait until that pass to mark the generators. > > > > There are a lot of micro-optimisations that are actually context > independent, so moving them before the symtable pass should be quite > feasible - e.g. replacing "return None" with "return", stripping dead code > after a return statement, changing a "if not" statement into an "if" > statement with the two suites reversed, changing "(1, 2, 3)" into a stored > constant, folding "1 + 2" into the constant "3". > > I believe the goal is to see how many of the current bytecode optimisations > can actually be brought forward to the AST generation stage, rather than > waiting until after the bytecode symtable calculation and compilation > passes. > > The current structure goes: > > tokenisation->AST construction->symtable construction->bytecode > compilation->bytecode optimisation > > My understanding of what Thomas is trying to do is to make it look more > like this: > > tokenisation->AST construction->AST optimisation->symtable > construction->bytecode compilation > > There may still be a direct bytecode optimisation step left at the end if > it turns out something relies on information that isn't worked out until the > symtable construction pass (and that can't easily be worked out earlier, > such as adding an 'is_generator' flag to the FunctionDef AST node). The symbol table pass is not important for deciding where to place the optimizations. Either the optimization operates on the AST itself, in which case it needs to run before bytecode compilation or it operators on the lowered bytecode representation, in which case it needs to run after bytecode compilation. Since the symbol table passes doesn't change the representation, you ought to be able to put the optimizations before or after it. But I haven't looked closely at the actual code, so maybe I'm missing something. Jeremy > > > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > From barry at python.org Thu May 8 15:20:42 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 09:20:42 -0400 Subject: [Python-Dev] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <4822F062.7090305@cheimes.de> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> Message-ID: <617FFC34-8C94-48ED-951D-59F3467E6DC7@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 8, 2008, at 8:21 AM, Christian Heimes wrote: > Barry Warsaw schrieb: >> This is exactly what I'm thinking about! > > -1 > > A technical solution never solves a social problem. It's just going to > cause more social and technical problems. In this case I disagree. Given our global nature and the vast amounts of email we all get, I think a friendly little svn commit hook reminder is a simple and workable solution. This commit lock really doesn't need to be in place for very long. Optimistically, I only need it long enough to create the tags, which /normally/ should take me 10 minutes. > All community members with svn write privileges must subscribe to the > Python developer list. Committers must check the lists prior to a > check > in if a release is immanent. Releases are announced at least four days > prior to svn freeze so it's not going to be a problem. The problem > often > lies with occasional committers and maintainers of stdlib packages. > People need to show more discipline or eventually we have to > (temporarily) revoke their privileges. Or aggressively back out any changes from freeze time to tag time. If we don't add the commit hook lock, I will be very strict about this come the betas. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCL+KnEjvBPtnXfVAQIkxgQAqXXwZjHyI93L1xEvrIPYGkTugxlgEva/ bj9ip59XqB6EYS8NnciJU29WZhcc3WnEoOsdWk7qwYV0qOc2YOgYh775GF4Q2S/A 5qVw+oePFIGCWMhezVG/JYph8V6T0QL36hhgd78WqBJKa2C7IpKEjh3HATwY8DQL nouyqdmIDJo= =Vohh -----END PGP SIGNATURE----- From ncoghlan at gmail.com Thu May 8 15:23:11 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 08 May 2008 23:23:11 +1000 Subject: [Python-Dev] [Python-3000] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <4822F062.7090305@cheimes.de> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> Message-ID: <4822FEBF.9060800@gmail.com> Christian Heimes wrote: > Barry Warsaw schrieb: >> This is exactly what I'm thinking about! > > -1 > > A technical solution never solves a social problem. It's just going to > cause more social and technical problems. > > All community members with svn write privileges must subscribe to the > Python developer list. Committers must check the lists prior to a check > in if a release is immanent. Releases are announced at least four days > prior to svn freeze so it's not going to be a problem. The problem often > lies with occasional committers and maintainers of stdlib packages. > People need to show more discipline or eventually we have to > (temporarily) revoke their privileges. It's actually the time zone issues that get me in relation to code freezes... so I just try to avoid committing anything for a day or two :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From murman at gmail.com Thu May 8 15:41:20 2008 From: murman at gmail.com (Michael Urman) Date: Thu, 8 May 2008 08:41:20 -0500 Subject: [Python-Dev] [Python-3000] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <617FFC34-8C94-48ED-951D-59F3467E6DC7@python.org> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> <617FFC34-8C94-48ED-951D-59F3467E6DC7@python.org> Message-ID: On Thu, May 8, 2008 at 8:20 AM, Barry Warsaw wrote: > Or aggressively back out any changes from freeze time to tag time. If we > don't add the commit hook lock, I will be very strict about this come the > betas. I know this way is fairly entrenched in the python release process, but it sounds like it's using the tools incorrectly. In particular with subversion is very easy (compared to cvs) to branch and to switch branches locally. Why not create a new prerelease branch at the beginning of freeze and only merge in the critical changes? This way only the release manager need know or care about the branch, and nobody else has to really modify his behavior. Then tag, move, and/or delete the branch as desired. The obvious stumbling blocks include buildbots not following the new branch (this could be a blocker), and release scripts possibly needing modifications if they contain direct svn url references. -- Michael Urman From barry at python.org Thu May 8 15:51:29 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 09:51:29 -0400 Subject: [Python-Dev] Freeze lifted Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've created the tags for 3.0a5 and 2.6a3, and the tarballs look good, so I'm lifting the commit freeze for these two branches. Thanks everyone, and look for the release announcements in a little while. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCMFYXEjvBPtnXfVAQKwyAP/bVnUtGzHMaJcwdc6BZR+kZJ0M22k/Vbp Nk1IfPts3HPKC7cNWzEkpWlqeXnGC0piuqDGrv2igY2Ori7LVMaTOea1xj8L1KqA QxiSHT0qtkW9J/io/q3Vw4cdXjshUQahSVPL2upafmCF1ROGDM0IKODq6kzjxgGV I8XI4BciN20= =NIY+ -----END PGP SIGNATURE----- From steve at pearwood.info Thu May 8 16:14:55 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 9 May 2008 00:14:55 +1000 Subject: [Python-Dev] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <4822F062.7090305@cheimes.de> References: <20080508043520.B60821E400E@bag.python.org> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> Message-ID: <200805090014.55459.steve@pearwood.info> On Thu, 8 May 2008 10:21:54 pm Christian Heimes wrote: > Barry Warsaw schrieb: > > This is exactly what I'm thinking about! > > -1 > > A technical solution never solves a social problem. It's just going > to cause more social and technical problems. That's certainly not true. The social problem of people throwing their "night soil" out into the streets had a technical solution, namely sewerage systems. Or if you prefer a more computer-related example, the social problem of having people on a shared computer have access to files you don't want them to have access to is solved by the technical solution of file permissions and ACLs. I have no opinion on this specific problem, but it is equally fallacious to say that social problems_never_ have technical solutions as it is to say that they _always_ have technical solutions. -- Steven D'Aprano From amk at amk.ca Thu May 8 16:23:56 2008 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 8 May 2008 10:23:56 -0400 Subject: [Python-Dev] [Python-checkins] [Python-3000] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <4822FEBF.9060800@gmail.com> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> <4822FEBF.9060800@gmail.com> Message-ID: <20080508142356.GA8525@amk-desktop.matrixgroup.net> On Thu, May 08, 2008 at 11:23:11PM +1000, Nick Coghlan wrote: > It's actually the time zone issues that get me in relation to code > freezes... so I just try to avoid committing anything for a day or two :) Subscribers to the python-dev digests may also not see a posting immediately, waiting until the next digest is sent. --amk From barry at python.org Thu May 8 16:24:16 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 10:24:16 -0400 Subject: [Python-Dev] [Python-3000] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> <617FFC34-8C94-48ED-951D-59F3467E6DC7@python.org> Message-ID: <694A97D7-5CB1-4DC4-B17F-2B157DE89CF4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 8, 2008, at 9:41 AM, Michael Urman wrote: > On Thu, May 8, 2008 at 8:20 AM, Barry Warsaw wrote: >> Or aggressively back out any changes from freeze time to tag time. >> If we >> don't add the commit hook lock, I will be very strict about this >> come the >> betas. > > I know this way is fairly entrenched in the python release process, > but it sounds like it's using the tools incorrectly. In particular > with subversion is very easy (compared to cvs) to branch and to switch > branches locally. Why not create a new prerelease branch at the > beginning of freeze and only merge in the critical changes? This way > only the release manager need know or care about the branch, and > nobody else has to really modify his behavior. Then tag, move, and/or > delete the branch as desired. > > The obvious stumbling blocks include buildbots not following the new > branch (this could be a blocker), and release scripts possibly needing > modifications if they contain direct svn url references. I definitely think we'd want the buildbots to track the release branches, and it's a bit of a pain to get the release scripts to deal with the svn switches. Right now I think the freeze window is pretty short (barring unforeseen networking snafus) that it's not worth it. However, once the release process is smooth enough, maybe this little freeze hiccup will be worth eliminating. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCMNEHEjvBPtnXfVAQIDogP+NVpyE7AhUS1Eerqv/N+ERTuKnmy/rSNQ wQhOlAxlvx/lPgm0Mi70C9cA60ogxwGE+nJPf0RQxN2bVfhE/+fvElRl9x7xuoo3 wAK6/zzItqMCP4bpaT8sbsqn4tPB4OCKr0eM/SgZMxrHZkHHZwLTVAw81h40Fmr3 A30V6JpZpdU= =q3uu -----END PGP SIGNATURE----- From aahz at pythoncraft.com Thu May 8 19:36:01 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 8 May 2008 10:36:01 -0700 Subject: [Python-Dev] [Python-checkins] [Python-3000] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <20080508142356.GA8525@amk-desktop.matrixgroup.net> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> <4822FEBF.9060800@gmail.com> <20080508142356.GA8525@amk-desktop.matrixgroup.net> Message-ID: <20080508173601.GB5547@panix.com> On Thu, May 08, 2008, A.M. Kuchling wrote: > On Thu, May 08, 2008 at 11:23:11PM +1000, Nick Coghlan wrote: >> >> It's actually the time zone issues that get me in relation to code >> freezes... so I just try to avoid committing anything for a day or two :) > > Subscribers to the python-dev digests may also not see a posting > immediately, waiting until the next digest is sent. Heck, I'm not on the digest and I still have been seeing posts after a week (though that's the fault of a balky mailserver that should be fixed). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Help a hearing-impaired person: http://rule6.info/hearing.html From lists at cheimes.de Fri May 9 00:05:19 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 09 May 2008 00:05:19 +0200 Subject: [Python-Dev] PEP 370 extras Message-ID: <4823791F.2050507@cheimes.de> I'm working on some extra cool and useful additions to PEP 370. It's going to make compilation and usage of custom libraries very easy. Sit back and watch the slide show. :) The site module has two new options. The --user-base option prints the path to the user base directory to stdout. $ ~/dev/python/trunk/python -m site --user-base /home/heimes/.local I install libxslt-1.1.19 into my ~/.local directory. libxslt-1.1.19$ ./configure --prefix `~/dev/python/trunk/python -m site --user-base` libxslt-1.1.19$ make libxslt-1.1.19$ make install ... test -z "/home/heimes/.local/lib" || mkdir -p -- "/home/heimes/.local/lib" /bin/bash ../libtool --mode=install /usr/bin/install -c 'libxslt.la' '/home/heimes/.local/lib/libxslt.la' /usr/bin/install -c .libs/libxslt.so.1.1.19 /home/heimes/.local/lib/libxslt.so.1.1.19 Now I compile lxml against the libraries and include directories in ~/.local. But first I have to add ~/.local/bin to PATH so lxml picks up the right xslt-config binary. $ export PATH=~/.local/bin:$PATH lxml-1.3.6$ ~/dev/python/trunk/python setup.py build_ext --user Building lxml version 1.3.6-8122 setupinfo.py:137: DeprecationWarning: os.popen3 is deprecated. Use the subprocess module. wf, rf, ef = os.popen3(cmd) /home/heimes/dev/python/trunk/Lib/distutils/dist.py:263: UserWarning: Unknown distribution option: 'zip_safe' warnings.warn(msg) running build_ext building 'lxml.etree' extension creating build creating build/temp.linux-i686-2.6-pydebug creating build/temp.linux-i686-2.6-pydebug/src creating build/temp.linux-i686-2.6-pydebug/src/lxml gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -fPIC -I/home/heimes/.local/include -I/usr/include/libxml2 -I/home/heimes/dev/python/trunk/Include -I/home/heimes/dev/python/trunk -I/home/heimes/.local/include -c src/lxml/etree.c -o build/temp.linux-i686-2.6-pydebug/src/lxml/etree.o -w creating build/lib.linux-i686-2.6-pydebug creating build/lib.linux-i686-2.6-pydebug/lxml gcc -pthread -shared build/temp.linux-i686-2.6-pydebug/src/lxml/etree.o -L/home/heimes/.local/lib -L/home/heimes/.local/lib -Wl,-R/home/heimes/.local/lib -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-i686-2.6-pydebug/lxml/etree.so building 'lxml.objectify' extension gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -fPIC -I/home/heimes/.local/include -I/usr/include/libxml2 -I/home/heimes/dev/python/trunk/Include -I/home/heimes/dev/python/trunk -I/home/heimes/.local/include -c src/lxml/objectify.c -o build/temp.linux-i686-2.6-pydebug/src/lxml/objectify.o -w gcc -pthread -shared build/temp.linux-i686-2.6-pydebug/src/lxml/objectify.o -L/home/heimes/.local/lib -L/home/heimes/.local/lib -Wl,-R/home/heimes/.local/lib -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-i686-2.6-pydebug/lxml/objectify.so building 'lxml.pyclasslookup' extension gcc -pthread -fno-strict-aliasing -g -Wall -Wstrict-prototypes -fPIC -I/home/heimes/.local/include -I/usr/include/libxml2 -I/home/heimes/dev/python/trunk/Include -I/home/heimes/dev/python/trunk -I/home/heimes/.local/include -c src/lxml/pyclasslookup.c -o build/temp.linux-i686-2.6-pydebug/src/lxml/pyclasslookup.o -w gcc -pthread -shared build/temp.linux-i686-2.6-pydebug/src/lxml/pyclasslookup.o -L/home/heimes/.local/lib -L/home/heimes/.local/lib -Wl,-R/home/heimes/.local/lib -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-i686-2.6-pydebug/lxml/pyclasslookup.so lxml-1.3.6$ ~/dev/python/trunk/python setup.py install --user ... As you can see etree.so is going to load my local libxslt library instead of the system one. $ ldd /home/heimes/.local/lib/python2.6/site-packages/lxml/etree.so linux-gate.so.1 => (0xb7fc5000) libxslt.so.1 => /home/heimes/.local/lib/libxslt.so.1 (0xb7e89000) libexslt.so.0 => /home/heimes/.local/lib/libexslt.so.0 (0xb7e78000) libxml2.so.2 => /usr/lib/libxml2.so.2 (0xb7d3d000) libz.so.1 => /usr/lib/libz.so.1 (0xb7d28000) libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7d03000) libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7ceb000) libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7b9c000) libgcrypt.so.11 => /lib/libgcrypt.so.11 (0xb7b4e000) libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7b4a000) /lib/ld-linux.so.2 (0xb7fc6000) libgpg-error.so.0 => /lib/libgpg-error.so.0 (0xb7b46000) Comments? :) Christian From brett at python.org Fri May 9 00:28:34 2008 From: brett at python.org (Brett Cannon) Date: Fri, 9 May 2008 00:28:34 +0200 Subject: [Python-Dev] PEP 370 extras In-Reply-To: <4823791F.2050507@cheimes.de> References: <4823791F.2050507@cheimes.de> Message-ID: On Fri, May 9, 2008 at 12:05 AM, Christian Heimes wrote: > I'm working on some extra cool and useful additions to PEP 370. It's > going to make compilation and usage of custom libraries very easy. Sit > back and watch the slide show. :) > > The site module has two new options. The --user-base option prints the > path to the user base directory to stdout. > What is the second option? And BTW, I wish we had this in 2.5 since I could finally install packages that can be used by both the OS X Python installation as well as my own build. =) This will be a handy feature once it is in a widely distributed version of Python. -Brett From lists at cheimes.de Fri May 9 00:30:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 09 May 2008 00:30:24 +0200 Subject: [Python-Dev] PEP 370 extras In-Reply-To: References: <4823791F.2050507@cheimes.de> Message-ID: <48237F00.3040802@cheimes.de> Brett Cannon schrieb: > On Fri, May 9, 2008 at 12:05 AM, Christian Heimes wrote: >> I'm working on some extra cool and useful additions to PEP 370. It's >> going to make compilation and usage of custom libraries very easy. Sit >> back and watch the slide show. :) >> >> The site module has two new options. The --user-base option prints the >> path to the user base directory to stdout. >> > > What is the second option? --user-site $ ~/dev/python/trunk/python -m site --help /home/heimes/dev/python/trunk/Lib/site.py [--user-base] [--user-site] Without arguments print some useful information With arguments print the value of USER_BASE and/or USER_SITE separated by ':'. Exit codes with --user-base or --user-site: 0 - user site directory is enabled 1 - user site diretory is disabled by user 2 - uses site directory is disabled by super user or for security reasons >2 - unknown error From brett at python.org Fri May 9 00:39:50 2008 From: brett at python.org (Brett Cannon) Date: Fri, 9 May 2008 00:39:50 +0200 Subject: [Python-Dev] PEP 370 extras In-Reply-To: <48237F00.3040802@cheimes.de> References: <4823791F.2050507@cheimes.de> <48237F00.3040802@cheimes.de> Message-ID: On Fri, May 9, 2008 at 12:30 AM, Christian Heimes wrote: > Brett Cannon schrieb: > > > On Fri, May 9, 2008 at 12:05 AM, Christian Heimes wrote: > >> I'm working on some extra cool and useful additions to PEP 370. It's > >> going to make compilation and usage of custom libraries very easy. Sit > >> back and watch the slide show. :) > >> > >> The site module has two new options. The --user-base option prints the > >> path to the user base directory to stdout. > >> > > > > What is the second option? > > --user-site > Ah, OK. > $ ~/dev/python/trunk/python -m site --help > /home/heimes/dev/python/trunk/Lib/site.py [--user-base] [--user-site] > > Without arguments print some useful information > With arguments print the value of USER_BASE and/or USER_SITE separated > by ':'. > > Exit codes with --user-base or --user-site: > 0 - user site directory is enabled > 1 - user site diretory is disabled by user Typo: missing a "c" in "directory". > 2 - uses site directory is disabled by super user > or for security reasons > >2 - unknown error > > Yeah, they should both be useful for various path-related stuff. -Brett From tom at vector-seven.com Fri May 9 01:22:18 2008 From: tom at vector-seven.com (Thomas Lee) Date: Fri, 09 May 2008 09:22:18 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <4822EB63.1040608@gmail.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> Message-ID: <48238B2A.1090409@vector-seven.com> Nick Coghlan wrote: > > There are a lot of micro-optimisations that are actually context > independent, so moving them before the symtable pass should be quite > feasible - e.g. replacing "return None" with "return", stripping dead > code after a return statement, changing a "if not" statement into an > "if" statement with the two suites reversed, changing "(1, 2, 3)" into > a stored constant, folding "1 + 2" into the constant "3". > > I believe the goal is to see how many of the current bytecode > optimisations can actually be brought forward to the AST generation > stage, rather than waiting until after the bytecode symtable > calculation and compilation passes. > That's been the aim so far. It's been largely successful with the exception of a few edge cases (most notably the functions vs. generator stuff). The elimination of unreachable paths (whether they be things like "if 0: ..." or "return; ... more code ...") completely breaks generators since we might potentially be blowing away "yield" statements during the elimination process. The rest of the optimizations, as far as I can see, are much less scary. > The current structure goes: > > tokenisation->AST construction->symtable construction->bytecode > compilation->bytecode optimisation > > My understanding of what Thomas is trying to do is to make it look > more like this: > > tokenisation->AST construction->AST optimisation->symtable > construction->bytecode compilation > That's exactly right. I made a quick and dirty attempt at moving the AST optimization step after the symtable generation on the train home last night to see if Jeremy's suggestion gives us anything. Now, it does make the detection of generators a little easier, but it really doesn't give us all that much else. I'm happy to post a patch so you can see what I mean, but for now I think visiting the FunctionDef subtree to check for Yield nodes will be fine. Again, I might be wrong (as I've often been throughout this process!) but let's see how it goes. Obviously a bit less efficient, but function bodies really shouldn't be all that deep anyway. I've got a good idea about how I'm going to go forward with this. Cheers, T From rhamph at gmail.com Fri May 9 01:36:14 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 8 May 2008 17:36:14 -0600 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <48238B2A.1090409@vector-seven.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> <48238B2A.1090409@vector-seven.com> Message-ID: On Thu, May 8, 2008 at 5:22 PM, Thomas Lee wrote: > Nick Coghlan wrote: >> >> There are a lot of micro-optimisations that are actually context >> independent, so moving them before the symtable pass should be quite >> feasible - e.g. replacing "return None" with "return", stripping dead code >> after a return statement, changing a "if not" statement into an "if" >> statement with the two suites reversed, changing "(1, 2, 3)" into a stored >> constant, folding "1 + 2" into the constant "3". >> >> I believe the goal is to see how many of the current bytecode >> optimisations can actually be brought forward to the AST generation stage, >> rather than waiting until after the bytecode symtable calculation and >> compilation passes. >> > That's been the aim so far. It's been largely successful with the exception > of a few edge cases (most notably the functions vs. generator stuff). The > elimination of unreachable paths (whether they be things like "if 0: ..." or > "return; ... more code ...") completely breaks generators since we might > potentially be blowing away "yield" statements during the elimination > process. Also breaks various sanity checks relating to the global statement. -- Adam Olsen, aka Rhamphoryncus From barry at python.org Fri May 9 01:50:06 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 8 May 2008 19:50:06 -0400 Subject: [Python-Dev] RELEASED Python 2.6a3 and 3.0a5 Message-ID: <88DFD025-8670-42FA-9B73-AFF5193FB0AE@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team and the Python community, I am happy to announce the third alpha release of Python 2.6, and the fifth alpha release of Python 3.0. Please note that these are alpha releases, and as such are not suitable for production environments. We continue to strive for a high degree of quality, but there are still some known problems and the feature sets have not been finalized. These alphas are being released to solicit feedback and hopefully discover bugs, as well as allowing you to determine how changes in 2.6 and 3.0 might impact you. If you find things broken or incorrect, please submit a bug report at http://bugs.python.org For more information and downloadable distributions, see the Python 2.6 website: http://www.python.org/download/releases/2.6/ and the Python 3.0 web site: http://www.python.org/download/releases/3.0/ These are the last planned alphas for both versions. If all goes well, next month will see the first beta releases of both, which will also signal feature freeze. Two beta releases are planned, with the final releases scheduled for September 3, 2008. See PEP 361 for release details: http://www.python.org/dev/peps/pep-0361/ Enjoy, - -Barry Barry Warsaw barry at python.org Python 2.6/3.0 Release Manager (on behalf of the entire python-dev team) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCORrnEjvBPtnXfVAQIK+QQAgEUtAvW7uo0BxMiT1bCAo2E9ZecWJ9xe DBgd/5IK8moITkqhqGAH5UvfytV6uPkOMgGIS/Uvk4hzhU3jwSopEIDJLFQ5nGtC lCzOHzkDjSNZ8Q2OOAI9mbSHY8grvVxCMB4X2SVXIEMZ6M/X1AcV2b0utp9O1w/l T/PEvP8U1uY= =2Tnb -----END PGP SIGNATURE----- From tom at vector-seven.com Fri May 9 01:54:20 2008 From: tom at vector-seven.com (Thomas Lee) Date: Fri, 09 May 2008 09:54:20 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> <48238B2A.1090409@vector-seven.com> Message-ID: <482392AC.4010407@vector-seven.com> Adam Olsen wrote: > On Thu, May 8, 2008 at 5:22 PM, Thomas Lee wrote: > >> Nick Coghlan wrote: >> >>> There are a lot of micro-optimisations that are actually context >>> independent, so moving them before the symtable pass should be quite >>> feasible - e.g. replacing "return None" with "return", stripping dead code >>> after a return statement, changing a "if not" statement into an "if" >>> statement with the two suites reversed, changing "(1, 2, 3)" into a stored >>> constant, folding "1 + 2" into the constant "3". >>> >>> I believe the goal is to see how many of the current bytecode >>> optimisations can actually be brought forward to the AST generation stage, >>> rather than waiting until after the bytecode symtable calculation and >>> compilation passes. >>> >>> >> That's been the aim so far. It's been largely successful with the exception >> of a few edge cases (most notably the functions vs. generator stuff). The >> elimination of unreachable paths (whether they be things like "if 0: ..." or >> "return; ... more code ...") completely breaks generators since we might >> potentially be blowing away "yield" statements during the elimination >> process. >> > > Also breaks various sanity checks relating to the global statement. > > What sanity checks are these exactly? Is this related to the lnotab? Cheers, T From musiccomposition at gmail.com Fri May 9 02:02:29 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 8 May 2008 19:02:29 -0500 Subject: [Python-Dev] [Python-3000] RELEASED Python 2.6a3 and 3.0a5 In-Reply-To: <88DFD025-8670-42FA-9B73-AFF5193FB0AE@python.org> References: <88DFD025-8670-42FA-9B73-AFF5193FB0AE@python.org> Message-ID: <1afaf6160805081702i522aade4ife0a3562d4c904bd@mail.gmail.com> On Thu, May 8, 2008 at 6:50 PM, Barry Warsaw wrote: > On behalf of the Python development team and the Python community, I am > happy to announce the third alpha release of Python 2.6, and the fifth alpha > release of Python 3.0. Special thanks goes to Christian Heimes for his last minute detection and fix of a ref leak. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From skip at pobox.com Fri May 9 02:35:09 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 8 May 2008 19:35:09 -0500 Subject: [Python-Dev] [Python-checkins] [Python-3000] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <20080508142356.GA8525@amk-desktop.matrixgroup.net> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> <4822FEBF.9060800@gmail.com> <20080508142356.GA8525@amk-desktop.matrixgroup.net> Message-ID: <18467.39997.908036.839451@montanaro-dyndns-org.local> amk> Subscribers to the python-dev digests may also not see a posting amk> immediately, waiting until the next digest is sent. Or python-dev subscribers might delete the announcement of the freeze without paying close attention to it then wonder a few hours later whether or not it's okay to check anything in. (Speaking as someone with a problem in this area.) I'm all in favor of a technical fix to my memory problems. <0.5 wink> Skip From steve at holdenweb.com Fri May 9 03:12:57 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 08 May 2008 21:12:57 -0400 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <482392AC.4010407@vector-seven.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> <48238B2A.1090409@vector-seven.com> <482392AC.4010407@vector-seven.com> Message-ID: Thomas Lee wrote: > Adam Olsen wrote: >> On Thu, May 8, 2008 at 5:22 PM, Thomas Lee wrote: >> >>> Nick Coghlan wrote: >>> >>>> There are a lot of micro-optimisations that are actually context >>>> independent, so moving them before the symtable pass should be quite >>>> feasible - e.g. replacing "return None" with "return", stripping >>>> dead code >>>> after a return statement, changing a "if not" statement into an "if" >>>> statement with the two suites reversed, changing "(1, 2, 3)" into a >>>> stored >>>> constant, folding "1 + 2" into the constant "3". >>>> >>>> I believe the goal is to see how many of the current bytecode >>>> optimisations can actually be brought forward to the AST generation >>>> stage, >>>> rather than waiting until after the bytecode symtable calculation and >>>> compilation passes. >>>> >>>> >>> That's been the aim so far. It's been largely successful with the >>> exception >>> of a few edge cases (most notably the functions vs. generator stuff). >>> The >>> elimination of unreachable paths (whether they be things like "if 0: >>> ..." or >>> "return; ... more code ...") completely breaks generators since we might >>> potentially be blowing away "yield" statements during the elimination >>> process. >>> >> >> Also breaks various sanity checks relating to the global statement. >> >> > What sanity checks are these exactly? Is this related to the lnotab? > > Cheers, > T > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > > While not strictly related to the global statement, perhaps Adam refers to the possibility of optimizing away code with an assignment which would make a name be recognized as local? If you're worried about "yield" disappearing you should also be worried about assignments disappearing, since that might cause names to be interpreted as globals. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From rhamph at gmail.com Fri May 9 03:38:29 2008 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 8 May 2008 19:38:29 -0600 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <482392AC.4010407@vector-seven.com> References: <481846D2.6090509@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> <48238B2A.1090409@vector-seven.com> <482392AC.4010407@vector-seven.com> Message-ID: On Thu, May 8, 2008 at 5:54 PM, Thomas Lee wrote: > Adam Olsen wrote: >> >> On Thu, May 8, 2008 at 5:22 PM, Thomas Lee wrote: >> >>> >>> Nick Coghlan wrote: >>> >>>> >>>> There are a lot of micro-optimisations that are actually context >>>> independent, so moving them before the symtable pass should be quite >>>> feasible - e.g. replacing "return None" with "return", stripping dead >>>> code >>>> after a return statement, changing a "if not" statement into an "if" >>>> statement with the two suites reversed, changing "(1, 2, 3)" into a >>>> stored >>>> constant, folding "1 + 2" into the constant "3". >>>> >>>> I believe the goal is to see how many of the current bytecode >>>> optimisations can actually be brought forward to the AST generation >>>> stage, >>>> rather than waiting until after the bytecode symtable calculation and >>>> compilation passes. >>>> >>>> >>> >>> That's been the aim so far. It's been largely successful with the >>> exception >>> of a few edge cases (most notably the functions vs. generator stuff). The >>> elimination of unreachable paths (whether they be things like "if 0: ..." >>> or >>> "return; ... more code ...") completely breaks generators since we might >>> potentially be blowing away "yield" statements during the elimination >>> process. >>> >> >> Also breaks various sanity checks relating to the global statement. >> >> > > What sanity checks are these exactly? Is this related to the lnotab? Here we are. In 2.4.4: >>> def foo(): ... print test ... if 0: ... import test ... >>> foo() Traceback (most recent call last): File "", line 1, in ? File "", line 2, in foo NameError: global name 'test' is not defined 2.5.1 correctly raises an UnboundLocalError instead. Searching the bug DB for old bugs involving the optimizer showed several variations on this, such as "if 0: yield None" at module scope not raising a SyntaxError (Issue 1875, still open in trunk). Clearly there needs to be a general approach to avoid affecting these checks with optimizations. -- Adam Olsen, aka Rhamphoryncus From ncoghlan at gmail.com Fri May 9 03:38:51 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 09 May 2008 11:38:51 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> <48238B2A.1090409@vector-seven.com> <482392AC.4010407@vector-seven.com> Message-ID: <4823AB2B.90702@gmail.com> Steve Holden wrote: > While not strictly related to the global statement, perhaps Adam refers > to the possibility of optimizing away code with an assignment which > would make a name be recognized as local? If you're worried about > "yield" disappearing you should also be worried about assignments > disappearing, since that might cause names to be interpreted as globals. And once you start annotating functions as generators or not, and variable names as locals or cell variables or globals, you're starting to build up a substantial fraction of the information that is already collected during the symtable construction pass. Perhaps the initial attempt at this should just focus on identifying those operations which have the potential to alter the results of the symtable construction, and leave those to the bytecode optimisation step for the moment. Doing the symtable pass twice seems fairly undesirable, even if it does let us trim some dead code out of the AST. Cheers, Nick. P.S. Here's a nonsensical example where optimising away the dead code after the return statement would significantly change the code's semantics: >>> def silly(): ... def inner(): ... print cell_var ... return inner ... cell_var = None ... >>> silly()() Traceback (most recent call last): File "", line 1, in File "", line 3, in inner NameError: free variable 'cell_var' referenced before assignment in enclosing scope -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tom at vector-seven.com Fri May 9 03:51:08 2008 From: tom at vector-seven.com (Thomas Lee) Date: Fri, 09 May 2008 11:51:08 +1000 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <4823AB2B.90702@gmail.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> <481B51A8.5050609@v.loewis.de> <4821C31D.9000506@gmail.com> <4821CE31.40609@vector-seven.com> <4822EB63.1040608@gmail.com> <48238B2A.1090409@vector-seven.com> <482392AC.4010407@vector-seven.com> <4823AB2B.90702@gmail.com> Message-ID: <4823AE0C.5010400@vector-seven.com> Nick Coghlan wrote: > Steve Holden wrote: >> While not strictly related to the global statement, perhaps Adam >> refers to the possibility of optimizing away code with an assignment >> which would make a name be recognized as local? If you're worried >> about "yield" disappearing you should also be worried about >> assignments disappearing, since that might cause names to be >> interpreted as globals. > > And once you start annotating functions as generators or not, and > variable names as locals or cell variables or globals, you're starting > to build up a substantial fraction of the information that is already > collected during the symtable construction pass. > > Perhaps the initial attempt at this should just focus on identifying > those operations which have the potential to alter the results of the > symtable construction, and leave those to the bytecode optimisation > step for the moment. Doing the symtable pass twice seems fairly > undesirable, even if it does let us trim some dead code out of the AST. > Sounds good. We can always come back to it. Cheers, T From stephen at xemacs.org Fri May 9 10:11:08 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 09 May 2008 17:11:08 +0900 Subject: [Python-Dev] [Python-3000] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> <617FFC34-8C94-48ED-951D-59F3467E6DC7@python.org> Message-ID: <8763tnnbhf.fsf@uwakimon.sk.tsukuba.ac.jp> Michael Urman writes: > I know this way is fairly entrenched in the python release process, > but it sounds like it's using the tools incorrectly. In particular > with subversion is very easy (compared to cvs) to branch and to switch > branches locally. Why not create a new prerelease branch at the > beginning of freeze and only merge in the critical changes? Well, speaking from experience: - some of the "critical changes" may only get committed on the release branch - something different from what's in the mainline may get committed on the release branch - the milestones are on a sideline, not on the mainline. Getting these points right is essential to ensure that the beta testers' work is actually relevant to the development process, that bisection searches work correctly, etc. > only the release manager need know or care about the branch, and > nobody else has to really modify his behavior. Behavior modification is the main point of having a release cycle. Setting deadlines, changing the nature of the patches, bringing issues to closure, etc. A release without a freeze is like a sentence without a period, IMO. From mal at egenix.com Fri May 9 12:44:01 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 09 May 2008 12:44:01 +0200 Subject: [Python-Dev] [Python-3000] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> Message-ID: <48242AF1.906@egenix.com> On 2008-05-08 13:59, Barry Warsaw wrote: > On May 8, 2008, at 7:54 AM, Benjamin Peterson wrote: > >> On Thu, May 8, 2008 at 6:32 AM, Barry Warsaw wrote: >>> Since the trunk buildbots appear to be mostly happy (well those that are >>> connected anyway), and because I couldn't get the releases out last >>> night, >>> I'll let this one slide. I'd like to find a way to more forcefully >>> enforce >>> commit freezes for the betas though. > >> I wonder if you couldn't alter the server side commit hook to reject >> everything with the message "Sorry, we're in a freeze." (You'd have to >> make an exception for yourself.) > > This is exactly what I'm thinking about! +1, that's easy to do with Subversion and doesn't hurt anyone. Please also use a term like "freeze" or "frozen" in the subject line of the announcement - perhaps even in capital letters. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 09 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From barry at python.org Fri May 9 14:10:06 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 9 May 2008 08:10:06 -0400 Subject: [Python-Dev] PEP 370 extras In-Reply-To: <4823791F.2050507@cheimes.de> References: <4823791F.2050507@cheimes.de> Message-ID: <57897D5F-D6F6-4A3D-80D3-FA0A367657EB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 8, 2008, at 6:05 PM, Christian Heimes wrote: > I'm working on some extra cool and useful additions to PEP 370. It's > going to make compilation and usage of custom libraries very easy. Sit > back and watch the slide show. :) > > Comments? :) Christian, you get this week's Awesome Award. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCQ/HnEjvBPtnXfVAQK7QgQAumf3wL8nQeH6CVC4hlaqjNQ330KSaVfj /o2bKusu77R+MYyYq/OV5k3q2/eo1XyPUDlE4+JXldvnhmjmx33zTv+5MDZLIWbN PPTfmYosHcXMh4CG6iqFqZxCoobhGUJNWuEt/X5bcOkmh/v4TPqCO0QYarV3cA69 jVQzSPgIce0= =x/tK -----END PGP SIGNATURE----- From barry at python.org Fri May 9 14:22:32 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 9 May 2008 08:22:32 -0400 Subject: [Python-Dev] [Python-3000] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <4822F062.7090305@cheimes.de> <617FFC34-8C94-48ED-951D-59F3467E6DC7@python.org> Message-ID: <520D2894-2296-4C6D-97FF-8521520E8E81@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 8, 2008, at 3:03 PM, Georg Brandl wrote: > > While I'm +0 on the commit hook, it would help if a mail that > announces > a freeze would > - not be hidden in a thread on python-dev and > - have a easily recognizable title, like "[TRUNK FREEZE] ....". I will make the freeze announcement more recognizable in the future, but I also want to point out that the entire release schedule has been published far in advance in PEP 361. At this point, the freeze dates should come as no surprise. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCRCCnEjvBPtnXfVAQKJgAQAojZ5vIg2K4q4e+XEHogQKeFjxkh5+o6U eWDjmkeVImwe1Sylb+mCqrxQ7JNY6d1m35hQsna/Ghan1IVIQ857fCBXS84aIUGl AGAnbrzxAt7RoYz/dyhz2twf1Uui5OVGOCYnmZ3ExZhTrEHN7ze43C+Blir0sH+4 DCuDj4xmpMM= =6W75 -----END PGP SIGNATURE----- From skip at pobox.com Fri May 9 14:15:21 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 9 May 2008 07:15:21 -0500 Subject: [Python-Dev] Code Freeze - full or partial? In-Reply-To: <48242AF1.906@egenix.com> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <48242AF1.906@egenix.com> Message-ID: <18468.16473.206010.819980@montanaro-dyndns-org.local> In the past I seem to recall that the Python code proper might be frozen (for a day or two) before a release, but that it was okay to still commit changes to non-code files such as documentation or files in Misc. Is this still the case in the new release-early-release-often regime? Is the intention to make the duration of the code freeze so short (a few minutes or hours) that it's not worth the effort to make this distinction? Skip From mal at egenix.com Fri May 9 15:15:31 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 09 May 2008 15:15:31 +0200 Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? Message-ID: <48244E73.7060603@egenix.com> Is there a tool available that can convert 2.x code automagically to the .format() method syntax ? Just did a quick grep of our code base and it has some 2000 lines of code that would need to be changed. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 09 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From barry at python.org Fri May 9 15:25:17 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 9 May 2008 09:25:17 -0400 Subject: [Python-Dev] [Python-3000] [Python-checkins] r62848 - python/trunk/Objects/setobject.c In-Reply-To: <48242AF1.906@egenix.com> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <48242AF1.906@egenix.com> Message-ID: <308D6BF1-936D-45F2-960B-0A42D04186A5@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 9, 2008, at 6:44 AM, M.-A. Lemburg wrote: > On 2008-05-08 13:59, Barry Warsaw wrote: >> On May 8, 2008, at 7:54 AM, Benjamin Peterson wrote: >>> On Thu, May 8, 2008 at 6:32 AM, Barry Warsaw >>> wrote: >>>> Since the trunk buildbots appear to be mostly happy (well those >>>> that are >>>> connected anyway), and because I couldn't get the releases out >>>> last night, >>>> I'll let this one slide. I'd like to find a way to more >>>> forcefully enforce >>>> commit freezes for the betas though. >>> I wonder if you couldn't alter the server side commit hook to reject >>> everything with the message "Sorry, we're in a freeze." (You'd >>> have to >>> make an exception for yourself.) >> This is exactly what I'm thinking about! > > +1, that's easy to do with Subversion and doesn't hurt anyone. Agreed. Look for it for the first beta. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCRQvXEjvBPtnXfVAQKyLwP8D0AVX+jgvy04hM207eeWRZb3JcHMtZuP ZcOuBQsCsVFppCxAreYIwfa0e6TD2LHBV4uz/G7Nxt6qNI6SY7lHQezNg4RezFwJ e93HAGdD0djj4BrL/xCr0wrK6wCwjodcvcjFdqTjEdLnkS7KGM9ooW8ZdYjQp6jI E+ZLDdhQ/KY= =24yM -----END PGP SIGNATURE----- From skip at pobox.com Fri May 9 15:29:17 2008 From: skip at pobox.com (skip at pobox.com) Date: Fri, 9 May 2008 08:29:17 -0500 Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? In-Reply-To: <48244E73.7060603@egenix.com> References: <48244E73.7060603@egenix.com> Message-ID: <18468.20909.201736.367371@montanaro-dyndns-org.local> mal> Is there a tool available that can convert 2.x code automagically mal> to the .format() method syntax ? mal> Just did a quick grep of our code base and it has some 2000 lines mal> of code that would need to be changed. I suggested a 2to3 fixer for this but was shot down. Skip From barry at python.org Fri May 9 15:30:11 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 9 May 2008 09:30:11 -0400 Subject: [Python-Dev] [Python-3000] Code Freeze - full or partial? In-Reply-To: <18468.16473.206010.819980@montanaro-dyndns-org.local> References: <20080508043520.B60821E400E@bag.python.org> <718F3CF3-E61B-43DC-81CC-E6EF0AAB9403@python.org> <1afaf6160805080454m3e0422b5sb9b7931591b703c3@mail.gmail.com> <18BD2693-5453-450D-A7C4-6B4B6F5C3AB9@python.org> <48242AF1.906@egenix.com> <18468.16473.206010.819980@montanaro-dyndns-org.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 9, 2008, at 8:15 AM, skip at pobox.com wrote: > In the past I seem to recall that the Python code proper might be > frozen > (for a day or two) before a release, but that it was okay to still > commit > changes to non-code files such as documentation or files in Misc. > Is this > still the case in the new release-early-release-often regime? Is the > intention to make the duration of the code freeze so short (a few > minutes or > hours) that it's not worth the effort to make this distinction? For the alphas, that's certainly been the case because it hasn't been necessary to coordinate all the Experts. IOW, it's okay for the Windows installer to get uploaded a few hours after the tarballs. For the betas, rcs and finals, I think we want a little bit more coordination (correct me if you disagree). So in that case, there may be a longer freeze. Even in that case, I don't envision more than a 24 hour freeze hopefully. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCRR5HEjvBPtnXfVAQLa+gP8CL9koa5eGBvP8g+CA8l61SIuluHNbPkq SH7uOiPMeuIX392xy82ixnXjYTlCJn9epWouYkiWta3GA+ZaCcmTFFavZ3ZbLbE3 uxfzhCWsZ5EUW5/iDCOUrlEwuxXJ6FU4naRTaTCBTELXRKvb3sI5C2pFjrb6JTZc hP2hP6m+A2Y= =avCD -----END PGP SIGNATURE----- From barry at python.org Fri May 9 15:35:32 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 9 May 2008 09:35:32 -0400 Subject: [Python-Dev] [Python-checkins] r62916 - sandbox/trunk/release/release.py In-Reply-To: <48244A0C.7070506@gmail.com> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <48244A0C.7070506@gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 9, 2008, at 8:56 AM, Nick Coghlan wrote: > Barry Warsaw wrote: >> Single quotes are easier to type so I prefer them. I very rarely >> find myself using apostrophes in non-triple quoted strings. I like >> it when things are easier to type, but I feel a vague sense of deja >> vu here <>wink<>. :) > > I'm certainly not suggesting that anyway else needs to agree with > that particular idiosyncrasy of mine - there are plenty of reasons > to prefer to do things the other way around. And when I'm changing > existing code, I try to stick to whatever convention that code > already uses. Agreed, and sorry, I didn't mean to imply you (or anybody else) was wrong in your preferences! I still have this idea of playing "Stump The Guido" some day at Pycon, where we get a dozen or so Python hackers to write some similar code in their own idiosyncratic styles and we see if Guido can match the code with the hacker. At least one of /my/ major tells will have been eradicated by then. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCRTJHEjvBPtnXfVAQLsvwQAkexLiblEWUiJ/af/0hXAeemUjaCjplfZ gMdXNGjCTALFjhq2608CbXOY+fyrHhDC5D2JTl2LziH328HUD84hOL32fWin8R4X r270zzypE3PnLStEjObW1kTBNe0ROZv4DkgclUn+Y7xC7pvJFzQxEAX0OqiHZQUu UjylCKEw+ZI= =HHcl -----END PGP SIGNATURE----- From mal at egenix.com Fri May 9 16:02:39 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 09 May 2008 16:02:39 +0200 Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? In-Reply-To: <18468.20909.201736.367371@montanaro-dyndns-org.local> References: <48244E73.7060603@egenix.com> <18468.20909.201736.367371@montanaro-dyndns-org.local> Message-ID: <4824597F.3010003@egenix.com> On 2008-05-09 15:29, skip at pobox.com wrote: > mal> Is there a tool available that can convert 2.x code automagically > mal> to the .format() method syntax ? > > mal> Just did a quick grep of our code base and it has some 2000 lines > mal> of code that would need to be changed. > > I suggested a 2to3 fixer for this but was shot down. Well, ideally such a tool should address 2to2 :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 09 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From rbp at isnomore.net Fri May 9 16:02:58 2008 From: rbp at isnomore.net (Rodrigo Bernardo Pimentel) Date: Fri, 9 May 2008 11:02:58 -0300 Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? In-Reply-To: <18468.20909.201736.367371@montanaro-dyndns-org.local> References: <48244E73.7060603@egenix.com> <18468.20909.201736.367371@montanaro-dyndns-org.local> Message-ID: <20080509140257.GD6022@isnomore.net> On Fri, May 09 2008 at 10:29:17AM BRT, skip at pobox.com wrote: > > mal> Is there a tool available that can convert 2.x code automagically > mal> to the .format() method syntax ? > > mal> Just did a quick grep of our code base and it has some 2000 lines > mal> of code that would need to be changed. > > I suggested a 2to3 fixer for this but was shot down. I had mentioned it during the thread where it was shot down, but I have a Google Summer of Code project (now approved and in motion - huzza!) to add confidence ranking for 2to3 fixers. As a test for the project, I'll write a fixer for string conversion and see if having a confidence value is enough to make such a fixer useful. rbp -- Rodrigo Bernardo Pimentel | GPG: <0x0DB14978> From status at bugs.python.org Fri May 9 18:06:30 2008 From: status at bugs.python.org (Tracker) Date: Fri, 9 May 2008 18:06:30 +0200 (CEST) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080509160630.7DE657827D@psf.upfronthosting.co.za> ACTIVITY SUMMARY (05/02/08 - 05/09/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1860 open (+38) / 12766 closed (+26) / 14626 total (+64) Open issues with patches: 561 Average duration of open issues: 709 days. Median duration of open issues: 1369 days. Open Issues Breakdown open 1834 (+38) pending 26 ( +0) Issues Created Or Reopened (64) _______________________________ Cmd module doesn't support readline completion on OSX Leopard 05/02/08 http://bugs.python.org/issue2740 created ruben documentation -- do serversockets work with ipv6 05/02/08 http://bugs.python.org/issue2741 created mrm example code does not work 05/02/08 http://bugs.python.org/issue2742 created mrm Fix module output for warnings from the interpreter 05/02/08 CLOSED http://bugs.python.org/issue2743 created brett.cannon Fix test_cProfile 05/02/08 http://bugs.python.org/issue2744 created benjamin.peterson Add support for IsWow64Process 05/03/08 CLOSED http://bugs.python.org/issue2745 created mhammond patch ElementTree ProcessingInstruction uses character entities in con 05/03/08 http://bugs.python.org/issue2746 created waveform Documentation of new gobject types fails 05/03/08 http://bugs.python.org/issue2747 created aafshar ceil(), floor() and round() broken in Decimal 05/03/08 CLOSED http://bugs.python.org/issue2748 created marketdickinson patch Raise a DeprecationWarning for warnings.showwarning(.., line) 05/03/08 CLOSED http://bugs.python.org/issue2749 created brett.cannon easy Add simplejson to Python 2.6/3.0 standard library 05/03/08 http://bugs.python.org/issue2750 created bob.ippolito patch Regression for executing packages 05/04/08 CLOSED http://bugs.python.org/issue2751 created benjamin.peterson Wrong comment in socket documentation 05/04/08 CLOSED http://bugs.python.org/issue2752 created giampaolo.rodola Test issue (different spelling) 05/04/08 CLOSED http://bugs.python.org/issue2753 created loewis patch Mac version of IDLE doesn't scroll as expected 05/04/08 http://bugs.python.org/issue2754 created alextrob IDLE ignores module change before restart 05/04/08 http://bugs.python.org/issue2755 created mooz123 urllib2 add_header fails with existing unredirected_header 05/04/08 http://bugs.python.org/issue2756 created zathras patch Little error in unittest documentation 05/04/08 CLOSED http://bugs.python.org/issue2757 created humitos patch arg 2 of PyErr_SetFromErrnoWithFilename should be const 05/04/08 http://bugs.python.org/issue2758 created ukleinek Bool makes filter(None,it) redundant; remove None option? 05/04/08 CLOSED http://bugs.python.org/issue2759 reopened tjreedy Recent change in socket.py breaks urllib2 05/04/08 CLOSED http://bugs.python.org/issue2760 created therve Doc: built-in callables and map 05/04/08 CLOSED http://bugs.python.org/issue2761 created tjreedy Language Reference: obsolute 2.x notes 05/04/08 CLOSED http://bugs.python.org/issue2762 created tjreedy A socket example code shown in doc doesn't work on FreeBSD 05/04/08 http://bugs.python.org/issue2763 created giampaolo.rodola c_char doesn't implement py3k buffer interface 05/04/08 http://bugs.python.org/issue2764 created christian.heimes warnings: error in test_missing_filename_main_with_argv_empty_st 05/04/08 CLOSED http://bugs.python.org/issue2765 created christian.heimes Doubtfull code in 'doctest.DocTestSuite' 05/05/08 http://bugs.python.org/issue2766 created dmaurer "doctest.DocTestCase.debug" clears "test.globs" too early 05/05/08 http://bugs.python.org/issue2767 created dmaurer os.fstat and other os.f* methods should use PyObject_AsFileDescr 05/05/08 http://bugs.python.org/issue2768 created christian.heimes easy Small typo in whatsnew/2.6.rst 05/05/08 CLOSED http://bugs.python.org/issue2769 created quentin.gallet-gilles patch Subversion 1.5 Working Copies Break Setuptools 05/05/08 http://bugs.python.org/issue2770 created durin42 test issue 05/05/08 CLOSED http://bugs.python.org/issue2771 created gvanrossum Add PendingDeprecationWarning for % formatting 05/06/08 http://bugs.python.org/issue2772 created eric.smith patch, patch Wrong description of 'g' conversion type of string formatting op 05/06/08 CLOSED http://bugs.python.org/issue2773 created zybi ctypes documentation not effective 05/06/08 http://bugs.python.org/issue2774 created janssen Implement PEP 3108 05/06/08 http://bugs.python.org/issue2775 created brett.cannon easy urllib2.urlopen() gets confused with path with // in it 05/06/08 http://bugs.python.org/issue2776 created ambarish subprocess unit tests for kill, term and send_signal flaky 05/06/08 http://bugs.python.org/issue2777 created christian.heimes set_swap_bodies is unsafe 05/07/08 CLOSED http://bugs.python.org/issue2778 created Rhamphoryncus patch Python 3.0a4 crashes on script in a path with non-ASCII characte 05/07/08 CLOSED http://bugs.python.org/issue2779 created pietzcker Python 3.0a4 crashes on script in a path with non-ASCII characte 05/07/08 CLOSED http://bugs.python.org/issue2780 created pietzcker Tiny patch to _winreg docs 05/07/08 CLOSED http://bugs.python.org/issue2781 created tim.golden patch datetime/date strftime() method and time.strftime() inconsistenc 05/07/08 http://bugs.python.org/issue2782 created mishok13 ctypes.util.find_library() doesn't find OS X .bundle or .so libr 05/07/08 http://bugs.python.org/issue2783 created janssen easy fast builtin sum may leak 05/07/08 http://bugs.python.org/issue2784 created MrJean1 patch alternate fast builtin sum 05/07/08 CLOSED http://bugs.python.org/issue2785 created MrJean1 patch Names in traceback should have class names, if they're methods 05/07/08 http://bugs.python.org/issue2786 created smurfix Patch to flush unittest output 05/07/08 http://bugs.python.org/issue2787 created Rhamphoryncus patch ignore file for Mercurial 05/07/08 CLOSED http://bugs.python.org/issue2788 created pitrou Comparsion datetime objects with None 05/08/08 http://bugs.python.org/issue2789 created shura_zam patch sys.flags is missing bytes_warning 05/08/08 CLOSED http://bugs.python.org/issue2790 created schmir patch subprocess.py leaks fd in communicate 05/08/08 http://bugs.python.org/issue2791 created jrosdahl alternate fast builtin sum, rev'd 05/08/08 CLOSED http://bugs.python.org/issue2792 created MrJean1 patch Dictionary fails to index when adding list when in a deeply nest 05/08/08 CLOSED http://bugs.python.org/issue2793 created jreadon Figure directive not handled for latex writing in Sphinx 05/08/08 http://bugs.python.org/issue2794 created aafshar patch rename "Caveat" to "Warning" 05/09/08 CLOSED http://bugs.python.org/issue2795 created benjamin.peterson Build issues in DragonFly BSD 05/09/08 http://bugs.python.org/issue2796 created hasso patch Trunk build issues in DragonFly BSD 05/09/08 http://bugs.python.org/issue2797 created hasso patch Crash on non-Windows if Python runs from a non-ASCII directory 05/09/08 http://bugs.python.org/issue2798 created pitrou patch Remove PyUnicode_AsString(), rework PyUnicode_AsStringAndSize(), 05/09/08 http://bugs.python.org/issue2799 created lemburg make htmlhelp creates python30a5.chm but msi.py expects pydoc.ch 05/09/08 http://bugs.python.org/issue2800 created christian.heimes Py30a5 float.is_integer() raises ValueError 05/09/08 http://bugs.python.org/issue2801 created mark str.format() :n integer output 05/09/08 http://bugs.python.org/issue2802 created mark heapq.heappush called with too few arguments in sched.py 05/09/08 http://bugs.python.org/issue2803 created mthard patch Issues Now Closed (55) ______________________ Fast BytesIO implementation + misc changes 121 days http://bugs.python.org/issue1751 alexandre.vassalotti patch PEP 370 118 days http://bugs.python.org/issue1799 christian.heimes patch Potential overflows due to incorrect usage of PyUnicode_AsString 97 days http://bugs.python.org/issue1950 lemburg patch make sqlite.Row hashable correctly 75 days http://bugs.python.org/issue2152 ghaering patch sqlite numeric type conversion problems 67 days http://bugs.python.org/issue2157 ghaering patch confusing exception when opening a filename with nonprintable ch 71 days http://bugs.python.org/issue2158 gregory.p.smith namedtuple breaks refleak tests 48 days http://bugs.python.org/issue2448 christian.heimes Exception state lives too long in 3.0 40 days http://bugs.python.org/issue2507 barry patch macerrors.py cannot be imported due to non-ascii characters in c 32 days http://bugs.python.org/issue2520 ronaldoussoren patch If HAVE_LONG_LONG is not defined, longval will not be initialize 31 days http://bugs.python.org/issue2540 ghaering patch, patch Py30a4 RELNOTES only cover 30a1 and 30a2 34 days http://bugs.python.org/issue2547 barry Can't change the framework name on OS X builds 25 days http://bugs.python.org/issue2573 benjamin.peterson patch Vista UAC/elevation support for bdist_wininst 25 days http://bugs.python.org/issue2581 mhammond patch, patch Autodoc should skip inherited methods 16 days http://bugs.python.org/issue2656 georg.brandl Ignore case when checking algorithm in urllib2 9 days http://bugs.python.org/issue2695 georg.brandl patch Logging ancestors ignored after configuration 9 days http://bugs.python.org/issue2697 acooke Tiny fix for os.walk docstring 8 days http://bugs.python.org/issue2707 gregory.p.smith easy tempfile.mkstempf 9 days http://bugs.python.org/issue2717 skip.montanaro patch, patch, easy make compiling struct be passed around to all ast helpers 4 days http://bugs.python.org/issue2720 benjamin.peterson patch Autodoc's autoclass directive should include constructor docstri 7 days http://bugs.python.org/issue2726 abhik Failing decimal doctest 2 days http://bugs.python.org/issue2728 facundobatista patch Fix module output for warnings from the interpreter 0 days http://bugs.python.org/issue2743 brett.cannon Add support for IsWow64Process 1 days http://bugs.python.org/issue2745 loewis patch ceil(), floor() and round() broken in Decimal 6 days http://bugs.python.org/issue2748 marketdickinson patch Raise a DeprecationWarning for warnings.showwarning(.., line) 2 days http://bugs.python.org/issue2749 brett.cannon easy Regression for executing packages 0 days http://bugs.python.org/issue2751 ncoghlan Wrong comment in socket documentation 2 days http://bugs.python.org/issue2752 georg.brandl Test issue (different spelling) 0 days http://bugs.python.org/issue2753 loewis patch Little error in unittest documentation 0 days http://bugs.python.org/issue2757 georg.brandl patch Bool makes filter(None,it) redundant; remove None option? 1 days http://bugs.python.org/issue2759 georg.brandl Recent change in socket.py breaks urllib2 1 days http://bugs.python.org/issue2760 gregory.p.smith Doc: built-in callables and map 1 days http://bugs.python.org/issue2761 georg.brandl Language Reference: obsolute 2.x notes 1 days http://bugs.python.org/issue2762 georg.brandl warnings: error in test_missing_filename_main_with_argv_empty_st 0 days http://bugs.python.org/issue2765 benjamin.peterson Small typo in whatsnew/2.6.rst 0 days http://bugs.python.org/issue2769 georg.brandl patch test issue 0 days http://bugs.python.org/issue2771 gvanrossum Wrong description of 'g' conversion type of string formatting op 0 days http://bugs.python.org/issue2773 georg.brandl set_swap_bodies is unsafe 2 days http://bugs.python.org/issue2778 Rhamphoryncus patch Python 3.0a4 crashes on script in a path with non-ASCII characte 0 days http://bugs.python.org/issue2779 amaury.forgeotdarc Python 3.0a4 crashes on script in a path with non-ASCII characte 1 days http://bugs.python.org/issue2780 christian.heimes Tiny patch to _winreg docs 2 days http://bugs.python.org/issue2781 georg.brandl patch alternate fast builtin sum 0 days http://bugs.python.org/issue2785 rhettinger patch ignore file for Mercurial 1 days http://bugs.python.org/issue2788 georg.brandl sys.flags is missing bytes_warning 0 days http://bugs.python.org/issue2790 schmir patch alternate fast builtin sum, rev'd 0 days http://bugs.python.org/issue2792 rhettinger patch Dictionary fails to index when adding list when in a deeply nest 0 days http://bugs.python.org/issue2793 amaury.forgeotdarc rename "Caveat" to "Warning" 0 days http://bugs.python.org/issue2795 georg.brandl scripts destination directory not on default path 1741 days http://bugs.python.org/issue778799 ronaldoussoren MacPython builds with DESTROOT need fixup 1641 days http://bugs.python.org/issue835790 ronaldoussoren Support "hybrid" framework/unix style 1400 days http://bugs.python.org/issue984242 ronaldoussoren Ensure -single_module on Mac OS X 1382 days http://bugs.python.org/issue995019 ronaldoussoren Mac: make frameworkinstall skips docs, scripts 1231 days http://bugs.python.org/issue1087737 ronaldoussoren patch for bug 1170311 "zipfile UnicodeDecodeError" 330 days http://bugs.python.org/issue1734346 loewis patch platform.mac_ver() returning incorrect patch version 274 days http://bugs.python.org/issue1770190 ronaldoussoren patch urlparse.urljoin does not obey current uri rfc (rfc 3986) 258 days http://bugs.python.org/issue1779700 georg.brandl Top Issues Most Discussed (10) ______________________________ 37 Add simplejson to Python 2.6/3.0 standard library 6 days open http://bugs.python.org/issue2750 12 range: lean and mean 8 days open http://bugs.python.org/issue2735 11 set_swap_bodies is unsafe 2 days closed http://bugs.python.org/issue2778 11 Add PendingDeprecationWarning for % formatting 3 days open http://bugs.python.org/issue2772 10 repr() should not escape non-ASCII characters 25 days open http://bugs.python.org/issue2630 9 Ignore case when checking algorithm in urllib2 9 days closed http://bugs.python.org/issue2695 8 sys.flags is missing bytes_warning 0 days closed http://bugs.python.org/issue2790 8 ceil(), floor() and round() broken in Decimal 6 days closed http://bugs.python.org/issue2748 8 re.escape should not escape underscore 22 days open http://bugs.python.org/issue2650 7 performance problem in socket._fileobject.read 25 days open http://bugs.python.org/issue2632 From jek-gmane at kleckner.net Fri May 9 18:25:39 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Fri, 09 May 2008 09:25:39 -0700 Subject: [Python-Dev] RELEASED Python 2.6a3 and 3.0a5 In-Reply-To: <88DFD025-8670-42FA-9B73-AFF5193FB0AE@python.org> References: <88DFD025-8670-42FA-9B73-AFF5193FB0AE@python.org> Message-ID: Barry Warsaw wrote: ... > For more information and downloadable distributions, see the Python > 2.6 website: > > http://www.python.org/download/releases/2.6/ The .msi for 2.6a3 seems to be missing from: http://www.python.org/ftp/python/2.6/ and the download link from this page gives a 404: http://www.python.org/download/releases/2.6/ (This 3.0a5 .msi files are also missing) From janssen at parc.com Fri May 9 19:06:07 2008 From: janssen at parc.com (Bill Janssen) Date: Fri, 9 May 2008 10:06:07 PDT Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? In-Reply-To: <4824597F.3010003@egenix.com> References: <48244E73.7060603@egenix.com> <18468.20909.201736.367371@montanaro-dyndns-org.local> <4824597F.3010003@egenix.com> Message-ID: <08May9.100611pdt."58696"@synergy1.parc.xerox.com> > On 2008-05-09 15:29, skip at pobox.com wrote: > > mal> Is there a tool available that can convert 2.x code automagically > > mal> to the .format() method syntax ? > > > > mal> Just did a quick grep of our code base and it has some 2000 lines > > mal> of code that would need to be changed. > > > > I suggested a 2to3 fixer for this but was shot down. > > Well, ideally such a tool should address 2to2 :-) The curse of back-porting... I, too, would find such a tool helpful; I've got a thousand lines of %-formatting use just in UpLib alone. Bill From hall.jeff at gmail.com Fri May 9 19:15:11 2008 From: hall.jeff at gmail.com (Jeff Hall) Date: Fri, 9 May 2008 13:15:11 -0400 Subject: [Python-Dev] [Python-checkins] r62916 - sandbox/trunk/release/release.py In-Reply-To: References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <48244A0C.7070506@gmail.com> Message-ID: <1bc395c10805091015q141521a8le21366af2268579e@mail.gmail.com> I think a better "stump the guido" would be to just have people write the most convoluted things for simple processes and see how minimalist he could make the final code... I think we could have some fun with this ;) -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri May 9 19:18:23 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 9 May 2008 13:18:23 -0400 Subject: [Python-Dev] RELEASED Python 2.6a3 and 3.0a5 In-Reply-To: References: <88DFD025-8670-42FA-9B73-AFF5193FB0AE@python.org> Message-ID: <237EF689-8207-4897-B03A-1C0DC0087552@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 9, 2008, at 12:25 PM, Jim Kleckner wrote: > Barry Warsaw wrote: > ... >> For more information and downloadable distributions, see the Python >> 2.6 website: >> http://www.python.org/download/releases/2.6/ > > The .msi for 2.6a3 seems to be missing from: > http://www.python.org/ftp/python/2.6/ > and the download link from this page gives a 404: > http://www.python.org/download/releases/2.6/ > > (This 3.0a5 .msi files are also missing) Yes, Martin hasn't uploaded these yet, but they should show up any day now. For the betas, rcs and finals, we'll coordinate more closely. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBSCSHYHEjvBPtnXfVAQIWzgP+JsgRe12aq0Mqfz6+FpD9Wf5rJsGGqu0x M2xihvWQSlcoMvlJeTvTmRg0TiIXrM9EsCgtunwDiZcoA5blWz+QIh55ierv7G4N EhTrmmiLc1mD36otuTEBU7qgIl++BMC2eVyRr61+0mZYx49c8IAWpJpsSriPfXGZ 6qIuBYU3gUc= =KPAq -----END PGP SIGNATURE----- From martin at v.loewis.de Sat May 10 01:18:23 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 10 May 2008 01:18:23 +0200 Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? In-Reply-To: <48244E73.7060603@egenix.com> References: <48244E73.7060603@egenix.com> Message-ID: <4824DBBF.5000103@v.loewis.de> > Is there a tool available that can convert 2.x code automagically > to the .format() method syntax ? > > Just did a quick grep of our code base and it has some 2000 lines of code > that would need to be changed. Why do you think this code needs to change? I'd leave all the code as-is, and might not start using .format before Python 3.2, unless some coding convention says I have to. Regards, Martin From martin at v.loewis.de Sat May 10 01:38:58 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 10 May 2008 01:38:58 +0200 Subject: [Python-Dev] Warn about mktemp once again? In-Reply-To: <18466.23488.560586.183048@montanaro-dyndns-org.local> References: <18452.65322.843226.299769@montanaro-dyndns-org.local> <48214C48.2040005@v.loewis.de> <48224638.7040803@canterbury.ac.nz> <18466.23488.560586.183048@montanaro-dyndns-org.local> Message-ID: <4824E092.2080909@v.loewis.de> skip at pobox.com wrote: > This has been a fairly interesting discussion about tempfile.mktemp, but can > we return to the original question? Should we begin warning about its use > again? Should we ever warn about it? Given that there people continue to point out that there are completely acceptable use cases of it, I don't think we should ever warn about it. Regards, Martin From d.rothe at semantics.de Sat May 10 01:27:56 2008 From: d.rothe at semantics.de (Dirk Rothe) Date: Sat, 10 May 2008 01:27:56 +0200 Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? In-Reply-To: <08May9.100611pdt.\"58696\"@synergy1.parc.xerox.com> References: <48244E73.7060603@egenix.com> <18468.20909.201736.367371@montanaro-dyndns-org.local> <4824597F.3010003@egenix.com> <08May9.100611pdt.\"58696\"@synergy1.parc.xerox.com> Message-ID: On Fri, 09 May 2008 19:06:07 +0200, Bill Janssen wrote: > The curse of back-porting... I, too, would find such a tool helpful; > I've got a thousand lines of %-formatting use just in UpLib alone. yea, I hope such a Fixer will emerge, formatting via % is just too painless atm in 2.5. I guess our company should check out the 'preferred' way to do it real soon, or else the conversion process will be really time consuming. --dirk From martin at v.loewis.de Sat May 10 12:00:04 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 10 May 2008 12:00:04 +0200 Subject: [Python-Dev] GSoC Student Introduction, again In-Reply-To: References: <48228721.1030600@v.loewis.de> Message-ID: <48257224.7090909@v.loewis.de> > I've already wrapped all the Ttk functionality by now, and I will > complete the documentation till the first betas. > But as you know there isn't much people using (I know myself and a Tcl > guy) so I'm not sure if it would be acceptable to include this module > right into these betas, or would it be ? Should I start doing > something more/else ? No. My question really was whether you see any reason not to bundle Tk 8.5 with Python 2.6 (on Windows). Whether or not the Ttk API is exposed is a different matter - you might consider providing it as a distutils package. Regards, Martin From ggpolo at gmail.com Sat May 10 13:46:37 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Sat, 10 May 2008 08:46:37 -0300 Subject: [Python-Dev] GSoC Student Introduction, again In-Reply-To: <48257224.7090909@v.loewis.de> References: <48228721.1030600@v.loewis.de> <48257224.7090909@v.loewis.de> Message-ID: 2008/5/10 "Martin v. L?wis" : >> I've already wrapped all the Ttk functionality by now, and I will >> complete the documentation till the first betas. >> But as you know there isn't much people using (I know myself and a Tcl >> guy) so I'm not sure if it would be acceptable to include this module >> right into these betas, or would it be ? Should I start doing >> something more/else ? > > No. My question really was whether you see any reason not to bundle > Tk 8.5 with Python 2.6 (on Windows). No reasons that I'm aware of, Martin. I haven't tested it under Windows myself but given that it is not supposed to break tk 8.4 apps and I haven't experienced bugs here in Linux, I would say it is a safe bet to start bundling Tk 8.5 within the first betas. > Whether or not the Ttk API is > exposed is a different matter - you might consider providing it > as a distutils package. > > Regards, > Martin > Thanks, -- -- Guilherme H. Polo Goncalves From mal at egenix.com Sat May 10 15:24:42 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 10 May 2008 15:24:42 +0200 Subject: [Python-Dev] Tool for converting %-formatting to .format()ing ? In-Reply-To: <4824DBBF.5000103@v.loewis.de> References: <48244E73.7060603@egenix.com> <4824DBBF.5000103@v.loewis.de> Message-ID: <4825A21A.4040006@egenix.com> On 2008-05-10 01:18, Martin v. L?wis wrote: >> Is there a tool available that can convert 2.x code automagically >> to the .format() method syntax ? >> >> Just did a quick grep of our code base and it has some 2000 lines of code >> that would need to be changed. > > Why do you think this code needs to change? > > I'd leave all the code as-is, and might not start using .format before > Python 3.2, unless some coding convention says I have to. True, just wanted to know whether there is such a tool. I personally like the %-notation a lot, mainly because it's more or less the same as in C. %i, %s and %r are by far the most used format characters in our code base. Determining the position index and writing {0!s} or {0!r} instead (which requires quite a finger dance on a German keyboard) doesn't make .format() really attractive, IMHO. Perhaps you're right and it's better to wait a few rounds of refinements of .format() before jumping on that train :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 10 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From skip at pobox.com Sat May 10 16:29:45 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 10 May 2008 09:29:45 -0500 Subject: [Python-Dev] On quote styles In-Reply-To: References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> Message-ID: <18469.45401.859822.563618@montanaro-dyndns-org.local> Moving from python-checkins and giving this topic a proper subject. The original thread started here with a checkin by Benjamin: http://mail.python.org/pipermail/python-checkins/2008-May/069181.html While Python doesn't have a char type (yet), I still find the distinction between 'c' and "abc" useful to show intent (especially given my C background and occasional but increasingly rare dip into the CPython implementation). I must admit though that I'm not as consistent with this as I used to be or would like to be. I guess that's because I'm lazy and the byte compiler doesn't enforce that style. I do prefer to use """ as the triple quote string delimiter because I find that for most of my uses it makes more sense as apostrophes tend to be the quote chars most frequently embedded in them (I don't do much, if any, HTML these days). This is, of course, all personal preference, and definitely not PEP-8-able. maybe-if-we--uncle-timmy-will-sort-it-out-ly y'rs, Skip From g.brandl at gmx.net Sat May 10 17:51:06 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 10 May 2008 17:51:06 +0200 Subject: [Python-Dev] On quote styles In-Reply-To: <18469.45401.859822.563618@montanaro-dyndns-org.local> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> Message-ID: skip at pobox.com schrieb: > Moving from python-checkins and giving this topic a proper subject. The > original thread started here with a checkin by Benjamin: > > http://mail.python.org/pipermail/python-checkins/2008-May/069181.html > > While Python doesn't have a char type (yet), I still find the distinction > between 'c' and "abc" useful to show intent (especially given my C > background and occasional but increasingly rare dip into the CPython > implementation). I must admit though that I'm not as consistent with this > as I used to be or would like to be. I guess that's because I'm lazy and > the byte compiler doesn't enforce that style. > > I do prefer to use """ as the triple quote string delimiter because I find > that for most of my uses it makes more sense as apostrophes tend to be the > quote chars most frequently embedded in them (I don't do much, if any, HTML > these days). > > This is, of course, all personal preference, and definitely not > PEP-8-able. > > maybe-if-we--uncle-timmy-will-sort-it-out-ly y'rs, I think some people use the two string literal types to distinguish between text (potentially translatable strings) and data. This way, it's much easier to add i18n later if needed. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From tjreedy at udel.edu Sat May 10 20:15:26 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 10 May 2008 14:15:26 -0400 Subject: [Python-Dev] On quote styles References: <20080509005554.97A9E1E400C@bag.python.org><4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> Message-ID: wrote in message news:18469.45401.859822.563618 at montanaro-dyndns-org.local... | | Moving from python-checkins and giving this topic a proper subject. The | original thread started here with a checkin by Benjamin: | | http://mail.python.org/pipermail/python-checkins/2008-May/069181.html To me, those changes are all in the wrong direction and make the code *harder* to read. I tend to use 'hello' for constants and words and "Some original text" for words. I presume this is what Georg B. meant. tjr From greg.ewing at canterbury.ac.nz Sun May 11 02:29:04 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 May 2008 12:29:04 +1200 Subject: [Python-Dev] On quote styles In-Reply-To: <18469.45401.859822.563618@montanaro-dyndns-org.local> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> Message-ID: <48263DD0.90702@canterbury.ac.nz> skip at pobox.com wrote: > While Python doesn't have a char type (yet), I still find the distinction > between 'c' and "abc" useful to show intent (especially given my C > background The way I tend to use them is that "xxx" is for data operated on by the program and seen by the user, and 'xxx' is for things that are only used internally, e.g. enumerated type values and attribute names for use by getattr(). I wouldn't like to see any guideline that said you should only use double quotes or something like that. -- Greg From skip at pobox.com Sun May 11 04:51:39 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 10 May 2008 21:51:39 -0500 Subject: [Python-Dev] On quote styles In-Reply-To: <48263DD0.90702@canterbury.ac.nz> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> <48263DD0.90702@canterbury.ac.nz> Message-ID: <18470.24379.202238.513904@montanaro-dyndns-org.local> Greg> I wouldn't like to see any guideline that said you should only use Greg> double quotes or something like that. It might be useful to have a wiki page which identified some of the conventions people use. Skip From ncoghlan at gmail.com Sun May 11 04:52:08 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 11 May 2008 12:52:08 +1000 Subject: [Python-Dev] On quote styles In-Reply-To: <48263DD0.90702@canterbury.ac.nz> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> <48263DD0.90702@canterbury.ac.nz> Message-ID: <48265F58.8030301@gmail.com> Greg Ewing wrote: > I wouldn't like to see any guideline that said you > should only use double quotes or something like that. Nah, it was more a philosophical discussion prompted by a comment from me regarding my personal reasons for preferring different styles of quotes in certain situations. The various other comments that have come up is that this is one of the cases where having more than one way to do something is a benefit rather than a problem. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From brett at python.org Sun May 11 05:30:35 2008 From: brett at python.org (Brett Cannon) Date: Sat, 10 May 2008 20:30:35 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module Message-ID: I just realized that PEP 3108 was missing one proposal from the stdlib SIG (originally proposed by Facundo Batista) of copying the cgi.parse_qs() function over to the new urllib.parse module so that people no longer need to import the cgi module just for that one parsing function. Does anyone object if I update the PEP to include that change? -Brett From brett at python.org Sun May 11 05:38:00 2008 From: brett at python.org (Brett Cannon) Date: Sat, 10 May 2008 20:38:00 -0700 Subject: [Python-Dev] Trickery with moving urllib Message-ID: There is going to be an issue with the current proposal for keeping around urllib. Since the package is to be named the same thing as the module, to handle the new name that means urllib.__init__ will need to gain the Py3K warning for the new name. But that doesn't quite work as the package's module will be imported automatically any time the module within the package is used. I see three solutions for dealing with this. 1. Have stubs for the entire urllib API in urllib.__init__ that raise a DeprecationWarning either specifying the new name or saying the function/class is deprecated. 2. Rename urllib to urllib.fetch or urllib.old_request to get people to move over to urllib.request (aka urllib2) at some point. 3. Ditch urllib completely and have people move to urllib.request directly from 2.6 going into 3.0. I was a fan of 3 in the stdlib SIG, but MAL argued well for 2 since so much code uses urllib (although I have always wondered how many *really* use urllib and how many could easily just move over to urllib2). The current solution of keeping around urllib's classes was a compromise, but I had not thought out the transition yet (and Jeremy Hylton will be running into this as he has volunteered to handle this). What do people think we should do? -Brett From skip at pobox.com Sun May 11 05:43:17 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 10 May 2008 22:43:17 -0500 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: Message-ID: <18470.27477.880117.113109@montanaro-dyndns-org.local> Brett> There is going to be an issue with the current proposal for Brett> keeping around urllib. Since the package is to be named the same Brett> thing as the module.... Is this the only module morphing into a package of the same name? Skip From guido at python.org Sun May 11 05:49:40 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 10 May 2008 20:49:40 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: Works for me. The other thing I always use from cgi is escape() -- will that be available somewhere else too? On Sat, May 10, 2008 at 8:30 PM, Brett Cannon wrote: > I just realized that PEP 3108 was missing one proposal from the stdlib > SIG (originally proposed by Facundo Batista) of copying the > cgi.parse_qs() function over to the new urllib.parse module so that > people no longer need to import the cgi module just for that one > parsing function. Does anyone object if I update the PEP to include > that change? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake at acm.org Sun May 11 05:56:47 2008 From: fdrake at acm.org (Fred Drake) Date: Sat, 10 May 2008 23:56:47 -0400 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: > Works for me. The other thing I always use from cgi is escape() -- > will that be available somewhere else too? xml.sax.saxutils.escape() would be an appropriate replacement, though the location is a little funky. -Fred -- Fred Drake From alexandre at peadrop.com Sun May 11 08:46:23 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 11 May 2008 02:46:23 -0400 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: <18470.27477.880117.113109@montanaro-dyndns-org.local> References: <18470.27477.880117.113109@montanaro-dyndns-org.local> Message-ID: On Sat, May 10, 2008 at 11:43 PM, wrote: > > Brett> There is going to be an issue with the current proposal for > Brett> keeping around urllib. Since the package is to be named the same > Brett> thing as the module.... > > Is this the only module morphing into a package of the same name? > No, it is not. The dbm package will have the same issue. -- Alexandre From alexandre at peadrop.com Sun May 11 08:58:26 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 11 May 2008 02:58:26 -0400 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: Message-ID: On Sat, May 10, 2008 at 11:38 PM, Brett Cannon wrote: > I see three solutions for dealing with this. > > 1. Have stubs for the entire urllib API in urllib.__init__ that raise > a DeprecationWarning either specifying the new name or saying the > function/class is deprecated. > > 2. Rename urllib to urllib.fetch or urllib.old_request to get people > to move over to urllib.request (aka urllib2) at some point. > I am probably missing something, because I don't see how this solution would solve the problem. The warning in urllib.__init__ will still be issued when people will import urllib.fetch (or urllib.fetch). -- Alexandre From g.brandl at gmx.net Sun May 11 09:00:10 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 May 2008 09:00:10 +0200 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: Fred Drake schrieb: > On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >> Works for me. The other thing I always use from cgi is escape() -- >> will that be available somewhere else too? > > > xml.sax.saxutils.escape() would be an appropriate replacement, though > the location is a little funky. Ideally it would be xml.escape(). Perhaps there are more of these generally- useful XML thingies to put there? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From fdrake at acm.org Sun May 11 09:04:06 2008 From: fdrake at acm.org (Fred Drake) Date: Sun, 11 May 2008 03:04:06 -0400 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: Message-ID: <1786A4A3-F89F-4BCA-B53E-CEA8F9473E99@acm.org> On Sat, May 10, 2008 at 11:38 PM, Brett Cannon wrote: > I see three solutions for dealing with this. > > 1. Have stubs for the entire urllib API in urllib.__init__ that raise > a DeprecationWarning either specifying the new name or saying the > function/class is deprecated. On May 11, 2008, at 2:58 AM, Alexandre Vassalotti wrote: > I am probably missing something, because I don't see how this solution > would solve the problem. The warning in urllib.__init__ will still be > issued when people will import urllib.fetch (or urllib.fetch). Were the warnings issued on import, yes, but I think Brett's suggestion was really about issuing warnings when the stub functions were called. For classes, I guess this could be handled using the __init__ methods, but I'm not sure I like that, or that it will be easy to get exactly the right behavior in all cases. -Fred -- Fred Drake From g.brandl at gmx.net Sun May 11 17:31:07 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 May 2008 17:31:07 +0200 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: Message-ID: Brett Cannon schrieb: > There is going to be an issue with the current proposal for keeping > around urllib. Since the package is to be named the same thing as the > module, to handle the new name that means urllib.__init__ will need to > gain the Py3K warning for the new name. But that doesn't quite work as > the package's module will be imported automatically any time the > module within the package is used. In this context, what's become of PEP 364? Is it still alive? Does it cover this case? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From jeremy at alum.mit.edu Sun May 11 18:02:38 2008 From: jeremy at alum.mit.edu (Jeremy Hylton) Date: Sun, 11 May 2008 12:02:38 -0400 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: Message-ID: On Sat, May 10, 2008 at 11:38 PM, Brett Cannon wrote: > There is going to be an issue with the current proposal for keeping > around urllib. Since the package is to be named the same thing as the > module, to handle the new name that means urllib.__init__ will need to > gain the Py3K warning for the new name. But that doesn't quite work as > the package's module will be imported automatically any time the > module within the package is used. > > I see three solutions for dealing with this. > > 1. Have stubs for the entire urllib API in urllib.__init__ that raise > a DeprecationWarning either specifying the new name or saying the > function/class is deprecated. > > 2. Rename urllib to urllib.fetch or urllib.old_request to get people > to move over to urllib.request (aka urllib2) at some point. > > 3. Ditch urllib completely and have people move to urllib.request > directly from 2.6 going into 3.0. > > I was a fan of 3 in the stdlib SIG, but MAL argued well for 2 since so > much code uses urllib (although I have always wondered how many > *really* use urllib and how many could easily just move over to > urllib2). The current solution of keeping around urllib's classes was > a compromise, but I had not thought out the transition yet (and Jeremy > Hylton will be running into this as he has volunteered to handle > this). > > What do people think we should do? I am migrating all the stdlib code to use urllib2 APIs when possible. It seems straightforward enough to keep the old urllib classes in urllib.request, but change the standard API functions (urlopen, urlretrieve) to use the urllib2 implementation. We could probably have a big collection of deprecation warnings in the urllib/__init__ without much trouble. It doesn't have any code at this point. Jeremy > > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu > From bcannon at gmail.com Sun May 11 19:36:55 2008 From: bcannon at gmail.com (Brett C.) Date: Sun, 11 May 2008 10:36:55 -0700 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: <18470.27477.880117.113109@montanaro-dyndns-org.local> References: <18470.27477.880117.113109@montanaro-dyndns-org.local> Message-ID: <767BE43F-16D4-4488-9971-76AB0920B124@gmail.com> On 10-May-08, at 20:43, skip at pobox.com wrote: > > Brett> There is going to be an issue with the current proposal for > Brett> keeping around urllib. Since the package is to be named > the same > Brett> thing as the module.... > > Is this the only module morphing into a package of the same name? > Yes. -Brett > Skip From bcannon at gmail.com Sun May 11 19:38:06 2008 From: bcannon at gmail.com (Brett C.) Date: Sun, 11 May 2008 10:38:06 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: <56F4F6C2-7F99-42CB-879B-E58778DE6D98@gmail.com> -Brett [from his iPod touch] On 10-May-08, at 20:49, "Guido van Rossum" wrote: > Works for me. The other thing I always use from cgi is escape() -- > will that be available somewhere else too? > It can be. =) -Brett > On Sat, May 10, 2008 at 8:30 PM, Brett Cannon > wrote: >> I just realized that PEP 3108 was missing one proposal from the >> stdlib >> SIG (originally proposed by Facundo Batista) of copying the >> cgi.parse_qs() function over to the new urllib.parse module so that >> people no longer need to import the cgi module just for that one >> parsing function. Does anyone object if I update the PEP to include >> that change? > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) From bcannon at gmail.com Sun May 11 19:40:51 2008 From: bcannon at gmail.com (Brett C.) Date: Sun, 11 May 2008 10:40:51 -0700 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: <18470.27477.880117.113109@montanaro-dyndns-org.local> Message-ID: <8D7BA5DD-AF9E-4693-99D4-53B4E1D9F7C7@gmail.com> -Brett [from his iPod touch] On 10-May-08, at 23:46, "Alexandre Vassalotti" wrote: > On Sat, May 10, 2008 at 11:43 PM, wrote: >> >> Brett> There is going to be an issue with the current proposal for >> Brett> keeping around urllib. Since the package is to be named >> the same >> Brett> thing as the module.... >> >> Is this the only module morphing into a package of the same name? >> > > No, it is not. The dbm package will have the same issue. > Ah nuts, forgot about that one. -Brett > -- Alexandre From bcannon at gmail.com Sun May 11 19:42:37 2008 From: bcannon at gmail.com (Brett C.) Date: Sun, 11 May 2008 10:42:37 -0700 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: Message-ID: <76409034-70D2-4A16-8A08-161C7156C0AF@gmail.com> -Brett [from his iPod touch] On 10-May-08, at 23:58, "Alexandre Vassalotti" wrote: > On Sat, May 10, 2008 at 11:38 PM, Brett Cannon > wrote: >> I see three solutions for dealing with this. >> >> 1. Have stubs for the entire urllib API in urllib.__init__ that raise >> a DeprecationWarning either specifying the new name or saying the >> function/class is deprecated. >> >> 2. Rename urllib to urllib.fetch or urllib.old_request to get people >> to move over to urllib.request (aka urllib2) at some point. >> > > I am probably missing something, because I don't see how this solution > would solve the problem. The warning in urllib.__init__ will still be > issued when people will import urllib.fetch (or urllib.fetch). > No, you are probably right. My brain is mush at the moment. -brett > -- Alexandre From bcannon at gmail.com Sun May 11 19:50:01 2008 From: bcannon at gmail.com (Brett C.) Date: Sun, 11 May 2008 10:50:01 -0700 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: <1786A4A3-F89F-4BCA-B53E-CEA8F9473E99@acm.org> References: <1786A4A3-F89F-4BCA-B53E-CEA8F9473E99@acm.org> Message-ID: -Brett [from his iPod touch] On 11-May-08, at 0:04, Fred Drake wrote: > On Sat, May 10, 2008 at 11:38 PM, Brett Cannon > wrote: >> I see three solutions for dealing with this. >> >> 1. Have stubs for the entire urllib API in urllib.__init__ that raise >> a DeprecationWarning either specifying the new name or saying the >> function/class is deprecated. > > On May 11, 2008, at 2:58 AM, Alexandre Vassalotti wrote: >> I am probably missing something, because I don't see how this >> solution >> would solve the problem. The warning in urllib.__init__ will still be >> issued when people will import urllib.fetch (or urllib.fetch). > > > Were the warnings issued on import, yes, but I think Brett's > suggestion was really about issuing warnings when the stub functions > were called. For classes, I guess this could be handled using the > __init__ methods, but I'm not sure I like that, or that it will be > easy to get exactly the right behavior in all cases. > Another option is to not worry about warnings in this specific case and fully rely on 2to3. The issue of urllib and splitting its API across 2 files stands, though. That could be solved with a urllib._old module for now that eventually gets a deprecation warning. > > -Fred > > -- > Fred Drake > > > > From brett at python.org Sun May 11 21:37:37 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 12:37:37 -0700 Subject: [Python-Dev] Trickery with moving urllib In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 8:31 AM, Georg Brandl wrote: > Brett Cannon schrieb: >> >> There is going to be an issue with the current proposal for keeping >> around urllib. Since the package is to be named the same thing as the >> module, to handle the new name that means urllib.__init__ will need to >> gain the Py3K warning for the new name. But that doesn't quite work as >> the package's module will be imported automatically any time the >> module within the package is used. > > In this context, what's become of PEP 364? Is it still alive? Does it > cover this case? > PEP 364 was pre-2to3 (Barry and I came up with that solution at the same PyCon 2to3 was conceived). A 2to3 fixer that handles the renames does the same thing; so no, it doesn't solve the issue directly. -Brett From michele.simionato at gmail.com Fri May 2 08:18:57 2008 From: michele.simionato at gmail.com (michele.simionato at gmail.com) Date: Thu, 1 May 2008 23:18:57 -0700 (PDT) Subject: [Python-Dev] Problems with the new super() In-Reply-To: <481AAC86.3010308@canterbury.ac.nz> References: <481A7AA8.8010004@canterbury.ac.nz> <2CE5930E-A585-4D82-B70E-3F043B7EB151@northwestern.edu> <481AAC86.3010308@canterbury.ac.nz> Message-ID: <1eb743b7-fe15-4d08-8771-c873c433ec72@w7g2000hsa.googlegroups.com> On May 2, 7:54 am, Greg Ewing wrote: > > I can only speak from my own experience, which is that > whenever I've had a problem involving multiple inheritance, > super() didn't solve it. What did solve it was either > refactoring so that the classes being mixed were more > independent, or finding another solution that didn't > require multiple inheritance. > > Usually the new solution turned out to be better in > other ways as well, so I've come to regard multiple > inheritance issues as a code smell suggesting that > I need to rethink something. This is my experience as well. I have not found a real life problem yet that I could not solve with single inheritance + composition/delegation in a better and more maintainable way than using multiple inheritance. Also, I have come to believe that cooperative methods are a wart (it is too difficult to reason about them, they are fragile, and overall I see them as an unneeded complication in everyday coding). But we are stuck with multiple inheritance now and there is already a lot of code out there using it, so in the present situation we have to cope with it and to make it more usable, so I welcome the new super. I think that it should be made a keyword tough (it is too magic now not to be one). Michele Simionato From richard at lemurconsulting.com Fri May 2 14:49:42 2008 From: richard at lemurconsulting.com (Richard Boulton) Date: Fri, 02 May 2008 13:49:42 +0100 Subject: [Python-Dev] Reminder: last alphas next Wednesday 07-May-2008 In-Reply-To: <20080502092008.GW78165@nexus.in-nomine.org> References: <05760A6A-4F0F-4466-A640-062AE94E0470@python.org> <481A35D8.60604@cheimes.de> <20080502000324.25821.1160563467.divmod.xquotient.7178@joule.divmod.com> <20080502032549.25821.1219840827.divmod.xquotient.7352@joule.divmod.com> <481AD58D.2010201@holdenweb.com> <20080502092008.GW78165@nexus.in-nomine.org> Message-ID: <481B0DE6.30406@lemurconsulting.com> Jeroen Ruigrok van der Werven wrote: > -On [20080502 10:50], Steve Holden (steve at holdenweb.com) wrote: >> Groan. Then everyone else realizes what a "great idea" this is, and we see >> ~/Perl/, ~/Ruby/, ~/C# (that'll screw the Microsoft users, a directory with >> a comment market in its name), ~/Lisp/ and the rest? I don't think people >> would thank us for that in the long term. > > I'm +1 on just using $HOME/.local, but otherwise $HOME/.python makes sense > too. $HOME/.python.d doesn't do it for me, too clunky (and hardly used if I > look at my .files in $HOME). I'd vote for $HOME/.local if asked. I agree with most, if not all, of the arguments posted in favour of this, but would also like to point out the XDG-basedir specification: http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html This specification concerns where to put data files rather than libraries, but the relevance here is that the default location to put user-specific files according to it is in $HOME/.local/share Whilst by no means universally followed, uptake amongst Gnome and GTK applications seems to be rising - I certainly have a load of stuff in .local/share/ without ever having configured anything away from the Ubuntu install defaults. So, on Ubuntu computers at least, it seems likely that a $HOME/.local/ directory will already exist, with the beginnings of a unix style layout inside it. -- Richard From alexandre at peadrop.com Sun May 11 23:29:04 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 11 May 2008 17:29:04 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. Message-ID: Hello, I have been working the module renaming for PEP-3108, and I just noticed that some buildbots are throwing errors while updating their checkout. It seems the method I use for renaming modules hits a subversion bug on certain platforms. The error thrown looks like this: ... svn: In directory 'build/Lib/plat-mac' svn: Can't move source to dest svn: Can't move 'build/Lib/plat-mac/.svn/tmp/prop-base/pixmapwrapper.py.svn-base' to 'build/Lib/plat-mac/.svn/prop-base/pixmapwrapper.py.svn-base': No such file or directory program finished with exit code 1 (http://www.python.org/dev/buildbot/all/x86 osx.5 trunk/builds/201/step-svn/0) When I rename a module I use "svn copy", since "svn remove" doesn't pick up changes made to the "deleted" file. For example, here is what I did for PixMapWrapper: svn copy ./Lib/plat-mac/PixMapWrapper.py ./Lib/plat-mac/pixmapwrapper.py edit ./Lib/plat-mac/PixMapWrapper.py svn commit It seems that I could avoid this error by using "cp" instead of "svn copy" (which I did use for renaming copy_reg). However, I am not sure if this method doesn't preserve the full history of file. So, how should I do to fix the failing buildbots? -- Alexandre From p.f.moore at gmail.com Sun May 11 23:45:24 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 11 May 2008 22:45:24 +0100 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> References: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> Message-ID: <79990c6b0805111445i3fc47cd7j8b224cfd79d2a962@mail.gmail.com> Sorry, should have gone to the list: 2008/5/11 Alexandre Vassalotti : > When I rename a module I use "svn copy", since "svn remove" doesn't > pick up changes made to the "deleted" file. For example, here is what > I did for PixMapWrapper: > > svn copy ./Lib/plat-mac/PixMapWrapper.py ./Lib/plat-mac/pixmapwrapper.py > edit ./Lib/plat-mac/PixMapWrapper.py > svn commit That seems a very odd usage. You're renaming, not copying. Why aren't you using svn rename (svn move)? I can well imagine this causing serious confusion. Please be very careful here - if you introduce revisions which contain multiple files with names that differ only in case, you're going to really mess up history (and probably the only clean way to fix this will be to actually go back and edit the history). Paul (who has been bitten once too often with case clashes in history :-() From g.brandl at gmx.net Sun May 11 23:45:03 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 May 2008 23:45:03 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: Message-ID: Alexandre Vassalotti schrieb: > When I rename a module I use "svn copy", since "svn remove" doesn't > pick up changes made to the "deleted" file. For example, here is what > I did for PixMapWrapper: > > svn copy ./Lib/plat-mac/PixMapWrapper.py ./Lib/plat-mac/pixmapwrapper.py > edit ./Lib/plat-mac/PixMapWrapper.py > svn commit > > It seems that I could avoid this error by using "cp" instead of "svn > copy" (which I did use for renaming copy_reg). However, I am not sure > if this method doesn't preserve the full history of file. > > So, how should I do to fix the failing buildbots? In any case, how does SVN on Windows handle this situation, having two identical filenames -- except for case -- in one directory? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From phd at phd.pp.ru Mon May 12 00:05:56 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 12 May 2008 02:05:56 +0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: Message-ID: <20080511220556.GA19183@phd.pp.ru> On Sun, May 11, 2008 at 11:45:03PM +0200, Georg Brandl wrote: > In any case, how does SVN on Windows handle this situation, having two > identical filenames -- except for case -- in one directory? Very bad. It's completely confused. It can checkout one of the files and commit the other. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From alexandre at peadrop.com Mon May 12 00:16:14 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 11 May 2008 18:16:14 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> References: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> Message-ID: On Sun, May 11, 2008 at 5:44 PM, Paul Moore wrote: > 2008/5/11 Alexandre Vassalotti : >> When I rename a module I use "svn copy", since "svn remove" doesn't >> pick up changes made to the "deleted" file. For example, here is what >> I did for PixMapWrapper: >> >> svn copy ./Lib/plat-mac/PixMapWrapper.py ./Lib/plat-mac/pixmapwrapper.py >> edit ./Lib/plat-mac/PixMapWrapper.py >> svn commit > > That seems a very odd usage. You're renaming, not copying. Why aren't > you using svn rename (svn move)? I can well imagine this causing > serious confusion. > I wrote: > When I rename a module I use "svn copy", since "svn remove" doesn't > pick up changes made to the "deleted" file. For example, here is what > I did for PixMapWrapper: Oops, I meant "svn rename" when I said "svn remove". As I said, if I use "svn rename" I cannot make changes to the file being renamed. > Please be very careful here - if you introduce revisions which contain > multiple files with names that differ only in case, you're going to > really mess up history (and probably the only clean way to fix this > will be to actually go back and edit the history). Oh, you are right. I totally forgot about case-insensible filesystems. This is really going to make such case-change renamings nasty. -- Alexandre From g.brandl at gmx.net Mon May 12 00:20:51 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 12 May 2008 00:20:51 +0200 Subject: [Python-Dev] Adding start to enumerate() Message-ID: I believe the following is a common use-case for enumerate() (at least, I've used it quite some times): for lineno, line in enumerate(fileobject): ... For this, it would be nice to have a start parameter for enumerate(). The changes are minimal -- okay for 2.6? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From brett at python.org Mon May 12 00:31:10 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 15:31:10 -0700 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> Message-ID: On Sun, May 11, 2008 at 3:16 PM, Alexandre Vassalotti wrote: > On Sun, May 11, 2008 at 5:44 PM, Paul Moore wrote: >> 2008/5/11 Alexandre Vassalotti : >>> When I rename a module I use "svn copy", since "svn remove" doesn't >>> pick up changes made to the "deleted" file. For example, here is what >>> I did for PixMapWrapper: >>> >>> svn copy ./Lib/plat-mac/PixMapWrapper.py ./Lib/plat-mac/pixmapwrapper.py >>> edit ./Lib/plat-mac/PixMapWrapper.py >>> svn commit >> >> That seems a very odd usage. You're renaming, not copying. Why aren't >> you using svn rename (svn move)? I can well imagine this causing >> serious confusion. >> > > I wrote: >> When I rename a module I use "svn copy", since "svn remove" doesn't >> pick up changes made to the "deleted" file. For example, here is what >> I did for PixMapWrapper: > > Oops, I meant "svn rename" when I said "svn remove". As I said, if I > use "svn rename" I cannot make changes to the file being renamed. > > >> Please be very careful here - if you introduce revisions which contain >> multiple files with names that differ only in case, you're going to >> really mess up history (and probably the only clean way to fix this >> will be to actually go back and edit the history). > > Oh, you are right. I totally forgot about case-insensible filesystems. > This is really going to make such case-change renamings nasty. The PEP specifies the lib-old directory to hold the old case name so that the svn rename won't lead to two files in the same directory. I was hoping that creating the stub in lib-old would allow a simple ``svn rename`` for the original module on a case-sensitive file-system and the case-insensitive file-systems would just be able to deal with it. Is that just not going to work? Oh, and I am really sorry, Alexandre, but the PixMapWrapper rename should have been taken out of the PEP as the entire Mac directory is going away, so the rename is kind of pointless since the module is going to be deleted. -Brett From alexandre at peadrop.com Mon May 12 00:38:06 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 11 May 2008 18:38:06 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> Message-ID: On Sun, May 11, 2008 at 6:31 PM, Brett Cannon wrote: > The PEP specifies the lib-old directory to hold the old case name so > that the svn rename won't lead to two files in the same directory. I > was hoping that creating the stub in lib-old would allow a simple > ``svn rename`` for the original module on a case-sensitive file-system > and the case-insensitive file-systems would just be able to deal with > it. Is that just not going to work? > > Oh, and I am really sorry, Alexandre, but the PixMapWrapper rename > should have been taken out of the PEP as the entire Mac directory is > going away, so the rename is kind of pointless since the module is > going to be deleted. > Well, I guess I really messed up on that one. So, do you have any idea on how to revert the changes? -- Alexandre From brett at python.org Mon May 12 00:43:18 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 15:43:18 -0700 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> Message-ID: On Sun, May 11, 2008 at 3:38 PM, Alexandre Vassalotti wrote: > On Sun, May 11, 2008 at 6:31 PM, Brett Cannon wrote: >> The PEP specifies the lib-old directory to hold the old case name so >> that the svn rename won't lead to two files in the same directory. I >> was hoping that creating the stub in lib-old would allow a simple >> ``svn rename`` for the original module on a case-sensitive file-system >> and the case-insensitive file-systems would just be able to deal with >> it. Is that just not going to work? >> >> Oh, and I am really sorry, Alexandre, but the PixMapWrapper rename >> should have been taken out of the PEP as the entire Mac directory is >> going away, so the rename is kind of pointless since the module is >> going to be deleted. >> > > Well, I guess I really messed up on that one. Eh, how were you to know? It's something I sometimes forget on my Mac since it is case-preserving but case-insensitive. Hopefully the lib-old trick will solve the problem as PixMapWrapper was the only one that is in any form of a package and that module is no longer an issue. > So, do you have any idea > on how to revert the changes? See if having you move Queue.py into lib-old fixes things. Otherwise the patch will need to be reverted. I am on GTalk right now so you can IM me as soon as you have tried if you want and I can see if it fixes the problem on OS X. -Brett From brett at python.org Mon May 12 00:44:22 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 15:44:22 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 3:20 PM, Georg Brandl wrote: > I believe the following is a common use-case for enumerate() > (at least, I've used it quite some times): > > for lineno, line in enumerate(fileobject): > ... > > For this, it would be nice to have a start parameter for enumerate(). > The changes are minimal -- okay for 2.6? > Taking a new argument that has a default shouldn't be an issue. +1 from me. I assume it is just going to start the count at that number, not advance the iterable to that point, right? -Brett From g.brandl at gmx.net Mon May 12 00:50:38 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 12 May 2008 00:50:38 +0200 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: Message-ID: Brett Cannon schrieb: > On Sun, May 11, 2008 at 3:20 PM, Georg Brandl wrote: >> I believe the following is a common use-case for enumerate() >> (at least, I've used it quite some times): >> >> for lineno, line in enumerate(fileobject): >> ... >> >> For this, it would be nice to have a start parameter for enumerate(). >> The changes are minimal -- okay for 2.6? >> > > Taking a new argument that has a default shouldn't be an issue. +1 > from me. I assume it is just going to start the count at that number, > not advance the iterable to that point, right? Exactly. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From alexandre at peadrop.com Mon May 12 01:33:00 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 11 May 2008 19:33:00 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 5:29 PM, Alexandre Vassalotti wrote: > Hello, > > I have been working the module renaming for PEP-3108, and I just > noticed that some buildbots are throwing errors while updating their > checkout. It seems the method I use for renaming modules hits a > subversion bug on certain platforms. The error thrown looks like this: [SNIP] > So, how should I do to fix the failing buildbots? > I reverted the all problematic changes and the buildbots are green again. Thank you all for you support! -- Alexandre From steve at pearwood.info Mon May 12 01:42:45 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 12 May 2008 09:42:45 +1000 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: Message-ID: <200805120942.46259.steve@pearwood.info> On Mon, 12 May 2008 08:20:51 am Georg Brandl wrote: > I believe the following is a common use-case for enumerate() > (at least, I've used it quite some times): > > for lineno, line in enumerate(fileobject): > ... > > For this, it would be nice to have a start parameter for enumerate(). Why would it be nice? What would you use it for? The only thing I can think of is printing lines with line numbers, and starting those line numbers at one instead of zero. If that's the only use-case, should it require built-in support? -- Steven From guido at python.org Mon May 12 01:43:48 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 11 May 2008 16:43:48 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 12:00 AM, Georg Brandl wrote: > Fred Drake schrieb: >> >> On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >>> >>> Works for me. The other thing I always use from cgi is escape() -- >>> will that be available somewhere else too? >> >> >> xml.sax.saxutils.escape() would be an appropriate replacement, though the >> location is a little funky. More than a little IMO. :-( > Ideally it would be xml.escape(). Perhaps there are more of these generally- > useful XML thingies to put there? Not really -- I'd rather see the xml package as a pure container, with all its contents in submodules (or subpackages). Also, I don't see this exclusive to XML -- HTML isn't XML (yet). > Georg > > -- > Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. > Four shall be the number of spaces thou shalt indent, and the number of thy > indenting shall be four. Eight shalt thou not indent, nor either indent thou > two, excepting that thou then proceed to four. Tabs are right out. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon May 12 01:59:48 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 16:59:48 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805120942.46259.steve@pearwood.info> References: <200805120942.46259.steve@pearwood.info> Message-ID: On Sun, May 11, 2008 at 4:42 PM, Steven D'Aprano wrote: > On Mon, 12 May 2008 08:20:51 am Georg Brandl wrote: >> I believe the following is a common use-case for enumerate() >> (at least, I've used it quite some times): >> >> for lineno, line in enumerate(fileobject): >> ... >> >> For this, it would be nice to have a start parameter for enumerate(). > > Why would it be nice? What would you use it for? > > The only thing I can think of is printing lines with line numbers, and > starting those line numbers at one instead of zero. If that's the only > use-case, should it require built-in support? It's a common enough use-case, so I think it makes sense. With the cost being so minimal to add support I think this one use-case alone is enough to justify adding the support. -Brett From brett at python.org Mon May 12 02:01:10 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 17:01:10 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 4:43 PM, Guido van Rossum wrote: > On Sun, May 11, 2008 at 12:00 AM, Georg Brandl wrote: >> Fred Drake schrieb: >>> >>> On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >>>> >>>> Works for me. The other thing I always use from cgi is escape() -- >>>> will that be available somewhere else too? >>> >>> >>> xml.sax.saxutils.escape() would be an appropriate replacement, though the >>> location is a little funky. > > More than a little IMO. :-( > Well, if that function is better than who cares about the location; it will end up in urllib.parse as some function. -Brett From facundobatista at gmail.com Mon May 12 02:03:48 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Sun, 11 May 2008 21:03:48 -0300 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: <200805120942.46259.steve@pearwood.info> Message-ID: 2008/5/11, Brett Cannon : > It's a common enough use-case, so I think it makes sense. With the > cost being so minimal to add support I think this one use-case alone > is enough to justify adding the support. +1 -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From greg.ewing at canterbury.ac.nz Mon May 12 02:21:00 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 12 May 2008 12:21:00 +1200 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805120942.46259.steve@pearwood.info> References: <200805120942.46259.steve@pearwood.info> Message-ID: <48278D6C.2020205@canterbury.ac.nz> Steven D'Aprano wrote: > The only thing I can think of is printing lines with line numbers Parsing a file and wanting to be able to print error messages with line numbers would seem to be a fairly likely use. -- Greg From scott+python-dev at scottdial.com Mon May 12 03:23:01 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sun, 11 May 2008 21:23:01 -0400 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: Message-ID: <48279BF5.6010107@scottdial.com> Brett Cannon wrote: > Taking a new argument that has a default shouldn't be an issue. +1 > from me. I assume it is just going to start the count at that number, > not advance the iterable to that point, right? I wonder if it would be best for it to be a keyword-only argument. So many of the utility functions on iterables are foo(*iterables) that I might be inclined to think enumerate(foo, bar) is equivalent to enumerate(chain(foo, bar)), but enumerate(foo, start=bar) is pretty obvious. And if you consider that the enumeration is prepended to the values of foo, enumerate(foo, bar) is "backwards." Just saying.. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From guido at python.org Mon May 12 03:23:53 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 11 May 2008 18:23:53 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 5:01 PM, Brett Cannon wrote: > On Sun, May 11, 2008 at 4:43 PM, Guido van Rossum wrote: >> On Sun, May 11, 2008 at 12:00 AM, Georg Brandl wrote: >>> Fred Drake schrieb: >>>> >>>> On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >>>>> >>>>> Works for me. The other thing I always use from cgi is escape() -- >>>>> will that be available somewhere else too? >>>> >>>> >>>> xml.sax.saxutils.escape() would be an appropriate replacement, though the >>>> location is a little funky. >> >> More than a little IMO. :-( >> > > Well, if that function is better than who cares about the location; it > will end up in urllib.parse as some function. It's a trivial function; it shouldn't pull in three packages and lots of other cruft. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon May 12 03:24:58 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 11 May 2008 18:24:58 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <48279BF5.6010107@scottdial.com> References: <48279BF5.6010107@scottdial.com> Message-ID: +1 to this. On Sun, May 11, 2008 at 6:23 PM, Scott Dial wrote: > Brett Cannon wrote: >> >> Taking a new argument that has a default shouldn't be an issue. +1 >> from me. I assume it is just going to start the count at that number, >> not advance the iterable to that point, right? > > I wonder if it would be best for it to be a keyword-only argument. So > many of the utility functions on iterables are foo(*iterables) that I > might be inclined to think enumerate(foo, bar) is equivalent to > enumerate(chain(foo, bar)), but enumerate(foo, start=bar) is pretty > obvious. And if you consider that the enumeration is prepended to the > values of foo, enumerate(foo, bar) is "backwards." Just saying.. > > -Scott > > -- > Scott Dial > scott at scottdial.com > scodial at cs.indiana.edu > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon May 12 03:34:03 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 18:34:03 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 6:23 PM, Guido van Rossum wrote: > On Sun, May 11, 2008 at 5:01 PM, Brett Cannon wrote: >> On Sun, May 11, 2008 at 4:43 PM, Guido van Rossum wrote: >>> On Sun, May 11, 2008 at 12:00 AM, Georg Brandl wrote: >>>> Fred Drake schrieb: >>>>> >>>>> On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >>>>>> >>>>>> Works for me. The other thing I always use from cgi is escape() -- >>>>>> will that be available somewhere else too? >>>>> >>>>> >>>>> xml.sax.saxutils.escape() would be an appropriate replacement, though the >>>>> location is a little funky. >>> >>> More than a little IMO. :-( >>> >> >> Well, if that function is better than who cares about the location; it >> will end up in urllib.parse as some function. > > It's a trivial function; it shouldn't pull in three packages and lots > of other cruft. So are you saying that it isn't that much better in urllib.parse? That only cuts the package count down by one. -Brett From guido at python.org Mon May 12 03:35:31 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 11 May 2008 18:35:31 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 6:34 PM, Brett Cannon wrote: > On Sun, May 11, 2008 at 6:23 PM, Guido van Rossum wrote: >> On Sun, May 11, 2008 at 5:01 PM, Brett Cannon wrote: >>> On Sun, May 11, 2008 at 4:43 PM, Guido van Rossum wrote: >>>> On Sun, May 11, 2008 at 12:00 AM, Georg Brandl wrote: >>>>> Fred Drake schrieb: >>>>>> >>>>>> On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >>>>>>> >>>>>>> Works for me. The other thing I always use from cgi is escape() -- >>>>>>> will that be available somewhere else too? >>>>>> >>>>>> >>>>>> xml.sax.saxutils.escape() would be an appropriate replacement, though the >>>>>> location is a little funky. >>>> >>>> More than a little IMO. :-( >>>> >>> >>> Well, if that function is better than who cares about the location; it >>> will end up in urllib.parse as some function. >> >> It's a trivial function; it shouldn't pull in three packages and lots >> of other cruft. > > So are you saying that it isn't that much better in urllib.parse? That > only cuts the package count down by one. I didn't mean to say, but it does seem the wrong module -- escape() is for HTML, not for URLs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon May 12 03:35:45 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 18:35:45 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <48279BF5.6010107@scottdial.com> References: <48279BF5.6010107@scottdial.com> Message-ID: On Sun, May 11, 2008 at 6:23 PM, Scott Dial wrote: > Brett Cannon wrote: >> >> Taking a new argument that has a default shouldn't be an issue. +1 >> from me. I assume it is just going to start the count at that number, >> not advance the iterable to that point, right? > > I wonder if it would be best for it to be a keyword-only argument. So > many of the utility functions on iterables are foo(*iterables) that I > might be inclined to think enumerate(foo, bar) is equivalent to > enumerate(chain(foo, bar)), but enumerate(foo, start=bar) is pretty > obvious. And if you consider that the enumeration is prepended to the > values of foo, enumerate(foo, bar) is "backwards." Just saying.. > Sure, making it 'start' or something and having it be keyword-only makes sense. -Brett From brett at python.org Mon May 12 03:39:02 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 18:39:02 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 6:35 PM, Guido van Rossum wrote: > On Sun, May 11, 2008 at 6:34 PM, Brett Cannon wrote: >> On Sun, May 11, 2008 at 6:23 PM, Guido van Rossum wrote: >>> On Sun, May 11, 2008 at 5:01 PM, Brett Cannon wrote: >>>> On Sun, May 11, 2008 at 4:43 PM, Guido van Rossum wrote: >>>>> On Sun, May 11, 2008 at 12:00 AM, Georg Brandl wrote: >>>>>> Fred Drake schrieb: >>>>>>> >>>>>>> On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >>>>>>>> >>>>>>>> Works for me. The other thing I always use from cgi is escape() -- >>>>>>>> will that be available somewhere else too? >>>>>>> >>>>>>> >>>>>>> xml.sax.saxutils.escape() would be an appropriate replacement, > though the >>>>>>> location is a little funky. >>>>> >>>>> More than a little IMO. :-( >>>>> >>>> >>>> Well, if that function is better than who cares about the location; it >>>> will end up in urllib.parse as some function. >>> >>> It's a trivial function; it shouldn't pull in three packages and lots >>> of other cruft. >> >> So are you saying that it isn't that much better in urllib.parse? That >> only cuts the package count down by one. > > I didn't mean to say, but it does seem the wrong module -- escape() is > for HTML, not for URLs. OK. I will only worry about moving cgi.parse_qs() to urllib.parse and cgi.escape() to the html package somewhere (either 'html', 'html.parser' which is currently HTMLParser, or some new module; I prefer the first option). -Brett From brett at python.org Mon May 12 03:44:12 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 18:44:12 -0700 Subject: [Python-Dev] How to help out with PEP 3108 Message-ID: I have tried to update PEP 3108 (http://www.python.org/dev/peps/pep-3108/) as best as I can to list what needs to be done for each module in order to get it removed if there is some extra step. The ones without comment most likely need no special work and thus I can take care of really quickly myself (have it down to instinct at this point). But anything with special needs I would appreciate the help in dealing with. I have also tried to make the steps necessary to deal with the various modules as clear as possible in the PEP itself. Although I created issue 2775 to track stuff, it is probably wiser to create individual issues for the modules that require any form of work to deal with. Just make sure that if you do that you make it a dependency in issue 2775 (and if you can, list it in the PEP). I would like to not have to hold up the betas just because of this PEP, but I don't know if I can finish the PEP completely on my own. So help is appreciated. Thanks to anyone who can help (or who already have)! -Brett From phd at phd.pp.ru Mon May 12 03:50:24 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Mon, 12 May 2008 05:50:24 +0400 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: <20080512015024.GA26348@phd.pp.ru> On Sun, May 11, 2008 at 06:39:02PM -0700, Brett Cannon wrote: > cgi.escape() to the html package somewhere (either 'html', > 'html.parser' which is currently HTMLParser, or some new module; html.parser.escape() seems a bit backward; html.escape() fits the best! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Mon May 12 04:02:32 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 11 May 2008 19:02:32 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: <20080512015024.GA26348@phd.pp.ru> References: <20080512015024.GA26348@phd.pp.ru> Message-ID: On Sun, May 11, 2008 at 6:50 PM, Oleg Broytmann wrote: > On Sun, May 11, 2008 at 06:39:02PM -0700, Brett Cannon wrote: >> cgi.escape() to the html package somewhere (either 'html', >> 'html.parser' which is currently HTMLParser, or some new module; > > html.parser.escape() seems a bit backward; html.escape() fits the best! +1 -- --Guido van Rossum (home page: http://www.python.org/~guido/) From amk at amk.ca Mon May 12 04:02:32 2008 From: amk at amk.ca (A.M. Kuchling) Date: Sun, 11 May 2008 22:02:32 -0400 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <48278D6C.2020205@canterbury.ac.nz> References: <200805120942.46259.steve@pearwood.info> <48278D6C.2020205@canterbury.ac.nz> Message-ID: <20080512020232.GA12958@amk.local> On Mon, May 12, 2008 at 12:21:00PM +1200, Greg Ewing wrote: > Parsing a file and wanting to be able to print > error messages with line numbers would seem to > be a fairly likely use. Couldn't people be using the fileinput module for this, though? --amk From ggpolo at gmail.com Mon May 12 04:25:42 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Sun, 11 May 2008 23:25:42 -0300 Subject: [Python-Dev] How to help out with PEP 3108 In-Reply-To: References: Message-ID: 2008/5/11 Brett Cannon : > I have tried to update PEP 3108 > (http://www.python.org/dev/peps/pep-3108/) as best as I can to list > what needs to be done for each module in order to get it removed if > there is some extra step. The ones without comment most likely need no > special work and thus I can take care of really quickly myself (have > it down to instinct at this point). But anything with special needs I > would appreciate the help in dealing with. I have also tried to make > the steps necessary to deal with the various modules as clear as > possible in the PEP itself. > > Although I created issue 2775 to track stuff, it is probably wiser to > create individual issues for the modules that require any form of work > to deal with. Just make sure that if you do that you make it a > dependency in issue 2775 (and if you can, list it in the PEP). > > I would like to not have to hold up the betas just because of this > PEP, but I don't know if I can finish the PEP completely on my own. So > help is appreciated. Thanks to anyone who can help (or who already > have)! > > -Brett Hi Brett, I've sent a patch regarding the move of Tkinter to the tkinter package. But I'm in doubt about stub modules for this case, I'm not sure if using lib-old is good for Tkinter specifically, since lib-tk is included in sys.path. How should I proceed on this ? -- -- Guilherme H. Polo Goncalves From brett at python.org Mon May 12 04:34:31 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 19:34:31 -0700 Subject: [Python-Dev] How best to handle the docs for a renamed module? Message-ID: For the sake of argument, let's consider the Queue module. It is now named queue. For 2.6 I plan on having both Queue and queue listed in the index, with Queue deprecated with instructions to use the new name. But what to do about all the references. Should we leave them pointing at Queue to lessen confusion for people who read about some module on some other site that isn't using the new name, or update everything in 2.6 to use the new name? -Brett From guido at python.org Mon May 12 05:17:35 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 11 May 2008 20:17:35 -0700 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: References: Message-ID: On a sunny day, Brett Cannon wrote: > For the sake of argument, let's consider the Queue module. It is now > named queue. For 2.6 I plan on having both Queue and queue listed in > the index, with Queue deprecated with instructions to use the new > name. > > But what to do about all the references. Should we leave them pointing > at Queue to lessen confusion for people who read about some module on > some other site that isn't using the new name, or update everything in > 2.6 to use the new name? I'd say use the new names, so people can get used to them sooner -- otherwise all you do is postpone the confusion until they're going to use 3.0. People who've only head of Queue will be fine as long as searching for Queue in the index and TOC will find them a pointer to queue (and that one has a "versionadded 2.6 -- renamed from Queue" note). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Mon May 12 05:35:41 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 20:35:41 -0700 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: References: Message-ID: On Sun, May 11, 2008 at 8:17 PM, Guido van Rossum wrote: > On a sunny day, Brett Cannon wrote: >> For the sake of argument, let's consider the Queue module. It is now >> named queue. For 2.6 I plan on having both Queue and queue listed in >> the index, with Queue deprecated with instructions to use the new >> name. >> >> But what to do about all the references. Should we leave them pointing >> at Queue to lessen confusion for people who read about some module on >> some other site that isn't using the new name, or update everything in >> 2.6 to use the new name? > > I'd say use the new names, so people can get used to them sooner -- > otherwise all you do is postpone the confusion until they're going to > use 3.0. People who've only head of Queue will be fine as long as > searching for Queue in the index and TOC will find them a pointer to > queue (and that one has a "versionadded 2.6 -- renamed from Queue" > note). > OK, documented as such. -Brett From brett at python.org Mon May 12 05:42:44 2008 From: brett at python.org (Brett Cannon) Date: Sun, 11 May 2008 20:42:44 -0700 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: <20080512015024.GA26348@phd.pp.ru> Message-ID: On Sun, May 11, 2008 at 7:02 PM, Guido van Rossum wrote: > On Sun, May 11, 2008 at 6:50 PM, Oleg Broytmann wrote: >> On Sun, May 11, 2008 at 06:39:02PM -0700, Brett Cannon wrote: >>> cgi.escape() to the html package somewhere (either 'html', >>> 'html.parser' which is currently HTMLParser, or some new module; >> >> html.parser.escape() seems a bit backward; html.escape() fits the best! > > +1 Issue 2829 is a bug report for copying cgi.parse_qs() to urllib.parse and 2830 for copying cgi.escape() to html. -Brett From scott+python-dev at scottdial.com Mon May 12 05:56:44 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Sun, 11 May 2008 23:56:44 -0400 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: <48279BF5.6010107@scottdial.com> Message-ID: <4827BFFC.1060609@scottdial.com> Brett Cannon wrote: > Sure, making it 'start' or something and having it be keyword-only makes sense. http://bugs.python.org/issue2831 -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From martin at v.loewis.de Mon May 12 09:40:09 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 May 2008 09:40:09 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: Message-ID: <4827F459.4010907@v.loewis.de> > When I rename a module I use "svn copy", since "svn remove" doesn't > pick up changes made to the "deleted" file. For example, here is what > I did for PixMapWrapper: You want to make changes to the deleted file? Why? Regards, Martin From martin at v.loewis.de Mon May 12 09:49:24 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 May 2008 09:49:24 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> Message-ID: <4827F684.6070307@v.loewis.de> > Well, I guess I really messed up on that one. So, do you have any idea > on how to revert the changes? If the changes where in a single revision N, do svn merge -rN:N-1 . svn commit -m "revert rN" If they span over several subsequent revisions, use N-k instead. If they span over several revisions with intermediate revisions that you don't want to revert, try multiple merge commands before a single commit; if that fails, revert and commit each range of changes separately. Regards, Martin P.S. If you want to get the buildbots back in shape (in case they aren't), build a non-existing branch through the UI (which will cause a recursive removal of the entire checkout), then either wait for the next regular commit, or force a build of the respective branch (branches/py3k or trunk). On Windows, if there is still a python.exe process holding onto its binary, that fails, and we need support from the slave admin. From g.brandl at gmx.net Mon May 12 12:10:22 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 12 May 2008 12:10:22 +0200 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: References: Message-ID: Brett Cannon schrieb: > For the sake of argument, let's consider the Queue module. It is now > named queue. For 2.6 I plan on having both Queue and queue listed in > the index, with Queue deprecated with instructions to use the new > name. > > But what to do about all the references. Should we leave them pointing > at Queue to lessen confusion for people who read about some module on > some other site that isn't using the new name, or update everything in > 2.6 to use the new name? I've now updated docs for the Queue, SocketServer and copy_reg modules in the trunk. * File names (only important for copy_reg) and references use the new name. * There are two "module" directives in the file; one for the old, one for the new name. The synopsis of the old name is "Old name of the XXX module." * There is a note after the module directives that tells The :mod:`XXX` module has been renamed to :mod:`YYY` in Python 3.0. It is importable under both names in Python 2.6 and the rest of the 2.x series. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From p.f.moore at gmail.com Mon May 12 13:18:32 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 May 2008 12:18:32 +0100 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: Message-ID: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> 2008/5/12 Alexandre Vassalotti : > On Sun, May 11, 2008 at 5:29 PM, Alexandre Vassalotti > wrote: > > Hello, > > > > I have been working the module renaming for PEP-3108, and I just > > noticed that some buildbots are throwing errors while updating their > > checkout. It seems the method I use for renaming modules hits a > > subversion bug on certain platforms. The error thrown looks like this: > [SNIP] > > > So, how should I do to fix the failing buildbots? > > > > I reverted the all problematic changes and the buildbots are green again. > > Thank you all for you support! Revision 63129 is not valid on case folding filesystems. In particular, this horribly breaks using hg-svn to make a local mirror of the Python repository: >\Apps\HGsvn\hgimportsvn.exe -r 63120 http://svn.python.org/projects/python/trunk foo >cd foo >\apps\hgsvn\hgpullsvn >hg log Lib\socketserver.py changeset: 2:e8856fdf9300 branch: trunk tag: svn.63129 user: alexandre.vassalotti date: Mon May 12 02:37:10 2008 +0100 summary: [svn] Renamed SocketServer to 'socketserver'. >hg up -r2 abort: case-folding collision between Lib/socketserver.py and Lib/SocketServer.py >hg up -rtip abort: case-folding collision between Lib/socketserver.py and Lib/SocketServer.py The hg repository is now totally broken. I'm not sure there is a fix for this, short of going back and manually fixing the history (and I don't know if this is possible at all using Subversion). I know it's a rare situation, but can people PLEASE be careful not to introduce case issues like this - they pollute history for ever (I still hit problems with some of the case funkiness in svn from the early 1990's!!!) Bazaar seems to be OK (ish) with this - it picks just one version to show. Paul. From p.f.moore at gmail.com Mon May 12 13:23:18 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 May 2008 12:23:18 +0100 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> Message-ID: <79990c6b0805120423p44cdb7edmdd6b5de7e983a07c@mail.gmail.com> 2008/5/12 Paul Moore : > Bazaar seems to be OK (ish) with this - it picks just one version to show. It seems to be unable to find the history using either bzr log Lib\socketserver.py or bzr log Lib\SocketServer.py, though (but bzr log seems pretty unintuitive to me, so maybe I was using it wrongly...) Paul. From martin at v.loewis.de Mon May 12 14:26:19 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 May 2008 14:26:19 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> Message-ID: <4828376B.9090601@v.loewis.de> > Revision 63129 is not valid on case folding filesystems. In > particular, this horribly breaks using hg-svn to make a local mirror > of the Python repository: That would be a bug in hg-svn, right? Yes, the revision is not valid on case-folding systems - but why should that break hg-svn? The tool should be able to represent such stuff in its repository (whatever concept of repository hg may have); it should then only fail if you also want to check out that specific revision. There is no way to remove this revision from the repository now (other than a full dump|filter|load cycle, which I would rather want to avoid). > I know it's a rare situation, but can people PLEASE be careful not to > introduce case issues like this - they pollute history for ever (I > still hit problems with some of the case funkiness in svn from the > early 1990's!!!) Subversion's first release was in October 2000; it wasn't self-hosting until 2001 :-) Regards, Martin From ijmorlan at cs.uwaterloo.ca Mon May 12 15:03:05 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Mon, 12 May 2008 09:03:05 -0400 (EDT) Subject: [Python-Dev] On quote styles In-Reply-To: <48263DD0.90702@canterbury.ac.nz> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> <48263DD0.90702@canterbury.ac.nz> Message-ID: On Sun, 11 May 2008, Greg Ewing wrote: >> While Python doesn't have a char type (yet), I still find the distinction >> between 'c' and "abc" useful to show intent (especially given my C >> background > > The way I tend to use them is that "xxx" is for data > operated on by the program and seen by the user, > and 'xxx' is for things that are only used internally, > e.g. enumerated type values and attribute names for use > by getattr(). I find myself doing something similar. I use '' for anything that is a dictionary key and "" for things that are strings not meant to be used for indexing, in particular the things that are looked up. Except when I don't. When I am generating HTML using the ll.xist library, which uses constructs like this: html.a (u'link text', href="http://url.goes.here/") I have just started using '' for text element contents and "" for attribute values. This way the attribute values (keyword parameters) look almost the exact same as in actual HTML (only difference is the commas) (and the escaping), and my editor seems to be designed for C-style languages where '' are for character constants and "" are for strings so it highlights them differently. My personal opinion is that one should not use the different quoting styles at random, but I am inclined to believe that there is no single guideline that can apply to everybody. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From alexandre at peadrop.com Mon May 12 15:16:00 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 12 May 2008 09:16:00 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <4827F459.4010907@v.loewis.de> References: <4827F459.4010907@v.loewis.de> Message-ID: On Mon, May 12, 2008 at 3:40 AM, "Martin v. L?wis" wrote: > > When I rename a module I use "svn copy", since "svn remove" doesn't > > pick up changes made to the "deleted" file. For example, here is what > > I did for PixMapWrapper: > > You want to make changes to the deleted file? Why? > The idea was to replace the orignial module file with its stub. However, the "svn copy" and edit process isn't the cause of the problems. It is the fact that 2 files existed in the same directory differing only by a case-change. Anyway, all the buildbot seems okay now. -- Alexandre From alexandre at peadrop.com Mon May 12 15:18:46 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 12 May 2008 09:18:46 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <4827F684.6070307@v.loewis.de> References: <79990c6b0805111444ldc9a549o845f027fbdb91635@mail.gmail.com> <4827F684.6070307@v.loewis.de> Message-ID: On Mon, May 12, 2008 at 3:49 AM, "Martin v. L?wis" wrote: > > Well, I guess I really messed up on that one. So, do you have any idea > > on how to revert the changes? > > If the changes where in a single revision N, do > > svn merge -rN:N-1 . > svn commit -m "revert rN" > > If they span over several subsequent revisions, use N-k > instead. If they span over several revisions with intermediate > revisions that you don't want to revert, try multiple merge > commands before a single commit; if that fails, revert and commit > each range of changes separately. > Yes. That is exactly what I did to revert the changes. > > P.S. If you want to get the buildbots back in shape (in case they > aren't), build a non-existing branch through the UI > (which will cause a recursive removal of the entire checkout), then > either wait for the next regular commit, or force a build of the > respective branch (branches/py3k or trunk). On Windows, if there > is still a python.exe process holding onto its binary, that fails, > and we need support from the slave admin. > Thanks for the tip. Now, I just hope that I will never have to use it. ;-) -- Alexandre From alexandre at peadrop.com Mon May 12 15:21:34 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 12 May 2008 09:21:34 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> Message-ID: On Mon, May 12, 2008 at 7:18 AM, Paul Moore wrote: > Revision 63129 is not valid on case folding filesystems. In > particular, this horribly breaks using hg-svn to make a local mirror > of the Python repository: > > >\Apps\HGsvn\hgimportsvn.exe -r 63120 > http://svn.python.org/projects/python/trunk foo > >cd foo > >\apps\hgsvn\hgpullsvn > >hg log Lib\socketserver.py > changeset: 2:e8856fdf9300 > branch: trunk > tag: svn.63129 > user: alexandre.vassalotti > date: Mon May 12 02:37:10 2008 +0100 > summary: [svn] Renamed SocketServer to 'socketserver'. > > >hg up -r2 > abort: case-folding collision between Lib/socketserver.py and > Lib/SocketServer.py > > >hg up -rtip > abort: case-folding collision between Lib/socketserver.py and > Lib/SocketServer.py > > The hg repository is now totally broken. > Which version of mercurial are you using? I know that versions prior 1.0 had some bug with handling case-changes on case-insensitive filesystems. -- Alexandre From martin at v.loewis.de Mon May 12 15:24:18 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 12 May 2008 15:24:18 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: <4827F459.4010907@v.loewis.de> Message-ID: <48284502.3000204@v.loewis.de> > The idea was to replace the orignial module file with its stub. > However, the "svn copy" and edit process isn't the cause of the > problems. It is the fact that 2 files existed in the same directory > differing only by a case-change. I still don't understand. You wanted to replace the file with a stub, and then delete it? Why not just delete it (or use svn mv in the first place)? Regards, Martin From alexandre at peadrop.com Mon May 12 15:47:17 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 12 May 2008 09:47:17 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <48284502.3000204@v.loewis.de> References: <4827F459.4010907@v.loewis.de> <48284502.3000204@v.loewis.de> Message-ID: On Mon, May 12, 2008 at 9:24 AM, "Martin v. L?wis" wrote: > > The idea was to replace the orignial module file with its stub. > > However, the "svn copy" and edit process isn't the cause of the > > problems. It is the fact that 2 files existed in the same directory > > differing only by a case-change. > > I still don't understand. You wanted to replace the file with a stub, > and then delete it? Why not just delete it (or use svn mv in the first > place)? No. That is exactly what I wanted to avoid by using "svn copy", instead of "svn move". "svn move" mark the original file for removal. which makes it impossible to modify the original file on the same commit. Anyway, Brett updated the PEP with renaming procedure that avoids this problem completely. -- Alexandre From p.f.moore at gmail.com Mon May 12 16:08:14 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 May 2008 15:08:14 +0100 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <4828376B.9090601@v.loewis.de> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> <4828376B.9090601@v.loewis.de> Message-ID: <79990c6b0805120708q49371aecte5f351823bc6f908@mail.gmail.com> 2008/5/12 "Martin v. L?wis" : > > Revision 63129 is not valid on case folding filesystems. In > > particular, this horribly breaks using hg-svn to make a local mirror > > of the Python repository: > > That would be a bug in hg-svn, right? Yes, the revision is not valid > on case-folding systems - but why should that break hg-svn? The tool > should be able to represent such stuff in its repository (whatever > concept of repository hg may have); it should then only fail if you also > want to check out that specific revision. Absolutely it's a bug in hgsvn (which is a converter for svn -> hg, this isn't a bug in hg), but it's a design issue (hgsvn works by checking out each revision in turn and checking it into hg) triggered by having a svn revision which cannot be represented on a case-folsing system. > There is no way to remove this revision from the repository now > (other than a full dump|filter|load cycle, which I would rather want to > avoid). OK, there's no way this is justified for such an obscure issue - all I'm really asking is that people remember case-folding systems when making changes like this. > > I know it's a rare situation, but can people PLEASE be careful not to > > introduce case issues like this - they pollute history for ever (I > > still hit problems with some of the case funkiness in svn from the > > early 1990's!!!) > > Subversion's first release was in October 2000; it wasn't self-hosting > until 2001 :-) I assume it's pre-svn history, converted from CVS. Or I'm misreading something. Whatever, it's from a long time ago :-) Paul. From p.f.moore at gmail.com Mon May 12 16:11:58 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 May 2008 15:11:58 +0100 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> Message-ID: <79990c6b0805120711t230a935eme18c5290479f2ffd@mail.gmail.com> 2008/5/12 Alexandre Vassalotti : > Which version of mercurial are you using? I know that versions prior > 1.0 had some bug with handling case-changes on case-insensitive > filesystems. 1.0. There are still case folding bugs in 1.0 - I'm working on fixing them. But there will never be a complete fix for this situation, as it's simply not possible to checkout the exact svn layout of that revision on a case-folding system, it simply can't be supported (without name mangling of files in the working directory). As I said, it's no big deal, just something people working on case-sensitive systems need to watch out for when interoperation with case-insensitive systems is important. Paul. From p.f.moore at gmail.com Mon May 12 16:13:38 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 May 2008 15:13:38 +0100 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805120708q49371aecte5f351823bc6f908@mail.gmail.com> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> <4828376B.9090601@v.loewis.de> <79990c6b0805120708q49371aecte5f351823bc6f908@mail.gmail.com> Message-ID: <79990c6b0805120713s15c13e8ci36dbf4ca8baf3194@mail.gmail.com> 2008/5/12 Paul Moore : > 2008/5/12 "Martin v. L?wis" : > > Subversion's first release was in October 2000; it wasn't self-hosting > > until 2001 :-) > > I assume it's pre-svn history, converted from CVS. Or I'm misreading > something. Whatever, it's from a long time ago :-) Hmm, Guido's been at the time machine again :-) >svn log http://svn.python.org/projects/python/trunk | tail ------------------------------------------------------------------------ r2161 | guido | 1990-08-09 15:25:15 +0100 (Thu, 09 Aug 1990) | 2 lines Initial revision ------------------------------------------------------------------------ r2160 | (no author) | 1990-08-09 15:25:15 +0100 (Thu, 09 Aug 1990) | 1 line New repository initialized by cvs2svn. ------------------------------------------------------------------------ Paul. From martin at v.loewis.de Mon May 12 16:18:33 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 May 2008 16:18:33 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805120713s15c13e8ci36dbf4ca8baf3194@mail.gmail.com> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> <4828376B.9090601@v.loewis.de> <79990c6b0805120708q49371aecte5f351823bc6f908@mail.gmail.com> <79990c6b0805120713s15c13e8ci36dbf4ca8baf3194@mail.gmail.com> Message-ID: <482851B9.1070304@v.loewis.de> > Hmm, Guido's been at the time machine again :-) > >> svn log http://svn.python.org/projects/python/trunk | tail > ------------------------------------------------------------------------ > r2161 | guido | 1990-08-09 15:25:15 +0100 (Thu, 09 Aug 1990) | 2 lines > > Initial revision > > ------------------------------------------------------------------------ > r2160 | (no author) | 1990-08-09 15:25:15 +0100 (Thu, 09 Aug 1990) | 1 line > > New repository initialized by cvs2svn. > ------------------------------------------------------------------------ Actually, I will borrow the machine five years from now to create these files, after we switch back from hg to svn :-) svn log -r2158 http://svn.python.org/projects ------------------------------------------------------------------------ r2158 | bwarsaw | 2005-10-24 00:39:17 +0200 (Mo, 24 Okt 2005) | 3 lines PEP 351, the freeze protocol. ------------------------------------------------------------------------ Regards, Martin From martin at v.loewis.de Mon May 12 16:28:17 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 May 2008 16:28:17 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805120711t230a935eme18c5290479f2ffd@mail.gmail.com> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> <79990c6b0805120711t230a935eme18c5290479f2ffd@mail.gmail.com> Message-ID: <48285401.9080308@v.loewis.de> > 1.0. There are still case folding bugs in 1.0 - I'm working on fixing > them. But there will never be a complete fix for this situation, as > it's simply not possible to checkout the exact svn layout of that > revision on a case-folding system, it simply can't be supported > (without name mangling of files in the working directory). Assuming this is NTFS, notice that it is not the file system that is case-folding, but the API. If you pass FILE_FLAGS_POSIX_SEMANTICS to the API, you can get case-sensitive behavior (to some degree, as there are bugs in that API, as well). Regards, Martin From p.f.moore at gmail.com Mon May 12 16:30:04 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 12 May 2008 15:30:04 +0100 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <48285401.9080308@v.loewis.de> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> <79990c6b0805120711t230a935eme18c5290479f2ffd@mail.gmail.com> <48285401.9080308@v.loewis.de> Message-ID: <79990c6b0805120730v799d680i4bd4062b861f1dc6@mail.gmail.com> 2008/5/12 "Martin v. L?wis" : > > 1.0. There are still case folding bugs in 1.0 - I'm working on fixing > > them. But there will never be a complete fix for this situation, as > > it's simply not possible to checkout the exact svn layout of that > > revision on a case-folding system, it simply can't be supported > > (without name mangling of files in the working directory). > > Assuming this is NTFS, notice that it is not the file system that is > case-folding, but the API. If you pass FILE_FLAGS_POSIX_SEMANTICS to > the API, you can get case-sensitive behavior (to some degree, as there > are bugs in that API, as well). Hmm, that's interesting information. I suspect I could easily create more bugs than I solve by using it, but it's interesting to know nevertheless. Thanks, Paul. From tonynelson at georgeanelson.com Mon May 12 16:36:35 2008 From: tonynelson at georgeanelson.com (Tony Nelson) Date: Mon, 12 May 2008 10:36:35 -0400 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: At 11:56 PM -0400 5/10/08, Fred Drake wrote: >On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >> Works for me. The other thing I always use from cgi is escape() -- >> will that be available somewhere else too? > > >xml.sax.saxutils.escape() would be an appropriate replacement, though >the location is a little funky. At least it's right next to the valuable quoteattr(). -- ____________________________________________________________________ TonyN.:' ' From martin at v.loewis.de Mon May 12 16:43:54 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 12 May 2008 16:43:54 +0200 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: <79990c6b0805120730v799d680i4bd4062b861f1dc6@mail.gmail.com> References: <79990c6b0805120418m713bdc9bxa3457c6ec31b34d7@mail.gmail.com> <79990c6b0805120711t230a935eme18c5290479f2ffd@mail.gmail.com> <48285401.9080308@v.loewis.de> <79990c6b0805120730v799d680i4bd4062b861f1dc6@mail.gmail.com> Message-ID: <482857AA.9080004@v.loewis.de> > Hmm, that's interesting information. I suspect I could easily create > more bugs than I solve by using it, but it's interesting to know > nevertheless. I've been trying to use for a few applications. The biggest short-coming is that it won't work for all cases. In particular, for the following operations I could not find a way to make them case-sensitive (as they don't accept any flags parameter): DeleteFile (apparently possible through NtDeleteFile), CreateDirectory, RemoveDirectory, FindFirstFile (and probably others as well). Regards, Martin From barry at python.org Mon May 12 16:44:52 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 12 May 2008 10:44:52 -0400 Subject: [Python-Dev] Buildbots have trouble checking out the repository due to recent changes. In-Reply-To: References: <4827F459.4010907@v.loewis.de> <48284502.3000204@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 12, 2008, at 9:47 AM, Alexandre Vassalotti wrote: > On Mon, May 12, 2008 at 9:24 AM, "Martin v. L?wis" > wrote: >>> The idea was to replace the orignial module file with its stub. >>> However, the "svn copy" and edit process isn't the cause of the >>> problems. It is the fact that 2 files existed in the same directory >>> differing only by a case-change. >> >> I still don't understand. You wanted to replace the file with a stub, >> and then delete it? Why not just delete it (or use svn mv in the >> first >> place)? > > No. That is exactly what I wanted to avoid by using "svn copy", > instead of "svn move". "svn move" mark the original file for removal. > which makes it impossible to modify the original file on the same > commit. Anyway, Brett updated the PEP with renaming procedure that > avoids this problem completely. bzr to the rescue . - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSChX5nEjvBPtnXfVAQK2NwQAgA3Gaa7Ew6uAVrqJe3B9wj/lseMuS88j y4/gtlCcb35Vtff9CpEveCD44IelaB+OFH2iG6TKXxxD47HhYfJD3STRWnphimb8 j4UXJyfJTlMIW7ISWGzv62c+RlyXle8P0VXwdfDvem8h3o8cWDBBACvlqA/6wjb0 FFQuLHedW+8= =r0hs -----END PGP SIGNATURE----- From ijmorlan at cs.uwaterloo.ca Mon May 12 16:48:04 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Mon, 12 May 2008 10:48:04 -0400 (EDT) Subject: [Python-Dev] Problems with the new super() In-Reply-To: <481A776E.9080908@canterbury.ac.nz> References: <481A776E.9080908@canterbury.ac.nz> Message-ID: On Fri, 2 May 2008, Greg Ewing wrote: > Guido van Rossum wrote: >> The >> alternative would be to make it a keyword, which seemed excessive >> (plus, it would be odd if super() were a keyword when self is not). > > If it's really such a useful thing as to warrant so much > magic to support it, then I think it deserves to have a > keyword. > > Conversely, I would say that if it doesn't deserve a > keyword, it also doesn't deserve that much magic. One might even go further and say that if it's that magic, then it's a keyword by definition. Certainly I would want a syntax-highlighting editor to highlight it somehow, and if the editor has a "rename" feature to rename a variable and replace all references to it (but not other uses of the same identifiers in different scope), then it probably should at least alert the programmer before allowing a rename to or from the magic name. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From alexandre at peadrop.com Mon May 12 19:40:12 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 12 May 2008 13:40:12 -0400 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: References: Message-ID: On Mon, May 12, 2008 at 6:10 AM, Georg Brandl wrote: > I've now updated docs for the Queue, SocketServer and copy_reg modules in > the trunk. > Thank you, Georg, for updating docs! -- Alexandre From steve at holdenweb.com Mon May 12 19:42:49 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 12 May 2008 13:42:49 -0400 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805120942.46259.steve@pearwood.info> References: <200805120942.46259.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > On Mon, 12 May 2008 08:20:51 am Georg Brandl wrote: >> I believe the following is a common use-case for enumerate() >> (at least, I've used it quite some times): >> >> for lineno, line in enumerate(fileobject): >> ... >> >> For this, it would be nice to have a start parameter for enumerate(). > > Why would it be nice? What would you use it for? > > The only thing I can think of is printing lines with line numbers, and > starting those line numbers at one instead of zero. If that's the only > use-case, should it require built-in support? > It's fairly common in financial applications to number checks in batches, from a starting point that depends on the number of checks issued in previous runs. Having a start point would allow this to be done more simply, though it's not anyway what I would call an onerous task. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From thomaspinckney3 at gmail.com Mon May 12 22:58:47 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Mon, 12 May 2008 16:58:47 -0400 Subject: [Python-Dev] Copying cgi.parse_qs() to the urllib.parse module In-Reply-To: References: Message-ID: Is there any thought to extending escape to escape / unescape to by default handle characters other than <, >, and &? At a minimum it should handle arbitrary &xxx; values. Ideally, it would also handle common other symbolic names besides < > etc. HTML from common web sites such as nytimes.com frequently has a variety of characters escaped. Consider the page at http://travel.nytimes.com/travel/guides/europe/france/provence-and-the-french-riviera/overview.html It lists its content type as: content="text/html; charset=UTF-8" And contains text like: There’s the Côte d’ Ideally, we would decode ’ into ? and ô into ?. Unfortunately, #146 is really an error -- it's not a utf-8 encoded unicode character but really a MS codepage 1252 character for apostrophe (apparently may HTML editing systems intermingle unicode and codepage 1252 content for apostrophes and a few other common characters). I'm happy to contribute some additional code for these other cases if people agree it's useful. On May 12, 2008, at 10:36 AM, Tony Nelson wrote: > At 11:56 PM -0400 5/10/08, Fred Drake wrote: >> On May 10, 2008, at 11:49 PM, Guido van Rossum wrote: >>> Works for me. The other thing I always use from cgi is escape() -- >>> will that be available somewhere else too? >> >> >> xml.sax.saxutils.escape() would be an appropriate replacement, though >> the location is a little funky. > > At least it's right next to the valuable quoteattr(). > -- > ____________________________________________________________________ > TonyN.:' > ' > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/thomaspinckney3%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ggpolo at gmail.com Mon May 12 22:58:48 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Mon, 12 May 2008 17:58:48 -0300 Subject: [Python-Dev] lib2to3, need some light on the imports fixer Message-ID: Hello, Would someone tell me how can I add a new entry in the MAPPING dict in the lib2to3/fixes/fix_imports.py that does the following: "import A" gets fixed as "import C.D as A" Right now it is fixing by doing "import C.D" and changing several other lines in the code to use this new "C.D" name. I wanted to avoid these changes if possible. Regards, -- -- Guilherme H. Polo Goncalves From collinw at gmail.com Mon May 12 23:13:56 2008 From: collinw at gmail.com (Collin Winter) Date: Mon, 12 May 2008 14:13:56 -0700 Subject: [Python-Dev] lib2to3, need some light on the imports fixer In-Reply-To: References: Message-ID: <43aa6ff70805121413vb42a783mabe4d7764704b5a0@mail.gmail.com> On Mon, May 12, 2008 at 1:58 PM, Guilherme Polo wrote: > Hello, > > Would someone tell me how can I add a new entry in the MAPPING dict in > the lib2to3/fixes/fix_imports.py that does the following: > > "import A" gets fixed as "import C.D as A" > > Right now it is fixing by doing "import C.D" and changing several > other lines in the code to use this new "C.D" name. I wanted to avoid > these changes if possible. I don't believe there's a way to do that, but adding support for it should be fairly straight-forward. Assign the patch to me for review. Collin Winter From benji at zope.com Mon May 12 23:16:30 2008 From: benji at zope.com (Benji York) Date: Mon, 12 May 2008 17:16:30 -0400 Subject: [Python-Dev] On quote styles In-Reply-To: <18470.24379.202238.513904@montanaro-dyndns-org.local> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> <48263DD0.90702@canterbury.ac.nz> <18470.24379.202238.513904@montanaro-dyndns-org.local> Message-ID: On Sat, May 10, 2008 at 10:51 PM, wrote: > It might be useful to have a wiki page which identified some of the > conventions people use. One such entry could be "Do what python does.": >>> 'I am a string.' 'I am a string.' >>> "I'm a string" "I'm a string" -- Benji York From barry at python.org Mon May 12 23:28:58 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 12 May 2008 17:28:58 -0400 Subject: [Python-Dev] On quote styles In-Reply-To: References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> <48263DD0.90702@canterbury.ac.nz> <18470.24379.202238.513904@montanaro-dyndns-org.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 12, 2008, at 5:16 PM, Benji York wrote: > On Sat, May 10, 2008 at 10:51 PM, wrote: > >> It might be useful to have a wiki page which identified some of the >> conventions people use. > > One such entry could be "Do what python does.": > >>>> 'I am a string.' > 'I am a string.' >>>> "I'm a string" > "I'm a string" Nice. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSCi2mnEjvBPtnXfVAQJWHAP/b/2Yebt7abj+UuysjYFgzetl+26SFSFo jXq/JZUnieIhaiCwMkJW8BW8QmxtkCn3EPDf7VUgMhxMmxDLMQ5Ougzk+/Mq5JLs VM5zJ7kFRU4kBKReGumobSxOMUnteWNYCeUcpUDRHzEEGWFaRM0luNWUiN09iWin DYDvVS5nSCw= =/vGr -----END PGP SIGNATURE----- From ncoghlan at gmail.com Tue May 13 00:32:59 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 May 2008 08:32:59 +1000 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: Message-ID: <4828C59B.8070008@gmail.com> Georg Brandl wrote: > I believe the following is a common use-case for enumerate() > (at least, I've used it quite some times): > > for lineno, line in enumerate(fileobject): > ... > > For this, it would be nice to have a start parameter for enumerate(). > The changes are minimal -- okay for 2.6? I'd like to hear from Raymond before we do this. I'm pretty sure we had a reason for *not* doing it that way in when enumerate() was added, but I can't remember what that reason might have been... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tjreedy at udel.edu Tue May 13 01:52:23 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 May 2008 19:52:23 -0400 Subject: [Python-Dev] Adding start to enumerate() References: <4828C59B.8070008@gmail.com> Message-ID: "Nick Coghlan" wrote in message news:4828C59B.8070008 at gmail.com... | I'd like to hear from Raymond before we do this. I'm pretty sure we had | a reason for *not* doing it that way in when enumerate() was added, but | I can't remember what that reason might have been... http://bugs.python.org/issue2831 From skip at pobox.com Tue May 13 02:00:30 2008 From: skip at pobox.com (skip at pobox.com) Date: Mon, 12 May 2008 19:00:30 -0500 Subject: [Python-Dev] On quote styles In-Reply-To: References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> <48263DD0.90702@canterbury.ac.nz> <18470.24379.202238.513904@montanaro-dyndns-org.local> Message-ID: <18472.55838.305222.771661@montanaro-dyndns-org.local> >>>>> "Benji" == Benji York writes: Benji> One such entry could be "Do what python does.": >>> 'I am a string.' 'I am a string.' >>> "I'm a string" "I'm a string" That would be the principle of fewest backslashes. ;-) Skip From borowitz at cs.stanford.edu Tue May 13 02:26:53 2008 From: borowitz at cs.stanford.edu (David Borowitz) Date: Mon, 12 May 2008 17:26:53 -0700 Subject: [Python-Dev] On quote styles In-Reply-To: <18472.55838.305222.771661@montanaro-dyndns-org.local> References: <20080509005554.97A9E1E400C@bag.python.org> <4823AD7D.70309@gmail.com> <18469.45401.859822.563618@montanaro-dyndns-org.local> <48263DD0.90702@canterbury.ac.nz> <18470.24379.202238.513904@montanaro-dyndns-org.local> <18472.55838.305222.771661@montanaro-dyndns-org.local> Message-ID: <70e6cf130805121726l7d6f035cqfb7a7b2dc4e9e011@mail.gmail.com> Python itself doesn't appear to follow that principle: >>> "Ain't nothin' stoppin' this from usin' \"double quotes\"." 'Ain\'t nothin\' stoppin\' this from usin\' "double quotes".' IMHO it's a useful rule of thumb, but like most of the other alternatives presented in this thread, taken to extremes, it can get silly. (Imagine a string with 10 's and 9 "s, and adding two more "s to it...) But I suppose that's another reason these are just guidelines. Dave Borowitz On Mon, May 12, 2008 at 5:00 PM, wrote: > >>>>> "Benji" == Benji York writes: > > Benji> One such entry could be "Do what python does.": > > >>> 'I am a string.' > 'I am a string.' > >>> "I'm a string" > "I'm a string" > > That would be the principle of fewest backslashes. ;-) > > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ddborowitz%40gmail.com > -- It is better to be quotable than to be honest. -Tom Stoppard Borowitz -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue May 13 11:40:57 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 May 2008 19:40:57 +1000 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: <4828C59B.8070008@gmail.com> Message-ID: <48296229.9000800@gmail.com> Terry Reedy wrote: > "Nick Coghlan" wrote in message > news:4828C59B.8070008 at gmail.com... > | I'd like to hear from Raymond before we do this. I'm pretty sure we had > | a reason for *not* doing it that way in when enumerate() was added, but > | I can't remember what that reason might have been... > > http://bugs.python.org/issue2831 Thanks. I think this part is the main reason I see a start argument to enumerate as potentially problematic: """all variants can easily be misread as starting at the nth item in the sequence (much like islice() does now): enumerate(3, 'abcdefg') --> (3,'d') (4,'e') (5, 'f') (6, 'g').""" Is the need to use zip(count(3), seq) for the offset index case really such a burden given the associated benefits in keeping the builtin function really simple and easy to understand? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Tue May 13 16:25:01 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 13 May 2008 07:25:01 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <48296229.9000800@gmail.com> References: <4828C59B.8070008@gmail.com> <48296229.9000800@gmail.com> Message-ID: On Tue, May 13, 2008 at 2:40 AM, Nick Coghlan wrote: > Terry Reedy wrote: >> >> "Nick Coghlan" wrote in message >> news:4828C59B.8070008 at gmail.com... >> | I'd like to hear from Raymond before we do this. I'm pretty sure we had >> | a reason for *not* doing it that way in when enumerate() was added, but >> | I can't remember what that reason might have been... >> >> http://bugs.python.org/issue2831 > > Thanks. I think this part is the main reason I see a start argument to > enumerate as potentially problematic: > > """all variants can easily be misread as starting at the nth item in the > sequence (much like islice() does now): enumerate(3, 'abcdefg') --> > (3,'d') (4,'e') (5, 'f') (6, 'g').""" So the ambiguity is that enumerate(it, start=N) could be taken as skipping the first N items of it rather than adding N to the index it returns. (And it is my own argument!) I'd like to withdraw this argument. There are two separate use cases for using enumerate(): one is to iterate over a sequence and to have a handy index by which to update the value in the sequence. Another is for 1-based counting, usually when printing 1-based ordinals (such as line numbers in files, dates in a month or months in a year, etc.). N-based counting is less common but still conceivable. However I see no use for skipping items from the start, and if that use case ever came up, passing a slice to enumerate() would be the appropriate thing to do. In fact, if you passed in a slice, you might also want to pass a corresponding start value so the indices produced match those of the original sequence. So, I am still in favor of adding a new argument to enumerate(). I'm neutral on the need for a keyword (don't think it would hurt, not sure how much it matters). I'm strongly against making it an optional *leading* argument like Raymond proposed; that's a style I just don't want to promote, range() and the curses module notwithstanding. > Is the need to use zip(count(3), seq) for the offset index case really such > a burden given the associated benefits in keeping the builtin function > really simple and easy to understand? Yes, zip(count(3), seq) is too complex for this simple use case. I've always solved this so far with this less-than-elegant but certainly simpler idiom (except for users stuck in the tradition of for-loops in certain older languages :-): for i, line in enumerat(lines): i += 1 print "%4d. %s" % (i, line) and variants thereof. [Also added to the issue.] -- --Guido van Rossum (home page: http://www.python.org/~guido/) From duncan.booth at suttoncourtenay.org.uk Tue May 13 17:07:02 2008 From: duncan.booth at suttoncourtenay.org.uk (Duncan Booth) Date: Tue, 13 May 2008 15:07:02 +0000 (UTC) Subject: [Python-Dev] Adding start to enumerate() References: <200805120942.46259.steve@pearwood.info> Message-ID: "Steven D'Aprano" wrote: > On Mon, 12 May 2008 08:20:51 am Georg Brandl wrote: >> I believe the following is a common use-case for enumerate() >> (at least, I've used it quite some times): >> >> for lineno, line in enumerate(fileobject): >> ... >> >> For this, it would be nice to have a start parameter for enumerate(). > > Why would it be nice? What would you use it for? > > The only thing I can think of is printing lines with line numbers, and > starting those line numbers at one instead of zero. If that's the only > use-case, should it require built-in support? > If you are generating paginated output then a function to generate an arbitrary page would likely want to enumerate starting at some value larger than one. Of course in that case you'll also want to skip part way through the data, but I think it is more likely that you'll want to enumerate the partial data (e.g. if it is a database query) rather than slice the enumeration. From greg at krypto.org Tue May 13 20:33:53 2008 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 13 May 2008 11:33:53 -0700 Subject: [Python-Dev] heads up on svn.python.org ssh keys - debian/ubuntu users may need new ones Message-ID: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> Heads up. Debian screwed up. As a result all ssh and ssl keys generated in the last 18 months on debian and ubuntu systems may be compromised due to not using a good random number generator seed. http://lists.debian.org/debian-security-announce/2008/msg00152.html and http://www.links.org/?p=327 If you generated your python subversion ssh key during this time on a machine fitting the description above, please consider replacing your keys. apt-get update ; apt-get upgrade on debian will provide you with a ssh-vulnkey program that can be used to test if your ssh keys are valid or not. -gps From ggpolo at gmail.com Tue May 13 20:45:02 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Tue, 13 May 2008 15:45:02 -0300 Subject: [Python-Dev] heads up on svn.python.org ssh keys - debian/ubuntu users may need new ones In-Reply-To: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> References: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> Message-ID: 2008/5/13 Gregory P. Smith : > Heads up. > > Debian screwed up. As a result all ssh and ssl keys generated in the > last 18 months on debian and ubuntu systems may be compromised due to > not using a good random number generator seed. > http://lists.debian.org/debian-security-announce/2008/msg00152.html > and http://www.links.org/?p=327 > > If you generated your python subversion ssh key during this time on a > machine fitting the description above, please consider replacing your > keys. > > apt-get update ; apt-get upgrade on debian will provide you with a > ssh-vulnkey program that can be used to test if your ssh keys are > valid or not. > Thanks for pointing it out Gregory. ssh-vulnkey says most of my keys are compromised, including the one used for python's svn. > -gps > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com > -- -- Guilherme H. Polo Goncalves From steve at pearwood.info Tue May 13 20:53:06 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 14 May 2008 04:53:06 +1000 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: <48296229.9000800@gmail.com> Message-ID: <200805140453.07013.steve@pearwood.info> On Wed, 14 May 2008 12:25:01 am Guido van Rossum wrote: > However I see no use for skipping items > from the start, You've never had to deal with data where the first N items were special in some way? e.g. skipping over a header line in a file? I know I've written code like this before: it = iter(whatever()) for i in xrange(N): # skip the first N items it.next() for item in it: process(item) > and if that use case ever came up, passing a slice to > enumerate() would be the appropriate thing to do. While slices are wonderfully useful things, they aren't panaceas. They're not so useful with iterators, and they make a copy of the data, which can be problematic if there's a *lot* of it. [tongue firmly in cheek] Perhaps what we need is a more flexible enumerate function? enumerate(iterable, start_at_index=0, count_from=0) Having a consistent index provided by enumerate reduces the amount of thought I have to put into the loop structure, and avoids the temptation to do this: for i, item in enumerate(seq, start=1): print "line %d: %s" % (i, item) # subtract one from the index to adjust for one-based counting seq[i-1] = foo(item) I never need to think about whether it is zero-based, one-based, or some other N-based counting, because it is always zero-based. -0 on any change to enumerate. -- Steven D'Aprano From facundobatista at gmail.com Tue May 13 20:59:03 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 13 May 2008 15:59:03 -0300 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805140453.07013.steve@pearwood.info> References: <48296229.9000800@gmail.com> <200805140453.07013.steve@pearwood.info> Message-ID: 2008/5/13, Steven D'Aprano : > Perhaps what we need is a more flexible enumerate function? > enumerate(iterable, start_at_index=0, count_from=0) +1 to provide both options: they're not intrusive (as I can keep using enumerate without those), and having both helps in the understanding of the function. I mean, if I find an option that is "start", I can confuse if it will start counting or giving me the elements of the iterable... if I found both parameters, it will be easier to understand. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From guido at python.org Tue May 13 21:01:20 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 13 May 2008 12:01:20 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805140453.07013.steve@pearwood.info> References: <48296229.9000800@gmail.com> <200805140453.07013.steve@pearwood.info> Message-ID: On Tue, May 13, 2008 at 11:53 AM, Steven D'Aprano wrote: > On Wed, 14 May 2008 12:25:01 am Guido van Rossum wrote: > > > However I see no use for skipping items > > from the start, > > You've never had to deal with data where the first N items were special > in some way? e.g. skipping over a header line in a file? Of course I have. But never in the argument to enumerate(). [...] > > and if that use case ever came up, passing a slice to > > enumerate() would be the appropriate thing to do. > > While slices are wonderfully useful things, they aren't panaceas. > They're not so useful with iterators, and they make a copy of the data, > which can be problematic if there's a *lot* of it. That's why we have itertools.islice(). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 13 21:05:36 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 13 May 2008 12:05:36 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: <48296229.9000800@gmail.com> <200805140453.07013.steve@pearwood.info> Message-ID: On Tue, May 13, 2008 at 11:59 AM, Facundo Batista wrote: > 2008/5/13, Steven D'Aprano : > > > > Perhaps what we need is a more flexible enumerate function? > > enumerate(iterable, start_at_index=0, count_from=0) > > +1 to provide both options: they're not intrusive (as I can keep using > enumerate without those), and having both helps in the understanding > of the function. > > I mean, if I find an option that is "start", I can confuse if it will > start counting or giving me the elements of the iterable... if I found > both parameters, it will be easier to understand. Actually, having both dramatically increases the potential for confusion. Once you have the starting index option, you're always going to be worried about whether the first index generated defaults to zero or to the starting index, since depending on your use case one or the other is vastly more useful... We already have itertools.islice() which can handle both of these easily (slice the input or slice the output). -1 on providing a start index. +1 on providing a start value for the count, making it a positional-with-optional-keyword-name ('start') parameter. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Tue May 13 21:22:57 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 13 May 2008 21:22:57 +0200 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: <48296229.9000800@gmail.com> <200805140453.07013.steve@pearwood.info> Message-ID: Guido van Rossum schrieb: > On Tue, May 13, 2008 at 11:59 AM, Facundo Batista > wrote: >> 2008/5/13, Steven D'Aprano : >> >> >> > Perhaps what we need is a more flexible enumerate function? >> > enumerate(iterable, start_at_index=0, count_from=0) >> >> +1 to provide both options: they're not intrusive (as I can keep using >> enumerate without those), and having both helps in the understanding >> of the function. >> >> I mean, if I find an option that is "start", I can confuse if it will >> start counting or giving me the elements of the iterable... if I found >> both parameters, it will be easier to understand. > > Actually, having both dramatically increases the potential for > confusion. Once you have the starting index option, you're always > going to be worried about whether the first index generated defaults > to zero or to the starting index, since depending on your use case one > or the other is vastly more useful... > > We already have itertools.islice() which can handle both of these > easily (slice the input or slice the output). > > -1 on providing a start index. > +1 on providing a start value for the count, making it a > positional-with-optional-keyword-name ('start') parameter. This is what's now implemented in SVN. Thanks for the discussion! cheers, Georg From jcea at jcea.es Tue May 13 21:14:44 2008 From: jcea at jcea.es (Jesus Cea) Date: Tue, 13 May 2008 21:14:44 +0200 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <48278D6C.2020205@canterbury.ac.nz> References: <200805120942.46259.steve@pearwood.info> <48278D6C.2020205@canterbury.ac.nz> Message-ID: <4829E8A4.9040207@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Greg Ewing wrote: | Steven D'Aprano wrote: |> The only thing I can think of is printing lines with line numbers | | Parsing a file and wanting to be able to print | error messages with line numbers would seem to | be a fairly likely use. What is wrong with: for lineno,line in enumerate(file) : ~ print lineno+1,line :-? - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSCnopJlgi5GaxT1NAQJSAAP/V8U23owV+YjGiAKQbZOb6FIXA6Wfll65 /oLg1WuzQ/KBre41LJ3sw4RS8+FThrfihe56nYmJagnkvo353rceb/W45T85b17Y +VXg33jQy8UkoJX2a0YwFP8jZPO+5iWlO6zUc44iisBmh8BeLCMQAJHl6+0pJfqQ /cp/BLcxEYI= =T1Bc -----END PGP SIGNATURE----- From jcea at jcea.es Tue May 13 21:09:59 2008 From: jcea at jcea.es (Jesus Cea) Date: Tue, 13 May 2008 21:09:59 +0200 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? Message-ID: <4829E787.5080605@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just committed pybsddb 4.6.4 to python svn. My next step (after a successfull buildbot cycle, I hope!) is to commit the new testsuite. First I need to review any changes there since I maintain pybsddb. The testsuite creates a lot of files/directories while working. Fine, since it cleans later, unless some test crashes hard. My testcode defines a variable to the path I want to use to keep those files/dirs. Currently it is "/tmp/z-BerkeleyDB", or something like that. Since my code allows for testcases to be run in multiple threads and/or processes, where would be a sensible path choice for temporal files?. Current working directory would be fine, if "somebody" is cleaning after running all tests. Ramdisk ("/tmp" in some systems, for example Solaris) would be nice, since some test uses transactions, and transactions are write-synchronous. In my system, runnning all tests serially (you can run all of them in parallel, if you wish) takes about 15-20 seconds. Far faster than current tests in python svn, and that can be improved even more. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSCnngZlgi5GaxT1NAQLhPwP9GMMyNMgz6mfmfHXVOV8bDJGOQRok95uL 14A+K9zXW3/wlp1rhvoPmCHYqvRoCLVkPZ/7pLEQlo1ZksOlHy6BH3MDeDJEjVax 69XlzUUeuqplGbTiMdX8qd0dPi2Jp+Akg7U6ZmBdADhF7D21umU474OalKO2eIoL ba/wnlMvens= =1lai -----END PGP SIGNATURE----- From brett at python.org Tue May 13 21:25:59 2008 From: brett at python.org (Brett Cannon) Date: Tue, 13 May 2008 21:25:59 +0200 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: <4829E787.5080605@jcea.es> References: <4829E787.5080605@jcea.es> Message-ID: On Tue, May 13, 2008 at 9:09 PM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I just committed pybsddb 4.6.4 to python svn. My next step (after a > successfull buildbot cycle, I hope!) is to commit the new testsuite. > First I need to review any changes there since I maintain pybsddb. > > The testsuite creates a lot of files/directories while working. Fine, > since it cleans later, unless some test crashes hard. My testcode > defines a variable to the path I want to use to keep those files/dirs. > Currently it is "/tmp/z-BerkeleyDB", or something like that. > > Since my code allows for testcases to be run in multiple threads and/or > processes, where would be a sensible path choice for temporal files?. > Current working directory would be fine, if "somebody" is cleaning after > running all tests. > It is up to the tests to clean up after themselves since they can be executed directly, so there is not magical location that gets cleaned up in case a test screws up. You can use test.test_support.TESTFN if you want a location. Or something from tempfile. -Brett From jcea at jcea.es Tue May 13 21:41:59 2008 From: jcea at jcea.es (Jesus Cea) Date: Tue, 13 May 2008 21:41:59 +0200 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: References: <4829E787.5080605@jcea.es> Message-ID: <4829EF07.2070000@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brett Cannon wrote: | It is up to the tests to clean up after themselves since they can be | executed directly, so there is not magical location that gets cleaned | up in case a test screws up. You can use test.test_support.TESTFN if | you want a location. Or something from tempfile. I'm worried about the case when a test crashes and leave garbage behind... that will crash future test runs. I guess I would delete the directory before using it, but I guess somebody already had this problem and solved it, somehow. :-? - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSCnvA5lgi5GaxT1NAQIM7wP9EEOsr+WYpQtA+0tbHTv247kplxzNDo7T WTIWU+pfs575gMSbK5NUCrr6TXuFsqlgybOaFqA3wTKHiKslcRhkBrmjtd/Wj4iF /bdv8C/SnLdM443ZnakWFnPAH+Y/horuRxKvsx01lNdaNtbADZG9bClOCfVBfoWM j/fUSb5RtAU= =GkQb -----END PGP SIGNATURE----- From brett at python.org Tue May 13 22:56:12 2008 From: brett at python.org (Brett Cannon) Date: Tue, 13 May 2008 13:56:12 -0700 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: <4829EF07.2070000@jcea.es> References: <4829E787.5080605@jcea.es> <4829EF07.2070000@jcea.es> Message-ID: On Tue, May 13, 2008 at 12:41 PM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Brett Cannon wrote: > | It is up to the tests to clean up after themselves since they can be > | executed directly, so there is not magical location that gets cleaned > | up in case a test screws up. You can use test.test_support.TESTFN if > | you want a location. Or something from tempfile. > > I'm worried about the case when a test crashes and leave garbage > behind... that will crash future test runs. > Then don't crash on tests. =) > I guess I would delete the directory before using it, but I guess > somebody already had this problem and solved it, somehow. :-? That's usually the other way to go about it. In your test_main() or your setUp() methods you make sure everything is clean before you start. -Brett From tjreedy at udel.edu Tue May 13 23:09:59 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 13 May 2008 17:09:59 -0400 Subject: [Python-Dev] Adding start to enumerate() References: <200805120942.46259.steve@pearwood.info> Message-ID: "Duncan Booth" wrote in message news:Xns9A9DA3EB5E8EBduncanrcpcouk at 127.0.0.1... | | If you are generating paginated output then a function to generate an | arbitrary page would likely want to enumerate starting at some value larger | than one. | | Of course in that case you'll also want to skip part way through the data, | but I think it is more likely that you'll want to enumerate the partial | data (e.g. if it is a database query) rather than slice the enumeration. Not if the data is split across multiple files. I remember editing and printing a book from multiple files. Decent word processors have an option to start page numbering at something other than 1. Much bigger memories perhaps make this less common now. From musiccomposition at gmail.com Tue May 13 23:46:20 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 13 May 2008 16:46:20 -0500 Subject: [Python-Dev] heads up on svn.python.org ssh keys - debian/ubuntu users may need new ones In-Reply-To: References: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> Message-ID: <1afaf6160805131446g5cc2f17dp986fd3e2b3190fe5@mail.gmail.com> On Tue, May 13, 2008 at 1:45 PM, Guilherme Polo wrote:\ > Thanks for pointing it out Gregory. > > ssh-vulnkey says most of my keys are compromised, including the one > used for python's svn. Then according to the developer FAQ, you should change your keys and email pydotorg at python.org . -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From steve at pearwood.info Wed May 14 00:11:23 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 14 May 2008 08:11:23 +1000 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: References: <200805140453.07013.steve@pearwood.info> Message-ID: <200805140811.24004.steve@pearwood.info> On Wed, 14 May 2008 05:01:20 am you wrote: > > ?While slices are wonderfully useful things, they aren't panaceas. > > ?They're not so useful with iterators, and they make a copy of the > > data, which can be problematic if there's a *lot* of it. > > That's why we have itertools.islice(). I always forget itertools. Perhaps I should engrave it on my monitor as a reminder. With iterators being such a fundamental part of Python these days, perhaps one day we'll see the functions in the itertools module become iterator methods, as happened with strings. But that's a discussion for another day. -- Steven From guido at python.org Wed May 14 00:18:44 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 13 May 2008 15:18:44 -0700 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805140811.24004.steve@pearwood.info> References: <200805140453.07013.steve@pearwood.info> <200805140811.24004.steve@pearwood.info> Message-ID: On Tue, May 13, 2008 at 3:11 PM, Steven D'Aprano wrote: > With iterators being such a fundamental part of Python these days, > perhaps one day we'll see the functions in the itertools module become > iterator methods, as happened with strings. But that's a discussion for > another day. Unlikely. Every method needs to be reimplemented for each type that is expected to support it. There are only two string types, so that's manageable. But there are many dozens of iterator types. Insisting on a common base class would be unpythonic. Plus, the functional approach makes it possible to treat various non-iterators (like sequences and other iterables) the same way. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed May 14 01:12:51 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 14 May 2008 01:12:51 +0200 Subject: [Python-Dev] heads up on svn.python.org ssh keys - debian/ubuntu users may need new ones In-Reply-To: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> References: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> Message-ID: <482A2073.4000400@v.loewis.de> > If you generated your python subversion ssh key during this time on a > machine fitting the description above, please consider replacing your > keys. > > apt-get update ; apt-get upgrade on debian will provide you with a > ssh-vulnkey program that can be used to test if your ssh keys are > valid or not. I'll ping all committers for which ssh-vulnkey reports COMPROMISED. I personally don't think the threat is severe - unless people also published their public SSH keys somewhere, there is little chance that somebody can break in by just guessing them remotely - you still need to try a lot of combinations for user names and passwords, plus with subversion, we'll easily recognize doubtful checkins (as we do even if the committer is legitimate :-). Regards, Martin From jcea at jcea.es Wed May 14 01:49:51 2008 From: jcea at jcea.es (Jesus Cea) Date: Wed, 14 May 2008 01:49:51 +0200 Subject: [Python-Dev] BSDDB3 In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E2456D05A@EXMBX04.exchhosting.com> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> <47CF0B7C.109@v.loewis.de> <480CABFB.2080302@argo.es> <87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com> <480EA87D.5040107@argo.es> <87D3F9C72FBF214DB39FA4E3FE618CDC6E2456D05A@EXMBX04.exchhosting.com> Message-ID: <482A291F.3070107@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: |> Next week I will (if nothing goes wrong) publish pybsddb |> 4.6.4. This release supports distributed transactions and |> replication, testsuite is way faster, and rewritten to be |> able to launch tests from multiple threads/processes if you |> wish, setuptools/pypi support, etc. | | Great! Once you've settled in with your changes let me know and I'll help with doing the necessary things on the Windows-side to set up the bsddb46.vcproj and switching the build to use that. Python SVN updated. Let me know if you need anything from me. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSCopHZlgi5GaxT1NAQKrCQQAj/HTk5oqSbF2PYkZpCw2T7Dd6yEgddlY L+qW1Cde/b3duClEfawy7kf5DkSjlGLVZ9XSS+njAMKszzYK6ZIg9N4GEu5A9TO2 Rg2PiytaPbs88Jo5IIlDjvaVFPPqsOasn7WH1wcawtUKNei8whMReOveZgYXfFFf QphSJ7zspNU= =L6jr -----END PGP SIGNATURE----- From jcea at jcea.es Wed May 14 01:54:35 2008 From: jcea at jcea.es (Jesus Cea) Date: Wed, 14 May 2008 01:54:35 +0200 Subject: [Python-Dev] BSDDB3 In-Reply-To: <52dc1c820804231317h67954dcbmdb50b58da35b3b64@mail.gmail.com> References: <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> <47CF0B7C.109@v.loewis.de> <480CABFB.2080302@argo.es> <87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com> <480EA87D.5040107@argo.es> <52dc1c820804231317h67954dcbmdb50b58da35b3b64@mail.gmail.com> Message-ID: <482A2A3B.8090403@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gregory P. Smith wrote: | After that, merging his changes into trunk will be relatively easy and I | think we should give jcea commit access and let him do it and henceforth | use python svn as the official repository for the pybsddb code and | documentation. | | I do not see this as a daunting task assuming jcea is willing to be | volunteered to do the merge and to handle any py3k merge issues that | come up. Well, I would like to keep bsddb3 alive outside python SVN. There are a few reasons (like being able to accept foreign patches, for instance); the main one is: I've published *four* bsddb3 versions in the last 3-4 months. Each release gives a quantum leap improvement in Berkeley DB support (for example, replication and distributed transactions). If I link the bsddb releases to python ones, progress would be very slow. In current state, bsddb3 still needs a lot of work to catch current Berkeley DB feature sets. I think I can publish a new release per month until 2009. Those releases will be consistent improvements in Berkeley DB support. But I don't want to link Python with this much needed bsddb3 releases. My intention is to upgrade python bsddb3 versions from time to time, enough to keep them fresh without burden you with constant patches. On time for official releases. Moreover, keeping the bsddb3 separate will allow me to publish a new release when Oracle updates Berkeley DB. This has been a serious issue in the past. bsddb3, as a separate project, can support last Berkeley DB release without python build servers needing a BDB upgrade and users waiting a full release cycle (months). A last point: I can provide bsddb3 packages for python 2.3, 2.4 and 2.5, also. That said, I'm very interested in being a good python-dev netizen. I will do what needed to merge patches back and forth. I will read your comments with maximum interest. I will have the buildbot webpage open permanently :-). I will do the python 3.0 necessary changes when time comes. And Python will integrate a bsddb3 library proved in the wild, verified against 5 different python releases and 8 different Berkeley DB ones. And I will love to do additional work in python, aside bsddb. Let me show you that I can do it... - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSCoqO5lgi5GaxT1NAQJP1wP6A46t6f7wajIyrUoScLE8RSqJSncnW1jo r/7A69hRC6UW2I0JbMJ+wFVcXC2507XaCiC0l0Xljq9IqH7JAU6H/gwv6Cyp4eVF tmwiUVFmlw+wyZMCnvBAa4EOF4/ZBwrILIdNywFNpoRmCLH8RmiBDE+x4jgjcSCh N5yssDDLzPk= =/YFF -----END PGP SIGNATURE----- From brett at python.org Wed May 14 02:12:58 2008 From: brett at python.org (Brett Cannon) Date: Tue, 13 May 2008 17:12:58 -0700 Subject: [Python-Dev] BSDDB3 In-Reply-To: <482A2A3B.8090403@jcea.es> References: <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> <47CF0B7C.109@v.loewis.de> <480CABFB.2080302@argo.es> <87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com> <480EA87D.5040107@argo.es> <52dc1c820804231317h67954dcbmdb50b58da35b3b64@mail.gmail.com> <482A2A3B.8090403@jcea.es> Message-ID: On Tue, May 13, 2008 at 4:54 PM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Gregory P. Smith wrote: > | After that, merging his changes into trunk will be relatively easy and I > | think we should give jcea commit access and let him do it and henceforth > | use python svn as the official repository for the pybsddb code and > | documentation. > | > | I do not see this as a daunting task assuming jcea is willing to be > | volunteered to do the merge and to handle any py3k merge issues that > | come up. > > Well, I would like to keep bsddb3 alive outside python SVN. There are a > few reasons (like being able to accept foreign patches, for instance); > the main one is: I've published *four* bsddb3 versions in the last 3-4 > months. Each release gives a quantum leap improvement in Berkeley DB > support (for example, replication and distributed transactions). If I > link the bsddb releases to python ones, progress would be very slow. > This is why I think bsddb should just be an external package. The few wrappers we still have in the core (sqlite and tkinter) do not have a high churn rate at all since both are fairly stable and not constantly changing (or at least that I can tell). Regardless, we are trying to prevent external maintenance from happening anymore. Python is not like a Linux distribution where we push patches upstream; we are the upstream. Patches should end with us. > In current state, bsddb3 still needs a lot of work to catch current > Berkeley DB feature sets. I think I can publish a new release per month > until 2009. Those releases will be consistent improvements in Berkeley > DB support. But I don't want to link Python with this much needed bsddb3 > releases. My intention is to upgrade python bsddb3 versions from time to > time, enough to keep them fresh without burden you with constant > patches. On time for official releases. > If the wrapper is still behind then I think this is another reason to remove the package from the core and just keep it external. Why have something in the core that is outdated when its committed? > Moreover, keeping the bsddb3 separate will allow me to publish a new > release when Oracle updates Berkeley DB. This has been a serious issue > in the past. bsddb3, as a separate project, can support last Berkeley DB > release without python build servers needing a BDB upgrade and users > waiting a full release cycle (months). > > A last point: I can provide bsddb3 packages for python 2.3, 2.4 and 2.5, > also. > > That said, I'm very interested in being a good python-dev netizen. I > will do what needed to merge patches back and forth. I will read your > comments with maximum interest. I will have the buildbot webpage open > permanently :-). I will do the python 3.0 necessary changes when time > comes. And Python will integrate a bsddb3 library proved in the wild, > verified against 5 different python releases and 8 different Berkeley DB > ones. > > And I will love to do additional work in python, aside bsddb. > > Let me show you that I can do it... I am always happy to have developers to step forward and want to help! And the last thing I want to do is discourage another contributor, but I still think bsddb should be gone from 3.0. The churn rate sounds too high on Berkeley DB itself to warrant making it an included package that we continue to support internally. I also worry about maintenance in terms of multiple people being able to help. bsddb sat there for a while when Gregory scaled back his work. I remember trying to do the initial port of bsddb to 3.0 and it was hard and not many people wanted to work on it (and I don't blame them). sqlite and tkinter, on the other hand, was not as difficult and we have never had issues finding people willing to help with the code. While Jesus is up for doing all of this work at this moment (which is great!), if he fell off the face of the earth we will be stuck with a stale package again. -Brett From alexandre at peadrop.com Wed May 14 02:17:53 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Tue, 13 May 2008 20:17:53 -0400 Subject: [Python-Dev] heads up on svn.python.org ssh keys - debian/ubuntu users may need new ones In-Reply-To: <482A2073.4000400@v.loewis.de> References: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> <482A2073.4000400@v.loewis.de> Message-ID: On Tue, May 13, 2008 at 7:12 PM, "Martin v. L?wis" wrote: > > If you generated your python subversion ssh key during this time on a > > machine fitting the description above, please consider replacing your > > keys. > > > > apt-get update ; apt-get upgrade on debian will provide you with a > > ssh-vulnkey program that can be used to test if your ssh keys are > > valid or not. > > I'll ping all committers for which ssh-vulnkey reports COMPROMISED. > > I personally don't think the threat is severe - unless people also > published their public SSH keys somewhere, there is little chance that > somebody can break in by just guessing them remotely - you still need > to try a lot of combinations for user names and passwords, plus with > subversion, we'll easily recognize doubtful checkins (as we do even > if the committer is legitimate :-). > Well, I had a break in on my public server (peadrop.com) this week, which had a copy my ssh pubkey. I don't know if the attacker took a look at my pubkeys, but I won't take any change. So, I definitely have to change my key, ASAP. -- Alexandre From greg.ewing at canterbury.ac.nz Wed May 14 02:24:48 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 May 2008 12:24:48 +1200 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805140811.24004.steve@pearwood.info> References: <200805140453.07013.steve@pearwood.info> <200805140811.24004.steve@pearwood.info> Message-ID: <482A3150.3060802@canterbury.ac.nz> Steven D'Aprano wrote: > With iterators being such a fundamental part of Python these days, > perhaps one day we'll see the functions in the itertools module become > iterator methods I hope not. The set of potential functions that operate on iterators is open-ended, and there's no reason to single out a particular subset and make them methods. Also, it would force all iterators to inherit from a common base class in order to support those methods, which is not a Pythonic thing to do. Strings are different, because (a) they're a concrete class, and (b) string methods benefit from having access to implementation details of the string. -- Greg From jnoller at gmail.com Wed May 14 02:52:35 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 13 May 2008 20:52:35 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. Message-ID: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> I am looking for any questions, concerns or benchmarks python-dev has regarding the possible inclusion of the pyprocessing module to the standard library - preferably in the 2.6 timeline. In March, I began working on the PEP for the inclusion of the pyprocessing (processing) module into the python standard library[1]. The original email to the stdlib-sig can be found here, it includes a basic overview of the module: http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html The processing module mirrors/mimics the API of the threading module - and with simple import/subclassing changes depending on the code, allows you to leverage multi core machines via an underlying forking mechanism. The module also supports the sharing of data across groups of networked machines - a feature obviously not part of the core threading module, but useful in a distributed environment. As I am trying to finish up the PEP, I want to see if I can address any questions or include any other useful data (including benchmarks) in the PEP prior to publishing it. I am also intending to include basic benchmarks for both the processing module against the threading module as a comparison. -Jesse [1] Processing page: http://pyprocessing.berlios.de/ From greg.ewing at canterbury.ac.nz Wed May 14 02:57:45 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 May 2008 12:57:45 +1200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: <482A3909.5060806@canterbury.ac.nz> Jesse Noller wrote: > I am looking for any questions, concerns or benchmarks python-dev has > regarding the possible inclusion of the pyprocessing module to the > standard library Sounds good, but I'd suggest giving a more specific name than "processing", which is so generic as to be meaningless. -- Greg From jnoller at gmail.com Wed May 14 03:22:20 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 13 May 2008 21:22:20 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482A3909.5060806@canterbury.ac.nz> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482A3909.5060806@canterbury.ac.nz> Message-ID: <744245CB-B1BC-437F-801B-FA79F25A6AF9@gmail.com> On May 13, 2008, at 8:57 PM, Greg Ewing wrote: > Jesse Noller wrote: >> I am looking for any questions, concerns or benchmarks python-dev has >> regarding the possible inclusion of the pyprocessing module to the >> standard library > > Sounds good, but I'd suggest giving a more specific > name than "processing", which is so generic as to > be meaningless. > > -- > Greg > ____ Any suggestions? Subprocess is taken and concurrent might be presumptuous unless it was added as concurrent.processing or some like permutation. -Jesse From thomaspinckney3 at gmail.com Wed May 14 03:23:02 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Tue, 13 May 2008 21:23:02 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: Why not use MPI? It's cross platform, cross language and very widely supported already. And there're Python bindings already. On May 13, 2008, at 8:52 PM, Jesse Noller wrote: > I am looking for any questions, concerns or benchmarks python-dev has > regarding the possible inclusion of the pyprocessing module to the > standard library - preferably in the 2.6 timeline. In March, I began > working on the PEP for the inclusion of the pyprocessing (processing) > module into the python standard library[1]. The original email to the > stdlib-sig can be found here, it includes a basic overview of the > module: > > http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html > > The processing module mirrors/mimics the API of the threading module - > and with simple import/subclassing changes depending on the code, > allows you to leverage multi core machines via an underlying forking > mechanism. The module also supports the sharing of data across groups > of networked machines - a feature obviously not part of the core > threading module, but useful in a distributed environment. > > As I am trying to finish up the PEP, I want to see if I can address > any questions or include any other useful data (including benchmarks) > in the PEP prior to publishing it. I am also intending to include > basic benchmarks for both the processing module against the threading > module as a comparison. > > -Jesse > > [1] Processing page: http://pyprocessing.berlios.de/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/thomaspinckney3%40gmail.com From barry at python.org Wed May 14 03:37:32 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 13 May 2008 21:37:32 -0400 Subject: [Python-Dev] heads up on svn.python.org ssh keys - debian/ubuntu users may need new ones In-Reply-To: <482A2073.4000400@v.loewis.de> References: <52dc1c820805131133q61565b6dj839ad824194ad508@mail.gmail.com> <482A2073.4000400@v.loewis.de> Message-ID: <06914F51-DCE5-497A-B9E9-F2C61D77541D@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 13, 2008, at 7:12 PM, Martin v. L?wis wrote: >> If you generated your python subversion ssh key during this time on a >> machine fitting the description above, please consider replacing your >> keys. >> >> apt-get update ; apt-get upgrade on debian will provide you with a >> ssh-vulnkey program that can be used to test if your ssh keys are >> valid or not. > > I'll ping all committers for which ssh-vulnkey reports COMPROMISED. > > I personally don't think the threat is severe - unless people also > published their public SSH keys somewhere, there is little chance that > somebody can break in by just guessing them remotely - you still need > to try a lot of combinations for user names and passwords, plus with > subversion, we'll easily recognize doubtful checkins (as we do even > if the committer is legitimate :-). It's also probably worth checking the keys for everyone who has shell access on the python.org machines. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSCpCXHEjvBPtnXfVAQLy5gP+MZJ7/RKDqw9QKvNr9rlGm7GjOBkuWR3B UA91clzb4Iuy+51+V4B3iUcdmwGtpfYum8/2+1/qpi7abO/IiIQvvOKczQzkv5XL ALh59zR2iiBuNg1BVW0JPdkyNt6qr2oe8kKdUZfyrwRSKIukX+e40Oa+1zvfp0E7 9AumiqMUCtI= =EXC8 -----END PGP SIGNATURE----- From jnoller at gmail.com Wed May 14 03:41:58 2008 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 13 May 2008 21:41:58 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: The inclusion of the processing module does not exclude the potential to also use or one day include MPI bindings. The goal is to add a module with a "known" API and semantics which allows programmer using cPython to easily take advantage of multple cores (and as a side benefit, machines). In theory - one could also use MPI within an application using the processing module as a communications/message passing system. -Jesse On May 13, 2008, at 9:23 PM, Tom Pinckney wrote: > Why not use MPI? It's cross platform, cross language and very widely > supported already. And there're Python bindings already. > > On May 13, 2008, at 8:52 PM, Jesse Noller wrote: > >> I am looking for any questions, concerns or benchmarks python-dev has >> regarding the possible inclusion of the pyprocessing module to the >> standard library - preferably in the 2.6 timeline. In March, I began >> working on the PEP for the inclusion of the pyprocessing (processing) >> module into the python standard library[1]. The original email to the >> stdlib-sig can be found here, it includes a basic overview of the >> module: >> >> http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html >> >> The processing module mirrors/mimics the API of the threading >> module - >> and with simple import/subclassing changes depending on the code, >> allows you to leverage multi core machines via an underlying forking >> mechanism. The module also supports the sharing of data across groups >> of networked machines - a feature obviously not part of the core >> threading module, but useful in a distributed environment. >> >> As I am trying to finish up the PEP, I want to see if I can address >> any questions or include any other useful data (including benchmarks) >> in the PEP prior to publishing it. I am also intending to include >> basic benchmarks for both the processing module against the threading >> module as a comparison. >> >> -Jesse >> >> [1] Processing page: http://pyprocessing.berlios.de/ >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/thomaspinckney3%40gmail.com > From talin at acm.org Wed May 14 04:02:39 2008 From: talin at acm.org (Talin) Date: Tue, 13 May 2008 19:02:39 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482A3909.5060806@canterbury.ac.nz> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482A3909.5060806@canterbury.ac.nz> Message-ID: <482A483F.6060303@acm.org> multiprocessing Greg Ewing wrote: > Jesse Noller wrote: >> I am looking for any questions, concerns or benchmarks python-dev has >> regarding the possible inclusion of the pyprocessing module to the >> standard library > > Sounds good, but I'd suggest giving a more specific > name than "processing", which is so generic as to > be meaningless. From ncoghlan at gmail.com Wed May 14 06:32:14 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 May 2008 14:32:14 +1000 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <482A3150.3060802@canterbury.ac.nz> References: <200805140453.07013.steve@pearwood.info> <200805140811.24004.steve@pearwood.info> <482A3150.3060802@canterbury.ac.nz> Message-ID: <482A6B4E.8080108@gmail.com> Greg Ewing wrote: > Steven D'Aprano wrote: >> With iterators being such a fundamental part of Python these days, >> perhaps one day we'll see the functions in the itertools module become >> iterator methods > > I hope not. The set of potential functions that operate > on iterators is open-ended, and there's no reason to > single out a particular subset and make them methods. I've occasionally experimented with the idea of a 'FlexibleIterator' class that accepted an arbitrary iterable in it's constructor and then used itertools internally to provide native support for concatenation (via itertools.chain) and slicing (via itertools.islice), but have never been able to come up with anything which didn't lend itself to fundamental misunderstanding of what it was doing. I've come to the conclusion that there are some aspects of dealing with arbitrary iterators where trying to make it look 'pretty' starts to hide things that the programmer really needs to be aware of in order to reason correctly about the program. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg at krypto.org Wed May 14 06:33:47 2008 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 13 May 2008 21:33:47 -0700 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: <4829E787.5080605@jcea.es> References: <4829E787.5080605@jcea.es> Message-ID: <52dc1c820805132133s1ad0be0em2d7c25296fece78c@mail.gmail.com> btw, I fixed the Lib/test/test_bsddb3.py file for the updated Lib/bsddb/test/ modules. Thats how the test suite gets run by the buildbots (run the test suite from a python trunk sandbox using "./python Lib/test/regrtest.py -v -u bsddb test_bsddb test_bsddb3" to reproduce exactly how it is run yourself). I'm seeing what appears to be a memory corruption error when using a debug build of trunk (configure --with-pydebug). It causes a crash while deallocating the argument tuple passed to DBEnv.lock_get on the second call in bsddb.test.test_lock's test03_lock_timeout() test. Platform tested on: OS X 10.4, gcc 4.0.1 x86. BerkeleyDB 4.5. I haven't tested on other platforms but I expect other buildbots will also have issues now that the test suite isn't being skipped. -gps On Tue, May 13, 2008 at 12:09 PM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I just committed pybsddb 4.6.4 to python svn. My next step (after a > successfull buildbot cycle, I hope!) is to commit the new testsuite. > First I need to review any changes there since I maintain pybsddb. > > The testsuite creates a lot of files/directories while working. Fine, > since it cleans later, unless some test crashes hard. My testcode > defines a variable to the path I want to use to keep those files/dirs. > Currently it is "/tmp/z-BerkeleyDB", or something like that. > > Since my code allows for testcases to be run in multiple threads and/or > processes, where would be a sensible path choice for temporal files?. > Current working directory would be fine, if "somebody" is cleaning after > running all tests. > > Ramdisk ("/tmp" in some systems, for example Solaris) would be nice, > since some test uses transactions, and transactions are write-synchronous. > > In my system, runnning all tests serially (you can run all of them in > parallel, if you wish) takes about 15-20 seconds. Far faster than > current tests in python svn, and that can be improved even more. > > - -- > Jesus Cea Avion _/_/ _/_/_/ _/_/_/ > jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ > jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ > ~ _/_/ _/_/ _/_/ _/_/ _/_/ > "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ > "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ > "El amor es poner tu felicidad en la felicidad de otro" - Leibniz > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.8 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iQCVAwUBSCnngZlgi5GaxT1NAQLhPwP9GMMyNMgz6mfmfHXVOV8bDJGOQRok95uL > 14A+K9zXW3/wlp1rhvoPmCHYqvRoCLVkPZ/7pLEQlo1ZksOlHy6BH3MDeDJEjVax > 69XlzUUeuqplGbTiMdX8qd0dPi2Jp+Akg7U6ZmBdADhF7D21umU474OalKO2eIoL > ba/wnlMvens= > =1lai > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > From ncoghlan at gmail.com Wed May 14 06:37:47 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 May 2008 14:37:47 +1000 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: References: <4829E787.5080605@jcea.es> <4829EF07.2070000@jcea.es> Message-ID: <482A6C9B.70003@gmail.com> Brett Cannon wrote: > On Tue, May 13, 2008 at 12:41 PM, Jesus Cea wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Brett Cannon wrote: >> | It is up to the tests to clean up after themselves since they can be >> | executed directly, so there is not magical location that gets cleaned >> | up in case a test screws up. You can use test.test_support.TESTFN if >> | you want a location. Or something from tempfile. >> >> I'm worried about the case when a test crashes and leave garbage >> behind... that will crash future test runs. >> > > Then don't crash on tests. =) > >> I guess I would delete the directory before using it, but I guess >> somebody already had this problem and solved it, somehow. :-? > > That's usually the other way to go about it. In your test_main() or > your setUp() methods you make sure everything is clean before you > start. Personally, I use the tempfile module to get a temporary directory and do anything I need in there (see test_runpy.py or test_cmdline_script.py for examples). That way, even if the test does manage to crash in a way that bypasses the shutil.rmtree cleanup call (which is actually fairly hard to do when the cleanup call is in a finally clause or a tearDown method), it won't affect subsequent executions (although it may accumulate clutter in the buildbot machine's temporary directories over time). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed May 14 06:39:50 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 May 2008 14:39:50 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482A483F.6060303@acm.org> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482A3909.5060806@canterbury.ac.nz> <482A483F.6060303@acm.org> Message-ID: <482A6D16.4020300@gmail.com> Talin wrote: > multiprocessing multiprocessing would work for me - it adds the concurrency implications that threading has, but 'processing' on its own would be missing. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at holdenweb.com Wed May 14 06:41:10 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 14 May 2008 00:41:10 -0400 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805140453.07013.steve@pearwood.info> References: <48296229.9000800@gmail.com> <200805140453.07013.steve@pearwood.info> Message-ID: <482A6D66.7020808@holdenweb.com> Steven D'Aprano wrote: [...] > [tongue firmly in cheek] > Perhaps what we need is a more flexible enumerate function? > enumerate(iterable, start_at_index=0, count_from=0) Super idea. Then we can have a thread about whether it belongs in itertools or somewhere else. [...] Please take your tongue out of your cheek immediately. I am beginning to feel somewhat uncomfortable on your behalf. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Wed May 14 06:41:10 2008 From: steve at holdenweb.com (Steve Holden) Date: Wed, 14 May 2008 00:41:10 -0400 Subject: [Python-Dev] Adding start to enumerate() In-Reply-To: <200805140453.07013.steve@pearwood.info> References: <48296229.9000800@gmail.com> <200805140453.07013.steve@pearwood.info> Message-ID: <482A6D66.7020808@holdenweb.com> Steven D'Aprano wrote: [...] > [tongue firmly in cheek] > Perhaps what we need is a more flexible enumerate function? > enumerate(iterable, start_at_index=0, count_from=0) Super idea. Then we can have a thread about whether it belongs in itertools or somewhere else. [...] Please take your tongue out of your cheek immediately. I am beginning to feel somewhat uncomfortable on your behalf. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ncoghlan at gmail.com Wed May 14 06:42:44 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 May 2008 14:42:44 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: <482A6DC4.7040103@gmail.com> Tom Pinckney wrote: > Why not use MPI? It's cross platform, cross language and very widely > supported already. And there're Python bindings already. The point of pyprocessing is that fact that the API is the same as that of the threading module - making it very easy to convert a multithreaded program to a multiprocessing one, and hence making it easy to exploit multiple processors in a CPU bound Python program, regardless of the restrictions imposed by Python's GIL. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Wed May 14 07:10:51 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 May 2008 17:10:51 +1200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482A6D16.4020300@gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482A3909.5060806@canterbury.ac.nz> <482A483F.6060303@acm.org> <482A6D16.4020300@gmail.com> Message-ID: <482A745B.2040706@canterbury.ac.nz> Nick Coghlan wrote: > Talin wrote: > >> multiprocessing > > multiprocessing would work for me Me, too. -- Greg From martin at v.loewis.de Wed May 14 10:45:05 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 14 May 2008 10:45:05 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: <482AA691.6040802@v.loewis.de> Jesse Noller wrote: > I am looking for any questions, concerns or benchmarks python-dev has > regarding the possible inclusion of the pyprocessing module to the > standard library - preferably in the 2.6 timeline. I think for inclusion in 2.6 it's to late. For 3.0, it's definitely too late - the PEP acceptance deadline was a year ago (IIRC). > As I am trying to finish up the PEP, I want to see if I can address > any questions or include any other useful data (including benchmarks) > in the PEP prior to publishing it. I am also intending to include > basic benchmarks for both the processing module against the threading > module as a comparison. I'm worried whether it's stable, what user base it has, whether users (other than the authors) are lobbying for inclusion. Statistically, it seems to be not ready yet: it is not even a year old, and has not reached version 1.0 yet. Regards, Martin From lists at cheimes.de Wed May 14 11:45:23 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 14 May 2008 11:45:23 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482AA691.6040802@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> Message-ID: <482AB4B3.7020409@cheimes.de> Martin v. L?wis schrieb: > I'm worried whether it's stable, what user base it has, whether users > (other than the authors) are lobbying for inclusion. Statistically, > it seems to be not ready yet: it is not even a year old, and has not > reached version 1.0 yet. I'm on Martin's side here. Although I like to see some sort of multi processing mechanism in Python 'cause I need it for lots of projects I'm against the inclusion of pyprocessing in 2.6 and 3.0. The project isn't old and mature enough and it has some competitors like pp (parallel processing). On the one hand the inclusion of a package gives it an unfair advantage over similar packages. On the other hand it slows down future development because a new feature release must be synced with Python releases about every 1.5 years. -0.5 from me Christian From ncoghlan at gmail.com Wed May 14 12:01:40 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 May 2008 20:01:40 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482AB4B3.7020409@cheimes.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> Message-ID: <482AB884.304@gmail.com> Christian Heimes wrote: > Martin v. L?wis schrieb: >> I'm worried whether it's stable, what user base it has, whether users >> (other than the authors) are lobbying for inclusion. Statistically, >> it seems to be not ready yet: it is not even a year old, and has not >> reached version 1.0 yet. > > I'm on Martin's side here. Although I like to see some sort of multi > processing mechanism in Python 'cause I need it for lots of projects I'm > against the inclusion of pyprocessing in 2.6 and 3.0. The project isn't > old and mature enough and it has some competitors like pp (parallel > processing). > > On the one hand the inclusion of a package gives it an unfair advantage > over similar packages. On the other hand it slows down future > development because a new feature release must be synced with Python > releases about every 1.5 years. > > -0.5 from me It also isn't something which needs to be done *right now*. Leaving this for 3.x/2.7 seems like a much better idea to me. With the continuing rise of multi-processor desktop machines, the parallel processing approaches that are out there should see a fair amount of use and active development over the next 18 months. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tom at vector-seven.com Wed May 14 12:37:50 2008 From: tom at vector-seven.com (Thomas Lee) Date: Wed, 14 May 2008 20:37:50 +1000 Subject: [Python-Dev] availability of httplib.HTTPResponse.close Message-ID: <482AC0FE.1070701@vector-seven.com> I was debating whether this was truly a question for python-dev or if I should take it to one of the user lists. Ultimately it feels like a question about the implementation of a core module, so hopefully nobody minds me posting it here. :) Although not listed as a public API method in the documentation, it would seem the httplib.HTTPResponse.close method might be useful in the event we don't want to actually read any of the data being sent back from the server. If I'm to assume that an undocumented method is considered "private", currently the only way to force the underlying file-like socket wrapper to close is by calling read() until no more data remains (as per the documentation). What's the reasoning behind requiring callers to read() all the pending data vs. just closing the socket? Is the close method really off limits to code using HTTPResponse? Cheers, Tom From nick at craig-wood.com Wed May 14 12:58:57 2008 From: nick at craig-wood.com (Nick Craig-Wood) Date: Wed, 14 May 2008 11:58:57 +0100 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: <20080514105858.0279814C64B@irishsea.home.craig-wood.com> Jesse Noller wrote: > I am looking for any questions, concerns or benchmarks python-dev has > regarding the possible inclusion of the pyprocessing module to the > standard library - preferably in the 2.6 timeline. In March, I began > working on the PEP for the inclusion of the pyprocessing (processing) > module into the python standard library[1]. The original email to the > stdlib-sig can be found here, it includes a basic overview of the > module: > > http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html > > The processing module mirrors/mimics the API of the threading module - > and with simple import/subclassing changes depending on the code, > allows you to leverage multi core machines via an underlying forking > mechanism. The module also supports the sharing of data across groups > of networked machines - a feature obviously not part of the core > threading module, but useful in a distributed environment. I think processing looks interesting and useful, especially since it works on Windows as well as Un*x. However I'd like to see a review of the security - anything which can run across networks of machines has security implications and I didn't see these spelt out in the documentation. Networked running should certainly be disabled by default and need explicitly enabling by the user - I'd hate for a new version of python to come with a remote exploit by default... -- Nick Craig-Wood -- http://www.craig-wood.com/nick From jnoller at gmail.com Wed May 14 14:06:15 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 14 May 2008 08:06:15 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482AA691.6040802@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> Message-ID: <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> On Wed, May 14, 2008 at 4:45 AM, "Martin v. L?wis" wrote: > Jesse Noller wrote: > > I am looking for any questions, concerns or benchmarks python-dev has > > regarding the possible inclusion of the pyprocessing module to the > > standard library - preferably in the 2.6 timeline. > > I think for inclusion in 2.6 it's to late. For 3.0, it's definitely > too late - the PEP acceptance deadline was a year ago (IIRC). > That's a fair point - however, as this is a proposed inclusion of a standalone library that includes tests, documentation and benchmarks - I was hoping it would be possible to attempt to get it in. A target for 3.1 or 2.7 is also possible. > > > As I am trying to finish up the PEP, I want to see if I can address > > any questions or include any other useful data (including benchmarks) > > in the PEP prior to publishing it. I am also intending to include > > basic benchmarks for both the processing module against the threading > > module as a comparison. > > I'm worried whether it's stable, what user base it has, whether users > (other than the authors) are lobbying for inclusion. Statistically, > it seems to be not ready yet: it is not even a year old, and has not > reached version 1.0 yet. Just as a point of clarification: I am not the module author. I am simply a relatively passionate user using it to solve real problems. I am working on the PEP after speaking with the author about it. One of the outstanding issues I have listed is the "lack" of a 1.0 version - however given the fairly random way that most open source projects pick numbering schemes - I don't see this as much more than an administrative issue. As for user base - at PyCon this year, I was involved in several open space discussion about the concurrency/parallelism "problem" as it relates to Python, as well as giving a lightning talk on this precise subject including mentioning this module and alternatives. The sheer number of people working on "big" problems who approached me to talk about this was astounding. The presence online of the same like-minded people is also quite large. Overwhelmingly, many of the python programmers I spoke to are looking for "a solution" that does not require the alteration of a known programming paradigm (i.e: threads) to allow them to take advantage of systems which are not getting "faster" - instead, they are getting wider. Simply put due to the GIL - pure python applications can not take advantage of these machines which are now more common than not without switching to an alternative interpreter - which for many - myself included - is not an attractive option. Guido, and others called this module out specifically as a "work-around" to the GIL for concurrency within cPython applications. The API matches a known one and in most (if not all) cases, it is an easy drop in to allow programmers to achieve what they may not be able to do with the standard library as it is today. Obviously, there are other solutions that aim to solve this problem domain - Twisted, ParallelPython, etc all "work around" the GIL through various methods - but the fundamental problem with those is that they ask a user to "re-learn" how to construct their applications. While the concept of shared memory and "threads" may not be the "final answer" in the concurrent programming world - fundamentally, it's wrong for us to say that in order for people to really use python to solve concurrent problems they have to: 1: Re-architect their applications 2: Re-learn how to program (i.e: Async programming, Actor models, shared-nothing) I really do feel that inclusion of this library offers us the best of both worlds - it gives us (as a community) an easy answer to those people who would dismiss python due to the GIL and it also allows users to easily implement their applications. The GIL is currently advantageous to the cPython code base for simple ease of implementation and maintenance. Barring the one-day adoption of the bulk of Adam Olsen's safe-threading work or a massive trend in programming to "rid" ourselves of the threaded mindset - this gives cPython a leg-up in the concurrency world and gives users a tool to solve hard problems. -Jesse From jnoller at gmail.com Wed May 14 14:15:08 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 14 May 2008 08:15:08 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482AB4B3.7020409@cheimes.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> Message-ID: <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> On Wed, May 14, 2008 at 5:45 AM, Christian Heimes wrote: > Martin v. L?wis schrieb: > > > I'm worried whether it's stable, what user base it has, whether users > > (other than the authors) are lobbying for inclusion. Statistically, > > it seems to be not ready yet: it is not even a year old, and has not > > reached version 1.0 yet. > > I'm on Martin's side here. Although I like to see some sort of multi > processing mechanism in Python 'cause I need it for lots of projects I'm > against the inclusion of pyprocessing in 2.6 and 3.0. The project isn't > old and mature enough and it has some competitors like pp (parallel > processing). > > On the one hand the inclusion of a package gives it an unfair advantage > over similar packages. On the other hand it slows down future > development because a new feature release must be synced with Python > releases about every 1.5 years. > > -0.5 from me > > Christian > I said this in reply to Martin - but the competitors (in my mind) are not as compelling due to the "alternative" paradigm for application construction they propose. The processing module is an "easy win" for us if included. Personally - I don't see how inclusion in the stdlib would slow down development - yes, you have to stick with the same release cycle as python-core, but if the module is "feature complete" and provides a stable API as it stands I don't see following python-core timelines as overly onerous. The module itself doesn't change that frequently - the last release in April was a bugfix release and API consistency change (the API would have to be locked for inclusion obviously - targeting a 2.7/3.1 release may be advantageous to achieve this). -jesse From jnoller at gmail.com Wed May 14 14:18:05 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 14 May 2008 08:18:05 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080514105858.0279814C64B@irishsea.home.craig-wood.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> Message-ID: <4222a8490805140518sbca0bbgc139d9132fe01dbe@mail.gmail.com> On Wed, May 14, 2008 at 6:58 AM, Nick Craig-Wood wrote: > Jesse Noller wrote: > > I am looking for any questions, concerns or benchmarks python-dev has > > regarding the possible inclusion of the pyprocessing module to the > > standard library - preferably in the 2.6 timeline. In March, I began > > working on the PEP for the inclusion of the pyprocessing (processing) > > module into the python standard library[1]. The original email to the > > stdlib-sig can be found here, it includes a basic overview of the > > module: > > > > http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html > > > > The processing module mirrors/mimics the API of the threading module - > > and with simple import/subclassing changes depending on the code, > > allows you to leverage multi core machines via an underlying forking > > mechanism. The module also supports the sharing of data across groups > > of networked machines - a feature obviously not part of the core > > threading module, but useful in a distributed environment. > > I think processing looks interesting and useful, especially since it > works on Windows as well as Un*x. > > However I'd like to see a review of the security - anything which can > run across networks of machines has security implications and I didn't > see these spelt out in the documentation. > > Networked running should certainly be disabled by default and need > explicitly enabling by the user - I'd hate for a new version of python > to come with a remote exploit by default... > > -- > Nick Craig-Wood -- http://www.craig-wood.com/nick > See the Manager documentation: http://pyprocessing.berlios.de/doc/manager-objects.html And the Listener/Client documentation: http://pyprocessing.berlios.de/doc/connection-ref.html Remote access is not implicit - it is explicit - you must spawn a Manager/Client instance and tell it to use the network instead of it being "always networked". I'll add a security audit to the list of open issues though - that's a good point. -jesse From janssen at parc.com Wed May 14 17:15:17 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 14 May 2008 08:15:17 PDT Subject: [Python-Dev] availability of httplib.HTTPResponse.close In-Reply-To: <482AC0FE.1070701@vector-seven.com> References: <482AC0FE.1070701@vector-seven.com> Message-ID: <08May14.081525pdt."58696"@synergy1.parc.xerox.com> Thomas, I think this is related to issue 1348. Bill From mal at egenix.com Wed May 14 17:46:25 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 14 May 2008 17:46:25 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> Message-ID: <482B0951.2020708@egenix.com> On 2008-05-14 14:15, Jesse Noller wrote: > On Wed, May 14, 2008 at 5:45 AM, Christian Heimes wrote: >> Martin v. L?wis schrieb: >> >>> I'm worried whether it's stable, what user base it has, whether users >> > (other than the authors) are lobbying for inclusion. Statistically, >> > it seems to be not ready yet: it is not even a year old, and has not >> > reached version 1.0 yet. >> >> I'm on Martin's side here. Although I like to see some sort of multi >> processing mechanism in Python 'cause I need it for lots of projects I'm >> against the inclusion of pyprocessing in 2.6 and 3.0. The project isn't >> old and mature enough and it has some competitors like pp (parallel >> processing). >> >> On the one hand the inclusion of a package gives it an unfair advantage >> over similar packages. On the other hand it slows down future >> development because a new feature release must be synced with Python >> releases about every 1.5 years. >> >> -0.5 from me >> >> Christian >> > > I said this in reply to Martin - but the competitors (in my mind) are > not as compelling due to the "alternative" paradigm for application > construction they propose. The processing module is an "easy win" for > us if included. > > Personally - I don't see how inclusion in the stdlib would slow down > development - yes, you have to stick with the same release cycle as > python-core, but if the module is "feature complete" and provides a > stable API as it stands I don't see following python-core timelines as > overly onerous. > > The module itself doesn't change that frequently - the last release in > April was a bugfix release and API consistency change (the API would > have to be locked for inclusion obviously - targeting a 2.7/3.1 > release may be advantageous to achieve this). Why don't you start a parallel-sig and then hash this out with other distributed computing users ? You could then reach a decision by the time 2.7 is scheduled for release and then add the chosen module to the stdlib. The API of the processing module does look simple and nice, but parallel processing is a minefield - esp. when it comes to handling error situations (e.g. a worker failing, network going down, fail-over, etc.). What I'm missing with the processing module is a way to spawn processes on clusters (rather than just on a single machine). In the scientific world, MPI is the standard API of choice for doing parallel processing, so if we're after standards, supporting MPI would seem to be more attractive than the processing module. http://pypi.python.org/pypi/mpi4py In the enterprise world, you often find CORBA based solutions. http://omniorb.sourceforge.net/ And then, of course, you have a gazillion specialized solutions such as PyRO: http://pyro.sourceforge.net/ OTOH, perhaps the stdlib should just include entry-level support for some form of parallel processing, in which case processing does look attractive. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 14 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From jnoller at gmail.com Wed May 14 17:57:51 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 14 May 2008 11:57:51 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482B0951.2020708@egenix.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> Message-ID: <4222a8490805140857y3580102cx45cefb561c84565a@mail.gmail.com> On Wed, May 14, 2008 at 11:46 AM, M.-A. Lemburg wrote: > On 2008-05-14 14:15, Jesse Noller wrote: >> >> On Wed, May 14, 2008 at 5:45 AM, Christian Heimes >> wrote: >>> >>> Martin v. L?wis schrieb: >>> >>>> I'm worried whether it's stable, what user base it has, whether users >>> >>> > (other than the authors) are lobbying for inclusion. Statistically, >>> > it seems to be not ready yet: it is not even a year old, and has not >>> > reached version 1.0 yet. >>> >>> I'm on Martin's side here. Although I like to see some sort of multi >>> processing mechanism in Python 'cause I need it for lots of projects I'm >>> against the inclusion of pyprocessing in 2.6 and 3.0. The project isn't >>> old and mature enough and it has some competitors like pp (parallel >>> processing). >>> >>> On the one hand the inclusion of a package gives it an unfair advantage >>> over similar packages. On the other hand it slows down future >>> development because a new feature release must be synced with Python >>> releases about every 1.5 years. >>> >>> -0.5 from me >>> >>> Christian >>> >> >> I said this in reply to Martin - but the competitors (in my mind) are >> not as compelling due to the "alternative" paradigm for application >> construction they propose. The processing module is an "easy win" for >> us if included. >> >> Personally - I don't see how inclusion in the stdlib would slow down >> development - yes, you have to stick with the same release cycle as >> python-core, but if the module is "feature complete" and provides a >> stable API as it stands I don't see following python-core timelines as >> overly onerous. >> >> The module itself doesn't change that frequently - the last release in >> April was a bugfix release and API consistency change (the API would >> have to be locked for inclusion obviously - targeting a 2.7/3.1 >> release may be advantageous to achieve this). > > Why don't you start a parallel-sig and then hash this out with other > distributed computing users ? > > You could then reach a decision by the time 2.7 is scheduled for release > and then add the chosen module to the stdlib. > > The API of the processing module does look simple and nice, but > parallel processing is a minefield - esp. when it comes to handling > error situations (e.g. a worker failing, network going down, fail-over, > etc.). > > What I'm missing with the processing module is a way to spawn processes > on clusters (rather than just on a single machine). > > In the scientific world, MPI is the standard API of choice for doing > parallel processing, so if we're after standards, supporting MPI > would seem to be more attractive than the processing module. > > http://pypi.python.org/pypi/mpi4py > > In the enterprise world, you often find CORBA based solutions. > > http://omniorb.sourceforge.net/ > > And then, of course, you have a gazillion specialized solutions > such as PyRO: > > http://pyro.sourceforge.net/ > > OTOH, perhaps the stdlib should just include entry-level support > for some form of parallel processing, in which case processing > does look attractive. > > -- > Marc-Andre Lemburg > eGenix.com > Thanks for bringing up something I was going to mention - I am not attempting to "solve" the distributed computing problem with this proposal - you are right in mentioning there's a variety of technologies out there for achieving "true" loosely-coupled distributed computing, including all of those which you pointed out. I am proposing exactly what you mentioned: Entry level parallel processing. The fact that the processing module does have remote capabilities is a bonus: Not core to the proposal. While in a "perfect" world - a system might exist which truly insulates programmers from the difference between local concurrency and distributed systems - the two are really different problems. My concern is the taking advantage of the 8 core machine sitting under my desk (or the 10 or so I have in the lab)- the processing module allows me to do that - easily. The module is basic enough to be flexible in other technologies to use with it for highly distributed systems but it is also simple enough to act as an entry point for those people just starting out in the domain. Think of it like the difference between asyncore and Twisted. I could easily see more loosely-coupled-highly-distributed tools being built on top of the basics it has provided. -jesse From amcnabb at mcnabbs.org Wed May 14 18:32:13 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Wed, 14 May 2008 10:32:13 -0600 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482B0951.2020708@egenix.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> Message-ID: <20080514163213.GA2949@mcnabbs.org> On Wed, May 14, 2008 at 05:46:25PM +0200, M.-A. Lemburg wrote: > > What I'm missing with the processing module is a way to spawn processes > on clusters (rather than just on a single machine). > > In the scientific world, MPI is the standard API of choice for doing > parallel processing, so if we're after standards, supporting MPI > would seem to be more attractive than the processing module. Think of the processing module as an alternative to the threading module, not as an alternative to MPI. In Python, multi-threading can be extremely slow. The processing module gives you a way to convert from using multiple threads to using multiple processes. If it made people feel better, maybe it should be called threading2 instead of multiprocessing. The word "processing" seems to make people think of parallel processing and clusters, which is missing the point. Anyway, I would love to see the processing module included in the standard library. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From amcnabb at mcnabbs.org Wed May 14 18:39:23 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Wed, 14 May 2008 10:39:23 -0600 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> Message-ID: <20080514163923.GB2949@mcnabbs.org> On Wed, May 14, 2008 at 08:06:15AM -0400, Jesse Noller wrote: > > Overwhelmingly, many of the python programmers I spoke to are looking > for "a solution" that does not require the alteration of a known > programming paradigm (i.e: threads) to allow them to take advantage of > systems which are not getting "faster" - instead, they are getting > wider. Simply put due to the GIL - pure python applications can not > take advantage of these machines which are now more common than not > without switching to an alternative interpreter - which for many - > myself included - is not an attractive option. On Newegg, there are currently 15 single core processors listed, but there are 57 dual core processors and 52 quad core processors. By the time Python 2.6 comes out, it will be hard to buy a new computer without multiple cores. In my opinion, the sooner Python has a nice simple library for inter-process communication, the better. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: From martin at v.loewis.de Wed May 14 19:29:06 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 14 May 2008 19:29:06 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482B0951.2020708@egenix.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> Message-ID: <482B2162.1010109@v.loewis.de> > In the scientific world, MPI is the standard API of choice for doing > parallel processing, so if we're after standards, supporting MPI > would seem to be more attractive than the processing module. > > http://pypi.python.org/pypi/mpi4py Of course, for MPI, pyprocessing's main functionality (starting new activities) isn't needed - you use the vendor's mpirun binary, which will create as many processes as you wish, following a policy that was set up by the cluster administration, or that you chose in a product-specific manner (e.g. what nodes to involve in the job). If my task was high-performance computing, I would indeed use MPI and ignore pyprocessing. > In the enterprise world, you often find CORBA based solutions. > > http://omniorb.sourceforge.net/ Same here: I would prefer CORBA of pyprocessing when I want a componentized, distributed application. > And then, of course, you have a gazillion specialized solutions > such as PyRO: > > http://pyro.sourceforge.net/ I personally would not use that library, although I know others are very fond of it. Regards, Martin From thomaspinckney3 at gmail.com Wed May 14 19:35:11 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Wed, 14 May 2008 13:35:11 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080514163213.GA2949@mcnabbs.org> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> Message-ID: <06A0D80E-0254-4BF2-87ED-3E6EC92027D4@gmail.com> On May 14, 2008, at 12:32 PM, Andrew McNabb wrote: >> > > Think of the processing module as an alternative to the threading > module, not as an alternative to MPI. In Python, multi-threading > can be > extremely slow. The processing module gives you a way to convert from > using multiple threads to using multiple processes. > > If it made people feel better, maybe it should be called threading2 > instead of multiprocessing. The word "processing" seems to make > people > think of parallel processing and clusters, which is missing the point. > > Anyway, I would love to see the processing module included in the > standard library. > Is the goal of the pyprocessing module to be exactly drop in compatible with threading, Queue and friends? I guess the idea would be that if my problem is compute bound I'd use pyprocessing and if it was I/O bound I might just use the existing threading library? Can I write a program using only threading and Queue interfaces for inter-thread communication and just change my import statements and have my program work? Currently, it looks like the pyprocessing.Queue interface is slightly different than than Queue.Queue for example (no task_done() etc). Perhaps a stdlib version of pyprocessing could be simplified down to not try to be a cross-machine computing environment and just be a same- machine threading replacement? This would make the maintenance easier and reduce confusion with users about how they should do cross-machine multiprocessing. By the way, a thread-safe simple dict in the style of Queue would be extremely helpful in writing multi-threaded programs, whether using the threading or pyprocessing modules. From martin at v.loewis.de Wed May 14 19:52:58 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 14 May 2008 19:52:58 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> Message-ID: <482B26FA.6060007@v.loewis.de> > I really do feel that inclusion of this library offers us the best of > both worlds - it gives us (as a community) an easy answer to those > people who would dismiss python due to the GIL and it also allows > users to easily implement their applications. I really feel that you can get the best of both worlds even without inclusion of the module in the standard library. It should be fairly easy to install, assuming pre-compiled packages are available. For inclusion into Python, a number of things need to be changed, in particular with respect to the C code. It duplicates a lot of code from the socket and os modules, and should (IMO) be merged into these, rather than being separate - or perhaps merged into the subprocess module. Why isn't it possible to implement all this in pure Python, on top of the subprocess module? If there are limitations in Python that make it impossible to implement such functionality in pure Python - then *those* limitations should be overcome, rather than including the code wholesale. Regards, Martin From charlesc-lists-python-dev at pyropus.ca Wed May 14 19:55:58 2008 From: charlesc-lists-python-dev at pyropus.ca (Charles Cazabon) Date: 14 May 2008 11:55:58 -0600 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080514163213.GA2949@mcnabbs.org> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> Message-ID: <20080514175558.GA20779@pyropus.ca> Andrew McNabb wrote: > > Think of the processing module as an alternative to the threading > module, not as an alternative to MPI. In Python, multi-threading can be > extremely slow. The processing module gives you a way to convert from > using multiple threads to using multiple processes. Indeed; pyprocessing was exactly what I wanted, and worked exactly as it said it did. It had essentially no learning curve at all because of its implementation of essentially the same interface as the threading module. It's remoting capabilities are almost surplus to requirements; if I wanted that, I might use MPI or something else. > If it made people feel better, maybe it should be called threading2 > instead of multiprocessing. The word "processing" seems to make people > think of parallel processing and clusters, which is missing the point. "threading" is to threads as "processing" is to processes; that's why it was named processing. But the choice of name shouldn't affect the decision as to whether it should be included or not. > Anyway, I would love to see the processing module included in the > standard library. I would as well; I'm using it in a current project, and can see opportunities for it to be useful in other projects. My only note is that based on my experience, the code needs a little cleanup for portability reasons before it is included with the Python standard library -- it worked fine as-is on Linux, but needed some hackery to get running on a slightly elderly Solaris 8 box. I didn't test it on anything else. Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From greg.ewing at canterbury.ac.nz Thu May 15 02:07:38 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 May 2008 12:07:38 +1200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482B0951.2020708@egenix.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> Message-ID: <482B7ECA.9090309@canterbury.ac.nz> M.-A. Lemburg wrote: > The API of the processing module does look simple and nice, but > parallel processing is a minefield - esp. when it comes to handling > error situations (e.g. a worker failing, network going down, fail-over, > etc.). > > What I'm missing with the processing module is a way to spawn processes > on clusters (rather than just on a single machine). Perhaps one-size-fits-all isn't the right approach here. I think there's room for more than one module -- a simple one for people who just want to spawn some extra processes on the same CPU to take advantage of multiple cores, and a fancier one (maybe based on MPI) for people who want grid-computing style distribution with error handling, fault tolerance, etc. (I didn't set out to justify that paragraph, btw -- it just happened!) -- Greg From greg.ewing at canterbury.ac.nz Thu May 15 02:19:21 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 May 2008 12:19:21 +1200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080514163213.GA2949@mcnabbs.org> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> Message-ID: <482B8189.2090300@canterbury.ac.nz> Andrew McNabb wrote: > If it made people feel better, maybe it should be called threading2 > instead of multiprocessing. I think that errs in the other direction, making it sound like just another way of doing single-process threading, which it's not. Maybe "multicore" would help give the right impression? (Still allows for networking -- nobody says all the cores have to be on the same machine.:-) -- Greg From greg.ewing at canterbury.ac.nz Thu May 15 02:45:41 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 May 2008 12:45:41 +1200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080514175558.GA20779@pyropus.ca> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> <20080514175558.GA20779@pyropus.ca> Message-ID: <482B87B5.9000007@canterbury.ac.nz> Charles Cazabon wrote: > "threading" is to threads as "processing" is to processes; that's why it was > named processing. Unfortunately, the word "processing" is already used in the field of computing with a very general meaning -- any kind of data transfomation at all can be, and is, referred to as "processing". So the intended meaning fails to come across. -- Greg From skip at pobox.com Thu May 15 03:44:35 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 14 May 2008 20:44:35 -0500 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482B8189.2090300@canterbury.ac.nz> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> <482B8189.2090300@canterbury.ac.nz> Message-ID: <18475.38275.762515.598355@montanaro-dyndns-org.local> Greg> Maybe "multicore" would help give the right impression? "multiproc", "multiprocess"? Skip From pje at telecommunity.com Thu May 15 03:48:15 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 May 2008 21:48:15 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482B8189.2090300@canterbury.ac.nz> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> <482B8189.2090300@canterbury.ac.nz> Message-ID: <20080515014752.B836E3A4061@sparrow.telecommunity.com> At 12:19 PM 5/15/2008 +1200, Greg Ewing wrote: >Andrew McNabb wrote: > >>If it made people feel better, maybe it should be called threading2 >>instead of multiprocessing. > >I think that errs in the other direction, making it sound >like just another way of doing single-process threading, >which it's not. > >Maybe "multicore" would help give the right impression? Sounds like a marketing win to me, since it directly addresses the "python doesn't do multicore" meme. From greg at krypto.org Thu May 15 05:55:57 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 14 May 2008 20:55:57 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080515014752.B836E3A4061@sparrow.telecommunity.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> <482B8189.2090300@canterbury.ac.nz> <20080515014752.B836E3A4061@sparrow.telecommunity.com> Message-ID: <52dc1c820805142055i77533320k8e2a9f6ba1c8f0c1@mail.gmail.com> On Wed, May 14, 2008 at 6:48 PM, Phillip J. Eby wrote: > At 12:19 PM 5/15/2008 +1200, Greg Ewing wrote: >> >> Andrew McNabb wrote: >> >>> If it made people feel better, maybe it should be called threading2 >>> instead of multiprocessing. >> >> I think that errs in the other direction, making it sound >> like just another way of doing single-process threading, >> which it's not. >> >> Maybe "multicore" would help give the right impression? > > Sounds like a marketing win to me, since it directly addresses the "python > doesn't do multicore" meme. > -1 on "multicore" - multiprocess or multiprocessing are a fine names. cores are irrelevant. systems have multiple cpus real or virtual regardless of how many dies, sockets and cores there are. +0.5 on inclusion. that means i am happy if it does but don't think it needs to make it into 2.6/3.0. leave inclusion for 2.7/3.1. its easy for people to install from an external source for now if they want it. -gps From martin at v.loewis.de Thu May 15 10:05:21 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 15 May 2008 10:05:21 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <52dc1c820805142055i77533320k8e2a9f6ba1c8f0c1@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> <482B8189.2090300@canterbury.ac.nz> <20080515014752.B836E3A4061@sparrow.telecommunity.com> <52dc1c820805142055i77533320k8e2a9f6ba1c8f0c1@mail.gmail.com> Message-ID: <482BEEC1.9050203@v.loewis.de> > -1 on "multicore" - multiprocess or multiprocessing are a fine names. > cores are irrelevant. Thanks you for saying that. I'm constantly amazed how people think multi-core systems are a new thing, conceptually, when they are really just a different packaging for multi-processor systems (perhaps with NUMA consequences that the operating system, but not application, needs to deal with). So -1 on "multicore" from me as well, despise marketing (typo intentional :-) Regards, Martin From ncoghlan at gmail.com Thu May 15 12:10:01 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 May 2008 20:10:01 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <52dc1c820805142055i77533320k8e2a9f6ba1c8f0c1@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> <482B8189.2090300@canterbury.ac.nz> <20080515014752.B836E3A4061@sparrow.telecommunity.com> <52dc1c820805142055i77533320k8e2a9f6ba1c8f0c1@mail.gmail.com> Message-ID: <482C0BF9.60108@gmail.com> Gregory P. Smith wrote: > -1 on "multicore" - multiprocess or multiprocessing are a fine names. > cores are irrelevant. systems have multiple cpus real or virtual > regardless of how many dies, sockets and cores there are. > > +0.5 on inclusion. that means i am happy if it does but don't think > it needs to make it into 2.6/3.0. leave inclusion for 2.7/3.1. its > easy for people to install from an external source for now if they > want it. This is a nice summary of my opinion as well. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From lists at cheimes.de Thu May 15 12:48:15 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 15 May 2008 12:48:15 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <52dc1c820805142055i77533320k8e2a9f6ba1c8f0c1@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <482AB4B3.7020409@cheimes.de> <4222a8490805140515m4e010218l3c23ace5ac008ca2@mail.gmail.com> <482B0951.2020708@egenix.com> <20080514163213.GA2949@mcnabbs.org> <482B8189.2090300@canterbury.ac.nz> <20080515014752.B836E3A4061@sparrow.telecommunity.com> <52dc1c820805142055i77533320k8e2a9f6ba1c8f0c1@mail.gmail.com> Message-ID: <482C14EF.4000202@cheimes.de> Gregory P. Smith schrieb: > +0.5 on inclusion. that means i am happy if it does but don't think > it needs to make it into 2.6/3.0. leave inclusion for 2.7/3.1. its > easy for people to install from an external source for now if they > want it. I'm still -0.5 on inclusion *NOW*, but +1 on inclusion in 2.7/3.1. IMHO pyprocessing isn't mature enough - the author has labeled it as 'beta' - and the proposal came too late. It's going to take a while to give the package a security audit and integrate the C code into Python. Christian From mhammond at skippinet.com.au Thu May 15 13:37:25 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu, 15 May 2008 21:37:25 +1000 Subject: [Python-Dev] rename of ConfigParser module? Message-ID: <003001c8b680$0c7ff280$257fd780$@com.au> Trying to build pywin32 from the trunk, I see: from distutils.config import PyPIRCCommand File "C:\src\python-svn\lib\distutils\config.py", line 8, in from ConfigParser import ConfigParser ImportError: No module named ConfigParser Digging a little deeper, I see: |r63242 | alexandre.vassalotti | 2008-05-15 08:07:07 +1000 (Thu, 15 May 2008) | 2 lines | |Renamed the ConfigParser module to 'configparser'. Is there something I've missed? Cheers, Mark From lxander.m at gmail.com Thu May 15 14:57:59 2008 From: lxander.m at gmail.com (Alexander Michael) Date: Thu, 15 May 2008 08:57:59 -0400 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <003001c8b680$0c7ff280$257fd780$@com.au> References: <003001c8b680$0c7ff280$257fd780$@com.au> Message-ID: <525f23e80805150557m6b441865xf2c8c74453e5903@mail.gmail.com> On Thu, May 15, 2008 at 7:37 AM, Mark Hammond wrote: > Trying to build pywin32 from the trunk, I see: > > from distutils.config import PyPIRCCommand > File "C:\src\python-svn\lib\distutils\config.py", line 8, in > from ConfigParser import ConfigParser > ImportError: No module named ConfigParser > > Digging a little deeper, I see: > > |r63242 | alexandre.vassalotti | 2008-05-15 08:07:07 +1000 (Thu, 15 May > 2008) | 2 lines > | > |Renamed the ConfigParser module to 'configparser'. > > Is there something I've missed? ? From ncoghlan at gmail.com Thu May 15 15:48:21 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 May 2008 23:48:21 +1000 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <003001c8b680$0c7ff280$257fd780$@com.au> References: <003001c8b680$0c7ff280$257fd780$@com.au> Message-ID: <482C3F25.5060700@gmail.com> Mark Hammond wrote: > Trying to build pywin32 from the trunk, I see: > > from distutils.config import PyPIRCCommand > File "C:\src\python-svn\lib\distutils\config.py", line 8, in > from ConfigParser import ConfigParser > ImportError: No module named ConfigParser > > Digging a little deeper, I see: > > |r63242 | alexandre.vassalotti | 2008-05-15 08:07:07 +1000 (Thu, 15 May > 2008) | 2 lines > | > |Renamed the ConfigParser module to 'configparser'. > > Is there something I've missed? My money would be on the warning from the build process that Modules/Setup.dist is newer than Modules/Setup :) To keep NTFS and any other case insensitive filesystems from choking on module renames which only differ by case, the old names have been moved to the lib-old directory. You need the newer Setup file to get that directory to appear in sys.path. (Hmm, is changing Modules/Setup enough to sort the Windows build out as well? Or does that need a separate change to some of the Visual Studio files?) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From thomaspinckney3 at gmail.com Thu May 15 17:56:01 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Thu, 15 May 2008 11:56:01 -0400 Subject: [Python-Dev] Python parallel benchmark Message-ID: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> All the discussion recently about pyprocessing got me interested in actually benchmarking Python's multiprocessing performance to see if reality matched my expectations around what would scale up and what would not. I knew Python threads wouldn't be good for compute bound problems, but I was curious to see how well they worked for i/o bound problems. The short answer is that for i/o bound problems, python threads worked just as well as using multiple operating system processes. I wrote two simple benchmarks, one compute bound and the other i/o bound. The compute bound one did a parallel matrix multiply and the i/ o bound one read random records from a remote MySQL database. I ran each benchmark via python's thread module and via MPI (using mpi4py and openmpi and Send()/Recv() for communication). Each test was run multiple times and the numbers were consistent between test runs. I ran the tests on a dual-core Macbook Pro running OS X 10.5 and the included python 2.5.1. 1) Python threads a) compute bound: 1 thread - 16 seconds 2 threads - 21 seconds b) i/o bound: 1 thread -- 13 seconds 4 threads -- 10 seconds 8 threads -- 5 seconds 12 threads - 4 seconds 2) MPI a) compute bound: 1 thread - 17 seconds 2 threads -- 11 seconds b) i/o bound 1 thread -- 13 seconds 4 threads -- 10 seconds 8 threads -- 6 seconds 12 threads -- 4 seconds From amk at amk.ca Thu May 15 22:33:53 2008 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 15 May 2008 16:33:53 -0400 Subject: [Python-Dev] Distutils & configparser rename Message-ID: <20080515203353.GA24080@amk-desktop.matrixgroup.net> Python 2.6 renames the ConfigParser module to be configparser. Distutils imports ConfigParser in various places. I just made a commit updating the import in one places, and then noticed that part of commit r63248, which made the same change, was reverted in order to preserve backward-compatibility. Instead, the default path will include lib-old again to keep the old module name available. I suggest dropping that goal, though. We've preserved compatibility but I'm not aware that anyone uses the Python 2.x Distutils with earlier versions of Python. In particular: * There's no standalone distutils package on PyPI, nor can I find such a package with a general web search. Am I missing it? * I do not see users advising other users to use a later version of Distutils to fix their problems. Is anyone actually benefiting from the effort of maintaining backward compatibility? --amk From musiccomposition at gmail.com Thu May 15 23:54:28 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 15 May 2008 16:54:28 -0500 Subject: [Python-Dev] platform module testing Message-ID: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> At the moment, the test for the platform module merely calls each function. I realize that this is a hard module to test well, but are there some assumptions we can make? For example, if sys.platform is 'java', can it be assumed that java_ver is going to return something which is not empty. When sys.platform is 'darwin' and the gestalt module is present, is it reasonable to require that a empty tuple not be returned? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From martin at v.loewis.de Fri May 16 00:30:28 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 16 May 2008 00:30:28 +0200 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <482C3F25.5060700@gmail.com> References: <003001c8b680$0c7ff280$257fd780$@com.au> <482C3F25.5060700@gmail.com> Message-ID: <482CB984.60007@v.loewis.de> > (Hmm, is changing Modules/Setup enough to sort the Windows build out as > well? Or does that need a separate change to some of the Visual Studio > files?) The latter. Whenever you add, remove, or rename an extension module, you need to adjust the PCbuild files as well. (technically, you also have to adjust PC/os2emx/Makefile, PC/os2vacpp/makefile.omk, and PC/VC6, but these changes are waived). Regards, Martin From ncoghlan at gmail.com Fri May 16 00:43:30 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 16 May 2008 08:43:30 +1000 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> Message-ID: <482CBC92.5050603@gmail.com> Tom Pinckney wrote: > All the discussion recently about pyprocessing got me interested in > actually benchmarking Python's multiprocessing performance to see if > reality matched my expectations around what would scale up and what > would not. I knew Python threads wouldn't be good for compute bound > problems, but I was curious to see how well they worked for i/o bound > problems. The short answer is that for i/o bound problems, python > threads worked just as well as using multiple operating system processes. Interesting - given that your example compute bound problem happened to be a matrix multiply, I'd be curious what the results are when using python threads with numpy to do the same thing (my understanding is that numpy will usually release the GIL while doing serious number-crunching) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri May 16 00:49:22 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 16 May 2008 08:49:22 +1000 Subject: [Python-Dev] Distutils & configparser rename In-Reply-To: <20080515203353.GA24080@amk-desktop.matrixgroup.net> References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> Message-ID: <482CBDF2.3050206@gmail.com> A.M. Kuchling wrote: > Python 2.6 renames the ConfigParser module to be configparser. > > Distutils imports ConfigParser in various places. I just made a > commit updating the import in one places, and then noticed that part > of commit r63248, which made the same change, was reverted in order to > preserve backward-compatibility. Instead, the default path will > include lib-old again to keep the old module name available. Well, lib-old got added back to the path to support external users (since the deprecated ConfigParser name will emit a warning only under the -3 Py3k warning flag). Since it would be nice for the standard library to not emit any warnings with the -3 flag, perhaps distutils should at least be trying the new name first, and only falling back to the old name on an ImportError (assuming we do decide we want to be able to run the 2.6 distutils on older versions of Python). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jnoller at gmail.com Fri May 16 00:50:12 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 15 May 2008 18:50:12 -0400 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> Message-ID: <6554D51A-43B6-41A4-8A11-78F7E0523128@gmail.com> Do you have the code posted someplace for this? I'd like to add it into the tests I am running On May 15, 2008, at 11:56 AM, Tom Pinckney wrote: > All the discussion recently about pyprocessing got me interested in > actually benchmarking Python's multiprocessing performance to see if > reality matched my expectations around what would scale up and what > would not. I knew Python threads wouldn't be good for compute bound > problems, but I was curious to see how well they worked for i/o > bound problems. The short answer is that for i/o bound problems, > python threads worked just as well as using multiple operating > system processes. > > I wrote two simple benchmarks, one compute bound and the other i/o > bound. The compute bound one did a parallel matrix multiply and the > i/o bound one read random records from a remote MySQL database. I > ran each benchmark via python's thread module and via MPI (using > mpi4py and openmpi and Send()/Recv() for communication). Each test > was run multiple times and the numbers were consistent between test > runs. I ran the tests on a dual-core Macbook Pro running OS X 10.5 > and the included python 2.5.1. > > 1) Python threads > > a) compute bound: > > 1 thread - 16 seconds > 2 threads - 21 seconds > > b) i/o bound: > > 1 thread -- 13 seconds > 4 threads -- 10 seconds > 8 threads -- 5 seconds > 12 threads - 4 seconds > > 2) MPI > > a) compute bound: > > 1 thread - 17 seconds > 2 threads -- 11 seconds > > b) i/o bound > > 1 thread -- 13 seconds > 4 threads -- 10 seconds > 8 threads -- 6 seconds > 12 threads -- 4 seconds > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com From eric+python-dev at trueblade.com Fri May 16 00:54:36 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 15 May 2008 18:54:36 -0400 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <6554D51A-43B6-41A4-8A11-78F7E0523128@gmail.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <6554D51A-43B6-41A4-8A11-78F7E0523128@gmail.com> Message-ID: <482CBF2C.9090204@trueblade.com> Jesse Noller wrote: > Do you have the code posted someplace for this? I'd like to add it into > the tests I am running It would also be interesting to see how pyprocessing performs. Eric. From jnoller at gmail.com Fri May 16 00:56:28 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 15 May 2008 18:56:28 -0400 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <482CBF2C.9090204@trueblade.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <6554D51A-43B6-41A4-8A11-78F7E0523128@gmail.com> <482CBF2C.9090204@trueblade.com> Message-ID: On May 15, 2008, at 6:54 PM, Eric Smith wrote: > Jesse Noller wrote: >> Do you have the code posted someplace for this? I'd like to add it >> into the tests I am running > > It would also be interesting to see how pyprocessing performs. > > Eric. I'm working on exactly that - I have some I/O work - mandlebrot crunching, prime crunching and othe tests to run with threads, pyprocessing, parallel python (as soon as I figure out why it's hitting open file handle issues) and single threaded -Jesse From brett at python.org Fri May 16 01:02:47 2008 From: brett at python.org (Brett Cannon) Date: Thu, 15 May 2008 16:02:47 -0700 Subject: [Python-Dev] platform module testing In-Reply-To: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> References: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> Message-ID: On Thu, May 15, 2008 at 2:54 PM, Benjamin Peterson wrote: > At the moment, the test for the platform module merely calls each > function. I realize that this is a hard module to test well, but are > there some assumptions we can make? For example, if sys.platform is > 'java', can it be assumed that java_ver is going to return something > which is not empty. When sys.platform is 'darwin' and the gestalt > module is present, is it reasonable to require that a empty tuple not > be returned? > Sounds reasonable to me. Probably the best way to verify is to do it and see what the buildbots say. -Brett From brett at python.org Fri May 16 01:04:45 2008 From: brett at python.org (Brett Cannon) Date: Thu, 15 May 2008 16:04:45 -0700 Subject: [Python-Dev] Distutils & configparser rename In-Reply-To: <20080515203353.GA24080@amk-desktop.matrixgroup.net> References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> Message-ID: On Thu, May 15, 2008 at 1:33 PM, A.M. Kuchling wrote: > Python 2.6 renames the ConfigParser module to be configparser. > > Distutils imports ConfigParser in various places. I just made a > commit updating the import in one places, and then noticed that part > of commit r63248, which made the same change, was reverted in order to > preserve backward-compatibility. Instead, the default path will > include lib-old again to keep the old module name available. > > I suggest dropping that goal, though. We've preserved compatibility > but I'm not aware that anyone uses the Python 2.x Distutils with > earlier versions of Python. In particular: > > * There's no standalone distutils package on PyPI, nor can I find > such a package with a general web search. Am I missing it? > > * I do not see users advising other users to use a later version of > Distutils to fix their problems. > > Is anyone actually benefiting from the effort of maintaining backward > compatibility? The change was reverted at MAL's request, but he didn't qualify it beyond wanting the backwards-compatibility. -Brett From lists at cheimes.de Fri May 16 01:09:18 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 16 May 2008 01:09:18 +0200 Subject: [Python-Dev] platform module testing In-Reply-To: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> References: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> Message-ID: <482CC29E.3050204@cheimes.de> Benjamin Peterson schrieb: > At the moment, the test for the platform module merely calls each > function. I realize that this is a hard module to test well, but are > there some assumptions we can make? For example, if sys.platform is > 'java', can it be assumed that java_ver is going to return something > which is not empty. When sys.platform is 'darwin' and the gestalt > module is present, is it reasonable to require that a empty tuple not > be returned? A student has created a patch during the GHOP contest which adds some features and tests for Linux. AFAIK the patch hasn't been applied yet. See http://bugs.python.org/issue1322 Christian From lists at cheimes.de Fri May 16 01:04:42 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 16 May 2008 01:04:42 +0200 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <482CB984.60007@v.loewis.de> References: <003001c8b680$0c7ff280$257fd780$@com.au> <482C3F25.5060700@gmail.com> <482CB984.60007@v.loewis.de> Message-ID: <482CC18A.8030801@cheimes.de> Martin v. L?wis schrieb: >> (Hmm, is changing Modules/Setup enough to sort the Windows build out as >> well? Or does that need a separate change to some of the Visual Studio >> files?) > > The latter. Whenever you add, remove, or rename an extension module, you > need to adjust the PCbuild files as well. (technically, you also have to > adjust PC/os2emx/Makefile, PC/os2vacpp/makefile.omk, and PC/VC6, but > these changes are waived). Don't forget PC/config.c and the build files for VC 6, VS 7.1 and VS 8.0 in the PC/ directory. :] Christian From lists at cheimes.de Fri May 16 01:25:04 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 16 May 2008 01:25:04 +0200 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <482CC3D4.1010406@v.loewis.de> References: <003001c8b680$0c7ff280$257fd780$@com.au> <482C3F25.5060700@gmail.com> <482CB984.60007@v.loewis.de> <482CC18A.8030801@cheimes.de> <482CC3D4.1010406@v.loewis.de> Message-ID: <482CC650.6090009@cheimes.de> Martin v. L?wis schrieb: >> Don't forget PC/config.c and the build files for VC 6, VS 7.1 and VS 8.0 >> in the PC/ directory. :] > > Aren't the 7.1 and 8.0 files generated? Only the 8.0 files are generated from the 9.0 files. The 6.0 and 7.1 files still require manual work. It might be possible to create the 7.1 files from the 9.0 as well. But nobody has written a tool so far. Christian From martin at v.loewis.de Fri May 16 01:14:28 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 16 May 2008 01:14:28 +0200 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <482CC18A.8030801@cheimes.de> References: <003001c8b680$0c7ff280$257fd780$@com.au> <482C3F25.5060700@gmail.com> <482CB984.60007@v.loewis.de> <482CC18A.8030801@cheimes.de> Message-ID: <482CC3D4.1010406@v.loewis.de> > Don't forget PC/config.c and the build files for VC 6, VS 7.1 and VS 8.0 > in the PC/ directory. :] Aren't the 7.1 and 8.0 files generated? Regards, Martin From mhammond at skippinet.com.au Fri May 16 01:35:52 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri, 16 May 2008 09:35:52 +1000 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <482CB984.60007@v.loewis.de> References: <003001c8b680$0c7ff280$257fd780$@com.au> <482C3F25.5060700@gmail.com> <482CB984.60007@v.loewis.de> Message-ID: <009a01c8b6e4$6eca75a0$4c5f60e0$@com.au> > > (Hmm, is changing Modules/Setup enough to sort the Windows build out > > as > > well? Or does that need a separate change to some of the Visual > > Studio files?) > > The latter. Whenever you add, remove, or rename an extension module, > you need to adjust the PCbuild files as well. (technically, you also > have to adjust PC/os2emx/Makefile, PC/os2vacpp/makefile.omk, and PC/VC6, > but these changes are waived). Correct me if I'm wrong, but in this case we are talking about adding a new directory to sys.path, correct? On Windows, this means a change in PCBuild/pyconfig.h and tools/msi/msi.py. A patch seems simple, so I'll check one in (assuming noone beats me) after clarification of a couple of points. * I assume lib-old should go at the end of the sys.path managed by the above files? * What is the story with "plat-win"? pyconfig.h includes a 'Lib\plat-win' directory, and although this directory does not exist, it usually ends up on sys.path. msi.py does *not* include this entry, which means it doesn't end up on sys.path in certain embedding environments (specifically, when the hosting application is using a python.dll located outside the Python directory). I'm sure there are pros and cons for including that directory, but I can't see justification for it to sometimes be used and sometimes not. So should I change msi.py to include this directory, or change pyconfig.h to not include it? Thanks, Mark From jek-gmane at kleckner.net Fri May 16 01:44:38 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Thu, 15 May 2008 16:44:38 -0700 Subject: [Python-Dev] Visual Studio 2008 compiler option EHsc ? Message-ID: In building a package with several platforms, I ran across the warning message below from Visual Studio 2008. Should we add the /EHsc option to the compile_options in distutils for MSVC? Or is it more complex than that... This link describes the warning: http://tinyurl.com/4fmjue http://msdn.microsoft.com/en-us/library/2axwkyt4.aspx This link describes a patch: http://tinyurl.com/63bqo2 http://projects.scipy.org/pipermail/numpy-discussion/2007-January/025527.html === Warning message: C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\xstring(2092) : warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\xstring(2083) : while compiling class template member function 'void std::basic_string<_Elem,_Traits,_Ax>::_Copy(unsigned int,unsigned int)' with [ _Elem=char, _Traits=std::char_traits, _Ax=std::allocator ] C:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\xstring(2221) : see reference to class template instantiation 'std::basic_string<_Elem,_Traits,_Ax>' being compiled with [ _Elem=char, _Traits=std::char_traits, _Ax=std::allocator ] From brett at python.org Fri May 16 02:05:46 2008 From: brett at python.org (Brett Cannon) Date: Thu, 15 May 2008 17:05:46 -0700 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <009a01c8b6e4$6eca75a0$4c5f60e0$@com.au> References: <003001c8b680$0c7ff280$257fd780$@com.au> <482C3F25.5060700@gmail.com> <482CB984.60007@v.loewis.de> <009a01c8b6e4$6eca75a0$4c5f60e0$@com.au> Message-ID: On Thu, May 15, 2008 at 4:35 PM, Mark Hammond wrote: >> > (Hmm, is changing Modules/Setup enough to sort the Windows build out >> > as >> > well? Or does that need a separate change to some of the Visual >> > Studio files?) >> >> The latter. Whenever you add, remove, or rename an extension module, >> you need to adjust the PCbuild files as well. (technically, you also >> have to adjust PC/os2emx/Makefile, PC/os2vacpp/makefile.omk, and PC/VC6, >> but these changes are waived). > > Correct me if I'm wrong, but in this case we are talking about adding a new > directory to sys.path, correct? On Windows, this means a change in > PCBuild/pyconfig.h and tools/msi/msi.py. A patch seems simple, so I'll > check one in (assuming noone beats me) after clarification of a couple of > points. > > * I assume lib-old should go at the end of the sys.path managed by the above > files? > Yes. -Brett From thomaspinckney3 at gmail.com Fri May 16 03:02:29 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Thu, 15 May 2008 21:02:29 -0400 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <482CBC92.5050603@gmail.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <482CBC92.5050603@gmail.com> Message-ID: <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> I switched to using numpy for the matrix multiply and while the overall time to do the matrix multiply is much faster, there is still no speed up from using more than one python thread. If I look at top while running 2 or more threads, both cores are being used 100% and there is no idle time on the system. I did a quick google search and didn't find anything conclusive about numpy releasing the GIL. The most conclusive and recent reference I found was http://mail.python.org/pipermail/python-list/2007-October/463148.html I found some other references where people were expressing concern over numpy releasing the GIL due to the fact that other C extensions could call numpy and unexpectedly have the GIL released on them (or something like that). On May 15, 2008, at 6:43 PM, Nick Coghlan wrote: > Tom Pinckney wrote: >> All the discussion recently about pyprocessing got me interested in >> actually benchmarking Python's multiprocessing performance to see >> if reality matched my expectations around what would scale up and >> what would not. I knew Python threads wouldn't be good for compute >> bound problems, but I was curious to see how well they worked for i/ >> o bound problems. The short answer is that for i/o bound problems, >> python threads worked just as well as using multiple operating >> system processes. > > Interesting - given that your example compute bound problem happened > to be a matrix multiply, I'd be curious what the results are when > using python threads with numpy to do the same thing (my > understanding is that numpy will usually release the GIL while doing > serious number-crunching) > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org From tnelson at onresolve.com Wed May 14 16:27:54 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 14 May 2008 15:27:54 +0100 Subject: [Python-Dev] PEP 370 extras References: <4823791F.2050507@cheimes.de> <57897D5F-D6F6-4A3D-80D3-FA0A367657EB@python.org> Message-ID: <6167796BFEB5D0438720AC212E89A6B0078726F3@exchange.onresolve.com> > Christian, you get this week's Awesome Award. Hah, 'Weekly Awesome Award', we should make that one of our things, like QOTW. Trent. From tnelson at onresolve.com Wed May 14 17:08:53 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 14 May 2008 16:08:53 +0100 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? Message-ID: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> I was about to commit an initial import of the bsddb 4.6.21 source to the 'external' area in the repo, which I obtained from the following URL: http://www.oracle.com/technology/software/products/berkeley-db/index.html I downloaded the source that includes AES encryption, for no reason other than it was first on the list. I'm now wondering if we should only be importing the 'NC' source that doesn't contain any encryption? Jesus, does pybsddb use any of the Berkeley DB encryption facilities? Would anything break if we built the bsddb module without encryption? (Note that all of this only applies to Windows.) Trent. From alexandre at peadrop.com Fri May 16 03:59:22 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Thu, 15 May 2008 21:59:22 -0400 Subject: [Python-Dev] Distutils & configparser rename In-Reply-To: <482CBDF2.3050206@gmail.com> References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> <482CBDF2.3050206@gmail.com> Message-ID: On Thu, May 15, 2008 at 6:49 PM, Nick Coghlan wrote: > Since it would be nice for the standard library to not emit any warnings > with the -3 flag, perhaps distutils should at least be trying the new name > first, and only falling back to the old name on an ImportError (assuming we > do decide we want to be able to run the 2.6 distutils on older versions of > Python). > Well, that is a good idea. And, that will silence the Windows buildbots while other developers find out how to add lib-old/ to the sys.path. -- Alexandre From greg.ewing at canterbury.ac.nz Fri May 16 04:55:03 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 May 2008 14:55:03 +1200 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <482CBC92.5050603@gmail.com> <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> Message-ID: <482CF787.70703@canterbury.ac.nz> Tom Pinckney wrote: > If I look at top while > running 2 or more threads, both cores are being used 100% and there is > no idle time on the system. If you run it with just one thread, does it use up only one core's worth of CPU? If so, this suggests that the GIL is being released. If it wasn't, two threads would still only use one core's worth. Also, if you have only two cores, using more than two threads isn't going to gain you anything whatever happens. -- Greg From greg at krypto.org Fri May 16 06:04:42 2008 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 15 May 2008 21:04:42 -0700 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> Message-ID: <52dc1c820805152104m54ab85d8r6d2e44284eaab8c3@mail.gmail.com> In the past I believe we've built it with encryption. Regardless, we already ship encryption with Python thanks to the ssl module and I'm assuming the PSF has taken care of the necessary silly export document filing for the US so I see no reason to exclude it from bsddb. On Wed, May 14, 2008 at 8:08 AM, Trent Nelson wrote: > I was about to commit an initial import of the bsddb 4.6.21 source to the 'external' area in the repo, which I obtained from the following URL: > > http://www.oracle.com/technology/software/products/berkeley-db/index.html > > I downloaded the source that includes AES encryption, for no reason other than it was first on the list. I'm now wondering if we should only be importing the 'NC' source that doesn't contain any encryption? Jesus, does pybsddb use any of the Berkeley DB encryption facilities? Would anything break if we built the bsddb module without encryption? > > (Note that all of this only applies to Windows.) > > > Trent. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > From greg at krypto.org Fri May 16 06:08:13 2008 From: greg at krypto.org (Gregory P. Smith) Date: Thu, 15 May 2008 21:08:13 -0700 Subject: [Python-Dev] Distutils & configparser rename In-Reply-To: References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> Message-ID: <52dc1c820805152108wc386902u7cacd9e7f5ae812d@mail.gmail.com> On Thu, May 15, 2008 at 4:04 PM, Brett Cannon wrote: > On Thu, May 15, 2008 at 1:33 PM, A.M. Kuchling wrote: >> Python 2.6 renames the ConfigParser module to be configparser. >> >> Distutils imports ConfigParser in various places. I just made a >> commit updating the import in one places, and then noticed that part >> of commit r63248, which made the same change, was reverted in order to >> preserve backward-compatibility. Instead, the default path will >> include lib-old again to keep the old module name available. >> >> I suggest dropping that goal, though. We've preserved compatibility >> but I'm not aware that anyone uses the Python 2.x Distutils with >> earlier versions of Python. In particular: >> >> * There's no standalone distutils package on PyPI, nor can I find >> such a package with a general web search. Am I missing it? >> >> * I do not see users advising other users to use a later version of >> Distutils to fix their problems. >> >> Is anyone actually benefiting from the effort of maintaining backward >> compatibility? > > The change was reverted at MAL's request, but he didn't qualify it > beyond wanting the backwards-compatibility. > > -Brett backwards compatibility? whats wrong with just doing: try: import configparser as ConfigParser except ImportError: import ConfigParser in the distutils code that needs to be. From thomaspinckney3 at gmail.com Fri May 16 06:33:08 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Fri, 16 May 2008 00:33:08 -0400 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <482CF787.70703@canterbury.ac.nz> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <482CBC92.5050603@gmail.com> <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> <482CF787.70703@canterbury.ac.nz> Message-ID: Interestingly, I think there's something magic going on with numpy.dot() on my mac. If I just run a program without threading--that is just a numpy matrix multiply such as: import numpy a = numpy.empty((4000,4000)) b = numpy.empty((4000,4000)) c = numpy.dot(a,b) then I see both cores fully maxed out on my mac. On a dual-core linux machine I see only one core maxed out by this program and it runs VASTLY slower on the linux box. It turns out that numpy on Mac's uses Apple's Accelerate.framekwork BLAS and LAPACK which in turn is multi-threaded as of OS X 10.4.8. On May 15, 2008, at 10:55 PM, Greg Ewing wrote: > Tom Pinckney wrote: >> If I look at top while running 2 or more threads, both cores are >> being used 100% and there is no idle time on the system. > > If you run it with just one thread, does it use up only > one core's worth of CPU? > > If so, this suggests that the GIL is being released. If > it wasn't, two threads would still only use one core's worth. > > Also, if you have only two cores, using more than two > threads isn't going to gain you anything whatever happens. > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/thomaspinckney3%40gmail.com From brett at python.org Fri May 16 07:18:53 2008 From: brett at python.org (Brett Cannon) Date: Thu, 15 May 2008 22:18:53 -0700 Subject: [Python-Dev] Distutils & configparser rename In-Reply-To: <52dc1c820805152108wc386902u7cacd9e7f5ae812d@mail.gmail.com> References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> <52dc1c820805152108wc386902u7cacd9e7f5ae812d@mail.gmail.com> Message-ID: On Thu, May 15, 2008 at 9:08 PM, Gregory P. Smith wrote: > On Thu, May 15, 2008 at 4:04 PM, Brett Cannon wrote: >> On Thu, May 15, 2008 at 1:33 PM, A.M. Kuchling wrote: >>> Python 2.6 renames the ConfigParser module to be configparser. >>> >>> Distutils imports ConfigParser in various places. I just made a >>> commit updating the import in one places, and then noticed that part >>> of commit r63248, which made the same change, was reverted in order to >>> preserve backward-compatibility. Instead, the default path will >>> include lib-old again to keep the old module name available. >>> >>> I suggest dropping that goal, though. We've preserved compatibility >>> but I'm not aware that anyone uses the Python 2.x Distutils with >>> earlier versions of Python. In particular: >>> >>> * There's no standalone distutils package on PyPI, nor can I find >>> such a package with a general web search. Am I missing it? >>> >>> * I do not see users advising other users to use a later version of >>> Distutils to fix their problems. >>> >>> Is anyone actually benefiting from the effort of maintaining backward >>> compatibility? >> >> The change was reverted at MAL's request, but he didn't qualify it >> beyond wanting the backwards-compatibility. >> >> -Brett > > backwards compatibility? whats wrong with just doing: > > try: > import configparser as ConfigParser > except ImportError: > import ConfigParser > > in the distutils code that needs to be. > Nothing's wrong with it. It's just that Alexandre did what we have done for all of the stdlib and just moved entirely over to the new name. -Brett From martin at v.loewis.de Fri May 16 07:27:32 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 16 May 2008 07:27:32 +0200 Subject: [Python-Dev] Visual Studio 2008 compiler option EHsc ? In-Reply-To: References: Message-ID: <482D1B44.3090507@v.loewis.de> > In building a package with several platforms, I > ran across the warning message below from Visual > Studio 2008. Should we add the /EHsc option to the > compile_options in distutils for MSVC? Or is it more > complex than that... Who is "we"? If you have a module that uses C++ exceptions, you should certainly enable compiler support for exceptions, for that module. Regards, Martin From brett at python.org Fri May 16 07:39:30 2008 From: brett at python.org (Brett Cannon) Date: Thu, 15 May 2008 22:39:30 -0700 Subject: [Python-Dev] Help with finishing PEP 3108 Message-ID: I need help to finish implementing PEP 3108. While over 80 modules are now deprecated in Python 2.6 (of which I did over 50 of), there are still over 20 tasks left to do in relation to the PEP. My free time is being sucked away since I have a conference paper deadline of June 1. And I am moving May 31. And I have to fly down to California to help my mother move on June 4. And other personal stuff (see a certain trend in my life at the moment?). So if you have time to help, please see issue 2775 (http://bugs.python.org/issue2775) and the dependencies list. The issues range from doing patch reviews of work people have already done, renaming a module, creating a new package, removing some use from the stdlib, or even backporting some changes made to 3.0 that were never merged into 2.6. In other words a wide variety of things. =) The PEP outlines the steps necessary to deprecate a module for deletion or for a rename in a step-by-step manner so you don't need to worry about forgetting a step. If you can't choose what to do, the issues that will lead to a module be deleted are the highest priority as renames can be handled by 2to3 in a later version while module deletions are harder to get pushed through and accepted. The modules left to still remove are still there because they are still used somehow in the stdlib. The module renames are mostly done at this point, but the new packages have not been handled yet. Obviously I don't want the beta to be held up by this, nor do I want to see any of the work left out because I couldn't get to it all. So any and all help is appreciated. -Brett From p.f.moore at gmail.com Fri May 16 09:49:22 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 16 May 2008 08:49:22 +0100 Subject: [Python-Dev] [Python-3000] PEP 3138- String representation in Python 3000 In-Reply-To: <797440730805151047t79017a0ds6a1655884ace120e@mail.gmail.com> References: <797440730805060655j3250ab22le3220c76e5403ea4@mail.gmail.com> <48232FB2.3020205@egenix.com> <797440730805091023p75f64540r9071d77247d155fd@mail.gmail.com> <482B10DF.50105@egenix.com> <797440730805150913h78ddde9bk435959b33a6987ee@mail.gmail.com> <79990c6b0805150949k29d95adfn9fb30f54df618d55@mail.gmail.com> <797440730805151047t79017a0ds6a1655884ace120e@mail.gmail.com> Message-ID: <79990c6b0805160049u60d82614s362a21755c2a0d39@mail.gmail.com> 2008/5/15 Atsuo Ishimoto : > With my proposal, print("Hello\u03C8") prints "Hello\u03C8" instead of > raising an exception. And print(repr("Hello\u03C8")) prints > "'Hello\u03C8'", so no garbage are printed. > > Now, let's say you are Greek and working on Greek version of XP. > print("Hello\u03C8") prints "Hello"+collect Greek character(GREEK > SMALL LETTER PSI). And print(repr("Hello\u03C8")) prints > "'Hello"+collect Greek character+"'". If you have Greek font, you can > try this if you swich your command prompt by "chcp 1253" (change > codepage to 1253) on your command prompt. > [...] > Python detects user's capabilities, since Python 2.x(or 1.6? I forgot.) > On Windows, Python detects user's encoding from codepage. On Unix, > locale is used to detect encoding. Ah, thanks. I hadn't realised this - I've had trouble printing Unicode in the past, and assumed it was a result of Windows' strange console handling (OEM code pages vs Windows code pages confuse me). I use Unicode so rarely that it wasn't worth worrying about it, though. I guess the problem was my understanding, rather than code page detection not working. Sorry for the confusion. >> Like it or not, a large proportion of Python's users still work in >> environments where much of the Unicode character space is not >> displayed readably. >> > > I agree. So rejecting my proposal as "Not common use-case" might be > reasonable. But I should argue to get sympathy, anyway:). As Oleg pointed out, my comment "a large proportion" was a guess, and an unfounded one at that. And regardless, you definitely have my sympathy, this is an issue that needs solving :-) (Heck, just the fact that you have to write your emails to this group in a foreign language is enough to get you my sympathy!!!) > I can understand your aware. Perhaps you don't want see your terminal > flash by escape sequence, beep, endless graphic characters, etc. For > legacy byte-string applications(whether written in C or Python), > printing arbitrary string can cause such mess. But this is unlikely to > happen by printing the Unicode string, since the characters your > terminal cannot understand will be escaped or be converted to > character such as '?'. Ah, that's what the switching of the error mode is for. I understand more clearly now. > Hope this helps. It does - thanks for being patient with me. Paul. From hrvoje.niksic at avl.com Fri May 16 10:17:25 2008 From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=) Date: Fri, 16 May 2008 10:17:25 +0200 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <482CBC92.5050603@gmail.com> <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> Message-ID: <1210925845.31997.5.camel@localhost> On Thu, 2008-05-15 at 21:02 -0400, Tom Pinckney wrote: > I found some other references where people were expressing concern > over numpy releasing the GIL due to the fact that other C extensions > could call numpy and unexpectedly have the GIL released on them (or > something like that). Could you please post links to those? I'm asking because AFAIK that concern doesn't really stand. Any (correct) code that releases the GIL is responsible for reacquiring it before calling *any* Python code, in fact before doing anything that might touch a Python object or its refcount. From ulrich.berning at denviso.de Fri May 16 10:32:30 2008 From: ulrich.berning at denviso.de (Ulrich Berning) Date: Fri, 16 May 2008 10:32:30 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080514105858.0279814C64B@irishsea.home.craig-wood.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> Message-ID: <482D469E.7080609@denviso.de> Nick Craig-Wood wrote: >Jesse Noller wrote: > > >> I am looking for any questions, concerns or benchmarks python-dev has >> regarding the possible inclusion of the pyprocessing module to the >> standard library - preferably in the 2.6 timeline. In March, I began >> working on the PEP for the inclusion of the pyprocessing (processing) >> module into the python standard library[1]. The original email to the >> stdlib-sig can be found here, it includes a basic overview of the >> module: >> >> http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html >> >> The processing module mirrors/mimics the API of the threading module - >> and with simple import/subclassing changes depending on the code, >> allows you to leverage multi core machines via an underlying forking >> mechanism. The module also supports the sharing of data across groups >> of networked machines - a feature obviously not part of the core >> threading module, but useful in a distributed environment. >> >> > >I think processing looks interesting and useful, especially since it >works on Windows as well as Un*x. > >However I'd like to see a review of the security - anything which can >run across networks of machines has security implications and I didn't >see these spelt out in the documentation. > >Networked running should certainly be disabled by default and need >explicitly enabling by the user - I'd hate for a new version of python >to come with a remote exploit by default... > > > As long as the ctypes extension doesn't build on major Un*x platforms (AIX, HP-UX), I don't like to see ctypes dependend modules included into the stdlib. Please keep the stdlib as portable as possible. More and more people tend to say "Runs on Un*x" when they really mean "Tested on Linux". Un*x is not Linux. Ulli From andymac at bullseye.apana.org.au Fri May 16 11:21:58 2008 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Fri, 16 May 2008 20:21:58 +1100 Subject: [Python-Dev] rename of ConfigParser module? In-Reply-To: <482CB984.60007@v.loewis.de> References: <003001c8b680$0c7ff280$257fd780$@com.au> <482C3F25.5060700@gmail.com> <482CB984.60007@v.loewis.de> Message-ID: <482D5236.1000101@bullseye.andymac.org> Martin v. Lo"wis wrote: >> (Hmm, is changing Modules/Setup enough to sort the Windows build out as >> well? Or does that need a separate change to some of the Visual Studio >> files?) > > The latter. Whenever you add, remove, or rename an extension module, you > need to adjust the PCbuild files as well. (technically, you also have to > adjust PC/os2emx/Makefile, PC/os2vacpp/makefile.omk, and PC/VC6, but > these changes are waived). OS/2 fixes are something I'll address during the betas. -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From ygingras at ygingras.net Fri May 16 12:52:35 2008 From: ygingras at ygingras.net (Yannick Gingras) Date: Fri, 16 May 2008 06:52:35 -0400 Subject: [Python-Dev] Symbolic errno values in error messages Message-ID: <873aoifrm4.fsf@enceladus.ygingras.net> Hi, I spent some time googleing for "OSError 4" before it occurred to me that "4" was actually an irrelevant implementation detail. As soon as I searched for "EINTR", I found exactly what I was looking for. (not really but this is another story) I jumped to the conclusion that OSError.__str__() should return the symbolic error number instead of the integer value. I was about to write a patch but I just noticed that OSError and friends are implemented in C so I guess it will be a bit harder than expected. The error message comes from EnvironmentError.__str__() which is implemented by EnvironmentError_str() in Objects/exceptions.c. A Python implementation could have replaced self.errno in the format string args by errno.errorcode[self.errno]. However, it's not clear to me if there is such a mapping available in C. Since my fix is not as trivial as I expected, I ask for advice regarding the following questions: 1) Should OSError.__str__() print the symbolic name of errno? 2) Where can I find the symbolic name in C? Best regards, -- Yannick Gingras From scott+python-dev at scottdial.com Fri May 16 13:10:42 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 16 May 2008 07:10:42 -0400 Subject: [Python-Dev] Distutils & configparser rename In-Reply-To: <20080515203353.GA24080@amk-desktop.matrixgroup.net> References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> Message-ID: <482D6BB2.7080807@scottdial.com> A.M. Kuchling wrote: > * There's no standalone distutils package on PyPI, nor can I find > such a package with a general web search. Am I missing it? > > * I do not see users advising other users to use a later version of > Distutils to fix their problems. > > Is anyone actually benefiting from the effort of maintaining backward > compatibility? I remember this discussion coming up before the 2.5 release when the distutils version number was changed to depend on the python instance instead of a static string. MAL was made very upset that the SVN trunk version of distutils could no longer be pulled out seperately. But at that time, you had removed it from PEP 291, officially renouncing this practice. I believe the conclusion was that MAL's request to maintain psuedo-backward-compatibility would be upheld despite not having official status. http://mail.python.org/pipermail/python-dev/2006-August/068250.html -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From thomaspinckney3 at gmail.com Fri May 16 14:04:14 2008 From: thomaspinckney3 at gmail.com (Tom Pinckney) Date: Fri, 16 May 2008 08:04:14 -0400 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <1210925845.31997.5.camel@localhost> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <482CBC92.5050603@gmail.com> <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> <1210925845.31997.5.camel@localhost> Message-ID: <6575DBA1-1B98-4F7B-A9EE-9314F23709A8@gmail.com> Here's one example, albeit from a few years ago http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/1625465 But, I am a numpy novice and so no idea what it actually does in its current form. On May 16, 2008, at 4:17 AM, Hrvoje Nik?i? wrote: > On Thu, 2008-05-15 at 21:02 -0400, Tom Pinckney wrote: >> I found some other references where people were expressing concern >> over numpy releasing the GIL due to the fact that other C extensions >> could call numpy and unexpectedly have the GIL released on them (or >> something like that). > > Could you please post links to those? I'm asking because AFAIK that > concern doesn't really stand. Any (correct) code that releases the > GIL > is responsible for reacquiring it before calling *any* Python code, in > fact before doing anything that might touch a Python object or its > refcount. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/thomaspinckney3%40gmail.com From hrvoje.niksic at avl.com Fri May 16 14:14:44 2008 From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=) Date: Fri, 16 May 2008 14:14:44 +0200 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <6575DBA1-1B98-4F7B-A9EE-9314F23709A8@gmail.com> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <482CBC92.5050603@gmail.com> <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> <1210925845.31997.5.camel@localhost> <6575DBA1-1B98-4F7B-A9EE-9314F23709A8@gmail.com> Message-ID: <1210940084.32116.3.camel@localhost> On Fri, 2008-05-16 at 08:04 -0400, Tom Pinckney wrote: > Here's one example, albeit from a few years ago > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/1625465 Thanks for the pointer. I'm not sure I fully understand Konrad Hinsen's concerns, but maybe the problem is that Numpy's "number-crunching" needs to call back into Python frequently. The notion of "releasing the GIL for number-crunching" assumes that the code is structured like this: 1. code that works with python objects ... 2. acquire pointer to a C struct/array ... 3. release GIL 4. work with C objects ("crunch the numbers") without calling any Python code and without touching Python objects or refcounts 5. reacquire GIL If step 4 needs to call into Python frequently, then this strategy won't really work. From matthieu.brucher at gmail.com Fri May 16 14:20:02 2008 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Fri, 16 May 2008 14:20:02 +0200 Subject: [Python-Dev] Python parallel benchmark In-Reply-To: <1210940084.32116.3.camel@localhost> References: <7F11760E-863F-4F20-80FF-0D5D2A2DCA41@gmail.com> <482CBC92.5050603@gmail.com> <95CCDDEC-2A3B-4465-A066-1C7E34BB3C2A@gmail.com> <1210925845.31997.5.camel@localhost> <6575DBA1-1B98-4F7B-A9EE-9314F23709A8@gmail.com> <1210940084.32116.3.camel@localhost> Message-ID: 2008/5/16 Hrvoje Nik?i? : > On Fri, 2008-05-16 at 08:04 -0400, Tom Pinckney wrote: > > Here's one example, albeit from a few years ago > > > > http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/1625465 > > Thanks for the pointer. I'm not sure I fully understand Konrad Hinsen's > concerns, but maybe the problem is that Numpy's "number-crunching" needs > to call back into Python frequently. The notion of "releasing the GIL > for number-crunching" assumes that the code is structured like this: > > 1. code that works with python objects ... > 2. acquire pointer to a C struct/array ... > 3. release GIL > 4. work with C objects ("crunch the numbers") without calling any Python > code and without touching Python objects or refcounts > 5. reacquire GIL > > If step 4 needs to call into Python frequently, then this strategy won't > really work. > Hi, The current version of Numpy releases as soon as possible the GIL. The usual macros for starting and stopping a GIL release (as advertised in the documentation) are present for every step 4 work, whenever is possible (for instance it is the case for universal functions which are used in a lot of numpy's functions). Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Fri May 16 14:40:52 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 16 May 2008 16:40:52 +0400 Subject: [Python-Dev] SSH connection sharing Message-ID: <20080516124052.GB23518@phd.pp.ru> Hello! I would like to share a useful tip on how to speedup many short ssh (scp, rsync, svn over ssh) sessions (on Unix). Sorry if this information is widely known; I learned the trick a week ago and I am pretty happy with it. OpenSSH4 on Unix allows connection sharing - utilizing one authenticated TCP connection for a number of SSH sessions. The advantage is that subsequent connections don't need to open a new TCP connections and don't need to pass the authentication phase, thus they initialize a session much faster. This is very useful for running a lot of short ssh sessions (scp, sftp, rsync, svn over ssh). Detailed docs are in man ssh_config; see options ControlMaster and ControlPath. There is a lot of information_ on the web. .. _information: http://www.google.com/search?hl=en&ie=utf-8&oe=utf-8&q=ssh%20ControlPath Add the following to your ~/.ssh/config:: ControlMaster auto ControlPath ~/.ssh/.%r@%h:%p *ControlMaster auto* means the first client becomes the master, it opens TCP connection, authenticates and opens a unix socket named in ControlPath. Subsequent clients use the unix socket to connect to the server via the already established connection. In case the user tries to close the first (master) client it will wait until all sessions are finished; in case the user kills the master, it kills all its shared sessions. I use all 3 recommended macros, because I use ssh/scp/sftp/rsync/svn to connect to a number of servers. Drawbacks: the subsequent connections inherit some attributes of the master; agent forwarding, e.g. If ssh (or the OS, or even the entire computer) crashes one needs to remove unix sockets manually; to remedy this I changed ControlPath to ~/.ssh/controls/%r\@%h:%p and added to my crontab:: @reboot rm -f .ssh/controls/* Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From mal at egenix.com Fri May 16 14:43:32 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 16 May 2008 14:43:32 +0200 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: References: Message-ID: <482D8174.7030407@egenix.com> On 2008-05-12 04:34, Brett Cannon wrote: > For the sake of argument, let's consider the Queue module. It is now > named queue. For 2.6 I plan on having both Queue and queue listed in > the index, with Queue deprecated with instructions to use the new > name. > > But what to do about all the references. Should we leave them pointing > at Queue to lessen confusion for people who read about some module on > some other site that isn't using the new name, or update everything in > 2.6 to use the new name? How hard would it be to add a redirects from the old pages to the new ones ? mod_rewrite does wonders - well, provided you find the right patterns... -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From nd at perlig.de Fri May 16 14:52:19 2008 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Fri, 16 May 2008 14:52:19 +0200 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: <482D8174.7030407@egenix.com> References: <482D8174.7030407@egenix.com> Message-ID: <200805161452.20000.nd@perlig.de> * M.-A. Lemburg wrote: > On 2008-05-12 04:34, Brett Cannon wrote: > > For the sake of argument, let's consider the Queue module. It is now > > named queue. For 2.6 I plan on having both Queue and queue listed in > > the index, with Queue deprecated with instructions to use the new > > name. > > > > But what to do about all the references. Should we leave them pointing > > at Queue to lessen confusion for people who read about some module on > > some other site that isn't using the new name, or update everything in > > 2.6 to use the new name? > > How hard would it be to add a redirects from the old pages to the > new ones ? > > mod_rewrite does wonders - well, provided you find the right patterns... The "pattern" can be a simple text file maintained in subversion:: oldurl newurl ... And then you can utilize RewriteMap to get that into the apache. nd From mal at egenix.com Fri May 16 15:00:26 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 16 May 2008 15:00:26 +0200 Subject: [Python-Dev] Distutils & configparser rename In-Reply-To: <20080515203353.GA24080@amk-desktop.matrixgroup.net> References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> Message-ID: <482D856A.802@egenix.com> On 2008-05-15 22:33, A.M. Kuchling wrote: > Python 2.6 renames the ConfigParser module to be configparser. > > Distutils imports ConfigParser in various places. I just made a > commit updating the import in one places, and then noticed that part > of commit r63248, which made the same change, was reverted in order to > preserve backward-compatibility. Instead, the default path will > include lib-old again to keep the old module name available. > > I suggest dropping that goal, though. We've preserved compatibility > but I'm not aware that anyone uses the Python 2.x Distutils with > earlier versions of Python. In particular: > > * There's no standalone distutils package on PyPI, nor can I find > such a package with a general web search. Am I missing it? > > * I do not see users advising other users to use a later version of > Distutils to fix their problems. > > Is anyone actually benefiting from the effort of maintaining backward > compatibility? Yes: all the folks who want to create distutils packages for more than just the current Python version. I've argued for this a couple of times in the past. Some background: In order to build a Python package for a previous Python version, you have to run distutils using that older Python version. Now, as distutils evolves, new features are added, bugs are fixed, etc. so as packager you always want to use the latest distutils version available - even with older Python releases. In some cases, e.g. PyPI registration, this may even be necessary, since the new versions of those commands need to be kept in sync with the PyPI repository. Another aspect is keeping package setup.py files working. If you need to support multiple Python versions, then your setup.py will have to work with multiple different versions of distutils. Since performance doesn't really matter for distutils, it is well possible and easy to keep compatibility with a few releases back. This has worked great in the past and I don't see why we should break this, as recent distutils checkins have done. Note that Python doesn't exactly make it easy to ship Python packages. You have several different dimensions to take into consideration: * Python version * UCS2/UCS4 * Platform and processor type * 32/64-bit So there already is a lot of porting effort needed to support a reasonable number of targets. I don't think it takes a lot of effort to keep distutils running with Python 2.3 and 2.4. In the past I've usually rewritten parts of distutils that were modified in incompatible ways. I haven't been able to that for the recent checkins that broke distutils even on Python 2.4. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Fri May 16 15:01:43 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 16 May 2008 15:01:43 +0200 Subject: [Python-Dev] platform module testing In-Reply-To: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> References: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> Message-ID: <482D85B7.7000909@egenix.com> On 2008-05-15 23:54, Benjamin Peterson wrote: > At the moment, the test for the platform module merely calls each > function. I realize that this is a hard module to test well, but are > there some assumptions we can make? For example, if sys.platform is > 'java', can it be assumed that java_ver is going to return something > which is not empty. When sys.platform is 'darwin' and the gestalt > module is present, is it reasonable to require that a empty tuple not > be returned? Yes, at least the main elements should be present and filled in. If not, we have to fix platform.py. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From amk at amk.ca Fri May 16 15:24:51 2008 From: amk at amk.ca (A.M. Kuchling) Date: Fri, 16 May 2008 09:24:51 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482D469E.7080609@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> Message-ID: <20080516132451.GA7257@amk-desktop.matrixgroup.net> On Fri, May 16, 2008 at 10:32:30AM +0200, Ulrich Berning wrote: > As long as the ctypes extension doesn't build on major Un*x platforms > (AIX, HP-UX), I don't like to see ctypes dependend modules included into > the stdlib. Please keep the stdlib as portable as possible. > More and more people tend to say "Runs on Un*x" when they really mean > "Tested on Linux". Un*x is not Linux. Python development doesn't seem to have any volunteers who use AIX or HP-UX and can fix bugs on these platforms. Searching bugs.python.org, I find about 20 open AIX issues, 4 or 5 HP-UX issues, 6 IRIX bugs, and about 15 Solaris bugs; but I don't know if any of the developers here use these platforms. (There is a Solaris buildbot, at least.) This means that if people didn't use ctypes and wrote C extension modules instead, those extension modules probably wouldn't work on AIX and HP-UX either. I'd rather see a standard library that's as featureful and useful as possible, and not constrain it in order to support platforms that we don't really seem to be supporting anyway. There's a bug report about ctypes on AIX (1637120) and an old one about ctypes on Solaris that looks like it's fixed, but I can't find a report for HP-UX. Part of the problem seems to be that libffi is written for GCC, so using the vendor compiler often causes difficulties. --amk From ncoghlan at gmail.com Fri May 16 15:44:04 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 16 May 2008 23:44:04 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482D469E.7080609@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> Message-ID: <482D8FA4.7040509@gmail.com> Ulrich Berning wrote: > More and more people tend to say "Runs on Un*x" when they really mean > "Tested on Linux". Un*x is not Linux. Hmm, perhaps that would be why there are Solaris, FreeBSD and Tru64 machines amongst the main Python buildbots, to go along with the assorted OS X, Windows and Linux boxes - and as far as I know test_ctypes runs quite happily on all of them. On the specific problems with AIX, HP-UX and ctypes, was it ctypes itself that was failing to build, or the underlying libffi? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Fri May 16 15:47:03 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 16 May 2008 15:47:03 +0200 Subject: [Python-Dev] [Distutils] Distutils & configparser rename In-Reply-To: References: <20080515203353.GA24080@amk-desktop.matrixgroup.net> <482D856A.802@egenix.com> Message-ID: <482D9057.3060602@egenix.com> On 2008-05-16 15:28, Rasmus Andersson wrote: > > On 16 maj 2008, at 15.00, M.-A. Lemburg wrote: > >> On 2008-05-15 22:33, A.M. Kuchling wrote: >>> >> >> So there already is a lot of porting effort needed to support >> a reasonable number of targets. >> >> I don't think it takes a lot of effort to keep distutils >> running with Python 2.3 and 2.4. > > Speaking of which; is there any reliable statistics of Python versions > and platforms? Would be really interesting! Reliable statistics are hard to obtain, since Python is distributed along with many Unix distributions, as part of many products, and also in source form (where you basically lose any platform or Python version information). FWIW, these are the stats for our OSS Python package egenix-mx-base: Downloads egenix-mx-base 3.0 in 2008: Python 1.5.2: 0 Python 2.0 : 0 Python 2.1 : 31 Python 2.2 : 54 Python 2.3 : 483 Python 2.4 : 3368 Python 2.5 : 7504 Source ZIP : 4238 Source TGZ : 2918 These are just downloads, so likely new installations (the 3.0 version was released about one year ago). It's hard to tell what the installed user base looks like, esp with respect to the used Python versions. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From alexander.belopolsky at gmail.com Fri May 16 16:03:43 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 16 May 2008 14:03:43 +0000 (UTC) Subject: [Python-Dev] Symbolic errno values in error messages References: <873aoifrm4.fsf@enceladus.ygingras.net> Message-ID: Yannick Gingras ygingras.net> writes: .. > > 1) Should OSError.__str__() print the symbolic name of errno? > +1 for the change > 2) Where can I find the symbolic name in C? Use standard C library char* strerror(int errnum) function. You can see an example usage in Modules/posixmodule.c (posix_strerror). From ncoghlan at gmail.com Fri May 16 16:13:45 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 17 May 2008 00:13:45 +1000 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: <873aoifrm4.fsf@enceladus.ygingras.net> References: <873aoifrm4.fsf@enceladus.ygingras.net> Message-ID: <482D9699.5000903@gmail.com> Yannick Gingras wrote: > 1) Should OSError.__str__() print the symbolic name of errno? +1 (assuming the performance hit for doing so is incurred only when the exception is actually printed) > 2) Where can I find the symbolic name in C? Modules/errnomodule.c Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri May 16 16:15:23 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 17 May 2008 00:15:23 +1000 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: References: <873aoifrm4.fsf@enceladus.ygingras.net> Message-ID: <482D96FB.2040700@gmail.com> Alexander Belopolsky wrote: > Yannick Gingras ygingras.net> writes: >> 2) Where can I find the symbolic name in C? > > Use standard C library char* strerror(int errnum) function. You can see > an example usage in Modules/posixmodule.c (posix_strerror). I don't believe that would provide adequate Windows support. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From exarkun at divmod.com Fri May 16 16:38:56 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 16 May 2008 10:38:56 -0400 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: <482D96FB.2040700@gmail.com> Message-ID: <20080516143856.6859.1687756730.divmod.quotient.64266@ohm> On Sat, 17 May 2008 00:15:23 +1000, Nick Coghlan wrote: >Alexander Belopolsky wrote: >>Yannick Gingras ygingras.net> writes: >>>2) Where can I find the symbolic name in C? >> >>Use standard C library char* strerror(int errnum) function. You can see >>an example usage in Modules/posixmodule.c (posix_strerror). > >I don't believe that would provide adequate Windows support. > It's not C, but maybe it's interesting to look at anyway: http://twistedmatrix.com/trac/browser/trunk/twisted/python/win32.py?rev=21682#L94 However, neither strerror nor the linked code gives out symbolic names for errnos. They both produce messages like "Interrupted system call", whereas the symbolic name would be "EINTR". Modules/errnomodule.c might be worth looking at, although its solution is somewhat disappointing. Jean-Paul From alexander.belopolsky at gmail.com Fri May 16 16:39:13 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 16 May 2008 10:39:13 -0400 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: <482D96FB.2040700@gmail.com> References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> Message-ID: On Fri, May 16, 2008 at 10:15 AM, Nick Coghlan wrote: > Alexander Belopolsky wrote: .. >> Use standard C library char* strerror(int errnum) function. You can see >> an example usage in Modules/posixmodule.c (posix_strerror). > > I don't believe that would provide adequate Windows support. > Until recently, python had its own cross-platform implementation of strerror, but it was removed because it was deemed redundant. This tells me that it should work on windows. On the other hand, IOError object and in fact EnvironmentError object already has strerror member which is printed when available: >>> try: ... open('/') ... except Exception,e: ... pass ... >>> print e [Errno 21] Is a directory So now I am not sure what OP is proposing. Do you want to replace 21 with EISDIR in the above? From ygingras at ygingras.net Fri May 16 16:52:34 2008 From: ygingras at ygingras.net (Yannick Gingras) Date: Fri, 16 May 2008 10:52:34 -0400 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: (Alexander Belopolsky's message of "Fri\, 16 May 2008 10\:39\:13 -0400") References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> Message-ID: <87y76ae1xp.fsf@enceladus.ygingras.net> "Alexander Belopolsky" writes: >>>> try: > ... open('/') > ... except Exception,e: > ... pass > ... >>>> print e > [Errno 21] Is a directory > > So now I am not sure what OP is proposing. Do you want to replace 21 > with EISDIR in the above? Yes, that's what I had in mind. -- Yannick Gingras From alexander.belopolsky at gmail.com Fri May 16 17:02:26 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 16 May 2008 11:02:26 -0400 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: <87y76ae1xp.fsf@enceladus.ygingras.net> References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> <87y76ae1xp.fsf@enceladus.ygingras.net> Message-ID: On Fri, May 16, 2008 at 10:52 AM, Yannick Gingras wrote: >>>>> print e >> [Errno 21] Is a directory >> >> So now I am not sure what OP is proposing. Do you want to replace 21 >> with EISDIR in the above? > > Yes, that's what I had in mind. > In this case, I have a more drastic proposal. Lets change EnvironmentError errno attribute (myerrno in C) to string. 'EXYZ' strings can be interned, which will make them more efficient than integers for lookups and comparisons (to literals). A half-way and backward compatible solution would be to stick 'EXYZ' code at the end of the args tuple and add an errnosym attribute. From theller at ctypes.org Fri May 16 17:22:46 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 16 May 2008 17:22:46 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482D469E.7080609@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> Message-ID: Ulrich Berning schrieb: > Nick Craig-Wood wrote: > >>Jesse Noller wrote: >> >> >>> I am looking for any questions, concerns or benchmarks python-dev has >>> regarding the possible inclusion of the pyprocessing module to the >>> standard library - preferably in the 2.6 timeline. In March, I began >>> working on the PEP for the inclusion of the pyprocessing (processing) >>> module into the python standard library[1]. The original email to the >>> stdlib-sig can be found here, it includes a basic overview of the >>> module: >>> >>> http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html >>> >>> The processing module mirrors/mimics the API of the threading module - >>> and with simple import/subclassing changes depending on the code, >>> allows you to leverage multi core machines via an underlying forking >>> mechanism. The module also supports the sharing of data across groups >>> of networked machines - a feature obviously not part of the core >>> threading module, but useful in a distributed environment. >>> >>> >> >>I think processing looks interesting and useful, especially since it >>works on Windows as well as Un*x. >> >>However I'd like to see a review of the security - anything which can >>run across networks of machines has security implications and I didn't >>see these spelt out in the documentation. >> >>Networked running should certainly be disabled by default and need >>explicitly enabling by the user - I'd hate for a new version of python >>to come with a remote exploit by default... >> >> >> > As long as the ctypes extension doesn't build on major Un*x platforms > (AIX, HP-UX), I don't like to see ctypes dependend modules included into > the stdlib. Please keep the stdlib as portable as possible. > More and more people tend to say "Runs on Un*x" when they really mean > "Tested on Linux". Un*x is not Linux. Hm, I've never looked at the processing module. Does it depend on ctypes? Anyway, the trunk version of ctypes is a lot more portable than the release25-maint version. I have once tried to build the trunk on HP-UX machines, and, IIRC, it did build on IA64 and PA machines. Of course only with GCC, not with the vendor compilers. Thomas From mal at egenix.com Fri May 16 17:33:11 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 16 May 2008 17:33:11 +0200 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: <482D96FB.2040700@gmail.com> References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> Message-ID: <482DA937.9050408@egenix.com> On 2008-05-16 16:15, Nick Coghlan wrote: > Alexander Belopolsky wrote: >> Yannick Gingras ygingras.net> writes: >>> 2) Where can I find the symbolic name in C? >> >> Use standard C library char* strerror(int errnum) function. You can see >> an example usage in Modules/posixmodule.c (posix_strerror). > > I don't believe that would provide adequate Windows support. Well, there's still the idea of a winerror module: http://bugs.python.org/issue1505257 Perhaps someone can pick it up and turn it into a (generated) C module ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From jnoller at gmail.com Fri May 16 17:33:52 2008 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 16 May 2008 11:33:52 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> Message-ID: <4222a8490805160833v4b318a60p800448d5ec4e86a3@mail.gmail.com> On Fri, May 16, 2008 at 11:22 AM, Thomas Heller wrote: > Ulrich Berning schrieb: >> Nick Craig-Wood wrote: >> >>>Jesse Noller wrote: >>> >>> >>>> I am looking for any questions, concerns or benchmarks python-dev has >>>> regarding the possible inclusion of the pyprocessing module to the >>>> standard library - preferably in the 2.6 timeline. In March, I began >>>> working on the PEP for the inclusion of the pyprocessing (processing) >>>> module into the python standard library[1]. The original email to the >>>> stdlib-sig can be found here, it includes a basic overview of the >>>> module: >>>> >>>> http://mail.python.org/pipermail/stdlib-sig/2008-March/000129.html >>>> >>>> The processing module mirrors/mimics the API of the threading module - >>>> and with simple import/subclassing changes depending on the code, >>>> allows you to leverage multi core machines via an underlying forking >>>> mechanism. The module also supports the sharing of data across groups >>>> of networked machines - a feature obviously not part of the core >>>> threading module, but useful in a distributed environment. >>>> >>>> >>> >>>I think processing looks interesting and useful, especially since it >>>works on Windows as well as Un*x. >>> >>>However I'd like to see a review of the security - anything which can >>>run across networks of machines has security implications and I didn't >>>see these spelt out in the documentation. >>> >>>Networked running should certainly be disabled by default and need >>>explicitly enabling by the user - I'd hate for a new version of python >>>to come with a remote exploit by default... >>> >>> >>> >> As long as the ctypes extension doesn't build on major Un*x platforms >> (AIX, HP-UX), I don't like to see ctypes dependend modules included into >> the stdlib. Please keep the stdlib as portable as possible. >> More and more people tend to say "Runs on Un*x" when they really mean >> "Tested on Linux". Un*x is not Linux. > > Hm, I've never looked at the processing module. Does it depend on ctypes? > > Anyway, the trunk version of ctypes is a lot more portable than the release25-maint > version. I have once tried to build the trunk on HP-UX machines, and, > IIRC, it did build on IA64 and PA machines. Of course only with GCC, not > with the vendor compilers. > > Thomas > Yes, it requires ctypes. From mal at egenix.com Fri May 16 17:35:37 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 16 May 2008 17:35:37 +0200 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> <87y76ae1xp.fsf@enceladus.ygingras.net> Message-ID: <482DA9C9.7090606@egenix.com> On 2008-05-16 17:02, Alexander Belopolsky wrote: > On Fri, May 16, 2008 at 10:52 AM, Yannick Gingras wrote: > >>>>>> print e >>> [Errno 21] Is a directory >>> >>> So now I am not sure what OP is proposing. Do you want to replace 21 >>> with EISDIR in the above? >> Yes, that's what I had in mind. >> > > In this case, I have a more drastic proposal. Lets change > EnvironmentError errno attribute (myerrno in C) to string. -1 You never want to change an integer field to a string. >'EXYZ' > strings can be interned, which will make them more efficient than > integers for lookups and comparisons (to literals). A half-way and > backward compatible solution would be to stick 'EXYZ' code at the end > of the args tuple and add an errnosym attribute. Actually, you don't have to put it into any tuple. Just add it to the error object as attribute. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From facundobatista at gmail.com Fri May 16 18:42:38 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 16 May 2008 13:42:38 -0300 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: 2008/5/13 Jesse Noller : > I am looking for any questions, concerns or benchmarks python-dev has > regarding the possible inclusion of the pyprocessing module to the > standard library - preferably in the 2.6 timeline. In March, I began +1 to include this module in the library, if... - it's included in 2.7/3.1 after a stabilization period. - it's stop reinventing the wheel and starts using the already-in-stdlib infrastructure (this is from Martin's post). - a champion appears willing to work with it and maintain it afterwards (which is the opinion of the module owner about this?) I think that including in the stdlib a threading-like-API module is a clear win. Of course, this module won't solve all the problems or necessities of multiprocessing world, but it's an included easy to use start (in the same spirit that SimpleHTTPServer it's not an Apache, but it's very useful if your problem area fits its limitations). Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From alexandre at peadrop.com Fri May 16 18:58:15 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Fri, 16 May 2008 12:58:15 -0400 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: <87y76ae1xp.fsf@enceladus.ygingras.net> References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> <87y76ae1xp.fsf@enceladus.ygingras.net> Message-ID: On Fri, May 16, 2008 at 10:52 AM, Yannick Gingras wrote: > "Alexander Belopolsky" writes: > >>>>> try: >> ... open('/') >> ... except Exception,e: >> ... pass >> ... >>>>> print e >> [Errno 21] Is a directory >> >> So now I am not sure what OP is proposing. Do you want to replace 21 >> with EISDIR in the above? > > Yes, that's what I had in mind. > Then, check out EnvironmentError_str in Objects/exceptions.c. You should be able import the errno module and fetch its errorcode dictionary. -- Alexandre From python at rcn.com Fri May 16 19:19:20 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 16 May 2008 10:19:20 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: <008f01c8b778$fca46590$ae00a8c0@RaymondLaptop1> [Facundo] > I think that including in the stdlib a threading-like-API module is a > clear win. I also think this is vital and should not wait for Py2.7. Raymond From ggpolo at gmail.com Fri May 16 19:30:05 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 16 May 2008 14:30:05 -0300 Subject: [Python-Dev] lib2to3, need some light on the imports fixer In-Reply-To: <43aa6ff70805121413vb42a783mabe4d7764704b5a0@mail.gmail.com> References: <43aa6ff70805121413vb42a783mabe4d7764704b5a0@mail.gmail.com> Message-ID: 2008/5/12 Collin Winter : > On Mon, May 12, 2008 at 1:58 PM, Guilherme Polo wrote: >> Hello, >> >> Would someone tell me how can I add a new entry in the MAPPING dict in >> the lib2to3/fixes/fix_imports.py that does the following: >> >> "import A" gets fixed as "import C.D as A" >> >> Right now it is fixing by doing "import C.D" and changing several >> other lines in the code to use this new "C.D" name. I wanted to avoid >> these changes if possible. > > I don't believe there's a way to do that, but adding support for it > should be fairly straight-forward. Assign the patch to me for review. > Is there something I could read get some 2to3 understanding ? How do I construct patterns ? > Collin Winter > -- -- Guilherme H. Polo Goncalves From collinw at gmail.com Fri May 16 19:33:20 2008 From: collinw at gmail.com (Collin Winter) Date: Fri, 16 May 2008 10:33:20 -0700 Subject: [Python-Dev] lib2to3, need some light on the imports fixer In-Reply-To: References: <43aa6ff70805121413vb42a783mabe4d7764704b5a0@mail.gmail.com> Message-ID: <43aa6ff70805161033o4e5bf43ct4e8091b47b1974a5@mail.gmail.com> On Fri, May 16, 2008 at 10:30 AM, Guilherme Polo wrote: > 2008/5/12 Collin Winter : >> On Mon, May 12, 2008 at 1:58 PM, Guilherme Polo wrote: >>> Hello, >>> >>> Would someone tell me how can I add a new entry in the MAPPING dict in >>> the lib2to3/fixes/fix_imports.py that does the following: >>> >>> "import A" gets fixed as "import C.D as A" >>> >>> Right now it is fixing by doing "import C.D" and changing several >>> other lines in the code to use this new "C.D" name. I wanted to avoid >>> these changes if possible. >> >> I don't believe there's a way to do that, but adding support for it >> should be fairly straight-forward. Assign the patch to me for review. >> > > Is there something I could read get some 2to3 understanding ? The README is a very shallow overview, but it needs to be beefed up to talk about architecture. Honestly, the best way (currently) is to read the existing fixers. Mea culpa. > How do I construct patterns ? find_pattern.py is the tool I wrote/use to find patterns. You give it some Python code and it spits out the pattern for it. It's hard to hand-construct patterns due to some optimizations done to eliminate unneeded nodes (nodes with only a single child are eliminated, pulling the child to its grandparent). From ggpolo at gmail.com Fri May 16 19:42:44 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 16 May 2008 14:42:44 -0300 Subject: [Python-Dev] lib2to3, need some light on the imports fixer In-Reply-To: <43aa6ff70805161033o4e5bf43ct4e8091b47b1974a5@mail.gmail.com> References: <43aa6ff70805121413vb42a783mabe4d7764704b5a0@mail.gmail.com> <43aa6ff70805161033o4e5bf43ct4e8091b47b1974a5@mail.gmail.com> Message-ID: 2008/5/16 Collin Winter : > On Fri, May 16, 2008 at 10:30 AM, Guilherme Polo wrote: >> 2008/5/12 Collin Winter : >>> On Mon, May 12, 2008 at 1:58 PM, Guilherme Polo wrote: >>>> Hello, >>>> >>>> Would someone tell me how can I add a new entry in the MAPPING dict in >>>> the lib2to3/fixes/fix_imports.py that does the following: >>>> >>>> "import A" gets fixed as "import C.D as A" >>>> >>>> Right now it is fixing by doing "import C.D" and changing several >>>> other lines in the code to use this new "C.D" name. I wanted to avoid >>>> these changes if possible. >>> >>> I don't believe there's a way to do that, but adding support for it >>> should be fairly straight-forward. Assign the patch to me for review. >>> >> >> Is there something I could read get some 2to3 understanding ? > > The README is a very shallow overview, but it needs to be beefed up to > talk about architecture. Honestly, the best way (currently) is to read > the existing fixers. Mea culpa. > >> How do I construct patterns ? > > find_pattern.py is the tool I wrote/use to find patterns. You give it > some Python code and it spits out the pattern for it. It's hard to > hand-construct patterns due to some optimizations done to eliminate > unneeded nodes (nodes with only a single child are eliminated, pulling > the child to its grandparent). > I see find_pattern.py is only at http://svn.python.org/projects/sandbox/trunk/2to3. Now I remember I saw it some time ago, but was trying to find it at python's repo and couldn't find it =) Thanks Collin. -- -- Guilherme H. Polo Goncalves From collinw at gmail.com Fri May 16 19:46:02 2008 From: collinw at gmail.com (Collin Winter) Date: Fri, 16 May 2008 10:46:02 -0700 Subject: [Python-Dev] lib2to3, need some light on the imports fixer In-Reply-To: References: <43aa6ff70805121413vb42a783mabe4d7764704b5a0@mail.gmail.com> <43aa6ff70805161033o4e5bf43ct4e8091b47b1974a5@mail.gmail.com> Message-ID: <43aa6ff70805161046s7e6cd3bvc068788034eae205@mail.gmail.com> On Fri, May 16, 2008 at 10:42 AM, Guilherme Polo wrote: > 2008/5/16 Collin Winter : >> On Fri, May 16, 2008 at 10:30 AM, Guilherme Polo wrote: >>> 2008/5/12 Collin Winter : >>>> On Mon, May 12, 2008 at 1:58 PM, Guilherme Polo wrote: >>>>> Hello, >>>>> >>>>> Would someone tell me how can I add a new entry in the MAPPING dict in >>>>> the lib2to3/fixes/fix_imports.py that does the following: >>>>> >>>>> "import A" gets fixed as "import C.D as A" >>>>> >>>>> Right now it is fixing by doing "import C.D" and changing several >>>>> other lines in the code to use this new "C.D" name. I wanted to avoid >>>>> these changes if possible. >>>> >>>> I don't believe there's a way to do that, but adding support for it >>>> should be fairly straight-forward. Assign the patch to me for review. >>>> >>> >>> Is there something I could read get some 2to3 understanding ? >> >> The README is a very shallow overview, but it needs to be beefed up to >> talk about architecture. Honestly, the best way (currently) is to read >> the existing fixers. Mea culpa. >> >>> How do I construct patterns ? >> >> find_pattern.py is the tool I wrote/use to find patterns. You give it >> some Python code and it spits out the pattern for it. It's hard to >> hand-construct patterns due to some optimizations done to eliminate >> unneeded nodes (nodes with only a single child are eliminated, pulling >> the child to its grandparent). >> > > I see find_pattern.py is only at > http://svn.python.org/projects/sandbox/trunk/2to3. > Now I remember I saw it some time ago, but was trying to find it at > python's repo and couldn't find it =) Please make your changes to http://svn.python.org/projects/sandbox/trunk/2to3; the Python copy should be synced from there, not the other way around. From brett at python.org Fri May 16 20:04:12 2008 From: brett at python.org (Brett Cannon) Date: Fri, 16 May 2008 11:04:12 -0700 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: <482D8174.7030407@egenix.com> References: <482D8174.7030407@egenix.com> Message-ID: On Fri, May 16, 2008 at 5:43 AM, M.-A. Lemburg wrote: > On 2008-05-12 04:34, Brett Cannon wrote: >> >> For the sake of argument, let's consider the Queue module. It is now >> named queue. For 2.6 I plan on having both Queue and queue listed in >> the index, with Queue deprecated with instructions to use the new >> name. >> >> But what to do about all the references. Should we leave them pointing >> at Queue to lessen confusion for people who read about some module on >> some other site that isn't using the new name, or update everything in >> 2.6 to use the new name? > > How hard would it be to add a redirects from the old pages to the > new ones ? > > mod_rewrite does wonders - well, provided you find the right patterns... Well, the problem is that only solves the online docs issue, not offline viewing. And now that the docs can be built easily, I at least find myself referring to the docs mostly offline rather than through docs.python.org. -Brett From r.m.oudkerk at googlemail.com Fri May 16 20:16:55 2008 From: r.m.oudkerk at googlemail.com (r.m.oudkerk) Date: Fri, 16 May 2008 19:16:55 +0100 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482B26FA.6060007@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> <482B26FA.6060007@v.loewis.de> Message-ID: 2008/5/14 "Martin v. L?wis" : >> I really do feel that inclusion of this library offers us the best of >> both worlds - it gives us (as a community) an easy answer to those >> people who would dismiss python due to the GIL and it also allows >> users to easily implement their applications. I am the author of processing -- sorry for arriving late in this thread. > For inclusion into Python, a number of things need to be changed, > in particular with respect to the C code. It duplicates a lot of > code from the socket and os modules, and should (IMO) be merged into > these, rather than being separate - or perhaps merged into the > subprocess module. Now that socket.fromfd() and socket._dup() is available on Windows in 2.6 and 3.0 (after a patch from me) some of the code could be removed. I would also like to see recvfd() and sendfd() get added to the socket module -- these functions allow the transfer of file descriptors between processes over a Unix domain socket. But I am not sure that there is much duplication with the socket module. The Connection type is basically a message oriented version of a socket with builtin pickling/unpickling -- it could be implemented in pure Python but that would *much* slower. I cannot see any opportunity to reuse code from socketmodule.c in implementing the Connection type, particularly given that the code also allows the use of Windows named pipes which are much faster than sockets on Windows. As far as I know there is no duplication with the os modules. Basically the C code separates as follows (1) a Connection type (plus PipeConnection on windows) (2) a "process shared" lock/semaphore type (3) Win32 functions/constants to allow use of named pipes (4) A few other Win32 functions/constants (5) A wrapper making PyObject_AsWriteBuffer() available from Python I suppose (4) and perhaps (3) could sensibly be added/merged with _subprocess.c. (5) could also be moved somewhere else. > Why isn't it possible to implement all this in pure Python, on > top of the subprocess module? I am not sure how much you include when you say "all this" -- (2) certainly cannot be done in pure python, and I certainly believe that (1) needs to be done in C for the sake of performance. On Windows I did use subprocess.py in earlier versions, but I had bug reports saying that in non-console programs it would choke because it was failing to duplicate GetStdHandle(STD_OUTPUT_HANDLE) etc. Using _subprocess.CreateProcess() directly avoided the problem and did not add noticeably to code length. I also found that the Popen.__del__() methods kept raising errors on process shutdown because of unavailable globals, so I was happy to get shot of subprocess.py. (Off-topic but I think that the way that subprocess.Popen.__del__() keeps the object alive if the process has not finished is a little distasteful. The __del__() method and the _cleanup() function are completely unecessary on Windows since Windows does not have zombie processes. On Unix __del__() could simply save the pid to the _active list and _cleanup() could be rewritten to use os.waitpid().) > If there are limitations in Python that make it impossible to > implement such functionality in pure Python - then *those* > limitations should be overcome, rather than including the code > wholesale. Apart from (1) and (2) which I think are unavoidable, nearly all the C code is Windows specific. I assume you don't want to bring in the whole of pywin32 into stdlib... > Regards, > Martin Richard PS. I am more than willing to maintain the code if it goes in. From martin at v.loewis.de Sat May 17 00:22:53 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 17 May 2008 00:22:53 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> <482B26FA.6060007@v.loewis.de> Message-ID: <482E093D.8030609@v.loewis.de> > Apart from (1) and (2) which I think are unavoidable, nearly all the C > code is Windows specific. I assume you don't want to bring > in the whole of pywin32 into stdlib... I wouldn't structure it the same way as pywin32, but yes, I have pondered exposing all of Win32 in a module, as part of the standard library. This has somewhat lost relevance with ctypes. Regards, Martin From martin at v.loewis.de Sat May 17 00:25:07 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 17 May 2008 00:25:07 +0200 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> Message-ID: <482E09C3.1060000@v.loewis.de> > Until recently, python had its own cross-platform implementation of > strerror, but it was removed because it was deemed redundant. This > tells me that it should work on windows. That conclusion is incorrect. It works on MSVCRT, but for this specific aspect, using MSVCRT is a bad idea (because it artificially renumbers system errors, just to provide an illusion for what Microsoft considers POSIX). Regards, Martin From python at rcn.com Sat May 17 00:42:02 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 16 May 2008 15:42:02 -0700 Subject: [Python-Dev] How best to handle the docs for a renamed module? References: <482D8174.7030407@egenix.com> Message-ID: <002801c8b7a6$10212180$ad00a8c0@RaymondLaptop1> [Brett[ >>> For the sake of argument, let's consider the Queue module. It is now >>> named queue. For 2.6 I plan on having both Queue and queue listed in >>> the index, with Queue deprecated with instructions to use the new >>> name. FWIW, I don't consider this to be a value added backport. Renaming doesn't benefit a 2.6 user. All it does is add confusion and it may break code where users or third-party libraries are already using the new name. I say leave the new names in 3.0 and let the 2-to-3 tool do its job. Why confuse the 2.6 landscape with double naming clutter. Raymond From greg at krypto.org Sat May 17 00:52:33 2008 From: greg at krypto.org (Gregory P. Smith) Date: Fri, 16 May 2008 15:52:33 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482D469E.7080609@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> Message-ID: <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> On Fri, May 16, 2008 at 1:32 AM, Ulrich Berning wrote: > > As long as the ctypes extension doesn't build on major Un*x platforms (AIX, > HP-UX), I don't like to see ctypes dependend modules included into the > stdlib. Please keep the stdlib as portable as possible. Nice in theory but ctypes already works on at least the top 3 popular platforms. Lets not hold Python's stdlib back because nobody who uses IBM and HP proprietary stuff has contributed the necessary support. Making nice libraries available for other platforms is a good way to encourage people to either pitch in and add support or consider their platform choices in the future. -gps From jek-gmane at kleckner.net Sat May 17 01:16:43 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Fri, 16 May 2008 16:16:43 -0700 Subject: [Python-Dev] Visual Studio 2008 compiler option EHsc ? In-Reply-To: <482D1B44.3090507@v.loewis.de> References: <482D1B44.3090507@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> In building a package with several platforms, I >> ran across the warning message below from Visual >> Studio 2008. Should we add the /EHsc option to the >> compile_options in distutils for MSVC? Or is it more >> complex than that... > > Who is "we"? If you have a module that uses C++ exceptions, > you should certainly enable compiler support for exceptions, > for that module. In my original email, I referred to the patch for distutils at: http://tinyurl.com/63bqo2 Based on that patch, I made the attached patch in distutils to msvc9compiler.py and that made the complaints disappear (and presumably made the exceptions work?). Is there any downside to doing this by default? Or should I parameterize my build_ext for that platform to include that option? Some of the web pages on the topic suggest that this is due to a change in the compiler behavior with VS2008. So this may bite other people as 2.6 rolls out. I am not an expert in structured exception handling and all the nuances so welcome any advice. Thanks - Jim -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: patch.txt URL: From ncoghlan at gmail.com Sat May 17 02:49:54 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 17 May 2008 10:49:54 +1000 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: <002801c8b7a6$10212180$ad00a8c0@RaymondLaptop1> References: <482D8174.7030407@egenix.com> <002801c8b7a6$10212180$ad00a8c0@RaymondLaptop1> Message-ID: <482E2BB2.9080103@gmail.com> Raymond Hettinger wrote: > I say leave the new names in 3.0 and let the 2-to-3 tool do its job. > Why confuse the 2.6 landscape with double naming clutter. To increase the common subset of code which can run on both 2.6 and 3.0 *without* invoking 2to3? PEP-8'ifying these names also makes them easier for at least me to remember (since I don't have to worry about whether or not the module name includes unexpected underscores or strange capitalisation) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From brett at python.org Sat May 17 07:07:08 2008 From: brett at python.org (Brett Cannon) Date: Fri, 16 May 2008 22:07:08 -0700 Subject: [Python-Dev] How best to handle the docs for a renamed module? In-Reply-To: <482E2BB2.9080103@gmail.com> References: <482D8174.7030407@egenix.com> <002801c8b7a6$10212180$ad00a8c0@RaymondLaptop1> <482E2BB2.9080103@gmail.com> Message-ID: On Fri, May 16, 2008 at 5:49 PM, Nick Coghlan wrote: > Raymond Hettinger wrote: >> >> I say leave the new names in 3.0 and let the 2-to-3 tool do its job. >> Why confuse the 2.6 landscape with double naming clutter. > > To increase the common subset of code which can run on both 2.6 and 3.0 > *without* invoking 2to3? > Right. 2.6 is trying to be as much of a reasonable subset of Python 3.0 as possible. This includes module names. > PEP-8'ifying these names also makes them easier for at least me to remember > (since I don't have to worry about whether or not the module name includes > unexpected underscores or strange capitalisation) That too. =) -Brett From matthieu.brucher at gmail.com Sat May 17 10:34:01 2008 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Sat, 17 May 2008 10:34:01 +0200 Subject: [Python-Dev] Visual Studio 2008 compiler option EHsc ? In-Reply-To: References: <482D1B44.3090507@v.loewis.de> Message-ID: Hi, 2008/5/17 Jim Kleckner : > Martin v. L?wis wrote: > >> In building a package with several platforms, I >>> ran across the warning message below from Visual >>> Studio 2008. Should we add the /EHsc option to the >>> compile_options in distutils for MSVC? Or is it more >>> complex than that... >>> >> >> Who is "we"? If you have a module that uses C++ exceptions, >> you should certainly enable compiler support for exceptions, >> for that module. >> > > In my original email, I referred to the patch for distutils at: > http://tinyurl.com/63bqo2 > > Based on that patch, I made the attached patch in distutils to > msvc9compiler.py and that made the complaints disappear > (and presumably made the exceptions work?). > > Is there any downside to doing this by default? > Or should I parameterize my build_ext for that platform > to include that option? This option is only for C++ code, and not for every C++ code. Some people don't want exception to be handled (take a look at Boost for instance that let the user decide whether its exception handling is done by the compiler or by the user), as it can add overhead in some places. With Scons, the problem is the same, you have to add by hand this flag, and I think it is the correct way of doing things. Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Sat May 17 11:05:55 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Sat, 17 May 2008 11:05:55 +0200 Subject: [Python-Dev] Module renaming and pickle mechanisms Message-ID: <482E9FF3.4060200@egenix.com> I'd like to bring a potential problem to attention that is caused by the recent module renaming approach: Object serialization protocols like e.g. pickle usually store the complete module path to the object class together with the object. They access this module path by looking at the __module__ attribute of the object classes. With the renaming, all objects which use classes from the renamed modules will now refer to the renamed modules in their serialized form, e.g. queue.Queue instead of Queue.Queue (just to name one example). While this is nice for forward compatibility, it causes rather serious problems for making object serialization backwards compatible, since the older Python versions can no longer unserialize objects due to missing modules. This can happen in client-server setups where e.g. the server uses Python 2.6 and the clients some other Python version (e.g. Python 2.5). It can also happen in storage setups where Python objects are stored using e.g. pickle, ZODB being a prominent example. As soon as a Python 2.6 application starts writing to such storages, Python 2.5 and lower versions will no longer be able to read back all the data. Now, I think there's a way to solve this puzzle: Instead of renaming the modules (e.g. Queue -> queue), we leave the code in the existing modules and packages and instead add the new module names and package structure with pointers and redirects to the existing 2.5 modules. Code can (and probably should) still be changed to try to import the new module name. In cases where backwards compatibility is needed, this can also be done using try: import newname except ImportError: import oldname Later on, when porting applications to 3.0, the 2to3 script can then apply the final renaming in the source code. Example: queue.py: --------- import sys, Queue sys.modules[__name__] = Queue -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 17 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From alexandre at peadrop.com Sat May 17 16:59:05 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sat, 17 May 2008 10:59:05 -0400 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <482E9FF3.4060200@egenix.com> References: <482E9FF3.4060200@egenix.com> Message-ID: On Sat, May 17, 2008 at 5:05 AM, M.-A. Lemburg wrote: > I'd like to bring a potential problem to attention that is caused > by the recent module renaming approach: > > Object serialization protocols like e.g. pickle usually store the > complete module path to the object class together with the object. Thanks for bringing this up. I was aware of the problem myself, but I hadn't yet worked out a good solution to it. > It can also happen in storage setups where Python > objects are stored using e.g. pickle, ZODB being a prominent > example. As soon as a Python 2.6 application starts writing to > such storages, Python 2.5 and lower versions will no longer be > able to read back all the data. > The opposite problem exists for Python 3.0, too. Pickle streams written by Python 2.x applications will not be readable by Python 3.0. And, one solution to this is to use Python 2.6 to regenerate pickle stream. Another solution would be to write a 2to3 pickle converter using the pickletools module. It is surely not the most elegant or robust solution, but I could work. > Now, I think there's a way to solve this puzzle: > > Instead of renaming the modules (e.g. Queue -> queue), we leave > the code in the existing modules and packages and instead add > the new module names and package structure with pointers and > redirects to the existing 2.5 modules. This would certainly work for simple modules, but what about packages? For packages, you can't use the ``sys.modules[__name__] = Queue`` to preserve module identity. Therefore, pickle will use the new package name when writing its streams. So, we are back to the same problem again. A possible solution could be writing a compatibility layer for the Pickler class, which would map new module names to their old at runtime. Again, this is neither an elegant, nor robust, solution, but it should work in most cases. -- Alexandre From alexandre at peadrop.com Sat May 17 17:07:07 2008 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sat, 17 May 2008 11:07:07 -0400 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> Message-ID: Errata: On Sat, May 17, 2008 at 10:59 AM, Alexandre Vassalotti wrote: > And, one solution to this is to use Python 2.6 to regenerate pickle > stream. ... to regenerate *the* pickle *streams*. > It is surely not the most elegant or robust solution, but I could work. ... but *it* could work. > This would certainly work for simple modules, but what about packages? > For packages, you can't use the ``sys.modules[__name__] = Queue`` to > preserve module identity. ... you can't use the ``sys.modules[__name__] = Queue`` *trick* to preserve module identity. > A possible solution could be writing a compatibility layer for the ... could be *to write* a compatibility layer... I guess I should start proofreading my emails before sending them, not after... -- Alexandre From steve at holdenweb.com Sat May 17 17:15:26 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 17 May 2008 11:15:26 -0400 Subject: [Python-Dev] platform module testing In-Reply-To: <482CC29E.3050204@cheimes.de> References: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> <482CC29E.3050204@cheimes.de> Message-ID: Christian Heimes wrote: > Benjamin Peterson schrieb: >> At the moment, the test for the platform module merely calls each >> function. I realize that this is a hard module to test well, but are >> there some assumptions we can make? For example, if sys.platform is >> 'java', can it be assumed that java_ver is going to return something >> which is not empty. When sys.platform is 'darwin' and the gestalt >> module is present, is it reasonable to require that a empty tuple not >> be returned? > > A student has created a patch during the GHOP contest which adds some > features and tests for Linux. AFAIK the patch hasn't been applied yet. > See http://bugs.python.org/issue1322 > Contributor agreement? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From ggpolo at gmail.com Sat May 17 18:02:53 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Sat, 17 May 2008 13:02:53 -0300 Subject: [Python-Dev] Possible problem that may be caused by Tkinter renaming Message-ID: Hello, I smell a problem caused by this line at tkinter/__init__.py: tkinter = _tkinter # b/w compat for export This was fine when tkinter was a Tkinter module, but I believe it would be better to rename this to something else. Given that python has a lot of users, I'm sure you will be able to find all sort of crazy things over the internet and one of them could be: import tkinter from tkinter import * something = tkinter._default_root Which is not that crazy at all, for some values of crazy, but will fail now. Regards, -- -- Guilherme H. Polo Goncalves From g.brandl at gmx.net Sat May 17 21:22:57 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 17 May 2008 21:22:57 +0200 Subject: [Python-Dev] platform module testing In-Reply-To: References: <1afaf6160805151454o198b04f3me51ffccfa8981601@mail.gmail.com> <482CC29E.3050204@cheimes.de> Message-ID: Steve Holden schrieb: > Christian Heimes wrote: >> Benjamin Peterson schrieb: >>> At the moment, the test for the platform module merely calls each >>> function. I realize that this is a hard module to test well, but are >>> there some assumptions we can make? For example, if sys.platform is >>> 'java', can it be assumed that java_ver is going to return something >>> which is not empty. When sys.platform is 'darwin' and the gestalt >>> module is present, is it reasonable to require that a empty tuple not >>> be returned? >> >> A student has created a patch during the GHOP contest which adds some >> features and tests for Linux. AFAIK the patch hasn't been applied yet. >> See http://bugs.python.org/issue1322 >> > Contributor agreement? Is present, see . Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From greg at krypto.org Sat May 17 21:51:29 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 17 May 2008 12:51:29 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> Message-ID: <52dc1c820805171251r5305c77ao8307e4047d9c1@mail.gmail.com> On Sat, May 17, 2008 at 7:59 AM, Alexandre Vassalotti wrote: > Another solution would be to write a 2to3 pickle converter using the > pickletools module. It is surely not the most elegant or robust > solution, but I could work. This could be done even for 2.x <--> 2.6 to be translate module names at unpickling and pickling time. IMHO thats preferable to leaving stub modules with the old names around. Anyways I'm not a heavy user of pickle so people who are should decide. From ggpolo at gmail.com Sat May 17 22:53:32 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Sat, 17 May 2008 17:53:32 -0300 Subject: [Python-Dev] Possible problem that may be caused by Tkinter renaming In-Reply-To: References: Message-ID: 2008/5/17 Guilherme Polo : > Hello, > > I smell a problem caused by this line at tkinter/__init__.py: > > tkinter = _tkinter # b/w compat for export Georg and me decided to remove that line as a solution to this problem. > > This was fine when tkinter was a Tkinter module, but I believe it > would be better to rename this to something else. > Given that python has a lot of users, I'm sure you will be able to > find all sort of crazy things over the internet and one of them could > be: > > import tkinter > from tkinter import * > > something = tkinter._default_root > > Which is not that crazy at all, for some values of crazy, but will fail now. > > > Regards, > > -- > -- Guilherme H. Polo Goncalves > -- -- Guilherme H. Polo Goncalves From greg.ewing at canterbury.ac.nz Sun May 18 00:22:35 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 18 May 2008 10:22:35 +1200 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> Message-ID: <482F5AAB.901@canterbury.ac.nz> Alexandre Vassalotti wrote: > On Sat, May 17, 2008 at 5:05 AM, M.-A. Lemburg wrote: > >>Object serialization protocols like e.g. pickle usually store the >>complete module path to the object class together with the object. > > The opposite problem exists for Python 3.0, too. This is just one manifestation of what I consider a serious shortcoming of the pickle format for long-term storage: it ties the data to implementation details of the program. When I brought this up earlier, various people assured me that it wasn't a problem in practice. I think we're seeing one situation here where it *is* a problem. -- Greg From glyph at divmod.com Sun May 18 01:44:38 2008 From: glyph at divmod.com (glyph at divmod.com) Date: Sat, 17 May 2008 23:44:38 -0000 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <482F5AAB.901@canterbury.ac.nz> References: <482E9FF3.4060200@egenix.com> <482F5AAB.901@canterbury.ac.nz> Message-ID: <20080517234438.25821.2020774119.divmod.xquotient.8407@joule.divmod.com> On 10:22 pm, greg.ewing at canterbury.ac.nz wrote: >When I brought this up earlier, various people assured >me that it wasn't a problem in practice. I think we're >seeing one situation here where it *is* a problem. Just my two cents here - experience has taught me that it's definitely a problem in practice. One big problem with pickle is that it's even difficult to tell when or how much your persistence format depends on your application code. For example, if you're pickling a dict that is supposed to map strings to integers, but you have a bug which accidentally ends up using a string subclass instead, it can be very difficult to figure out that this ever happened. pickletools is really neat, and can help with this problem once you're stuck, but it's a better idea to use a more explicit persistence mechanism in the first place if you can. From musiccomposition at gmail.com Sun May 18 02:42:52 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Sat, 17 May 2008 19:42:52 -0500 Subject: [Python-Dev] oct and hex Py3k warnings Message-ID: <1afaf6160805171742w474bdc65pb6ccdaedc371dcfc@mail.gmail.com> I know Guido has opposed adding Py3k warnings to oct and hex for "practical reasons," but I would like to make the case again. future_buitlins oct and hex will give you the correct Py3k behavior, but 2to3 can't convert old hex and oct usage to the new ones. That's why is would be helpful to warn about it and encourage people to import from future_builtins. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From mal at egenix.com Sun May 18 14:20:29 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Sun, 18 May 2008 14:20:29 +0200 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> Message-ID: <48301F0D.6080900@egenix.com> On 2008-05-17 16:59, Alexandre Vassalotti wrote: > On Sat, May 17, 2008 at 5:05 AM, M.-A. Lemburg wrote: >> I'd like to bring a potential problem to attention that is caused >> by the recent module renaming approach: >> >> Object serialization protocols like e.g. pickle usually store the >> complete module path to the object class together with the object. > > Thanks for bringing this up. I was aware of the problem myself, but I > hadn't yet worked out a good solution to it. > > >> It can also happen in storage setups where Python >> objects are stored using e.g. pickle, ZODB being a prominent >> example. As soon as a Python 2.6 application starts writing to >> such storages, Python 2.5 and lower versions will no longer be >> able to read back all the data. >> > > The opposite problem exists for Python 3.0, too. Pickle streams > written by Python 2.x applications will not be readable by Python 3.0. > And, one solution to this is to use Python 2.6 to regenerate pickle > stream. > > Another solution would be to write a 2to3 pickle converter using the > pickletools module. It is surely not the most elegant or robust > solution, but I could work. I'm not really worried much about going from 2.x to 3.x. Breakage is allowed for that transition. However, the case is different for going from 2.5 to 2.6. Breakage should be avoided if at all possible. >> Now, I think there's a way to solve this puzzle: >> >> Instead of renaming the modules (e.g. Queue -> queue), we leave >> the code in the existing modules and packages and instead add >> the new module names and package structure with pointers and >> redirects to the existing 2.5 modules. > > This would certainly work for simple modules, but what about packages? > For packages, you can't use the ``sys.modules[__name__] = Queue`` to > preserve module identity. Therefore, pickle will use the new package > name when writing its streams. So, we are back to the same problem > again. > > A possible solution could be writing a compatibility layer for the > Pickler class, which would map new module names to their old at > runtime. Again, this is neither an elegant, nor robust, solution, but > it should work in most cases. While it's possible to fix pickle (at least the Python version), this would not help with other serialization formats that rely on the .__module__ attribute mapping to an existing module. It's better to address the problem at the module level. Perhaps I have a misunderstanding of the reasoning behind doing the renaming in the 2.x branch, but it appears that the only reason is to get used to the new names. That's a rather low priority argument in comparison to the breakage the renaming will cause in the 2.x branch. I think it's much better to have 2to3.py do the renaming and only add warnings to the renamed modules in 2.x (without actually applying any renaming). It would also be possible to seed sys.modules with module proxy objects (see e.g. mx.Misc.LazyModule from egenix-mx-base) which only turn into real module object if the module is referenced. This would allow adding a "from __future__ import new_module_names" which then results in loading proxies for all renamed modules (without actually loading the modules until they are used under their new names). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 18 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ncoghlan at gmail.com Sun May 18 15:14:00 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 18 May 2008 23:14:00 +1000 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <48301F0D.6080900@egenix.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> Message-ID: <48302B98.7040304@gmail.com> M.-A. Lemburg wrote: > Perhaps I have a misunderstanding of the reasoning behind > doing the renaming in the 2.x branch, but it appears that > the only reason is to get used to the new names. That's a > rather low priority argument in comparison to the breakage > the renaming will cause in the 2.x branch. I think this is the key point here. The possibility of breaking pickling compatibility never came up during the PEP 3108 discussions, so wasn't taken into account in deciding whether or not backporting the name changes was a good idea. I think it's pretty clear that the code needs to be moved back into the modules with the old names for 2.6. The only question is whether or not we put any effort into making the new stdlib organisation usable in 2.x, or just rely on 2to3 to fix it (note that the "increasing the common subset" argument doesn't really apply, since you can catch the import errors in order to try both names). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fwierzbicki at gmail.com Sun May 18 20:49:09 2008 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Sun, 18 May 2008 14:49:09 -0400 Subject: [Python-Dev] Optimization of Python ASTs: How should we deal with constant values? In-Reply-To: <481B236D.80807@vector-seven.com> References: <481846D2.6090509@vector-seven.com> <4818D854.1060906@v.loewis.de> <481B236D.80807@vector-seven.com> Message-ID: <4dab5f760805181149y41fc1afdx8e2d16eddb34c93f@mail.gmail.com> On Fri, May 2, 2008 at 10:21 AM, Thomas Lee wrote: > Any Jython folk care to weigh in on this? If there are no major objections I > think I'm going to forge ahead with an independant Const() node. I suspect that having a marker for non-int non-str constants could also be used for some optimization of Jython bytecode emissions as well, though of course it would depend on the details. If the path from raw AST to optimized AST isn't too crazy Jython can just grow the same logic. We have really good AST comparison tests these days, so I bet it won't be a huge problem for us when the time comes. -Frank From fwierzbicki at gmail.com Sun May 18 21:05:59 2008 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Sun, 18 May 2008 15:05:59 -0400 Subject: [Python-Dev] Documentation for ability to execute zipfiles & directories In-Reply-To: <20080304173605.GA9119@amk-desktop.matrixgroup.net> References: <47CD421E.2010402@gmail.com> <79990c6b0803040455q7ce211d0y78956b3a395c3bad@mail.gmail.com> <20080304173605.GA9119@amk-desktop.matrixgroup.net> Message-ID: <4dab5f760805181205s4b2e61u6480374150344cea@mail.gmail.com> On Tue, Mar 4, 2008 at 1:36 PM, A.M. Kuchling wrote: > On Tue, Mar 04, 2008 at 08:58:57AM -0500, Steve Holden wrote: >> While I hesitate to suggest a change of such magnitude, there's >> something to recommend the old IBM mainframe approach of separating out >> "Principles of Operation" (which would be the reference manuals, in >> Python's case the Language and Library refs) from "Users' Guide" which >> contains the practical stuff you need to actually make use of a product. > > Good suggestion. Using the debugger and profiler could also be > covered in the User's Guide. > > Would splitting up the docs make them more useful for > IronPython/Jython? For example, Jython could eventually take the 2.6 > language docs as-is, but modify the library reference to remove > unsupported modules and add Jython-specific ones. Speaking for Jython, this would be extremely helpful for us. Once we get caught up, better docs will become one of our most important priorities I think. -Frank From martin at v.loewis.de Sun May 18 22:02:56 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 18 May 2008 22:02:56 +0200 Subject: [Python-Dev] Visual Studio 2008 compiler option EHsc ? In-Reply-To: References: <482D1B44.3090507@v.loewis.de> Message-ID: <48308B70.4020209@v.loewis.de> > Is there any downside to doing this by default? The most obvious one is the (slight) loss of performance. The mere fact that it is an option, and not enabled by default, indicates that there might be problems enabling it. One problem might be that some applications may require support for asynchronous (structured) exceptions and want to specify /EHa, or that they have cases where C functions throw C++ exceptions, so they want to specify /EHs (no c). So yes, unfortunately, you'll have to add the option into your own setup.py, it seems. Regards, Martin From brett at python.org Sun May 18 22:24:45 2008 From: brett at python.org (Brett Cannon) Date: Sun, 18 May 2008 13:24:45 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <48302B98.7040304@gmail.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> Message-ID: On Sun, May 18, 2008 at 6:14 AM, Nick Coghlan wrote: > M.-A. Lemburg wrote: >> >> Perhaps I have a misunderstanding of the reasoning behind >> doing the renaming in the 2.x branch, but it appears that >> the only reason is to get used to the new names. That's a >> rather low priority argument in comparison to the breakage >> the renaming will cause in the 2.x branch. > > I think this is the key point here. The possibility of breaking pickling > compatibility never came up during the PEP 3108 discussions, so wasn't taken > into account in deciding whether or not backporting the name changes was a > good idea. > > I think it's pretty clear that the code needs to be moved back into the > modules with the old names for 2.6. The only question is whether or not we > put any effort into making the new stdlib organisation usable in 2.x, or > just rely on 2to3 to fix it (note that the "increasing the common subset" > argument doesn't really apply, since you can catch the import errors in > order to try both names). Problem with this is it makes forward-porting revisions to 3.0 a PITA. By keeping the module names consistent between the versions merging a revision is just a matter of ``svnmerge merge`` with the usual 3.0-specific changes. Reverting the modules back to the old name will make forward-porting much more difficult as I don't think svn keeps rename information around (and thus map the old name to the new name in terms of diffs). Alexandre's idea of teaching pickle the mapping of old names to new might be the best solution. We could have a flag to pickle that deactivates the renaming. Otherwise we could bump the pickle version number so that the new number doesn't do the mapping while the old versions to the implicit module mapping. And as Greg and Glpyh have pointed out, this is a problem that might need to be addressed in the future with some changes to our serialization method (I have no clue how since I don't deal with pickle very much). -Brett From dlenski at gmail.com Mon May 19 00:59:05 2008 From: dlenski at gmail.com (Dan Lenski) Date: Sun, 18 May 2008 22:59:05 +0000 (UTC) Subject: [Python-Dev] buffer interface for C extensions Message-ID: Hi all, I've written a small C extension to submit commands to SCSI devices via Linux's sg_io driver (for a camera hacking project). The extension is just a wrapper around a couple ioctl()'s with Pythonic exception handling thrown in. One of my extension methods is called like this from python: sg.write(fd, command[, data, timeout) Both command and data are binary strings. I would like to be able to use either a regular Python string or an array('B', ...) for these read-only arguments. So I tried to use the "t#" argument specifier to indicate that these arguments could be either strings or objects that implement the read- only buffer interface: if (!PyArg_ParseTuple(args, "it#|t#i:write", &sg_fd, &cmd, &cmdLen, &buf, &bufLen, &timeout)) return NULL; Now, this works fine with strings, but when I call it with an array I get a TypeError: TypeError: write() argument 2 must be string or read-only character buffer, not array.array So, I then tried changing "t#" to "w#" to indicate that the arguments must implement the /read-write/ buffer interface. Now the array objects work, but when I try a string argument, I naturally get this error: TypeError: Cannot use string as modifiable buffer So here's what I don't understand. Why doesn't the "t#" argument specifier support read-write buffers as well as read-only buffers? Aren't read-write buffers a *superset* of read-only buffers?? Is there something I'm doing wrong or a quick fix to get this to work appropriately? Thanks for any help! Dan From hitchmanr at gmail.com Mon May 19 03:45:07 2008 From: hitchmanr at gmail.com (Ryan Hitchman) Date: Sun, 18 May 2008 19:45:07 -0600 Subject: [Python-Dev] Conditional For Statements Message-ID: I'd like to propose an addition to the for statement's syntax: for {variable} in {iterable} if {condition}: {block} which is equivalent to for {variable} in {iterable}: if not {condition}: continue {block} and for {variable} in filter(lambda: {condition}, iterable): {block} This would make the syntax closer to that of generators, which have 'for variable in iterable if condition', and would improve code clarity by increased brevity and not negating boolean expressions. Following are examples of current code with what the new code would be, taken from the Python 3.0a5 tarball. Demo/tkinter/guido/ss1.py:163: for (x, y), cell in self.cells.items(): if x <= 0 or y <= 0: continue for (x, y), cell in self.cells.items() if x > 0 and y > 0: Lib/encodings/__init__.py:91: for modname in modnames: if not modname or '.' in modname: continue for modname in modnames if modname and '.' not in modname: Lib/idlelib/AutoExpand.py:70: for w in wafter: if dict.get(w): continue for w in wafter if w not in dict: Lib/Cookie.py:483: for K,V in items: if V == "": continue if K not in attrs: continue for K,V in items if V != "" and K not in attrs: Lib/hashlib.py:108: for opensslFuncName in filter(lambda n: n.startswith('openssl_'), dir(_hashlib)): for opensslFuncName in dir(_hashlib) if opensslFuncName.startswith('openssl_'): There are many more examples of this in the standard library, and likely even more in production code. I am not familiar with LL(1) parsing, so this may impossible under that constraint. From eli at courtwright.org Mon May 19 03:57:24 2008 From: eli at courtwright.org (Eli Courtwright) Date: Sun, 18 May 2008 21:57:24 -0400 Subject: [Python-Dev] Conditional For Statements In-Reply-To: References: Message-ID: <3f6c86f50805181857s1f627d8n319d27699ef35853@mail.gmail.com> First, the best list on which to discuss this would be Python-Ideas not Python-Dev. Second, this was brought up there a few months ago, although not much discussion ensued: http://mail.python.org/pipermail/python-ideas/2008-March/001407.html - Eli On Sun, May 18, 2008 at 9:45 PM, Ryan Hitchman wrote: > I'd like to propose an addition to the for statement's syntax: > > for {variable} in {iterable} if {condition}: > {block} > > which is equivalent to > > for {variable} in {iterable}: > if not {condition}: > continue > {block} > > and > > for {variable} in filter(lambda: {condition}, iterable): > {block} > > This would make the syntax closer to that of generators, which have > 'for variable in iterable if condition', and would improve code > clarity by increased brevity and not negating boolean expressions. > > Following are examples of current code with what the new code would > be, taken from the Python 3.0a5 tarball. > > Demo/tkinter/guido/ss1.py:163: > for (x, y), cell in self.cells.items(): > if x <= 0 or y <= 0: > continue > for (x, y), cell in self.cells.items() if x > 0 and y > 0: > > Lib/encodings/__init__.py:91: > for modname in modnames: > if not modname or '.' in modname: > continue > for modname in modnames if modname and '.' not in modname: > > Lib/idlelib/AutoExpand.py:70: > for w in wafter: > if dict.get(w): > continue > for w in wafter if w not in dict: > > Lib/Cookie.py:483: > for K,V in items: > if V == "": continue > if K not in attrs: continue > for K,V in items if V != "" and K not in attrs: > > Lib/hashlib.py:108: > for opensslFuncName in filter(lambda n: n.startswith('openssl_'), > dir(_hashlib)): > for opensslFuncName in dir(_hashlib) if > opensslFuncName.startswith('openssl_'): > > There are many more examples of this in the standard library, and > likely even more in production code. > > I am not familiar with LL(1) parsing, so this may impossible under > that constraint. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/eli%40courtwright.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ijmorlan at cs.uwaterloo.ca Mon May 19 04:07:15 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Sun, 18 May 2008 22:07:15 -0400 (EDT) Subject: [Python-Dev] Conditional For Statements In-Reply-To: References: Message-ID: On Sun, 18 May 2008, Ryan Hitchman wrote: > I'd like to propose an addition to the for statement's syntax: > > for {variable} in {iterable} if {condition}: > {block} > > which is equivalent to > > for {variable} in {iterable}: > if not {condition}: > continue > {block} > > and > > for {variable} in filter(lambda: {condition}, iterable): > {block} > > This would make the syntax closer to that of generators, which have > 'for variable in iterable if condition', and would improve code > clarity by increased brevity and not negating boolean expressions. http://mail.python.org/pipermail/python-dev/2006-May/065090.html This was proposed and rejected before, after which it was again proposed and rejected. I think it was also proposed and rejected sometime in 2007, but I can't seem to find it in the archives so my memory may be faulty. The idea is apparently not dead yet. Hang around a couple of minutes, though. It won't be long. Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From tjreedy at udel.edu Mon May 19 05:55:28 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 18 May 2008 23:55:28 -0400 Subject: [Python-Dev] Conditional For Statements References: Message-ID: "Ryan Hitchman" wrote in message news:e60c31470805181845u7fb0cafdsb0878db491eb7a3c at mail.gmail.com... | I'd like to propose an addition to the for statement's syntax: | | for {variable} in {iterable} if {condition}: | {block} | | which is equivalent to | | for {variable} in {iterable}: | if not {condition}: | continue | {block} Is this different from for var in filter(lambda x: condition(x), iterable)): ? | and | | for {variable} in filter(lambda: {condition}, iterable): | {block} IDLE 3.0a5 >>> for i in filter(lambda i: i%2, range(10)): print(i) 1 3 5 7 9 From ulrich.berning at denviso.de Mon May 19 10:05:09 2008 From: ulrich.berning at denviso.de (Ulrich Berning) Date: Mon, 19 May 2008 10:05:09 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <482D8FA4.7040509@gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> Message-ID: <483134B5.9000300@denviso.de> Nick Coghlan wrote: > Ulrich Berning wrote: > >> More and more people tend to say "Runs on Un*x" when they really mean >> "Tested on Linux". Un*x is not Linux. > > > Hmm, perhaps that would be why there are Solaris, FreeBSD and Tru64 > machines amongst the main Python buildbots, to go along with the > assorted OS X, Windows and Linux boxes - and as far as I know > test_ctypes runs quite happily on all of them. > > On the specific problems with AIX, HP-UX and ctypes, was it ctypes > itself that was failing to build, or the underlying libffi? > > Cheers, > Nick. > On HP-UX-11.00, HP ANSI C++ B3910B A.03.73, Python-2.5.2, I get configure: error: "libffi has not been ported to hppa2.0w-hp-hpux11.00." On AIX-4.3.3, C for AIX Compiler Version 6, Python-2.5.2, I get "build/temp.aix-4.3-2.5/libffi/include/ffi.h", line 123.4: 1506-205 (S) #error "no 64-bit data type supported" On Solaris-10/x86, Sun C 5.8 Patch 121016-07 2007/10/03, Python-2.5.2, I get "build/temp.solaris-2.10-i86pc-2.5/libffi/include/ffitarget.h", line 64: undefined symbol: FFI_DEFAULT_ABI On Solaris-8/sparc, Sun C 5.8 2005/10/13, Python-2.5.2, I get "build/temp.solaris-2.8-sun4u-2.5/libffi/include/ffi.h", line 225: syntax error before or at: __attribute__ On IRIX-6.5, gcc-3.4.4, Python-2.5.2, ffi_closure is undefined, because only the old O32 binary format is supported, not the new N32/N64 format. I'm trying to use the vendor specific compilers whenever possible, because using gcc puts in additional dependencies (libgcc), I want to avoid, and even if I could live with these dependencies, it's not easy to get/build the 'right' gcc version, if your software also depends on other big packages like Qt and PyQt. I'm not using these platforms for my own pleasure (in fact, I would be happy if these platforms would disappear from the market), but as long as our customers use these platforms, we want to promise our software runs on those platforms. I have no problem with the fact that ctypes doesn't build on those platforms because I don't use it, but if more and more essential packages depend on ctypes, I'm running into trouble. PyOpenGL is an example of an extension, that moved completely from C-Source (SWIG generated) to ctypes usage. Ulli From ulrich.berning at denviso.de Mon May 19 11:03:27 2008 From: ulrich.berning at denviso.de (Ulrich Berning) Date: Mon, 19 May 2008 11:03:27 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> Message-ID: <4831425F.2050102@denviso.de> Gregory P. Smith wrote: >On Fri, May 16, 2008 at 1:32 AM, Ulrich Berning > wrote: > > >>As long as the ctypes extension doesn't build on major Un*x platforms (AIX, >>HP-UX), I don't like to see ctypes dependend modules included into the >>stdlib. Please keep the stdlib as portable as possible. >> >> > >Nice in theory but ctypes already works on at least the top 3 popular >platforms. Lets not hold Python's stdlib back because nobody who uses >IBM and HP proprietary stuff has contributed the necessary support. >Making nice libraries available for other platforms is a good way to >encourage people to either pitch in and add support or consider their >platform choices in the future. > >-gps > > > It's not my platform choice, it's the choice of our customers. I'm not using these platforms just for fun (in fact it isn't fun compared to Linux or Windows). If porting libffi to AIX, HP-UX, IRIX, Solaris... (especially using vendor compilers) would be an easy job, I'm sure it would have been done already. If more and more essential packages depend on ctypes, we should make a clear statement, that Python isn't supported any longer on platform/compiler combinations where libffi/ctypes doesn't build. This would give me arguments to drop support of our software on those platforms. Ulli From ncoghlan at gmail.com Mon May 19 12:30:40 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 19 May 2008 20:30:40 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <483134B5.9000300@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> Message-ID: <483156D0.8010801@gmail.com> Ulrich Berning wrote: > I'm trying to use the vendor specific compilers whenever possible, > because using gcc puts in additional dependencies (libgcc), I want to > avoid, and even if I could live with these dependencies, it's not easy > to get/build the 'right' gcc version, if your software also depends on > other big packages like Qt and PyQt. > > I'm not using these platforms for my own pleasure (in fact, I would be > happy if these platforms would disappear from the market), but as long > as our customers use these platforms, we want to promise our software > runs on those platforms. > > I have no problem with the fact that ctypes doesn't build on those > platforms because I don't use it, but if more and more essential > packages depend on ctypes, I'm running into trouble. PyOpenGL is an > example of an extension, that moved completely from C-Source (SWIG > generated) to ctypes usage. Hmm, perhaps the ctypes documentation could use a more prominent warning that it may not be available on some Unix platforms (HP-UX, AIX, IRIX), and that it may require the use of GCC rather than the vendor compiler on others (Solaris). At the moment, I suspect some projects may be switching to using it without realising the implications for cross-platform portability. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From regebro at gmail.com Mon May 19 12:42:29 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 19 May 2008 12:42:29 +0200 Subject: [Python-Dev] Conditional For Statements In-Reply-To: References: Message-ID: <319e029f0805190342x70b30ecbi30ca58781e16e8df@mail.gmail.com> On Mon, May 19, 2008 at 5:55 AM, Terry Reedy wrote: > > "Ryan Hitchman" wrote in message > news:e60c31470805181845u7fb0cafdsb0878db491eb7a3c at mail.gmail.com... > | I'd like to propose an addition to the for statement's syntax: > | > | for {variable} in {iterable} if {condition}: > | {block} > | > | which is equivalent to > | > | for {variable} in {iterable}: > | if not {condition}: > | continue > | {block} > > Is this different from > for var in filter(lambda x: condition(x), iterable)): > ? > > | and > | > | for {variable} in filter(lambda: {condition}, iterable): > | {block} > > IDLE 3.0a5 >>>> for i in filter(lambda i: i%2, range(10)): > print(i) > > > 1 > 3 > 5 > 7 > 9 How was it again? "One and only one way"? :-) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From mal at egenix.com Mon May 19 14:08:23 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 19 May 2008 14:08:23 +0200 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> Message-ID: <48316DB7.2040506@egenix.com> On 2008-05-18 22:24, Brett Cannon wrote: > On Sun, May 18, 2008 at 6:14 AM, Nick Coghlan wrote: >> M.-A. Lemburg wrote: >>> Perhaps I have a misunderstanding of the reasoning behind >>> doing the renaming in the 2.x branch, but it appears that >>> the only reason is to get used to the new names. That's a >>> rather low priority argument in comparison to the breakage >>> the renaming will cause in the 2.x branch. >> I think this is the key point here. The possibility of breaking pickling >> compatibility never came up during the PEP 3108 discussions, so wasn't taken >> into account in deciding whether or not backporting the name changes was a >> good idea. >> >> I think it's pretty clear that the code needs to be moved back into the >> modules with the old names for 2.6. The only question is whether or not we >> put any effort into making the new stdlib organisation usable in 2.x, or >> just rely on 2to3 to fix it (note that the "increasing the common subset" >> argument doesn't really apply, since you can catch the import errors in >> order to try both names). > > Problem with this is it makes forward-porting revisions to 3.0 a PITA. > By keeping the module names consistent between the versions merging a > revision is just a matter of ``svnmerge merge`` with the usual > 3.0-specific changes. Reverting the modules back to the old name will > make forward-porting much more difficult as I don't think svn keeps > rename information around (and thus map the old name to the new name > in terms of diffs). svnmerge is written in Python, so wouldn't it be possible to add support for maintaining such renaming to that tool ? I don't think that an administrative problem such as forward- porting patches to 3.x warrants breakage in the 2.x branch. After all, the renaming was approached for Python 3.0 and not 2.6 *because* it introduces major breakage. AFAIR, the discussion on the stdlib-sig also didn't include the plan to backport such changes to 2.6. Otherwise, we would have hashed them out there. > Alexandre's idea of teaching pickle the mapping of old names to new > might be the best solution. We could have a flag to pickle that > deactivates the renaming. Otherwise we could bump the pickle version > number so that the new number doesn't do the mapping while the old > versions to the implicit module mapping. > > And as Greg and Glpyh have pointed out, this is a problem that might > need to be addressed in the future with some changes to our > serialization method (I have no clue how since I don't deal with > pickle very much). It is possible to make pickle aware of the module renames, but that doesn't solve problems with other forms of serialization or use of the .__module__ attribute in general. Why can't we just provide a "from __future__ import renamed_modules" which then provides all the new name to old name mappings in some form (e.g. module proxies or whatever) and leave the existing modules in 2.x untouched ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 19 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ncoghlan at gmail.com Mon May 19 14:19:25 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 19 May 2008 22:19:25 +1000 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <48316DB7.2040506@egenix.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> Message-ID: <4831704D.1060201@gmail.com> M.-A. Lemburg wrote: > I don't think that an administrative problem such as forward- > porting patches to 3.x warrants breakage in the 2.x branch. > > After all, the renaming was approached for Python 3.0 and not > 2.6 *because* it introduces major breakage. > > AFAIR, the discussion on the stdlib-sig also didn't include the > plan to backport such changes to 2.6. Otherwise, we would have > hashed them out there. I think MAL is 100% correct here (and I expect Raymond will chime in to support him at some point as well). Taking the time to fix out mistake may mean we need to do another alpha rather than being able to go into the betas as planned, but I think that would be a much better option than breaking any 2.x code that relies on __module__ staying the same across releases. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From steve at pearwood.info Mon May 19 14:33:02 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 19 May 2008 22:33:02 +1000 Subject: [Python-Dev] Conditional For Statements In-Reply-To: <319e029f0805190342x70b30ecbi30ca58781e16e8df@mail.gmail.com> References: <319e029f0805190342x70b30ecbi30ca58781e16e8df@mail.gmail.com> Message-ID: <200805192233.03069.steve@pearwood.info> On Mon, 19 May 2008 08:42:29 pm Lennart Regebro wrote: > How was it again? "One and only one way"? :-) Certainly not. What on earth gave you the idea that that ridiculous statement is a Python philosophy? I know some Perl developers like to contrast the supposed flexibility of Perl ("more than one way to do it") with the imagined poverty of Python, but that really is a silly claim to make about any Turing Complete language. If nothing else, any programming language that lets you perform arithmetic would not be so restrictive: x = (1+2)*3 + 3*3 x = (3+2)*3 + 1*3 Which one should the compiler prohibit? I strongly suggest that you look at the Zen of Python: >>> import this The Zen of Python, by Tim Peters ... There should be one-- and preferably only one --obvious way to do it. There should be ONE OBVIOUS way to do it, not "only one way". -- Steven From tjreedy at udel.edu Mon May 19 14:45:05 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 19 May 2008 08:45:05 -0400 Subject: [Python-Dev] Module renaming and pickle mechanisms References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com><48302B98.7040304@gmail.com><48316DB7.2040506@egenix.com> <4831704D.1060201@gmail.com> Message-ID: "Nick Coghlan" wrote in message news:4831704D.1060201 at gmail.com... | M.-A. Lemburg wrote: | > I don't think that an administrative problem such as forward- | > porting patches to 3.x warrants breakage in the 2.x branch. | > | > After all, the renaming was approached for Python 3.0 and not | > 2.6 *because* it introduces major breakage. | > | > AFAIR, the discussion on the stdlib-sig also didn't include the | > plan to backport such changes to 2.6. Otherwise, we would have | > hashed them out there. | | I think MAL is 100% correct here (and I expect Raymond will chime in to | support him at some point as well). For what little it's worth, I was surprised too that the 3.0 renames were backported as thr default versions. It strikes me as possibly a 'bridge too far' ;-). tjr From ygingras at ygingras.net Mon May 19 15:04:25 2008 From: ygingras at ygingras.net (Yannick Gingras) Date: Mon, 19 May 2008 09:04:25 -0400 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: (Alexandre Vassalotti's message of "Fri\, 16 May 2008 12\:58\:15 -0400") References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> <87y76ae1xp.fsf@enceladus.ygingras.net> Message-ID: <87abimcuna.fsf@enceladus.ygingras.net> "Alexandre Vassalotti" writes: >>> So now I am not sure what OP is proposing. Do you want to replace 21 >>> with EISDIR in the above? >> >> Yes, that's what I had in mind. >> > > Then, check out EnvironmentError_str in Objects/exceptions.c. You > should be able import the errno module and fetch its errorcode > dictionary. It wasn't as hard as I expected. It's the first time that I play with the Python C API; I didn't expect the API to be that high level. I attached a patch to convert errno to its symbolic value when an EnvironmentError is printed. Should attach it to a ticket on bugs.python.org? I'm sure there is a style guide like PEP-8 for C code, feel free to point me to it because my patch is probably not fully style compliant. With Emacs, doing M-x c-set-style python doesn't seems to do the right thing. Are you all using a bunch of shared settings in you .emacs files? -- Yannick Gingras -------------- next part -------------- A non-text attachment was scrubbed... Name: python-26-sympolic-errno.diff Type: text/x-diff Size: 2288 bytes Desc: not available URL: From mhammond at skippinet.com.au Mon May 19 15:35:06 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 19 May 2008 23:35:06 +1000 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <4831704D.1060201@gmail.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> <4831704D.1060201@gmail.com> Message-ID: <02ba01c8b9b5$27245210$756cf630$@com.au> Nick writes: > M.-A. Lemburg wrote: > > I don't think that an administrative problem such as forward- > > porting patches to 3.x warrants breakage in the 2.x branch. > > > > After all, the renaming was approached for Python 3.0 and not > > 2.6 *because* it introduces major breakage. > > > > AFAIR, the discussion on the stdlib-sig also didn't include the > > plan to backport such changes to 2.6. Otherwise, we would have > > hashed them out there. > > I think MAL is 100% correct here (and I expect Raymond will chime in to > support him at some point as well). And until then, a +1 for MAL's position from me as well. 2.x should be quite conservative about such changes... Cheers, Mark From ncoghlan at gmail.com Mon May 19 16:16:51 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 May 2008 00:16:51 +1000 Subject: [Python-Dev] Conditional For Statements In-Reply-To: References: Message-ID: <48318BD3.9050304@gmail.com> Ryan Hitchman wrote: > This would make the syntax closer to that of generators, which have > 'for variable in iterable if condition', Incorporating the filter condition into the list comprehension syntax (and later into generator expressions) is necessary because they need to be written as a single expression. The same doesn't hold true for loop filters: it is quite possible to keep the looping and filtering concerns separated into different statements, and there's no compelling reason for merging them. > would improve code clarity by increased brevity Increased brevity != improved code clarity (in fact, being too terse can obfuscate things - just look at Perl) > not negating boolean expressions. Except, of course, for those cases where it is easier to define the condition for items to skip than it is to define the condition for items to include. > Following are examples of current code with what the new code would > be, taken from the Python 3.0a5 tarball. Using just the standard idiom of a separate if statement to do the filtering: for (x, y), cell in self.cells.items(): if x <= 0 or y <= 0: continue # ... for modname in modnames: if not modname or '.' in modname: continue # ... for w in wafter: if dict.get(w): continue # ... for K,V in items: if V == "": continue if K not in attrs: continue # ... for opensslFuncName in dir(_hashlib): if opensslFuncName.startswith('openssl_'): continue # ... I'm not really seeing how the readability of any of those examples is enhanced by being able to merge the filter condition into the first line of the for loop. The one example which was already written that way through the use of filter() is actually easy to read with the addition of the vertical whitespace needed for the idiomatic form. The separate statement approach also scales far more cleanly to more complex filter conditions. For example, some code of mine includes a loop that looks like: for name in dirnames: if not os.path.exists(name): continue if not os.path.isdir(name): continue # Process named directory This conveys the order of checks far more clearly than would be the case if the os.path calls were hidden on the end of the loop invocation. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From aleaxit at gmail.com Mon May 19 16:57:33 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Mon, 19 May 2008 07:57:33 -0700 Subject: [Python-Dev] Conditional For Statements In-Reply-To: <200805192233.03069.steve@pearwood.info> References: <319e029f0805190342x70b30ecbi30ca58781e16e8df@mail.gmail.com> <200805192233.03069.steve@pearwood.info> Message-ID: On Mon, May 19, 2008 at 5:33 AM, Steven D'Aprano wrote: ... >>>> import this > The Zen of Python, by Tim Peters > ... > There should be one-- and preferably only one --obvious way to do it. > > There should be ONE OBVIOUS way to do it, not "only one way". The "only one way" phrasing is in the "Spirit of C" section of the ISO C Standard `Rationale' (non-normative, but highly informative), specifically in point four, "provide only one way to do an operation". I contend that the somewhat different phrasing is somewhat accidental and not particularly relevant: the *intent* of this *language design guideline* is exactly the same (Tim was writing on his own, while the C Standard was written by a committee, so it's hardly surprising that the Standard's wording is drier and less funny than Tim's;-). IOW, I've often argued that this is a deep commonality between the design philosophy of C and Python (together with "trust the programmer" and "keep the language small and simple", two more points in the "Spirit of C" section) which makes them conceptually "closer" to each other than C is to languages sharing more of its syntax, from C++ to Javascript; it should be particularly easy to see the contrast with languages deliberately designed to provide multiple equivalent synonyms (such as foo.size and foo.length meaning the same thing in Ruby) or going out of their way to provide multiple equivalent syntax approaches for exactly the same semantics (such as Cobol's "ADD A TO B GIVING C" and "C = A + B", or SQL's explicit [INNER] JOIN form and the equivalent form where the inner join is left implicit in the WHERE clause). A design guideline is not a strict rule -- e.g., every C programmer has met situations where a while statement, and a for statement with some of the three subparts in the control clause omitted, can both make sense, and of course every time you write within your loop some "if(condition) continue" you wonder if in this case it might not be better to instead use "if(!condition) { ... }" around the rest of the loop's body, etc, etc -- so much for "only one way", as the various ways to perform elementary operations can click together in a large number of combinations and permutations; and adding "obvious" would not help much either, since both alternatives can easily happen to be "equally obvious". Python is just in the same situation. Nevertheless, keeping the guideline firmly in mind (e.g., eschewing synonyms in APIs you design that provide no added value beyond that of catering to different tastes or to a meta-taste for variety over uniformity;-) is still a helpful heuristic -- with or without the "obvious" word (i.e., whether you're working with C, which doesn't have the "obvious" in its phrasing of the principle, or with Python, which does). Alex From guido at python.org Mon May 19 17:10:29 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 19 May 2008 08:10:29 -0700 Subject: [Python-Dev] oct and hex Py3k warnings In-Reply-To: <1afaf6160805171742w474bdc65pb6ccdaedc371dcfc@mail.gmail.com> References: <1afaf6160805171742w474bdc65pb6ccdaedc371dcfc@mail.gmail.com> Message-ID: On Sat, May 17, 2008 at 5:42 PM, Benjamin Peterson wrote: > I know Guido has opposed adding Py3k warnings to oct and hex for > "practical reasons," but I would like to make the case again. > > future_buitlins oct and hex will give you the correct Py3k behavior, > but 2to3 can't convert old hex and oct usage to the new ones. That's > why is would be helpful to warn about it and encourage people to > import from future_builtins. But the difference only matters for people who are parsing the output, right? I think they'd find out soon enough. I'm still -1. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Mon May 19 17:39:40 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 19 May 2008 08:39:40 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com><4831704D.1060201@gmail.com> <02ba01c8b9b5$27245210$756cf630$@com.au> Message-ID: <009401c8b9c6$8e719250$ac00a8c0@RaymondLaptop1> > Nick writes: >> M.-A. Lemburg wrote: >> > I don't think that an administrative problem such as forward- >> > porting patches to 3.x warrants breakage in the 2.x branch. >> > >> > After all, the renaming was approached for Python 3.0 and not >> > 2.6 *because* it introduces major breakage. >> > >> > AFAIR, the discussion on the stdlib-sig also didn't include the >> > plan to backport such changes to 2.6. Otherwise, we would have >> > hashed them out there. >> >> I think MAL is 100% correct here (and I expect Raymond will chime in to >> support him at some point as well). > > And until then, a +1 for MAL's position from me as well. 2.x should be > quite conservative about such changes... I concur. Raymond From regebro at gmail.com Mon May 19 18:00:43 2008 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 19 May 2008 18:00:43 +0200 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <48316DB7.2040506@egenix.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> Message-ID: <319e029f0805190900u3e6b0f79xc9a0369d08510485@mail.gmail.com> On Mon, May 19, 2008 at 2:08 PM, M.-A. Lemburg wrote: > Why can't we just provide a "from __future__ import renamed_modules" > which then provides all the new name to old name mappings in > some form (e.g. module proxies or whatever) and leave the > existing modules in 2.x untouched ? If I understand this correctly, the pickles would then be compatible between 2.6 and 2.5, unless you did from __future__ import renamed_modules, which would make the pickles compatible between 2.6 and 3.0. This sounds like the best solution to me, especially if the old names are still available after the future import, as all that would then be needed it to repickle all the pickles to convert from 2.5 to 3.0 pickles, right? So, if I understood this correctly, that sounds like a perfect solution. :) -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From guido at python.org Mon May 19 18:22:54 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 19 May 2008 09:22:54 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <009401c8b9c6$8e719250$ac00a8c0@RaymondLaptop1> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> <4831704D.1060201@gmail.com> <02ba01c8b9b5$27245210$756cf630$@com.au> <009401c8b9c6$8e719250$ac00a8c0@RaymondLaptop1> Message-ID: On Mon, May 19, 2008 at 8:39 AM, Raymond Hettinger wrote: >> Nick writes: >>> >>> M.-A. Lemburg wrote: >>> > I don't think that an administrative problem such as forward- >>> > porting patches to 3.x warrants breakage in the 2.x branch. >>> > >>> > After all, the renaming was approached for Python 3.0 and not >>> > 2.6 *because* it introduces major breakage. >>> > >>> > AFAIR, the discussion on the stdlib-sig also didn't include the >>> > plan to backport such changes to 2.6. Otherwise, we would have >>> > hashed them out there. >>> >>> I think MAL is 100% correct here (and I expect Raymond will chime in to >>> support him at some point as well). >> >> And until then, a +1 for MAL's position from me as well. 2.x should be >> quite conservative about such changes... > > I concur. And a "me too" post about being conservative by default as well. I'm not sure how effective a "from __future__ import renamed_modules" would be, since such future imports are meant to affect the semantics of the *current* module only, whereas which name to use when pickling a module reference is most likely a global choice. So perhaps some other way to changing the default behavior globally would be more appropriate. Assuming it's really the pickle module that needs to know about this, how about making this a per-Pickler-instance flag? if you wanted to write 3.0 compatible pickles you'd have to do something like p = pikle.Pickler() p.use_new_module_names(True) pkl = p.dump() We could supply an extra flag to the dump() and dumps() convenience functions as well. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Mon May 19 19:02:36 2008 From: janssen at parc.com (Bill Janssen) Date: Mon, 19 May 2008 10:02:36 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <483156D0.8010801@gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> Message-ID: <08May19.100239pdt."58696"@synergy1.parc.xerox.com> > Hmm, perhaps the ctypes documentation could use a more prominent warning > that it may not be available on some Unix platforms (HP-UX, AIX, IRIX), > and that it may require the use of GCC rather than the vendor compiler > on others (Solaris). > > At the moment, I suspect some projects may be switching to using it > without realising the implications for cross-platform portability. I think it's a tad more problematic. As other modules, both in and out of the standard library, move to use ctypes, it implies that *Python* isn't supported on those combinations. Personally, that's fine with me (as long as there's a workaround for Solaris!), but I think that Ulrich is right in saying this should be more prominent. Bill From jcea at jcea.es Mon May 19 19:07:40 2008 From: jcea at jcea.es (Jesus Cea) Date: Mon, 19 May 2008 19:07:40 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080516132451.GA7257@amk-desktop.matrixgroup.net> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <20080516132451.GA7257@amk-desktop.matrixgroup.net> Message-ID: <4831B3DC.1030902@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A.M. Kuchling wrote: | Python development doesn't seem to have any volunteers who use AIX or | HP-UX and can fix bugs on these platforms. Searching bugs.python.org, | I find about 20 open AIX issues, 4 or 5 HP-UX issues, 6 IRIX bugs, and | about 15 Solaris bugs; but I don't know if any of the developers here | use these platforms. (There is a Solaris buildbot, at least.) I develop under Solaris 10 x86. My environment is 64 bits, but my python deployment/development in 32 bits. I could try 64 bits if asked for. I can look at any Solaris related issue. Just ask. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDGz2Jlgi5GaxT1NAQIx6wP+NudycEV/nkaCEGLUwnj5BP19urVURedj x6Udb690/Hon/fv6tkplIw5pLramdv3hH8KKsxWr+kzrp4iHUZ7JPyItC0DBXImQ 3wRrV7Yhu1nVmFV2qm5Kdbu8+x7fCiWnzESLzQmIjEKQd0dlO2vStXUNmndoJ21h pDwIW0bUriA= =jbHH -----END PGP SIGNATURE----- From hitchmanr at gmail.com Mon May 19 20:05:11 2008 From: hitchmanr at gmail.com (Ryan Hitchman) Date: Mon, 19 May 2008 12:05:11 -0600 Subject: [Python-Dev] Conditional For Statements In-Reply-To: <48318BD3.9050304@gmail.com> References: <48318BD3.9050304@gmail.com> Message-ID: On Sun, May 21, 2006 at 17:38:49 CEST, Guido van Rossum wrote > ... > Also, it would be a parsing conflict for the new conditional > expressions (x if T else y). > ... That's all I needed to know. Sorry, everyone, I'll try not to waste your time in the future. From theller at ctypes.org Mon May 19 20:50:39 2008 From: theller at ctypes.org (Thomas Heller) Date: Mon, 19 May 2008 20:50:39 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <08May19.100239pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> Message-ID: Bill Janssen schrieb: >> Hmm, perhaps the ctypes documentation could use a more prominent warning >> that it may not be available on some Unix platforms (HP-UX, AIX, IRIX), >> and that it may require the use of GCC rather than the vendor compiler >> on others (Solaris). >> >> At the moment, I suspect some projects may be switching to using it >> without realising the implications for cross-platform portability. > > I think it's a tad more problematic. As other modules, both in and > out of the standard library, move to use ctypes, it implies that > *Python* isn't supported on those combinations. Personally, that's > fine with me (as long as there's a workaround for Solaris!), but I > think that Ulrich is right in saying this should be more prominent. I won't object if anyone adds this notice to the Python docs, so please go ahead. A table of platforms (on the wiki?) where ctypes builds/works or does not may also be helpful. Myself I would rather spend my energy to make ctypes more portable, within my skills and the platforms I have access to. Thomas From brett at python.org Mon May 19 21:05:02 2008 From: brett at python.org (Brett Cannon) Date: Mon, 19 May 2008 12:05:02 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> <4831704D.1060201@gmail.com> <02ba01c8b9b5$27245210$756cf630$@com.au> <009401c8b9c6$8e719250$ac00a8c0@RaymondLaptop1> Message-ID: On Mon, May 19, 2008 at 9:22 AM, Guido van Rossum wrote: > On Mon, May 19, 2008 at 8:39 AM, Raymond Hettinger wrote: >>> Nick writes: >>>> >>>> M.-A. Lemburg wrote: >>>> > I don't think that an administrative problem such as forward- >>>> > porting patches to 3.x warrants breakage in the 2.x branch. >>>> > >>>> > After all, the renaming was approached for Python 3.0 and not >>>> > 2.6 *because* it introduces major breakage. >>>> > >>>> > AFAIR, the discussion on the stdlib-sig also didn't include the >>>> > plan to backport such changes to 2.6. Otherwise, we would have >>>> > hashed them out there. >>>> >>>> I think MAL is 100% correct here (and I expect Raymond will chime in to >>>> support him at some point as well). >>> >>> And until then, a +1 for MAL's position from me as well. 2.x should be >>> quite conservative about such changes... >> >> I concur. > > And a "me too" post about being conservative by default as well. > I will update the PEP some time today. I think if we take MAL's idea of doing the __dict__.update() trick and suppress the Py3K warnings then it should be able to keep the warnings (it will require a very specific filter). Otherwise the Py3K warnings will just have to go. -Brett From brett at python.org Mon May 19 21:26:17 2008 From: brett at python.org (Brett Cannon) Date: Mon, 19 May 2008 12:26:17 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <48316DB7.2040506@egenix.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> Message-ID: On Mon, May 19, 2008 at 5:08 AM, M.-A. Lemburg wrote: > On 2008-05-18 22:24, Brett Cannon wrote: >> >> On Sun, May 18, 2008 at 6:14 AM, Nick Coghlan wrote: >>> >>> M.-A. Lemburg wrote: >>>> >>>> Perhaps I have a misunderstanding of the reasoning behind >>>> doing the renaming in the 2.x branch, but it appears that >>>> the only reason is to get used to the new names. That's a >>>> rather low priority argument in comparison to the breakage >>>> the renaming will cause in the 2.x branch. >>> >>> I think this is the key point here. The possibility of breaking pickling >>> compatibility never came up during the PEP 3108 discussions, so wasn't >>> taken >>> into account in deciding whether or not backporting the name changes was >>> a >>> good idea. >>> >>> I think it's pretty clear that the code needs to be moved back into the >>> modules with the old names for 2.6. The only question is whether or not >>> we >>> put any effort into making the new stdlib organisation usable in 2.x, or >>> just rely on 2to3 to fix it (note that the "increasing the common subset" >>> argument doesn't really apply, since you can catch the import errors in >>> order to try both names). >> >> Problem with this is it makes forward-porting revisions to 3.0 a PITA. >> By keeping the module names consistent between the versions merging a >> revision is just a matter of ``svnmerge merge`` with the usual >> 3.0-specific changes. Reverting the modules back to the old name will >> make forward-porting much more difficult as I don't think svn keeps >> rename information around (and thus map the old name to the new name >> in terms of diffs). > > svnmerge is written in Python, so wouldn't it be possible to add > support for maintaining such renaming to that tool ? > Don't know, possibly. But I am not about to try to figure out. > I don't think that an administrative problem such as forward- > porting patches to 3.x warrants breakage in the 2.x branch. > That's why I suggested changing pickle to deal with the rename, but obviously I am in the minority in that idea. > After all, the renaming was approached for Python 3.0 and not > 2.6 *because* it introduces major breakage. > > AFAIR, the discussion on the stdlib-sig also didn't include the > plan to backport such changes to 2.6. Otherwise, we would have > hashed them out there. > Never came up. >> Alexandre's idea of teaching pickle the mapping of old names to new >> might be the best solution. We could have a flag to pickle that >> deactivates the renaming. Otherwise we could bump the pickle version >> number so that the new number doesn't do the mapping while the old >> versions to the implicit module mapping. >> >> And as Greg and Glpyh have pointed out, this is a problem that might >> need to be addressed in the future with some changes to our >> serialization method (I have no clue how since I don't deal with >> pickle very much). > > It is possible to make pickle aware of the module renames, but > that doesn't solve problems with other forms of serialization > or use of the .__module__ attribute in general. > > Why can't we just provide a "from __future__ import renamed_modules" > which then provides all the new name to old name mappings in > some form (e.g. module proxies or whatever) and leave the > existing modules in 2.x untouched ? I have started a discussion on the stdlib SIG on how to handle this, so I will defer this discussion to there. But one thing that needs to be decided is if we are ever going to allow ourselves to rename modules without a major version bump, and if so how to deal with this problem. I would hope we don't have to wait another eight years before there is another chance to shift things around if it becomes apparent that some new package should be introduced since 2to3 gives us a very nice way to handle the mechanical aspect of porting code. -Brett From brett at python.org Mon May 19 21:36:25 2008 From: brett at python.org (Brett Cannon) Date: Mon, 19 May 2008 12:36:25 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4831425F.2050102@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> Message-ID: On Mon, May 19, 2008 at 2:03 AM, Ulrich Berning wrote: > Gregory P. Smith wrote: > >> On Fri, May 16, 2008 at 1:32 AM, Ulrich Berning >> wrote: >> >>> >>> As long as the ctypes extension doesn't build on major Un*x platforms >>> (AIX, >>> HP-UX), I don't like to see ctypes dependend modules included into the >>> stdlib. Please keep the stdlib as portable as possible. >>> >> >> Nice in theory but ctypes already works on at least the top 3 popular >> platforms. Lets not hold Python's stdlib back because nobody who uses >> IBM and HP proprietary stuff has contributed the necessary support. >> Making nice libraries available for other platforms is a good way to >> encourage people to either pitch in and add support or consider their >> platform choices in the future. >> >> -gps >> >> > > It's not my platform choice, it's the choice of our customers. I'm not using > these platforms just for fun (in fact it isn't fun compared to Linux or > Windows). > > If porting libffi to AIX, HP-UX, IRIX, Solaris... (especially using vendor > compilers) would be an easy job, I'm sure it would have been done already. Well, ctypes isn't simple. =) > If more and more essential packages depend on ctypes, we should make a clear > statement, that Python isn't supported any longer on platform/compiler > combinations where libffi/ctypes doesn't build. This would give me arguments > to drop support of our software on those platforms. You are mixing the stdlib in with the language in terms of what is required for Python to work, which I think is unfair. Just because some part of the stdlib isn't portable to some OS does not mean Python is not supported on that platform. If you can run a pure Python module that does not depend on any C extension, then that platform has the support needed to run Python. Everything else is extra (which is why we have modules in the stdlib only available on specific platforms). -Brett From mal at egenix.com Mon May 19 21:41:19 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 19 May 2008 21:41:19 +0200 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> Message-ID: <4831D7DF.1090804@egenix.com> On 2008-05-19 21:26, Brett Cannon wrote: >> It is possible to make pickle aware of the module renames, but >> that doesn't solve problems with other forms of serialization >> or use of the .__module__ attribute in general. >> >> Why can't we just provide a "from __future__ import renamed_modules" >> which then provides all the new name to old name mappings in >> some form (e.g. module proxies or whatever) and leave the >> existing modules in 2.x untouched ? > > I have started a discussion on the stdlib SIG on how to handle this, > so I will defer this discussion to there. Thanks. > But one thing that needs to be decided is if we are ever going to > allow ourselves to rename modules without a major version bump, and if > so how to deal with this problem. I would hope we don't have to wait > another eight years before there is another chance to shift things > around if it becomes apparent that some new package should be > introduced since 2to3 gives us a very nice way to handle the > mechanical aspect of porting code. We could some kind of module aliasing support to Python. Backporting name changes would then be a matter of loading the right aliasing map into the older Python version. This could probably be done by adding a line if hasattr(sys, 'module_aliases'): modname = sys.module_aliases.get(modname, modname) to the __import__ implementation. By turning .__module__ into a property and applying the same aliasing there, we should be able to resolve most technical issues with a renaming. Alas, too late to change 2.4 and 2.5 :-/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 19 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ygingras at ygingras.net Mon May 19 21:48:26 2008 From: ygingras at ygingras.net (Yannick Gingras) Date: Mon, 19 May 2008 15:48:26 -0400 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: (Guido van Rossum's message of "Mon\, 19 May 2008 09\:13\:09 -0700") References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> <87y76ae1xp.fsf@enceladus.ygingras.net> <87abimcuna.fsf@enceladus.ygingras.net> Message-ID: <87y766axdh.fsf@enceladus.ygingras.net> "Guido van Rossum" writes: > Have you considered whether this works on all platforms? (E.g. > Windows, or embedded non-Unix-based.) Yes but I guess I didn't comment it properly. The line printed_errno = errno_str ? errno_str : self->myerrno; ensures that we gracefully fallback to numeric errno when here is no symbolic value available. The code is only invoked if there is an errno value, which takes care of most non-Unix platforms. -- Yannick Gingras From g.brandl at gmx.net Mon May 19 22:04:41 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 19 May 2008 22:04:41 +0200 Subject: [Python-Dev] Symbolic errno values in error messages In-Reply-To: <87abimcuna.fsf@enceladus.ygingras.net> References: <873aoifrm4.fsf@enceladus.ygingras.net> <482D96FB.2040700@gmail.com> <87y76ae1xp.fsf@enceladus.ygingras.net> <87abimcuna.fsf@enceladus.ygingras.net> Message-ID: Yannick Gingras schrieb: > "Alexandre Vassalotti" writes: > >>>> So now I am not sure what OP is proposing. Do you want to replace 21 >>>> with EISDIR in the above? >>> >>> Yes, that's what I had in mind. >>> >> >> Then, check out EnvironmentError_str in Objects/exceptions.c. You >> should be able import the errno module and fetch its errorcode >> dictionary. > > It wasn't as hard as I expected. It's the first time that I play with > the Python C API; I didn't expect the API to be that high level. > > I attached a patch to convert errno to its symbolic value when an > EnvironmentError is printed. Should attach it to a ticket on > bugs.python.org? > > I'm sure there is a style guide like PEP-8 for C code, feel free to > point me to it because my patch is probably not fully style compliant. > > With Emacs, doing > > M-x c-set-style python > > doesn't seems to do the right thing. Are you all using a bunch of > shared settings in you .emacs files? For new-style Python C files, this style definition works well: (c-add-style "python-new" '((indent-tabs-mode . nil) (fill-column . 78) (c-basic-offset . 4) (c-offsets-alist . ((substatement-open . 0) (inextern-lang . 0) (arglist-intro . +) (knr-argdecl-intro . +))) (c-hanging-braces-alist . ((brace-list-open) (brace-list-intro) (brace-list-close) (brace-entry-open) (substatement-open after) (block-close . c-snug-do-while))) (c-block-comment-prefix . "* ")) ) This is a very crude hook that auto-selects the C style depending on whether it finds a line starting with tab in the first 3000 characters in the file: (defun c-select-style () (save-excursion (if (re-search-forward "^\t" 3000 t) (c-set-style "python") (c-set-style "python-new")))) (add-hook 'c-mode-hook 'c-select-style) HTH, Georg From amk at amk.ca Mon May 19 22:38:20 2008 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 19 May 2008 16:38:20 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> Message-ID: <20080519203820.GA12661@amk-desktop.matrixgroup.net> On Mon, May 19, 2008 at 08:50:39PM +0200, Thomas Heller wrote: > Myself I would rather spend my energy to make ctypes more portable, within my > skills and the platforms I have access to. Someone could run Solaris x86 inside a hosted virtual machine and make it available to the Python developers. Is it possible to find similar hosting for HP-UX and AIX? Or might IBM or HP be willing to donate a low-end machine to the PSF for porting use? --amk From musiccomposition at gmail.com Tue May 20 00:26:17 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 19 May 2008 17:26:17 -0500 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <48316DB7.2040506@egenix.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> Message-ID: <1afaf6160805191526p20caa74atdd782d8e3210369c@mail.gmail.com> On Mon, May 19, 2008 at 7:08 AM, M.-A. Lemburg wrote: > On 2008-05-18 22:24, Brett Cannon wrote: >> >> On Sun, May 18, 2008 at 6:14 AM, Nick Coghlan wrote: >>> >>> M.-A. Lemburg wrote: >>>> >>>> Perhaps I have a misunderstanding of the reasoning behind >>>> doing the renaming in the 2.x branch, but it appears that >>>> the only reason is to get used to the new names. That's a >>>> rather low priority argument in comparison to the breakage >>>> the renaming will cause in the 2.x branch. >>> >>> I think this is the key point here. The possibility of breaking pickling >>> compatibility never came up during the PEP 3108 discussions, so wasn't >>> taken >>> into account in deciding whether or not backporting the name changes was >>> a >>> good idea. >>> >>> I think it's pretty clear that the code needs to be moved back into the >>> modules with the old names for 2.6. The only question is whether or not >>> we >>> put any effort into making the new stdlib organisation usable in 2.x, or >>> just rely on 2to3 to fix it (note that the "increasing the common subset" >>> argument doesn't really apply, since you can catch the import errors in >>> order to try both names). >> >> Problem with this is it makes forward-porting revisions to 3.0 a PITA. >> By keeping the module names consistent between the versions merging a >> revision is just a matter of ``svnmerge merge`` with the usual >> 3.0-specific changes. Reverting the modules back to the old name will >> make forward-porting much more difficult as I don't think svn keeps >> rename information around (and thus map the old name to the new name >> in terms of diffs). > > svnmerge is written in Python, so wouldn't it be possible to add > support for maintaining such renaming to that tool ? svnmerge.py is mostly a wrapper over svn merge, and svn merge can't handle it, so I don't think is easily possible. > > I don't think that an administrative problem such as forward- > porting patches to 3.x warrants breakage in the 2.x branch. I am a bit worried for the sanity of the Merger, though. Merges into non-existent files are skipped automatically, so it doesn't make life any easier. Bazaar can handle renames correctly. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From jcea at jcea.es Tue May 20 00:46:55 2008 From: jcea at jcea.es (Jesus Cea) Date: Tue, 20 May 2008 00:46:55 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> Message-ID: <4832035F.3010708@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: | I downloaded the source that includes AES encryption, for no reason | other than it was first on the list. I'm now wondering if we should | only be importing the 'NC' source that doesn't contain any | encryption? Jesus, does pybsddb use any of the Berkeley DB | encryption facilities? Would anything break if we built the | bsddb module without encryption? Yes, pybsddb3 4.6.4 supports cryptography if the underlying Berkeley DB library is crypto enabled. In principle, you can compile BDB without crypto, and pybsddb3 should work, but you would lose ability to open any DB formerly created using page encryption or page checksum. Export laws aside, we better compile with crypto :). Details: http://www.jcea.es/programacion/pybsddb_doc/dbenv.html#set_encrypt - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDIDWplgi5GaxT1NAQKaagP+Myn6t364B9cVMUtlKjTIX1LBZkkEG7SX pv+hjUZ68r0Suw8SkhXnBbL2Ek3/yFhvNHH8qAZvEtRGsFGmKiNXccn9Ce3oy0me rAYhuuICleAUJ8RO8FZz+sGK3SR4kITrqnGfMnG46u0rXDQy82NSRW2+uWz/D96d U+X/wfV2HgQ= =io+q -----END PGP SIGNATURE----- From graham.horler at gmail.com Tue May 20 00:46:32 2008 From: graham.horler at gmail.com (Graham Horler) Date: Mon, 19 May 2008 23:46:32 +0100 Subject: [Python-Dev] Unaccompanied Patch Message-ID: <6saeel$9fm6e8@mercury.karoo.kcom.com> Hi all, I created a patch on 2007-08-15: http://bugs.python.org/issue1775025 I wonder, will it just stay in patches unnoticed until it has its very own associated bug report to keep it company? Thanks, Graham From brett at python.org Tue May 20 00:57:17 2008 From: brett at python.org (Brett Cannon) Date: Mon, 19 May 2008 15:57:17 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <1afaf6160805191526p20caa74atdd782d8e3210369c@mail.gmail.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> <1afaf6160805191526p20caa74atdd782d8e3210369c@mail.gmail.com> Message-ID: On Mon, May 19, 2008 at 3:26 PM, Benjamin Peterson wrote: > On Mon, May 19, 2008 at 7:08 AM, M.-A. Lemburg wrote: >> On 2008-05-18 22:24, Brett Cannon wrote: >>> >>> On Sun, May 18, 2008 at 6:14 AM, Nick Coghlan wrote: >>>> >>>> M.-A. Lemburg wrote: >>>>> >>>>> Perhaps I have a misunderstanding of the reasoning behind >>>>> doing the renaming in the 2.x branch, but it appears that >>>>> the only reason is to get used to the new names. That's a >>>>> rather low priority argument in comparison to the breakage >>>>> the renaming will cause in the 2.x branch. >>>> >>>> I think this is the key point here. The possibility of breaking pickling >>>> compatibility never came up during the PEP 3108 discussions, so wasn't >>>> taken >>>> into account in deciding whether or not backporting the name changes was >>>> a >>>> good idea. >>>> >>>> I think it's pretty clear that the code needs to be moved back into the >>>> modules with the old names for 2.6. The only question is whether or not >>>> we >>>> put any effort into making the new stdlib organisation usable in 2.x, or >>>> just rely on 2to3 to fix it (note that the "increasing the common subset" >>>> argument doesn't really apply, since you can catch the import errors in >>>> order to try both names). >>> >>> Problem with this is it makes forward-porting revisions to 3.0 a PITA. >>> By keeping the module names consistent between the versions merging a >>> revision is just a matter of ``svnmerge merge`` with the usual >>> 3.0-specific changes. Reverting the modules back to the old name will >>> make forward-porting much more difficult as I don't think svn keeps >>> rename information around (and thus map the old name to the new name >>> in terms of diffs). >> >> svnmerge is written in Python, so wouldn't it be possible to add >> support for maintaining such renaming to that tool ? > > svnmerge.py is mostly a wrapper over svn merge, and svn merge can't > handle it, so I don't think is easily possible. I think MAL was suggesting add some property that kept track of skipped merges or something. >> >> I don't think that an administrative problem such as forward- >> porting patches to 3.x warrants breakage in the 2.x branch. > > I am a bit worried for the sanity of the Merger, though. Merges into > non-existent files are skipped automatically, so it doesn't make life > any easier. > It will either have to be done in 2.6 and the immediately forward-ported or done in 3.0 and backported. I will follow the latter IF I feel like bothering with the backport. > > Bazaar can handle renames > correctly. Yeah, yeah. One thing at a time. -Brett From brett at python.org Tue May 20 00:59:37 2008 From: brett at python.org (Brett Cannon) Date: Mon, 19 May 2008 15:59:37 -0700 Subject: [Python-Dev] Unaccompanied Patch In-Reply-To: <6saeel$9fm6e8@mercury.karoo.kcom.com> References: <6saeel$9fm6e8@mercury.karoo.kcom.com> Message-ID: On Mon, May 19, 2008 at 3:46 PM, Graham Horler wrote: > Hi all, > > I created a patch on 2007-08-15: > http://bugs.python.org/issue1775025 > > I wonder, will it just stay in patches unnoticed until it has its very > own associated bug report to keep it company? Being a patch does not mean it is ignore (actually, it raises the chance it will be noticed). It's just that developer time is limited and so someone has to just take enough interest in the patch to review it, apply it, etc. We no longer distinguish issues between patches and bugs; when a bug gets an attached patch we just set the proper keyword. So creating a separate issue will just lead to it being closed as being superceded by the patch. -Brett From musiccomposition at gmail.com Tue May 20 01:00:20 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 19 May 2008 18:00:20 -0500 Subject: [Python-Dev] Unaccompanied Patch In-Reply-To: <6saeel$9fm6e8@mercury.karoo.kcom.com> References: <6saeel$9fm6e8@mercury.karoo.kcom.com> Message-ID: <1afaf6160805191600j2df5f537n324d180ecb296171@mail.gmail.com> On Mon, May 19, 2008 at 5:46 PM, Graham Horler wrote: > Hi all, > > I created a patch on 2007-08-15: > http://bugs.python.org/issue1775025 > > I wonder, will it just stay in patches unnoticed until it has its very > own associated bug report to keep it company? At the moment, I only see a doc patch. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Tue May 20 01:02:42 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 19 May 2008 18:02:42 -0500 Subject: [Python-Dev] Unaccompanied Patch In-Reply-To: <1afaf6160805191600j2df5f537n324d180ecb296171@mail.gmail.com> References: <6saeel$9fm6e8@mercury.karoo.kcom.com> <1afaf6160805191600j2df5f537n324d180ecb296171@mail.gmail.com> Message-ID: <1afaf6160805191602k980af92r6f7f72e34586d0e4@mail.gmail.com> On Mon, May 19, 2008 at 6:00 PM, Benjamin Peterson wrote: > On Mon, May 19, 2008 at 5:46 PM, Graham Horler wrote: >> Hi all, >> >> I created a patch on 2007-08-15: >> http://bugs.python.org/issue1775025 >> >> I wonder, will it just stay in patches unnoticed until it has its very >> own associated bug report to keep it company? > > At the moment, I only see a doc patch. Sorry, that didn't come out they way I wanted it to. I meant that the doc patch will have to be converted to reST. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From dlenski at gmail.com Tue May 20 01:32:38 2008 From: dlenski at gmail.com (Dan Lenski) Date: Mon, 19 May 2008 23:32:38 +0000 (UTC) Subject: [Python-Dev] buffer interface for C extensions References: Message-ID: On Sun, 18 May 2008 22:59:05 +0000, Dan Lenski wrote: > So here's what I don't understand. Why doesn't the "t#" argument > specifier support read-write buffers as well as read-only buffers? > Aren't read-write buffers a *superset* of read-only buffers?? Is there > something I'm doing wrong or a quick fix to get this to work > appropriately? So... I've answered my own question on this. I had a look in getargs.c, and apparently the "t#" argument specifier only accepts *character-based* readable buffers. Is there any chance anyone of adding an argument specifier for readable buffers in general? Say, "r#"? Can anyone explain to me the rationale for having some data type support the readable buffer interface, but /NOT/ the character-based buffer interface? I can't find any information that explains the fundamental distinction between the two... and I don't see why array('B') shouldn't support both. Dan From janssen at parc.com Tue May 20 02:15:05 2008 From: janssen at parc.com (Bill Janssen) Date: Mon, 19 May 2008 17:15:05 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> Message-ID: <08May19.171510pdt."58696"@synergy1.parc.xerox.com> > If you can run a pure Python module > that does not depend on any C extension, then that platform has the > support needed to run Python. This is certainly a point of view. One that many end-users wouldn't understand :-). Bill From brett at python.org Tue May 20 02:23:29 2008 From: brett at python.org (Brett Cannon) Date: Mon, 19 May 2008 17:23:29 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <8926242303997388828@unknownmsgid> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> Message-ID: On Mon, May 19, 2008 at 5:15 PM, Bill Janssen wrote: >> If you can run a pure Python module >> that does not depend on any C extension, then that platform has the >> support needed to run Python. > > This is certainly a point of view. One that many end-users wouldn't > understand :-). Perhaps, but it's clear-cut. Is OS X not properly supported because it can't run the _winreg module? I just don't like labeling a platform as unsupported just because ctypes doesn't compile on it. -Brett From janssen at parc.com Tue May 20 03:13:11 2008 From: janssen at parc.com (Bill Janssen) Date: Mon, 19 May 2008 18:13:11 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> Message-ID: <08May19.181320pdt."58696"@synergy1.parc.xerox.com> > On Mon, May 19, 2008 at 5:15 PM, Bill Janssen wrote: > >> If you can run a pure Python module > >> that does not depend on any C extension, then that platform has the > >> support needed to run Python. > > > > This is certainly a point of view. One that many end-users wouldn't > > understand :-). > > Perhaps, but it's clear-cut. Is OS X not properly supported because it > can't run the _winreg module? I just don't like labeling a platform as > unsupported just because ctypes doesn't compile on it. I assume _winreg runs everywhere Python is said to run, and there's a Windows registry to examine, so I think it's a different kettle of fish. ctypes doesn't run everywhere Python is said to run, and there are dynamic libraries to call into. I think it would be great if we could get some AIX, HP-UX, and Solaris boxes for Thomas to work on. What would motivate IBM, H-P, and Sun to donate such gear? Perhaps each of the companies have an office somewhere that could help with this resource allocation? For instance, talking to the "AIX Collaboration Center" of IBM (aixcc at us.ibm.com)? And these are all SYSV derivatives, aren't they? So perhaps it's some common fix for all three? Bill From steve at holdenweb.com Tue May 20 04:47:16 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 19 May 2008 22:47:16 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <08May19.181320pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt."58696"@synergy1.parc.xerox.com> Message-ID: <48323BB4.7020706@holdenweb.com> Bill Janssen wrote: >> On Mon, May 19, 2008 at 5:15 PM, Bill Janssen wrote: >>>> If you can run a pure Python module >>>> that does not depend on any C extension, then that platform has the >>>> support needed to run Python. >>> This is certainly a point of view. One that many end-users wouldn't >>> understand :-). >> Perhaps, but it's clear-cut. Is OS X not properly supported because it >> can't run the _winreg module? I just don't like labeling a platform as >> unsupported just because ctypes doesn't compile on it. > > I assume _winreg runs everywhere Python is said to run, and there's a > Windows registry to examine, so I think it's a different kettle of > fish. ctypes doesn't run everywhere Python is said to run, and there > are dynamic libraries to call into. > > I think it would be great if we could get some AIX, HP-UX, and Solaris > boxes for Thomas to work on. What would motivate IBM, H-P, and Sun to > donate such gear? Perhaps each of the companies have an office > somewhere that could help with this resource allocation? For > instance, talking to the "AIX Collaboration Center" of IBM > (aixcc at us.ibm.com)? > > And these are all SYSV derivatives, aren't they? So perhaps it's some > common fix for all three? > OK, I know people in Sun and possibly H-P I could ask, and I may have an arm or two to twist to get to IBM. But worst-case, what budget would the infrastructure committee need for the hardware and (more importantly) if the hardware materialized, would there be manpower to maintain the platforms as "Python supported"? The more libraries that use ctypes to call into native functionality, the more important it becomes to have ctypes run, even if only to implement platform-specific functionality on the given platforms. I would like "being able to call a wide range of native libraries" to be a Python claim on all platforms, even when the native libraries are platform-proprietary. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue May 20 04:47:16 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 19 May 2008 22:47:16 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <08May19.181320pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt."58696"@synergy1.parc.xerox.com> Message-ID: <48323BB4.7020706@holdenweb.com> Bill Janssen wrote: >> On Mon, May 19, 2008 at 5:15 PM, Bill Janssen wrote: >>>> If you can run a pure Python module >>>> that does not depend on any C extension, then that platform has the >>>> support needed to run Python. >>> This is certainly a point of view. One that many end-users wouldn't >>> understand :-). >> Perhaps, but it's clear-cut. Is OS X not properly supported because it >> can't run the _winreg module? I just don't like labeling a platform as >> unsupported just because ctypes doesn't compile on it. > > I assume _winreg runs everywhere Python is said to run, and there's a > Windows registry to examine, so I think it's a different kettle of > fish. ctypes doesn't run everywhere Python is said to run, and there > are dynamic libraries to call into. > > I think it would be great if we could get some AIX, HP-UX, and Solaris > boxes for Thomas to work on. What would motivate IBM, H-P, and Sun to > donate such gear? Perhaps each of the companies have an office > somewhere that could help with this resource allocation? For > instance, talking to the "AIX Collaboration Center" of IBM > (aixcc at us.ibm.com)? > > And these are all SYSV derivatives, aren't they? So perhaps it's some > common fix for all three? > OK, I know people in Sun and possibly H-P I could ask, and I may have an arm or two to twist to get to IBM. But worst-case, what budget would the infrastructure committee need for the hardware and (more importantly) if the hardware materialized, would there be manpower to maintain the platforms as "Python supported"? The more libraries that use ctypes to call into native functionality, the more important it becomes to have ctypes run, even if only to implement platform-specific functionality on the given platforms. I would like "being able to call a wide range of native libraries" to be a Python claim on all platforms, even when the native libraries are platform-proprietary. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue May 20 05:34:43 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 19 May 2008 23:34:43 -0400 Subject: [Python-Dev] Unaccompanied Patch In-Reply-To: <1afaf6160805191602k980af92r6f7f72e34586d0e4@mail.gmail.com> References: <6saeel$9fm6e8@mercury.karoo.kcom.com> <1afaf6160805191600j2df5f537n324d180ecb296171@mail.gmail.com> <1afaf6160805191602k980af92r6f7f72e34586d0e4@mail.gmail.com> Message-ID: <483246D3.9030203@holdenweb.com> Benjamin Peterson wrote: > On Mon, May 19, 2008 at 6:00 PM, Benjamin Peterson > wrote: >> On Mon, May 19, 2008 at 5:46 PM, Graham Horler wrote: >>> Hi all, >>> >>> I created a patch on 2007-08-15: >>> http://bugs.python.org/issue1775025 >>> >>> I wonder, will it just stay in patches unnoticed until it has its very >>> own associated bug report to keep it company? >> At the moment, I only see a doc patch. > > Sorry, that didn't come out they way I wanted it to. I meant that the > doc patch will have to be converted to reST. > > Well, looky here: zipfile_62920.diff grahamh, 2008-05-19 23:49 Updated + doc -> reST That was quick. I'm not in a position to svn update were I am. Looks like someone's paying attention, though. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue May 20 05:34:43 2008 From: steve at holdenweb.com (Steve Holden) Date: Mon, 19 May 2008 23:34:43 -0400 Subject: [Python-Dev] Unaccompanied Patch In-Reply-To: <1afaf6160805191602k980af92r6f7f72e34586d0e4@mail.gmail.com> References: <6saeel$9fm6e8@mercury.karoo.kcom.com> <1afaf6160805191600j2df5f537n324d180ecb296171@mail.gmail.com> <1afaf6160805191602k980af92r6f7f72e34586d0e4@mail.gmail.com> Message-ID: <483246D3.9030203@holdenweb.com> Benjamin Peterson wrote: > On Mon, May 19, 2008 at 6:00 PM, Benjamin Peterson > wrote: >> On Mon, May 19, 2008 at 5:46 PM, Graham Horler wrote: >>> Hi all, >>> >>> I created a patch on 2007-08-15: >>> http://bugs.python.org/issue1775025 >>> >>> I wonder, will it just stay in patches unnoticed until it has its very >>> own associated bug report to keep it company? >> At the moment, I only see a doc patch. > > Sorry, that didn't come out they way I wanted it to. I meant that the > doc patch will have to be converted to reST. > > Well, looky here: zipfile_62920.diff grahamh, 2008-05-19 23:49 Updated + doc -> reST That was quick. I'm not in a position to svn update were I am. Looks like someone's paying attention, though. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From brett at python.org Tue May 20 06:44:23 2008 From: brett at python.org (Brett Cannon) Date: Mon, 19 May 2008 21:44:23 -0700 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> Message-ID: On Mon, May 19, 2008 at 12:26 PM, Brett Cannon wrote: > On Mon, May 19, 2008 at 5:08 AM, M.-A. Lemburg wrote: >> On 2008-05-18 22:24, Brett Cannon wrote: >>> >>> On Sun, May 18, 2008 at 6:14 AM, Nick Coghlan wrote: >>>> >>>> M.-A. Lemburg wrote: >>>>> >>>>> Perhaps I have a misunderstanding of the reasoning behind >>>>> doing the renaming in the 2.x branch, but it appears that >>>>> the only reason is to get used to the new names. That's a >>>>> rather low priority argument in comparison to the breakage >>>>> the renaming will cause in the 2.x branch. >>>> >>>> I think this is the key point here. The possibility of breaking pickling >>>> compatibility never came up during the PEP 3108 discussions, so wasn't >>>> taken >>>> into account in deciding whether or not backporting the name changes was >>>> a >>>> good idea. >>>> >>>> I think it's pretty clear that the code needs to be moved back into the >>>> modules with the old names for 2.6. The only question is whether or not >>>> we >>>> put any effort into making the new stdlib organisation usable in 2.x, or >>>> just rely on 2to3 to fix it (note that the "increasing the common subset" >>>> argument doesn't really apply, since you can catch the import errors in >>>> order to try both names). >>> >>> Problem with this is it makes forward-porting revisions to 3.0 a PITA. >>> By keeping the module names consistent between the versions merging a >>> revision is just a matter of ``svnmerge merge`` with the usual >>> 3.0-specific changes. Reverting the modules back to the old name will >>> make forward-porting much more difficult as I don't think svn keeps >>> rename information around (and thus map the old name to the new name >>> in terms of diffs). >> >> svnmerge is written in Python, so wouldn't it be possible to add >> support for maintaining such renaming to that tool ? >> > > Don't know, possibly. But I am not about to try to figure out. > >> I don't think that an administrative problem such as forward- >> porting patches to 3.x warrants breakage in the 2.x branch. >> > > That's why I suggested changing pickle to deal with the rename, but > obviously I am in the minority in that idea. > >> After all, the renaming was approached for Python 3.0 and not >> 2.6 *because* it introduces major breakage. >> >> AFAIR, the discussion on the stdlib-sig also didn't include the >> plan to backport such changes to 2.6. Otherwise, we would have >> hashed them out there. >> > > Never came up. > >>> Alexandre's idea of teaching pickle the mapping of old names to new >>> might be the best solution. We could have a flag to pickle that >>> deactivates the renaming. Otherwise we could bump the pickle version >>> number so that the new number doesn't do the mapping while the old >>> versions to the implicit module mapping. >>> >>> And as Greg and Glpyh have pointed out, this is a problem that might >>> need to be addressed in the future with some changes to our >>> serialization method (I have no clue how since I don't deal with >>> pickle very much). >> >> It is possible to make pickle aware of the module renames, but >> that doesn't solve problems with other forms of serialization >> or use of the .__module__ attribute in general. >> >> Why can't we just provide a "from __future__ import renamed_modules" >> which then provides all the new name to old name mappings in >> some form (e.g. module proxies or whatever) and leave the >> existing modules in 2.x untouched ? > > I have started a discussion on the stdlib SIG on how to handle this, > so I will defer this discussion to there. > The decision was made (by me) to just revert all of the renames in 2.6. A note will be in the docs stating the rename, but otherwise 2to3 will be relied upon for all transitions from old names to new names. I have updated the PEP to note about which modules need to be reverted and the new steps to rename a module, and added/re-opened the appropriate issues (all attached to issue 2775). -Brett From g.brandl at gmx.net Tue May 20 08:29:45 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 20 May 2008 08:29:45 +0200 Subject: [Python-Dev] Module renaming and pickle mechanisms In-Reply-To: <1afaf6160805191526p20caa74atdd782d8e3210369c@mail.gmail.com> References: <482E9FF3.4060200@egenix.com> <48301F0D.6080900@egenix.com> <48302B98.7040304@gmail.com> <48316DB7.2040506@egenix.com> <1afaf6160805191526p20caa74atdd782d8e3210369c@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: >> svnmerge is written in Python, so wouldn't it be possible to add >> support for maintaining such renaming to that tool ? > > svnmerge.py is mostly a wrapper over svn merge, and svn merge can't > handle it, so I don't think is easily possible. >> >> I don't think that an administrative problem such as forward- >> porting patches to 3.x warrants breakage in the 2.x branch. > > I am a bit worried for the sanity of the Merger, though. Merges into > non-existent files are skipped automatically, so it doesn't make life > any easier. > > > Bazaar can handle renames > correctly. So can dozens of other VCSs. Just to keep perspective. Georg From martin at v.loewis.de Tue May 20 10:22:37 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 May 2008 10:22:37 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING Message-ID: <48328A4D.9020704@v.loewis.de> I'd like to propose a new environment variable PYTHONSTDOUTENCODING. This is meant to solve various problems that people had with Python not detecting their terminal encoding correctly; it would override any detection that Python would use for determining the encoding of stdout (and stdin - but that's less relevant in 2.x). In particular, setting this environment variable would also disable the detection of whether stdout is a terminal. This is desirable for cases as the pydev eclipse plugin, where Python currently fails to detect that the output is a terminal (and technically, what Eclipse provides is not a terminal, but just a pipe, as you can't do pseudoterms in Java). This would have the additional effect that the encoding also gets in effect when redirecting stdout to a file. Whether or not this is a good thing might be debatable; giving the user the control over it (to set or clear that variable) is a good thing, IMO. Naming contest: it probably would be the longest of the PYTHON* variables. I would not want to call it PYTHONENCODING, or PYTHONSTDENCODING, though, because people might infer that it affects sys.getdefaultencoding(), which it shouldn't. Regards, Martin From mal at egenix.com Tue May 20 10:41:43 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 May 2008 10:41:43 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <48328A4D.9020704@v.loewis.de> References: <48328A4D.9020704@v.loewis.de> Message-ID: <48328EC7.7000106@egenix.com> On 2008-05-20 10:22, Martin v. L?wis wrote: > I'd like to propose a new environment variable PYTHONSTDOUTENCODING. > This is meant to solve various problems that people had with Python > not detecting their terminal encoding correctly; it would override > any detection that Python would use for determining the encoding of > stdout (and stdin - but that's less relevant in 2.x). How is this relevant for 2.x ? In 2.x, stdin and stdout are just files without any io wrappers around them. Writing Unicode to stdout will still use the default encoding ASCII to convert it to an 8-bit string. All other 8-bit strings will be passed to stdout as-is. For 3.x, I'd like to see a PYTHONSTDINENCODING, because the current way of relying on the terminal encoding does work well... it then falls back to ASCII, which prevents entering e.g. German Umlauts. > In particular, setting this environment variable would also disable > the detection of whether stdout is a terminal. This is desirable > for cases as the pydev eclipse plugin, where Python currently > fails to detect that the output is a terminal (and technically, > what Eclipse provides is not a terminal, but just a pipe, as you > can't do pseudoterms in Java). > > This would have the additional effect that the encoding also gets > in effect when redirecting stdout to a file. Whether or not this > is a good thing might be debatable; giving the user the control over > it (to set or clear that variable) is a good thing, IMO. > > Naming contest: it probably would be the longest of the PYTHON* > variables. I would not want to call it PYTHONENCODING, or > PYTHONSTDENCODING, though, because people might infer that it > affects sys.getdefaultencoding(), which it shouldn't. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From thomas at python.org Tue May 20 12:16:32 2008 From: thomas at python.org (Thomas Wouters) Date: Tue, 20 May 2008 12:16:32 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <48328EC7.7000106@egenix.com> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> Message-ID: <9e804ac0805200316mba393f5pb8bf81d7e7e87abe@mail.gmail.com> On Tue, May 20, 2008 at 10:41 AM, M.-A. Lemburg wrote: > On 2008-05-20 10:22, Martin v. L?wis wrote: > >> I'd like to propose a new environment variable PYTHONSTDOUTENCODING. >> This is meant to solve various problems that people had with Python >> not detecting their terminal encoding correctly; it would override >> any detection that Python would use for determining the encoding of >> stdout (and stdin - but that's less relevant in 2.x). >> > > How is this relevant for 2.x ? > > In 2.x, stdin and stdout are just files without any io wrappers > around them. > > Writing Unicode to stdout will still use the default encoding > ASCII to convert it to an 8-bit string. All other 8-bit strings > will be passed to stdout as-is. You're forgetting about print; in Python 2.x, when stdout is connected to a terminal, the locale settings (typically the LANG, LC_ALL and LC_CTYPE environment variables) are taken into account when 'print' writes to sys.stdout. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Tue May 20 12:20:06 2008 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 20 May 2008 12:20:06 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <9e804ac0805200316mba393f5pb8bf81d7e7e87abe@mail.gmail.com> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> <9e804ac0805200316mba393f5pb8bf81d7e7e87abe@mail.gmail.com> Message-ID: <319e029f0805200320p6c41b4b9u79ddacf3418b9cc5@mail.gmail.com> On Tue, May 20, 2008 at 12:16 PM, Thomas Wouters wrote: > You're forgetting about print; in Python 2.x, when stdout is connected to a > terminal, the locale settings (typically the LANG, LC_ALL and LC_CTYPE > environment variables) are taken into account when 'print' writes to > sys.stdout. Isn't it then enough to make sure your locale setting are correct? (Never had any problems myself, if works great in Ubuntu). -- Lennart Regebro: Zope and Plone consulting. http://www.colliberty.com/ +33 661 58 14 64 From mal at egenix.com Tue May 20 12:36:39 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 May 2008 12:36:39 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <9e804ac0805200316mba393f5pb8bf81d7e7e87abe@mail.gmail.com> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> <9e804ac0805200316mba393f5pb8bf81d7e7e87abe@mail.gmail.com> Message-ID: <4832A9B7.5050508@egenix.com> On 2008-05-20 12:16, Thomas Wouters wrote: > On Tue, May 20, 2008 at 10:41 AM, M.-A. Lemburg wrote: > >> On 2008-05-20 10:22, Martin v. L?wis wrote: >> >>> I'd like to propose a new environment variable PYTHONSTDOUTENCODING. >>> This is meant to solve various problems that people had with Python >>> not detecting their terminal encoding correctly; it would override >>> any detection that Python would use for determining the encoding of >>> stdout (and stdin - but that's less relevant in 2.x). >>> >> How is this relevant for 2.x ? >> >> In 2.x, stdin and stdout are just files without any io wrappers >> around them. >> >> Writing Unicode to stdout will still use the default encoding >> ASCII to convert it to an 8-bit string. All other 8-bit strings >> will be passed to stdout as-is. > > > You're forgetting about print; in Python 2.x, when stdout is connected to a > terminal, the locale settings (typically the LANG, LC_ALL and LC_CTYPE > environment variables) are taken into account when 'print' writes to > sys.stdout. Thanks for reminding me. I had forgotten about that special case. So "sys.stdout.write(unicode)" will always use the default encoding, whereas "print unicode" uses the sys.stdout.encoding, correct ? Hmm, wouldn't it be better to always use .encoding and also make it adjustable from Python (it is adjustable from C) ?! PYTHONSTDOUTENCODING could then provide the default to sys.stdout.encoding. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From phd at phd.pp.ru Tue May 20 12:48:23 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 20 May 2008 14:48:23 +0400 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <48328A4D.9020704@v.loewis.de> References: <48328A4D.9020704@v.loewis.de> Message-ID: <20080520104823.GA1689@phd.pp.ru> On Tue, May 20, 2008 at 10:22:37AM +0200, "Martin v. L?wis" wrote: > I'd like to propose a new environment variable PYTHONSTDOUTENCODING. > This is meant to solve various problems that people had with Python > not detecting their terminal encoding correctly; it would override > any detection that Python would use for determining the encoding of > stdout (and stdin - but that's less relevant in 2.x). Is it to override locale settings in case the user wants a different encoding? for such cases as redirected stdout, or windows console (which has an "OEM" encoding that differs from the locale encoding)? > Naming contest: it probably would be the longest of the PYTHON* > variables. I would not want to call it PYTHONENCODING, or > PYTHONSTDENCODING, though, because people might infer that it > affects sys.getdefaultencoding(), which it shouldn't. PYTHONIOENCODING? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ziade.tarek at gmail.com Tue May 20 13:33:41 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 20 May 2008 13:33:41 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <20080520104823.GA1689@phd.pp.ru> References: <48328A4D.9020704@v.loewis.de> <20080520104823.GA1689@phd.pp.ru> Message-ID: <94bdd2610805200433ud89dd1hc5c39c06dc2db0b@mail.gmail.com> On Tue, May 20, 2008 at 12:48 PM, Oleg Broytmann wrote: > On Tue, May 20, 2008 at 10:22:37AM +0200, "Martin v. L?wis" wrote: > > I'd like to propose a new environment variable PYTHONSTDOUTENCODING. > > This is meant to solve various problems that people had with Python > > not detecting their terminal encoding correctly; it would override > > any detection that Python would use for determining the encoding of > > stdout (and stdin - but that's less relevant in 2.x). > > Is it to override locale settings in case the user wants a different > encoding? for such cases as redirected stdout, or windows console (which > has an "OEM" encoding that differs from the locale encoding)? > > > Naming contest: it probably would be the longest of the PYTHON* > > variables. I would not want to call it PYTHONENCODING, or > > PYTHONSTDENCODING, though, because people might infer that it > > affects sys.getdefaultencoding(), which it shouldn't. > > PYTHONIOENCODING? > What about PYTHONLANG ? or something that tries to reflect which environment variables are used for this ? (LC_CTYPE -> PYTHONCTYPE ? if the code uses just LC_CTYPE) http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html#tag_08_02 Just for my own knowledge: why it has to be one word ? can't it be PYTHON_LANG ? Tarek > > Oleg. > -- > Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru > Programmers don't die, they just GOSUB without RETURN. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From amk at amk.ca Tue May 20 14:16:16 2008 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 20 May 2008 08:16:16 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <08May19.181320pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt."58696"@synergy1.parc.xerox.com> Message-ID: <20080520121616.GA6040@amk-desktop.matrixgroup.net> On Mon, May 19, 2008 at 06:13:11PM -0700, Bill Janssen wrote: > And these are all SYSV derivatives, aren't they? So perhaps it's some > common fix for all three? This reminds of a Tim-ism: ================== > Just for the record, on AIX, the following C program: Oh no you don't! I followed AIX threads for the first year it came out, but eventually decided there was no future in investing time in baffling discussions that usually ended with "oh, never mind -- turns out it's a bug" <0.9 wink>. Vladimir Marangozov and Tim Peters, 23 Jun 1998 ================== 10 years later, things don't seem to be much different. For ctypes it looks like libffi often fails to compile when not using GCC; http://bugs.python.org/issue1637120 is an AIX bug report. For the curses module, I've noticed that HP-UX seems to require including additional header files or defining _XOPEN_SOURCE to make curses.h work. So I don't think there's a common problem with all these minority platforms, and we really would need developers on all of them. --amk From ncoghlan at gmail.com Tue May 20 14:24:26 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 20 May 2008 22:24:26 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <08May19.181320pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt."58696"@synergy1.parc.xerox.com> Message-ID: <4832C2FA.6010304@gmail.com> Bill Janssen wrote: > And these are all SYSV derivatives, aren't they? So perhaps it's some > common fix for all three? I've never personally had the pleasure(?!) of having to get stuff working on all of AIX/HP-UX/Tru64/Solaris, but what little dealings I've had with people who have suggests that the differences between these platforms make the discrepancies between different Linux distributions look utterly trivial by comparison. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From charlesc-lists-python-dev at pyropus.ca Tue May 20 15:17:19 2008 From: charlesc-lists-python-dev at pyropus.ca (Charles Cazabon) Date: 20 May 2008 07:17:19 -0600 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080519203820.GA12661@amk-desktop.matrixgroup.net> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> <20080519203820.GA12661@amk-desktop.matrixgroup.net> Message-ID: <20080520131719.GA2317@pyropus.ca> A.M. Kuchling wrote: > On Mon, May 19, 2008 at 08:50:39PM +0200, Thomas Heller wrote: > > Myself I would rather spend my energy to make ctypes more portable, within my > > skills and the platforms I have access to. > > Someone could run Solaris x86 inside a hosted virtual machine and make > it available to the Python developers. Is it possible to find similar > hosting for HP-UX and AIX? Or might IBM or HP be willing to donate a > low-end machine to the PSF for porting use? You can test HPUX and VMS with a (free) HP Testdrive account (along with a bunch of Linuxes and *BSDs) on various hardware platforms. I used it extensively when doing portability testing of another project. http://www.testdrive.hp.com/ Charles -- ----------------------------------------------------------------------- Charles Cazabon GPL'ed software available at: http://pyropus.ca/software/ ----------------------------------------------------------------------- From theller at ctypes.org Tue May 20 15:43:26 2008 From: theller at ctypes.org (Thomas Heller) Date: Tue, 20 May 2008 15:43:26 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080520131719.GA2317@pyropus.ca> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> <20080519203820.GA12661@amk-desktop.matrixgroup.net> <20080520131719.GA2317@pyropus.ca> Message-ID: Charles Cazabon schrieb: > A.M. Kuchling wrote: >> On Mon, May 19, 2008 at 08:50:39PM +0200, Thomas Heller wrote: >> > Myself I would rather spend my energy to make ctypes more portable, within my >> > skills and the platforms I have access to. >> >> Someone could run Solaris x86 inside a hosted virtual machine and make >> it available to the Python developers. Is it possible to find similar >> hosting for HP-UX and AIX? Or might IBM or HP be willing to donate a >> low-end machine to the PSF for porting use? > > You can test HPUX and VMS with a (free) HP Testdrive account (along with a > bunch of Linuxes and *BSDs) on various hardware platforms. I used it > extensively when doing portability testing of another project. > http://www.testdrive.hp.com/ > > Charles I'm doing that. From time to time, these systems are a pain to use IMO. A buildbot is soooo much better... -- Thanks, Thomas From ncoghlan at gmail.com Tue May 20 16:33:40 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 May 2008 00:33:40 +1000 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 Message-ID: <4832E144.8030309@gmail.com> One of the tasks where classic classes are currently far superior to new-style classes is in writing proxy classes like weakref.proxy - cases where nearly all operations need to be delegated to some other object, with only a few being handled via the proxy type object itself. With classic classes, this is trivial, since __getattr__ is always consulted, even for retrieval of special methods. With new-style classes, however, the __getattribute__ machinery can be bypassed, meaning the only way to proxy an arbitrary instance is to define all of the special methods that have C-level slots. This issue was actually first raised five and a half years ago [1], but has never been a particularly pressing problem, as anyone with any sense that needed to write a proxy object just used a classic class instead of a new-style one. In 3.0, with the demise of classic classes, that workaround goes away. So what do people think of including a ProxyBase implementation in 2.6 and 3.0 that explicitly delegates all of the C-level slots to a designated target instance? For some proxy class implementers, it would be possible to use this class as a base-class to get the special method delegation 'for free', and for others with more esoteric needs, it would at least provide a reference for which special methods needed to be provided explicitly by the proxy type. I attached a sample implementation to [1] which is essentially equivalent to weakref.proxy, but with a strong reference to the target, and written in Python rather than C. I expect the target audience for such a feature to be quite small, but without better support for it in the standard library, I also suspect it could prove to be a show-stopper for the affected developers as far as Py3k migration is concerned. Cheers, Nick. [1] http://bugs.python.org/issue643841 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fuzzyman at voidspace.org.uk Tue May 20 16:38:02 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 20 May 2008 15:38:02 +0100 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <4832E144.8030309@gmail.com> References: <4832E144.8030309@gmail.com> Message-ID: <4832E24A.7080508@voidspace.org.uk> Nick Coghlan wrote: > One of the tasks where classic classes are currently far superior to > new-style classes is in writing proxy classes like weakref.proxy - > cases where nearly all operations need to be delegated to some other > object, with only a few being handled via the proxy type object itself. I've needed to do this a few times when wrapping libraries. Michael Foord > > With classic classes, this is trivial, since __getattr__ is always > consulted, even for retrieval of special methods. > > With new-style classes, however, the __getattribute__ machinery can be > bypassed, meaning the only way to proxy an arbitrary instance is to > define all of the special methods that have C-level slots. > > This issue was actually first raised five and a half years ago [1], > but has never been a particularly pressing problem, as anyone with any > sense that needed to write a proxy object just used a classic class > instead of a new-style one. In 3.0, with the demise of classic > classes, that workaround goes away. > > So what do people think of including a ProxyBase implementation in 2.6 > and 3.0 that explicitly delegates all of the C-level slots to a > designated target instance? For some proxy class implementers, it > would be possible to use this class as a base-class to get the special > method delegation 'for free', and for others with more esoteric needs, > it would at least provide a reference for which special methods needed > to be provided explicitly by the proxy type. > > I attached a sample implementation to [1] which is essentially > equivalent to weakref.proxy, but with a strong reference to the > target, and written in Python rather than C. > > I expect the target audience for such a feature to be quite small, but > without better support for it in the standard library, I also suspect > it could prove to be a show-stopper for the affected developers as far > as Py3k migration is concerned. > > Cheers, > Nick. > > [1] http://bugs.python.org/issue643841 > From Ted.Leung at sun.com Tue May 20 18:54:43 2008 From: Ted.Leung at sun.com (Ted Leung) Date: Tue, 20 May 2008 09:54:43 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48323BB4.7020706@holdenweb.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> Message-ID: On May 19, 2008, at 7:47 PM, Steve Holden wrote: >> > OK, I know people in Sun and possibly H-P I could ask, and I may > have an arm or two to twist to get to IBM. But worst-case, what > budget would the infrastructure committee need for the hardware and > (more importantly) if the hardware materialized, would there be > manpower to maintain the platforms as "Python supported"? You can ask me as far as Sun goes. There are probably a variety of options: 1) Get the open source version of VirtualBox and use it to run a virtualized OpenSolaris 2) I can see if we can get folks access to a box running Solaris 3) I can ask and see if there are people at Sun who would be willing to jump in and help Ted From adlaiff6 at gmail.com Tue May 20 20:03:48 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Tue, 20 May 2008 14:03:48 -0400 (EDT) Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <6237575648078673977@unknownmsgid> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <6237575648078673977@unknownmsgid> Message-ID: On Mon, 19 May 2008, Bill Janssen wrote: > > On Mon, May 19, 2008 at 5:15 PM, Bill Janssen wrote: > > >> If you can run a pure Python module > > >> that does not depend on any C extension, then that platform has the > > >> support needed to run Python. > > > > > > This is certainly a point of view. One that many end-users wouldn't > > > understand :-). > > > > Perhaps, but it's clear-cut. Is OS X not properly supported because it > > can't run the _winreg module? I just don't like labeling a platform as > > unsupported just because ctypes doesn't compile on it. > > I assume _winreg runs everywhere Python is said to run, and there's a > Windows registry to examine, so I think it's a different kettle of > fish. ctypes doesn't run everywhere Python is said to run, and there > are dynamic libraries to call into. > > I think it would be great if we could get some AIX, HP-UX, and Solaris > boxes for Thomas to work on. What would motivate IBM, H-P, and Sun to > donate such gear? Perhaps each of the companies have an office > somewhere that could help with this resource allocation? For > instance, talking to the "AIX Collaboration Center" of IBM > (aixcc at us.ibm.com)? > > And these are all SYSV derivatives, aren't they? So perhaps it's some > common fix for all three? I just spent a semester in "Advanced Systems Programming in Unix/C", and all we did was port old software to solaris, hpux, irix, and friends (I don't think we used AIX, but I believe he has one running). Most of these were virtual machines, but they adequately represented the annoyances present in their respective architectures. I've cc'd my professor, hopefully he will let me know if there are any tips he has for setting up a VM. -- Cheers, Leif From martin at v.loewis.de Tue May 20 20:23:37 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 May 2008 20:23:37 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <48328EC7.7000106@egenix.com> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> Message-ID: <48331729.5080204@v.loewis.de> > Writing Unicode to stdout will still use the default encoding > ASCII to convert it to an 8-bit string. That's not true. Regards, Martin From martin at v.loewis.de Tue May 20 20:25:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 May 2008 20:25:42 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <319e029f0805200320p6c41b4b9u79ddacf3418b9cc5@mail.gmail.com> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> <9e804ac0805200316mba393f5pb8bf81d7e7e87abe@mail.gmail.com> <319e029f0805200320p6c41b4b9u79ddacf3418b9cc5@mail.gmail.com> Message-ID: <483317A6.9080300@v.loewis.de> >> You're forgetting about print; in Python 2.x, when stdout is connected to a >> terminal, the locale settings (typically the LANG, LC_ALL and LC_CTYPE >> environment variables) are taken into account when 'print' writes to >> sys.stdout. > > Isn't it then enough to make sure your locale setting are correct? > (Never had any problems myself, if works great in Ubuntu). It's much more difficult on OS X, for example, which doesn't really support the concept of locales (at least prior to 10.5). There are other odd cases, like the Eclipse one I mentioned. Setting the locale doesn't help there. Regards, Martin From janssen at parc.com Tue May 20 20:28:55 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 20 May 2008 11:28:55 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48323BB4.7020706@holdenweb.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt."58696"@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> Message-ID: <08May20.112856pdt."58696"@synergy1.parc.xerox.com> Steve Holden wrote: > The more libraries that use ctypes to call into native functionality, > the more important it becomes to have ctypes run, even if only to > implement platform-specific functionality on the given platforms. I > would like "being able to call a wide range of native libraries" to be a > Python claim on all platforms, even when the native libraries are > platform-proprietary. Yes, I'd like to see this happen, too. Bill From martin at v.loewis.de Tue May 20 20:36:01 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 May 2008 20:36:01 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <4832A9B7.5050508@egenix.com> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> <9e804ac0805200316mba393f5pb8bf81d7e7e87abe@mail.gmail.com> <4832A9B7.5050508@egenix.com> Message-ID: <48331A11.2080509@v.loewis.de> > PYTHONSTDOUTENCODING could then provide the default to > sys.stdout.encoding. Right - that's the proposal. Regards, Martin From dima at hlabs.spb.ru Tue May 20 20:01:14 2008 From: dima at hlabs.spb.ru (Dmitry Vasiliev) Date: Tue, 20 May 2008 22:01:14 +0400 Subject: [Python-Dev] Possible bug in re module? Message-ID: <483311EA.9030600@hlabs.spb.ru> I've just found a strange re behavior: >>> import re >>> re.sub("(?:ab|b|a)", "+", "cbacbabcabc") 'c++c++c+c' >>> re.sub("(?:ab|b|a){2}", "+", "cbacbabcabc") 'c+c+c+c' In the last case |-separated expressions seems don't tried from left to right. Is it bug or just me? -- Dmitry Vasiliev http://hlabs.spb.ru -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From mal at egenix.com Tue May 20 20:56:26 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 May 2008 20:56:26 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <48331729.5080204@v.loewis.de> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> <48331729.5080204@v.loewis.de> Message-ID: <48331EDA.2050503@egenix.com> On 2008-05-20 20:23, Martin v. L?wis wrote: >> Writing Unicode to stdout will still use the default encoding >> ASCII to convert it to an 8-bit string. > > That's not true. Are you sure ? > setenv LC_ALL de_DE.utf8 > python2.5 Python 2.5 (r25:51908, May 9 2007, 00:53:06) >>> u = u'???' >>> sys.stdout.write(u) Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordinal not in range(128) >>> print u ?????? >>> Only "print" will set the Py_PRINT_RAW flag to trigger the conversion from Unicode to 8-bit strings using .encoding in PyFile_WriteObject(). If not set, the default encoding is used. I'm not exactly sure why, since using .encoding would be useful in all cases. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From martin at v.loewis.de Tue May 20 21:34:32 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 20 May 2008 21:34:32 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <48331EDA.2050503@egenix.com> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> <48331729.5080204@v.loewis.de> <48331EDA.2050503@egenix.com> Message-ID: <483327C8.8050904@v.loewis.de> > I'm not exactly sure why, since using .encoding would be useful > in all cases. Right, I think it should use the file's encoding also for .write. Regards, Martin From adlaiff6 at gmail.com Tue May 20 21:34:47 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Tue, 20 May 2008 15:34:47 -0400 (EDT) Subject: [Python-Dev] Possible bug in re module? In-Reply-To: <483311EA.9030600@hlabs.spb.ru> References: <483311EA.9030600@hlabs.spb.ru> Message-ID: On Tue, 20 May 2008, Dmitry Vasiliev wrote: > I've just found a strange re behavior: > > >>> import re > >>> re.sub("(?:ab|b|a)", "+", "cbacbabcabc") > 'c++c++c+c' > >>> re.sub("(?:ab|b|a){2}", "+", "cbacbabcabc") > 'c+c+c+c' > > In the last case |-separated expressions seems don't tried from left to right. > Is it bug or just me? What were you expecting, 'c+c+cabc'? The re engine should try everything possible for a match with the entire re, not just match the first thing it finds for each section. -- Cheers, Leif From janssen at parc.com Tue May 20 21:52:11 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 20 May 2008 12:52:11 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> Message-ID: <08May20.125217pdt."58696"@synergy1.parc.xerox.com> > Bill Janssen schrieb: > >> Hmm, perhaps the ctypes documentation could use a more prominent warning > >> that it may not be available on some Unix platforms (HP-UX, AIX, IRIX), > >> and that it may require the use of GCC rather than the vendor compiler > >> on others (Solaris). > >> > >> At the moment, I suspect some projects may be switching to using it > >> without realising the implications for cross-platform portability. > > > > I think it's a tad more problematic. As other modules, both in and > > out of the standard library, move to use ctypes, it implies that > > *Python* isn't supported on those combinations. Personally, that's > > fine with me (as long as there's a workaround for Solaris!), but I > > think that Ulrich is right in saying this should be more prominent. > > I won't object if anyone adds this notice to the Python docs, > so please go ahead. A table of platforms (on the wiki?) where ctypes > builds/works or does not may also be helpful. What happens on those platforms where ctypes doesn't work? Does the module fail to import, either because it isn't present, or because it can't load the libffi library? Or does it fail silently in some way? Bill From nd at perlig.de Tue May 20 22:13:54 2008 From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=) Date: Tue, 20 May 2008 22:13:54 +0200 Subject: [Python-Dev] Possible bug in re module? In-Reply-To: <483311EA.9030600@hlabs.spb.ru> References: <483311EA.9030600@hlabs.spb.ru> Message-ID: <200805202213.55183@news.perlig.de> * Dmitry Vasiliev wrote: > I've just found a strange re behavior: > >>> import re > >>> re.sub("(?:ab|b|a)", "+", "cbacbabcabc") > > 'c++c++c+c' > > >>> re.sub("(?:ab|b|a){2}", "+", "cbacbabcabc") > > 'c+c+c+c' > > In the last case |-separated expressions seems don't tried from left to > right. Is it bug or just me? Looks fine to me. (Although if I understand that correctly, the userlist would be more appropriate). nd -- "Das Verhalten von Gates hatte mir bewiesen, dass ich auf ihn und seine beiden Gef?hrten nicht zu z?hlen brauchte" -- Karl May, "Winnetou III" Im Westen was neues: From mal at egenix.com Tue May 20 22:15:48 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 20 May 2008 22:15:48 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <483327C8.8050904@v.loewis.de> References: <48328A4D.9020704@v.loewis.de> <48328EC7.7000106@egenix.com> <48331729.5080204@v.loewis.de> <48331EDA.2050503@egenix.com> <483327C8.8050904@v.loewis.de> Message-ID: <48333174.6010102@egenix.com> On 2008-05-20 21:34, Martin v. L?wis wrote: >> I'm not exactly sure why, since using .encoding would be useful >> in all cases. > > Right, I think it should use the file's encoding also for .write. Could you add that to the proposal ?! Please also add the ability to set the .encoding on file objects. This would make adjusting e.g. sys.stdout.encoding easier from within Python. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 20 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From martin at v.loewis.de Tue May 20 22:22:03 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 May 2008 22:22:03 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <20080520104823.GA1689@phd.pp.ru> References: <48328A4D.9020704@v.loewis.de> <20080520104823.GA1689@phd.pp.ru> Message-ID: <483332EB.6090407@v.loewis.de> > Is it to override locale settings in case the user wants a different > encoding? for such cases as redirected stdout, or windows console (which > has an "OEM" encoding that differs from the locale encoding)? On Windows, the setlocale mechanism isn't used at all, since it doesn't support nl_langinfo (let alone CODESET). And yes, that is to override whatever determination Python would make on its own. >> Naming contest: it probably would be the longest of the PYTHON* >> variables. I would not want to call it PYTHONENCODING, or >> PYTHONSTDENCODING, though, because people might infer that it >> affects sys.getdefaultencoding(), which it shouldn't. > > PYTHONIOENCODING? Unprecise in a different way (as it is meant to apply only to stdout, not to all IO), but shorter. Regards, Martin From martin at v.loewis.de Tue May 20 22:24:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 20 May 2008 22:24:00 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <94bdd2610805200433ud89dd1hc5c39c06dc2db0b@mail.gmail.com> References: <48328A4D.9020704@v.loewis.de> <20080520104823.GA1689@phd.pp.ru> <94bdd2610805200433ud89dd1hc5c39c06dc2db0b@mail.gmail.com> Message-ID: <48333360.4010302@v.loewis.de> > What about PYTHONLANG ? > > or something that tries to reflect which environment variables are used > for this ? > > (LC_CTYPE -> PYTHONCTYPE ? if the code uses just LC_CTYPE) It's not meant to name a locale, but an encoding. In fact, that the encoding is tied to the locale is IMO a misconception in the POSIX locale machinery. > http://www.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html#tag_08_02 > > Just for my own knowledge: why it has to be one word ? can't it be > PYTHON_LANG ? No technical reason - just because PYTHONPATH, PYTHONHOME, PYTHONUNBUFFERED, PYTHONVERBOSE, PYTHONSTARTUP, and PYTHONCASEOK don't have underscores, either. Regards, Martin From lists at cheimes.de Tue May 20 22:24:18 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 20 May 2008 22:24:18 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> <482B26FA.6060007@v.loewis.de> Message-ID: <48333372.1080800@cheimes.de> r.m.oudkerk schrieb: > Now that socket.fromfd() and socket._dup() is available on Windows in > 2.6 and 3.0 (after a patch from me) some of the code could be removed. > I would also like to see recvfd() and sendfd() get added to the socket > module -- these functions allow the transfer of file descriptors > between processes over a Unix domain socket. I can neither find recvfd in my man pages nor in my header files in /usr/include on Linux (Ubuntu 8.04 i686). I assume recvfd and sendfd aren't syscalls but the proposed names for the functions. > (1) a Connection type (plus PipeConnection on windows) > (2) a "process shared" lock/semaphore type > (3) Win32 functions/constants to allow use of named pipes > (4) A few other Win32 functions/constants > (5) A wrapper making PyObject_AsWriteBuffer() available from Python > > I suppose (4) and perhaps (3) could sensibly be added/merged with > _subprocess.c. (5) could also be moved somewhere else. Why do you want to put the named pipes into the _subprocess module? IMHO the socket module is more appropriate for communication channels like name pipes. > (Off-topic but I think that the way that subprocess.Popen.__del__() > keeps the object alive if the process has not finished is a little > distasteful. The __del__() method and the _cleanup() function are > completely unecessary on Windows since Windows does not have zombie > processes. On Unix __del__() could simply save the pid to the _active > list and _cleanup() could be rewritten to use os.waitpid().) Interesting idea. The approach could safe us some trouble. I'm always +1 when it comes to removing ugly hacks. Christian From phd at phd.pp.ru Tue May 20 22:43:13 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 21 May 2008 00:43:13 +0400 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <483332EB.6090407@v.loewis.de> References: <48328A4D.9020704@v.loewis.de> <20080520104823.GA1689@phd.pp.ru> <483332EB.6090407@v.loewis.de> Message-ID: <20080520204313.GA552@phd.pp.ru> On Tue, May 20, 2008 at 10:22:03PM +0200, "Martin v. L?wis" wrote: > > PYTHONIOENCODING? > > Unprecise in a different way (as it is meant to apply only to stdout, > not to all IO), but shorter. I don't think you can make it both precise and short. If you want to be precise and have both PYTHON and STDOUT - shorten ENCODING to ENC. If you agree to sacrifice PYTHON - make it PYSTDOUTENCODING. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ncoghlan at gmail.com Tue May 20 23:03:07 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 May 2008 07:03:07 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> Message-ID: <48333C8B.3010000@gmail.com> Ted Leung wrote: > > On May 19, 2008, at 7:47 PM, Steve Holden wrote: >>> >> OK, I know people in Sun and possibly H-P I could ask, and I may have >> an arm or two to twist to get to IBM. But worst-case, what budget >> would the infrastructure committee need for the hardware and (more >> importantly) if the hardware materialized, would there be manpower to >> maintain the platforms as "Python supported"? > > You can ask me as far as Sun goes. There are probably a variety of > options: > > 1) Get the open source version of VirtualBox and use it to run a > virtualized OpenSolaris > 2) I can see if we can get folks access to a box running Solaris > 3) I can ask and see if there are people at Sun who would be willing to > jump in and help We actually have a couple of Solaris buildbots already - as I understand it, the issue there isn't Solaris as such, it's being able to use the Sun compiler instead of GCC to compile ctypes/libffi. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From musiccomposition at gmail.com Tue May 20 23:53:54 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 20 May 2008 16:53:54 -0500 Subject: [Python-Dev] disappearing exceptions Message-ID: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> In several places in the C code, there are instances where all exceptions can be completely ignored (akin to a bare except statement.) after a PyObject_GetAttr call. A week ago, I fixed one of these in hasattr (issue 2196) by propagating exceptions that don't inherit Exception (SystemExit, KeyboardInterrupt). However, there is another patch that propagates anything that is not a AttributeError (issue 1574217). How should we approach this? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From Ted.Leung at Sun.COM Wed May 21 00:25:41 2008 From: Ted.Leung at Sun.COM (Ted Leung) Date: Tue, 20 May 2008 15:25:41 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48333C8B.3010000@gmail.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> <48333C8B.3010000@gmail.com> Message-ID: <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> On May 20, 2008, at 2:03 PM, Nick Coghlan wrote: > > We actually have a couple of Solaris buildbots already - as I > understand it, the issue there isn't Solaris as such, it's being > able to use the Sun compiler instead of GCC to compile ctypes/libffi. Does that mean that we need access to the Sun compiler or that the Sun compiler has bugs which prevent ctypes from compiling? Ted From lists at cheimes.de Wed May 21 00:38:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 21 May 2008 00:38:28 +0200 Subject: [Python-Dev] disappearing exceptions In-Reply-To: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> References: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> Message-ID: <483352E4.8020407@cheimes.de> Benjamin Peterson schrieb: > In several places in the C code, there are instances where all > exceptions can be completely ignored (akin to a bare except > statement.) after a PyObject_GetAttr call. > > A week ago, I fixed one of these in hasattr (issue 2196) by > propagating exceptions that don't inherit Exception (SystemExit, > KeyboardInterrupt). However, there is another patch that propagates > anything that is not a AttributeError (issue 1574217). > > How should we approach this? Oh, you are bringing something to my mind. In Python 2.x hasattr() swallows every exception. This is a major issue for applications like ZODB, because database conflict errors must be propagated to the conflict resolution machinery. Shane Hathaway said once: --- That said, I was surprised to discover that Python 2.3 implements hasattr this way (from bltinmodule.c): v = PyObject_GetAttr(v, name); if (v == NULL) { PyErr_Clear(); Py_INCREF(Py_False); return Py_False; } Py_DECREF(v); Py_INCREF(Py_True); return Py_True; It should not swallow all errors, especially now that descriptors make computed attributes quite common. getattr() only recently started catching only AttributeErrors, but apparently hasattr is lagging behind. I suggest the consistency between getattr and hasattr should be fixed in Python, not Zope. --- Thankfully this issue was fixed in Python 2.6 and 3.0. In newer versions of Python hasattr() only swallows exception based on the Exception class but not BaseExceptions. We should make sure all code in the core behaves the same way. Exceptions based on BaseException must *never* be swallowed. The behavior may even be worse a macro and it should be documented in large, friendly and red letters in the C API docs. *wink* Christian From steve at holdenweb.com Wed May 21 00:43:29 2008 From: steve at holdenweb.com (Steve Holden) Date: Tue, 20 May 2008 18:43:29 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> <48333C8B.3010000@gmail.com> <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> Message-ID: <48335411.4030807@holdenweb.com> Ted Leung wrote: > > On May 20, 2008, at 2:03 PM, Nick Coghlan wrote: >> >> We actually have a couple of Solaris buildbots already - as I >> understand it, the issue there isn't Solaris as such, it's being able >> to use the Sun compiler instead of GCC to compile ctypes/libffi. > > Does that mean that we need access to the Sun compiler or that the Sun > compiler has bugs which prevent ctypes from compiling? > I don't think anyone's mentioned Solaris in this context, but there are claims that libffi "doesn't build" for AIX and HP-UX. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From musiccomposition at gmail.com Wed May 21 01:25:56 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 20 May 2008 18:25:56 -0500 Subject: [Python-Dev] disappearing exceptions In-Reply-To: <483352E4.8020407@cheimes.de> References: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> <483352E4.8020407@cheimes.de> Message-ID: <1afaf6160805201625m1859b03j92624eb50a7e6148@mail.gmail.com> On Tue, May 20, 2008 at 5:38 PM, Christian Heimes wrote: > Thankfully this issue was fixed in Python 2.6 and 3.0. In newer versions > of Python hasattr() only swallows exception based on the Exception class > but not BaseExceptions. We should make sure all code in the core behaves > the same way. Exceptions based on BaseException must *never* be > swallowed. The behavior may even be worse a macro and it should be > documented in large, friendly and red letters in the C API docs. *wink* Speaking of the C-API, I was thinking of introducing a new function called PyObject_SafeHasAttr that functions just like PyObject_HasAttr except it can fail. > > Christian > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From janssen at parc.com Wed May 21 01:28:50 2008 From: janssen at parc.com (Bill Janssen) Date: Tue, 20 May 2008 16:28:50 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48335411.4030807@holdenweb.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> <48333C8B.3010000@gmail.com> <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> <48335411.4030807@holdenweb.com> Message-ID: <08May20.162852pdt."58696"@synergy1.parc.xerox.com> > > Does that mean that we need access to the Sun compiler or that the Sun > > compiler has bugs which prevent ctypes from compiling? > > > I don't think anyone's mentioned Solaris in this context, but there are > claims that libffi "doesn't build" for AIX and HP-UX. I think there was also an issue with using the Sun compiler to build ctypes and/or libffi. Bill From lists at cheimes.de Wed May 21 01:44:56 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 21 May 2008 01:44:56 +0200 Subject: [Python-Dev] disappearing exceptions In-Reply-To: <1afaf6160805201625m1859b03j92624eb50a7e6148@mail.gmail.com> References: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> <483352E4.8020407@cheimes.de> <1afaf6160805201625m1859b03j92624eb50a7e6148@mail.gmail.com> Message-ID: <48336278.3000005@cheimes.de> Benjamin Peterson schrieb: > Speaking of the C-API, I was thinking of introducing a new function > called PyObject_SafeHasAttr that functions just like PyObject_HasAttr > except it can fail. How do you define "fail" in this context? Christian From musiccomposition at gmail.com Wed May 21 01:47:39 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Tue, 20 May 2008 18:47:39 -0500 Subject: [Python-Dev] disappearing exceptions In-Reply-To: <48336278.3000005@cheimes.de> References: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> <483352E4.8020407@cheimes.de> <1afaf6160805201625m1859b03j92624eb50a7e6148@mail.gmail.com> <48336278.3000005@cheimes.de> Message-ID: <1afaf6160805201647q4f3a0b40s108c04a4428aef0b@mail.gmail.com> On Tue, May 20, 2008 at 6:44 PM, Christian Heimes wrote: > Benjamin Peterson schrieb: >> Speaking of the C-API, I was thinking of introducing a new function >> called PyObject_SafeHasAttr that functions just like PyObject_HasAttr >> except it can fail. > > How do you define "fail" in this context? If PyObject_GetAttr raises an exception that is not an AttributeError, it can return -1 to fail. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From greg.ewing at canterbury.ac.nz Wed May 21 01:55:58 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 May 2008 11:55:58 +1200 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <4832E144.8030309@gmail.com> References: <4832E144.8030309@gmail.com> Message-ID: <4833650E.9080204@canterbury.ac.nz> Nick Coghlan wrote: > So what do people think of including a ProxyBase implementation in 2.6 > and 3.0 that explicitly delegates all of the C-level slots to a > designated target instance? Sounds good to me. -- Greg From greg.ewing at canterbury.ac.nz Wed May 21 02:13:09 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 May 2008 12:13:09 +1200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48333372.1080800@cheimes.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> <482B26FA.6060007@v.loewis.de> <48333372.1080800@cheimes.de> Message-ID: <48336915.8080007@canterbury.ac.nz> Christian Heimes wrote: > I assume recvfd and sendfd > aren't syscalls but the proposed names for the functions. Yes, the functionality in question is accessed through the sendmsg() and recvmsg() system calls. -- Greg From greg.ewing at canterbury.ac.nz Wed May 21 02:19:42 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 May 2008 12:19:42 +1200 Subject: [Python-Dev] disappearing exceptions In-Reply-To: <483352E4.8020407@cheimes.de> References: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> <483352E4.8020407@cheimes.de> Message-ID: <48336A9E.5010104@canterbury.ac.nz> Christian Heimes wrote: > Thankfully this issue was fixed in Python 2.6 and 3.0. In newer versions > of Python hasattr() only swallows exception based on the Exception class > but not BaseExceptions. Shouldn't it only be catching AttributeError, though? > We should make sure all code in the core behaves > the same way. Exceptions based on BaseException must *never* be > swallowed. Seems to me that all code in the core that catches exceptions should only be catching the exceptions it really needs to catch. That requires looking at each case individually rather than a blanket "catch Exception" recommendation (although that might still be an improvement over the status quo). -- Greg From greg.ewing at canterbury.ac.nz Wed May 21 02:26:48 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 May 2008 12:26:48 +1200 Subject: [Python-Dev] disappearing exceptions In-Reply-To: <48336A9E.5010104@canterbury.ac.nz> References: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> <483352E4.8020407@cheimes.de> <48336A9E.5010104@canterbury.ac.nz> Message-ID: <48336C48.2080701@canterbury.ac.nz> I wrote: > Shouldn't it only be catching AttributeError, though? Forget that, I forgot that PyObject_HasAttr can't signal an exception. The Py3 C API should be designed to fix this, I think. -- Greg From fdrake at acm.org Wed May 21 02:39:39 2008 From: fdrake at acm.org (Fred Drake) Date: Tue, 20 May 2008 20:39:39 -0400 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <4833650E.9080204@canterbury.ac.nz> References: <4832E144.8030309@gmail.com> <4833650E.9080204@canterbury.ac.nz> Message-ID: <6244070B-03CA-4B1F-AFFD-D948268F627B@acm.org> Nick Coghlan wrote: > So what do people think of including a ProxyBase implementation in > 2.6 and 3.0 that explicitly delegates all of the C-level slots to a > designated target instance? On May 20, 2008, at 7:55 PM, Greg Ewing wrote: > Sounds good to me. Same here. There's an implementation in zope.proxy which might be useful as a starting point: http://pypi.python.org/pypi/zope.proxy/3.4.0 I'd be willing to write the required documentation, since I'm partly to blame for the implementation. The short description of the package on PyPI contains a typo; that's been fixed in Subversion. :-) -Fred -- Fred Drake From mhammond at skippinet.com.au Wed May 21 03:51:52 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Wed, 21 May 2008 11:51:52 +1000 Subject: [Python-Dev] disappearing exceptions In-Reply-To: <48336A9E.5010104@canterbury.ac.nz> References: <1afaf6160805201453h13e24d1bpb77a6c46f9fea4bb@mail.gmail.com> <483352E4.8020407@cheimes.de> <48336A9E.5010104@canterbury.ac.nz> Message-ID: <03ee01c8bae5$40b94560$c22bd020$@com.au> Greg writes: > Christian Heimes wrote: > > Thankfully this issue was fixed in Python 2.6 and 3.0. In newer > > versions of Python hasattr() only swallows exception based on > > the Exception class but not BaseExceptions. > > Shouldn't it only be catching AttributeError, though? This has come up before - eg http://www.mail-archive.com/python-3000 at python.org/msg11630.html - which itself starts with the words "This issue has been raised before" :) It points at http://bugs.python.org/issue504714 regarding 2.x - however, if you follow that thread above, it seems Guido is against changing this behaviour even for 3k. cheers, Mark From martin at v.loewis.de Wed May 21 06:08:01 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 May 2008 06:08:01 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48333372.1080800@cheimes.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> <482B26FA.6060007@v.loewis.de> <48333372.1080800@cheimes.de> Message-ID: <4833A021.3070706@v.loewis.de> > I can neither find recvfd in my man pages nor in my header files in > /usr/include on Linux (Ubuntu 8.04 i686). I assume recvfd and sendfd > aren't syscalls but the proposed names for the functions. Yes (and no). The system call is sendmsg, with a cmsg_type of SCM_RIGHTS. I don't think there is a plan to add library functions with that name. Regards, Martin From martin at v.loewis.de Wed May 21 06:32:58 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 May 2008 06:32:58 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> <48333C8B.3010000@gmail.com> <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> Message-ID: <4833A5FA.8010703@v.loewis.de> >> We actually have a couple of Solaris buildbots already - as I >> understand it, the issue there isn't Solaris as such, it's being able >> to use the Sun compiler instead of GCC to compile ctypes/libffi. > > Does that mean that we need access to the Sun compiler or that the Sun > compiler has bugs which prevent ctypes from compiling? Neither, nor. ctypes (or, rather, libffi) has code specific to gcc (or, rather, the GNU assembler) that makes the Sun compiler reject it. IIRC, it chokes on the attempt to compile assembler code that has C preprocessor macros in it (can't test it right now). Regards, Martin From theller at ctypes.org Wed May 21 08:00:40 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 21 May 2008 08:00:40 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4833A5FA.8010703@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> <48333C8B.3010000@gmail.com> <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> <4833A5FA.8010703@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >>> We actually have a couple of Solaris buildbots already - as I >>> understand it, the issue there isn't Solaris as such, it's being able >>> to use the Sun compiler instead of GCC to compile ctypes/libffi. >> >> Does that mean that we need access to the Sun compiler or that the Sun >> compiler has bugs which prevent ctypes from compiling? > > Neither, nor. ctypes (or, rather, libffi) has code specific to gcc > (or, rather, the GNU assembler) that makes the Sun compiler reject > it. IIRC, it chokes on the attempt to compile assembler code that > has C preprocessor macros in it (can't test it right now). Could it be a solution to build libffi with gcc, then configure Python with '--with-system--ffi' and compile with the sun compiler, or would this introduce the dependencies on libgcc or whatever again that Ulrich wanted to avoid? Thomas From martin at v.loewis.de Wed May 21 08:40:09 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 May 2008 08:40:09 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> <48333C8B.3010000@gmail.com> <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> <4833A5FA.8010703@v.loewis.de> Message-ID: <4833C3C9.5030403@v.loewis.de> > Could it be a solution to build libffi with gcc, then configure Python > with '--with-system--ffi' and compile with the sun compiler, or would this > introduce the dependencies on libgcc or whatever again that Ulrich wanted > to avoid? It depends on the processor and the code, but chances are high that this would indeed create a dependency on libgcc. Regards, Martin From g.brandl at gmx.net Wed May 21 09:33:44 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 May 2008 09:33:44 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4833A021.3070706@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <482AA691.6040802@v.loewis.de> <4222a8490805140506r5ce394a0mcad3b71b174595bc@mail.gmail.com> <482B26FA.6060007@v.loewis.de> <48333372.1080800@cheimes.de> <4833A021.3070706@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >> I can neither find recvfd in my man pages nor in my header files in >> /usr/include on Linux (Ubuntu 8.04 i686). I assume recvfd and sendfd >> aren't syscalls but the proposed names for the functions. > > Yes (and no). The system call is sendmsg, with a cmsg_type of > SCM_RIGHTS. I don't think there is a plan to add library functions > with that name. There is at least a patch: http://bugs.python.org/issue1194378 Georg From ulrich.berning at denviso.de Wed May 21 09:39:02 2008 From: ulrich.berning at denviso.de (Ulrich Berning) Date: Wed, 21 May 2008 09:39:02 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> Message-ID: <4833D196.7050205@denviso.de> Brett Cannon wrote: >On Mon, May 19, 2008 at 2:03 AM, Ulrich Berning > wrote: > > >>Gregory P. Smith wrote: >> >> >> >>>On Fri, May 16, 2008 at 1:32 AM, Ulrich Berning >>> wrote: >>> >>> >>> >>>>As long as the ctypes extension doesn't build on major Un*x platforms >>>>(AIX, >>>>HP-UX), I don't like to see ctypes dependend modules included into the >>>>stdlib. Please keep the stdlib as portable as possible. >>>> >>>> >>>> >>>Nice in theory but ctypes already works on at least the top 3 popular >>>platforms. Lets not hold Python's stdlib back because nobody who uses >>>IBM and HP proprietary stuff has contributed the necessary support. >>>Making nice libraries available for other platforms is a good way to >>>encourage people to either pitch in and add support or consider their >>>platform choices in the future. >>> >>>-gps >>> >>> >>> >>> >>It's not my platform choice, it's the choice of our customers. I'm not using >>these platforms just for fun (in fact it isn't fun compared to Linux or >>Windows). >> >>If porting libffi to AIX, HP-UX, IRIX, Solaris... (especially using vendor >>compilers) would be an easy job, I'm sure it would have been done already. >> >> > >Well, ctypes isn't simple. =) > > > >>If more and more essential packages depend on ctypes, we should make a clear >>statement, that Python isn't supported any longer on platform/compiler >>combinations where libffi/ctypes doesn't build. This would give me arguments >>to drop support of our software on those platforms. >> >> > >You are mixing the stdlib in with the language in terms of what is >required for Python to work, which I think is unfair. Just because >some part of the stdlib isn't portable to some OS does not mean Python >is not supported on that platform. If you can run a pure Python module >that does not depend on any C extension, then that platform has the >support needed to run Python. Everything else is extra (which is why >we have modules in the stdlib only available on specific platforms). > >-Brett > > > I don't think it is unfair. If the development team decides one day to reimplement essential extensions like math, _socket, select, _ssl, pwd, grp, time, _locale, zlib... based on ctypes because it may be much easier to maintain python modules instead of dealing with complicated C code, Python will become pretty useless. It's like a cool radio without the chance to get any batteries for it, pretty useless. Platform specific modules are documented as such and nobody would expect a _winreg module on AIX or HP-UX. As said before, PyOpenGL is an example of an extension that moved from C code to Python/ctypes, luckily we don't use it, but what if the maintainers of MySQL-Python or cx_Oracle decide to move to ctypes. Having the ctypes extension in the stdlib doesn't imply it runs on any platform where python runs. Extension writers should keep this in mind when they decide to use ctypes. They should document, that their extension depends on ctypes and therefore doesn't run on platforms where ctypes doesn't work. Ulli From arigo at tunes.org Wed May 21 10:45:48 2008 From: arigo at tunes.org (Armin Rigo) Date: Wed, 21 May 2008 10:45:48 +0200 Subject: [Python-Dev] Proposal: new environment variable PYTHONSTDOUTENCODING In-Reply-To: <48328A4D.9020704@v.loewis.de> References: <48328A4D.9020704@v.loewis.de> Message-ID: <20080521084548.GA8100@code0.codespeak.net> Hi Martin, On Tue, May 20, 2008 at 10:22:37AM +0200, "Martin v. L?wis" wrote: > In particular, setting this environment variable would also disable > the detection of whether stdout is a terminal. In this case, it seems to me that existing programs that start python as a non-interactive subprocess, via a mecanism like os.popen2() or the equivalent in other languages, will receive bogus data (the python banner) and/or hang in unexpected ways (the subprocess waiting for more input after the prompt). A bientot, Armin From ncoghlan at gmail.com Wed May 21 11:41:42 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 May 2008 19:41:42 +1000 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <6244070B-03CA-4B1F-AFFD-D948268F627B@acm.org> References: <4832E144.8030309@gmail.com> <4833650E.9080204@canterbury.ac.nz> <6244070B-03CA-4B1F-AFFD-D948268F627B@acm.org> Message-ID: <4833EE56.5010705@gmail.com> Fred Drake wrote: > Nick Coghlan wrote: >> So what do people think of including a ProxyBase implementation in 2.6 >> and 3.0 that explicitly delegates all of the C-level slots to a >> designated target instance? > > > On May 20, 2008, at 7:55 PM, Greg Ewing wrote: >> Sounds good to me. > > Same here. There's an implementation in zope.proxy which might be > useful as a starting point: While a proxy class written in C would no doubt be faster than one written in Python, one of the things I'm hoping to achieve is for the stdlib generic proxy to serve as an example for people writing their own new-style proxy classes in addition to being usable as a base class (Adam Olsen suggested ProxyMixin instead of ProxyBase as a possible name). One other issue is where to put it - none of the existing stdlib modules seemed appropriate, so my sample implementation in the tracker issue just invents a new 'typetools' module. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fdrake at acm.org Wed May 21 13:36:19 2008 From: fdrake at acm.org (Fred Drake) Date: Wed, 21 May 2008 07:36:19 -0400 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <4833EE56.5010705@gmail.com> References: <4832E144.8030309@gmail.com> <4833650E.9080204@canterbury.ac.nz> <6244070B-03CA-4B1F-AFFD-D948268F627B@acm.org> <4833EE56.5010705@gmail.com> Message-ID: On May 21, 2008, at 5:41 AM, Nick Coghlan wrote: > While a proxy class written in C would no doubt be faster than one > written in Python, one of the things I'm hoping to achieve is for > the stdlib generic proxy to serve as an example for people writing > their own new-style proxy classes in addition to being usable as a > base class (Adam Olsen suggested ProxyMixin instead of ProxyBase as > a possible name). The idea that it would serve as an example seems odd; the reason to make things part of the standard library is because their implementation needs to track the Python core closely. For a proxy, it would be the need to reflect additional slot methods as they're added. A Python implementation may be able to do this just fine, but the performance penalty would make it uninteresting for many large applications. For what it's worth, zope.proxy does support subclassing. -Fred -- Fred Drake From ncoghlan at gmail.com Wed May 21 16:30:25 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 May 2008 00:30:25 +1000 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: References: <4832E144.8030309@gmail.com> <4833650E.9080204@canterbury.ac.nz> <6244070B-03CA-4B1F-AFFD-D948268F627B@acm.org> <4833EE56.5010705@gmail.com> Message-ID: <48343201.1040706@gmail.com> Fred Drake wrote: > On May 21, 2008, at 5:41 AM, Nick Coghlan wrote: >> While a proxy class written in C would no doubt be faster than one >> written in Python, one of the things I'm hoping to achieve is for the >> stdlib generic proxy to serve as an example for people writing their >> own new-style proxy classes in addition to being usable as a base >> class (Adam Olsen suggested ProxyMixin instead of ProxyBase as a >> possible name). > > > The idea that it would serve as an example seems odd; the reason to make > things part of the standard library is because their implementation > needs to track the Python core closely. For a proxy, it would be the > need to reflect additional slot methods as they're added. It does that too - the unit tests I just added to the tracker issue for this proposal will actually start to fail if something is added to the operator module without the unit tests for the proposed ProxyMixin being updated appropriately (it would actually probably be a good thing to have something similar in the weakref.proxy test suite to prevent a repeat of our effort with forgetting to update that when the operator.index slot was added) > A Python > implementation may be able to do this just fine, but the performance > penalty would make it uninteresting for many large applications. It should still be faster than the classic class __getattr__ based proxy implementations it is primarily intended to replace. People that are already using a C-based implementation like zope.proxy aren't going to be affected by the removal of classic classes in 3.0, since they weren't relying on them anyway. In many ways, the TestProxyMixin test suite may prove more useful in the long run than the ProxyMixin class itself, since the test suite provides a template for how to test that a proxy class is doing its job, and also to automatically detect when new C-level operations have been added to the interpreter that the proxy class doesn't support. > For what it's worth, zope.proxy does support subclassing. But not in a mixin style - since zope.proxy is an extension class in its own right, it can't be combined with other extension classes. The Python version can be combined with anything. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Wed May 21 17:11:05 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 21 May 2008 10:11:05 -0500 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4833D196.7050205@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> Message-ID: <18484.15241.265856.844772@montanaro-dyndns-org.local> Ulrich> Having the ctypes extension in the stdlib doesn't imply it runs Ulrich> on any platform where python runs. Maybe the presence of a functioning ctypes (can|might|should|will) become the operational definition of "Python runs on platform X". Skip From ziade.tarek at gmail.com Wed May 21 17:07:00 2008 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 21 May 2008 17:07:00 +0200 Subject: [Python-Dev] A smarter shutil.copytree ? In-Reply-To: References: <94bdd2610804170952p12f3bffan114d937178ebf046@mail.gmail.com> <94bdd2610804171527m6ec08d52g4e05517643541316@mail.gmail.com> <94bdd2610804201515j24a3f7e4uf7ab98cfa915fafb@mail.gmail.com> <94bdd2610804220056w24a34d07q16b1151796f1b9d7@mail.gmail.com> Message-ID: <94bdd2610805210807q7a0f584ib30801e184125920@mail.gmail.com> On Tue, Apr 22, 2008 at 7:04 PM, Steven Bethard wrote: > > The callable takes the src directory + its content as a list, and > > returns filter eligible for exclusion > > FWIW, that looks better to me. > > > That makes me wonder, like Alexander said on the bug tracker: > > In the glob-style patterns callable, do we want to deal with absolute > paths ? > > I think that it would be okay to document that > shutil.ignore_patterns() only accepts patterns matching individual > filenames (not complex paths). If someone needs to do something with > absolute paths, then they can write their own 'ignore' function, > right? > Yes, the patch has been changed and corrected by a few people (thanks), and so the doc, http://bugs.python.org/issue2663 So i guess it can be reviewed by a commiter at this stage Regards Tarek > > Steve > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phd.pp.ru Wed May 21 17:32:25 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 21 May 2008 19:32:25 +0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <18484.15241.265856.844772@montanaro-dyndns-org.local> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <18484.15241.265856.844772@montanaro-dyndns-org.local> Message-ID: <20080521153225.GA32556@phd.pp.ru> On Wed, May 21, 2008 at 10:11:05AM -0500, skip at pobox.com wrote: > Maybe the presence of a functioning ctypes (can|might|should|will) become > the operational definition of "Python runs on platform X". It is not black-or-white, runs or doesn't. PythonD, e.g., runs on DOS, can use sockets (from WatTCP library), but certainly cannot do multithreading or multitasking. So the wording should be "Python supports platform X with the following limitations: ..." Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From fuzzyman at voidspace.org.uk Wed May 21 17:26:29 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 21 May 2008 16:26:29 +0100 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <18484.15241.265856.844772@montanaro-dyndns-org.local> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <18484.15241.265856.844772@montanaro-dyndns-org.local> Message-ID: <48343F25.8080201@voidspace.org.uk> skip at pobox.com wrote: > Ulrich> Having the ctypes extension in the stdlib doesn't imply it runs > Ulrich> on any platform where python runs. > > Maybe the presence of a functioning ctypes (can|might|should|will) become > the operational definition of "Python runs on platform X". > And what about platforms like the JVM or CLR? Incidentally there were a small but vocal group of Pythonistas who were (are?) certain that IronPython is not Python because it doesn't have [all of...] the C extensions. Michael Foord > Skip > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From fuzzyman at voidspace.org.uk Wed May 21 17:45:13 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 21 May 2008 16:45:13 +0100 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <6D5AB552-2ABB-40F9-8D43-0EE4E2601A76@fuhm.net> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <18484.15241.265856.844772@montanaro-dyndns-org.local> <48343F25.8080201@voidspace.org.uk> <6D5AB552-2ABB-40F9-8D43-0EE4E2601A76@fuhm.net> Message-ID: <48344389.1030103@voidspace.org.uk> James Y Knight wrote: > > On May 21, 2008, at 11:26 AM, Michael Foord wrote: > >> And what about platforms like the JVM or CLR? >> >> Incidentally there were a small but vocal group of Pythonistas who >> were (are?) certain that IronPython is not Python because it doesn't >> have [all of...] the C extensions. > > It seems likely to be easier to port ctypes to Jython/IronPython than > it is to port 500 C extensions to Jython. So more software using ctype > seems like an improvement for those communities. > Yes, both of them have their own FFIs. Seo Sanghyeon started a port of ctypes for IronPython that should be revived by 'someone'. :-) Michael Foord > James From foom at fuhm.net Wed May 21 17:50:20 2008 From: foom at fuhm.net (James Y Knight) Date: Wed, 21 May 2008 11:50:20 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48343F25.8080201@voidspace.org.uk> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <18484.15241.265856.844772@montanaro-dyndns-org.local> <48343F25.8080201@voidspace.org.uk> Message-ID: <6D5AB552-2ABB-40F9-8D43-0EE4E2601A76@fuhm.net> On May 21, 2008, at 11:26 AM, Michael Foord wrote: > And what about platforms like the JVM or CLR? > > Incidentally there were a small but vocal group of Pythonistas who > were (are?) certain that IronPython is not Python because it doesn't > have [all of...] the C extensions. It seems likely to be easier to port ctypes to Jython/IronPython than it is to port 500 C extensions to Jython. So more software using ctype seems like an improvement for those communities. James From skip at pobox.com Wed May 21 18:20:46 2008 From: skip at pobox.com (skip at pobox.com) Date: Wed, 21 May 2008 11:20:46 -0500 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48343F25.8080201@voidspace.org.uk> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <18484.15241.265856.844772@montanaro-dyndns-org.local> <48343F25.8080201@voidspace.org.uk> Message-ID: <18484.19422.772780.894379@montanaro-dyndns-org.local> Skip> Maybe the presence of a functioning ctypes (can|might|should|will) Skip> become the operational definition of "Python runs on platform X". Michael> And what about platforms like the JVM or CLR? Sorry, allow me to rephrase: Maybe the presence of a functioning ctypes (can|might|should|will) become the operational definition of "CPython runs on platform X". Michael> Incidentally there were a small but vocal group of Pythonistas Michael> who were (are?) certain that IronPython is not Python because Michael> it doesn't have [all of...] the C extensions. >From my perspective IronPython isn't Python because it doesn't run on my platforms (Solaris & Mac). ;-) (At least not easily enough for an old codger like me to install.) Skip From theller at ctypes.org Wed May 21 18:54:36 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 21 May 2008 18:54:36 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48344389.1030103@voidspace.org.uk> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <18484.15241.265856.844772@montanaro-dyndns-org.local> <48343F25.8080201@voidspace.org.uk> <6D5AB552-2ABB-40F9-8D43-0EE4E2601A76@fuhm.net> <48344389.1030103@voidspace.org.uk> Message-ID: Michael Foord schrieb: > James Y Knight wrote: >> >> On May 21, 2008, at 11:26 AM, Michael Foord wrote: >> >>> And what about platforms like the JVM or CLR? >>> >>> Incidentally there were a small but vocal group of Pythonistas who >>> were (are?) certain that IronPython is not Python because it doesn't >>> have [all of...] the C extensions. >> >> It seems likely to be easier to port ctypes to Jython/IronPython than >> it is to port 500 C extensions to Jython. So more software using ctype >> seems like an improvement for those communities. >> > > Yes, both of them have their own FFIs. Seo Sanghyeon started a port of > ctypes for IronPython that should be revived by 'someone'. :-) OT: There is also some noise about a ctypes-js implementation for JavaScript; although I do not know how far they are. Thomas From theller at ctypes.org Wed May 21 19:00:24 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 21 May 2008 19:00:24 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <08May20.125217pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> <08May20.125217pdt."58696"@synergy1.parc.xerox.com> Message-ID: Bill Janssen schrieb: > What happens on those platforms where ctypes doesn't work? Does the > module fail to import, either because it isn't present, or because it > can't load the libffi library? Or does it fail silently in some way? It doesn't compile, and Python's setup.py script removes if afterwards IIRC so it cannot be imported. In some cases it compiles, but the ctypes unittests segfault (I have seen this some days ago on HP-UX PA with gcc). The cause for the segfault is either a ctypes bug or a libffi bug. Thomas From theller at ctypes.org Wed May 21 19:10:58 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 21 May 2008 19:10:58 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080519203820.GA12661@amk-desktop.matrixgroup.net> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> <20080519203820.GA12661@amk-desktop.matrixgroup.net> Message-ID: A.M. Kuchling schrieb: > On Mon, May 19, 2008 at 08:50:39PM +0200, Thomas Heller wrote: >> Myself I would rather spend my energy to make ctypes more portable, within my >> skills and the platforms I have access to. > > Someone could run Solaris x86 inside a hosted virtual machine and make > it available to the Python developers. Is it possible to find similar > hosting for HP-UX and AIX? Or might IBM or HP be willing to donate a > low-end machine to the PSF for porting use? I have a vmware appliance installed that runs "solaris 10 update 2", gcc and the sun compiler are installed. As expected (?), ctypes works fine when compiled with gcc, but fails to build with the siun compiler. There is also a solaris buildbot running on a sparc machine. Thomas From theller at ctypes.org Wed May 21 19:18:56 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 21 May 2008 19:18:56 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4831425F.2050102@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> Message-ID: Ulrich Berning schrieb: > > If porting libffi to AIX, HP-UX, IRIX, Solaris... (especially using > vendor compilers) would be an easy job, I'm sure it would have been done > already. If more and more essential packages depend on ctypes, we should > make a clear statement, that Python isn't supported any longer on > platform/compiler combinations where libffi/ctypes doesn't build. This > would give me arguments to drop support of our software on those platforms. Ulrich, if *you* have access to these platforms and want to help a good start which hopefully wouldn't take too much time would be to compile the current libffi release [1], run the test suite, and report the results. Compile with gcc, for a start, and note that the testsuite requires dejagnu (the PyObjC folks have written a dejagnu replacement in Python for testing libffi, but I haven't tried that). [1] http://sourceware.org/libffi/ -- Thanks, Thomas From tnelson at onresolve.com Wed May 21 19:22:18 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 21 May 2008 18:22:18 +0100 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? Message-ID: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> Gah. I just went and visited the Berkeley DB download site as I was preparing my commit message so I could refer to the exact .tar.gz being imported, only to notice that the latest version is now 4.7.25. Jesus, can we use this version? Trent. ________________________________________ From: python-dev-bounces+tnelson=onresolve.com at python.org [python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Jesus Cea [jcea at jcea.es] Sent: 19 May 2008 23:46 To: Python DEV Subject: Re: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: | I downloaded the source that includes AES encryption, for no reason | other than it was first on the list. I'm now wondering if we should | only be importing the 'NC' source that doesn't contain any | encryption? Jesus, does pybsddb use any of the Berkeley DB | encryption facilities? Would anything break if we built the | bsddb module without encryption? Yes, pybsddb3 4.6.4 supports cryptography if the underlying Berkeley DB library is crypto enabled. In principle, you can compile BDB without crypto, and pybsddb3 should work, but you would lose ability to open any DB formerly created using page encryption or page checksum. Export laws aside, we better compile with crypto :). Details: http://www.jcea.es/programacion/pybsddb_doc/dbenv.html#set_encrypt - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDIDWplgi5GaxT1NAQKaagP+Myn6t364B9cVMUtlKjTIX1LBZkkEG7SX pv+hjUZ68r0Suw8SkhXnBbL2Ek3/yFhvNHH8qAZvEtRGsFGmKiNXccn9Ce3oy0me rAYhuuICleAUJ8RO8FZz+sGK3SR4kITrqnGfMnG46u0rXDQy82NSRW2+uWz/D96d U+X/wfV2HgQ= =io+q -----END PGP SIGNATURE----- _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From fuzzyman at voidspace.org.uk Wed May 21 20:03:05 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 21 May 2008 19:03:05 +0100 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <18484.19422.772780.894379@montanaro-dyndns-org.local> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <18484.15241.265856.844772@montanaro-dyndns-org.local> <48343F25.8080201@voidspace.org.uk> <18484.19422.772780.894379@montanaro-dyndns-org.local> Message-ID: <483463D9.3060300@voidspace.org.uk> skip at pobox.com wrote: > Skip> Maybe the presence of a functioning ctypes (can|might|should|will) > Skip> become the operational definition of "Python runs on platform X". > > Michael> And what about platforms like the JVM or CLR? > > Sorry, allow me to rephrase: > > Maybe the presence of a functioning ctypes (can|might|should|will) > become the operational definition of "CPython runs on platform X". > > Michael> Incidentally there were a small but vocal group of Pythonistas > Michael> who were (are?) certain that IronPython is not Python because > Michael> it doesn't have [all of...] the C extensions. > > >From my perspective IronPython isn't Python because it doesn't run on my > platforms (Solaris & Mac). ;-) (At least not easily enough for an old codger > like me to install.) > IronPython runs very well on the Mac. Installing Mono is pretty much 'one click' these days and it *comes* with IronPython (and has done for a while). I'm writing "IronPython in Action" on the Mac. Solaris I don't care about... :-) Michael Foord > Skip > From jcea at jcea.es Wed May 21 20:14:45 2008 From: jcea at jcea.es (Jesus Cea) Date: Wed, 21 May 2008 20:14:45 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> Message-ID: <48346695.4030906@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: | Gah. I just went and visited the Berkeley DB download site as | I was preparing my commit message so I could refer to the | exact .tar.gz being imported, only to notice that the latest | version is now 4.7.25. Jesus, can we use this version? Err.... No. It is not clear to me that python 2.6/3.0 will be published with BDB 4.6 or 4.7 support. 4.6 has several known issues, apparently solved in 4.7. I will comment when I have spend some time with 4.7. Any comment, Greg?. In any case, what would be the procedure to update the buildbot infraestructure?. No, DO NOT update Berkeley DB yet!. I keep a private branch in pybsddb for BDB 4.7 support, waiting for Oracle publication. Since they already pushed 4.7.25 out (no pre warning for bindings developers, too bad!), I think I can publish pybsddb 4.7.0 in a couple of days. That done, I will update python version. PS: pybsddb 4.7.0 will support Berkeley DB 4.0 to 4.7. So, the buildbots don't need to be upgraded.. unless we decide that Python 4.6/3.0 will have Berkeley DB 4.7. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDRmkplgi5GaxT1NAQIGGwP/Rxn2A1TgEfKPU14utLBe2Q9oJlILvusX Fdn69/ZXr4Rk7AiVHrvIkokNxBxnQKMWKykXnZmXAF0Tu75IqbOFuokkE8bu7PG9 yvsPsISbgpFmxrdo1bbJcO4r2ZtsMivw+p5l47GSon9reInl8Dr4cOOG3PayiuSq 9icmszNADCo= =u5Qg -----END PGP SIGNATURE----- From martin at v.loewis.de Wed May 21 20:57:33 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 May 2008 20:57:33 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4833D196.7050205@denviso.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> Message-ID: <4834709D.7030604@v.loewis.de> > As said before, PyOpenGL is an example of an extension that moved from C > code to Python/ctypes, luckily we don't use it, but what if the > maintainers of MySQL-Python or cx_Oracle decide to move to ctypes. > Having the ctypes extension in the stdlib doesn't imply it runs on any > platform where python runs. Extension writers should keep this in mind > when they decide to use ctypes. They should document, that their > extension depends on ctypes and therefore doesn't run on platforms where > ctypes doesn't work. Plus, even if ctypes works, the code might be incorrect, because they had been assuming structure layouts and symbolic constants that have just a different definition on some other platform, causing the extension module to crash. Writing portable ctypes modules is really hard - significantly harder than writing portable C code (although writing non-portable ctypes code is apparently easier than writing non-portable C code). Regards, Martin From martin at v.loewis.de Wed May 21 21:03:15 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 21 May 2008 21:03:15 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <48346695.4030906@jcea.es> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es> Message-ID: <483471F3.2030305@v.loewis.de> > In any case, what would be the procedure to update the buildbot > infraestructure?. You need to import the sources into the Python subversion repository; I understand Trent was about to do that. Then you need to adjust Tools/msi/external.bat to have the slaves export the new version. Regards, Martin From janssen at parc.com Wed May 21 21:29:37 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 21 May 2008 12:29:37 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> <20080519203820.GA12661@amk-desktop.matrixgroup.net> Message-ID: <08May21.122939pdt."58696"@synergy1.parc.xerox.com> Thomas Heller schrieb: > A.M. Kuchling schrieb: > > On Mon, May 19, 2008 at 08:50:39PM +0200, Thomas Heller wrote: > >> Myself I would rather spend my energy to make ctypes more portable, within my > >> skills and the platforms I have access to. > > > > Someone could run Solaris x86 inside a hosted virtual machine and make > > it available to the Python developers. Is it possible to find similar > > hosting for HP-UX and AIX? Or might IBM or HP be willing to donate a > > low-end machine to the PSF for porting use? > > I have a vmware appliance installed that runs "solaris 10 update 2", > gcc and the sun compiler are installed. As expected (?), ctypes works > fine when compiled with gcc, but fails to build with the siun compiler. > > There is also a solaris buildbot running on a sparc machine. I just downloaded the Python 2.5.2 source tar, and tried to build it on a Solaris 11 machine with the SunPro 8 compiler (Sun CC 5.5), and failed: % ./configure [...] creating Modules/Setup creating Modules/Setup.local creating Makefile % make cc -c -DNDEBUG -O -I. -IInclude -I./Include -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c "/usr/include/sys/feature_tests.h", line 353: #error: "Compiler or options invalid for pre-UNIX 03 X/Open applications and pre-2001 POSIX applications" cc: acomp failed for ./Modules/python.c *** Error code 2 make: Fatal error: Command failed for target `Modules/python.o' % So maybe Python just doesn't run on Solaris with the Sun C compiler. Certainly doesn't build out of the box. Bill From theller at ctypes.org Wed May 21 21:42:36 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 21 May 2008 21:42:36 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4834709D.7030604@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <4834709D.7030604@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >> As said before, PyOpenGL is an example of an extension that moved from C >> code to Python/ctypes, luckily we don't use it, but what if the >> maintainers of MySQL-Python or cx_Oracle decide to move to ctypes. >> Having the ctypes extension in the stdlib doesn't imply it runs on any >> platform where python runs. Extension writers should keep this in mind >> when they decide to use ctypes. They should document, that their >> extension depends on ctypes and therefore doesn't run on platforms where >> ctypes doesn't work. > > Plus, even if ctypes works, the code might be incorrect, because they > had been assuming structure layouts and symbolic constants that have > just a different definition on some other platform, causing the > extension module to crash. > > Writing portable ctypes modules is really hard - significantly harder > than writing portable C code (although writing non-portable ctypes > code is apparently easier than writing non-portable C code). I would say that writing portable C code is hard as well, aren't there just more tools that help? Thomas From janssen at parc.com Wed May 21 21:54:41 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 21 May 2008 12:54:41 PDT Subject: [Python-Dev] Python & ctypes on Solaris (was: Addition of "pyprocessing" module to standard lib) In-Reply-To: <08May21.122939pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> <20080519203820.GA12661@amk-desktop.matrixgroup.net> <08May21.122939pdt."58696"@synergy1.parc.xerox.com> Message-ID: <08May21.125444pdt."58696"@synergy1.parc.xerox.com> > So maybe Python just doesn't run on Solaris with the Sun C compiler. > Certainly doesn't build out of the box. Hmmm, when I look at http://www.sun.com/software/solaris/freeware/, I see that Python is listed as "included with Solaris 10" as "Sun-supported software". But the version installed on my Solaris 11 system is 2.4.4. Still, this gives me hope that Sun is looking at Python 2.5, including ctypes, and will eventually make it work. Bill From exarkun at divmod.com Wed May 21 21:58:06 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 21 May 2008 15:58:06 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4834709D.7030604@v.loewis.de> Message-ID: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> On Wed, 21 May 2008 20:57:33 +0200, "\"Martin v. L?wis\"" wrote: >> As said before, PyOpenGL is an example of an extension that moved from C >> code to Python/ctypes, luckily we don't use it, but what if the >> maintainers of MySQL-Python or cx_Oracle decide to move to ctypes. >> Having the ctypes extension in the stdlib doesn't imply it runs on any >> platform where python runs. Extension writers should keep this in mind >> when they decide to use ctypes. They should document, that their >> extension depends on ctypes and therefore doesn't run on platforms where >> ctypes doesn't work. > >Plus, even if ctypes works, the code might be incorrect, because they >had been assuming structure layouts and symbolic constants that have >just a different definition on some other platform, causing the >extension module to crash. > >Writing portable ctypes modules is really hard - significantly harder >than writing portable C code (although writing non-portable ctypes >code is apparently easier than writing non-portable C code). True. There's some room for improvement in ctypes here, fortunately. For example, PyPy has some tools which resolve the particular problem you're talking about; the library is even available separately and can (and probably should) be used by anyone writing a ctypes module. Sample usage and installation instructions available here: http://codespeak.net/~fijal/configure.html Jean-Paul From foom at fuhm.net Wed May 21 22:32:57 2008 From: foom at fuhm.net (James Y Knight) Date: Wed, 21 May 2008 16:32:57 -0400 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> References: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> Message-ID: <0C8E8176-BBFC-438B-B1D9-539CBD7BFB59@fuhm.net> On May 21, 2008, at 3:58 PM, Jean-Paul Calderone wrote: >> Plus, even if ctypes works, the code might be incorrect, because they >> had been assuming structure layouts and symbolic constants that have >> just a different definition on some other platform, causing the >> extension module to crash. >> >> Writing portable ctypes modules is really hard - significantly harder >> than writing portable C code (although writing non-portable ctypes >> code is apparently easier than writing non-portable C code). > > True. There's some room for improvement in ctypes here, fortunately. > > For example, PyPy has some tools which resolve the particular problem > you're talking about; the library is even available separately and can > (and probably should) be used by anyone writing a ctypes module. > > Sample usage and installation instructions available here: > > http://codespeak.net/~fijal/configure.html The "csvn" subversion bindings use "ctypesgen" to grovel header files: http://code.google.com/p/ctypesgen/ There's also ctypeslib, which uses gcc-xml to parse the headers instead of a pure python C parser as ctypesgen does: http://svn.python.org/projects/ctypes/trunk/ctypeslib/ I don't really know how all these tools compare to eachother. It sure would be nice if someone could compare them, and figure out if one can be "blessed" and included with python so that doing the right thing is easy. James From python at rcn.com Wed May 21 22:53:08 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 21 May 2008 13:53:08 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. References: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> <0C8E8176-BBFC-438B-B1D9-539CBD7BFB59@fuhm.net> Message-ID: <9636FD7FF9B942759ED2E781B91324B7@RaymondLaptop1> This thread has diverged a bit from the original topic. I suggest going ahead and adding pyprocessing to the library. IMO, its functionality is going to be an essential capability as more and more computers ship with multiple processors. At this point, the basic API for pyprocessing seems well thought-out and somewhat stable. Over time, I expect the implementation will get tweaked in a number of ways including support more platforms as developers figure-out that they like the idea enough to write some platform dependent patches. Putting this functionality in 2.6/3.0 would provide a really nice incentive to update from Py2.5. It would be a sad lost opportunity if this module had to wait another couple years. Raymond From janssen at parc.com Wed May 21 22:55:38 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 21 May 2008 13:55:38 PDT Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <0C8E8176-BBFC-438B-B1D9-539CBD7BFB59@fuhm.net> References: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> <0C8E8176-BBFC-438B-B1D9-539CBD7BFB59@fuhm.net> Message-ID: <08May21.135539pdt."58696"@synergy1.parc.xerox.com> > The "csvn" subversion bindings use "ctypesgen" to grovel header files: > http://code.google.com/p/ctypesgen/ Thanks for the pointer. Looks nice. I've used ctypeslib before, but it takes a bit of infrastructure building (gcc-xml) before it works. Bill From aleaxit at gmail.com Wed May 21 23:46:41 2008 From: aleaxit at gmail.com (Alex Martelli) Date: Wed, 21 May 2008 14:46:41 -0700 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <9636FD7FF9B942759ED2E781B91324B7@RaymondLaptop1> References: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> <0C8E8176-BBFC-438B-B1D9-539CBD7BFB59@fuhm.net> <9636FD7FF9B942759ED2E781B91324B7@RaymondLaptop1> Message-ID: On Wed, May 21, 2008 at 1:53 PM, Raymond Hettinger wrote: > This thread has diverged a bit from the original topic. > > I suggest going ahead and adding pyprocessing to the library. > IMO, its functionality is going to be an essential capability as > more and more computers ship with multiple processors. > At this point, the basic API for pyprocessing seems well thought-out and > somewhat stable. Over time, I expect > the implementation will get tweaked in a number of ways > including support more platforms as developers figure-out > that they like the idea enough to write some platform dependent patches. > > Putting this functionality in 2.6/3.0 would provide a really > nice incentive to update from Py2.5. It would be a sad > lost opportunity if this module had to wait another couple years. I feel essentially the same way: it WOULD be sad to waste this excellent opportunity, so I second the suggestion to put pyprocessing in the library right now. Alex From greg.ewing at canterbury.ac.nz Thu May 22 00:44:48 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 May 2008 10:44:48 +1200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4833A5FA.8010703@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <8926242303997388828@unknownmsgid> <08May19.181320pdt.58696@synergy1.parc.xerox.com> <48323BB4.7020706@holdenweb.com> <48333C8B.3010000@gmail.com> <262FC728-2EF1-4B97-BA72-6FC50D762D08@Sun.COM> <4833A5FA.8010703@v.loewis.de> Message-ID: <4834A5E0.5080206@canterbury.ac.nz> Martin v. L?wis wrote: > IIRC, it chokes on the attempt to compile assembler code that > has C preprocessor macros in it (can't test it right now). Could the build process be modified to run the C preprocessor over the assembly language first? -- Greg From ncoghlan at gmail.com Thu May 22 01:08:48 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 May 2008 09:08:48 +1000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> <0C8E8176-BBFC-438B-B1D9-539CBD7BFB59@fuhm.net> <9636FD7FF9B942759ED2E781B91324B7@RaymondLaptop1> Message-ID: <4834AB80.3010208@gmail.com> Alex Martelli wrote: > On Wed, May 21, 2008 at 1:53 PM, Raymond Hettinger wrote: >> Putting this functionality in 2.6/3.0 would provide a really >> nice incentive to update from Py2.5. It would be a sad >> lost opportunity if this module had to wait another couple years. > > I feel essentially the same way: it WOULD be sad to waste this > excellent opportunity, so I second the suggestion to put pyprocessing > in the library right now. Thirded (I think I'm contradicting what I posted earlier in the thread, but I've had more of a chance to think about it and 18 months really is quite a long time to wait for this functionality...) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tnelson at onresolve.com Thu May 22 02:12:33 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 22 May 2008 01:12:33 +0100 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <48346695.4030906@jcea.es> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com>, <48346695.4030906@jcea.es> Message-ID: <6167796BFEB5D0438720AC212E89A6B0078657D9@exchange.onresolve.com> > Trent Nelson wrote: > | Gah. I just went and visited the Berkeley DB download site as > | I was preparing my commit message so I could refer to the > | exact .tar.gz being imported, only to notice that the latest > | version is now 4.7.25. Jesus, can we use this version? > > Err.... No. > > It is not clear to me that python 2.6/3.0 will be published with BDB 4.6 > or 4.7 support. 4.6 has several known issues, apparently solved in 4.7. I could have sworn I heard a few people mention that "4.5 has issues, but they're solved in 4.6" at PyCon ;-) > I keep a private branch in pybsddb for BDB 4.7 support, waiting for > Oracle publication. Since they already pushed 4.7.25 out (no pre warning > for bindings developers, too bad!), I think I can publish pybsddb 4.7.0 > in a couple of days. That done, I will update python version. > > PS: pybsddb 4.7.0 will support Berkeley DB 4.0 to 4.7. So, the buildbots > don't need to be upgraded.. unless we decide that Python 4.6/3.0 will > have Berkeley DB 4.7. Seems like the amount of work you have to do has doubled now that you've been added as an svn committer, given that you're maintaining multiple branches of code, one for pybsddb, and one for an official Python branch. I was under the impression that pybsddb would be assimilated into trunk/ Lib/bsddb and become the sole pybsddb incarnation. That is, you'd ditch the separate SourceForge pybsddb project and just work directly in the Python tree. I think I remember reading some concerns you had about doing this last week though, surrounding things like wanting to be able to release pybsddb versions more frequently than Python versions are released. Just because you live in trunk/Lib/bsddb shouldn't prevent you from doing this though; in fact, as long as you're sensitive to major release timeframes, I'm sure we'd love trunk to always track the latest Berkeley DB version; if all the buildbots stay green with 4.7 and there are no regressions in functionality, then hey, lets go with 4.7 for 2.6/3.0. It's probably safe to say that you're the one doing the most work with the code base and Berkeley DB in general, which means you'll be in a much better position to advise us as to which version we should be including or ignoring for a given Python release. In general, if we can support the latest release, we will. If Oracle come out with Berkeley DB 4.8 a week after 2.6 is released, that's fine, we'll keep release26-maint chugging along with 4.7, but we can switch trunk over to 4.8 once you're ready. By the time it's ready to cut 2.7, who knows, trunk's bsddb may be supporting Berkeley DB 5.2 at that stage. I would also think that you could cut independent releases (in the sense that they're not linked to any Python release schedule) of 'pybsddb' from the code living in trunk/Lib/bsddb. That way, users of 2.6 could still easy_install (or whatever) the latest pybsddb 4.8.0 to pick up the newest features. Trent. From greg at krypto.org Thu May 22 08:06:22 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 21 May 2008 23:06:22 -0700 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <6167796BFEB5D0438720AC212E89A6B0078657D9@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es> <6167796BFEB5D0438720AC212E89A6B0078657D9@exchange.onresolve.com> Message-ID: <52dc1c820805212306k41c34f18w194f1a7f5a99926b@mail.gmail.com> On Wed, May 21, 2008 at 5:12 PM, Trent Nelson wrote: >> Trent Nelson wrote: >> | Gah. I just went and visited the Berkeley DB download site as >> | I was preparing my commit message so I could refer to the >> | exact .tar.gz being imported, only to notice that the latest >> | version is now 4.7.25. Jesus, can we use this version? >> >> Err.... No. >> >> It is not clear to me that python 2.6/3.0 will be published with BDB 4.6 >> or 4.7 support. 4.6 has several known issues, apparently solved in 4.7. > > I could have sworn I heard a few people mention that "4.5 has issues, > but they're solved in 4.6" at PyCon ;-) The line I was giving at PyCon was "4.6 has issues, we might want to stick with 4.5" which is why I left setup.py limiting itself to (4, 5) despite the code being compatible with 4.6. Maybe 4.7 fixes the 4.6 issues; that was a quick release cycle. I haven't given it a whirl at all. I'll try to do that this weekend. But really all I'm doing with it at this point is running the bsddb.db test suite on OS X and Linux and if my build environment is working and I might try that as well. If it passes the entire test suite several times on windows x86 and amd64 builds when built with BerkeleyDB 4.7 (once Jesus commits the support code) I'm inclined to say we should ship with it and only revert back to 4.5 if someone has problems with it during a beta. >> I keep a private branch in pybsddb for BDB 4.7 support, waiting for >> Oracle publication. Since they already pushed 4.7.25 out (no pre warning >> for bindings developers, too bad!), I think I can publish pybsddb 4.7.0 >> in a couple of days. That done, I will update python version. Since 4.7.x is apparently out I suggest going ahead and committing the changes necessary to allow it to build and link against 4.7 regardless of if thats the version we use. No sense in holding back support for compiling. -gps From martin at v.loewis.de Thu May 22 09:08:18 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 09:08:18 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <4834709D.7030604@v.loewis.de> Message-ID: <48351BE2.8010501@v.loewis.de> >> Writing portable ctypes modules is really hard - significantly harder >> than writing portable C code (although writing non-portable ctypes >> code is apparently easier than writing non-portable C code). > > I would say that writing portable C code is hard as well, aren't there > just more tools that help? The C compiler in particular. It already gets symbolic constants and struct layouts right, something that ctypes can't do (because it doesn't use header files). Regards, Martin From adlaiff6 at gmail.com Thu May 22 09:21:55 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Thu, 22 May 2008 03:21:55 -0400 (EDT) Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <48351BE2.8010501@v.loewis.de> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <52dc1c820805161552u68a5d7e4v71259a0f5b691004@mail.gmail.com> <4831425F.2050102@denviso.de> <4833D196.7050205@denviso.de> <4834709D.7030604@v.loewis.de> <48351BE2.8010501@v.loewis.de> Message-ID: On Thu, 22 May 2008, "Martin v. L?wis" wrote: > > I would say that writing portable C code is hard as well, aren't there > > just more tools that help? > > The C compiler in particular. It already gets symbolic constants and > struct layouts right, something that ctypes can't do (because it doesn't > use header files). I don't think it makes much sense to talk about "The C Compiler" when you are discussing matters of portability. Just my $0.02. -- Cheers, Leif From michal at yogi.htu.tuwien.ac.at Thu May 22 13:45:04 2008 From: michal at yogi.htu.tuwien.ac.at (Michal Revucky) Date: Thu, 22 May 2008 13:45:04 +0200 Subject: [Python-Dev] vmgen based python interpretter Message-ID: <20080522114504.GB6135@yogi> hello everyone, i'm working on the branch: http://svn.python.org/projects/python/branches/release25-maint/ i'm trying to build a vmgen based interpreter for python: http://www.complang.tuwien.ac.at/anton/vmgen from vmgen's input file, which specifies the python's opcodes a c source is generated which may be included in ceval.c. when i include that generated source in ceval.c the whole interpreter crashes with this error: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1209710912 (LWP 28223)] PyObject_Malloc (nbytes=32) at Objects/obmalloc.c:747 747 if ((pool->freeblock = *(block **)bp) != NULL) { (gdb) so far i implemented only 4 opcodes (LOAD_CONST, PRINT_ITEM, PRINT_NEWLINE, RETURN_VALUE) on the other hand when i leave these 4 opcodes, as provided by ceval.c within the switch(opcode) statement every thing works well, the interpreter exits with an uknown opcode error, when an unknown opcode is executed. my question is, why does this segfault appear? as far as i know the PyObject_Malloc is only called by the compiler related code, and i changed code only in ceval.c it would would also be helpfull if someone could tell me, how to locate the code, which causes this SEG fault hopefully somebody can help me with that -- michal From ncoghlan at gmail.com Thu May 22 15:47:24 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 22 May 2008 23:47:24 +1000 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: References: <4832E144.8030309@gmail.com> <4833650E.9080204@canterbury.ac.nz> <6244070B-03CA-4B1F-AFFD-D948268F627B@acm.org> <4833EE56.5010705@gmail.com> Message-ID: <4835796C.6080101@gmail.com> Fred Drake wrote: > On May 21, 2008, at 5:41 AM, Nick Coghlan wrote: >> While a proxy class written in C would no doubt be faster than one >> written in Python, one of the things I'm hoping to achieve is for the >> stdlib generic proxy to serve as an example for people writing their >> own new-style proxy classes in addition to being usable as a base >> class (Adam Olsen suggested ProxyMixin instead of ProxyBase as a >> possible name). > > > The idea that it would serve as an example seems odd; the reason to make > things part of the standard library is because their implementation > needs to track the Python core closely. For a proxy, it would be the > need to reflect additional slot methods as they're added. A Python > implementation may be able to do this just fine, but the performance > penalty would make it uninteresting for many large applications. I've added a documentation file to the tracker item and kicked it in Barry's direction (I also bumped the priority to 'release blocker' because I think we need an explicit decision on this one from Barry or Guido before the first beta release of 3.0). Here's what I included in the proposed documentation for ProxyMixin (note that I modelled the implicit unwrapping behaviour directly on the behaviour of weakref.proxy): class:: ProxyMixin(target) A proxy class that ensures all special method invocations which may otherwise bypass the normal :method:`__getattribute__` lookup process are correctly delegated to the specified target object. Normal attribute manipulation operations are also delegated to the specified target object. All operations on a :class:`ProxyMixin` instance return an unproxied result. Operations involving multiple :class:`ProxyMixin` instances (e.g. addition) are permitted, and endeavour to return the same result as would be calculated if the proxy objects were not involved. Custom proxy class implementations may inherit from this class in order to automatically delegate all such special methods that the custom proxy class does not need to provide special handling for. This may not be practical for more complex delegation scenarios (e.g. a local interface to a remote object, or delegating to a weakly referenced target as in :code:`weakref.proxy`), but :class:`ProxyMixin` may still be used as a reference when developing such classes to identify which methods must be defined explicitly on the proxy type in order for them to be delegated correctly by the Python interpreter. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tnelson at onresolve.com Thu May 22 15:49:23 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Thu, 22 May 2008 14:49:23 +0100 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <483471F3.2030305@v.loewis.de> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es>,<483471F3.2030305@v.loewis.de> Message-ID: <6167796BFEB5D0438720AC212E89A6B0078657DA@exchange.onresolve.com> > > In any case, what would be the procedure to update the buildbot > > infraestructure?. > You need to import the sources into the Python subversion repository; > I understand Trent was about to do that. Indeed. Where do we stand with regards to svn commit e-mails being sent for 'external'? DB 4.6 is ~16MB... Trent. From facundobatista at gmail.com Thu May 22 17:28:47 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 22 May 2008 12:28:47 -0300 Subject: [Python-Dev] Slice as a copy... by design? Message-ID: Hi! A thread in PyAr raised the question that, considering that strings are immutable, why a slice of a string is a copy and not a reference to a part of that string. I couldn't answer why, so I'm asking here...Is it because the reference counting will be complicated? Is it because it'd be inefficient in other way? It's something else? Or is something that could be done... but is not done yet? Thank you very much! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From phd at phd.pp.ru Thu May 22 17:54:12 2008 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu, 22 May 2008 19:54:12 +0400 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <20080522155412.GB3325@phd.pp.ru> On Thu, May 22, 2008 at 12:28:47PM -0300, Facundo Batista wrote: > considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. I remember some discussions... let me see... google to help... aha: http://mail.python.org/pipermail/python-3000/2006-August/003224.html http://mail.python.org/pipermail/python-3000/2006-August/003242.html 2006, August... I don't remember what was the resolution of the discussion. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From gherron at islandtraining.com Thu May 22 17:35:35 2008 From: gherron at islandtraining.com (Gary Herron) Date: Thu, 22 May 2008 08:35:35 -0700 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <483592C7.20402@islandtraining.com> Facundo Batista wrote: > Hi! > > A thread in PyAr raised the question that, considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. > > I couldn't answer why, so I'm asking here...Is it because the > reference counting will be complicated? Is it because it'd be > inefficient in other way? It's something else? Or is something that > could be done... but is not done yet? > > Thank you very much! > > In fact, a slice is *not* always a copy! In at least some (simple) cases, a slice references the original string: >>> s = 'abc' >>> t = s[:] >>> s is t True >>> id(s) 3081872000L >>> id(t) 3081872000L Gary Herron From scott+python-dev at scottdial.com Thu May 22 18:22:53 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Thu, 22 May 2008 12:22:53 -0400 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <48359DDD.5000501@scottdial.com> Facundo Batista wrote: > I couldn't answer why, so I'm asking here...Is it because the > reference counting will be complicated? Is it because it'd be > inefficient in other way? It's something else? Or is something that > could be done... but is not done yet? If we changed Python to slice-by-reference, then tomorrow someone would be asking why it isn't slice-by-copy. There are pros and cons to both that are dependent on your application. It's not hard to imagine applications where you want to hold onto a small portion of a large string, thereby forcing the entire string to remain in memory. If a slices had a copy method, then I suppose this would be moot. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From facundobatista at gmail.com Thu May 22 18:24:06 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 22 May 2008 13:24:06 -0300 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: <20080522155412.GB3325@phd.pp.ru> References: <20080522155412.GB3325@phd.pp.ru> Message-ID: 2008/5/22 Oleg Broytmann : > I remember some discussions... let me see... google to help... aha: > > http://mail.python.org/pipermail/python-3000/2006-August/003224.html > http://mail.python.org/pipermail/python-3000/2006-August/003242.html These descussions are too general, and implies some work regarding the unification between slice and range, and discusses sequences in general... I'm just asking about strings (the discussion could apply similarly to bytes, but not bytearrays), and why slicing them returns a copy. Of course that when the slice has the step component different that 1, or reverses the string, this could not be done. But when the slice is mystring[x:y], being x < y, we *could* return just a view of the string (llot of emphasis to that "could"). Thanks. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ijmorlan at cs.uwaterloo.ca Thu May 22 18:26:51 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Thu, 22 May 2008 12:26:51 -0400 (EDT) Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: <483592C7.20402@islandtraining.com> References: <483592C7.20402@islandtraining.com> Message-ID: On Thu, 22 May 2008, Gary Herron wrote: > In fact, a slice is *not* always a copy! In at least some (simple) cases, a > slice references the original string: > >>>> s = 'abc' >>>> t = s[:] >>>> s is t > True >>>> id(s) > 3081872000L >>>> id(t) > 3081872000L I think the more interesting case is where the string objects are not the same object but use (portions of) the same underlying array in memory. If I understand correctly, Python does not do this, and I thought I read something about why not but I can't remember the details. Sharing contents is an obvious optimization which in some circumstances can dramatically reduce the amount of copying that goes on, but without a reasonably clever algorithm to decide when to let the underlying storage go (copying any part still in use), extremely bad behaviour can result - imagine reading in lots of long lines, then keeping just a short piece of each one. By contrast, the worst that can happen with no sharing is that performance and memory use is what you expect - the only "bad" is the apparent missed opportunity for optimization. I wonder if a "shared slice" object would be useful? That is, an object which behaves like a string obtained from a slicing operation except that it shares storage. It could have a .release method to go ahead and copy the underlying storage. One complexity comes to mind immediately - what happens if one takes a shared slice of a shared slice? Presumably it shares the original string's storage, but if the first shared slice is .released what happens to the second shared slice? It would be nice if it shared with the first shared slice, but keeping track of everything could get tricky. I'd be interested in pointers to any existing discussion on this issue. Trivia - right now there are *no* Google hits for 'python shared slice', although there are lots for 'python shared slices'. They don't appear to be talking about the same thing, however (without being exhaustive). Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From facundobatista at gmail.com Thu May 22 18:27:04 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 22 May 2008 13:27:04 -0300 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: <48359DDD.5000501@scottdial.com> References: <48359DDD.5000501@scottdial.com> Message-ID: 2008/5/22 Scott Dial : > If we changed Python to slice-by-reference, then tomorrow someone would be > asking why it isn't slice-by-copy. There are pros and cons to both that are Which are the cons of slice-by-reference of an immutable string? > dependent on your application. It's not hard to imagine applications where > you want to hold onto a small portion of a large string, thereby forcing the > entire string to remain in memory. If a slices had a copy method, then I This is a garbage collection issue. It's real, and maybe could be optimized somehow... but I think that this un-optimization is by far smaller than the optimization of not copying it in the first place. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From facundobatista at gmail.com Thu May 22 18:34:56 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 22 May 2008 13:34:56 -0300 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: <483592C7.20402@islandtraining.com> Message-ID: 2008/5/22 Isaac Morland : > By contrast, the worst that can happen with no sharing is that performance > and memory use is what you expect - the only "bad" is the apparent missed > opportunity for optimization. Exactly, "apparent". Also, this could be handled like a "good writing tip". For example, right now everybody knows that appending a letter to a string a zillion times is not efficient, you should store them in a list, and then .join() them. Similarly, we could know that slicing zillions of long lines and keeping small portion of them is not memory efficient, you should do everytime "shortstring = str(longstring[:2])", for example. Note that this "special coding" will be for an "special case"... in your normal life the code just will be more efficient... Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From jcea at jcea.es Thu May 22 18:40:09 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 22 May 2008 18:40:09 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <6167796BFEB5D0438720AC212E89A6B0078657DA@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es>, <483471F3.2030305@v.loewis.de> <6167796BFEB5D0438720AC212E89A6B0078657DA@exchange.onresolve.com> Message-ID: <4835A1E9.8090406@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: | Indeed. Where do we stand with regards to svn commit e-mails | being sent for 'external'? DB 4.6 is ~16MB... I just committed bsddb3 4.7.0. Please install Berkeley DB >=4.7.25. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDWh45lgi5GaxT1NAQLorQQAh3t/waArJtE94d3Fk4YbbxdO9i4+GmMO /xqcT1iHr+qg034LUYxVfWhGdjnbqNj2ZcTjWXEBVqgIqVT5yCYzbDrh0bKmROC0 QjQ0WaMJeMvqj1mGhmDCD3+QAYirUkEg3kAO8icrI9SkTedM6m4EMoT+GI2N6JSI k3SjvOPTKFw= =k2lF -----END PGP SIGNATURE----- From aahz at pythoncraft.com Thu May 22 19:01:44 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 22 May 2008 10:01:44 -0700 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <20080522170144.GA19714@panix.com> On Thu, May 22, 2008, Facundo Batista wrote: > > A thread in PyAr raised the question that, considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. Someone did a patch for this at one point, but I don't remember what happened. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From lists at cheimes.de Thu May 22 19:16:14 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 22 May 2008 19:16:14 +0200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: Facundo Batista schrieb: > Hi! > > A thread in PyAr raised the question that, considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. Because the reference approach is more complicated, harder to implement and may lead to unexpected behavior. For example: >>> a = "a long string with 500,000 chars ..." >>> b = a[0] >>> del a With the slice-as-copy design the string 'a' is immediately freed. The slice-as-reference design would keep the 500kB string in memory although you are only interested in the first character. The buffer interface was designed for the slice-as-copy use case: >>> a = "abcdefg" >>> b = buffer(a, 2, 3) >>> b >>> str(b) 'cde' >>> sys.getrefcount(a) 3 >>> del b >>> sys.getrefcount(a) 2 Christian From guido at python.org Thu May 22 19:37:06 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 22 May 2008 10:37:06 -0700 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: <20080522170144.GA19714@panix.com> References: <20080522170144.GA19714@panix.com> Message-ID: On Thu, May 22, 2008 at 10:01 AM, Aahz wrote: > On Thu, May 22, 2008, Facundo Batista wrote: >> >> A thread in PyAr raised the question that, considering that strings >> are immutable, why a slice of a string is a copy and not a reference >> to a part of that string. > > Someone did a patch for this at one point, but I don't remember what > happened. I rejected it because I came up with a worst-case scenario where the behavior with the patch in place was much, much worse than the behavior without it -- improved performance in other scenarios notwithstanding. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcea at jcea.es Thu May 22 20:04:35 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 22 May 2008 20:04:35 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <6167796BFEB5D0438720AC212E89A6B0078657D9@exchange.onresolve.com> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com>, <48346695.4030906@jcea.es> <6167796BFEB5D0438720AC212E89A6B0078657D9@exchange.onresolve.com> Message-ID: <4835B5B3.2010009@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: |> It is not clear to me that python 2.6/3.0 will be published with BDB 4.6 |> or 4.7 support. 4.6 has several known issues, apparently solved in 4.7. | | I could have sworn I heard a few people mention that "4.5 has issues, | but they're solved in 4.6" at PyCon ;-) Or course, it is natural to expect new releases to solve known issues. But initial releases of BDB 4.6 were crappy (current ones are fairly good, even if they didn't patch the "compact()" crash :). I glad to say than pybsddb3 testsuite has catched a few errors in Oracle BDB 4.7 pre-releases, so I'm fairly confident in the final version. | I think I remember reading some concerns you had about doing this last | week though, surrounding things like wanting to be able to release pybsddb | versions more frequently than Python versions are released. Just because | you live in trunk/Lib/bsddb shouldn't prevent you from doing this though; | in fact, as long as you're sensitive to major release timeframes, I'm | sure we'd love trunk to always track the latest Berkeley DB version; This issue is complex and I would like to delay a hard decision until it becomes a real issue. Currently, I rather prefer to keep them apart (but in sync). Aside the frequent release issue (something that would not be an issue when pybsddb3 cover the entire Berkeley DB API, some day), there are mailing lists, community support, open patch submission, etc. I know that merging can be a bit painful. Been there, done that. I'm waiting SVN 1.5 release and its much improved merge repeated merge, like the holy grial :-p. Someday Mercurial could be integrated in Python version control system, and I would sleep better at night. Meanwhile, bsddb patches in the python side are few and apart, and I can cope with them. I want to shield python-dev of this: """ [jcea at tesalia trunk]$ svn diff svn://svn.argo.es/jcea/pybsddb/tags/4.7.0 svn://svn.argo.es/jcea/pybsddb/trunk | wc ~ 1275 3554 37696 """ About ten big commits. And 4.7.0 was released mere hours ago... | if | all the buildbots stay green with 4.7 and there are no regressions in | functionality, then hey, lets go with 4.7 for 2.6/3.0. I will write about this, after you upgrade MS Windows buildbots to 4.7.25. Let's see how it behave. I rather prefer BDB 4.7.25 in Py2.6/3.0 than BDB 4.6.21, unless we discover some ugly bug in last release (as was discovered after 4.6 release; gracefully no python version was released using the faulty lib). I'm using this 4.7.0/4.7.25 release for... uhmm... 20 hours now. No issues so far, in a very BDB busy machine (mailserver, storing user mailboxes via Python->Durus->my durus backend for Berkeley DB->pybsddb3->BDB 4.7.25). A couple of millions emails per day :). | If Oracle come out with Berkeley DB 4.8 a week after 2.6 is released, | that's fine, we'll keep release26-maint chugging along with 4.7, but we | can switch trunk over to 4.8 once you're ready. One of the "problems" with Berkeley DB is that upgrading libraries can require "upgrade" also the database files. The "upgrade" is usually painless, but you can't go back. So, any BDB upgrade in a python release must be clearly marked, because it can become a no-return-point for some users. In Unix world, Python releases would use BDB installed in the system. So, the user/admin has control over BDB upgrades. But since MS Windows Python bundles BDB, we need to be extra-careful there. Someday we need to talk about pybsddb3 "eggs" (via pypi) supporting BDB DLL bundled by stock python in MS Windows. Current approach needs a new DLL. Some *other* day. So much to do, so little time... :). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDW1rplgi5GaxT1NAQJEpQP+IFcmL+0zqg1n8MrtyvPiRZk4oBSlwUzD Ov0SHVmx8MmhKk5NOI3FjpKSVZIVU55HZ+qFBuqz12VeYOucZOUeDlsE5LFnmXsC E4HR6+NCi7zHSOyESjy6j6M3rNPWqNiPOUnJ5kevDOcnoA+W8urdj97wHflArsJ3 e1M1R04UoMI= =lisl -----END PGP SIGNATURE----- From ijmorlan at cs.uwaterloo.ca Thu May 22 20:23:10 2008 From: ijmorlan at cs.uwaterloo.ca (Isaac Morland) Date: Thu, 22 May 2008 14:23:10 -0400 (EDT) Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: On Thu, 22 May 2008, Christian Heimes wrote: > The buffer interface was designed for the slice-as-copy use case: > >>>> a = "abcdefg" >>>> b = buffer(a, 2, 3) >>>> b > >>>> str(b) > 'cde' [....] This answers my musing about shared slices. But it points me at another question: why is buffer() listed in "Non-essential Built-in Functions"? While it is obviously not essential like str() or list(), it isn't deprecated like apply(). On the other hand, some other built-in functions listed under "Built-in Functions" are probably also not essential (I'm not going to go any further out on the limb by giving an example!). Perhaps I misunderstand the intent of this manual page. http://docs.python.org/lib/non-essential-built-in-funcs.html#l2h-88 Isaac Morland CSCF Web Guru DC 2554C, x36650 WWW Software Specialist From jcea at jcea.es Thu May 22 20:30:07 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 22 May 2008 20:30:07 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <52dc1c820805212306k41c34f18w194f1a7f5a99926b@mail.gmail.com> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es> <6167796BFEB5D0438720AC212E89A6B0078657D9@exchange.onresolve.com> <52dc1c820805212306k41c34f18w194f1a7f5a99926b@mail.gmail.com> Message-ID: <4835BBAF.1000507@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gregory P. Smith wrote: | On Wed, May 21, 2008 at 5:12 PM, Trent Nelson wrote: |>> Trent Nelson wrote: |>> | Gah. I just went and visited the Berkeley DB download site as |>> | I was preparing my commit message so I could refer to the |>> | exact .tar.gz being imported, only to notice that the latest |>> | version is now 4.7.25. Jesus, can we use this version? |>> |>> Err.... No. |>> |>> It is not clear to me that python 2.6/3.0 will be published with BDB 4.6 |>> or 4.7 support. 4.6 has several known issues, apparently solved in 4.7. |> I could have sworn I heard a few people mention that "4.5 has issues, |> but they're solved in 4.6" at PyCon ;-) | | The line I was giving at PyCon was "4.6 has issues, we might want to | stick with 4.5" which is why I left setup.py limiting itself to (4, 5) | despite the code being compatible with 4.6. I guess we need first all buildbots updated to 4.7.25 before changing the setup.py to support BDB 4.7, to avoid no updated ones linking against (old) buggy 4.6 releases. Reading the buildbot outputs, I see some machines using "ancient" BDB versions. For example, "amd64 gentoo trunk" uses 4.4.x. Would be nice to know the exact release :-). This is interesting, because a buildbot using a DBD < 4.7 will not exercise the module at full capacity, so buildbot value is, somewhat, reduced. For example, testsuite against a 4.0 BDB has 270 tests. Against 4.6, it has 292 test. Now that 4.7.25 is published, pybsddb 4.7.x testsuite will get even bigger, but only when testing against a 4.7 installation!. I would hate to find problems in Python beta releases because the buildbots are outdated :( | If it passes the entire test suite | several times on windows x86 and amd64 builds when built with | BerkeleyDB 4.7 (once Jesus commits the support code) I'm inclined to | say we should ship with it and only revert back to 4.5 if someone has | problems with it during a beta. :-). - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDW7qplgi5GaxT1NAQLWrwQAk9Xxm5tCB/p4dcCnoWQj3H1rvpJs+c9s g8m/g6+JL8rhlUWixEYApsK53komCGzSC5Ghf91c2QuIQugeGFA4dE3VcYGO89J+ GgvCw+YnJ3KG3DQ2Qw9Lus4LFuDS67bPlSTALQYuLk32ks7FTcTkhNO5qs5JTejs tpQde3MBlLw= =p2qf -----END PGP SIGNATURE----- From jcea at jcea.es Thu May 22 20:31:24 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 22 May 2008 20:31:24 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <483471F3.2030305@v.loewis.de> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es> <483471F3.2030305@v.loewis.de> Message-ID: <4835BBFC.5070705@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: |> In any case, what would be the procedure to update the buildbot |> infraestructure?. | | You need to import the sources into the Python subversion repository; | I understand Trent was about to do that. | | Then you need to adjust Tools/msi/external.bat to have the slaves export | the new version. I was thinking about the Unix buildbots. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDW7/Jlgi5GaxT1NAQL88AP+MXziTzh5wnF6oyjmZtAk9/75LVzHwcjU QyY/MF/DPsL9qnI81Lg7awl4adFaSmntzGX7AAh/TojI40TuW3LI53zFfMAUGsBB q8YaY6zswHENABry+NI8X/mo7J/VeCJstPNANsu1C6o7GSIVH7e6QhzDeOLgsp+O Ae3a1RE+9AU= =m5VB -----END PGP SIGNATURE----- From stefan_ml at behnel.de Thu May 22 20:39:06 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 22 May 2008 20:39:06 +0200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: Isaac Morland wrote: > On Thu, 22 May 2008, Christian Heimes wrote: > >> The buffer interface was designed for the slice-as-copy use case: >> >>>>> a = "abcdefg" >>>>> b = buffer(a, 2, 3) >>>>> b >> >>>>> str(b) >> 'cde' > [....] > > This answers my musing about shared slices. But it points me at another > question: why is buffer() listed in "Non-essential Built-in Functions"? > While it is obviously not essential like str() or list(), it isn't > deprecated like apply(). Even worse, it's gone in Py3: Python 3.0a5 (r30a5:62856, May 9 2008, 11:26:14) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> buffer Traceback (most recent call last): File "", line 1, in NameError: name 'buffer' is not defined Stefan From jcea at jcea.es Thu May 22 20:39:24 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 22 May 2008 20:39:24 +0200 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: <52dc1c820805132133s1ad0be0em2d7c25296fece78c@mail.gmail.com> References: <4829E787.5080605@jcea.es> <52dc1c820805132133s1ad0be0em2d7c25296fece78c@mail.gmail.com> Message-ID: <4835BDDC.9040709@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Gregory P. Smith wrote: | btw, I fixed the Lib/test/test_bsddb3.py file for the updated | Lib/bsddb/test/ modules. Thats how the test suite gets run by the | buildbots (run the test suite from a python trunk sandbox using | "./python Lib/test/regrtest.py -v -u bsddb test_bsddb test_bsddb3" to | reproduce exactly how it is run yourself). What about changing that file to read the testsuite from "Lib/bsddb/test", directly, to avoid skipping new tests again?. I would like to print the Berkeley DB release used, also. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDW93Jlgi5GaxT1NAQKUcwP/ZOu3jOGEpGHmDGZYZbChowHnDYnpwh0E DgVbHnP3hWRneo5UESNSg6yu481jJ4eLHF662zAYbLlJx1J15yF8jMXHxWuW4fH1 QNVU4DwtIVGU9FdAKHcDqKglgvTtKy/0rzDPBcjkgmbjsM9MkxtOu8lZjBw1RgaB T4ojB+pr7Ps= =JU9d -----END PGP SIGNATURE----- From lists at cheimes.de Thu May 22 20:52:54 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 22 May 2008 20:52:54 +0200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <4835C106.20001@cheimes.de> Stefan Behnel schrieb: > Even worse, it's gone in Py3: No, it has been replaced by a better system. Try "memoryview" Christian From stefan_ml at behnel.de Thu May 22 21:02:32 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 22 May 2008 21:02:32 +0200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: <4835C106.20001@cheimes.de> References: <4835C106.20001@cheimes.de> Message-ID: Hi, Christian Heimes wrote: > Stefan Behnel wrote: >> Even worse, it's gone in Py3: > > No, it has been replaced by a better system. > > Try "memoryview" I know. We are already discussing the buffer protocol on the Cython list. http://comments.gmane.org/gmane.comp.python.cython.devel/1763?set_lines=100000 Stefan From lists at cheimes.de Thu May 22 20:52:54 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 22 May 2008 20:52:54 +0200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <4835C106.20001@cheimes.de> Stefan Behnel schrieb: > Even worse, it's gone in Py3: No, it has been replaced by a better system. Try "memoryview" Christian From martin at v.loewis.de Thu May 22 22:09:38 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 22:09:38 +0200 Subject: [Python-Dev] vmgen based python interpretter In-Reply-To: <20080522114504.GB6135@yogi> References: <20080522114504.GB6135@yogi> Message-ID: <4835D302.4000605@v.loewis.de> > it would would also be helpfull if someone could tell me, how to locate the > code, which causes this SEG fault I think you already found the code; gdb tells you the exact location of the crash. What are the values of pool and bp? Regards, Martin From martin at v.loewis.de Thu May 22 22:17:45 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 22:17:45 +0200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <4835D4E9.1060005@v.loewis.de> > I couldn't answer why, so I'm asking here...Is it because the > reference counting will be complicated? Is it because it'd be > inefficient in other way? It's something else? Or is something that > could be done... but is not done yet? There are two problems with that approach: a) you may hold onto very long strings for a long time, even though you don't "see" the larger string anymore, e.g. when you have only a single line of an entire file that you hold onto. That, in turn, may cause the program to consume much more memory than the copying slicing. b) it is very difficult to implement and maintain. At any point in the code that deals with string representations, you have to make the distinction of whether you have a "real" string, or a slice. Nobody has contributed a correct and efficient implementation of that yet, let alone one that is also easy to maintain. c) (related to b) Currently, the string object's internal representation might be used in extension modules. We would need to find a way to protect against errors that may occur when we silently change the representation semantically. (as any good list of two items, this has three :-) Regards, Martin From martin at v.loewis.de Thu May 22 22:23:09 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 22:23:09 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <4835BBAF.1000507@jcea.es> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es> <6167796BFEB5D0438720AC212E89A6B0078657D9@exchange.onresolve.com> <52dc1c820805212306k41c34f18w194f1a7f5a99926b@mail.gmail.com> <4835BBAF.1000507@jcea.es> Message-ID: <4835D62D.4090606@v.loewis.de> > I guess we need first all buildbots updated to 4.7.25 before changing > the setup.py to support BDB 4.7, to avoid no updated ones linking > against (old) buggy 4.6 releases. Why do you think so? Updating all buildbots is not feasible, atleast not within this year. Some machines *intentionally* stay at old software, just to make sure Python continues to build. Would pybsddb 4.7 continue to build with all bsddb releases that it currently builds with? > Reading the buildbot outputs, I see some machines using "ancient" BDB > versions. For example, "amd64 gentoo trunk" uses 4.4.x. Would be nice to > know the exact release :-). Feel free to temporarily commit code to make it find out - either directly in the trunk, or by making a temporary branch and have that branch be built on the particular slave only. > This is interesting, because a buildbot using a DBD < 4.7 will not > exercise the module at full capacity, so buildbot value is, somewhat, > reduced. That's a matter of standpoint. It also tests whether the module works and passes its test suite with older releases, which is a good thing. So it's more valuable to have old releases on some slaves, rather than having them test all the same version (which is known to work in principle, anyway). > I would hate to find problems in Python beta releases because the > buildbots are outdated :( Don't worry, some slaves will surely have the latest version, at some point (if none else, the Windows ones will). Regards, Martin From martin at v.loewis.de Thu May 22 22:24:01 2008 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 22:24:01 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <4835BBFC.5070705@jcea.es> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es> <483471F3.2030305@v.loewis.de> <4835BBFC.5070705@jcea.es> Message-ID: <4835D661.5020204@v.loewis.de> > |> In any case, what would be the procedure to update the buildbot > |> infraestructure?. > | > | You need to import the sources into the Python subversion repository; > | I understand Trent was about to do that. > | > | Then you need to adjust Tools/msi/external.bat to have the slaves export > | the new version. > > I was thinking about the Unix buildbots. Ah. There is no way to upgrade them, and attempts to upgrade them should not be made (see my other message) Regards, Martin From martin at v.loewis.de Thu May 22 22:32:44 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 22:32:44 +0200 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: <4835BDDC.9040709@jcea.es> References: <4829E787.5080605@jcea.es> <52dc1c820805132133s1ad0be0em2d7c25296fece78c@mail.gmail.com> <4835BDDC.9040709@jcea.es> Message-ID: <4835D86C.6030406@v.loewis.de> > I would like to print the Berkeley DB release used, also. Could that be done at build time as well? Martin From p.f.moore at gmail.com Thu May 22 22:46:21 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 22 May 2008 21:46:21 +0100 Subject: [Python-Dev] Trying to understand svn history (was: Buildbots have trouble checking out the repository due to recent changes.) Message-ID: <79990c6b0805221346o1108b9fckfc8e5bd6c514333e@mail.gmail.com> 2008/5/12 Paul Moore : > Revision 63129 is not valid on case folding filesystems. In > particular, this horribly breaks using hg-svn to make a local mirror > of the Python repository: I'm still trying to identify what breaks in converting svn to Mercurial at around the time of your library renamings. Specifically, I cannot convert the svn repository history for r63104: >svn log -r63104 ------------------------------------------------------------------------ r63104 | alexandre.vassalotti | 2008-05-12 00:04:27 +0100 (Mon, 12 May 2008) | 2 lines Moved the Queue module stub in lib-old. >From what I can see of the changes, you simply added a Lib/lib-old/Queue.py in that revision - Lib/queue.py was already there. But your description "Moved..." implies you did something slightly different (and the text of your original message on this thread suggests that you might have done a svn copy Lib/queue.py Lib/lib-old/Queue.py followed by an edit). I know it's a while ago but can you recall exactly what you did? Or can some svn guru tell me how to get svn to tell me exactly what operations it believes went on? This is something of a showstopper for converting Python svn->hg on Windows, and I'd like to identify and if at all possible fix the bug. Thanks, Paul From martin at v.loewis.de Thu May 22 23:06:37 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 23:06:37 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <08May21.122939pdt."58696"@synergy1.parc.xerox.com> References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> <20080514105858.0279814C64B@irishsea.home.craig-wood.com> <482D469E.7080609@denviso.de> <482D8FA4.7040509@gmail.com> <483134B5.9000300@denviso.de> <483156D0.8010801@gmail.com> <08May19.100239pdt."58696"@synergy1.parc.xerox.com> <20080519203820.GA12661@amk-desktop.matrixgroup.net> <08May21.122939pdt."58696"@synergy1.parc.xerox.com> Message-ID: <4835E05D.3000904@v.loewis.de> > % make > cc -c -DNDEBUG -O -I. -IInclude -I./Include -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c > "/usr/include/sys/feature_tests.h", line 353: #error: "Compiler or > options invalid for pre-UNIX 03 X/Open applications and pre-2001 > POSIX applications" > cc: acomp failed for ./Modules/python.c which cc? It works fine for me, with cc: Sun C 5.9 SunOS_sparc Patch 124867-01 2007/07/12 (it just complains about not supporting -OPT:Olimit) > *** Error code 2 > make: Fatal error: Command failed for target `Modules/python.o' > % > > So maybe Python just doesn't run on Solaris with the Sun C compiler. > Certainly doesn't build out of the box. Apparently, there were some intermediate releases of SunPRO which had this problem; in these cases, setting -xc99=%none may help. Regards, Martin From mal at egenix.com Thu May 22 23:10:09 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 22 May 2008 23:10:09 +0200 Subject: [Python-Dev] buffer interface for C extensions In-Reply-To: References: Message-ID: <4835E131.9090101@egenix.com> On 2008-05-19 00:59, Dan Lenski wrote: > Hi all, > > I've written a small C extension to submit commands to SCSI devices via > Linux's sg_io driver (for a camera hacking project). The extension is > just a wrapper around a couple ioctl()'s with Pythonic exception handling > thrown in. One of my extension methods is called like this from python: > > sg.write(fd, command[, data, timeout) > > Both command and data are binary strings. I would like to be able to use > either a regular Python string or an array('B', ...) for these read-only > arguments. So I tried to use the "t#" argument specifier to indicate that > these arguments could be either strings or objects that implement the read- > only buffer interface: > > if (!PyArg_ParseTuple(args, "it#|t#i:write", &sg_fd, &cmd, > &cmdLen, &buf, &bufLen, &timeout)) > return NULL; > > Now, this works fine with strings, but when I call it with an array I get > a TypeError: > > TypeError: write() argument 2 must be string or read-only character > buffer, not array.array > > So, I then tried changing "t#" to "w#" to indicate that the arguments must > implement the /read-write/ buffer interface. Now the array objects work, > but when I try a string argument, I naturally get this error: > > TypeError: Cannot use string as modifiable buffer > > So here's what I don't understand. Why doesn't the "t#" argument > specifier support read-write buffers as well as read-only buffers? Aren't > read-write buffers a *superset* of read-only buffers?? Is there something > I'm doing wrong or a quick fix to get this to work appropriately? You should probably ask such questions on the capi-sig list. To answer your question: t# requires support for the read-only 8-bit character buffer interface s# can use the read buffer interface w# requires support for the write buffer interface Those are two different buffer interface slots, so whether a particular object works with t# or w# depends on whether it implements the slots in question. You should probably try s#, as this will also work with objects that implement the getreadbuffer slot. The details can be found in Python/getargs.c -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 22 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From p.f.moore at gmail.com Thu May 22 23:10:21 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 22 May 2008 22:10:21 +0100 Subject: [Python-Dev] Trying to understand svn history (was: Buildbots have trouble checking out the repository due to recent changes.) In-Reply-To: <79990c6b0805221346o1108b9fckfc8e5bd6c514333e@mail.gmail.com> References: <79990c6b0805221346o1108b9fckfc8e5bd6c514333e@mail.gmail.com> Message-ID: <79990c6b0805221410j2a88b8e9ud1c972c442a668a1@mail.gmail.com> 2008/5/22 Paul Moore : > I know it's a while ago but can you recall exactly what you did? Or > can some svn guru tell me how to get svn to tell me exactly what > operations it believes went on? Never mind, I found out how to do this using svn dump. Paul. From martin at v.loewis.de Thu May 22 23:17:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 May 2008 23:17:50 +0200 Subject: [Python-Dev] Trying to understand svn history (was: Buildbots have trouble checking out the repository due to recent changes.) In-Reply-To: <79990c6b0805221346o1108b9fckfc8e5bd6c514333e@mail.gmail.com> References: <79990c6b0805221346o1108b9fckfc8e5bd6c514333e@mail.gmail.com> Message-ID: <4835E2FE.7020603@v.loewis.de> >>From what I can see of the changes, you simply added a > Lib/lib-old/Queue.py in that revision - Lib/queue.py was already > there. But your description "Moved..." implies you did something > slightly different (and the text of your original message on this > thread suggests that you might have done a svn copy Lib/queue.py > Lib/lib-old/Queue.py followed by an edit). In this case, adding -v helps: Changed paths: D /python/trunk/Lib/Queue.py A /python/trunk/Lib/lib-old/Queue.py (from /python/trunk/Lib/Queue.py:63101) > I know it's a while ago but can you recall exactly what you did? Or > can some svn guru tell me how to get svn to tell me exactly what > operations it believes went on? See above. If you want to know whether it was a mere move, or whether something was changed as well, try svn diff svn+ssh://pythondev at svn.python.org/python/trunk/Lib/lib-old/Queue.py at 63104 svn+ssh://pythondev at svn.python.org/python/trunk/Lib/Queue.py at 63103 which shows no differences. In fact, Queue.py got its current contents in r63077, the same version where queue.py was copied from Queue.py. HTH, Martin From dlenski at gmail.com Thu May 22 23:25:09 2008 From: dlenski at gmail.com (Dan Lenski) Date: Thu, 22 May 2008 21:25:09 +0000 (UTC) Subject: [Python-Dev] buffer interface for C extensions References: <4835E131.9090101@egenix.com> Message-ID: On Thu, 22 May 2008 23:10:09 +0200, M.-A. Lemburg wrote: > On 2008-05-19 00:59, Dan Lenski wrote: > You should probably ask such questions on the capi-sig list. > > To answer your question: > > t# requires support for the read-only 8-bit character buffer interface > s# can use the read buffer interface > w# requires support for the write buffer interface I somehow overlooked the fact that the "s#" argument specifier supports exactly the functionality that I need. D'oh! > Those are two different buffer interface slots, so whether a particular > object works with t# or w# depends on whether it implements the slots in > question. > > You should probably try s#, as this will also work with objects that > implement the getreadbuffer slot. Worked like a charm! > The details can be found in Python/getargs.c I'm still unclear on the difference between the character/read buffer interfaces, but I'll read the code and try and figure that out. Thanks for your help, Marc-Andre!!! Dan From mal at egenix.com Thu May 22 23:31:42 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 22 May 2008 23:31:42 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <4832035F.3010708@jcea.es> References: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> <4832035F.3010708@jcea.es> Message-ID: <4835E63E.8050404@egenix.com> On 2008-05-20 00:46, Jesus Cea wrote: > Trent Nelson wrote: > | I downloaded the source that includes AES encryption, for no reason > | other than it was first on the list. I'm now wondering if we should > | only be importing the 'NC' source that doesn't contain any > | encryption? Jesus, does pybsddb use any of the Berkeley DB > | encryption facilities? Would anything break if we built the > | bsddb module without encryption? > > Yes, pybsddb3 4.6.4 supports cryptography if the underlying Berkeley DB > library is crypto enabled. > > In principle, you can compile BDB without crypto, and pybsddb3 should > work, but you would lose ability to open any DB formerly created using > page encryption or page checksum. > > Export laws aside, we better compile with crypto :). I hope you're only talking about the Windows build... In any case, if you do include crypto code in the Windows installer, please make sure that the PSF is informed, so that the proper reporting procedure can be put in place (whatever it is nowadays in the US). The installer already includes the ssl module, so it's not problem to include crypto code in general. BTW: AFAIK the _ssl module is built against OpenSSL. Since I couldn't find any OpenSSL DLLs in my Python install dir and due to the size of the _ssl.pyd, I assume that it is statically linked against OpenSSL. That's all fine, but then I'm missing the OpenSSL license and attribution notice somewhere in the installer, the README of the installation or elsewhere. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 22 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From greg.ewing at canterbury.ac.nz Fri May 23 00:35:19 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 23 May 2008 10:35:19 +1200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <4835F527.6050808@canterbury.ac.nz> Facundo Batista wrote: > A thread in PyAr raised the question that, considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. Because it would make it too easy to accidentally keep a large string alive via a reference to a small part of it. Some way of explicitly requesting a view into another string might be desirable, but it shouldn't be the default behaviour for string slicing. -- Greg From janssen at parc.com Fri May 23 01:15:58 2008 From: janssen at parc.com (Bill Janssen) Date: Thu, 22 May 2008 16:15:58 PDT Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <4835E63E.8050404@egenix.com> References: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> <4832035F.3010708@jcea.es> <4835E63E.8050404@egenix.com> Message-ID: <08May22.161607pdt."58698"@synergy1.parc.xerox.com> > That's all fine, but then I'm missing the OpenSSL license and > attribution notice somewhere in the installer, the README of the > installation or elsewhere. Good point. We need this for both the ssl module and the hashlib module. Bill From facundobatista at gmail.com Fri May 23 02:07:01 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 22 May 2008 21:07:01 -0300 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: 2008/5/22 Facundo Batista : > I couldn't answer why, so I'm asking here...Is it because the > reference counting will be complicated? Is it because it'd be > inefficient in other way? It's something else? Or is something that > could be done... but is not done yet? Thank you all for the information provided! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ncoghlan at gmail.com Fri May 23 10:35:11 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 23 May 2008 18:35:11 +1000 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: <4835F527.6050808@canterbury.ac.nz> References: <4835F527.6050808@canterbury.ac.nz> Message-ID: <483681BF.4030701@gmail.com> Greg Ewing wrote: > Some way of explicitly requesting a view into another > string might be desirable, but it shouldn't be the > default behaviour for string slicing. Backporting 3.0's memoryview to 2.6 would be the way to go about that. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From g.brandl at gmx.net Fri May 23 11:53:59 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 23 May 2008 11:53:59 +0200 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: <4834AB80.3010208@gmail.com> References: <20080521195806.4714.55722582.divmod.quotient.1686@ohm> <0C8E8176-BBFC-438B-B1D9-539CBD7BFB59@fuhm.net> <9636FD7FF9B942759ED2E781B91324B7@RaymondLaptop1> <4834AB80.3010208@gmail.com> Message-ID: Nick Coghlan schrieb: > Alex Martelli wrote: >> On Wed, May 21, 2008 at 1:53 PM, Raymond Hettinger wrote: >>> Putting this functionality in 2.6/3.0 would provide a really >>> nice incentive to update from Py2.5. It would be a sad >>> lost opportunity if this module had to wait another couple years. >> >> I feel essentially the same way: it WOULD be sad to waste this >> excellent opportunity, so I second the suggestion to put pyprocessing >> in the library right now. > > Thirded (I think I'm contradicting what I posted earlier in the thread, > but I've had more of a chance to think about it and 18 months really is > quite a long time to wait for this functionality...) This seems to be enough support to put an entry into PEP 361, which I've just done. Personally, I'd also find it useful to have such a thing in the stdlib, since at the moment multiple processes are really the way to go for parallelism in Python, and Python needs to stress this fact. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From mal at egenix.com Fri May 23 13:23:23 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 23 May 2008 13:23:23 +0200 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <08May22.161607pdt."58698"@synergy1.parc.xerox.com> References: <6167796BFEB5D0438720AC212E89A6B0078657B5@exchange.onresolve.com> <4832035F.3010708@jcea.es> <4835E63E.8050404@egenix.com> <08May22.161607pdt."58698"@synergy1.parc.xerox.com> Message-ID: <4836A92B.5000208@egenix.com> On 2008-05-23 01:15, Bill Janssen wrote: >> That's all fine, but then I'm missing the OpenSSL license and >> attribution notice somewhere in the installer, the README of the >> installation or elsewhere. > > Good point. We need this for both the ssl module and the hashlib > module. FYI: I've opened ticket #2949 to track this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 23 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From tnelson at onresolve.com Fri May 23 13:50:24 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Fri, 23 May 2008 12:50:24 +0100 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? Message-ID: <6167796BFEB5D0438720AC212E89A6B0078657E2@exchange.onresolve.com> > I just committed bsddb3 4.7.0. Please install Berkeley DB >=4.7.25. I'll be working on this over the weekend. Trent. From status at bugs.python.org Fri May 23 18:10:06 2008 From: status at bugs.python.org (Tracker) Date: Fri, 23 May 2008 18:10:06 +0200 (CEST) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080523161007.5AF9F782B3@psf.upfronthosting.co.za> ACTIVITY SUMMARY (05/16/08 - 05/23/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1884 open (+40) / 12889 closed (+16) / 14773 total (+56) Open issues with patches: 559 Average duration of open issues: 702 days. Median duration of open issues: 1428 days. Open Issues Breakdown open 1861 (+40) pending 23 ( +0) Issues Created Or Reopened (61) _______________________________ Backport oct() and hex() to use __index__ 05/17/08 http://bugs.python.org/issue2337 reopened benjamin.peterson patch, 26backport Moving lib-tk to tkinter package 05/20/08 CLOSED http://bugs.python.org/issue2839 reopened brett.cannon patch Write UserDict fixer for 2to3 05/20/08 http://bugs.python.org/issue2876 reopened brett.cannon Rename repr to reprlib 05/20/08 CLOSED http://bugs.python.org/issue2880 reopened brett.cannon Create the html package 05/20/08 CLOSED http://bugs.python.org/issue2882 reopened brett.cannon Crash in ParseTupleAndKeywords when passing byte string keywords 05/16/08 CLOSED http://bugs.python.org/issue2895 created scoder cmathmodule.c compilation error 05/16/08 CLOSED http://bugs.python.org/issue2896 created Juno include structmember.h in Python.h 05/16/08 http://bugs.python.org/issue2897 created benjamin.peterson Add memory footprint query 05/17/08 http://bugs.python.org/issue2898 created okkoto patch Fixers find, rfind, etc in 'string' module 05/17/08 http://bugs.python.org/issue2899 created bhy math.copysign(1, float('nan')) 05/17/08 CLOSED http://bugs.python.org/issue2900 created christian.heimes "error: can't allocate region" from mmap() when receiving big ch 05/18/08 http://bugs.python.org/issue2901 created hwaara tkinter uses MacOS 05/18/08 http://bugs.python.org/issue2902 created benjamin.peterson Add __name__ in globals of generated namedtuple namespace 05/18/08 http://bugs.python.org/issue2903 created xilun patch Cross-referencing to a method using an alternate title don't wor 05/18/08 http://bugs.python.org/issue2904 created llucax test_urllib.py crashes with bus error on Darwin 05/18/08 CLOSED http://bugs.python.org/issue2905 created cartman tkinter, assorted fixes 05/18/08 http://bugs.python.org/issue2906 created gpolo patch, patch ZeroDivisionError error message needs clarification 05/18/08 CLOSED http://bugs.python.org/issue2907 created chester fix_imports new entries for tkinter rename 05/18/08 CLOSED http://bugs.python.org/issue2908 reopened gpolo patch struct.Struct.unpack to return a namedtuple for easier attribute 05/18/08 http://bugs.python.org/issue2909 created habnabit patch Remove plat-mac from 3.0 05/18/08 http://bugs.python.org/issue2910 created brett.cannon rewrite test_struct as a unittest 05/18/08 http://bugs.python.org/issue2911 created benjamin.peterson easy let platform.uname try harder 05/19/08 http://bugs.python.org/issue2912 created benjamin.peterson easy idlelib/EditorWindow.py uses xrange() 05/19/08 http://bugs.python.org/issue2913 created mark Logging TimedRotatingFileHandler Feature Request 05/19/08 CLOSED http://bugs.python.org/issue2914 created vschmidt PyObject_IsInstance() doesn't find bases named in type(name, bas 05/19/08 CLOSED http://bugs.python.org/issue2915 created scoder urlgrabber.grabber calls setdefaulttimeout 05/19/08 http://bugs.python.org/issue2916 created kiilerix merge pickle and cPickle in 3.0 05/19/08 http://bugs.python.org/issue2917 created brett.cannon Merge StringIO/cStringIO in 3.0 05/19/08 http://bugs.python.org/issue2918 created brett.cannon Merge profile/cProfile in 3.0 05/19/08 http://bugs.python.org/issue2919 created brett.cannon Patch to print symbolic value or errno in EnvironmentError.__str 05/19/08 http://bugs.python.org/issue2920 created ygingras patch enable embedding: declare/#define only py* symbols in #includes 05/19/08 http://bugs.python.org/issue2921 created hfuru "No windows home dir" doc error 05/19/08 http://bugs.python.org/issue2922 created python at johnburnett.com Revert ConfigParser rename in 2.6 05/20/08 http://bugs.python.org/issue2923 created brett.cannon Revert copy_reg rename in 2.6 05/20/08 CLOSED http://bugs.python.org/issue2924 created brett.cannon Revert Queue rename in 2.6 05/20/08 http://bugs.python.org/issue2925 created brett.cannon Revert SocketServer rename in 2.6 05/20/08 http://bugs.python.org/issue2926 created brett.cannon expose html.parser.unescape 05/20/08 http://bugs.python.org/issue2927 created thomaspinckney3 patch Allow set/frozenset for __all__ 05/20/08 CLOSED http://bugs.python.org/issue2928 created Rhamphoryncus patch TimedRotatingFileHandler crashes on backup file deletion attempt 05/20/08 CLOSED http://bugs.python.org/issue2929 created blocki Cannot change variable defined in "__init__.py" after importing 05/20/08 CLOSED http://bugs.python.org/issue2930 created crayor optparse: various problems with unicode and gettext 05/20/08 http://bugs.python.org/issue2931 created ash patch Documenting Python: Syntax" 05/21/08 CLOSED http://bugs.python.org/issue2932 created owen "Documenting Python: Syntax" bug 05/21/08 CLOSED http://bugs.python.org/issue2933 created owen set() comparisons do not play well with others 05/21/08 CLOSED http://bugs.python.org/issue2934 created ncoghlan rfc822.py line 395 strings connection 05/21/08 http://bugs.python.org/issue2935 created erezbibi ctypes.util.find_library() doesn't consult LD_LIBRARY_PATH 05/21/08 http://bugs.python.org/issue2936 created janssen Incorrect rounding in floating-point operations with gcc/x87 05/21/08 http://bugs.python.org/issue2937 created marketdickinson Interactive help writes to the python install destination direct 05/22/08 http://bugs.python.org/issue2938 created r_mosaic Apache mod_python python-func strftime 05/22/08 http://bugs.python.org/issue2939 created raisachrisgun Building Python fails on SunOS 05/22/08 http://bugs.python.org/issue2940 created syraxes Propagate define to resurce mingw32 compile 05/22/08 http://bugs.python.org/issue2941 created shura_zam patch mingw/cygwin do not accept asm file as extension source 05/22/08 http://bugs.python.org/issue2942 created shura_zam patch Distutils should generate a better error message when the SDK is 05/22/08 http://bugs.python.org/issue2943 created cdavid asyncore doesn't handle connection refused correctly 05/22/08 http://bugs.python.org/issue2944 created shigin patch bdist_rpm does not list dist files (should effect upload) 05/22/08 http://bugs.python.org/issue2945 created htgoebel setuptools: bdist_wininst adds duplicate entry to dist_files 05/22/08 http://bugs.python.org/issue2946 created htgoebel subprocess (Replacing popen) - add a warning / hint 05/22/08 http://bugs.python.org/issue2947 created HWJ Unicode support for hashing algorithms 05/23/08 CLOSED http://bugs.python.org/issue2948 created vvro Windows installer doesn't include OpenSSL license and notice 05/23/08 http://bugs.python.org/issue2949 created lemburg silly readline module problem 05/23/08 http://bugs.python.org/issue2950 created AndyNovo Issues Now Closed (49) ______________________ merge urllib and urlparse functionality 208 days http://bugs.python.org/issue1333 brett.cannon toxml generates output that is not well formed 201 days http://bugs.python.org/issue1390 loewis patch ctypes.util.find_msvcrt() function 126 days http://bugs.python.org/issue1793 theller patch Cant open python gui using VISTA 116 days http://bugs.python.org/issue1948 kbk mmap segfaults when trying to write a block opened with PROT_REA 99 days http://bugs.python.org/issue2111 loewis Use of file.xreadlines() should raise a Py3K warning 61 days http://bugs.python.org/issue2353 georg.brandl patch, 26backport test_subprocess fails if your sys.executable is on a path with a 60 days http://bugs.python.org/issue2381 gregory.p.smith patch, easy Backport sys.maxsize to Python 2.6 55 days http://bugs.python.org/issue2488 loewis patch, easy, 26backport urlparse normalize URL path 42 days http://bugs.python.org/issue2583 facundobatista PyString_FromStringAndSize() to be considered unsafe 36 days http://bugs.python.org/issue2587 gregory.p.smith weakref.proxy fails to delegate tp_index slot 42 days http://bugs.python.org/issue2592 georg.brandl easy "{ +(}".format(**{' +(': 44}) should produce ValueError: invalid 42 days http://bugs.python.org/issue2598 eric.smith rename test_support to support 39 days http://bugs.python.org/issue2621 brett.cannon patch, easy Add simplejson to Python 2.6/3.0 standard library 13 days http://bugs.python.org/issue2750 benjamin.peterson patch ctypes.util.find_library() doesn't find OS X .bundle or .so libr 14 days http://bugs.python.org/issue2783 benjamin.peterson easy Moving lib-tk to tkinter package 0 days http://bugs.python.org/issue2839 georg.brandl patch int() lies about base parameter 8 days http://bugs.python.org/issue2844 rhettinger patch bsddb.db.DBEnv.lock_get test_lock.test03_set_timeout crashes 2 days http://bugs.python.org/issue2858 gregory.p.smith 2to3 fixer to rename markupbase to _markupbase 7 days http://bugs.python.org/issue2861 quentin.gallet-gilles patch Problem with urllib and urllib2 in urlopen? 2 days http://bugs.python.org/issue2868 benjamin.peterson Rename repr to reprlib 3 days http://bugs.python.org/issue2880 brett.cannon Create the html package 0 days http://bugs.python.org/issue2882 brett.cannon Create the tkinter package 1 days http://bugs.python.org/issue2884 georg.brandl patch dict.has_key() expected in h2py.py. 0 days http://bugs.python.org/issue2893 georg.brandl Crash in ParseTupleAndKeywords when passing byte string keywords 0 days http://bugs.python.org/issue2895 georg.brandl cmathmodule.c compilation error 1 days http://bugs.python.org/issue2896 georg.brandl math.copysign(1, float('nan')) 1 days http://bugs.python.org/issue2900 marketdickinson test_urllib.py crashes with bus error on Darwin 0 days http://bugs.python.org/issue2905 georg.brandl ZeroDivisionError error message needs clarification 0 days http://bugs.python.org/issue2907 georg.brandl fix_imports new entries for tkinter rename 0 days http://bugs.python.org/issue2908 georg.brandl patch Logging TimedRotatingFileHandler Feature Request 1 days http://bugs.python.org/issue2914 vsajip PyObject_IsInstance() doesn't find bases named in type(name, bas 2 days http://bugs.python.org/issue2915 benjamin.peterson Revert copy_reg rename in 2.6 0 days http://bugs.python.org/issue2924 georg.brandl Allow set/frozenset for __all__ 1 days http://bugs.python.org/issue2928 Rhamphoryncus patch TimedRotatingFileHandler crashes on backup file deletion attempt 0 days http://bugs.python.org/issue2929 vsajip Cannot change variable defined in "__init__.py" after importing 1 days http://bugs.python.org/issue2930 georg.brandl Documenting Python: Syntax" 0 days http://bugs.python.org/issue2932 benjamin.peterson "Documenting Python: Syntax" bug 0 days http://bugs.python.org/issue2933 georg.brandl set() comparisons do not play well with others 0 days http://bugs.python.org/issue2934 ncoghlan Unicode support for hashing algorithms 0 days http://bugs.python.org/issue2948 lemburg Tkinter.Misc has no __contains__ method 2061 days http://bugs.python.org/issue615772 georg.brandl sendmsg operation for unix sockets 1695 days http://bugs.python.org/issue814689 georg.brandl Text.edit_modified() fails 1451 days http://bugs.python.org/issue961805 georg.brandl patch bsddb3 testsuite failure when running more than one time 1369 days http://bugs.python.org/issue1010645 gregory.p.smith bsddb dbobj.DB.associate doesn't accept dbobj.DB param 1077 days http://bugs.python.org/issue1215023 gregory.p.smith Text.edit_modified() doesn't work 909 days http://bugs.python.org/issue1362475 gpolo patch thread + import => crashes? 87 days http://bugs.python.org/issue1720705 Rhamphoryncus Build on QNX 6 369 days http://bugs.python.org/issue1722225 loewis patch zipfile: Allow reading duplicate filenames 278 days http://bugs.python.org/issue1775025 georg.brandl patch Top Issues Most Discussed (10) ______________________________ 14 Full precision summation 12 days open http://bugs.python.org/issue2819 11 Add memory footprint query 6 days open http://bugs.python.org/issue2898 11 Add ctypes calling convention that allows safe access of errno 133 days open http://bugs.python.org/issue1798 10 New class special method lookup change 2006 days open http://bugs.python.org/issue643841 8 Moving lib-tk to tkinter package 0 days closed http://bugs.python.org/issue2839 8 Implement PEP 3108 17 days open http://bugs.python.org/issue2775 5 Unicode support for hashing algorithms 0 days closed http://bugs.python.org/issue2948 5 Allow set/frozenset for __all__ 1 days closed http://bugs.python.org/issue2928 5 2to3 fixer to rename markupbase to _markupbase 7 days closed http://bugs.python.org/issue2861 5 ctypes.util.find_library() doesn't find OS X .bundle or .so lib 14 days closed http://bugs.python.org/issue2783 From tnelson at onresolve.com Sat May 24 04:30:06 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Sat, 24 May 2008 03:30:06 +0100 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? In-Reply-To: <4835A1E9.8090406@jcea.es> References: <6167796BFEB5D0438720AC212E89A6B0078657D7@exchange.onresolve.com> <48346695.4030906@jcea.es>,<483471F3.2030305@v.loewis.de> <6167796BFEB5D0438720AC212E89A6B0078657DA@exchange.onresolve.com>, <4835A1E9.8090406@jcea.es> Message-ID: <6167796BFEB5D0438720AC212E89A6B0078657EA@exchange.onresolve.com> > I just committed bsddb3 4.7.0. Please install Berkeley DB >=4.7.25. Oh my. Just finished plugging 4.7.25 into the Windows build environment. Upon running regrtest.py -v -u bsddb test_bsddb test_bsddb3, I get about 300 lines like this printed to stderr: DB->key_range: method not permitted before handle's open method illegal flag specified to DB->remove Then about 15 lines like this: DB_ENV->rep_start interface requires an environment configured for the replication subsystem DB_ENV->rep_start interface requires an environment configured for the replication subsystem DB_ENV->rep_start interface requires an environment configured for the replication subsystem DB_ENV->rep_start interface requires an environment configured for the replication subsystem Then Python crashes. Quick perusal of stdout reveals this happening on just about every test: ====================================================================== ERROR: test__no_deadlock_first (test.test_bsddb.TestBTree) ---------------------------------------------------------------------- Traceback (most recent call last): File "O:\scm\svn+ssh\pythondev at svn.python.org\python\trunk\lib\test\test_bsddb.py", line 16, in setUp self.f = self.openmethod[0](self.fname, self.openflag, cachesize=32768) File "O:\scm\svn+ssh\pythondev at svn.python.org\python\trunk\lib\bsddb\__init__.py", line 320, in btopen e = _openDBEnv(cachesize) File "O:\scm\svn+ssh\pythondev at svn.python.org\python\trunk\lib\bsddb\__init__.py", line 360, in _openDBEnv e.open('.', db.DB_PRIVATE | db.DB_CREATE | db.DB_THREAD | db.DB_INIT_LOCK | db.DB_INIT_MPOOL) DBInvalidArgError: (22, 'Invalid argument') The last lines before python_d.exe crashes are as follows: testCheckElapsedTime (test.test_bsddb3.TimingCheck) ... ok test_pget (bsddb.test.test_cursor_pget_bug.pget_bugTestCase) ... ERROR testCheckElapsedTime (test.test_bsddb3.TimingCheck) ... ok test01_both (bsddb.test.test_dbobj.dbobjTestCase) ... ERROR test02_dbobj_dict_interface (bsddb.test.test_dbobj.dbobjTestCase) ... ERROR test03_dbobj_type_before_open (bsddb.test.test_dbobj.dbobjTestCase) ... FAIL Think I'll call it a night for now. Trent. From techtonik at gmail.com Sat May 24 09:23:40 2008 From: techtonik at gmail.com (techtonik) Date: Sat, 24 May 2008 09:23:40 +0200 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 Message-ID: Does anyone would like to include Curses support in Windows from version 2.6? It works ok already using the patch from issue #2889 and PDCurses library, but needs expertise of core developers to integrate the patch into the build system, add library to externals to ship next windows Python with embedded curses support. I would really like to see this as release blocker, because the problem raised once in a while in the past and the windows people who are interested in getting console interface tools usually lack the knowledge of unix platform to do the task. P.S. It is all about cross-platform roguelikes and text-mode players in Python [1] http://bugs.python.org/issue2889 -- --anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat May 24 11:48:19 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 24 May 2008 11:48:19 +0200 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: References: Message-ID: <4837E463.3030805@v.loewis.de> > I would really like to see this as release blocker, because the problem > raised once in a while in the past and the windows people who are > interested in getting console interface tools usually lack the knowledge > of unix platform to do the task. It certainly can't be a release blocker: there is no accepted PEP demanding it, it does not cause any crashes, and it is no regression over earlier releases. For this to have any chance, it *must* work with Visual Studio. Requiring gcc is unacceptable. Regards, Martin From techtonik at gmail.com Sat May 24 13:09:52 2008 From: techtonik at gmail.com (techtonik) Date: Sat, 24 May 2008 13:09:52 +0200 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: <4837E463.3030805@v.loewis.de> References: <4837E463.3030805@v.loewis.de> Message-ID: On Sat, May 24, 2008 at 11:48 AM, "Martin v. L?wis" wrote: > > I would really like to see this as release blocker, because the problem > > raised once in a while in the past and the windows people who are > > interested in getting console interface tools usually lack the knowledge > > of unix platform to do the task. > > It certainly can't be a release blocker: there is no accepted PEP > demanding it, it does not cause any crashes, and it is no regression > over earlier releases. > > For this to have any chance, it *must* work with Visual Studio. > Requiring gcc is unacceptable. > As for PDCurses library itself there is a Makefile in PDCurses distribution for Microsoft Visual C++ 2.0+ named vcwin32.mak I can't afford buying Visual Studio to test if it works with newer versions, but logically Visual Studio should be able to convert Makefile to a newer format. As for extension I've provided .bat file that uses GCC for compilation and I think I have some ancient Visual C++ compiler to make the similar .bat file for cl.exe. But I doubt I'll be able to provide a patch for Python build system to compile the module into windows distribution, because I do not have much experience with Python build system and with C linking/makefiles in general. I know that PDCurses must be added to externals/, but that's all. So even if can test and script Python part I am afraid I need help from someone with experience of Python+Make+Visual Studio. -- --anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Sat May 24 15:10:10 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 24 May 2008 14:10:10 +0100 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: References: Message-ID: <79990c6b0805240610u3243fde0k4f18df73a533d9f6@mail.gmail.com> 2008/5/24 techtonik : > Does anyone would like to include Curses support in Windows from version > 2.6? I'd like to see this, although it's hardly crucial for me - however, "from version 2.6" may be unrealistic at this point. > It works ok already using the patch from issue #2889 and PDCurses library, > but needs expertise of core developers to integrate the patch into the build > system, add library to externals to ship next windows Python with embedded > curses support. One other option would be to pull out the core curses module as an external project (maybe called "pdcurses"), link it with pdcurses and provide Windows binaries - then developers who cared could do try: import curses except ImportError: import pdcurses as curses > I would really like to see this as release blocker, because the problem > raised once in a while in the past and the windows people who are interested > in getting console interface tools usually lack the knowledge of unix > platform to do the task. No way it's a release blocker - we've lasted this long without it so it's in no way essential for 2.6. > P.S. It is all about cross-platform roguelikes and text-mode players in > Python Cross-platform any sort of console interface more complex than stdin/stdout. But such interfaces are increasingly rare these days... Paul. From p.f.moore at gmail.com Sat May 24 15:15:30 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 24 May 2008 14:15:30 +0100 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: References: <4837E463.3030805@v.loewis.de> Message-ID: <79990c6b0805240615l65975730i9a8d289e2315eada@mail.gmail.com> 2008/5/24 techtonik : >> For this to have any chance, it *must* work with Visual Studio. >> Requiring gcc is unacceptable. > > As for PDCurses library itself there is a Makefile in PDCurses distribution > for Microsoft Visual C++ 2.0+ named vcwin32.mak I can't afford buying > Visual Studio to test if it works with newer versions, but logically Visual > Studio should be able to convert Makefile to a newer format. Visual C++ 9.0 Express Edition builds Python quite happily these days. So you can certainly do the integration without buying anything. If you get stuck on technical details, there are people here who would happily give you advice. Windows developers are always welcome! > As for extension I've provided .bat file that uses GCC for compilation and I > think I have some ancient Visual C++ compiler to make the similar .bat file > for cl.exe. But I doubt I'll be able to provide a patch for Python build > system to compile the module into windows distribution, because I do not > have much experience with Python build system and with C linking/makefiles > in general. I know that PDCurses must be added to externals/, but that's > all. If you don't have the knowledge needed, and can't spare the time to learn (which is entirely acceptable) then you are indeed relying on another Windows developer to pick this up. You may be out of luck there - nobody has been interested enough to do this before now (it's not as if PDCurses is new) so there's not much reason to expect things to have changed. Paul. From matthieu.brucher at gmail.com Sat May 24 15:18:17 2008 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Sat, 24 May 2008 15:18:17 +0200 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: References: <4837E463.3030805@v.loewis.de> Message-ID: > > As for PDCurses library itself there is a Makefile in PDCurses distribution > for Microsoft Visual C++ 2.0+ named vcwin32.mak I can't afford buying > Visual Studio to test if it works with newer versions, but logically Visual > Studio should be able to convert Makefile to a newer format. You can start with Visual C++ Express, which is compeltely free ;) I don't know if Visual Studio updates the Makefiles. Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat May 24 17:47:57 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 24 May 2008 17:47:57 +0200 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: References: <4837E463.3030805@v.loewis.de> Message-ID: <483838AD.8070800@v.loewis.de> > As for PDCurses library itself there is a Makefile in PDCurses > distribution for Microsoft Visual C++ 2.0+ named vcwin32.mak I can't > afford buying Visual Studio to test if it works with newer versions, but > logically Visual Studio should be able to convert Makefile to a newer > format. Visual Studio 2008 Express is free of charge; so you should be able to work on this with that release. > So even if can test and script Python part I am afraid I need help from > someone with experience of Python+Make+Visual Studio. Then there is little chance that the patch gets accepted. I will have zero time for this until the beta release, unless the patch just needs to be applied, compiles without changes, and passes it the test suite. Regards, Martin From martin at v.loewis.de Sat May 24 17:51:10 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 24 May 2008 17:51:10 +0200 Subject: [Python-Dev] Issue2889: Add Curses for Windows as native module for 2.6 In-Reply-To: References: <4837E463.3030805@v.loewis.de> Message-ID: <4838396E.5040706@v.loewis.de> > You can start with Visual C++ Express, which is compeltely free ;) > I don't know if Visual Studio updates the Makefiles. I would like the makefiles to be ignored entirely. Trent has established a policy that we compile everything in a single project file, and while that means some extra work initially, it also has advantages (such as not having to worry about conflicting command line options between the library's build procedure, and Python's). Regards, Martin From jcea at jcea.es Sun May 25 06:25:17 2008 From: jcea at jcea.es (Jesus Cea) Date: Sun, 25 May 2008 06:25:17 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? Message-ID: <4838EA2D.7060402@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I can't find any PEP about detailed 2.6 -> 3000 migration guidelines, specially in the module (C code) extension area. Yes, I know about the 2to3 tool, but I'm interested in updating my 2.x code in order to keep (automatic via "2to3") difference between 2.x and 3.x codebase so small as possible. Also, 2to3 doesn't manage migration for C modules. Since I need to port bsddb3 to py3k, what I need to know?. Is any *updated* document out there?. PS: My plan is keep working in the python side under 2.x, and manage Python 3.0 via "2to3", for a long time. In the C side, I plan keeping the same codebase, with conditional compilation. Ideas?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDjqJplgi5GaxT1NAQLP3QP/Y2bWmNzHKsIq23dRz9sxd237TSfzbs5X UzEQ6Ze9Kv1lRXYRfiGjk37aOeiO4xqoKPfASc/WjMN/TmRkUNZxMQEQWwFYhmCA aAVqGxNMEX3KCXvRP9MkkZYLPa9RZg8boPTMYCn+uLDW1Ff2Pwj96hFKbWKG0dkJ RamNp8BFhkM= =ngK+ -----END PGP SIGNATURE----- From brett at python.org Sun May 25 07:43:00 2008 From: brett at python.org (Brett Cannon) Date: Sat, 24 May 2008 22:43:00 -0700 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <4838EA2D.7060402@jcea.es> References: <4838EA2D.7060402@jcea.es> Message-ID: On Sat, May 24, 2008 at 9:25 PM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I can't find any PEP about detailed 2.6 -> 3000 migration guidelines, > specially in the module (C code) extension area. Yes, I know about the > 2to3 tool, but I'm interested in updating my 2.x code in order to keep > (automatic via "2to3") difference between 2.x and 3.x codebase so small > as possible. Also, 2to3 doesn't manage migration for C modules. > > Since I need to port bsddb3 to py3k, what I need to know?. Is any > *updated* document out there?. > No definitive porting guide has been written, although it has been discussed as being needed before. One should probably be written ASAP if we are going to get any decent testing done by people with the beta (and I am definitely NOT volunteering to write the doc). -Brett From greg at krypto.org Sun May 25 09:18:38 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 25 May 2008 00:18:38 -0700 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <4838EA2D.7060402@jcea.es> References: <4838EA2D.7060402@jcea.es> Message-ID: <52dc1c820805250018h312c663bu13df1d2f56a9893c@mail.gmail.com> Of particular interest to bsddb is the pep3118 buffer API. I updated the existing bsddb module found in the py3k branch to use it last fall; you may want to use the changes made to it as an example. I agree, a single C code base is the only sane way forward for bsddb. The interesting part is that you also want to support older python versions (2.3-2.5?) as well so _bsddb.c & bsddb.h become a nice example of how to write code that spans multiple python versions. I expect it is less difficult than it sounds. http://www.python.org/dev/peps/pep-3118/ I believe the buffer api is supposed to be backported to 2.6? I haven't followed the status of that. If so, bsddb should use it in 2.6 as well. -gps On Sat, May 24, 2008 at 9:25 PM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I can't find any PEP about detailed 2.6 -> 3000 migration guidelines, > specially in the module (C code) extension area. Yes, I know about the > 2to3 tool, but I'm interested in updating my 2.x code in order to keep > (automatic via "2to3") difference between 2.x and 3.x codebase so small > as possible. Also, 2to3 doesn't manage migration for C modules. > > Since I need to port bsddb3 to py3k, what I need to know?. Is any > *updated* document out there?. > > PS: My plan is keep working in the python side under 2.x, and manage > Python 3.0 via "2to3", for a long time. In the C side, I plan keeping > the same codebase, with conditional compilation. Ideas?. > > - -- > Jesus Cea Avion _/_/ _/_/_/ _/_/_/ > jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ > jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ > ~ _/_/ _/_/ _/_/ _/_/ _/_/ > "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ > "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ > "El amor es poner tu felicidad en la felicidad de otro" - Leibniz > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.8 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iQCVAwUBSDjqJplgi5GaxT1NAQLP3QP/Y2bWmNzHKsIq23dRz9sxd237TSfzbs5X > UzEQ6Ze9Kv1lRXYRfiGjk37aOeiO4xqoKPfASc/WjMN/TmRkUNZxMQEQWwFYhmCA > aAVqGxNMEX3KCXvRP9MkkZYLPa9RZg8boPTMYCn+uLDW1Ff2Pwj96hFKbWKG0dkJ > RamNp8BFhkM= > =ngK+ > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > From greg at krypto.org Sun May 25 09:21:48 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 25 May 2008 00:21:48 -0700 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: <4835D86C.6030406@v.loewis.de> References: <4829E787.5080605@jcea.es> <52dc1c820805132133s1ad0be0em2d7c25296fece78c@mail.gmail.com> <4835BDDC.9040709@jcea.es> <4835D86C.6030406@v.loewis.de> Message-ID: <52dc1c820805250021j4854ac39la55b7e520f0719b2@mail.gmail.com> On Thu, May 22, 2008 at 1:32 PM, "Martin v. L?wis" wrote: >> I would like to print the Berkeley DB release used, also. > > Could that be done at build time as well? > > Martin I just added that to python's setup.py as well as made test_bsddb3 print the version to stderr when it gets run. -gps From stefan_ml at behnel.de Sun May 25 09:59:55 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 25 May 2008 09:59:55 +0200 Subject: [Python-Dev] Any PEP about 2.6 -> 3000 code transition? In-Reply-To: <52dc1c820805250018h312c663bu13df1d2f56a9893c@mail.gmail.com> References: <4838EA2D.7060402@jcea.es> <52dc1c820805250018h312c663bu13df1d2f56a9893c@mail.gmail.com> Message-ID: Gregory P. Smith wrote: > Of particular interest to bsddb is the pep3118 buffer API. I updated > the existing bsddb module found in the py3k branch to use it last > fall; you may want to use the changes made to it as an example. > > I agree, a single C code base is the only sane way forward for bsddb. > The interesting part is that you also want to support older python > versions (2.3-2.5?) as well so _bsddb.c & bsddb.h become a nice > example of how to write code that spans multiple python versions. > > I expect it is less difficult than it sounds. I would say so. Most things can be done with preprocessor conditionals and some redefinitions, e.g. "#define PyLong_Type PyInt_Type", etc. The method "generate_module_preamble()" in Cython's ModuleNode.py source file (from line 360 onwards) has a code block of things we found so far. http://hg.cython.org/cython-devel/file/tip/Cython/Compiler/ModuleNode.py > I believe the buffer api is supposed to be backported to 2.6? I > haven't followed the status of that. If so, bsddb should use it in > 2.6 as well. The API is there (in addition to the old buffer API), but it doesn't seem to get used. Stefan From lists at cheimes.de Sun May 25 16:59:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 25 May 2008 16:59:24 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 Message-ID: <48397ECC.9070805@cheimes.de> Hello! The first set of betas of Python 2.6 and 3.0 is fast apace. I like to grab the final chance and clean up the C API of 2.6 and 3.0. I know, I know, I brought up the topic two times in the past. But this time I mean it for real! :] Last time Guido said: --- I think it can actually be simplified. I think maintaining binary compatibility between 2.6 and earlier versions is hopeless anyway, so we might as well just rename PyString to PyBytes in 2.6 and 3.0, and have an extra set of macros so that code using PyString needs to be recompiled but not otherwise touched. E.g. typedef { ... } PyBytesObject; #define PyStringObject PyBytesObject ... PyString_Type; #define PyBytes_Type PyString_Type --- I like to follow Guido's advice and change the code as following: * replace PyBytes_ with PyByteArray_ * replace PyString with PyBytes_ * rename bytesobject.[ch] to bytearrayobject.[ch] * rename stringobject.[ch] to bytesobject.[ch] * add a new file stringobject.h which contains the aliases PyString_ -> PyBytes_ Christian From lists at cheimes.de Sun May 25 17:28:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 25 May 2008 17:28:53 +0200 Subject: [Python-Dev] Please svnmerge your changes Message-ID: <483985B5.6020705@cheimes.de> Hello fellow developers! I've been busy with personal work in the past weeks. At present I'm still moving into my new apartment. It has been a real challenge to install an IKEA kitchen in a house built before WW2 all by myself. On the one hand it's fun but on the other hand it costs me most of my free time at night. At least this building has a shelter in its cellar so I'm mostly protected in the case of an air strike. *g* In order to get all code merged before the first betas I need your help. Please everybody grab a couple of your checkins and merge them yourself. You can find the list of required merges at http://rafb.net/p/cghbTk63.html Christian From lists at cheimes.de Sun May 25 18:16:52 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 25 May 2008 18:16:52 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> Message-ID: <483990F4.30802@cheimes.de> Stefan Behnel schrieb: > will that be included by Python.h by default? Only in Python 2.6 Christian From ggpolo at gmail.com Sun May 25 19:10:25 2008 From: ggpolo at gmail.com (Guilherme Polo) Date: Sun, 25 May 2008 14:10:25 -0300 Subject: [Python-Dev] [Python-3000] Please svnmerge your changes In-Reply-To: <483985B5.6020705@cheimes.de> References: <483985B5.6020705@cheimes.de> Message-ID: 2008/5/25 Christian Heimes : > Hello fellow developers! > > I've been busy with personal work in the past weeks. At present I'm > still moving into my new apartment. It has been a real challenge to > install an IKEA kitchen in a house built before WW2 all by myself. On > the one hand it's fun but on the other hand it costs me most of my free > time at night. At least this building has a shelter in its cellar so I'm > mostly protected in the case of an air strike. *g* > > In order to get all code merged before the first betas I need your help. > Please everybody grab a couple of your checkins and merge them yourself. > > You can find the list of required merges at http://rafb.net/p/cghbTk63.html > I see there is one commit of mine that shouldn't be merged, maybe you could remove it from there ? > Christian Thanks, -- -- Guilherme H. Polo Goncalves From lists at cheimes.de Sun May 25 19:22:05 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 25 May 2008 19:22:05 +0200 Subject: [Python-Dev] [Python-3000] Please svnmerge your changes In-Reply-To: References: <483985B5.6020705@cheimes.de> Message-ID: <4839A03D.3080304@cheimes.de> Guilherme Polo schrieb: > I see there is one commit of mine that shouldn't be merged, maybe you > could remove it from there ? You can easily block it yourself with svnmerge.py: py3k$ svnmerge.py block -r1234 py3k$ svn ci -F svnmerge-commit-message.txt Christian From solipsis at pitrou.net Sun May 25 20:44:59 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 25 May 2008 18:44:59 +0000 (UTC) Subject: [Python-Dev] Mercurial mirrors of trunk and py3k Message-ID: Hello, For those of you who have been using my Mercurial mirrors of the Python SVN repository, I had to move them to another site as the original site has been down for a few days for unknown reasons. Here are the new URLs: http://hg.pitrou.net/public/cpython/trunk/ http://hg.pitrou.net/public/py3k/py3k/ If you want those to become the default pulling location, you just have to edit the .hg/hgrc for your local clone(s) and fill in the appropriate URL. Regards Antoine. From brett at python.org Mon May 26 00:02:32 2008 From: brett at python.org (Brett Cannon) Date: Sun, 25 May 2008 15:02:32 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <48397ECC.9070805@cheimes.de> References: <48397ECC.9070805@cheimes.de> Message-ID: On Sun, May 25, 2008 at 7:59 AM, Christian Heimes wrote: > Hello! > > The first set of betas of Python 2.6 and 3.0 is fast apace. I like to > grab the final chance and clean up the C API of 2.6 and 3.0. I know, I > know, I brought up the topic two times in the past. But this time I mean > it for real! :] > > Last time Guido said: > --- > I think it can actually be simplified. I think maintaining binary > compatibility between 2.6 and earlier versions is hopeless anyway, so > we might as well just rename PyString to PyBytes in 2.6 and 3.0, and > have an extra set of macros so that code using PyString needs to be > recompiled but not otherwise touched. E.g. > > typedef { ... } PyBytesObject; > #define PyStringObject PyBytesObject > > ... PyString_Type; > #define PyBytes_Type PyString_Type > > > --- > > I like to follow Guido's advice and change the code as following: > > * replace PyBytes_ with PyByteArray_ > * replace PyString with PyBytes_ > * rename bytesobject.[ch] to bytearrayobject.[ch] > * rename stringobject.[ch] to bytesobject.[ch] > * add a new file stringobject.h which contains the aliases PyString_ -> > PyBytes_ +1 from me. -Brett From brett at python.org Mon May 26 00:04:32 2008 From: brett at python.org (Brett Cannon) Date: Sun, 25 May 2008 15:04:32 -0700 Subject: [Python-Dev] [Python-3000] Please svnmerge your changes In-Reply-To: <483985B5.6020705@cheimes.de> References: <483985B5.6020705@cheimes.de> Message-ID: On Sun, May 25, 2008 at 8:28 AM, Christian Heimes wrote: > Hello fellow developers! > > I've been busy with personal work in the past weeks. At present I'm > still moving into my new apartment. It has been a real challenge to > install an IKEA kitchen in a house built before WW2 all by myself. On > the one hand it's fun but on the other hand it costs me most of my free > time at night. At least this building has a shelter in its cellar so I'm > mostly protected in the case of an air strike. *g* > > In order to get all code merged before the first betas I need your help. > Please everybody grab a couple of your checkins and merge them yourself. > > You can find the list of required merges at http://rafb.net/p/cghbTk63.html For stuff from the sandbox, would it help at all to block them explicitly even though they shouldn't get merged at all? -Brett From hrvoje.niksic at avl.com Mon May 26 11:21:35 2008 From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=) Date: Mon, 26 May 2008 11:21:35 +0200 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: <48359DDD.5000501@scottdial.com> Message-ID: <1211793695.32116.26.camel@localhost> On Thu, 2008-05-22 at 13:27 -0300, Facundo Batista wrote: > 2008/5/22 Scott Dial : > > > If we changed Python to slice-by-reference, then tomorrow someone would be > > asking why it isn't slice-by-copy. There are pros and cons to both that are > > Which are the cons of slice-by-reference of an immutable string? You have to consider the ramifications of such a design choice. There are two cases to consider: either slices return strings, or they return a different types. If they return strings, then all strings must grow three additional fields: start, end, and the reference to the actual string. That is 16 more bytes for *every* string, hardly a net win. You could argue that string slicing should return a separate type. But then, every piece of code that handles strings must be taught to handle this type in addition. PyString_Check and PyString_CheckExact must go. Instead of writing PyString_Foo, the C code would have to use PyObject_CallMethod and friends for the simplest tasks. Time-saving macros like PyString_AS_STRING and PyString_GET_SIZE become obsolete. Where performance matters (as it often does in low-level C code dealing with strings), this is a real problem. Besides, you lose various special optimizations, such as dicts being much faster when all keys are strings. From jason.orendorff at gmail.com Mon May 26 14:16:10 2008 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Mon, 26 May 2008 07:16:10 -0500 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: <1211793695.32116.26.camel@localhost> References: <48359DDD.5000501@scottdial.com> <1211793695.32116.26.camel@localhost> Message-ID: On Mon, May 26, 2008 at 4:21 AM, Hrvoje Nik?i? wrote: > On Thu, 2008-05-22 at 13:27 -0300, Facundo Batista wrote: >> 2008/5/22 Scott Dial : >> >> > If we changed Python to slice-by-reference, then tomorrow someone would be >> > asking why it isn't slice-by-copy. There are pros and cons to both that are >> >> Which are the cons of slice-by-reference of an immutable string? > > You have to consider the ramifications of such a design choice. There > are two cases to consider: either slices return strings, or they return > a different types. > > If they return strings, then all strings must grow three additional > fields: start, end, and the reference to the actual string. That is 16 > more bytes for *every* string, hardly a net win. A lot of dynamic language implementations have a complex string representation, where individual bits of the string tell what the rest of the representation is. Mozilla's JavaScript implementation is like this. At the moment, a string in JavaScript is two pointer-sized words, and JavaScript has O(1) slicing and, in many cases, O(len(s2)) string concatenation. There's a rather dense comment here explaining it: http://hg.mozilla.org/mozilla-central/index.cgi/file/79924d3b5bba/js/src/jsstr.h The equivalent of PyString_AS_STRING and PyString_GET_SIZE contains a branch. I don't think the implementation avoids the worst cases Guido was talking about; tiny substrings can keep huge strings alive. -j From musiccomposition at gmail.com Mon May 26 14:35:24 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 26 May 2008 07:35:24 -0500 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <48397ECC.9070805@cheimes.de> References: <48397ECC.9070805@cheimes.de> Message-ID: <1afaf6160805260535l1fd5d136vbdf2d24b1380e2be@mail.gmail.com> On Sun, May 25, 2008 at 9:59 AM, Christian Heimes wrote: > > I like to follow Guido's advice and change the code as following: > > * replace PyBytes_ with PyByteArray_ > * replace PyString with PyBytes_ > * rename bytesobject.[ch] to bytearrayobject.[ch] > * rename stringobject.[ch] to bytesobject.[ch] > * add a new file stringobject.h which contains the aliases PyString_ -> > PyBytes_ +1 Do you need any help? > > Christian -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From mal at egenix.com Mon May 26 15:29:07 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 26 May 2008 15:29:07 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <48397ECC.9070805@cheimes.de> References: <48397ECC.9070805@cheimes.de> Message-ID: <483ABB23.6050900@egenix.com> On 2008-05-25 16:59, Christian Heimes wrote: > Hello! > > The first set of betas of Python 2.6 and 3.0 is fast apace. I like to > grab the final chance and clean up the C API of 2.6 and 3.0. I know, I > know, I brought up the topic two times in the past. But this time I mean > it for real! :] > > Last time Guido said: > --- > I think it can actually be simplified. I think maintaining binary > compatibility between 2.6 and earlier versions is hopeless anyway, so > we might as well just rename PyString to PyBytes in 2.6 and 3.0, and > have an extra set of macros so that code using PyString needs to be > recompiled but not otherwise touched. E.g. > > typedef { ... } PyBytesObject; > #define PyStringObject PyBytesObject > > ... PyString_Type; > #define PyBytes_Type PyString_Type > > > --- > > I like to follow Guido's advice and change the code as following: > > * replace PyBytes_ with PyByteArray_ > * replace PyString with PyBytes_ > * rename bytesobject.[ch] to bytearrayobject.[ch] > * rename stringobject.[ch] to bytesobject.[ch] > * add a new file stringobject.h which contains the aliases PyString_ -> > PyBytes_ Since this is major break in the Python C API, please make sure that you bump the Python C API level used for module imports. Most imports will fail anyway at the link stage, since PyString_* APIs are probably the most used C APIs in Python extensions. One detail, I'm worried about is the change of the type name, since that is sometimes used in object serialization or proxy implementations. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 26 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 41 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From lists at cheimes.de Mon May 26 15:43:57 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 26 May 2008 15:43:57 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <1afaf6160805260535l1fd5d136vbdf2d24b1380e2be@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <1afaf6160805260535l1fd5d136vbdf2d24b1380e2be@mail.gmail.com> Message-ID: <483ABE9D.3070004@cheimes.de> Benjamin Peterson schrieb: > On Sun, May 25, 2008 at 9:59 AM, Christian Heimes wrote: >> I like to follow Guido's advice and change the code as following: >> >> * replace PyBytes_ with PyByteArray_ >> * replace PyString with PyBytes_ >> * rename bytesobject.[ch] to bytearrayobject.[ch] >> * rename stringobject.[ch] to bytesobject.[ch] >> * add a new file stringobject.h which contains the aliases PyString_ -> >> PyBytes_ > > +1 > > Do you need any help? I've renamed the functions and modules. Can you help me with updating the C API docs? In Python 2.6 the docs must still use PyString but you can add a note that PyBytes_ works, too. Christian From andymac at bullseye.apana.org.au Mon May 26 15:10:43 2008 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Tue, 27 May 2008 00:10:43 +1100 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <48397ECC.9070805@cheimes.de> References: <48397ECC.9070805@cheimes.de> Message-ID: <483AB6D3.8010207@bullseye.andymac.org> Christian Heimes wrote: > The first set of betas of Python 2.6 and 3.0 is fast apace. I like to > grab the final chance and clean up the C API of 2.6 and 3.0. I know, I > know, I brought up the topic two times in the past. But this time I mean > it for real! :] On the subject of stabilising the API, I assigned issue 2862 to you concerning tidying up freelist management interfaces for ints and floats (http://bugs.python.org/issue2862). Note that the patch in issue 2862 is essentially orthogonal to the patch in issue 2039, although any int/float freelist implementation changes would require amendments. Additionally, I notice that not all of the types with free lists have grown routines to clear them - dicts, lists and sets are missing these routines. I will add a patch for these in the next few days if no-one else gets there first. On the subject of issue 2039, I've come to the view that "explicit is better than implicit" applies to the freelist management, and with the addition of freelist clearing routines called from gc.collect() I see little reason to pursue bounding of freelist sizes (and would suggest removal of existing bounding code in those freelist implementations that currently have it). I have also come to the view that pymalloc's automatic attempts to return empty arenas to the OS should be changed to an on-demand cleaning, called after all other cleanup in gc.collect(). Returning arenas, while not expensive in general, is nonetheless not free (in performance terms). -- ------------------------------------------------------------------------- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac at bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac at pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From lists at cheimes.de Mon May 26 15:40:31 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 26 May 2008 15:40:31 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483ABB23.6050900@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> Message-ID: <483ABDCF.8000105@cheimes.de> M.-A. Lemburg schrieb: > Most imports will fail anyway at the link stage, since PyString_* APIs > are probably the most used C APIs in Python extensions. I think you have missed an important point. In Python 2.6 the names stay the same for the linker. Although the functions are now called PyBytes_Egg, they are redefined to PyString_Egg by a second header file. In Python 2.6 the renaming of PyString are purely for consistence with the new Python 3.0 names. The names for PyString stay the same for external code like the library and extension modules. PyBytes -> PyByteArray is a different story, though. > One detail, I'm worried about is the change of the type name, since > that is sometimes used in object serialization or proxy implementations. The type names aren't changed, too They are still "str" and "bytearray" in Python 2.6 (moved down) > Since this is major break in the Python C API, please make sure > that you bump the Python C API level used for module imports. Do you still think it's necessary to bump up the C API version level? Christian From mal at egenix.com Mon May 26 17:03:20 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 26 May 2008 17:03:20 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483ABDCF.8000105@cheimes.de> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> Message-ID: <483AD138.7000804@egenix.com> On 2008-05-26 15:40, Christian Heimes wrote: > M.-A. Lemburg schrieb: >> Most imports will fail anyway at the link stage, since PyString_* APIs >> are probably the most used C APIs in Python extensions. > > I think you have missed an important point. In Python 2.6 the names stay > the same for the linker. Although the functions are now called > PyBytes_Egg, they are redefined to PyString_Egg by a second header file. > > In Python 2.6 the renaming of PyString are purely for consistence with > the new Python 3.0 names. The names for PyString stay the same for > external code like the library and extension modules. Isn't that an awefuly confusing approach ? Wouldn't it be better to keep PyString APIs and definitions in stringobject.c|h and only add a new bytesobject.h header file that #defines the PyBytes APIs in terms of PyString APIs ? That maintains backwards compatibility and allows Python internals to use the new API names. With your approach, you've basically backported the confusing notion in Py3k that str() maps PyUnicode, only that in Py2 str() will now map to PyBytes. You'd have to add an aliase bytes -> str to the builtins to at least reduce the confusion a bit. However, that's bound to cause even more problems, since people will start using bytes() instead of str() in Py2 applications and as a result they won't run in older Python versions anymore. The same problem applies to Py2 extensions writers that wish to support older Python releases as well. > PyBytes -> PyByteArray is a different story, though. PyBytes was new in 2.6 anyway, so there's no breakage there. >> One detail, I'm worried about is the change of the type name, since >> that is sometimes used in object serialization or proxy implementations. > > The type names aren't changed, too They are still "str" and "bytearray" > in Python 2.6 Good. > (moved down) >> Since this is major break in the Python C API, please make sure >> that you bump the Python C API level used for module imports. > > Do you still think it's necessary to bump up the C API version level? Yes, but please let's first discuss this some more. I don't think that the timing was right.... you started this thread just yesterday and the patches are already checked in. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 26 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 41 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From lists at informa.tiker.net Mon May 26 17:29:49 2008 From: lists at informa.tiker.net (Andreas =?utf-8?q?Kl=C3=B6ckner?=) Date: Mon, 26 May 2008 11:29:49 -0400 Subject: [Python-Dev] itertools additions: one(), single_valued() Message-ID: <200805261129.51914.lists@informa.tiker.net> Hi all, I find the following two operations functions useful and general enough that I would like to propose them for addition to itertools: 8< ------------------------------------------------------------------------- def single_valued(iterable): it = iter(iterable) try: first_item = it.next() except StopIteration: raise ValueError, "empty iterable passed to 'single_valued()'" for other_item in it: if other_item != first_item: raise ValueError, "non-single-valued iterable'" return first_item 8< ------------------------------------------------------------------------- This first one embodies the assertion that all values of the iterable are identical. If they are, that value is returned. Otherwise, an exception is thrown. Maybe this should be rephrased such that the assertion part is evaluated via an actual "assert", thereby turning it off in optimized mode. Example use case: You get a list of lists, and expect each to be the same length. Typical treatment: list_len = len(lists[0]) New treatment: list_len = single_valued(len(l) for l in lists) Added benefits: The assumption is verified and clearly visible from the source. "lists" may now be an iterable. 8< ------------------------------------------------------------------------- def one(iterable): it = iter(iterable) try: v = it.next() except StopIteration: raise ValueError, "empty iterable passed to 'one()'" try: v2 = it.next() raise ValueError, "iterable with more than one entry passed to 'one()'" except StopIteration: return v 8< ------------------------------------------------------------------------- This one is a blatant rip-off from SQLAlchemy. It basically allows most searches to be written using generator expressions: what_i_am_looking_for = one(item for item in items if predicate(item)) This also encodes and checks the assumption that the sought item is unique within the list of candidates. Again, the assertion part could be turned off in optimized mode. Opinions? Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From bborcic at gmail.com Mon May 26 19:47:44 2008 From: bborcic at gmail.com (Boris Borcic) Date: Mon, 26 May 2008 19:47:44 +0200 Subject: [Python-Dev] itertools additions: one(), single_valued() In-Reply-To: <200805261129.51914.lists@informa.tiker.net> References: <200805261129.51914.lists@informa.tiker.net> Message-ID: > Opinions? > > Andreas my_target, = iterable my_target, = set(iterable) quite readably does it, imho, or else one can use [my_target] = iterable [my_target] = set(iterable) the error messages seem also OK. Cheers, BB From fijall at gmail.com Mon May 26 19:46:41 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 26 May 2008 19:46:41 +0200 Subject: [Python-Dev] zipimport._zip_directory_cache interface Message-ID: <693bc9ab0805261046r71f286a7od7adb948934cb2cb@mail.gmail.com> _zip_directory_cache is a cache for modules imported by zipimporter. It's a normal dictionary exposing gory details of zipimport implementation. It wouldn't be a bad thing on it's own, but pkgutil.py uses this knowledge, while _zip_directory_cache has no single test nor documentation (besides info in docstring). It's a bit of a pain to actually implement this in other python implementations (especially that cache can be internal and exposed differently). I think it should be either specified better (and stripped from _) or zipimport should expose some nicer interface to it's cache. What do you think? Cheers, fijal From g.brandl at gmx.net Mon May 26 20:27:35 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 26 May 2008 20:27:35 +0200 Subject: [Python-Dev] ABC issues Message-ID: Some ABC issues: * The 2.6-backported Mapping ABC has the 3.0 dict API, that is, it uses keys() that returns a view etc. * The 2.6 UserDict is not registered as a mapping. * collections.deque isn't registered as a MutableSequence. If there are no objections, I will correct these issues in the 2.6 and 3.0 branches. Georg From fijall at gmail.com Mon May 26 20:45:59 2008 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 26 May 2008 20:45:59 +0200 Subject: [Python-Dev] Slice as a copy... by design? Message-ID: <693bc9ab0805261145i1fbe2c61sbc15a694dab3cc65@mail.gmail.com> > Hi! > > A thread in PyAr raised the question that, considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. > > I couldn't answer why, so I'm asking here...Is it because the > reference counting will be complicated? Is it because it'd be > inefficient in other way? It's something else? Or is something that > could be done... but is not done yet? > > Thank you very much! We implemented this feature in pypy. It's turned of by default, because nobody could come out with a real world example benefiting from it. Regarding worst-case scenario, we only create this special object if slice length is not shorter than some percentage of source string (I think 75%, can be tuned). You still need to pay for a dispatch (even if small price) wherever you use string, to check if this is special object or not. Cheers, fijal From musiccomposition at gmail.com Mon May 26 20:46:10 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 26 May 2008 13:46:10 -0500 Subject: [Python-Dev] ABC issues In-Reply-To: References: Message-ID: <1afaf6160805261146u2d5c4e4fp5932eab86355dd79@mail.gmail.com> On Mon, May 26, 2008 at 1:27 PM, Georg Brandl wrote: > Some ABC issues: > > * The 2.6-backported Mapping ABC has the 3.0 dict API, > that is, it uses keys() that returns a view etc. How would you correct this in 2.6? Add the inter* variants and make keys, items, and values return a list? > > * The 2.6 UserDict is not registered as a mapping. > > * collections.deque isn't registered as a MutableSequence. > > If there are no objections, I will correct these issues > in the 2.6 and 3.0 branches. None on principle. > > Georg -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Mon May 26 20:49:11 2008 From: brett at python.org (Brett Cannon) Date: Mon, 26 May 2008 11:49:11 -0700 Subject: [Python-Dev] zipimport._zip_directory_cache interface In-Reply-To: <693bc9ab0805261046r71f286a7od7adb948934cb2cb@mail.gmail.com> References: <693bc9ab0805261046r71f286a7od7adb948934cb2cb@mail.gmail.com> Message-ID: On Mon, May 26, 2008 at 10:46 AM, Maciej Fijalkowski wrote: > _zip_directory_cache is a cache for modules imported by zipimporter. > It's a normal dictionary exposing gory details of zipimport > implementation. It wouldn't be a bad thing on it's own, but pkgutil.py > uses this knowledge, while _zip_directory_cache has no single test nor > documentation (besides info in docstring). It's a bit of a pain to > actually implement this in other python implementations (especially > that cache can be internal and exposed differently). I think it should > be either specified better (and stripped from _) or zipimport should > expose some nicer interface to it's cache. What do you think? > How useful is the cache access? If it is not necessary, I think pkgutil just needs to stop using it. If it is useful to access, the question becomes who exposes the cache; the module, the importers, or the loaders? -Brett From python at rcn.com Mon May 26 20:59:30 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 26 May 2008 11:59:30 -0700 Subject: [Python-Dev] ABC issues References: Message-ID: > * The 2.6-backported Mapping ABC has the 3.0 dict API, > that is, it uses keys() that returns a view etc. Curious to hear what Guido thinks about this one. A nice use of the Mapping ABC is to be able to get 3.0 behaviors. I thought that was the whole point of all these backports. If the ABC gets altered, then it just makes the 2-to-3 conversion harder. > * The 2.6 UserDict is not registered as a mapping. Since the API's are not currently the same, it makes sense that UserDict is not registered. If the Mapping ABC does get changed, only IterableUserDict should get registered. A regular UserDict does not comply. > * collections.deque isn't registered as a MutableSequence. Deque's do not support count(), insert() or __iadd__(). They should not be registered. General purpose indexing into a deque is typically a mis-use of the data structure. It was provided only to make it easier to substitute for lists in apps the operate only one ends (i.e.d[0], d[1], d[-1], d[-2] but not d[i] to somewhere in the middle). > If there are no objections, I will correct these issues > in the 2.6 and 3.0 branches. I think none of these changes should be made. Raymond From python at rcn.com Mon May 26 21:27:56 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 26 May 2008 12:27:56 -0700 Subject: [Python-Dev] itertools additions: one(), single_valued() References: <200805261129.51914.lists@informa.tiker.net> Message-ID: [Andreas] > I find the following two operations functions useful and general > enough that I would like to propose them for addition to itertools: No thanks. Variants can already be constructed from existing tools. And, they seem a little to specific to a data model where the first entry has some special significance depending on whether or not it is unique. > it = iter(iterable) > try: > first_item = it.next() > except StopIteration: > raise ValueError, "empty iterable passed to 'single_valued()'" > for other_item in it: > if other_item != first_item: > raise ValueError, "non-single-valued iterable'" > return first_item This looks like a set() operation with a couple odd special cases for exceptions. [] --> ValueError If [x] --> x [x x x] --> x [x x y x] --> ValueError The two non-exception cases both run the input iterable to exhaustion and as such do not fit it with the lazy-evaluation theme of the itertools module. > def one(iterable): > it = iter(iterable) > try: > v = it.next() > except StopIteration: > raise ValueError, "empty iterable passed to 'one()'" > try: > v2 = it.next() > raise ValueError, "iterable with more than one entry passed to 'one()'" > except StopIteration: > return v Looks similar to list(islice(iterable,2)) followed by regular list-like handling. > what_i_am_looking_for = one(item for item in items if predicate(item)) Looks similar to: wialf = ifilter(pred, items).next() > This also encodes and checks the assumption that the sought item is unique > within the list of candidates. Again, the assertion part could be turned off > in optimized mode. That is an odd assumption given that you're searching for a predicate match and not a single item match. Also, it is often a better design to enforce uniqueness constraints upon insertion, not upon lookup. Raymond From armin.ronacher at active-4.com Mon May 26 21:32:08 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Mon, 26 May 2008 19:32:08 +0000 (UTC) Subject: [Python-Dev] ABC issues References: Message-ID: Hi, Raymond Hettinger rcn.com> writes: > Curious to hear what Guido thinks about this one. > A nice use of the Mapping ABC is to be able to > get 3.0 behaviors. I thought that was the whole > point of all these backports. If the ABC gets altered, > then it just makes the 2-to-3 conversion harder. If the ABCs exists only for forward compatibility they will be mostly useless for every day usage. Mappings in Python 2.x look different and the ABCs should reflect that. The 2to3 script does replace d.iterkeys with d.keys as similar stuff, so that wouldn't make the conversion any harder. > > * The 2.6 UserDict is not registered as a mapping. > > Since the API's are not currently the same, it makes > sense that UserDict is not registered. Right, but I think the first one should be considered a bug. > If the Mapping ABC does get changed, only IterableUserDict > should get registered. A regular UserDict does not comply. If it's not a mapping it should at least be a container. But a typecheck for isinstance(UserDict, Container) gives an AttributeError currently because the __subclasshook__ doesn't handle old style classes. This is another issue the ABC backport currently has. IMO isinstance under Python 2.x shouldn't perform __*hook__ checks for old style classes. > > * collections.deque isn't registered as a MutableSequence. > > Deque's do not support count(), insert() or __iadd__(). > They should not be registered. General purpose indexing > into a deque is typically a mis-use of the data structure. > It was provided only to make it easier to substitute for lists > in apps the operate only one ends (i.e.d[0], d[1], d[-1], d[-2] > but not d[i] to somewhere in the middle). If it doesn't implement the MutableSequence protocol it still is a Sized container. However currently it's not registered as a container. Another issue is that builtin types don't accept ABCs currently. For example set() | SomeSet() gives a TypeError, SomeSet() | set() however works. Regards, Armin From armin.ronacher at active-4.com Mon May 26 21:40:49 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Mon, 26 May 2008 19:40:49 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?itertools_additions=3A_one=28=29=2C_single?= =?utf-8?b?X3ZhbHVlZCgp?= References: <200805261129.51914.lists@informa.tiker.net> Message-ID: Hi, Andreas Kl?ckner informa.tiker.net> writes: > list_len = single_valued(len(l) for l in lists) length, = set(map(len, lists)) > what_i_am_looking_for = one(item for item in items if predicate(item)) Is that really such a common case? In Python 2.6 you can use next for the same thing however without the check if there is just one item: what_i_am_looking_for = next(filter(predicate, items)) What I would like to see is a batch function defined like that: def batch(iterable, n): return izip(*repeat(iter(iterable), n)) It's a common pattern I use it very often. Regards, Armin From jek-gmane at kleckner.net Mon May 26 21:48:54 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Mon, 26 May 2008 12:48:54 -0700 Subject: [Python-Dev] VS8 include dirs grow without bound Message-ID: I tracked down a testing failure for Cython tests on the Windows platform to be the result of how the Visual C environment variables are being detected. In the function, query_vcvarsall, the .bat file: "C:\Program Files\Microsoft Visual Studio 9.0\VC\vcvarsall.bat" is being run with the arguments: x86 & set This in turn calls the platform-specific file vcvars32.bat (or other). In this file, the PATH, INCLUDE, and LIBPATH environment variables are simply being appended/prepended with their values. initialize() then just overrides the os.environ values with these monotonically growing variables. It seems that duplicate names should be filtered out from these paths to prevent overflow. The attached patch seems to fix this issue, if a bit brute force. Please review as a candidate. ===== @set PATH=%DevEnvDir%;%VCINSTALLDIR%\BIN;%VSINSTALLDIR%\Common7\Tools;%VSINSTALLDIR%\Common7\Tools\bin;%FrameworkDir%\%Framework35Version%;%FrameworkDir%\%Framework35Version%\Microsoft .NET Framework 3.5 (Pre-Release Version);%FrameworkDir%\%FrameworkVersion%;%VCINSTALLDIR%\VCPackages;%PATH% @set INCLUDE=%VCINSTALLDIR%\ATLMFC\INCLUDE;%VCINSTALLDIR%\INCLUDE;%INCLUDE% @set LIB=%VCINSTALLDIR%\ATLMFC\LIB;%VCINSTALLDIR%\LIB;%LIB% @set LIBPATH=%FrameworkDir%\%Framework35Version%;%FrameworkDir%\%FrameworkVersion%;%VCINSTALLDIR%\ATLMFC\LIB;%VCINSTALLDIR%\LIB;%LIBPATH% ===== -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dist.patch URL: From musiccomposition at gmail.com Mon May 26 21:53:24 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Mon, 26 May 2008 14:53:24 -0500 Subject: [Python-Dev] VS8 include dirs grow without bound In-Reply-To: References: Message-ID: <1afaf6160805261253m172aff2dr72f8f83ecfcc56e5@mail.gmail.com> On Mon, May 26, 2008 at 2:48 PM, Jim Kleckner wrote: > I tracked down a testing failure for Cython tests > on the Windows platform to be the result of how the > Visual C environment variables are being detected. Please put this (and it's patch) on the bug tracker: http://bugs.python.org -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From jek-gmane at kleckner.net Mon May 26 21:57:57 2008 From: jek-gmane at kleckner.net (Jim Kleckner) Date: Mon, 26 May 2008 12:57:57 -0700 Subject: [Python-Dev] VS8 include dirs grow without bound In-Reply-To: References: Message-ID: Jim Kleckner wrote: ... > initialize() then just overrides the os.environ values with these > monotonically growing variables. > > It seems that duplicate names should be filtered out from these paths > to prevent overflow. > > The attached patch seems to fix this issue, if a bit brute force. > Please review as a candidate. I created the following tracker item for this: http://bugs.python.org/issue2975 From armin.ronacher at active-4.com Mon May 26 22:03:38 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Mon, 26 May 2008 20:03:38 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Issue_643841=3A_Including_a_new-style_prox?= =?utf-8?q?y_base_class=09in_2=2E6/3=2E0?= References: <4832E144.8030309@gmail.com> Message-ID: Hi, Nick Coghlan gmail.com> writes: > One of the tasks where classic classes are currently far superior to > new-style classes is in writing proxy classes like weakref.proxy - cases > where nearly all operations need to be delegated to some other object, > with only a few being handled via the proxy type object itself. Yes. I stumbled upon that multiple times when writing proxies for thread locals. The solution I came up with is rather ugly but works at least: http://dev.pocoo.org/projects/werkzeug/browser/werkzeug/local.py#L202 The main problem with such proxies is that especially builtin types perform various type checks which break proxies sooner or later anyways (currently even abstract base classes implementing a compatible interface). So as soon as you start doing something like some_list + proxy_to_list you get a TypeError. Regards, Armin From jjl at pobox.com Mon May 26 22:38:43 2008 From: jjl at pobox.com (John J Lee) Date: Mon, 26 May 2008 21:38:43 +0100 (BST) Subject: [Python-Dev] API bug in new socket timeout interfaces Message-ID: I'm hoping to draw somebody's attention to this before it's too late. This affects multiple stdlib modules: http://bugs.python.org/issue2451 """ The new timeout support in 2.6 makes use of new function socket.create_connection(). socket.create_connection() provides no way to disable timeouts, other than by relying on socket.getdefaulttimeout() returning None. This is unfortunate, because it was the purpose of the new timeout support to allow control of timeouts without reliance on global state. """ The author of the patch that introduced this problem (Facundo Batista) has admitted that the behaviour he implemented does not match the discussion about the feature that took place on python-dev, but has not found time to review my patch in the last two months. I fear this broken API will make it into 2.6 (and 3.0?). More could be done here re other issues introduced by the patch (I've listed those that I noticed on issue 2451 -- including at least one other API issue), but the patch attached to issue2451 at least resolves the most clear-cut problem. Thanks John From kirklin.mcdonald at gmail.com Mon May 26 22:59:27 2008 From: kirklin.mcdonald at gmail.com (Kirk McDonald) Date: Mon, 26 May 2008 13:59:27 -0700 Subject: [Python-Dev] Slice as a copy... by design? In-Reply-To: References: Message-ID: <25bd58d10805261359n217987cej7275441d96eea688@mail.gmail.com> If I may, I'd like to describe the D programming language's slice and array semantics. This may seem like a digression (and it probably is), but this thread is already quite lengthy, and it may be interesting to look at a language which implements the slice-by-reference semantics we're talking about here. If you're unfamiliar with D, here are its relevant features: It is statically typed, it is garbage collected, it has dynamic arrays as a builtin type, and strings are simply arrays of characters. (In fact, strings are slightly more complicated than this, as D is fairly Unicode-aware, but this is beside the point.) It also styles itself as a systems programming language, meaning you can play directly with pointers if you wish to. A dynamic array in D is a sort of fat pointer, which looks like this: struct array { size_t length; void* ptr; } That is, it consists of two words, which are passed around by value. The language makes no distinction between an array and a slice of an array. They are both just a pointer and a length. In D, which is garbage collected, any pointer into a GC-allocated chunk of memory will keep that memory alive. Here is some code (its meaning should not be hard to divine): int[] a = [1, 2, 3, 4, 5]; int[] b = a[2..4]; Here we have allocated a dynamic array on the heap, and assigned a slice over the whole array to 'a'. Then we have assigned a sub-slice of the array to 'b'. The important point is that slicing is a very cheap operation in D. If we alter an element of b, the changes will be reflected in a: b[0] = 10; assert(a[2] == 10); (Already this is drifting off topic. Arrays in D are mutable, and strings in Python are not. But please bear with me.) If we get rid of the original reference to the entire array: a = null; Then b will still keep the entire array alive. This is probably the most common criticism of this behavior. If the original array is quite large, then even a single tiny slice of it will keep the entire thing alive. However, these slice semantics can be quite useful. Imagine an XML parser in D. One way you might implement it is to read the entire file into a buffer, then parse out the relevant sub-strings. These sub-strings are slices, and are very cheap to make. The important point here is that, outside of the original buffer, no heap allocations of any sort are made, and this can lead to a very efficient parser. And because this is the language's regular, default behavior, the code which does this looks very natural. This same behavior exists for any similar parsing problem, and can be particularly useful when the application is long-lived, such as an HTTP server parsing HTTP requests. While D's slice semantics are generally nice, I do not necessarily advocate their use in Python, since I don't think there's any good way to get this behavior in Python. (At least not without radically changing it in other ways.) It is still a worthy example of how slice-as-reference semantics can be useful. On Thu, May 22, 2008 at 8:28 AM, Facundo Batista wrote: > Hi! > > A thread in PyAr raised the question that, considering that strings > are immutable, why a slice of a string is a copy and not a reference > to a part of that string. > > I couldn't answer why, so I'm asking here...Is it because the > reference counting will be complicated? Is it because it'd be > inefficient in other way? It's something else? Or is something that > could be done... but is not done yet? > > Thank you very much! > > -- > . Facundo > > Blog: http://www.taniquetil.com.ar/plog/ > PyAr: http://www.python.org/ar/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/kirklin.mcdonald%40gmail.com > From lists at informa.tiker.net Mon May 26 23:18:21 2008 From: lists at informa.tiker.net (Andreas =?iso-8859-1?q?Kl=F6ckner?=) Date: Mon, 26 May 2008 17:18:21 -0400 Subject: [Python-Dev] itertools additions: one(), single_valued() In-Reply-To: References: <200805261129.51914.lists@informa.tiker.net> Message-ID: <200805261718.24981.lists@informa.tiker.net> On Montag 26 Mai 2008, Boris Borcic wrote: > > Opinions? > > > > Andreas > > my_target, = iterable > my_target, = set(iterable) > > quite readably does it, imho, or else one can use > > [my_target] = iterable > [my_target] = set(iterable) > > the error messages seem also OK. True, thanks for pointing this out. :) I guess the only places where my functions are useful are a) if you don't want the intermediate result in a variable or b) if you can't stomach the storage for the set. Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part. URL: From lists at cheimes.de Mon May 26 23:34:58 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 26 May 2008 23:34:58 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483AD138.7000804@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> Message-ID: <483B2D02.8040400@cheimes.de> M.-A. Lemburg schrieb: > Isn't that an awefuly confusing approach ? > > Wouldn't it be better to keep PyString APIs and definitions in > stringobject.c|h > > and only add a new bytesobject.h header file that #defines the > PyBytes APIs in terms of PyString APIs ? That maintains > backwards compatibility and allows Python internals to use the > new API names. > > With your approach, you've basically backported the confusing > notion in Py3k that str() maps PyUnicode, only that in Py2 > str() will now map to PyBytes. The last time I brought up the topic, I had a lengthy discussion with Guido. At first I wanted to rename the API in Python 3.0 only. Guido argued that it's going to cause too much merge conflicts. He then suggested the approach I implemented today. I find the approach less confusing than your suggestion and my initial idea. The internal API names are consistent for Python 2.6 and 3.0. The byte string C API is prefixed PyBytes and the unicode C API is prefixed PyUnicode. A core developer has just to remember that 'str' is a byte string in 2.x but an unicode object in 3.0. Extension developers don't have to worry at all. The ABI and external API is mostly the same and still exposes the 'str' functions as PyString. > You'd have to add an aliase bytes -> str to the builtins to > at least reduce the confusion a bit. Python 2.6 already has an alias bytes -> str > Yes, but please let's first discuss this some more. I don't think > that the timing was right.... you started this thread just yesterday > and the patches are already checked in. I'm sorry if I was too hasty for you. I got +1 from a couple of developers and it's basically Guido's suggestion. Christian From lists at cheimes.de Tue May 27 00:43:16 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 27 May 2008 00:43:16 +0200 Subject: [Python-Dev] optimization required: .format() is much slower than % Message-ID: Dear fellow Python developers! Ten minutes ago I raised a concern about speed differences between the old style % formatting and the new .format() code. Some quick benchmarking from Benjamin and me showed, that it's even worse than I expected. $ ./python -m timeit "'%s'.format('nothing')" 100000 loops, best of 3: 2.63 usec per loop $ ./python -m timeit "'%s' % 'nothing'" 10000000 loops, best of 3: 0.188 usec per loop $ ./python -m timeit "'some text with {0}'.format('nothing')" 100000 loops, best of 3: 4.34 usec per loop $ ./python -m timeit "'some text with %s' % 'nothing'" 100000 loops, best of 3: 2.04 usec per loop $ ./python -m timeit "'some text with {0} {1}'.format('nothing', 'more')" 100000 loops, best of 3: 6.77 usec per loop $ ./python -m timeit "'some text with %s %s' % ('nothing', 'more')" 100000 loops, best of 3: 2.22 usec per loop As you can clearly see the new .format() code is *much* slower than the old style % code. I recommend we spend some time on optimizing common code paths of the new .format() code. As first step I propose the move the __format__ method to a new type slot. __format__() is called for every object. My gut feeling says that a slot method is going to speed up the most common usage "{0}".format(some_string). Christian From python at rcn.com Tue May 27 01:11:30 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 26 May 2008 16:11:30 -0700 Subject: [Python-Dev] ABC issues References: Message-ID: <9A7CBBA4962140CA93089B46F97F8548@RaymondLaptop1> From: "Armin Ronacher" > If the ABCs exists only for forward compatibility they will be mostly useless > for every day usage. The Mapping ABC is still useful as a mixin in code that will run on both 2.6 and 3.0. It is the easiest way to start coding with views. >> Deque's do not support count(), insert() or __iadd__(). >> They should not be registered. > If it doesn't implement the MutableSequence protocol it still is a Sized > container. However currently it's not registered as a container. Seems useless to me. I don't think the intent of the ABC pep was to mandate that every class that defines __len__ must be registered as Sized. > Another issue is that builtin types don't accept ABCs currently. For example > set() | SomeSet() gives a TypeError, SomeSet() | set() however works. Pandora's Box -- sure you want to open it? Raymond From greg at krypto.org Tue May 27 02:17:31 2008 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 26 May 2008 17:17:31 -0700 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: References: Message-ID: <52dc1c820805261717t3842df91ubc44111872d19b98@mail.gmail.com> On Mon, May 26, 2008 at 3:43 PM, Christian Heimes wrote: > Dear fellow Python developers! > > Ten minutes ago I raised a concern about speed differences between the > old style % formatting and the new .format() code. Some quick > benchmarking from Benjamin and me showed, that it's even worse than I > expected. > > $ ./python -m timeit "'%s'.format('nothing')" > 100000 loops, best of 3: 2.63 usec per loop > $ ./python -m timeit "'%s' % 'nothing'" > 10000000 loops, best of 3: 0.188 usec per loop > > $ ./python -m timeit "'some text with {0}'.format('nothing')" > 100000 loops, best of 3: 4.34 usec per loop > $ ./python -m timeit "'some text with %s' % 'nothing'" > 100000 loops, best of 3: 2.04 usec per loop > > $ ./python -m timeit "'some text with {0} {1}'.format('nothing', 'more')" > 100000 loops, best of 3: 6.77 usec per loop > $ ./python -m timeit "'some text with %s %s' % ('nothing', 'more')" > 100000 loops, best of 3: 2.22 usec per loop > > As you can clearly see the new .format() code is *much* slower than the > old style % code. I recommend we spend some time on optimizing common > code paths of the new .format() code. > > As first step I propose the move the __format__ method to a new type > slot. __format__() is called for every object. My gut feeling says that > a slot method is going to speed up the most common usage > "{0}".format(some_string). My gut feels the same way... How about seeing the profile data of the new vs old string formatting functions first as a comparison. If those disagree vs the above then the ".format()" name lookup and function call overhead is likely consuming the time missing from a profile. -gps From facundobatista at gmail.com Tue May 27 02:28:45 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Mon, 26 May 2008 21:28:45 -0300 Subject: [Python-Dev] API bug in new socket timeout interfaces In-Reply-To: References: Message-ID: 2008/5/26 John J Lee : > The author of the patch that introduced this problem (Facundo Batista) has > admitted that the behaviour he implemented does not match the discussion > about the feature that took place on python-dev, but has not found time to > review my patch in the last two months. I fear this broken API will make it > into 2.6 (and 3.0?). No, I'll review it in less than two weeks. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From eric+python-dev at trueblade.com Tue May 27 03:04:47 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Mon, 26 May 2008 21:04:47 -0400 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <52dc1c820805261717t3842df91ubc44111872d19b98@mail.gmail.com> References: <52dc1c820805261717t3842df91ubc44111872d19b98@mail.gmail.com> Message-ID: <483B5E2F.6070308@trueblade.com> Gregory P. Smith wrote: > My gut feels the same way... How about seeing the profile data of the > new vs old string formatting functions first as a comparison. If > those disagree vs the above then the ".format()" name lookup and > function call overhead is likely consuming the time missing from a > profile. Because it's a more general and extensible mechanism, .format() will never be as fast as %-formatting. But I agree it could no doubt be improved. When implementing this, my gut feelings were that the performance difference is likely: - "__format__" lookup. - Each __format__() call returns an object. The overhead is non-trivial, compared to the hard coded types in %-formatting. Not that this means anything without actual profiling. Here's about the best direct comparison I could think of: $ ./python -m timeit "'nothing'.__format__('')" 1000000 loops, best of 3: 1.19 usec per loop $ ./python -m timeit "'{0}'.format('nothing')" 100000 loops, best of 3: 3.74 usec per loop Shows there is some overhead to the "format" lookup and result computation, but it's still much slower than: $ ./python -m timeit "'%s' % 'nothing'" 1000000 loops, best of 3: 0.223 usec per loop From ncoghlan at gmail.com Tue May 27 04:24:58 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 May 2008 12:24:58 +1000 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: References: <4832E144.8030309@gmail.com> Message-ID: <483B70FA.2080807@gmail.com> Armin Ronacher wrote: > Hi, > > Nick Coghlan gmail.com> writes: > >> One of the tasks where classic classes are currently far superior to >> new-style classes is in writing proxy classes like weakref.proxy - cases >> where nearly all operations need to be delegated to some other object, >> with only a few being handled via the proxy type object itself. > Yes. I stumbled upon that multiple times when writing proxies for thread > locals. The solution I came up with is rather ugly but works at least: > > http://dev.pocoo.org/projects/werkzeug/browser/werkzeug/local.py#L202 > > The main problem with such proxies is that especially builtin types perform > various type checks which break proxies sooner or later anyways (currently even > abstract base classes implementing a compatible interface). So as soon as you > start doing something like some_list + proxy_to_list you get a TypeError. What version are you using, and are your proxies correctly implementing all the __r*__ versions of the methods? While there are still some cases where types in the standard library raise TypeError directly instead of returning NotImplemented, they're generally pretty good about playing well with others (see the test_typetools.py file attached to the tracker item for #643841) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From tjreedy at udel.edu Tue May 27 07:22:13 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 27 May 2008 01:22:13 -0400 Subject: [Python-Dev] optimization required: .format() is much slowerthan % References: <52dc1c820805261717t3842df91ubc44111872d19b98@mail.gmail.com> Message-ID: "Gregory P. Smith" wrote in message news:52dc1c820805261717t3842df91ubc44111872d19b98 at mail.gmail.com... | On Mon, May 26, 2008 at 3:43 PM, Christian Heimes wrote: | > Dear fellow Python developers! | > | > Ten minutes ago I raised a concern about speed differences between the | > old style % formatting and the new .format() code. Some quick | > benchmarking from Benjamin and me showed, that it's even worse than I | > expected. | > | > $ ./python -m timeit "'%s'.format('nothing')" | > 100000 loops, best of 3: 2.63 usec per loop | > $ ./python -m timeit "'%s' % 'nothing'" | > 10000000 loops, best of 3: 0.188 usec per loop | > | > $ ./python -m timeit "'some text with {0}'.format('nothing')" | > 100000 loops, best of 3: 4.34 usec per loop | > $ ./python -m timeit "'some text with %s' % 'nothing'" | > 100000 loops, best of 3: 2.04 usec per loop | > | > $ ./python -m timeit "'some text with {0} {1}'.format('nothing', 'more')" | > 100000 loops, best of 3: 6.77 usec per loop | > $ ./python -m timeit "'some text with %s %s' % ('nothing', 'more')" | > 100000 loops, best of 3: 2.22 usec per loop | > | > As you can clearly see the new .format() code is *much* slower than the | > old style % code. I recommend we spend some time on optimizing common | > code paths of the new .format() code. | > | > As first step I propose the move the __format__ method to a new type | > slot. __format__() is called for every object. My gut feeling says that | > a slot method is going to speed up the most common usage | > "{0}".format(some_string). | | My gut feels the same way... How about seeing the profile data of the | new vs old string formatting functions first as a comparison. If | those disagree vs the above then the ".format()" name lookup In real usages, the format string is typically a constant. To mirror this, the .format name lookup and creation of the bound method object could be factored out of the testing loop and put in a setup string. IOW, I write code like emsg = 'Bad input {0}'.format for i in inputs: if squiggle(i): emsg(i) else: realwork(i) However, if I have done the following correctly (my first use of timeit), this seems not to be the slowdown. >>> t=timeit.Timer(stmt="'{0}'.format(None)") >>> t.timeit() 3.769231005616632 >>> t2=timeit.Timer(setup="m='{0}'.format", stmt='m(None)') >>> t2.timeit() 3.7023311622007213 tjr From ncoghlan at gmail.com Tue May 27 11:14:54 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 May 2008 19:14:54 +1000 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: References: Message-ID: <483BD10E.7050600@gmail.com> Following Terry's suggestion, I repeated Christian's tests with the lookup of the format method taken outside the loop (since one advantage of the new approach is to make that micro-optimisation easier), and also tried a few additional examples. First, there seems to be about a 0.25 microsecond hit (on my machine) which reflects the impact of the lookup of the "format" attribute itself. Secondly, the string % operator appears to have an explicit optimisation for the 'just return str(self)' case. This optimisation is missing from the new string format method. The only optimisation idea I came up with (other than the addition of a tp_format slot) is for string objects to be able to cache their (lookup:subformat) pairs rather than having to parse themselves every time. That has obvious memory consumption implications though. Cheers, Nick. Test results: # No formatting at all, just the method invocation: $ ./python -m timeit "''.format()" 1000000 loops, best of 3: 0.635 usec per loop $ ./python -m timeit -s "fmt = '%s'.format" "fmt()" 1000000 loops, best of 3: 0.48 usec per loop $ ./python -m timeit "'' % ()" 1000000 loops, best of 3: 0.389 usec per loop # Format one argument by position $ ./python -m timeit "'{0}'.format('nothing')" 1000000 loops, best of 3: 1.56 usec per loop $ ./python -m timeit -s "fmt = '{0}'.format" "fmt('nothing')" 1000000 loops, best of 3: 1.44 usec per loop $ ./python -m timeit "'%s' % 'nothing'" 10000000 loops, best of 3: 0.0736 usec per loop # Format one non-string argument by position $ ./python -m timeit "'{0}'.format(0)" 100000 loops, best of 3: 1.74 usec per loop $ ./python -m timeit -s "fmt = '{0}'.format" "fmt(0)" 1000000 loops, best of 3: 1.51 usec per loop $ ./python -m timeit "'%s' % 0" 10000000 loops, best of 3: 0.0987 usec per loop # Format one argument by position with additional text $ ./python -m timeit "'some text with {0}'.format('nothing')" 1000000 loops, best of 3: 1.7 usec per loop $ ./python -m timeit -s "fmt = 'some text with {0}'.format" "fmt('nothing')" 1000000 loops, best of 3: 1.45 usec per loop $ ./python -m timeit "'some text with %s' % 'nothing'" 1000000 loops, best of 3: 0.612 usec per loop # Format two arguments by position with additional text $ ./python -m timeit "'some text with {0} {1}'.format('nothing', 'more')" 100000 loops, best of 3: 2.63 usec per loop $ ./python -m timeit -s "fmt = 'some text with {0} {1}'.format" "fmt('nothing', 'more')" 100000 loops, best of 3: 2.31 usec per loop $ ./python -m timeit "'some text with %s %s' % ('nothing', 'more')" 1000000 loops, best of 3: 0.804 usec per loop # Format two non-string arguments by position with additional text $ ./python -m timeit "'some text with {0} {1}'.format(0, 1)" 100000 loops, best of 3: 2.81 usec per loop $ ./python -m timeit -s "fmt = 'some text with {0} {1}'.format" "fmt(0, 1)" 100000 loops, best of 3: 2.46 usec per loop $ ./python -m timeit "'some text with %s %s' % (0, 1)" 1000000 loops, best of 3: 0.851 usec per loop # Format two non-string arguments by name with additional text $ ./python -m timeit -s "d = dict(a=0, b=1)" "'some text with {a} {b}'.format(**d)" 100000 loops, best of 3: 2.81 usec per loop $ ./python -m timeit -s "d = dict(a=0, b=1)" -s "fmt = 'some text with {a} {b}'.format" "fmt(**d)" 100000 loops, best of 3: 2.58 usec per loop $ ./python -m timeit -s "d = dict(a=0, b=1)" "'some text with %(a)s %(b)s' % (d)" 1000000 loops, best of 3: 1 usec per loop -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mal at egenix.com Tue May 27 12:10:25 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 27 May 2008 12:10:25 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483B2D02.8040400@cheimes.de> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> Message-ID: <483BDE11.509@egenix.com> On 2008-05-26 23:34, Christian Heimes wrote: > M.-A. Lemburg schrieb: >> Isn't that an awefuly confusing approach ? >> >> Wouldn't it be better to keep PyString APIs and definitions in >> stringobject.c|h >> >> and only add a new bytesobject.h header file that #defines the >> PyBytes APIs in terms of PyString APIs ? That maintains >> backwards compatibility and allows Python internals to use the >> new API names. >> >> With your approach, you've basically backported the confusing >> notion in Py3k that str() maps PyUnicode, only that in Py2 >> str() will now map to PyBytes. > > The last time I brought up the topic, I had a lengthy discussion with > Guido. At first I wanted to rename the API in Python 3.0 only. Guido > argued that it's going to cause too much merge conflicts. He then > suggested the approach I implemented today. That's the same argument that came up in the module renaming discussion. I have a feeling that we should be looking for better merge tools, rather than implement code changes that cause more trouble than do good, just because our existing tools aren't smart enough. Wouldn't it be possible to have a 2to3.py converter take the 2.x code (including the C code), convert it and then apply any changes to the 3.x branch ? This wouldn't be merging in the classical sense, it would be automated forward porting. > I find the approach less confusing than your suggestion and my initial > idea. I disagree on that. Renaming old APIs to use the new names by adding a header file with #define is standard practice. Renaming the old APIs in the source code and undoing the renaming with a header file is not. > The internal API names are consistent for Python 2.6 and 3.0. The > byte string C API is prefixed PyBytes and the unicode C API is prefixed > PyUnicode. A core developer has just to remember that 'str' is a byte > string in 2.x but an unicode object in 3.0. So you've solved part of the problem for 3.x by moving the naming mixup back to 2.x. > Extension developers don't have to worry at all. The ABI and external > API is mostly the same and still exposes the 'str' functions as PyString. Well, yes, but only due to a preprocessor hack that turns the names used in bytesobject.c back into names you'd normally look for in stringobject.c. And all this, just because Subversion can't handle merging of symbol renaming. >> You'd have to add an aliase bytes -> str to the builtins to >> at least reduce the confusion a bit. > > Python 2.6 already has an alias bytes -> str > >> Yes, but please let's first discuss this some more. I don't think >> that the timing was right.... you started this thread just yesterday >> and the patches are already checked in. > > I'm sorry if I was too hasty for you. I got +1 from a couple of > developers and it's basically Guido's suggestion. Please discuss any changes of the 2.x code base on python-dev. Such major changes do need more discussion and possibly a PEP as well. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 27 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 40 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From solipsis at pitrou.net Tue May 27 12:12:53 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 May 2008 10:12:53 +0000 (UTC) Subject: [Python-Dev] optimization required: .format() is much slower than % References: <483BD10E.7050600@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > > The only optimisation idea I came up with (other than the addition of a > tp_format slot) is for string objects to be able to cache their > (lookup:subformat) pairs rather than having to parse themselves every > time. That has obvious memory consumption implications though. If a string is used as a formatting template, it is bound to be used again as such. So IMHO it sounds quite reasonable. In order to avoid memory consumption issues there could be a centralized cache as for regular expressions. It makes it easier to handle eviction based on various parameters, and it saves a few bytes for string objects which are never used as a formatting template. Regards Antoine. From bborcic at gmail.com Tue May 27 12:13:41 2008 From: bborcic at gmail.com (Boris Borcic) Date: Tue, 27 May 2008 12:13:41 +0200 Subject: [Python-Dev] itertools additions: one(), single_valued() In-Reply-To: <200805261718.24981.lists@informa.tiker.net> References: <200805261129.51914.lists@informa.tiker.net> <200805261718.24981.lists@informa.tiker.net> Message-ID: Andreas Kl?ckner wrote: > On Montag 26 Mai 2008, Boris Borcic wrote: (...) >> >> [my_target] = iterable >> [my_target] = set(iterable) >> >> the error messages seem also OK. > > True, thanks for pointing this out. :) > > I guess the only places where my functions are useful are > > a) if you don't want the intermediate result in a variable On the near side to ugly, one could attempt the expression equivalents: (lambda x:x)(*iterable) (lambda x:x)(*set(iterable)) > or > b) if you can't stomach the storage for the set. Yeah, I guess it deserves study. Cheers, BB From mal at egenix.com Tue May 27 13:54:26 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 27 May 2008 13:54:26 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> Message-ID: <483BF672.6020108@egenix.com> On 2008-05-27 12:56, Thomas Heller wrote: > M.-A. Lemburg schrieb: >> On 2008-05-26 23:34, Christian Heimes wrote: >>> M.-A. Lemburg schrieb: >>>> Isn't that an awefuly confusing approach ? >>>> >>>> Wouldn't it be better to keep PyString APIs and definitions in >>>> stringobject.c|h and only add a new bytesobject.h header file >>>> that #defines the PyBytes APIs in terms of PyString APIs ? That >>>> maintains backwards compatibility and allows Python internals to >>>> use the new API names. >>>> >>>> With your approach, you've basically backported the confusing >>>> notion in Py3k that str() maps PyUnicode, only that in Py2 str() >>>> will now map to PyBytes. >>> The last time I brought up the topic, I had a lengthy discussion >>> with Guido. At first I wanted to rename the API in Python 3.0 only. >>> Guido argued that it's going to cause too much merge conflicts. He >>> then suggested the approach I implemented today. >> That's the same argument that came up in the module renaming >> discussion. >> >> I have a feeling that we should be looking for better merge tools, >> rather than implement code changes that cause more trouble than do >> good, just because our existing tools aren't smart enough. > > There are applications out there that dynamically import the python dll > and link to the exported functions by name; they will all break. The exported APIs still use the old names. Just the source code versions of the APIs change to the new names and they now live in different files as well. > I believe in the past we have been more carefull with changes like these. > Even when python api functions were turned into cpp macros, we provided > exported functions for them; for a few examples see the function definitions > near line 1778 in file Python/pythonrun.c . IMO, we should keep using that strategy for Python 2.x. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 27 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 40 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From armin.ronacher at active-4.com Tue May 27 14:18:39 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Tue, 27 May 2008 12:18:39 +0000 (UTC) Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 References: <4832E144.8030309@gmail.com> <483B70FA.2080807@gmail.com> Message-ID: Hi, Nick Coghlan gmail.com> writes: > What version are you using, and are your proxies correctly implementing > all the __r*__ versions of the methods? The link to the ugly proxy is in the mail :-) And no, I'm currently not providing any __r*__ methods as I was too lazy to test on each call if the method that is proxied is providing an __rsomething__ or not, and if not come up with an ad-hoc implementation by calling __something__ and reversing the arguments passed. > While there are still some cases where types in the standard library > raise TypeError directly instead of returning NotImplemented, they're > generally pretty good about playing well with others (see the > test_typetools.py file attached to the tracker item for #643841) I also think that the stdlib should mention NotImplemented with a big warning. I see countless classes raising TypeError()s if __add__ or something fails which seem to work alright as long as someone tries to __radd__ it. Regards, Armin From jcea at jcea.es Tue May 27 14:38:22 2008 From: jcea at jcea.es (Jesus Cea) Date: Tue, 27 May 2008 14:38:22 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483BF672.6020108@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483BF672.6020108@egenix.com> Message-ID: <483C00BE.2040005@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 M.-A. Lemburg wrote: | The exported APIs still use the old names. Just the source code | versions of the APIs change to the new names and they now live | in different files as well. Do we need to change "PYTHON_API_VERSION"?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSDwAt5lgi5GaxT1NAQKvUwP+Lc605lolQAwcOc9lgM37m91lIqbBopws H7nzQqYjkQFISmXMNzeFwEzo6HPcLv0Phw1ghXF3zE1rQE6APcJX8CG5nMsvjSRO XTW4fKcnVEEcH0NH9WgSjp+v28gMOsfE4ppC1rjhYiJ5M3gIfVlFXrsNeboycvlf L0YM998L5Bw= =seNU -----END PGP SIGNATURE----- From theller at ctypes.org Tue May 27 14:47:15 2008 From: theller at ctypes.org (Thomas Heller) Date: Tue, 27 May 2008 14:47:15 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483BF672.6020108@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483BF672.6020108@egenix.com> Message-ID: M.-A. Lemburg schrieb: > On 2008-05-27 12:56, Thomas Heller wrote: >> M.-A. Lemburg schrieb: >>> On 2008-05-26 23:34, Christian Heimes wrote: >>>> M.-A. Lemburg schrieb: >>>>> Isn't that an awefuly confusing approach ? >>>>> >>>>> Wouldn't it be better to keep PyString APIs and definitions in >>>>> stringobject.c|h and only add a new bytesobject.h header file >>>>> that #defines the PyBytes APIs in terms of PyString APIs ? That >>>>> maintains backwards compatibility and allows Python internals to >>>>> use the new API names. >>>>> >>>>> With your approach, you've basically backported the confusing >>>>> notion in Py3k that str() maps PyUnicode, only that in Py2 str() >>>>> will now map to PyBytes. >>>> The last time I brought up the topic, I had a lengthy discussion >>>> with Guido. At first I wanted to rename the API in Python 3.0 only. >>>> Guido argued that it's going to cause too much merge conflicts. He >>>> then suggested the approach I implemented today. >>> That's the same argument that came up in the module renaming >>> discussion. >>> >>> I have a feeling that we should be looking for better merge tools, >>> rather than implement code changes that cause more trouble than do >>> good, just because our existing tools aren't smart enough. >> >> There are applications out there that dynamically import the python dll >> and link to the exported functions by name; they will all break. > > The exported APIs still use the old names. Just the source code > versions of the APIs change to the new names and they now live > in different files as well. > Sorry, then, I confused it with py3k. Thomas From ncoghlan at gmail.com Tue May 27 14:56:51 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 May 2008 22:56:51 +1000 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: References: <4832E144.8030309@gmail.com> <483B70FA.2080807@gmail.com> Message-ID: <483C0513.7040903@gmail.com> Armin Ronacher wrote: > Hi, > > Nick Coghlan gmail.com> writes: > >> What version are you using, and are your proxies correctly implementing >> all the __r*__ versions of the methods? > The link to the ugly proxy is in the mail :-) And no, I'm currently not > providing any __r*__ methods as I was too lazy to test on each call if the > method that is proxied is providing an __rsomething__ or not, and if not come up > with an ad-hoc implementation by calling __something__ and reversing the > arguments passed. I had another look - indeed, it is almost certainly the missing __r*__ methods which are causing problems for you when interacting with unproxied objects. The other differences between what you're doing and the proposed typetools.ProxyMixin are that: - ProxyMixin delegates the whole __getattribute__ operation to the target object so that descriptors work correctly (invoking object.__getattribute__ explicitly when it needs to retrieve the proxy object's _target attribute). - ProxyMixin correctly delegates in-place assignment operations via the __i*__ methods (note however that using in-place assignment will remove the proxy wrapper, just as it does for weakref.proxy) - ProxyMixin implicitly unwraps other ProxyMixin instances when it encounters them as an argument to an explicitly proxied operation - ProxyMixin doesn't support deprecated operations like __getslice__ or __methods__ > >> While there are still some cases where types in the standard library >> raise TypeError directly instead of returning NotImplemented, they're >> generally pretty good about playing well with others (see the >> test_typetools.py file attached to the tracker item for #643841) > I also think that the stdlib should mention NotImplemented with a big warning. > I see countless classes raising TypeError()s if __add__ or something fails which > seem to work alright as long as someone tries to __radd__ it. Returning NotImplemented is already mentioned in the documentation for the binary special methods, but I agree that it could be made more obvious that explicitly raising a TypeError from these methods is almost certainly the wrong thing to do. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From fuzzyman at voidspace.org.uk Tue May 27 15:09:28 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 27 May 2008 14:09:28 +0100 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <483C0513.7040903@gmail.com> References: <4832E144.8030309@gmail.com> <483B70FA.2080807@gmail.com> <483C0513.7040903@gmail.com> Message-ID: <483C0808.2030808@voidspace.org.uk> Nick Coghlan wrote: > Armin Ronacher wrote: >> Hi, >> >> Nick Coghlan gmail.com> writes: >> >>> What version are you using, and are your proxies correctly >>> implementing all the __r*__ versions of the methods? >> The link to the ugly proxy is in the mail :-) And no, I'm currently not >> providing any __r*__ methods as I was too lazy to test on each call >> if the >> method that is proxied is providing an __rsomething__ or not, and if >> not come up >> with an ad-hoc implementation by calling __something__ and reversing the >> arguments passed. > > I had another look - indeed, it is almost certainly the missing __r*__ > methods which are causing problems for you when interacting with > unproxied objects. > > The other differences between what you're doing and the proposed > typetools.ProxyMixin are that: > - ProxyMixin delegates the whole __getattribute__ operation to the > target object so that descriptors work correctly (invoking > object.__getattribute__ explicitly when it needs to retrieve the proxy > object's _target attribute). > - ProxyMixin correctly delegates in-place assignment operations via > the __i*__ methods (note however that using in-place assignment will > remove the proxy wrapper, just as it does for weakref.proxy) Couldn't in place operations wrap the return value with a proxy? Michael Foord > - ProxyMixin implicitly unwraps other ProxyMixin instances when it > encounters them as an argument to an explicitly proxied operation > - ProxyMixin doesn't support deprecated operations like __getslice__ > or __methods__ > > >> >>> While there are still some cases where types in the standard library >>> raise TypeError directly instead of returning NotImplemented, >>> they're generally pretty good about playing well with others (see >>> the test_typetools.py file attached to the tracker item for #643841) >> I also think that the stdlib should mention NotImplemented with a big >> warning. I see countless classes raising TypeError()s if __add__ or >> something fails which >> seem to work alright as long as someone tries to __radd__ it. > > Returning NotImplemented is already mentioned in the documentation for > the binary special methods, but I agree that it could be made more > obvious that explicitly raising a TypeError from these methods is > almost certainly the wrong thing to do. > > Cheers, > Nick. > From ncoghlan at gmail.com Tue May 27 15:32:38 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 27 May 2008 23:32:38 +1000 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <483C0808.2030808@voidspace.org.uk> References: <4832E144.8030309@gmail.com> <483B70FA.2080807@gmail.com> <483C0513.7040903@gmail.com> <483C0808.2030808@voidspace.org.uk> Message-ID: <483C0D76.6050507@gmail.com> Michael Foord wrote: > Nick Coghlan wrote: >> - ProxyMixin correctly delegates in-place assignment operations via >> the __i*__ methods (note however that using in-place assignment will >> remove the proxy wrapper, just as it does for weakref.proxy) > > Couldn't in place operations wrap the return value with a proxy? They could, but I tried to copy as much behaviour as possible from weakref.proxy which loses the proxy wrapper when you do an in-place operation: >>> from weakref import proxy as p >>> class MyInt(int): pass ... >>> x = MyInt(1) >>> px = p(x) >>> px >>> px += 1 >>> px 2 >>> class MyList(list): pass ... >>> y = MyList([]) >>> py = p(y) >>> py >>> py += [] >>> py [] That said, it wouldn't be hard (and might make more sense) to redefine the proxied in-place operations along the following lines: # See tracker item for _deref/_unwrap definitions def __iadd__(self, other): target = _deref(self) result = target + _unwrap(other) if result is target: # Mutated in-place, keep same proxy result = self else: # Returned a different object, make a new proxy result = type(self)(result) return result However, if we did something like that, I'd prefer to see the weakref.proxy implementation change to provide similar semantics. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From dalcinl at gmail.com Tue May 27 15:39:02 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Tue, 27 May 2008 10:39:02 -0300 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <48397ECC.9070805@cheimes.de> References: <48397ECC.9070805@cheimes.de> Message-ID: Chistian, I've posted some weeks ago some observation about the status of PyNumberMethods API. The thread link is below, I t did not received much atention. http://mail.python.org/pipermail/python-3000/2008-May/013594.html Now I sumarize that post * 'nb_nonzero' was renamed to 'nb_bool' * 'nb_inplace_divide' was removed * 'nb_hex', 'nb_oct', and 'nb_coerce' are there, but they are unused IMHO, the PyNumbersMethods struct should be left as in Py2, or it should be cleaned up, that is, all unused slots should be removed. On 5/25/08, Christian Heimes wrote: > Hello! > > The first set of betas of Python 2.6 and 3.0 is fast apace. I like to > grab the final chance and clean up the C API of 2.6 and 3.0. I know, I > know, I brought up the topic two times in the past. But this time I mean > it for real! :] > > Last time Guido said: > --- > I think it can actually be simplified. I think maintaining binary > compatibility between 2.6 and earlier versions is hopeless anyway, so > we might as well just rename PyString to PyBytes in 2.6 and 3.0, and > have an extra set of macros so that code using PyString needs to be > recompiled but not otherwise touched. E.g. > > typedef { ... } PyBytesObject; > #define PyStringObject PyBytesObject > > ... PyString_Type; > #define PyBytes_Type PyString_Type > > > --- > > I like to follow Guido's advice and change the code as following: > > * replace PyBytes_ with PyByteArray_ > * replace PyString with PyBytes_ > * rename bytesobject.[ch] to bytearrayobject.[ch] > * rename stringobject.[ch] to bytesobject.[ch] > * add a new file stringobject.h which contains the aliases PyString_ -> > PyBytes_ > > Christian > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/dalcinl%40gmail.com > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From eric+python-dev at trueblade.com Tue May 27 16:04:01 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 27 May 2008 10:04:01 -0400 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <483BD10E.7050600@gmail.com> References: <483BD10E.7050600@gmail.com> Message-ID: <483C14D1.1050007@trueblade.com> Nick Coghlan wrote: > Secondly, the string % operator appears to have an explicit optimisation > for the 'just return str(self)' case. This optimisation is missing from > the new string format method. I'll see if I can optimize this case. From eric+python-dev at trueblade.com Tue May 27 17:13:57 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 27 May 2008 11:13:57 -0400 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <483C14D1.1050007@trueblade.com> References: <483BD10E.7050600@gmail.com> <483C14D1.1050007@trueblade.com> Message-ID: <483C2535.2040306@trueblade.com> Eric Smith wrote: > Nick Coghlan wrote: >> Secondly, the string % operator appears to have an explicit >> optimisation for the 'just return str(self)' case. This optimisation >> is missing from the new string format method. > > I'll see if I can optimize this case. 3.0, from svn: $ ./python.exe -m timeit "'some text with {0}'.format('nothing')" 100000 loops, best of 3: 3.14 usec per loop 3.0, with PyUnicode_ExactCheck, skipping __format__ lookup: $ ./python.exe -m timeit "'some text with {0}'.format('nothing')" 100000 loops, best of 3: 2.32 usec per loop I could probably improve this some more, by skipping the creation of the object used to describe the format_spec. Compare with: $ ./python.exe -m timeit "'some text with %s' % 'nothing'" 1000000 loops, best of 3: 1.37 usec per loop From greg at krypto.org Tue May 27 18:38:24 2008 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 27 May 2008 09:38:24 -0700 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: References: <483BD10E.7050600@gmail.com> Message-ID: <52dc1c820805270938t147d6aeekb56de06fa58c1b47@mail.gmail.com> On Tue, May 27, 2008 at 3:12 AM, Antoine Pitrou wrote: > Nick Coghlan gmail.com> writes: >> >> The only optimisation idea I came up with (other than the addition of a >> tp_format slot) is for string objects to be able to cache their >> (lookup:subformat) pairs rather than having to parse themselves every >> time. That has obvious memory consumption implications though. > > If a string is used as a formatting template, it is bound to be used again as > such. So IMHO it sounds quite reasonable. > > In order to avoid memory consumption issues there could be a centralized cache > as for regular expressions. It makes it easier to handle eviction based on > various parameters, and it saves a few bytes for string objects which are never > used as a formatting template. > > Regards > > Antoine. +1 to that idea. From stefan_ml at behnel.de Tue May 27 18:47:13 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 27 May 2008 18:47:13 +0200 Subject: [Python-Dev] Why is type_modified() in typeobject.c not a public function? Message-ID: [reposting this to python-dev, as it affects both 2.6 and 3.0] Hi, when we build extension classes in Cython, we have to first build the type to make it available to user code, and then update the type's tp_dict while we run the class body code (PyObject_SetAttr() does not work here). In Py2.6+, this requires invalidating the method cache after each attribute change, which Python does internally using the type_modified() function. Could this function get a public interface? I do not think Cython is the only case where C code wants to modify a type after its creation, and copying the code over seems like a hack to me. Stefan From guido at python.org Tue May 27 19:26:46 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 27 May 2008 10:26:46 -0700 Subject: [Python-Dev] ABC issues In-Reply-To: References: Message-ID: On Mon, May 26, 2008 at 11:59 AM, Raymond Hettinger wrote: >> * The 2.6-backported Mapping ABC has the 3.0 dict API, >> that is, it uses keys() that returns a view etc. > > Curious to hear what Guido thinks about this one. > A nice use of the Mapping ABC is to be able to > get 3.0 behaviors. I thought that was the whole > point of all these backports. If the ABC gets altered, > then it just makes the 2-to-3 conversion harder. It's wrong if the ABC doesn't describe the behavior of actual implementations; that is its primary purpose, the mixin class is a nice side benefit. We could make the incompatible mixin classes available separately though, if you think they're useful. >> * The 2.6 UserDict is not registered as a mapping. > > Since the API's are not currently the same, it makes > sense that UserDict is not registered. > If the Mapping ABC does get changed, only IterableUserDict > should get registered. A regular UserDict does not comply. Fair enough. I recomment to fix the Mapping ABC and register IterableUserDict. >> * collections.deque isn't registered as a MutableSequence. > > Deque's do not support count(), insert() or __iadd__(). > They should not be registered. General purpose indexing > into a deque is typically a mis-use of the data structure. > It was provided only to make it easier to substitute for lists > in apps the operate only one ends (i.e.d[0], d[1], d[-1], d[-2] > but not d[i] to somewhere in the middle). Hopefully they aren't registered in 3.0 either. :-) >> If there are no objections, I will correct these issues >> in the 2.6 and 3.0 branches. > > I think none of these changes should be made. I'm in the middle. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 27 19:33:26 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 27 May 2008 10:33:26 -0700 Subject: [Python-Dev] ABC issues In-Reply-To: <9A7CBBA4962140CA93089B46F97F8548@RaymondLaptop1> References: <9A7CBBA4962140CA93089B46F97F8548@RaymondLaptop1> Message-ID: On Mon, May 26, 2008 at 4:11 PM, Raymond Hettinger wrote: >>> Deque's do not support count(), insert() or __iadd__(). >>> They should not be registered. > >> If it doesn't implement the MutableSequence protocol it still is a Sized >> container. However currently it's not registered as a container. > > Seems useless to me. I don't think the intent of the ABC pep was > to mandate that every class that defines __len__ must be registered > as Sized. There's no need to register as Sized -- the Sized ABC recognizes classes that define __len__ automatically. The Container class does the same looking for __contains__. Since the deque class doesn't implement __contains__, it is not considered a Container -- correctly IMO. >> Another issue is that builtin types don't accept ABCs currently. For >> example >> set() | SomeSet() gives a TypeError, SomeSet() | set() however works. > > Pandora's Box -- sure you want to open it? In 3.0 I'd like to; this was my original intent. In 2.6 I think it's not worth the complexity, though I won't complain. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 27 19:38:07 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 27 May 2008 10:38:07 -0700 Subject: [Python-Dev] Why is type_modified() in typeobject.c not a public function? In-Reply-To: References: Message-ID: I'm fine with giving it a public interface. Please submit a patch with docs included. On Tue, May 27, 2008 at 9:47 AM, Stefan Behnel wrote: > [reposting this to python-dev, as it affects both 2.6 and 3.0] > > Hi, > > when we build extension classes in Cython, we have to first build the type to > make it available to user code, and then update the type's tp_dict while we > run the class body code (PyObject_SetAttr() does not work here). In Py2.6+, > this requires invalidating the method cache after each attribute change, which > Python does internally using the type_modified() function. > > Could this function get a public interface? I do not think Cython is the only > case where C code wants to modify a type after its creation, and copying the > code over seems like a hack to me. > > Stefan > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue May 27 19:44:31 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 May 2008 10:44:31 -0700 Subject: [Python-Dev] ABC issues References: Message-ID: <35771740E8F7443FBF0F1A309BEE1EDC@RaymondLaptop1> >>> * The 2.6-backported Mapping ABC has the 3.0 dict API, >>> that is, it uses keys() that returns a view etc. >> >> Curious to hear what Guido thinks about this one. >> A nice use of the Mapping ABC is to be able to >> get 3.0 behaviors. I thought that was the whole >> point of all these backports. If the ABC gets altered, >> then it just makes the 2-to-3 conversion harder. > > It's wrong if the ABC doesn't describe the behavior of actual > implementations; that is its primary purpose, the mixin class is a > nice side benefit. ISTM, the one purpose of backporting is to make 2.6 closer to 3.0. Altering the API will just make them further apart. Raymond From lists at cheimes.de Tue May 27 20:05:19 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 27 May 2008 20:05:19 +0200 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: References: <483BD10E.7050600@gmail.com> Message-ID: <483C4D5F.9080508@cheimes.de> Antoine Pitrou schrieb: > In order to avoid memory consumption issues there could be a centralized cache > as for regular expressions. It makes it easier to handle eviction based on > various parameters, and it saves a few bytes for string objects which are never > used as a formatting template. Good idea! I suggest you hook into the string interning code and use a similar approach. Christian From eric+python-dev at trueblade.com Tue May 27 20:14:35 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Tue, 27 May 2008 14:14:35 -0400 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <483C4D5F.9080508@cheimes.de> References: <483BD10E.7050600@gmail.com> <483C4D5F.9080508@cheimes.de> Message-ID: <483C4F8B.7010600@trueblade.com> Christian Heimes wrote: > Antoine Pitrou schrieb: >> In order to avoid memory consumption issues there could be a centralized cache >> as for regular expressions. It makes it easier to handle eviction based on >> various parameters, and it saves a few bytes for string objects which are never >> used as a formatting template. > > Good idea! > I suggest you hook into the string interning code and use a similar > approach. I don't think parsing the strings is where it spends its time. I think the time is spent in object creation, __format__ lookup, and building the result. I'm looking into some of these other optimizations now. From guido at python.org Tue May 27 20:42:57 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 27 May 2008 11:42:57 -0700 Subject: [Python-Dev] ABC issues In-Reply-To: <35771740E8F7443FBF0F1A309BEE1EDC@RaymondLaptop1> References: <35771740E8F7443FBF0F1A309BEE1EDC@RaymondLaptop1> Message-ID: On Tue, May 27, 2008 at 10:44 AM, Raymond Hettinger wrote: >>>> * The 2.6-backported Mapping ABC has the 3.0 dict API, >>>> that is, it uses keys() that returns a view etc. >>> >>> Curious to hear what Guido thinks about this one. >>> A nice use of the Mapping ABC is to be able to >>> get 3.0 behaviors. I thought that was the whole >>> point of all these backports. If the ABC gets altered, >>> then it just makes the 2-to-3 conversion harder. >> >> It's wrong if the ABC doesn't describe the behavior of actual >> implementations; that is its primary purpose, the mixin class is a >> nice side benefit. > > ISTM, the one purpose of backporting is to make 2.6 closer to 3.0. Altering > the API will just make them further apart. Well, the ABCs have two sides -- they describe the API and they provide a mix-in. I feel strongly that the primary function of ABCs is to describe the API (in fact many ABCs do only that, e.g. look at Sized and Container). I want isinstance({}, collections.Mapping) to be true. If you want to use the 3.0 mixins in 2.6, perhaps an alternate set of APIs could be imported from the future? E.g. from future_collections import Mapping. IIRC a similar mechanism was proposed for some built-in functions, even though I see no traces of an implementation yet. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue May 27 20:50:34 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 May 2008 11:50:34 -0700 Subject: [Python-Dev] ABC issues References: <35771740E8F7443FBF0F1A309BEE1EDC@RaymondLaptop1> Message-ID: <7CC8D250A0884576A50331DF39DADBEF@RaymondLaptop1> > If you want to use the 3.0 mixins in 2.6, perhaps an alternate set of > APIs could be imported from the future? E.g. from future_collections > import Mapping. IIRC a similar mechanism was proposed for some > built-in functions, even though I see no traces of an implementation > yet. Any know what happened to the effort to put the 3.0 dict in future_builtins? That would nicely sync-up the ABC with a concrete implementation and bring 2.6 a little closer to 3.0. Raymond From armin.ronacher at active-4.com Tue May 27 21:16:28 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Tue, 27 May 2008 19:16:28 +0000 (UTC) Subject: [Python-Dev] ABC issues References: <9A7CBBA4962140CA93089B46F97F8548@RaymondLaptop1> Message-ID: Hi, Guido van Rossum python.org> writes: > There's no need to register as Sized -- the Sized ABC recognizes > classes that define __len__ automatically. The Container class does > the same looking for __contains__. Since the deque class doesn't > implement __contains__, it is not considered a Container -- correctly > IMO. True. deque doesn't implement __contains__. However "in" still works because of the __iter__ fallback. So from the API's perspective it's still compatible, even though it doesn't implement it. The same probably affects old style iterators (__getitem__ with index). One could argue that they are still iterable or containers, but that's harder to check so probably not worth the effort. > >> Another issue is that builtin types don't accept ABCs currently. For > >> example > >> set() | SomeSet() gives a TypeError, SomeSet() | set() however works. > > > > Pandora's Box -- sure you want to open it? > > In 3.0 I'd like to; this was my original intent. In 2.6 I think it's > not worth the complexity, though I won't complain. I would love to help on that as I'm very interested in that feature. Regards, Armin From armin.ronacher at active-4.com Tue May 27 21:32:54 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Tue, 27 May 2008 19:32:54 +0000 (UTC) Subject: [Python-Dev] Iterable String Redux (aka String ABC) Message-ID: Hi, Strings are currently iterable and it was stated multiple times that this is a good idea and shouldn't change. While I still don't think that that's a good idea I would like to propose a solution for the problem many people are experiencing by introducing an abstract base class for strings. Basically *the* problematic situation with iterable strings is something like a `flatten` function that flattens out every iterable object except of strings. Imagine it's implemented in a way similar to that:: def flatten(iterable): for item in iterable: try: if isinstance(item, basestring): raise TypeError() iterator = iter(item) except TypeError: yield item else: for i in flatten(iterator): yield i A problem comes up as soon as user defined strings (such as UserString) is passed to the function. In my opinion a good solution would be a "String" ABC one could test against. Regards, Armin From guido at python.org Tue May 27 21:39:45 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 27 May 2008 12:39:45 -0700 Subject: [Python-Dev] ABC issues In-Reply-To: References: <9A7CBBA4962140CA93089B46F97F8548@RaymondLaptop1> Message-ID: On Tue, May 27, 2008 at 12:16 PM, Armin Ronacher wrote: > Hi, > > Guido van Rossum python.org> writes: > >> There's no need to register as Sized -- the Sized ABC recognizes >> classes that define __len__ automatically. The Container class does >> the same looking for __contains__. Since the deque class doesn't >> implement __contains__, it is not considered a Container -- correctly >> IMO. > True. deque doesn't implement __contains__. However "in" still works > because of the __iter__ fallback. Sure, but that fallback is slow, and intentionally does not trigger the 'Container' ABC. > So from the API's perspective it's > still compatible, even though it doesn't implement it. The same probably > affects old style iterators (__getitem__ with index). One could argue that > they are still iterable or containers, but that's harder to check so > probably not worth the effort. The ABCs do not intend to capture partial behavioral compatibility that way -- the intent is to capture interface (in the duck typing sense). Whether __iter__ is a good enough substitute for __contains__ depends on a lot of things -- surely for a huge list it isn't, and for an iterator it isn't either (since it modifies the iterator's state). >> >> Another issue is that builtin types don't accept ABCs currently. For >> >> example >> >> set() | SomeSet() gives a TypeError, SomeSet() | set() however works. >> > >> > Pandora's Box -- sure you want to open it? >> >> In 3.0 I'd like to; this was my original intent. In 2.6 I think it's >> not worth the complexity, though I won't complain. > I would love to help on that as I'm very interested in that feature. Please do submit patches! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue May 27 21:42:48 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 27 May 2008 12:42:48 -0700 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: Message-ID: [+python-3000] On Tue, May 27, 2008 at 12:32 PM, Armin Ronacher wrote: > Strings are currently iterable and it was stated multiple times that this is a > good idea and shouldn't change. While I still don't think that that's a good > idea I would like to propose a solution for the problem many people are > experiencing by introducing an abstract base class for strings. > > Basically *the* problematic situation with iterable strings is something like > a `flatten` function that flattens out every iterable object except of strings. > Imagine it's implemented in a way similar to that:: > > def flatten(iterable): > for item in iterable: > try: > if isinstance(item, basestring): > raise TypeError() > iterator = iter(item) > except TypeError: > yield item > else: > for i in flatten(iterator): > yield i > > A problem comes up as soon as user defined strings (such as UserString) is > passed to the function. In my opinion a good solution would be a "String" > ABC one could test against. I'm not against this, but so far I've not been able to come up with a good set of methods to endow the String ABC with. Another problem is that not everybody draws the line in the same place -- how should instances of bytes, bytearray, array.array, memoryview (buffer in 2.6) be treated? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From benji at benjiyork.com Tue May 27 22:09:47 2008 From: benji at benjiyork.com (Benji York) Date: Tue, 27 May 2008 16:09:47 -0400 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: Message-ID: On Tue, May 27, 2008 at 3:42 PM, Guido van Rossum wrote: > [+python-3000] > > On Tue, May 27, 2008 at 12:32 PM, Armin Ronacher > wrote: >> Basically *the* problematic situation with iterable strings is something like >> a `flatten` function that flattens out every iterable object except of strings. >> Imagine it's implemented in a way similar to that:: > > I'm not against this, but so far I've not been able to come up with a > good set of methods to endow the String ABC with. Another problem is > that not everybody draws the line in the same place -- how should > instances of bytes, bytearray, array.array, memoryview (buffer in 2.6) > be treated? Maybe the opposite approach would be more fruitful. Flattening is about removing nested "containers", so perhaps there should be an ABC that things like lists and tuples provide, but strings don't. No idea what that might be. -- Benji York From solipsis at pitrou.net Tue May 27 22:18:07 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 May 2008 20:18:07 +0000 (UTC) Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: (just my 2 eurocents) Guido van Rossum python.org> writes: > > I'm not against this, but so far I've not been able to come up with a > good set of methods to endow the String ABC with. If we stay minimalistic we could consider that the three basic operations that define a string are: - testing for substring containment - splitting on a substring into a list of substrings - slicing in order to extract a substring Which gives us ['__contains__', 'split', '__getitem__'], and expands intuitively to ['__contains__', 'find', 'index', 'split', 'rsplit', '__getitem__']. > Another problem is > that not everybody draws the line in the same place -- how should > instances of bytes, bytearray, array.array, memoryview (buffer in 2.6) > be treated? In the followup of the flatten() example, bytes and bytearray should be Strings, but array.array and memoryview shouldn't. array.array is really a different kind of container rather than a proper string, and as for memoryview... well, since it's not documented I don't know what it's supposed to do :-) Regards Antoine. From g.brandl at gmx.net Tue May 27 22:44:59 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 27 May 2008 22:44:59 +0200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: Message-ID: Antoine Pitrou schrieb: > (just my 2 eurocents) > > Guido van Rossum python.org> writes: >> >> I'm not against this, but so far I've not been able to come up with a >> good set of methods to endow the String ABC with. > > If we stay minimalistic we could consider that the three basic operations that > define a string are: > - testing for substring containment > - splitting on a substring into a list of substrings > - slicing in order to extract a substring > Which gives us ['__contains__', 'split', '__getitem__'], and expands intuitively > to ['__contains__', 'find', 'index', 'split', 'rsplit', '__getitem__']. I'd argue that "find" is more primitive than "split" -- split is intuitively implemented using find and slicing, but implementing find using split and len is unintuitive. (Of course, "index" can be used instead of "find".) >> Another problem is >> that not everybody draws the line in the same place -- how should >> instances of bytes, bytearray, array.array, memoryview (buffer in 2.6) >> be treated? > > In the followup of the flatten() example, bytes and bytearray should be Strings, > but array.array and memoryview shouldn't. array.array is really a different kind > of container rather than a proper string, and as for memoryview... well, since > it's not documented I don't know what it's supposed to do :-) This is really a problem -- since the PEP 3118 authors don't seem to bother, I'll have to write up something based on the PEP, but I don't know if it is still up-to-date. Georg From solipsis at pitrou.net Tue May 27 23:15:12 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 May 2008 21:15:12 +0000 (UTC) Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: Georg Brandl gmx.net> writes: > I'd argue that "find" is more primitive than "split" -- split is intuitively > implemented using find and slicing, but implementing find using split and > len is unintuitive. (Of course, "index" can be used instead of "find".) I meant semantically primitive. I think the difference between a String and a plain Sequence is that, in a String, the existence and relative position of substrings has a meaning. This is true for character strings but it can also be true for other kinds of strings (think genome strings, they are usually represented using ASCII letters but it's out of convenience - they could be made of opaque objects instead). That's why, in string classes, you have methods like split() to deal with the processing of substrings - which you do not have on lists, not that's it more difficult to implement or algorithmically less efficient, but because it makes no point. Well I hope it makes at least a bit of sense :-) Regards Antoine. From armin.ronacher at active-4.com Tue May 27 23:19:49 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Tue, 27 May 2008 21:19:49 +0000 (UTC) Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: Hi, Georg Brandl gmx.net> writes: > I'd argue that "find" is more primitive than "split" -- split is intuitively > implemented using find and slicing, but implementing find using split and > len is unintuitive. (Of course, "index" can be used instead of "find".) It surely is, but it would probably make sense to require both. Maybe have something like this: class SymbolSequence(Sequence) class String(SymbolSequence) String would be the base of str/unicode and CharacterSequence of str/bytes. A SymbolSequence is basically a sequence based on one type of symbols that implements slicing, getting symbols by index, count() and index(). A String is basically everything a str/unicode provides as method except of those which depend on informatio based on the symbol. For example upper() / isupper() etc would go. Additionally I guess it makes sense to get rid of encode() / decode() / format(). Regards, Armin From g.brandl at gmx.net Tue May 27 23:31:06 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 27 May 2008 23:31:06 +0200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: Message-ID: Antoine Pitrou schrieb: > Georg Brandl gmx.net> writes: >> I'd argue that "find" is more primitive than "split" -- split is intuitively >> implemented using find and slicing, but implementing find using split and >> len is unintuitive. (Of course, "index" can be used instead of "find".) > > I meant semantically primitive. I think the difference between a String and a > plain Sequence is that, in a String, the existence and relative position of > substrings has a meaning. This is true for character strings but it can also be > true for other kinds of strings (think genome strings, they are usually > represented using ASCII letters but it's out of convenience - they could be made > of opaque objects instead). > > That's why, in string classes, you have methods like split() to deal with the > processing of substrings - which you do not have on lists, not that's it more > difficult to implement or algorithmically less efficient, but because it makes > no point. > > Well I hope it makes at least a bit of sense :-) It does, but I don't see how it contradicts my proposition. find() takes a substring as well. Georg From solipsis at pitrou.net Tue May 27 23:35:25 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 May 2008 21:35:25 +0000 (UTC) Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: Georg Brandl gmx.net> writes: > > It does, but I don't see how it contradicts my proposition. find() takes a > substring as well. Well, I'm not sure what your proposal was :-) Did you mean to keep split() out of the String interface, or to provide a default implementation of it based on find() and slicing? From g.brandl at gmx.net Tue May 27 23:44:06 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 27 May 2008 23:44:06 +0200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: Message-ID: Antoine Pitrou schrieb: > Georg Brandl gmx.net> writes: >> >> It does, but I don't see how it contradicts my proposition. find() takes a >> substring as well. > > Well, I'm not sure what your proposal was :-) > Did you mean to keep split() out of the String interface, or to provide a > default implementation of it based on find() and slicing? You wrote: > If we stay minimalistic we could consider that the three basic operations that > define a string are: > - testing for substring containment > - splitting on a substring into a list of substrings > - slicing in order to extract a substring I argued that instead of split, find belongs into that list. (BTW, length inquiry would be a fourth.) That the other methods, among them split, can be implemented in terms of those, follows from both sets of basic operations. Georg From solipsis at pitrou.net Tue May 27 23:59:25 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 27 May 2008 21:59:25 +0000 (UTC) Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: Georg Brandl gmx.net> writes: > You wrote: > > > If we stay minimalistic we could consider that the three basic operations that > > define a string are: > > - testing for substring containment > > - splitting on a substring into a list of substrings > > - slicing in order to extract a substring > > I argued that instead of split, find belongs into that list. > (BTW, length inquiry would be a fourth.) Well, find() does test for substring containment, so in essence it is in that list, although in my first post I chose '__contains__' as the canonical representative of substring containment :-) And, you are right, length inquiry belongs into it too. > That the other methods, among them split, can be implemented in terms > of those, follows from both sets of basic operations. When I wrote "the three basic operations that define a string", perhaps I should have written "the three essential operations" instead. I was not attempting to give implementation guidelines but to propose a semantic definition of what constitutes a string and distinguishes it from other kinds of objects. Anyway, I think we are picking on words here. Do we agree on the following basic String interface : ['__len__', '__contains__', '__getitem__', 'find', 'index', 'split', 'rsplit']? cheers Antoine. From python at rcn.com Wed May 28 00:23:38 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 May 2008 15:23:38 -0700 Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> [Armin Ronacher] > Basically *the* problematic situation with iterable strings is something like > a `flatten` function that flattens out every iterable object except of strings. Stated more generally: The problematic situation is that flatten() implementations typically need some way to decide what kinds of objects are atomic. Different apps draw the line in different places (chars, words, paragraphs, blobs, files, directories, xml elements with attributes, xml bodies, csv records, csv fields, etc.). > A problem comes up as soon as user defined strings (such as UserString) is > passed to the function. In my opinion a good solution would be a "String" > ABC one could test against. Conceptually, this is a fine idea, but three things bug me. First, there is a mismatch between the significance of the problem being addressed versus the weight of the solution. The tiny "problem" is a sense that the simplest version of a flatten recipe isn't perfectly general. The "solution" is to introduce yet another ABC, require adherence to the huge string API and require that everything that purports to be a string register itself. IMO, that is trying to kill a mosquito with a cannon. Second, this seems like the wrong solution to the problem as it places the responsibility in the wrong place and thereby hardwires its notion of what kind of objects should be split. A flatten() implementation doesn't really care about whether an input is a string which supports all the string-like methods such as capitalize(). Wouldn't it be better to write your version of flatten() with a registration function so that a user could specify which objects are atomic? Otherwise, you will have to continually re-edit your flatten() code as you run across other non-stringlike objects that also need to be treated as atomic. Third, I thought ABCs were introduced as an optional feature to support large apps that needed both polymorphic object flexibility and rigorous API matching. Now, it seems that even the tiniest recipe is going to expose its internals and insist on objects being registered as one of several supported abstract types. I suppose this is better than insisting on one of several concrete types, but it still smells like an anti-pattern. Raymond From jimjjewett at gmail.com Wed May 28 00:40:48 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 27 May 2008 18:40:48 -0400 Subject: [Python-Dev] [Python-3000] Iterable String Redux (aka String ABC) In-Reply-To: References: Message-ID: On 5/27/08, Benji York wrote: > Guido van Rossum wrote: > > Armin Ronacher wrote: > >> Basically *the* problematic situation with iterable strings is something like > >> a `flatten` function that flattens out every iterable object except of strings. > > I'm not against this, but so far I've not been able to come up with a > > good set of methods to endow the String ABC with. Another problem is > > that not everybody draws the line in the same place -- how should > > instances of bytes, bytearray, array.array, memoryview (buffer in 2.6) > > be treated? > Maybe the opposite approach would be more fruitful. Flattening is about > removing nested "containers", so perhaps there should be an ABC that > things like lists and tuples provide, but strings don't. No idea what > that might be. It isn't really stringiness that matters, it is that you have to terminate even though you still have an iterable container. The test is roughly (1==len(v) and v[0]==v), except that you want to stop a layer sooner. Guido had at least a start in Searchable, back when ABC were still in the sandbox: http://svn.python.org/view/sandbox/trunk/abc/abc.py?rev=55321&view=auto Searchable represented the fact that (x in c) =/=> (x in iter(c)) because of sequence searches like ("Error" in results) -jJ From python at rcn.com Wed May 28 00:54:20 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 May 2008 15:54:20 -0700 Subject: [Python-Dev] [Python-3000] Iterable String Redux (aka StringABC) References: Message-ID: <1791A55949334749A9DF448461C9B11A@RaymondLaptop1> "Jim Jewett" > It isn't really stringiness that matters, it is that you have to > terminate even though you still have an iterable container. Well said. > Guido had at least a start in Searchable, back when ABC > were still in the sandbox: Have to disagree here. An object cannot know in general whether a flattener wants to split it or not. That is an application dependent decision. A better answer is be able to tell the flattener what should be considered atomic in a given circumstance. Raymond From tjreedy at udel.edu Wed May 28 00:58:14 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 27 May 2008 18:58:14 -0400 Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: "Armin Ronacher" wrote in message news:loom.20080527T192243-415 at post.gmane.org... | Basically *the* problematic situation with iterable strings is something like | a `flatten` function that flattens out every iterable object except of strings. In most real cases I can imagine, this is way too broad. For instance, trying to 'flatten' an infinite iterable makes the flatten output one also. Flattening a set imposes an arbitrary order (but that is ok if one feeds the output to set(), which de-orders it). Flattening a dict decouples keys and values. Flattening iterable set-theoretic numbers (0={}, n = {n-1, {n-1}}, or something like that) would literaly yield nothing. | Imagine it's implemented in a way similar to that:: | | def flatten(iterable): | for item in iterable: | try: | if isinstance(item, basestring): | raise TypeError() | iterator = iter(item) | except TypeError: | yield item | else: | for i in flatten(iterator): | yield i I can more easily imagine wanting to flatten only certain classes, such and tuples and lists, or frozensets and sets. def flatten(iterable, classes): for item in iterable: if type(item) in classes: for i in flatten(item, classes): yield i else: yield item | A problem comes up as soon as user defined strings (such as UserString) is | passed to the function. In my opinion a good solution would be a "String" | ABC one could test against. This might be a good idea regardless of my comments. tjr From steve at pearwood.info Wed May 28 02:07:59 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 28 May 2008 10:07:59 +1000 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> Message-ID: <200805281007.59902.steve@pearwood.info> (If you receive this twice, please excuse the duplicate email. User-error on my part, sorry.) On Wed, 28 May 2008 08:23:38 am Raymond Hettinger wrote: > A flatten() implementation doesn't really care about whether > an input is a string which supports all the string-like methods > such as capitalize(). ? Wouldn't it be better to write your > version of flatten() with a registration function so that a user > could specify which objects are atomic? ?Otherwise, you > will have to continually re-edit your flatten() code as you > run across other non-stringlike objects that also need to > be treated as atomic. Just throwing a suggestion out there... def atomic(obj, _atomic=(basestring,)): try: return bool(obj.__atomic__) except AttributeError: if isinstance(obj, _atomic): return True else: try: iter(obj) except TypeError: return True return False assert atomic("abc") assert not atomic(['a', 'b', 'c']) If built-in objects grew an __atomic__ attribute, you could simplify the atomic() function greatly: def atomic(obj): return bool(obj.__atomic__) However atomic() is defined, now flatten() is easy: def flatten(obj): if atomic(obj): yield obj else: for item in obj: for i in flatten(item): yield i If you needed more control, you could customise it using standard techniques e.g. shadow the atomic() function with your own version, sub-class the types you wish to treat differently, make __atomic__ a computed property instead of a simple attribute, etc. Re-writing the above to match Python 3 is left as an exercise. -- Steven From python at rcn.com Wed May 28 02:31:04 2008 From: python at rcn.com (Raymond Hettinger) Date: Tue, 27 May 2008 17:31:04 -0700 Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <200805281005.58076.limeguin@pearwood.info> Message-ID: <38BFCD04E12240C1AFC7A455CF84A94F@RaymondLaptop1> > Steven D'Aprano" > If built-in objects grew an __atomic__ attribute, you could > simplify the atomic() function greatly: I may not have been clear enough in my previous post. Atomicity is not an intrinsic property of an object or class. How could you know in advance what various applications would want to consider atomic? The decision is application specific and best left to the caller of the flattener: def flatten(obj, predicate=None): if predicate is not None and predicate(obj): yield obj else: for item in obj: for i in flatten(item): yield i > However atomic() is defined, now flatten() is easy: Rule of thumb: if you find a need to change the language (adding a new builtin, adding a new protocol, and adding a property to every builtin and pure python container) just to implement a simple recipe, then it is the recipe that needs fixing, not the language. Raymond P.S. You're on the right track by factoring the decision away from the internals of flatten(); however, the atomic() predicate needs to be user definable for a given application not hardwired into the language itself. From greg.ewing at canterbury.ac.nz Wed May 28 02:30:04 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 28 May 2008 12:30:04 +1200 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: References: <4832E144.8030309@gmail.com> <483B70FA.2080807@gmail.com> Message-ID: <483CA78C.1050009@canterbury.ac.nz> Armin Ronacher wrote: > I'm currently not > providing any __r*__ methods as I was too lazy to test on each call if the > method that is proxied is providing an __rsomething__ or not, and if not come up > with an ad-hoc implementation by calling __something__ and reversing the > arguments passed. I don't see why you should have to do that, as the reversed method of the proxy will only be called if a prior non-reversed call failed. All the proxy should have to do is delegate any lookups of its own reversed methods to corresponding methods of the proxied object, no differently from any other method. -- Greg From greg.ewing at canterbury.ac.nz Wed May 28 02:38:09 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 28 May 2008 12:38:09 +1200 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <483C0D76.6050507@gmail.com> References: <4832E144.8030309@gmail.com> <483B70FA.2080807@gmail.com> <483C0513.7040903@gmail.com> <483C0808.2030808@voidspace.org.uk> <483C0D76.6050507@gmail.com> Message-ID: <483CA971.4000508@canterbury.ac.nz> Nick Coghlan wrote: > else: > # Returned a different object, make a new proxy > result = type(self)(result) You might want to check that the result has the same type as the proxied object before doing that. -- Greg From tjreedy at udel.edu Wed May 28 05:15:53 2008 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 27 May 2008 23:15:53 -0400 Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <200805281007.59902.steve@pearwood.info> Message-ID: "Steven D'Aprano" wrote in message news:200805281007.59902.steve at pearwood.info... Just throwing a suggestion out there... def atomic(obj, _atomic=(basestring,)): try: return bool(obj.__atomic__) except AttributeError: if isinstance(obj, _atomic): return True else: try: iter(obj) except TypeError: return True return False assert atomic("abc") assert not atomic(['a', 'b', 'c']) If built-in objects grew an __atomic__ attribute, you could simplify the atomic() function greatly: def atomic(obj): return bool(obj.__atomic__) However atomic() is defined, now flatten() is easy: def flatten(obj): if atomic(obj): yield obj else: for item in obj: for i in flatten(item): yield i If you needed more control, you could customise it using standard techniques e.g. shadow the atomic() function with your own version, sub-class the types you wish to treat differently, make __atomic__ a computed property instead of a simple attribute, etc. ================== This is a lot of work to avoid being explicit about either atomic or non-atomic classes on an site, package, module, or call basis ;-) From stefan_ml at behnel.de Wed May 28 07:47:16 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 28 May 2008 07:47:16 +0200 Subject: [Python-Dev] Why is type_modified() in typeobject.c not a public function? In-Reply-To: References: Message-ID: Hi, Guido van Rossum wrote: > On Tue, May 27, 2008 at 9:47 AM, Stefan Behnel wrote: >> Could this function get a public interface? I do not think Cython is the only >> case where C code wants to modify a type after its creation, and copying the >> code over seems like a hack to me. >> > I'm fine with giving it a public interface. Please submit a patch with > docs included. Straight forward patch is attached (against 3.0a5). Works for me in Cython. I thought about a name like "Taint(t)" or "ClearTypeCache(t)", but then went with the coward solution of calling the function PyType_Modified() as it was (almost) named internally. BTW, I noticed that the code in typeobject.c uses "DECREF before set" two times, like this: method_cache[h].version = type->tp_version_tag; method_cache[h].value = res; /* borrowed */ Py_INCREF(name); Py_DECREF(method_cache[h].name); method_cache[h].name = name; During the call to Py_DECREF, the cache content is incorrect, so can't this run into the same problem that Py_CLEAR() aims to solve? I attached a patch for that, too, just in case. Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: pytype_modified.patch Type: text/x-patch Size: 3095 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: possible-decref-before-set-fix.patch Type: text/x-patch Size: 1311 bytes Desc: not available URL: From techtonik at gmail.com Wed May 28 08:45:34 2008 From: techtonik at gmail.com (techtonik) Date: Wed, 28 May 2008 09:45:34 +0300 Subject: [Python-Dev] Deprecated Cookie classes in Py3k Message-ID: I've noticed that some classes in Cookies module (namely SerialCookie and SmartCookie) deprecated since 2.3 still present in Python3000 documentation. http://docs.python.org/dev/3.0/library/http.cookies.html Is it because ... ?: 1. Docs are not synchronized with API 2. Classes are not removed yet (http://wiki.python.org/moin/Py3kDeprecatedis actually a TODO) 3. Manual reference should contain information about all historical API changes -- --anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Wed May 28 12:07:45 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 28 May 2008 11:07:45 +0100 Subject: [Python-Dev] Issue 643841: Including a new-style proxy base class in 2.6/3.0 In-Reply-To: <483CA971.4000508@canterbury.ac.nz> References: <4832E144.8030309@gmail.com> <483B70FA.2080807@gmail.com> <483C0513.7040903@gmail.com> <483C0808.2030808@voidspace.org.uk> <483C0D76.6050507@gmail.com> <483CA971.4000508@canterbury.ac.nz> Message-ID: <483D2EF1.5070704@voidspace.org.uk> Greg Ewing wrote: > Nick Coghlan wrote: > >> else: >> # Returned a different object, make a new proxy >> result = type(self)(result) > > You might want to check that the result has the > same type as the proxied object before doing that. > Yep - and I really think it would need to do this. To have a proxy where: proxy_instance += 1 unwraps the proxy is really no good! (At least not for my use cases...) Michael Foord -- http://www.ironpythoninaction.com/ http://www.theotherdelia.co.uk/ http://www.voidspace.org.uk/ http://www.ironpython.info/ http://www.resolverhacks.net/ From mal at egenix.com Wed May 28 12:12:27 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 28 May 2008 12:12:27 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483BDE11.509@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> Message-ID: <483D300B.5090309@egenix.com> I'm beginning to wonder whether I'm the only one who cares about the Python 2.x branch not getting cluttered up with artifacts caused by a broken forward merge strategy. How can it be that we allow major C API changes such as the renaming of the PyString APIs to go into the trunk without discussion or a PEP ? We're having lengthy discussions about the addition of single method to an object, but such major changes just go in like that and nobody seems to really care. Puzzled, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 28 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 39 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 On 2008-05-27 12:10, M.-A. Lemburg wrote: > On 2008-05-26 23:34, Christian Heimes wrote: >> M.-A. Lemburg schrieb: >>> Isn't that an awefuly confusing approach ? >>> >>> Wouldn't it be better to keep PyString APIs and definitions in >>> stringobject.c|h >>> >>> and only add a new bytesobject.h header file that #defines the >>> PyBytes APIs in terms of PyString APIs ? That maintains >>> backwards compatibility and allows Python internals to use the >>> new API names. >>> >>> With your approach, you've basically backported the confusing >>> notion in Py3k that str() maps PyUnicode, only that in Py2 >>> str() will now map to PyBytes. >> >> The last time I brought up the topic, I had a lengthy discussion with >> Guido. At first I wanted to rename the API in Python 3.0 only. Guido >> argued that it's going to cause too much merge conflicts. He then >> suggested the approach I implemented today. > > That's the same argument that came up in the module renaming > discussion. > > I have a feeling that we should be looking for better merge > tools, rather than implement code changes that cause more trouble > than do good, just because our existing tools aren't smart > enough. > > Wouldn't it be possible to have a 2to3.py converter > take the 2.x code (including the C code), convert it and then > apply any changes to the 3.x branch ? > > This wouldn't be merging in the classical sense, it would be > automated forward porting. > >> I find the approach less confusing than your suggestion and my initial >> idea. > > I disagree on that. > > Renaming old APIs to use the new names by adding a header file with > #define is standard practice. > > Renaming the old APIs in the source code and undoing the renaming > with a header file is not. > >> The internal API names are consistent for Python 2.6 and 3.0. The >> byte string C API is prefixed PyBytes and the unicode C API is prefixed >> PyUnicode. A core developer has just to remember that 'str' is a byte >> string in 2.x but an unicode object in 3.0. > > So you've solved part of the problem for 3.x by moving the naming mixup > back to 2.x. > >> Extension developers don't have to worry at all. The ABI and external >> API is mostly the same and still exposes the 'str' functions as PyString. > > Well, yes, but only due to a preprocessor hack that turns the > names used in bytesobject.c back into names you'd normally look > for in stringobject.c. > > And all this, just because Subversion can't handle merging of > symbol renaming. > >>> You'd have to add an aliase bytes -> str to the builtins to >>> at least reduce the confusion a bit. >> >> Python 2.6 already has an alias bytes -> str >> >>> Yes, but please let's first discuss this some more. I don't think >>> that the timing was right.... you started this thread just yesterday >>> and the patches are already checked in. >> >> I'm sorry if I was too hasty for you. I got +1 from a couple of >> developers and it's basically Guido's suggestion. > > Please discuss any changes of the 2.x code base on python-dev. > > Such major changes do need more discussion and possibly a PEP as well. > > Thanks, From bborcic at gmail.com Wed May 28 13:18:32 2008 From: bborcic at gmail.com (Boris Borcic) Date: Wed, 28 May 2008 13:18:32 +0200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: Message-ID: Armin Ronacher wrote: > > Basically *the* problematic situation with iterable strings is something like > a `flatten` function that flattens out every iterable object except of strings. To flesh out the span of your "something like", recently I had a WSGI-based app that to some request mistakenly returned a 200K string instead of the same wrapped as a 1-element list; and the WSGI layer -according to spec- served it back character by character. Which "worked" - and durably confused not only me but IIS and a network router as well. While blame can certainly be assigned elsewhere - WSGI spec or implementation (wsgiref included) - unwelcome iterability of strings was a necessary cause. Cheers, BB From stefan_ml at behnel.de Wed May 28 13:41:57 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 28 May 2008 11:41:57 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Why_is_type=5Fmodified=28=29_in_typeobject?= =?utf-8?q?=2Ec_not_a_public_function=3F?= References: Message-ID: Stefan Behnel wrote: > Straight forward patch is attached (against 3.0a5). [...] > BTW, I noticed that the code in typeobject.c uses "DECREF before set" two > times. Filed on the bug tracker as issues 2989 and 2990. http://bugs.python.org/issue2989 http://bugs.python.org/issue2990 Stefan From lists at cheimes.de Wed May 28 13:43:46 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 28 May 2008 13:43:46 +0200 Subject: [Python-Dev] Why is type_modified() in typeobject.c not a public function? In-Reply-To: References: Message-ID: <483D4572.2090807@cheimes.de> Stefan Behnel schrieb: > Straight forward patch is attached (against 3.0a5). Works for me in Cython. I > thought about a name like "Taint(t)" or "ClearTypeCache(t)", but then went > with the coward solution of calling the function PyType_Modified() as it was > (almost) named internally. > > BTW, I noticed that the code in typeobject.c uses "DECREF before set" two > times, like this: > > method_cache[h].version = type->tp_version_tag; > method_cache[h].value = res; /* borrowed */ > Py_INCREF(name); > Py_DECREF(method_cache[h].name); > method_cache[h].name = name; > > During the call to Py_DECREF, the cache content is incorrect, so can't this > run into the same problem that Py_CLEAR() aims to solve? I attached a patch > for that, too, just in case. Please create two tickets in the bug tracker and append the patches. Christian From lists at cheimes.de Wed May 28 14:02:53 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 28 May 2008 14:02:53 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483BDE11.509@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> Message-ID: <483D49ED.8060907@cheimes.de> M.-A. Lemburg schrieb: > I have a feeling that we should be looking for better merge > tools, rather than implement code changes that cause more trouble > than do good, just because our existing tools aren't smart > enough. We don't have better tools at our hands. I don't think we'll get any tools in time or chance the VCS right before a major release. > Wouldn't it be possible to have a 2to3.py converter > take the 2.x code (including the C code), convert it and then > apply any changes to the 3.x branch ? Such a converter would be nice for 3rd party code but it's not an option for the core. In the past few months I've merged a lot of code from trunk to py3k. A 2to3 C converter doesn't help with merge conflicts. Naming differences make any merge more painful >> I find the approach less confusing than your suggestion and my initial >> idea. > > I disagree on that. > > Renaming old APIs to use the new names by adding a header file with > #define is standard practice. > > Renaming the old APIs in the source code and undoing the renaming > with a header file is not. I wasn't talking about standard practice here. I talked about less confusion for core developers. My approach doesn't split our internal API in two. And by the way it *is* a standard approach fore Python. Guido told me that the same approach was used during the 1.x to 2.0 migration. > And all this, just because Subversion can't handle merging of > symbol renaming. As I said earlier we don't have better tools at our disposal. We have to make some compromises. Sometimes practicality beat purity. > Please discuss any changes of the 2.x code base on python-dev. > > Such major changes do need more discussion and possibly a PEP as well. In the last few months I started at least three topics about the C API renaming. It's in the thread "2.6 and 3.0 tasks" http://permalink.gmane.org/gmane.comp.python.devel/93016 Christian From p.f.moore at gmail.com Wed May 28 14:17:14 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 May 2008 13:17:14 +0100 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> Message-ID: <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> On 27/05/2008, Raymond Hettinger wrote: > Conceptually, this is a fine idea, but three things bug me. > > First, there is a mismatch between the significance of the problem > being addressed versus the weight of the solution. Agreed, absolutely. > Second, this seems like the wrong solution to the problem > as it places the responsibility in the wrong place and thereby > hardwires its notion of what kind of objects should be split. Again, agreed. The flatten function is one of the canonical examples of the visitor patterns. I see no generalisation of this proposal to other visitor patterns. I'd rather see a solution which addressed the wider visitor use case (I think I just sprained my back bending over backwards to avoid mentioning generic functions :-)) > Third, I thought ABCs were introduced as an optional feature [...] Again, I agree absolutely. Paul. From p.f.moore at gmail.com Wed May 28 14:29:32 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 28 May 2008 13:29:32 +0100 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483D300B.5090309@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> Message-ID: <79990c6b0805280529vefcb2a6l200afda6222503f8@mail.gmail.com> On 28/05/2008, M.-A. Lemburg wrote: > I'm beginning to wonder whether I'm the only one who cares about > the Python 2.x branch not getting cluttered up with artifacts caused > by a broken forward merge strategy. I care, but I struggle to understand the implications and/or what is being proposed in many cases. Recent examples are the ABC backports and the current thread (string C API). I simply don't follow the issues well enough to comment. > How can it be that we allow major C API changes such as the renaming > of the PyString APIs to go into the trunk without discussion or > a PEP ? Christian has raised this a couple of times, but there has been little discussion. I suspect that this is because there is not enough clarity over the practical consequences. A PEP may help here, but I'm not sure how much - it could spark discussion, but would anyone actually end up any better informed? > We're having lengthy discussions about the addition of single method > to an object, but such major changes just go in like that and nobody > seems to really care. I suspect deadline pressure and burnout are involved here. In all honesty, there's been little or no work done on the C API, which is just as much in need of review and possible cleanup for 3.0 as the language. It's as close as makes no difference to too late now - does that mean we've lost the chance? Paul. From mal at egenix.com Wed May 28 14:59:33 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 28 May 2008 14:59:33 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <79990c6b0805280529vefcb2a6l200afda6222503f8@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <79990c6b0805280529vefcb2a6l200afda6222503f8@mail.gmail.com> Message-ID: <483D5735.4090608@egenix.com> On 2008-05-28 14:29, Paul Moore wrote: > On 28/05/2008, M.-A. Lemburg wrote: >> I'm beginning to wonder whether I'm the only one who cares about >> the Python 2.x branch not getting cluttered up with artifacts caused >> by a broken forward merge strategy. > > I care, but I struggle to understand the implications and/or what is > being proposed in many cases. Thanks, so I'm not the only :-) > Recent examples are the ABC backports and the current thread (string C > API). I simply don't follow the issues well enough to comment. > >> How can it be that we allow major C API changes such as the renaming >> of the PyString APIs to go into the trunk without discussion or >> a PEP ? > > Christian has raised this a couple of times, but there has been little > discussion. I suspect that this is because there is not enough clarity > over the practical consequences. A PEP may help here, but I'm not sure > how much - it could spark discussion, but would anyone actually end up > any better informed? Probably, yes. The reason is that if you have a PEP, more people are likely to review it and make comments. If you start a discussion with a general subject line which then results in lots of little sub-threads, important aspects of the discussion are likely to go unnoticed in the noise. >> We're having lengthy discussions about the addition of single method >> to an object, but such major changes just go in like that and nobody >> seems to really care. > > I suspect deadline pressure and burnout are involved here. > > In all honesty, there's been little or no work done on the C API, > which is just as much in need of review and possible cleanup for 3.0 > as the language. It's as close as makes no difference to too late now > - does that mean we've lost the chance? Perhaps, but the C API is certainly not used by as many people as the Python front-end and changes to the C API also have much deeper consequences due the API being written in C rather than Python. Overall, I don't think there's a lot to cleanup in the C API. Perhaps remove a few of those '...Ex()' APIs that were introduced to extend the original APIs and maybe remove or free up a few type slots that are no longer needed, but that's about it. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 28 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 39 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Wed May 28 14:47:00 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 28 May 2008 14:47:00 +0200 Subject: [Python-Dev] PyString -> PyBytes C API renaming (Stabilizing the C API of 2.6 and 3.0) In-Reply-To: <483D49ED.8060907@cheimes.de> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> Message-ID: <483D5444.8000705@egenix.com> On 2008-05-28 14:02, Christian Heimes wrote: > M.-A. Lemburg schrieb: >> I have a feeling that we should be looking for better merge >> tools, rather than implement code changes that cause more trouble >> than do good, just because our existing tools aren't smart >> enough. > > We don't have better tools at our hands. I don't think we'll get any > tools in time or chance the VCS right before a major release. > >> Wouldn't it be possible to have a 2to3.py converter >> take the 2.x code (including the C code), convert it and then >> apply any changes to the 3.x branch ? > > Such a converter would be nice for 3rd party code but it's not an option > for the core. In the past few months I've merged a lot of code from > trunk to py3k. A 2to3 C converter doesn't help with merge conflicts. > Naming differences make any merge more painful I was suggesting to not use SVN to merge changes directly, but to instead use an intermediate step in the process: Init: 1. grab the latest trunk 2. apply a 2to3 converter to the Python code and the C code, applying any renaming that may be necessary 3. save this converted version in a separate branch merge-branch Update: 1. checkout the merge-branch, . grab the latest trunk and 3.x branch 2. apply a 2to3 converter to the Python code and the C code, applying any renaming that may be necessary 3. copy the files over your working copy of the merge-branch 4. create a diff on the merge-branch 5. apply the diffs to 3.x branch, resolving any conflicts as necessary This doesn't require new tools (except for some C renaming support in the 2to3 tool). It only changes the procedure. We'd basically follow our own suggestions w/r to porting to 3.x, which is to make changes in the 2.x code, apply 2to3 and then apply remaining fixes there. I'm suggesting this, since 3.x is likely to introduce more Python stdlib and C API changes. The process would likely also makes a lot of other changes more easily manageable and reduce the overall merge conflicts. >>> I find the approach less confusing than your suggestion and my initial >>> idea. >> I disagree on that. >> >> Renaming old APIs to use the new names by adding a header file with >> #define is standard practice. >> >> Renaming the old APIs in the source code and undoing the renaming >> with a header file is not. > > I wasn't talking about standard practice here. I talked about less > confusion for core developers. My approach doesn't split our internal > API in two. No, but it does apply a well hidden renaming which will cause confusion when using a debugger to trace calls in C code. If you use PyBytes APIs, you expect to find PyBytes functions in the libs and also set breakpoints on these. With the renaming we don't have two sets of APIs (old and new) exposed in the lib, like what we normally do when applying changes to API names. > And by the way it *is* a standard approach fore Python. Guido told me > that the same approach was used during the 1.x to 2.0 migration. There was no API change between 1.6 and 2.0. You are probably talking about the great renaming between 1.4 and 1.5. That was different, since it changes almost all C APIs in Python. And it used the standard practice... from rename2.h in Python 1.5: /* This file contains a bunch of #defines that make it possible to use "old style" names (e.g. object) with the new style Python source distribution. */ #define True Py_True #define False Py_False #define None Py_None ie. #define >> And all this, just because Subversion can't handle merging of >> symbol renaming. > > As I said earlier we don't have better tools at our disposal. We have to > make some compromises. Sometimes practicality beat purity. See above. >> Please discuss any changes of the 2.x code base on python-dev. >> >> Such major changes do need more discussion and possibly a PEP as well. > > In the last few months I started at least three topics about the C API > renaming. It's in the thread "2.6 and 3.0 tasks" > http://permalink.gmane.org/gmane.comp.python.devel/93016 Thanks. I stopped reading that thread after Guido's reply in http://comments.gmane.org/gmane.comp.python.devel/92541 It would really help if subject lines were more specific. This thread also uses a much to general subject line (which is why I changed it). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 28 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 39 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ncoghlan at gmail.com Wed May 28 15:43:13 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 28 May 2008 23:43:13 +1000 Subject: [Python-Dev] [Python-3000] PyString -> PyBytes C API renaming (Stabilizing the C API of 2.6 and 3.0) In-Reply-To: <483D5444.8000705@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> <483D5444.8000705@egenix.com> Message-ID: <483D6171.5000208@gmail.com> M.-A. Lemburg wrote: > You are probably talking about the great renaming between 1.4 and 1.5. > That was different, since it changes almost all C APIs in Python. > And it used the standard practice... from rename2.h in Python 1.5: > > /* This file contains a bunch of #defines that make it possible to use > "old style" names (e.g. object) with the new style Python source > distribution. */ > > #define True Py_True > #define False Py_False > #define None Py_None > > ie. #define This is what I expected to see in stringobject.h, along with some code in stringobject.c to allow the linker to see the old names *as well as* the new names. At the moment, all the code appears to be using the new names, but stringobject.h implicitly converts the new names back to the old names - so trying to use ctypes to retrieve the PyBytes_* functions from the Python DLL will fail. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From g.brandl at gmx.net Wed May 28 17:57:45 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 28 May 2008 17:57:45 +0200 Subject: [Python-Dev] Deprecated Cookie classes in Py3k In-Reply-To: References: Message-ID: techtonik schrieb: > I've noticed that some classes in Cookies module (namely SerialCookie > and SmartCookie) deprecated since 2.3 still present in Python3000 > documentation. http://docs.python.org/dev/3.0/library/http.cookies.html > > Is it because ... ?: > 1. Docs are not synchronized with API > 2. Classes are not removed yet > (http://wiki.python.org/moin/Py3kDeprecated is actually a TODO) > 3. Manual reference should contain information about all historical API > changes It was most likely an oversight on my part. I've now removed the classes from the 3000 module. Thanks, Georg From techtonik at gmail.com Wed May 28 18:45:47 2008 From: techtonik at gmail.com (techtonik) Date: Wed, 28 May 2008 19:45:47 +0300 Subject: [Python-Dev] Deprecated Cookie classes in Py3k In-Reply-To: References: Message-ID: On Wed, May 28, 2008 at 6:57 PM, Georg Brandl wrote: > It was most likely an oversight on my part. I've now removed the classes > from the 3000 module. Nice! =) If you're up to fixing the docs, could you, please, also review the fresh issue about this module. http://bugs.python.org/issue2988 Browsers allow to create invalid (non RFC) cookies rather easily and they crash python web scripts on the same domain, because developers usually not aware of this problem to catch the exception. There is a test case and several proposals for workarounds in Cookie (http.cookies) module. -- --anatoly t. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnoller at gmail.com Wed May 28 19:03:56 2008 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 28 May 2008 13:03:56 -0400 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) Message-ID: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> Georg kindly published the PEP I submitted last night to the PEP site: http://www.python.org/dev/peps/pep-0371/ This PEP includes some of the previous discussion on the processing module's inclusion, and I hope clears up/clarifies some of the goals/non goals and issues. I also included benchmark data and a link to the code used for said benchmarks. I would like to renew the discussion now that "there is a PEP" to see if there are any outstanding things people would like to get resolved. I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback both here and elsewhere that people would rather see this in sooner in some form, rather than later (i.e.: 2.7/3.1). -Jesse From janssen at parc.com Wed May 28 19:08:23 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 28 May 2008 10:08:23 PDT Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483D300B.5090309@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> Message-ID: <08May28.100829pdt."58698"@synergy1.parc.xerox.com> > I'm beginning to wonder whether I'm the only one who cares about > the Python 2.x branch not getting cluttered up with artifacts caused > by a broken forward merge strategy. I share your concern. Seems to me that perhaps (not sure, but perhaps) the rush to back-port from 3.x, and the concern about minimizing pain of moving from 2.x to 3.x, has become the tail wagging the dog. Bill From brett at python.org Wed May 28 21:40:37 2008 From: brett at python.org (Brett Cannon) Date: Wed, 28 May 2008 12:40:37 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <8453256766467481803@unknownmsgid> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <8453256766467481803@unknownmsgid> Message-ID: On Wed, May 28, 2008 at 10:08 AM, Bill Janssen wrote: >> I'm beginning to wonder whether I'm the only one who cares about >> the Python 2.x branch not getting cluttered up with artifacts caused >> by a broken forward merge strategy. > > I share your concern. Seems to me that perhaps (not sure, but > perhaps) the rush to back-port from 3.x, and the concern about > minimizing pain of moving from 2.x to 3.x, has become the tail wagging > the dog. > Speaking for myself, I know that if fixing something in 2.x means a pain in forward-porting, I will just do it in 3.x and leave it someone else to back-port to 2.x which will lower the chances of the back-port ever occurring. I don't want to do this, but I am fighting damn hard against burn-out at this point and if I have to choose between complete burn-out and only working on the leading edge version of Python, I will choose the latter. So I for one appreciate Christian taking all of us into account in terms of the approach taken to make our lives easier when we work on Python. -Brett From greg at krypto.org Wed May 28 22:47:29 2008 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 28 May 2008 13:47:29 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483D300B.5090309@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> Message-ID: <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> On Wed, May 28, 2008 at 3:12 AM, M.-A. Lemburg wrote: > I'm beginning to wonder whether I'm the only one who cares about > the Python 2.x branch not getting cluttered up with artifacts caused > by a broken forward merge strategy. > > How can it be that we allow major C API changes such as the renaming > of the PyString APIs to go into the trunk without discussion or > a PEP ? I do not consider it a C API change. The API and ABI have not changed. Old code still compiles. Old binaries still dynamically load and work fine. (I just confirmed this by importing a couple python2.4 .so files into my non-debug build of 2.6 trunk) A of the PyString APIs are the real implementations in 2.x and are still there. We only switched to using their PyBytes equivalent names within the Python trunk code base. Are you objecting to our own code switching to use a different name even though the actual underlying API and ABI haven't changed? I suppose to people reading the code and going against old reference books it could be confusing but they've got to get used to the new names somehow and sometime. I strongly support changes like this one that makes the life of porting C code forwards and backwards between 2.x and 3.x easier without breaking compatibility with earlier 2.x version because that is going to be a serious pain for all of us otherwise. -gps From python at rcn.com Wed May 28 23:17:11 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 28 May 2008 14:17:11 -0700 Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: Message-ID: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> >>> I'm not against this, but so far I've not been able to come up with a >>> good set of methods to endow the String ABC with. >> >> If we stay minimalistic we could consider that the three basic operations that >> define a string are: >> - testing for substring containment >> - splitting on a substring into a list of substrings >> - slicing in order to extract a substring > >> Which gives us ['__contains__', 'split', '__getitem__'], and expands intuitively >> to ['__contains__', 'find', 'index', 'split', 'rsplit', '__getitem__']. With the Sequence ABC, you already get index, contains, __len__, count, __iter__, and __getitem__. What's the benefit of an additional ABC with just three more methods? What can be learned from any known use cases for abstract strings (iirc, idlelib has an interesting example of subclassing UserString). Is there anything about this proposal that is intrinsically texty? In the 3.0 world, text is an abstract sequence of code points. Would you want to require an encode() method so there will always be a way to make it concrete? The split()/rsplit() methods have a complex specification. Including them may make it hard to write a compliant class. >From what's been discussed so far, I don't see any advantage of isinstance(o, String) over hasattr(o, 'encode') or somesuch. Raymond From janssen at parc.com Wed May 28 23:33:17 2008 From: janssen at parc.com (Bill Janssen) Date: Wed, 28 May 2008 14:33:17 PDT Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> Message-ID: <08May28.143320pdt."58698"@synergy1.parc.xerox.com> > >>> I'm not against this, but so far I've not been able to come up with a > >>> good set of methods to endow the String ABC with. > >> > >> If we stay minimalistic we could consider that the three basic operations that > >> define a string are: > >> - testing for substring containment > >> - splitting on a substring into a list of substrings > >> - slicing in order to extract a substring > > > >> Which gives us ['__contains__', 'split', '__getitem__'], and expands intuitively > >> to ['__contains__', 'find', 'index', 'split', 'rsplit', '__getitem__']. > > With the Sequence ABC, you already get index, contains, __len__, count, > __iter__, and __getitem__. What's the benefit of an additional ABC > with just three more methods? What can be learned from any known use > cases for abstract strings (iirc, idlelib has an interesting example of > subclassing UserString). Is there anything about this proposal that is > intrinsically texty? > > In the 3.0 world, text is an abstract sequence of code points. > Would you want to require an encode() method so there will > always be a way to make it concrete? I would. > The split()/rsplit() methods have a complex specification. > Including them may make it hard to write a compliant class. > >From what's been discussed so far, I don't see any advantage > of isinstance(o, String) over hasattr(o, 'encode') or somesuch. Look, even if there were *no* additional methods, it's worth adding the base class, just to differentiate the class from the Sequence, as a marker, so that those of us who want to ask "isinstance(o, String)" can do so. Personally, I'd add in all the string methods to that class, in all their gory complexity. Those who need a compliant class should subclass the String base class, and override/add what they need. Bill From mike.klaas at gmail.com Thu May 29 01:01:34 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Wed, 28 May 2008 16:01:34 -0700 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <08May28.143320pdt."58698"@synergy1.parc.xerox.com> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt."58698"@synergy1.parc.xerox.com> Message-ID: <6B5CC2A3-EC91-4589-89F2-B4F51FB191B8@gmail.com> On 28-May-08, at 2:33 PM, Bill Janssen wrote: > >>> From what's been discussed so far, I don't see any advantage >> of isinstance(o, String) over hasattr(o, 'encode') or somesuch. > > Look, even if there were *no* additional methods, it's worth adding > the base class, just to differentiate the class from the Sequence, as > a marker, so that those of us who want to ask "isinstance(o, String)" > can do so. > > Personally, I'd add in all the string methods to that class, in all > their gory complexity. Those who need a compliant class should > subclass the String base class, and override/add what they need. I'm not sure I agree with you on the solution, but I definitely agree that although str/unicode are conceptually sequences of characters, it is rarely useful to think of them as iterables of objects, unlike all other Sequences. (Note: I don't dispute that it is occasionally useful to treat them as such.) In my perfect world, strings would be indicable and sliceable, but not iterable. A character iterator could be obtained using a new method, such as .chars(). s = 'hello world' list(s) # exception set(s) # exception tuple(s) # exception for char in s: # exception [ord(c) for c in s] # exception s[2] # ok s[::-1] # ok for char in s.chars(): # ok [ord(c) for c in s.chars()] # ok Though an argument could be made against this, I consider the current behaviour of strings one of the few instances where purity beats practicality in python. It is often the cause of errors that fail too late in my experience. -Mike From greg.ewing at canterbury.ac.nz Thu May 29 02:17:03 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 29 May 2008 12:17:03 +1200 Subject: [Python-Dev] A thought on generic functions In-Reply-To: <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> Message-ID: <483DF5FF.7010201@canterbury.ac.nz> Paul Moore wrote: > I'd rather see a solution which addressed the > wider visitor use case (I think I just sprained my back bending over > backwards to avoid mentioning generic functions :-)) Speaking of generic functions, while thinking about the recent discussion on proxy objects, it occurred to me that this is something you can do with an OO system that you can't do so easily with a generic function system. If the operations being proxied were generic functions rather than methods, you'd have to override them all individually instead of having a central point to catch them all. -- Greg From greg.ewing at canterbury.ac.nz Thu May 29 02:40:43 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 29 May 2008 12:40:43 +1200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <08May28.143320pdt.58698@synergy1.parc.xerox.com> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> Message-ID: <483DFB8B.5050909@canterbury.ac.nz> Bill Janssen wrote: > Look, even if there were *no* additional methods, it's worth adding > the base class, just to differentiate the class from the Sequence, as > a marker, so that those of us who want to ask "isinstance(o, String)" > can do so. Doesn't isinstance(x, basestring) already cover that? -- Greg From greg.ewing at canterbury.ac.nz Thu May 29 02:44:34 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 29 May 2008 12:44:34 +1200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <6B5CC2A3-EC91-4589-89F2-B4F51FB191B8@gmail.com> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <6B5CC2A3-EC91-4589-89F2-B4F51FB191B8@gmail.com> Message-ID: <483DFC72.1010106@canterbury.ac.nz> Mike Klaas wrote: > In my perfect world, strings would be indicable and sliceable, but not > iterable. An object that was indexable but not iterable would be a very strange thing. If it has __len__ and __getitem__, there's nothing to stop you iterating over it by hand anyway, so disallowing __iter__ would just seem perverse. A case could be made for strings being sliceable but neither indexable nor iterable, but it's probably too late to make such a radical change now. -- Greg From mike.klaas at gmail.com Thu May 29 04:59:02 2008 From: mike.klaas at gmail.com (Mike Klaas) Date: Wed, 28 May 2008 19:59:02 -0700 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <483DFC72.1010106@canterbury.ac.nz> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <6B5CC2A3-EC91-4589-89F2-B4F51FB191B8@gmail.com> <483DFC72.1010106@canterbury.ac.nz> Message-ID: <1C3D060A-9535-477F-AA41-C98282049022@gmail.com> On 28-May-08, at 5:44 PM, Greg Ewing wrote: > Mike Klaas wrote: > >> In my perfect world, strings would be indicable and sliceable, but >> not iterable. > > An object that was indexable but not iterable would > be a very strange thing. If it has __len__ and __getitem__, > there's nothing to stop you iterating over it by hand > anyway, so disallowing __iter__ would just seem perverse. Python has a beautiful abstraction in iteration: iter() is a generic function that allows you lazily consume a sequence of objects, whether it be lists, tuples, custom iterators, generators, or what have you. It is trivial to write your code to be agnostic to the type of iterable passed-in. Almost anything else a consumer of your code passes in will result in an immediate exception. Unfortunately, python has two extremely common data types which do not fail when this generic function is applied to them, and instead almost always returns a result which is not desired. Instead, it iterates over the characters of the string, a behaviour which is rarely needed in practice due to the wealth of methods available. I agree that it would be perverse to disallowing iterating over a string. I just wish that the way to do that wasn't glommed on to the object-iteration abstraction. As it stands, any consumer of iterables has to keep strings in mind. It is particularly irksome when the target input is an iterable of strings. I recall a function that accepts a list/iterable of item keys, hashes them, and then retrieves values based on the item hashes (usually over the network, so it is necessary to batch requests). This function is often used in the interactive interpreter, and it is thus very prone to being passed-in a string rather than a list. There was no good way to prevent the (frequent) mysterious "not found" errors save adding an explicit type check for basestring. String already behaves slightly differently from the way other sequences act: It is the only sequence for which 'seq in seq' is true, and the only sequence for which 'x in seq' can be true but 'any(x==item for item in seq)' is false. Abstractions are sometimes imperfect: this is why there is an explicit typecheck for strings in the sum() builtin. I'll stop here as I realize that the likelihood that this will be accepted is terribly small, especially considering the late stage of the process. But I would be willing to develop a patch that implements this behaviour on the off chance it is. -Mike From brett at python.org Thu May 29 06:38:05 2008 From: brett at python.org (Brett Cannon) Date: Wed, 28 May 2008 21:38:05 -0700 Subject: [Python-Dev] Finishing up PEP 3108 Message-ID: The issues related to PEP 3108 now total 14. With the beta (supposedly) in a week, I am hoping the last minor details can be pulled together or decisions made on what can be postponed and what should definitely be considered a release blocker. Issue 2847 - the aifc module still imports the cl module in 3.0. Problem is that the cl module is gone. =) So it seems silly to have the imports lying about. This can probably be changed to critical. Issue 2848 - mimetools has been deprecated for a while, but it is still used in a bunch of places. Since this has been deprecated in PEP 4 for a long time, should we add the removal warning in 2.6 now and then make its actual removal of usage something to do by another beta? Issue 2849 - rfc822 is the same problem as mimetools. Issue 2854 - gestalt needs to be added back into 3.0. This is Benjamin's issue. =) Issue 2873 - htmllib is slated to go, but pydoc still uses it. Then again, pydoc is busted thanks to the new doc format. Issue 2874 - the stat module is not so useful anymore, but it still has functions that are useful. Currently the value returned by os.stat() is a named tuple, but that won't support methods. So the object returned by os.stat() needs to probably become a proper object in posix. Issue 2876 - The UserDict module has been removed in 3.0, but two classes were moved and renamed in collections and another was removed. The removal is a 3.0 warning, but the class renaming might be a tricky 2to3 fixer (not sure if the fix_imports fixer can be tweaked to handle this). Issue 2877 - UserString.UserString moved. Just need to apply the patch. Issue 2878 - Ditto for UserList. Issue 2885 - Creation of the urllib package. Jeremy has been working on this. I believe his patch is up on rietveld. Issue 2917 - This is merging pickle and cPickle. Alexandre's thing. Issue 2918 - Same for StringIO/cStringIO. Issue 2919 - profile and cProfile needs to be merged. This has not been dealt with yet. Would it be reasonable to deprecate importing cProfile directly in 2.6 with the assumption the merge will work out for 3.0? So that is everything that's left. Issue 2775 is the tracking issue so you can look there to see what issues are still open and need work. I was hoping to spend Monday and Tuesday trying to tie up as many loose ends as possible, but the conference paper I have been working on that was due Sunday is now due a week later, and so Monday and Tuesday will be spent on that (supervisor's orders). Plus I am flying out Wednesday for 10 days to help my mother move and I don't know when I will get Net again. In other words, I still need help. =) -Brett P.S.: A huge thanks goes to everyone who has helped so far. My life has been nothing but stress for a while now and you guys have helped keep the stress from reaching epic proportions. From g.brandl at gmx.net Thu May 29 09:03:29 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 29 May 2008 09:03:29 +0200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <483DFB8B.5050909@canterbury.ac.nz> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Bill Janssen wrote: > >> Look, even if there were *no* additional methods, it's worth adding >> the base class, just to differentiate the class from the Sequence, as >> a marker, so that those of us who want to ask "isinstance(o, String)" >> can do so. > > Doesn't isinstance(x, basestring) already cover that? That doesn't cover UserString, for example. Georg From g.brandl at gmx.net Thu May 29 09:12:26 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 29 May 2008 09:12:26 +0200 Subject: [Python-Dev] Finishing up PEP 3108 In-Reply-To: References: Message-ID: Brett Cannon schrieb: > The issues related to PEP 3108 now total 14. With the beta > (supposedly) in a week, I am hoping the last minor details can be > pulled together or decisions made on what can be postponed and what > should definitely be considered a release blocker. > > Issue 2847 - the aifc module still imports the cl module in 3.0. > Problem is that the cl module is gone. =) So it seems silly to have > the imports lying about. This can probably be changed to critical. It shouldn't be a problem to rip everything cl-related out of aifc. The question is how useful aifc will be after that ... > Issue 2848 - mimetools has been deprecated for a while, but it is > still used in a bunch of places. Since this has been deprecated in PEP > 4 for a long time, should we add the removal warning in 2.6 now and > then make its actual removal of usage something to do by another beta? > > Issue 2849 - rfc822 is the same problem as mimetools. The problem is that nobody seems to know what exactly distinguishes mimetools/rfc822' classes and its successor's (email's) classes, so it's hard to replace it in the stdlib. > Issue 2873 - htmllib is slated to go, but pydoc still uses it. Then > again, pydoc is busted thanks to the new doc format. I will try to handle this in the coming week. > Issue 2919 - profile and cProfile needs to be merged. This has not > been dealt with yet. Would it be reasonable to deprecate importing > cProfile directly in 2.6 with the assumption the merge will work out > for 3.0? That's not the right way to go, you don't want to deprecate cStringIO or cPickle either. > So that is everything that's left. Issue 2775 is the tracking issue so > you can look there to see what issues are still open and need work. I > was hoping to spend Monday and Tuesday trying to tie up as many loose > ends as possible, but the conference paper I have been working on that > was due Sunday is now due a week later, and so Monday and Tuesday will > be spent on that (supervisor's orders). Plus I am flying out Wednesday > for 10 days to help my mother move and I don't know when I will get > Net again. In other words, I still need help. =) Let's hope we get this right in time. Then again, there are lots of other release blockers, so it may well be that the beta is delayed by some time. Georg From jcea at jcea.es Thu May 29 09:34:18 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 29 May 2008 09:34:18 +0200 Subject: [Python-Dev] [Python-3000] PyString -> PyBytes C API renaming (Stabilizing the C API of 2.6 and 3.0) In-Reply-To: <483D5444.8000705@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> <483D5444.8000705@egenix.com> Message-ID: <483E5C7A.2090507@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 M.-A. Lemburg wrote: | If you use PyBytes APIs, you expect to find PyBytes functions in | the libs and also set breakpoints on these. Very good point. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSD5ccplgi5GaxT1NAQIZwQP/SMW+GFHxPWui2/tjj2DgZtnzYigjQj/o T8/DYFXEwls65E1xukOi3zS9ePU49u+i36EaVOvYmYdasedTmODnV3anmBo49VFv rsWWr4BBbRwLj4TjjwWPGy7KNKCvyG/mIiBH0uq9tOe2oW9gZng67e1f3snBIite mw4qF6w9bmw= =1Rh8 -----END PGP SIGNATURE----- From solipsis at pitrou.net Thu May 29 11:22:02 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 29 May 2008 09:22:02 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Why_is_type=5Fmodified=28=29_in_typeobject?= =?utf-8?q?=2Ec_not_a_public_function=3F?= References: Message-ID: Stefan Behnel behnel.de> writes: > BTW, I noticed that the code in typeobject.c uses "DECREF before set" two > times, like this: > > method_cache[h].version = type->tp_version_tag; > method_cache[h].value = res; /* borrowed */ > Py_INCREF(name); > Py_DECREF(method_cache[h].name); > method_cache[h].name = name; Since this is so common, would it be useful to define a macro for the correct construct? Something like #define Py_SETREF(var, obj) \ { PyObject *tmp = (var); Py_INCREF(obj); \ (var) = (obj); Py_XDECREF(tmp); } Or, if we want to allow more optimizations, make two versions of it: #define Py_SETREF(var, obj) \ { PyObject *tmp = (var); Py_INCREF(obj); \ (var) = (obj); Py_DECREF(tmp); } #define Py_XSETREF(var, obj) \ { PyObject *tmp = (var); Py_INCREF(obj); \ (var) = (obj); Py_XDECREF(tmp); } Regards Antoine. From ncoghlan at gmail.com Thu May 29 11:53:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 May 2008 19:53:05 +1000 Subject: [Python-Dev] [Python-3000] PyString -> PyBytes C API renaming In-Reply-To: References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> <483D5444.8000705@egenix.com> Message-ID: <483E7D01.4010603@gmail.com> Stefan Behnel wrote: > M.-A. Lemburg wrote: >> If you use PyBytes APIs, you expect to find PyBytes functions in >> the libs and also set breakpoints on these. > > AFAICT, the PyBytes_* functions are in both Py2.6 and Py3 now, so no problem here. The PyBytes_* functions appear to be there, but a preprocessor macro means it is actually the PyString_* functions that appear in the Python DLL. That's great from a backwards compatibility point of view, but seriously confusing from the point of view of anyone trying to embed or otherwise debug Python 2.6. > Besides, how likely is it that users set a breakpoint on the PyBytes/PyString > functions? Not very likely at all - but it would still be nice if the PyBytes_* symbols were visible to the linker as well as the preprocessor. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From stefan_ml at behnel.de Thu May 29 11:55:38 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 29 May 2008 11:55:38 +0200 Subject: [Python-Dev] Py_SETREF/Py_XSETREF (was: Why is type_modified() in typeobject.c not a public function?) In-Reply-To: References: Message-ID: Antoine Pitrou wrote: > Stefan Behnel behnel.de> writes: >> BTW, I noticed that the code in typeobject.c uses "DECREF before set" two >> times, like this: >> >> method_cache[h].version = type->tp_version_tag; >> method_cache[h].value = res; /* borrowed */ >> Py_INCREF(name); >> Py_DECREF(method_cache[h].name); >> method_cache[h].name = name; > > Since this is so common, would it be useful to define a macro for the correct > construct? > > Something like > > #define Py_SETREF(var, obj) \ > { PyObject *tmp = (var); Py_INCREF(obj); \ > (var) = (obj); Py_XDECREF(tmp); } > > Or, if we want to allow more optimizations, make two versions of it: > > #define Py_SETREF(var, obj) \ > { PyObject *tmp = (var); Py_INCREF(obj); \ > (var) = (obj); Py_DECREF(tmp); } > > #define Py_XSETREF(var, obj) \ > { PyObject *tmp = (var); Py_INCREF(obj); \ > (var) = (obj); Py_XDECREF(tmp); } Both sound like a good idea to me. Having to think about whether or not DECREF poses a problem in a specific case is just cumbersome. Stefan From lists at cheimes.de Thu May 29 11:59:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 29 May 2008 11:59:28 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: References: <48397ECC.9070805@cheimes.de> Message-ID: <483E7E80.70403@cheimes.de> Stefan Behnel schrieb: > Christian Heimes wrote: >> * add a new file stringobject.h which contains the aliases PyString_ -> >> PyBytes_ > > Just a quick note that that file is still missing from SVN, so it's kind of > hard to compile existing code against the current branch state... No, the file is in SVN. It's just not in the py3k branch because it's not vital to the core. I had plans to add a Python 2.x compatibility header to Python 3.0 But I'm not going to spend any more time on the topic or any other development until we have reached an agreement on the naming. I don't want to waste more of my free time in vain. Christian From lists at cheimes.de Thu May 29 12:01:44 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 29 May 2008 12:01:44 +0200 Subject: [Python-Dev] PyString -> PyBytes C API renaming In-Reply-To: References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> <483D5444.8000705@egenix.com> Message-ID: <483E7F08.80704@cheimes.de> Stefan Behnel schrieb: > M.-A. Lemburg wrote: >> If you use PyBytes APIs, you expect to find PyBytes functions in >> the libs and also set breakpoints on these. > > AFAICT, the PyBytes_* functions are in both Py2.6 and Py3 now, so no problem here. In Python 2.6 the PyBytes_* functions are only available to the compiler but not to the linker. In 2.6 the ABI functions are PyString_* and in 3.0 it's PyBytes_* Christian From ncoghlan at gmail.com Thu May 29 12:03:36 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 May 2008 20:03:36 +1000 Subject: [Python-Dev] A thought on generic functions In-Reply-To: <483DF5FF.7010201@canterbury.ac.nz> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> <483DF5FF.7010201@canterbury.ac.nz> Message-ID: <483E7F78.8020109@gmail.com> Greg Ewing wrote: > Paul Moore wrote: >> I'd rather see a solution which addressed the >> wider visitor use case (I think I just sprained my back bending over >> backwards to avoid mentioning generic functions :-)) > > Speaking of generic functions, while thinking about the > recent discussion on proxy objects, it occurred to me > that this is something you can do with an OO system > that you can't do so easily with a generic function > system. If the operations being proxied were generic > functions rather than methods, you'd have to override > them all individually instead of having a central point > to catch them all. I don't think it would actually be that much worse - something like typetools.ProxyMixin would just involve a whole series of register calls instead of method definitions. I wouldn't expect the total amount of code involved to change much. That said, a recursive flatten() implementation is indeed a problem that generic functions are well suited to solving - have the default implementation attempt to iterate over the passed in object yielding its contents, yielding the object itself only if iteration fails, and then, for the types the application wishes to consider atomic, register an alternative implementation that just yields the object without attempting to iterate over it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu May 29 12:18:11 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 May 2008 20:18:11 +1000 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> Message-ID: <483E82E3.8040705@gmail.com> Jesse Noller wrote: > Georg kindly published the PEP I submitted last night to the PEP site: > > http://www.python.org/dev/peps/pep-0371/ > > This PEP includes some of the previous discussion on the processing > module's inclusion, and I hope clears up/clarifies some of the > goals/non goals and issues. I also included benchmark data and a link > to the code used for said benchmarks. > > I would like to renew the discussion now that "there is a PEP" to see > if there are any outstanding things people would like to get resolved. > I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback > both here and elsewhere that people would rather see this in sooner in > some form, rather than later (i.e.: 2.7/3.1). +1 from me (under the 'multiprocessing' name, with the understanding that some code duplication with other parts of the standard library may still remain in 2.6/3.0). From a non-web programmer's point of view, it seems like even more of a gain than standard library support for JSON ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From stefan_ml at behnel.de Thu May 29 12:21:32 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 29 May 2008 12:21:32 +0200 Subject: [Python-Dev] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483E7E80.70403@cheimes.de> References: <48397ECC.9070805@cheimes.de> <483E7E80.70403@cheimes.de> Message-ID: Christian Heimes wrote: > Stefan Behnel schrieb: >> Christian Heimes wrote: >>> * add a new file stringobject.h which contains the aliases PyString_ -> >>> PyBytes_ >> Just a quick note that that file is still missing from SVN, so it's kind of >> hard to compile existing code against the current branch state... > > No, the file is in SVN. It's just not in the py3k branch because it's > not vital to the core. It might be vital to those who try to test their code with Py3, though... :) > I had plans to add a Python 2.x compatibility header to Python 3.0 But > I'm not going to spend any more time on the topic or any other > development until we have reached an agreement on the naming. I don't > want to waste more of my free time in vain. Sounds to me like people should get back to real work rather than keeping bike-shedding in this thread... Stefan From fuzzyman at voidspace.org.uk Thu May 29 12:22:15 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 29 May 2008 11:22:15 +0100 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <483E82E3.8040705@gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <483E82E3.8040705@gmail.com> Message-ID: <483E83D7.2030907@voidspace.org.uk> Nick Coghlan wrote: > Jesse Noller wrote: > > Georg kindly published the PEP I submitted last night to the PEP site: > > > > http://www.python.org/dev/peps/pep-0371/ > > > > This PEP includes some of the previous discussion on the processing > > module's inclusion, and I hope clears up/clarifies some of the > > goals/non goals and issues. I also included benchmark data and a link > > to the code used for said benchmarks. > > > > I would like to renew the discussion now that "there is a PEP" to see > > if there are any outstanding things people would like to get resolved. > > I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback > > both here and elsewhere that people would rather see this in sooner in > > some form, rather than later (i.e.: 2.7/3.1). > > +1 from me (under the 'multiprocessing' name, with the understanding > that some code duplication with other parts of the standard library > may still remain in 2.6/3.0). +1 from me as well. I think multiple-processes is over played as a concurrency solution in Python (where you need to marshal lots of data in and out, the overheads of multiple processes can be very expensive) - but it is a very good solution for some problems. Michael Foord > > From a non-web programmer's point of view, it seems like even more of > a gain than standard library support for JSON ;) > > Cheers, > Nick. > -- http://www.ironpythoninaction.com/ http://www.theotherdelia.co.uk/ http://www.voidspace.org.uk/ http://www.ironpython.info/ http://www.resolverhacks.net/ From stefan_ml at behnel.de Thu May 29 12:28:21 2008 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 29 May 2008 12:28:21 +0200 Subject: [Python-Dev] [Python-3000] PyString -> PyBytes C API renaming In-Reply-To: <483E7D01.4010603@gmail.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> <483D5444.8000705@egenix.com> <483E7D01.4010603@gmail.com> Message-ID: Nick Coghlan wrote: > Stefan Behnel wrote: >> Besides, how likely is it that users set a breakpoint on the >> PyBytes/PyString functions? > > Not very likely at all - but it would still be nice if the PyBytes_* > symbols were visible to the linker as well as the preprocessor. Right, that's a nice-to-have, an add-on. Why don't we just let Christian finish his work, which is vital for the beta release? Then it's still time to file a bug report on the missing bits and provide a patch that adds linker symbols for PyBytes_*() in Py2.6 as an additional feature. Stefan From ncoghlan at gmail.com Thu May 29 12:34:58 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 May 2008 20:34:58 +1000 Subject: [Python-Dev] [Python-3000] PyString -> PyBytes C API renaming In-Reply-To: References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> <483D5444.8000705@egenix.com> <483E7D01.4010603@gmail.com> Message-ID: <483E86D2.5000809@gmail.com> Stefan Behnel wrote: > Nick Coghlan wrote: >> Stefan Behnel wrote: >>> Besides, how likely is it that users set a breakpoint on the >>> PyBytes/PyString functions? >> Not very likely at all - but it would still be nice if the PyBytes_* >> symbols were visible to the linker as well as the preprocessor. > > Right, that's a nice-to-have, an add-on. Why don't we just let Christian > finish his work, which is vital for the beta release? Then it's still time to > file a bug report on the missing bits and provide a patch that adds linker > symbols for PyBytes_*() in Py2.6 as an additional feature. Yeah, it took me a while to get my head around what he was trying to do, but GPS explained it pretty well elsewhere in this thread. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jnoller at gmail.com Thu May 29 14:49:17 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 29 May 2008 08:49:17 -0400 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <483E83D7.2030907@voidspace.org.uk> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <483E82E3.8040705@gmail.com> <483E83D7.2030907@voidspace.org.uk> Message-ID: <4222a8490805290549v70c76195r2cfc4bd68b79362f@mail.gmail.com> On Thu, May 29, 2008 at 6:22 AM, Michael Foord wrote: > Nick Coghlan wrote: >> >> Jesse Noller wrote: >> > Georg kindly published the PEP I submitted last night to the PEP site: >> > >> > http://www.python.org/dev/peps/pep-0371/ >> > >> > This PEP includes some of the previous discussion on the processing >> > module's inclusion, and I hope clears up/clarifies some of the >> > goals/non goals and issues. I also included benchmark data and a link >> > to the code used for said benchmarks. >> > >> > I would like to renew the discussion now that "there is a PEP" to see >> > if there are any outstanding things people would like to get resolved. >> > I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback >> > both here and elsewhere that people would rather see this in sooner in >> > some form, rather than later (i.e.: 2.7/3.1). >> >> +1 from me (under the 'multiprocessing' name, with the understanding that >> some code duplication with other parts of the standard library may still >> remain in 2.6/3.0). > > +1 from me as well. > > I think multiple-processes is over played as a concurrency solution in > Python (where you need to marshal lots of data in and out, the overheads of > multiple processes can be very expensive) - but it is a very good solution > for some problems. > > Michael Foord Agreed - this is a "step" rather than the final solution. As I pointed out in the PEP this is a method to side-step GIL limitations rather than to address the larger "GIL issue", I am implicitly assuming that no movement will be made on that front until the bulk of Adam Olsen's safethreading work is rolled into future versions. -jesse From qgallet at gmail.com Thu May 29 15:25:12 2008 From: qgallet at gmail.com (Quentin Gallet-Gilles) Date: Thu, 29 May 2008 15:25:12 +0200 Subject: [Python-Dev] Finishing up PEP 3108 In-Reply-To: References: Message-ID: <8b943f2b0805290625w19ab1fd3l48f00f40e630c39d@mail.gmail.com> On Thu, May 29, 2008 at 9:12 AM, Georg Brandl wrote: > Brett Cannon schrieb: > >> The issues related to PEP 3108 now total 14. With the beta >> (supposedly) in a week, I am hoping the last minor details can be >> pulled together or decisions made on what can be postponed and what >> should definitely be considered a release blocker. >> >> Issue 2847 - the aifc module still imports the cl module in 3.0. >> Problem is that the cl module is gone. =) So it seems silly to have >> the imports lying about. This can probably be changed to critical. >> > > It shouldn't be a problem to rip everything cl-related out of aifc. > The question is how useful aifc will be after that ... > Has someone already used that module ? I took a look into it, but I'm a bit confused about the various compression types, case-sensitivity and compatibility issues [1]. Are Apple's "alaw" and SGI's "ALAW" really the same encoding ? Can we use the audioop module for ALAW, just like it's already done for ULAW ? [1] http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/AIFF/AIFF.html Quentin -------------- next part -------------- An HTML attachment was scrubbed... URL: From skip at pobox.com Tue May 27 13:35:10 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 27 May 2008 06:35:10 -0500 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <483BD10E.7050600@gmail.com> References: <483BD10E.7050600@gmail.com> Message-ID: <18491.61934.571753.948998@montanaro-dyndns-org.local> Nick> $ ./python -m timeit "'' % ()" Nick> 1000000 loops, best of 3: 0.389 usec per loop vs. Nick> $ ./python -m timeit "'%s' % 'nothing'" Nick> 10000000 loops, best of 3: 0.0736 usec per loop I think you need to use a tuple for the second case to make it comparable to the first. Skip From jcea at jcea.es Thu May 29 15:39:09 2008 From: jcea at jcea.es (Jesus Cea) Date: Thu, 29 May 2008 15:39:09 +0200 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <483E83D7.2030907@voidspace.org.uk> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <483E82E3.8040705@gmail.com> <483E83D7.2030907@voidspace.org.uk> Message-ID: <483EB1FD.6070503@jcea.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Could you possibly extend the PEP to also document performance when, for instance, passing "big" objects via a queue, or sending "Events" back, testing "thread.isAlive()", and stuff like that?. What about mutexes? (not to protect shared objects, but files, for example). A share-nothing without data-passing doesn't need a new module :). I'm interesting in an almost direct conversion from thread module, and so I'm interested in knowing performance data outside "pyprocessing" sweet point (that is, "fire and forget" code, with little communication). How is implemented "thread.setDaemon()"?. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSD6x9plgi5GaxT1NAQI4DQP/dJWp2Mfj8e5M1zMM7JNJFFyBMRAxwagd 8EAS1Kbbl4sH8mhaucOwBntj6uXi55HOXo/+bcPD8OuMGq1gouIo7ZJvn3+SNoRM OKsL9Judks8GAZmSF40COAziqQ/Y9M73tOoRBHtqbSTFvexbeKbfY4DvfE+/erss g+oCjUi76FY= =erA0 -----END PGP SIGNATURE----- From ncoghlan at gmail.com Thu May 29 15:44:01 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 29 May 2008 23:44:01 +1000 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <18491.61934.571753.948998@montanaro-dyndns-org.local> References: <483BD10E.7050600@gmail.com> <18491.61934.571753.948998@montanaro-dyndns-org.local> Message-ID: <483EB321.2030901@gmail.com> skip at pobox.com wrote: > Nick> $ ./python -m timeit "'' % ()" > Nick> 1000000 loops, best of 3: 0.389 usec per loop > > vs. > > Nick> $ ./python -m timeit "'%s' % 'nothing'" > Nick> 10000000 loops, best of 3: 0.0736 usec per loop > > I think you need to use a tuple for the second case to make it comparable to > the first. It doesn't actually make that big a difference - I'm guessing a Py_INCREF shortcut ends up getting used either way: $ ./python -m timeit "'%s' % 'nothing'" 10000000 loops, best of 3: 0.0848 usec per loop $ ./python -m timeit "'%s' % 'nothing'," 10000000 loops, best of 3: 0.133 usec per loop $ ./python -m timeit "'' % ()" 1000000 loops, best of 3: 0.513 usec per loop (times are a bit variable at this very moment since I have a few different apps open) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jnoller at gmail.com Thu May 29 16:32:23 2008 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 29 May 2008 10:32:23 -0400 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <483EB1FD.6070503@jcea.es> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <483E82E3.8040705@gmail.com> <483E83D7.2030907@voidspace.org.uk> <483EB1FD.6070503@jcea.es> Message-ID: <4222a8490805290732o787fde66sf5ca3186a0c73d60@mail.gmail.com> On Thu, May 29, 2008 at 9:39 AM, Jesus Cea wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Could you possibly extend the PEP to also document performance when, for > instance, passing "big" objects via a queue, or sending "Events" back, > testing "thread.isAlive()", and stuff like that?. What about mutexes? > (not to protect shared objects, but files, for example). > > A share-nothing without data-passing doesn't need a new module :). I'm > interesting in an almost direct conversion from thread module, and so > I'm interested in knowing performance data outside "pyprocessing" sweet > point (that is, "fire and forget" code, with little communication). > > How is implemented "thread.setDaemon()"?. Alec Thomas sent me a bit of code to benchmark Queue-based object passing performance which I will incorporate when I get a chance. As for the provided examples/benchmarks - I can work on adding more, or if you want - as linked in the PEP, Oudkerk already has some of those outlined in a benchmark script here: http://pyprocessing.berlios.de/examples/benchmarks.py I chose not to recreate his tests directly, rather I chose to link to them. I will work on adding Queue-based numbers. I also wouldn't say I picked the "sweet spot" for the module - rather I picked the poor-spot for the threading module (parallel, python-based crunching). I do again want to point out that the goal is not to pick on threading, but to offer an API which mimics the existing threading API that allows for actual multi-processor/core usage. -jesse From mal at egenix.com Thu May 29 16:51:25 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 May 2008 16:51:25 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <08May28.100829pdt."58698"@synergy1.parc.xerox.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <08May28.100829pdt."58698"@synergy1.parc.xerox.com> Message-ID: <483EC2ED.7010104@egenix.com> On 2008-05-28 19:08, Bill Janssen wrote: >> I'm beginning to wonder whether I'm the only one who cares about >> the Python 2.x branch not getting cluttered up with artifacts caused >> by a broken forward merge strategy. > > I share your concern. Seems to me that perhaps (not sure, but > perhaps) the rush to back-port from 3.x, and the concern about > minimizing pain of moving from 2.x to 3.x, has become the tail wagging > the dog. Indeed. If the need to be able to forward merge changes from the 2.x trunk to the 3.x branch is the only reason for the current approach, then we need to find a better procedure for getting patches to 2.x forwarded to 3.x. I believe that everyone is aware that 3.x breaks things and that's fine. However, the reason for introducing such breakage in 3.x is that users have the option to decide whether and when to switch to the new major version. Being able to play with 3.x features in 2.x is nice, but I wouldn't really consider those essential for 2.x. It certainly doesn't warrant causing major problems in the 2.x releases. The module renaming backport was one example (which was undone again), the C API renaming is another. I expect more such features to be backported from 3.x to 2.x (even though I don't really think it's worth the trouble) and since this always means that changes have to applied in two worlds, we'll need a better process for getting changes in one major release ported to the other. Simply tweaking 2.x into shape so that the rather simple minded SVN merge command works, isn't a good enough procedure for this. That's why I suggested to use an intermediate form or branch for the merging - one that implements the 2.x with all renaming and syntax fixing applied. This would: * reduce the number of merge conflicts since the renaming would already have happened * reduce the patch sizes that have to be applied to 3.x in order to stay in sync with 2.x * result in a tool chain that makes it easier for all Python users to port their code to 3.x * simplify renaming or reorg of modules, functions, methods and C APIs without requiring major changes on either side -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 29 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 38 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From mal at egenix.com Thu May 29 17:22:58 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 May 2008 17:22:58 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> Message-ID: <483ECA52.6040000@egenix.com> On 2008-05-28 22:47, Gregory P. Smith wrote: > On Wed, May 28, 2008 at 3:12 AM, M.-A. Lemburg wrote: >> I'm beginning to wonder whether I'm the only one who cares about >> the Python 2.x branch not getting cluttered up with artifacts caused >> by a broken forward merge strategy. >> >> How can it be that we allow major C API changes such as the renaming >> of the PyString APIs to go into the trunk without discussion or >> a PEP ? > > I do not consider it a C API change. The API and ABI have not > changed. Old code still compiles. Old binaries still dynamically > load and work fine. (I just confirmed this by importing a couple > python2.4 .so files into my non-debug build of 2.6 trunk) > > A of the PyString APIs are the real implementations in 2.x and are > still there. We only switched to using their PyBytes equivalent names > within the Python trunk code base. > > Are you objecting to our own code switching to use a different name > even though the actual underlying API and ABI haven't changed? I > suppose to people reading the code and going against old reference > books it could be confusing but they've got to get used to the new > names somehow and sometime. > > I strongly support changes like this one that makes the life of > porting C code forwards and backwards between 2.x and 3.x easier > without breaking compatibility with earlier 2.x version because that > is going to be a serious pain for all of us otherwise. Well, first of all, it is a change in the C API: APIs have different names now, they live in different files, the Python documentation doesn't apply anymore, books have to be updated, programmers trained, etc. etc. That's fine for 3.x, it's not for 2.x. Second, if you leave out the "ease merging" argument, all of this is not really necessary in 2.x. If you absolutely want to have PyBytes APIs in 2.x, then you can *add* them, without removing the PyString APIs. We have done that on a smaller scale a couple of times in the past (turned functions into macros or vice-versa). And finally, the "merge" argument itself is not really all that strong. It's just a matter of getting the procedure corrected. Then you can rename and restructure as much as you want in 3.x - without affecting the stability and matureness of the 2.x branch. I suspect more of these backports to happen, so we better get things done right now instead of putting Python's reputation as stable and mature programming language at risk. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 29 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 38 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From qgallet at gmail.com Thu May 29 17:39:17 2008 From: qgallet at gmail.com (Quentin Gallet-Gilles) Date: Thu, 29 May 2008 17:39:17 +0200 Subject: [Python-Dev] [Python-3000] Finishing up PEP 3108 In-Reply-To: <483EC414.7080603@ibp.de> References: <8b943f2b0805290625w19ab1fd3l48f00f40e630c39d@mail.gmail.com> <483EC414.7080603@ibp.de> Message-ID: <8b943f2b0805290839s7a1f3238g9e21407a56c34159@mail.gmail.com> On Thu, May 29, 2008 at 4:56 PM, Lars Immisch wrote: > > >> Issue 2847 - the aifc module still imports the cl module in 3.0. >> Problem is that the cl module is gone. =) So it seems silly to have >> the imports lying about. This can probably be changed to critical. >> >> >> It shouldn't be a problem to rip everything cl-related out of aifc. >> The question is how useful aifc will be after that ... >> >> >> Has someone already used that module ? I took a look into it, but I'm a >> bit confused about the various compression types, case-sensitivity and >> compatibility issues [1]. Are Apple's "alaw" and SGI's "ALAW" really the >> same encoding ? Can we use the audioop module for ALAW, just like it's >> already done for ULAW ? >> > > There is just one alaw I've ever come across (G.711), and the audioop > implementation could be used (audioop's alaw support is younger than the > aifc module, BTW) > > The capitalisation is confusing, but your document [1] says: "Apple > Computer's QuickTime player recognize only the Apple compression types. > Although "ALAW" and "ULAW" contain identical sound samples to the "alaw" and > "ulaw" formats and were in use long before Apple introduced the new codes, > QuickTime does not recognize them." > > So this seems just a matter of naming in the AIFC, but not a matter of two > different alaw implementations. > > - Lars > Ok, I'll handle this issue. I'll be using the audioop implementation as a replacement of the SGI compression library. I'll also create a test suite, as Brett mentioned in the bug tracker the module was missing one. Quentin > > [1] http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/AIFF/AIFF.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at cheimes.de Thu May 29 17:45:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 29 May 2008 17:45:24 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483ECA52.6040000@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> Message-ID: <483ECF94.7060607@cheimes.de> M.-A. Lemburg schrieb: > Well, first of all, it is a change in the C API: > APIs have different names now, they live in different files, > the Python documentation doesn't apply anymore, books have to > be updated, programmers trained, etc. etc. That's fine for > 3.x, it's not for 2.x. No, that's not correct. The 2.x API is still the same. I've only changed the internal code. > Second, if you leave out the "ease merging" argument, all of > this is not really necessary in 2.x. If you absolutely want > to have PyBytes APIs in 2.x, then you can *add* them, without > removing the PyString APIs. We have done that on a smaller > scale a couple of times in the past (turned functions into > macros or vice-versa). The PyString methods are still available and the official API for dealing with str objects in 2.x. > And finally, the "merge" argument itself is not really all that > strong. It's just a matter of getting the procedure corrected. > Then you can rename and restructure as much as you want in > 3.x - without affecting the stability and matureness of the > 2.x branch. I'm volunteering to revert my chances if you are volunteering to keep the Python 2.x series in sync with the 3.x series. Christian From brett at python.org Thu May 29 19:32:04 2008 From: brett at python.org (Brett Cannon) Date: Thu, 29 May 2008 10:32:04 -0700 Subject: [Python-Dev] Finishing up PEP 3108 In-Reply-To: References: Message-ID: On Thu, May 29, 2008 at 12:12 AM, Georg Brandl wrote: > Brett Cannon schrieb: >> >> The issues related to PEP 3108 now total 14. With the beta >> (supposedly) in a week, I am hoping the last minor details can be >> pulled together or decisions made on what can be postponed and what >> should definitely be considered a release blocker. >> >> Issue 2847 - the aifc module still imports the cl module in 3.0. >> Problem is that the cl module is gone. =) So it seems silly to have >> the imports lying about. This can probably be changed to critical. > > It shouldn't be a problem to rip everything cl-related out of aifc. > The question is how useful aifc will be after that ... > If it ends up not being useful then the module can just go. >> Issue 2848 - mimetools has been deprecated for a while, but it is >> still used in a bunch of places. Since this has been deprecated in PEP >> 4 for a long time, should we add the removal warning in 2.6 now and >> then make its actual removal of usage something to do by another beta? >> >> Issue 2849 - rfc822 is the same problem as mimetools. > > The problem is that nobody seems to know what exactly distinguishes > mimetools/rfc822' classes and its successor's (email's) classes, so > it's hard to replace it in the stdlib. > Right. I have looked myself over the years and it never seemed brain-dead simple. >> Issue 2873 - htmllib is slated to go, but pydoc still uses it. Then >> again, pydoc is busted thanks to the new doc format. > > I will try to handle this in the coming week. > Fred had the interesting suggestion of removing pydoc in Py3K based on the thinking that documentation tools like pydoc should be external to Python. With the docs now so easy to generate directly, should pydoc perhaps just be gutted to only what is needed for help() to work? >> Issue 2919 - profile and cProfile needs to be merged. This has not >> been dealt with yet. Would it be reasonable to deprecate importing >> cProfile directly in 2.6 with the assumption the merge will work out >> for 3.0? > > That's not the right way to go, you don't want to deprecate cStringIO > or cPickle either. > Yeah, sorry, you're right. Guess my brain was not fully working when I wrote that. =) >> So that is everything that's left. Issue 2775 is the tracking issue so >> you can look there to see what issues are still open and need work. I >> was hoping to spend Monday and Tuesday trying to tie up as many loose >> ends as possible, but the conference paper I have been working on that >> was due Sunday is now due a week later, and so Monday and Tuesday will >> be spent on that (supervisor's orders). Plus I am flying out Wednesday >> for 10 days to help my mother move and I don't know when I will get >> Net again. In other words, I still need help. =) > > Let's hope we get this right in time. > > Then again, there are lots of other release blockers, so it may well be > that the beta is delayed by some time. Guess it depends on the whim of the release manager. =) -Brett From mal at egenix.com Thu May 29 20:08:57 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 29 May 2008 20:08:57 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483ECF94.7060607@cheimes.de> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> Message-ID: <483EF139.8000606@egenix.com> Christian, so far you have not responded to any of the suggestions made on this thread, only defended your checkin. That's not very helpful in getting to some conclusion. * What's so hard about going with a proper, standard solution that doesn't involve using your preprocessor hack ? * Why can't we have both PyString *and* PyBytes exposed in 2.x, with one redirecting to the other ? * Why should the 2.x code base turn to hacks, just because 3.x wants to restructure itself ? * Why aren't you even considering my proposed solution for this whole renaming and reorg problem ? BTW: Is there some PEP or wiki page explaining how you actually implement the merging from 2.x to 3.x ? I'm still under the assumption that you're only using svnmerge.py for this and doing straight merging from the trunk to the branch. Not sure how others feel about it, but if the only option you would feel comfortable with is not having the 3.x renaming backported, then I'd rather go with that, really. It's easy enough to add a header file to map PyString APIs to PyBytes if you want to port an extension to 3.x. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 29 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 38 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 On 2008-05-29 17:45, Christian Heimes wrote: > M.-A. Lemburg schrieb: >> Well, first of all, it is a change in the C API: >> APIs have different names now, they live in different files, >> the Python documentation doesn't apply anymore, books have to >> be updated, programmers trained, etc. etc. That's fine for >> 3.x, it's not for 2.x. > > No, that's not correct. The 2.x API is still the same. I've only changed > the internal code. > >> Second, if you leave out the "ease merging" argument, all of >> this is not really necessary in 2.x. If you absolutely want >> to have PyBytes APIs in 2.x, then you can *add* them, without >> removing the PyString APIs. We have done that on a smaller >> scale a couple of times in the past (turned functions into >> macros or vice-versa). > > The PyString methods are still available and the official API for > dealing with str objects in 2.x. > >> And finally, the "merge" argument itself is not really all that >> strong. It's just a matter of getting the procedure corrected. >> Then you can rename and restructure as much as you want in >> 3.x - without affecting the stability and matureness of the >> 2.x branch. > > I'm volunteering to revert my chances if you are volunteering to keep > the Python 2.x series in sync with the 3.x series. > > Christian > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com From steve at holdenweb.com Thu May 29 20:18:18 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 29 May 2008 14:18:18 -0400 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <483EB321.2030901@gmail.com> References: <483BD10E.7050600@gmail.com> <18491.61934.571753.948998@montanaro-dyndns-org.local> <483EB321.2030901@gmail.com> Message-ID: <483EF36A.70502@holdenweb.com> Nick Coghlan wrote: > skip at pobox.com wrote: >> Nick> $ ./python -m timeit "'' % ()" >> Nick> 1000000 loops, best of 3: 0.389 usec per loop >> >> vs. >> >> Nick> $ ./python -m timeit "'%s' % 'nothing'" >> Nick> 10000000 loops, best of 3: 0.0736 usec per loop >> >> I think you need to use a tuple for the second case to make it >> comparable to >> the first. > > It doesn't actually make that big a difference - I'm guessing a > Py_INCREF shortcut ends up getting used either way: > > $ ./python -m timeit "'%s' % 'nothing'" > 10000000 loops, best of 3: 0.0848 usec per loop > $ ./python -m timeit "'%s' % 'nothing'," > 10000000 loops, best of 3: 0.133 usec per loop Technically there you aren't using a tuple to provide the substitution argument, you are doing the substitution from a single string then putting the result in a tuple: >>> '%s' % 'nothing', ('nothing',) >>> On the basis of evidence like this: sholden at lifeboy ~ $ python -m timeit "'%s' % 'nothing'" 10000000 loops, best of 3: 0.0705 usec per loop sholden at lifeboy ~ $ python -m timeit "'%s' % ('nothing',)" 1000000 loops, best of 3: 0.691 usec per loop I'd say not using a tuple was the way to go if ever you needed to optimize the code for speed. > $ ./python -m timeit "'' % ()" > 1000000 loops, best of 3: 0.513 usec per loop > If you want an even stranger result, it appears to take the interpreter longer to substitute nothing from a tuple than it does to substitute the empty string from a string: sholden at lifeboy ~ $ python -m timeit "'' % ()" 1000000 loops, best of 3: 0.454 usec per loop sholden at lifeboy ~ $ python -m timeit "'%s' % ''" 10000000 loops, best of 3: 0.0715 usec per loop > (times are a bit variable at this very moment since I have a few > different apps open) > Me too. Timings are not benchmarks, I am not a lawyer, etc. Our machines seem to be of comparable speed. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Thu May 29 20:18:18 2008 From: steve at holdenweb.com (Steve Holden) Date: Thu, 29 May 2008 14:18:18 -0400 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <483EB321.2030901@gmail.com> References: <483BD10E.7050600@gmail.com> <18491.61934.571753.948998@montanaro-dyndns-org.local> <483EB321.2030901@gmail.com> Message-ID: <483EF36A.70502@holdenweb.com> Nick Coghlan wrote: > skip at pobox.com wrote: >> Nick> $ ./python -m timeit "'' % ()" >> Nick> 1000000 loops, best of 3: 0.389 usec per loop >> >> vs. >> >> Nick> $ ./python -m timeit "'%s' % 'nothing'" >> Nick> 10000000 loops, best of 3: 0.0736 usec per loop >> >> I think you need to use a tuple for the second case to make it >> comparable to >> the first. > > It doesn't actually make that big a difference - I'm guessing a > Py_INCREF shortcut ends up getting used either way: > > $ ./python -m timeit "'%s' % 'nothing'" > 10000000 loops, best of 3: 0.0848 usec per loop > $ ./python -m timeit "'%s' % 'nothing'," > 10000000 loops, best of 3: 0.133 usec per loop Technically there you aren't using a tuple to provide the substitution argument, you are doing the substitution from a single string then putting the result in a tuple: >>> '%s' % 'nothing', ('nothing',) >>> On the basis of evidence like this: sholden at lifeboy ~ $ python -m timeit "'%s' % 'nothing'" 10000000 loops, best of 3: 0.0705 usec per loop sholden at lifeboy ~ $ python -m timeit "'%s' % ('nothing',)" 1000000 loops, best of 3: 0.691 usec per loop I'd say not using a tuple was the way to go if ever you needed to optimize the code for speed. > $ ./python -m timeit "'' % ()" > 1000000 loops, best of 3: 0.513 usec per loop > If you want an even stranger result, it appears to take the interpreter longer to substitute nothing from a tuple than it does to substitute the empty string from a string: sholden at lifeboy ~ $ python -m timeit "'' % ()" 1000000 loops, best of 3: 0.454 usec per loop sholden at lifeboy ~ $ python -m timeit "'%s' % ''" 10000000 loops, best of 3: 0.0715 usec per loop > (times are a bit variable at this very moment since I have a few > different apps open) > Me too. Timings are not benchmarks, I am not a lawyer, etc. Our machines seem to be of comparable speed. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Thu May 29 21:28:54 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 29 May 2008 12:28:54 -0700 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <483E82E3.8040705@gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <483E82E3.8040705@gmail.com> Message-ID: On Thu, May 29, 2008 at 3:18 AM, Nick Coghlan wrote: > Jesse Noller wrote: >> Georg kindly published the PEP I submitted last night to the PEP site: >> >> http://www.python.org/dev/peps/pep-0371/ >> >> This PEP includes some of the previous discussion on the processing >> module's inclusion, and I hope clears up/clarifies some of the >> goals/non goals and issues. I also included benchmark data and a link >> to the code used for said benchmarks. >> >> I would like to renew the discussion now that "there is a PEP" to see >> if there are any outstanding things people would like to get resolved. >> I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback >> both here and elsewhere that people would rather see this in sooner in >> some form, rather than later (i.e.: 2.7/3.1). > > +1 from me (under the 'multiprocessing' name, with the understanding that > some code duplication with other parts of the standard library may still > remain in 2.6/3.0). +1 from me as well, and I support the rename to multiprocessing ("processing" is just too much of a common word). This will provide a good short-term answer to those who still clamor for dropping the GIL. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu May 29 21:42:37 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 29 May 2008 12:42:37 -0700 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com><483E82E3.8040705@gmail.com> Message-ID: >>> I would like to renew the discussion now that "there is a PEP" to see >>> if there are any outstanding things people would like to get resolved. >>> I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback >>> both here and elsewhere that people would rather see this in sooner in >>> some form, rather than later (i.e.: 2.7/3.1). >> >> +1 from me (under the 'multiprocessing' name, with the understanding that >> some code duplication with other parts of the standard library may still >> remain in 2.6/3.0). > > +1 from me as well, and I support the rename to multiprocessing > ("processing" is just too much of a common word). > > This will provide a good short-term answer to those who still clamor > for dropping the GIL. I'm very happy this is going forward. It's a big win. Raymond From musiccomposition at gmail.com Thu May 29 22:49:08 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 29 May 2008 15:49:08 -0500 Subject: [Python-Dev] Finishing up PEP 3108 In-Reply-To: References: Message-ID: <1afaf6160805291349i6ca25d6fhef22eb6c3abd5a7d@mail.gmail.com> On Wed, May 28, 2008 at 11:38 PM, Brett Cannon wrote: > > Issue 2854 - gestalt needs to be added back into 3.0. This is > Benjamin's issue. =) Is that your way of say "Check in the patch!" ? :) -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Thu May 29 23:01:52 2008 From: brett at python.org (Brett Cannon) Date: Thu, 29 May 2008 14:01:52 -0700 Subject: [Python-Dev] Finishing up PEP 3108 In-Reply-To: <1afaf6160805291349i6ca25d6fhef22eb6c3abd5a7d@mail.gmail.com> References: <1afaf6160805291349i6ca25d6fhef22eb6c3abd5a7d@mail.gmail.com> Message-ID: On Thu, May 29, 2008 at 1:49 PM, Benjamin Peterson wrote: > On Wed, May 28, 2008 at 11:38 PM, Brett Cannon wrote: >> >> Issue 2854 - gestalt needs to be added back into 3.0. This is >> Benjamin's issue. =) > > Is that your way of say "Check in the patch!" ? :) > More or less; specifically, "don't forget to do this." =) -Brett From greg.ewing at canterbury.ac.nz Fri May 30 00:30:45 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 30 May 2008 10:30:45 +1200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <1C3D060A-9535-477F-AA41-C98282049022@gmail.com> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <6B5CC2A3-EC91-4589-89F2-B4F51FB191B8@gmail.com> <483DFC72.1010106@canterbury.ac.nz> <1C3D060A-9535-477F-AA41-C98282049022@gmail.com> Message-ID: <483F2E95.8050903@canterbury.ac.nz> Mike Klaas wrote: > I agree that it would be perverse to disallowing iterating over a > string. Just to be clear, I'm saying that it would be perverse to disallow iterating *but* to allow indexing of individual characters. Either you should have both or you should have neither. -- Greg From greg.ewing at canterbury.ac.nz Fri May 30 00:41:50 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 30 May 2008 10:41:50 +1200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> Message-ID: <483F312E.1020302@canterbury.ac.nz> Georg Brandl wrote: > Greg Ewing schrieb: > >> Doesn't isinstance(x, basestring) already cover that? > > That doesn't cover UserString, for example. A better solution to that might be to have UserString inherit from basestring. -- Greg From ncoghlan at gmail.com Fri May 30 00:57:07 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 May 2008 08:57:07 +1000 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483EF139.8000606@egenix.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> Message-ID: <483F34C3.3050402@gmail.com> M.-A. Lemburg wrote: > * Why can't we have both PyString *and* PyBytes exposed in 2.x, > with one redirecting to the other ? We do have that - the PyString_* names still work perfectly fine in 2.x. They just won't be used in the Python core codebase anymore - everything in the Python core will use either PyBytes_* or PyUnicode_* regardless of which branch (2.x or 3.x) you're working on. I think that's a good thing for ease of maintenance in the future, even if it takes people a while to get their heads around it right now. > * Why should the 2.x code base turn to hacks, just because 3.x wants > to restructure itself ? With the better explanation from Greg of what the checked in approach achieves (i.e. preserving exact ABI compatibility for PyString_*, while allowing PyBytes_* to be used at the source code level), I don't see what has been done as being any more of a hack than the possibly more common "#define " (which *would* break binary compatibility). The only things that I think would tidy it up further would be to: - include an explanation of the approach and its effects on API and ABI backward and forward compatibility within 2.x and between 2.x and 3.x in stringobject.h - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Fri May 30 00:57:51 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 30 May 2008 10:57:51 +1200 Subject: [Python-Dev] A thought on generic functions In-Reply-To: <483E7F78.8020109@gmail.com> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> <483DF5FF.7010201@canterbury.ac.nz> <483E7F78.8020109@gmail.com> Message-ID: <483F34EF.5080301@canterbury.ac.nz> Nick Coghlan wrote: > I don't think it would actually be that much worse - something like > typetools.ProxyMixin would just involve a whole series of register calls > instead of method definitions. I wouldn't expect the total amount of > code involved to change much. I'm not thinking about the __xxx__ methods, they're an aberration. I'm thinking about all the user-defined methods and attributes that get caught in one go by the __getattr__ method of the proxy. > That said, a recursive flatten() implementation is indeed a problem that > generic functions are well suited to solving Yes, I agree with that. It was just something I thought of that shows that generic functions and OO are not quite equivalent in general. -- Greg From ncoghlan at gmail.com Fri May 30 01:23:25 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 May 2008 09:23:25 +1000 Subject: [Python-Dev] A thought on generic functions In-Reply-To: <483F34EF.5080301@canterbury.ac.nz> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> <483DF5FF.7010201@canterbury.ac.nz> <483E7F78.8020109@gmail.com> <483F34EF.5080301@canterbury.ac.nz> Message-ID: <483F3AED.4020302@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: >> I don't think it would actually be that much worse - something like >> typetools.ProxyMixin would just involve a whole series of register >> calls instead of method definitions. I wouldn't expect the total >> amount of code involved to change much. > > I'm not thinking about the __xxx__ methods, they're an > aberration. I'm thinking about all the user-defined > methods and attributes that get caught in one go by > the __getattr__ method of the proxy. Ah, I see what you mean. That's where the generic system itself needs to be based on generic functions - then you can hook the lookup function so that proxies get looked up based on their target type rather than the fact they're a proxy. It all gets very brain bending and self referential, which is when folks tend to throw generics in the 'too complicated' basket ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Fri May 30 01:34:01 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 30 May 2008 11:34:01 +1200 Subject: [Python-Dev] A thought on generic functions In-Reply-To: <483F3AED.4020302@gmail.com> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> <483DF5FF.7010201@canterbury.ac.nz> <483E7F78.8020109@gmail.com> <483F34EF.5080301@canterbury.ac.nz> <483F3AED.4020302@gmail.com> Message-ID: <483F3D69.5040706@canterbury.ac.nz> Nick Coghlan wrote: > That's where the generic system itself needs to > be based on generic functions - then you can hook the lookup function so > that proxies get looked up based on their target type rather than the > fact they're a proxy. It all gets very brain bending and self > referential, which is when folks tend to throw generics in the 'too > complicated' basket ;) Yep. :-) Also, when hooking into things at such a deep level, I'd be a bit worried that I was catching too *much*, or that I was imposing performance penalties on unrelated things. At least with __getattr__ you know what you're doing can *only* affect the proxy object and nothing else. -- Greg From humberto at digi.com.br Fri May 30 02:25:21 2008 From: humberto at digi.com.br (Humberto Diogenes) Date: Thu, 29 May 2008 21:25:21 -0300 Subject: [Python-Dev] [Python-3000] Finishing up PEP 3108 In-Reply-To: References: Message-ID: <21258346-305C-439D-A7F0-EE945FE77B37@digi.com.br> On 29/05/2008, at 14:32, Brett Cannon wrote: > On Thu, May 29, 2008 at 12:12 AM, Georg Brandl > wrote: >> >>> Issue 2848 - mimetools has been deprecated for a while, but it is >>> still used in a bunch of places. Since this has been deprecated in >>> PEP >>> 4 for a long time, should we add the removal warning in 2.6 now and >>> then make its actual removal of usage something to do by another >>> beta? >>> >>> Issue 2849 - rfc822 is the same problem as mimetools. >> >> The problem is that nobody seems to know what exactly distinguishes >> mimetools/rfc822' classes and its successor's (email's) classes, so >> it's hard to replace it in the stdlib. >> > > Right. I have looked myself over the years and it never seemed > brain-dead simple. Well, as documented in issue 2849, rfc822 is almost gone. I've already removed it from mailbox and test_urllib2 modules. It seems that there remains only one important use of it, which is in cgi.FieldStorage.read_multi(). I couldn't figure out how to replace it there, though, as read_multi's current implementation relies on the fact that rfc822.Message(fp) advances the file pointer just by the amount it needs, while email.parser.Parser.parse() reads the whole file. I believe that read_multi can be rewritten in a way that's compatible with email.parser, but I don't know how to do that... :\ -- Humberto Di?genes http://humberto.digi.com.br From luke.leighton at googlemail.com Tue May 13 20:06:35 2008 From: luke.leighton at googlemail.com (Luke Kenneth Casson Leighton) Date: Tue, 13 May 2008 18:06:35 -0000 Subject: [Python-Dev] http://bugs.python.org/issue708007 - telnetpopen Message-ID: my apologies, i found the issue "closed" after doing a search for my own work, requiring (again) the telnetlib code for yet another project. since 2001, the work done has been reimplemented time and time again, by different individuals, in different projects, in different ways, all solving exactly the same issues, providing exactly the same type of functionality, as that provided by TelnetPopen3. e.g. Pexpect is the one i found today, listed off of http://en.wikipedia.org/wiki/Expect l. From ctb at msu.edu Wed May 14 03:45:49 2008 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 14 May 2008 01:45:49 -0000 Subject: [Python-Dev] Addition of "pyprocessing" module to standard lib. In-Reply-To: References: <4222a8490805131752i4191bea6h7a19113c40a591b@mail.gmail.com> Message-ID: <20080514014536.GA6017@caltech.edu> On Tue, May 13, 2008 at 09:23:02PM -0400, Tom Pinckney wrote: -> Why not use MPI? It's cross platform, cross language and very widely -> supported already. And there're Python bindings already. MPI requires far more setup than the pyprocessing module does; it's by no means plug & play. --titus -- C. Titus Brown, ctb at msu.edu From Trent.Nelson at onresolve.com Fri May 16 03:06:01 2008 From: Trent.Nelson at onresolve.com (Trent Nelson) Date: Fri, 16 May 2008 01:06:01 -0000 Subject: [Python-Dev] BSDDB3 In-Reply-To: <482A291F.3070107@jcea.es> References: <47C32D2D.1030206@free.fr> <47C4813F.2080403@v.loewis.de> <52dc1c820802262146n7b7c211dk973eb44390a4b8f7@mail.gmail.com> <20080227141322.GA5871@amk-desktop.matrixgroup.net> <47CDDF57.1020006@argo.es> <47CF0B7C.109@v.loewis.de> <480CABFB.2080302@argo.es> <87D3F9C72FBF214DB39FA4E3FE618CDC6E243C7CA1@EXMBX04.exchhosting.com> <480EA87D.5040107@argo.es> <87D3F9C72FBF214DB39FA4E3FE618CDC6E2456D05A@EXMBX04.exchhosting.com> <482A291F.3070107@jcea.es> Message-ID: <6167796BFEB5D0438720AC212E89A6B0078726F0@exchange.onresolve.com> Thanks Jesus. I'll import BSDDB 4.6.4 into svn.python.org/projects/external today. Once that's done, I'll create a new bsddb46.vcproj and update the pcbuild.sln to use this project instead of the bsddb44 one currently being used. (Hopefully I'll be able to get that done today as well.) I assume you're just working on trunk at the moment? i.e. should I block or merge the changes to py3k? For everyone else: just a heads up that there will probably be a little bit of buildbot instability whilst we transition BSDDB versions on Windows. Trent. -----Original Message----- From: Jesus Cea [mailto:jcea at jcea.es] Sent: Wednesday, May 14, 2008 12:50 AM To: Trent Nelson Cc: python-dev at python.org Subject: Re: [Python-Dev] BSDDB3 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Trent Nelson wrote: |> Next week I will (if nothing goes wrong) publish pybsddb |> 4.6.4. This release supports distributed transactions and |> replication, testsuite is way faster, and rewritten to be |> able to launch tests from multiple threads/processes if you |> wish, setuptools/pypi support, etc. | | Great! Once you've settled in with your changes let me know and I'll help with doing the necessary things on the Windows-side to set up the bsddb46.vcproj and switching the build to use that. Python SVN updated. Let me know if you need anything from me. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSCopHZlgi5GaxT1NAQKrCQQAj/HTk5oqSbF2PYkZpCw2T7Dd6yEgddlY L+qW1Cde/b3duClEfawy7kf5DkSjlGLVZ9XSS+njAMKszzYK6ZIg9N4GEu5A9TO2 Rg2PiytaPbs88Jo5IIlDjvaVFPPqsOasn7WH1wcawtUKNei8whMReOveZgYXfFFf QphSJ7zspNU= =L6jr -----END PGP SIGNATURE----- From Trent.Nelson at onresolve.com Fri May 16 03:06:02 2008 From: Trent.Nelson at onresolve.com (Trent Nelson) Date: Fri, 16 May 2008 01:06:02 -0000 Subject: [Python-Dev] PEP 370 extras In-Reply-To: <57897D5F-D6F6-4A3D-80D3-FA0A367657EB@python.org> References: <4823791F.2050507@cheimes.de> <57897D5F-D6F6-4A3D-80D3-FA0A367657EB@python.org> Message-ID: <6167796BFEB5D0438720AC212E89A6B0078726EE@exchange.onresolve.com> > Christian, you get this week's Awesome Award. Hah, 'Weekly Awesome Award', we should make that one of our things, like QOTW. Trent. From Trent.Nelson at onresolve.com Fri May 16 03:06:03 2008 From: Trent.Nelson at onresolve.com (Trent Nelson) Date: Fri, 16 May 2008 01:06:03 -0000 Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? In-Reply-To: <4829E787.5080605@jcea.es> References: <4829E787.5080605@jcea.es> Message-ID: <6167796BFEB5D0438720AC212E89A6B0078726EF@exchange.onresolve.com> Hi Jesus, Regarding where to place test files, you might want to look at how the current bsddb test suite in Lib/bsddb/test handles the need to create temporary files and such. A bit of work has gone into this particular aspect to improve reliability and robustness on buildbots -- particularily when it comes to Windows. (I believe the current test suite comes up with a unique directory based on process ID..) Trent. -----Original Message----- From: python-dev-bounces+tnelson=onresolve.com at python.org [mailto:python-dev-bounces+tnelson=onresolve.com at python.org] On Behalf Of Jesus Cea Sent: Tuesday, May 13, 2008 8:10 PM To: Python DEV Subject: [Python-Dev] Committing bsddb 4.6.4, and where can I put testsuite temp files? -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just committed pybsddb 4.6.4 to python svn. My next step (after a successfull buildbot cycle, I hope!) is to commit the new testsuite. First I need to review any changes there since I maintain pybsddb. The testsuite creates a lot of files/directories while working. Fine, since it cleans later, unless some test crashes hard. My testcode defines a variable to the path I want to use to keep those files/dirs. Currently it is "/tmp/z-BerkeleyDB", or something like that. Since my code allows for testcases to be run in multiple threads and/or processes, where would be a sensible path choice for temporal files?. Current working directory would be fine, if "somebody" is cleaning after running all tests. Ramdisk ("/tmp" in some systems, for example Solaris) would be nice, since some test uses transactions, and transactions are write-synchronous. In my system, runnning all tests serially (you can run all of them in parallel, if you wish) takes about 15-20 seconds. Far faster than current tests in python svn, and that can be improved even more. - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBSCnngZlgi5GaxT1NAQLhPwP9GMMyNMgz6mfmfHXVOV8bDJGOQRok95uL 14A+K9zXW3/wlp1rhvoPmCHYqvRoCLVkPZ/7pLEQlo1ZksOlHy6BH3MDeDJEjVax 69XlzUUeuqplGbTiMdX8qd0dPi2Jp+Akg7U6ZmBdADhF7D21umU474OalKO2eIoL ba/wnlMvens= =1lai -----END PGP SIGNATURE----- _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/tnelson%40onresolve.com From Trent.Nelson at onresolve.com Fri May 16 03:12:37 2008 From: Trent.Nelson at onresolve.com (Trent Nelson) Date: Fri, 16 May 2008 01:12:37 -0000 Subject: [Python-Dev] Importing bsddb 4.6.21; with or without AES encryption? Message-ID: <6167796BFEB5D0438720AC212E89A6B0078726F1@exchange.onresolve.com> I was about to commit an initial import of the bsddb 4.6.21 source to the 'external' area in the repo, which I obtained from the following URL: http://www.oracle.com/technology/software/products/berkeley-db/index.html I downloaded the source that includes AES encryption, for no reason other than it was first on the list. I'm now wondering if we should only be importing the 'NC' source that doesn't contain any encryption? Jesus, does pybsddb use any of the Berkeley DB encryption facilities? Would anything break if we built the bsddb module without encryption? (Note that all of this only applies to Windows.) Trent. From status at bugs.python.org Fri May 16 18:08:30 2008 From: status at bugs.python.org (Tracker) Date: Fri, 16 May 2008 18:08:30 +0200 (CEST) Subject: [Python-Dev] Summary of Tracker Issues Message-ID: <20080516160830.A169E78174@psf.upfronthosting.co.za> ACTIVITY SUMMARY (05/09/08 - 05/16/08) Tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1874 open (+61) / 12843 closed (+30) / 14717 total (+91) Open issues with patches: 558 Average duration of open issues: 704 days. Median duration of open issues: 1397 days. Open Issues Breakdown open 1852 (+61) pending 22 ( +0) Issues Created Or Reopened (92) _______________________________ Exception state lives too long in 3.0 05/13/08 http://bugs.python.org/issue2507 reopened collinwinter patch Integer right shift raises OverflowError when second operand is 05/09/08 CLOSED http://bugs.python.org/issue2804 created marketdickinson patch 2to3 doesn't correct divisions 05/09/08 http://bugs.python.org/issue2805 created gpolo Py30a5: Tk Menu Alt-f behaves differently Linux vs Windows 05/09/08 http://bugs.python.org/issue2806 created mark Remove the API PyUnicode_AsString() 05/09/08 CLOSED http://bugs.python.org/issue2807 created alexandre.vassalotti asynchat forgets packets when push is called from a thread 05/10/08 http://bugs.python.org/issue2808 created xix xeaon string docstring doesn't mention that ' '.split() != ' '.split(' 05/10/08 CLOSED http://bugs.python.org/issue2809 created Neil Muller patch _winreg.EnumValue fails when the registry data includes multibyt 05/10/08 http://bugs.python.org/issue2810 created stutzbach doctest doesn't treat unicode literals as specified by the file 05/10/08 http://bugs.python.org/issue2811 created neves Document property.getter/setter/deleter etc 05/10/08 CLOSED http://bugs.python.org/issue2812 created benjamin.peterson patch No float formatting in PyString_FromFormat 05/10/08 http://bugs.python.org/issue2813 created duaneb Remove old classes from mailbox module 05/11/08 CLOSED http://bugs.python.org/issue2814 created hdiogenes patch Python Manuals: horizontal scrolling broken 05/11/08 CLOSED http://bugs.python.org/issue2815 created tjreedy Quote-type recognition bug 05/11/08 CLOSED http://bugs.python.org/issue2816 created chester Make Python create a tuple with one element in a clean way 05/11/08 CLOSED http://bugs.python.org/issue2817 created chester pulldom cannot handle xml file with large external entity proper 05/11/08 http://bugs.python.org/issue2818 created hanselda Full precision summation 05/11/08 http://bugs.python.org/issue2819 created MrJean1 patch Remove mac modules 05/11/08 CLOSED http://bugs.python.org/issue2820 created benjamin.peterson unittest.py sys.exit error 05/11/08 http://bugs.python.org/issue2821 created acgetchell Quote-type recognition bug [badly fixed last time] 05/11/08 CLOSED http://bugs.python.org/issue2822 created chester "Report bug" links 05/11/08 CLOSED http://bugs.python.org/issue2823 created techtonik zipfile to handle duplicate files in archive 05/11/08 http://bugs.python.org/issue2824 created techtonik Quote-type recognition bug [got a patch!] 05/11/08 CLOSED http://bugs.python.org/issue2825 created chester patch safe_substitute() and substitute() 05/11/08 CLOSED http://bugs.python.org/issue2826 created chester IDLE 3.0a5 cannot handle UTF-8 05/11/08 http://bugs.python.org/issue2827 created sven.siegmund Clean up undoc.rst 05/12/08 http://bugs.python.org/issue2828 created brett.cannon easy Copy cgi.parse_qs() to urllib.parse 05/12/08 http://bugs.python.org/issue2829 created brett.cannon Copy cgi.escape() to html 05/12/08 http://bugs.python.org/issue2830 created brett.cannon Adding start to enumerate() 05/12/08 CLOSED http://bugs.python.org/issue2831 created scottdial patch Line numbers reported by extract_stack are offset by the #-*- en 05/12/08 http://bugs.python.org/issue2832 created ddvoinikov __exit__ silences the active exception 05/12/08 http://bugs.python.org/issue2833 created ddvoinikov re.IGNORECASE not Unicode-ready 05/12/08 http://bugs.python.org/issue2834 created sven.siegmund Py30a5: webbrowser.open() inf recursion 05/12/08 CLOSED http://bugs.python.org/issue2835 created mark str.format() documentation needs to be backported to 2.6 05/12/08 CLOSED http://bugs.python.org/issue2836 created eric.smith 26backport OpenID wannabe 05/12/08 CLOSED http://bugs.python.org/issue2837 created techtonik Verify callback for SSL 05/12/08 CLOSED http://bugs.python.org/issue2838 created ruben Moving lib-tk to tkinter package 05/12/08 http://bugs.python.org/issue2839 created gpolo patch Expat parser locks XML source file if ContentHandler raises an e 05/12/08 http://bugs.python.org/issue2840 created craigneuro Windows: "Runtime Error!" crash from pythonw.exe (3.0a5) 05/12/08 http://bugs.python.org/issue2841 created tjreedy Dictionary methods: inconsistency 05/12/08 CLOSED http://bugs.python.org/issue2842 created chester New methods for existing Tkinter widgets 05/13/08 http://bugs.python.org/issue2843 created gpolo int() lies about base parameter 05/13/08 CLOSED http://bugs.python.org/issue2844 created jwilk patch shutil.copy2() copies permission bits 05/13/08 CLOSED http://bugs.python.org/issue2845 created igs Gzip cannot handle zero-padded output + patch 05/13/08 http://bugs.python.org/issue2846 created tadek patch Remove cl usage from aifc 05/14/08 http://bugs.python.org/issue2847 created brett.cannon Remove mimetools usage from the stdlib 05/14/08 http://bugs.python.org/issue2848 created brett.cannon Remove usage of rfc822 from the stdlib 05/14/08 http://bugs.python.org/issue2849 created brett.cannon Augmenting the Windows build to support code signing. 05/14/08 http://bugs.python.org/issue2850 created Trent.Nelson Eliminate Perl legacy in re flag names 05/14/08 CLOSED http://bugs.python.org/issue2851 created mark sidebar directive fails 05/14/08 http://bugs.python.org/issue2852 created coderanger *** glibc detected *** python: double free or corruption 05/14/08 http://bugs.python.org/issue2853 created nuddelaug Add gestalt back into Python 3.0 05/14/08 http://bugs.python.org/issue2854 created brett.cannon patch lookkey should INCREF/DECREF startkey around PyObject_RichCompar 05/14/08 http://bugs.python.org/issue2855 created Rhamphoryncus patch os.listdir doc should mention that Unicode decoding can fail 05/15/08 http://bugs.python.org/issue2856 created brodierao add codec for java modified utf-8 05/15/08 http://bugs.python.org/issue2857 created phr bsddb.db.DBEnv.lock_get test_lock.test03_set_timeout crashes 05/15/08 http://bugs.python.org/issue2858 created gregory.p.smith sphinx and virtualenv 05/15/08 http://bugs.python.org/issue2859 created peterK re module fails to handle digits in byte strings 05/15/08 http://bugs.python.org/issue2860 created scoder Patch to rename markupbase to _markupbase 05/15/08 http://bugs.python.org/issue2861 created quentin.gallet-gilles patch cleanup of freelist management 05/15/08 http://bugs.python.org/issue2862 created aimacintyre patch, patch Generator __name__ and better repr() 05/15/08 CLOSED http://bugs.python.org/issue2863 created georg.brandl patch, patch etree: Add XPath documentation 05/15/08 http://bugs.python.org/issue2864 created guettli syntax error in fix_imports.py 05/15/08 CLOSED http://bugs.python.org/issue2865 created Tinctorius patch syntax error in fix_imports.py 05/15/08 CLOSED http://bugs.python.org/issue2866 created Tinctorius patch curses-howto link in curses module documentation gives a 404 05/15/08 CLOSED http://bugs.python.org/issue2867 created Devin Jeanpierre Problem with urllib and urllib2 in urlopen? 05/15/08 http://bugs.python.org/issue2868 created ambarish Wrong doc for `calendar.Calendar.iterweekdays` 05/15/08 CLOSED http://bugs.python.org/issue2869 created yogi patch cmathmodule.c compile error 05/15/08 CLOSED http://bugs.python.org/issue2870 created Juno store thread.get_ident() thread identifier inside threading.Thre 05/15/08 http://bugs.python.org/issue2871 created irmen Remove commands for PEP 3108 05/16/08 http://bugs.python.org/issue2872 created brett.cannon Remove htmllib use in the stdlib 05/16/08 http://bugs.python.org/issue2873 created brett.cannon Remove use of the stat module in the stdlib 05/16/08 http://bugs.python.org/issue2874 created brett.cannon Rename the thread module to _thread 05/16/08 http://bugs.python.org/issue2875 created brett.cannon Backport UserDict move in 3.0 05/16/08 http://bugs.python.org/issue2876 created brett.cannon Backport UserString move from 3.0 05/16/08 http://bugs.python.org/issue2877 created brett.cannon Backport UserList move in 3.0 05/16/08 http://bugs.python.org/issue2878 created brett.cannon Rename _winreg to winreg 05/16/08 http://bugs.python.org/issue2879 created brett.cannon Rename repr to reprlib 05/16/08 CLOSED http://bugs.python.org/issue2880 created brett.cannon Create the dbm package 05/16/08 http://bugs.python.org/issue2881 created brett.cannon Create the html package 05/16/08 http://bugs.python.org/issue2882 created brett.cannon Create the http package 05/16/08 http://bugs.python.org/issue2883 created brett.cannon patch Create the tkinter package 05/16/08 http://bugs.python.org/issue2884 created brett.cannon patch Create the urllib package 05/16/08 http://bugs.python.org/issue2885 created brett.cannon Create the xmlrpc package 05/16/08 http://bugs.python.org/issue2886 created brett.cannon bsddb 4.6.4 needs to be ported to Python 3.0 05/16/08 http://bugs.python.org/issue2887 created alexandre.vassalotti pprint produces different output in 2.6 and 3.0 05/16/08 http://bugs.python.org/issue2888 created alexandre.vassalotti curses for windows (alternative patch) 05/16/08 http://bugs.python.org/issue2889 created techtonik O_ASYNC and FASYNC should be defined if available 05/16/08 CLOSED http://bugs.python.org/issue2890 created cartman patch urllib not handling ftp servers that do not support REST 05/16/08 http://bugs.python.org/issue2891 created daggett improve cElementTree iterparse error handling 05/16/08 http://bugs.python.org/issue2892 created hniksic dict.has_key() expected in h2py.py. 05/16/08 http://bugs.python.org/issue2893 created dbinger 2to3 discards comments before import statements 05/16/08 http://bugs.python.org/issue2894 created georg.brandl Issues Now Closed (81) ______________________ Patch to rename *Server modules to lower-case 266 days http://bugs.python.org/issue1000 brett.cannon patch Patch to rename HTMLParser module to lower_case 266 days http://bugs.python.org/issue1002 brett.cannon patch Patches to rename Queue module to queue 264 days http://bugs.python.org/issue1005 georg.brandl patch tracebacks from list comps (probably other comps) don't show fu 262 days http://bugs.python.org/issue1025 georg.brandl "internal" zipimport.zipimporter feature untested 199 days http://bugs.python.org/issue1326 georg.brandl patch PEP 341 is not reflected in the documentation 125 days http://bugs.python.org/issue1760 georg.brandl o(n*n) marshal.dumps performance for largish objects with patch 121 days http://bugs.python.org/issue1792 akuchling patch, easy Make .pypirc handle multiple servers 116 days http://bugs.python.org/issue1858 tarek patch test_cl.py converted to unittest 94 days http://bugs.python.org/issue2043 georg.brandl patch, easy Interpreter crash on shutdown 90 days http://bugs.python.org/issue2077 georg.brandl Link error of gethostbyaddr and gethostname in Python Manuals (t 85 days http://bugs.python.org/issue2129 georg.brandl int operations no longer overflow 81 days http://bugs.python.org/issue2147 georg.brandl TestCase.tmpdir(), TestCase.mock() 82 days http://bugs.python.org/issue2156 jcea Fix hasattr's exception problems 75 days http://bugs.python.org/issue2196 benjamin.peterson patch urllib2. strange behavior for getting chuncked transfer-ecnoded 67 days http://bugs.python.org/issue2243 georg.brandl rlcompleter raises Exception on bad input 65 days http://bugs.python.org/issue2250 georg.brandl patch, easy os.strerror does not check for out of range argument 54 days http://bugs.python.org/issue2413 georg.brandl patch inaccuracy in httplib timeout documentation 50 days http://bugs.python.org/issue2452 georg.brandl ldexp(x,n) misbehaves when abs(n) is large 45 days http://bugs.python.org/issue2487 marketdickinson patch duplicate Misc.lower 39 days http://bugs.python.org/issue2535 georg.brandl patch, easy textwrap handling of hyphenation 22 days http://bugs.python.org/issue2659 georg.brandl patch The position of special value tables (cmathmodule.c) 26 days http://bugs.python.org/issue2664 marketdickinson patch tk.rst possibly wrong ? 13 days http://bugs.python.org/issue2709 georg.brandl patch unittest.makeSuite undocumented and "obsolete" - but what to use 12 days http://bugs.python.org/issue2721 georg.brandl documentation -- do serversockets work with ipv6 9 days http://bugs.python.org/issue2741 georg.brandl example code does not work 9 days http://bugs.python.org/issue2742 georg.brandl patch A socket example code shown in doc doesn't work on FreeBSD 6 days http://bugs.python.org/issue2763 georg.brandl Doubtfull code in 'doctest.DocTestSuite' 7 days http://bugs.python.org/issue2766 georg.brandl "doctest.DocTestCase.debug" clears "test.globs" too early 7 days http://bugs.python.org/issue2767 georg.brandl Subversion 1.5 Working Copies Break Setuptools 7 days http://bugs.python.org/issue2770 georg.brandl set_swap_bodies is unsafe 8 days http://bugs.python.org/issue2778 Rhamphoryncus patch Patch to flush unittest output 4 days http://bugs.python.org/issue2787 georg.brandl patch Comparsion datetime objects with None 2 days http://bugs.python.org/issue2789 tjreedy patch Figure directive not handled for latex writing in Sphinx 8 days http://bugs.python.org/issue2794 georg.brandl patch Crash on non-Windows if Python runs from a non-ASCII directory 3 days http://bugs.python.org/issue2798 amaury.forgeotdarc patch make htmlhelp creates python30a5.chm but msi.py expects pydoc.ch 2 days http://bugs.python.org/issue2800 loewis Py30a5 float.is_integer() raises ValueError 0 days http://bugs.python.org/issue2801 marketdickinson str.format() :n integer output 2 days http://bugs.python.org/issue2802 eric.smith heapq.heappush called with too few arguments in sched.py 2 days http://bugs.python.org/issue2803 georg.brandl patch Integer right shift raises OverflowError when second operand is 1 days http://bugs.python.org/issue2804 marketdickinson patch Remove the API PyUnicode_AsString() 0 days http://bugs.python.org/issue2807 alexandre.vassalotti string docstring doesn't mention that ' '.split() != ' '.split(' 2 days http://bugs.python.org/issue2809 davidfraser patch Document property.getter/setter/deleter etc 1 days http://bugs.python.org/issue2812 georg.brandl patch Remove old classes from mailbox module 1 days http://bugs.python.org/issue2814 georg.brandl patch Python Manuals: horizontal scrolling broken 1 days http://bugs.python.org/issue2815 georg.brandl Quote-type recognition bug 0 days http://bugs.python.org/issue2816 chester Make Python create a tuple with one element in a clean way 0 days http://bugs.python.org/issue2817 chester Remove mac modules 1 days http://bugs.python.org/issue2820 brett.cannon Quote-type recognition bug [badly fixed last time] 0 days http://bugs.python.org/issue2822 gvanrossum "Report bug" links 0 days http://bugs.python.org/issue2823 georg.brandl Quote-type recognition bug [got a patch!] 0 days http://bugs.python.org/issue2825 gvanrossum patch safe_substitute() and substitute() 0 days http://bugs.python.org/issue2826 georg.brandl Adding start to enumerate() 2 days http://bugs.python.org/issue2831 georg.brandl patch Py30a5: webbrowser.open() inf recursion 0 days http://bugs.python.org/issue2835 amaury.forgeotdarc str.format() documentation needs to be backported to 2.6 0 days http://bugs.python.org/issue2836 georg.brandl 26backport OpenID wannabe 3 days http://bugs.python.org/issue2837 loewis Verify callback for SSL 2 days http://bugs.python.org/issue2838 janssen Dictionary methods: inconsistency 0 days http://bugs.python.org/issue2842 gvanrossum int() lies about base parameter 0 days http://bugs.python.org/issue2844 loewis patch shutil.copy2() copies permission bits 3 days http://bugs.python.org/issue2845 georg.brandl Eliminate Perl legacy in re flag names 0 days http://bugs.python.org/issue2851 gvanrossum Generator __name__ and better repr() 1 days http://bugs.python.org/issue2863 georg.brandl patch, patch syntax error in fix_imports.py 0 days http://bugs.python.org/issue2865 benjamin.peterson patch syntax error in fix_imports.py 0 days http://bugs.python.org/issue2866 collinwinter patch curses-howto link in curses module documentation gives a 404 1 days http://bugs.python.org/issue2867 georg.brandl Wrong doc for `calendar.Calendar.iterweekdays` 1 days http://bugs.python.org/issue2869 georg.brandl patch cmathmodule.c compile error 1 days http://bugs.python.org/issue2870 christian.heimes Rename repr to reprlib 0 days http://bugs.python.org/issue2880 alexandre.vassalotti O_ASYNC and FASYNC should be defined if available 0 days http://bugs.python.org/issue2890 georg.brandl patch Text widget, bindtags and Tabs 2541 days http://bugs.python.org/issue429031 gpolo TelnetPopen3, TelnetBase, Expect split 1879 days http://bugs.python.org/issue708007 lkcl patch mailbox should use email not rfc822 1789 days http://bugs.python.org/issue756982 georg.brandl patch, easy String interpolation needs PEP 237 update 1168 days http://bugs.python.org/issue1153769 georg.brandl patch using some_re.sub() often imports sre.__doc__ 1038 days http://bugs.python.org/issue1234985 georg.brandl pdb: implement "until",fix for 1248119 991 days http://bugs.python.org/issue1267629 benjamin.peterson patch 'imp' documentation does not mention that lock is re-entrant 496 days http://bugs.python.org/issue1625509 akuchling patch potential clash with C++ in ceval.h 467 days http://bugs.python.org/issue1649238 georg.brandl easy pprint handles depth argument incorrectly 374 days http://bugs.python.org/issue1712742 georg.brandl fix for 1712742: corrects pprint's handling of 'depth' 374 days http://bugs.python.org/issue1713041 georg.brandl patch, easy PLATFORM macro in PC/pyconfig.h already defined by Apache 310 days http://bugs.python.org/issue1749567 georg.brandl Pickle fails on BeautifulSoup's navigableString instances 299 days http://bugs.python.org/issue1757062 georg.brandl Top Issues Most Discussed (10) ______________________________ 30 Implement PEP 3108 10 days open http://bugs.python.org/issue2775 14 Full precision summation 5 days open http://bugs.python.org/issue2819 11 O_ASYNC and FASYNC should be defined if available 0 days closed http://bugs.python.org/issue2890 10 Adding start to enumerate() 2 days closed http://bugs.python.org/issue2831 10 Integer right shift raises OverflowError when second operand is 1 days closed http://bugs.python.org/issue2804 9 IDLE ignores module change before restart 12 days open http://bugs.python.org/issue2755 8 mailbox should use email not rfc822 1789 days closed http://bugs.python.org/issue756982 7 curselection() in Tkinter.py should return ints 1595 days open http://bugs.python.org/issue869780 7 int() lies about base parameter 0 days closed http://bugs.python.org/issue2844 6 add codec for java modified utf-8 2 days open http://bugs.python.org/issue2857 From hodgestar at gmail.com Tue May 27 01:19:39 2008 From: hodgestar at gmail.com (Simon Cross) Date: Tue, 27 May 2008 01:19:39 +0200 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: References: Message-ID: On Tue, May 27, 2008 at 12:43 AM, Christian Heimes wrote: > Ten minutes ago I raised a concern about speed differences between the > old style % formatting and the new .format() code. Some quick > benchmarking from Benjamin and me showed, that it's even worse than I > expected. My own tests show a less dire difference: $ ./python -m timeit "'some text with %s'.format('nothing')" 1000000 loops, best of 3: 0.578 usec per loop $ ./python -m timeit "'some text with %s' % 'nothing'" 1000000 loops, best of 3: 0.472 usec per loop > $ ./python -m timeit "'%s'.format('nothing')" > 100000 loops, best of 3: 2.63 usec per loop > $ ./python -m timeit "'%s' % 'nothing'" > 10000000 loops, best of 3: 0.188 usec per loop It struct me as odd that this one case shows such a big difference while the others show less of one. My tests show that the old-style % formatting is much faster when the final string is 20 characters or less: $ ./python -m timeit "'....|....|....|...%s' % '12'" 10000000 loops, best of 3: 0.0764 usec per loop $ ./python -m timeit "'....|....|....|...%s' % '123'" 1000000 loops, best of 3: 0.481 usec per loop A read through stringobject.c didn't give me any clue as to why 20 is the magic number. %d shows an even bigger jump: $ ./python -m timeit "'....|....|....|...%d' % 12" 10000000 loops, best of 3: 0.0764 usec per loop $ ./python -m timeit "'....|....|....|...%d' % 123" 1000000 loops, best of 3: 1.28 usec per loop Schiavo Simon From bryan.hartwell at ieso.ca Tue May 27 15:00:11 2008 From: bryan.hartwell at ieso.ca (Hartwell Bryan) Date: Tue, 27 May 2008 08:00:11 -0500 Subject: [Python-Dev] Obtaining short file path Message-ID: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int> Hi, Purpose: obtaining the system ("short") path from a full path Background: File dialogs (visual studio) return a full path (e.g. f="C:\this path has spaces\thisfilenameislongerthan8char.txt"). If this value is provided to Python, it will not recongize this as a file. In fact os.path.isfile(f) doesn't return false, it crashes. Likewise, when calling executables (from Python) with files as arguments a short path is required. VB FileSystemObject has the ShortPath method, while os.path and path (www.jorendorff.com) modules do not (at least as far as my googling could determine). Why bother creating a COM interface when you're just going to pass as shell run-time arguments all the values the server is better at computing? System: Python 2.3; Windows XP Sample Code: import win32com.client import time import os,sys import os.path #------------------------------------------------------------- def shortpath(x): z='' for y in x.split('\\'): if len(y.split('.')[0])>8: if ('.' in y): z=z+'\\'+y.split('.')[0][:6].upper()+'~1'+'.'+y.split('.')[1] else: z=z+'\\'+y[:6].upper()+'~1' else: z=z+'\\'+y return z[1:] #------------------------------------------------------------- xlApp = win32com.client.Dispatch("Excel.Application") xlBook = xlApp.ActiveWorkbook savFile = str(sys.argv[1]) rawFile = str(xlBook.Sheets("Timestamp").TextBox2) #print os.path.isfile(savFile) r=shortpath(rawFile) print r try: print os.path.isfile(r) except: print 'something rude' time.sleep(7) Notes: This code does not account for peer paths or files that share the first 8 characters (and file extension). I'm also aware that this is not the normal means for submitting a "patch", but in my job function I don't see myself regularly participating in python development (and I'm probably not savvy enough) so the effort wasn't worth it. However I still thought others might benefit from what seems to be (to me) a fundamental path function. Do with it, or ignore it, as you please. Cheers, Bryan Hartwell This message is intended only for the use of the intended recipients, and it may be privileged and confidential. If you are not the intended recipient, you are hereby notified that any review, retransmission, conversion to hard copy, copying, circulation or other use of this message is strictly prohibited. If you are not the intended recipient, please notify me immediately by return e-mail, and delete this message from your system. -------------- next part -------------- An HTML attachment was scrubbed... URL: From qrczak at knm.org.pl Thu May 29 12:05:20 2008 From: qrczak at knm.org.pl (=?UTF-8?Q?Marcin_=E2=80=98Qrczak=E2=80=99_Kowalczyk?=) Date: Thu, 29 May 2008 12:05:20 +0200 Subject: [Python-Dev] [Python-3000] PyString -> PyBytes C API renaming In-Reply-To: <483E7D01.4010603@gmail.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D49ED.8060907@cheimes.de> <483D5444.8000705@egenix.com> <483E7D01.4010603@gmail.com> Message-ID: <3f4107910805290305s63b97e73i87824755f7aa31fb@mail.gmail.com> 2008/5/29 Nick Coghlan : > it would still be nice if the PyBytes_* symbols > were visible to the linker as well as the preprocessor. If this is not a strict requirement but a useful extra, then it might be done in an unportable way. GCC has an 'alias' attribute: http://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html -- Marcin Kowalczyk qrczak at knm.org.pl http://qrnik.knm.org.pl/~qrczak/ From lars at ibp.de Thu May 29 16:56:20 2008 From: lars at ibp.de (Lars Immisch) Date: Thu, 29 May 2008 16:56:20 +0200 Subject: [Python-Dev] [Python-3000] Finishing up PEP 3108 In-Reply-To: <8b943f2b0805290625w19ab1fd3l48f00f40e630c39d@mail.gmail.com> References: <8b943f2b0805290625w19ab1fd3l48f00f40e630c39d@mail.gmail.com> Message-ID: <483EC414.7080603@ibp.de> > Issue 2847 - the aifc module still imports the cl module in 3.0. > Problem is that the cl module is gone. =) So it seems silly to have > the imports lying about. This can probably be changed to critical. > > > It shouldn't be a problem to rip everything cl-related out of aifc. > The question is how useful aifc will be after that ... > > > Has someone already used that module ? I took a look into it, but I'm a > bit confused about the various compression types, case-sensitivity and > compatibility issues [1]. Are Apple's "alaw" and SGI's "ALAW" really the > same encoding ? Can we use the audioop module for ALAW, just like it's > already done for ULAW ? There is just one alaw I've ever come across (G.711), and the audioop implementation could be used (audioop's alaw support is younger than the aifc module, BTW) The capitalisation is confusing, but your document [1] says: "Apple Computer's QuickTime player recognize only the Apple compression types. Although "ALAW" and "ULAW" contain identical sound samples to the "alaw" and "ulaw" formats and were in use long before Apple introduced the new codes, QuickTime does not recognize them." So this seems just a matter of naming in the AIFC, but not a matter of two different alaw implementations. - Lars [1] http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/AIFF/AIFF.html From aahz at pythoncraft.com Fri May 30 03:14:48 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 May 2008 18:14:48 -0700 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting Message-ID: <20080530011448.GA2964@panix.com> I just processed two weeks of backlog on python-dev (I'm only a backup moderator). Would anyone object to changing the list policy so that non-member posts get discarded instead of held? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From musiccomposition at gmail.com Fri May 30 03:17:37 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 29 May 2008 20:17:37 -0500 Subject: [Python-Dev] http://bugs.python.org/issue708007 - telnetpopen In-Reply-To: References: Message-ID: <1afaf6160805291817v6c5c4e31s3bb80838fe5d9d8f@mail.gmail.com> On Tue, May 13, 2008 at 1:06 PM, Luke Kenneth Casson Leighton wrote: > my apologies, i found the issue "closed" after doing a search for my > own work, requiring (again) the telnetlib code for yet another > project. I have reopened it for you. > > since 2001, the work done has been reimplemented time and time again, > by different individuals, in different projects, in different ways, > all solving exactly the same issues, providing exactly the same type > of functionality, as that provided by TelnetPopen3. e.g. Pexpect is > the one i found today, listed off of > http://en.wikipedia.org/wiki/Expect > > l. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/musiccomposition%40gmail.com > -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From musiccomposition at gmail.com Fri May 30 03:25:09 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 29 May 2008 20:25:09 -0500 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting In-Reply-To: <20080530011448.GA2964@panix.com> References: <20080530011448.GA2964@panix.com> Message-ID: <1afaf6160805291825p3cf01316n46ea7e8a6f3b005@mail.gmail.com> On Thu, May 29, 2008 at 8:14 PM, Aahz wrote: > I just processed two weeks of backlog on python-dev (I'm only a backup > moderator). Would anyone object to changing the list policy so that > non-member posts get discarded instead of held? Is any of the mail spam? -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From aahz at pythoncraft.com Fri May 30 03:28:39 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 May 2008 18:28:39 -0700 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting In-Reply-To: <1afaf6160805291825p3cf01316n46ea7e8a6f3b005@mail.gmail.com> References: <20080530011448.GA2964@panix.com> <1afaf6160805291825p3cf01316n46ea7e8a6f3b005@mail.gmail.com> Message-ID: <20080530012839.GA21352@panix.com> On Thu, May 29, 2008, Benjamin Peterson wrote: > On Thu, May 29, 2008 at 8:14 PM, Aahz wrote: >> >> I just processed two weeks of backlog on python-dev (I'm only a backup >> moderator). Would anyone object to changing the list policy so that >> non-member posts get discarded instead of held? > > Is any of the mail spam? Very yes -- more than half. (What gets through SpamBayes) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From musiccomposition at gmail.com Fri May 30 03:31:10 2008 From: musiccomposition at gmail.com (Benjamin Peterson) Date: Thu, 29 May 2008 20:31:10 -0500 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting In-Reply-To: <20080530012839.GA21352@panix.com> References: <20080530011448.GA2964@panix.com> <1afaf6160805291825p3cf01316n46ea7e8a6f3b005@mail.gmail.com> <20080530012839.GA21352@panix.com> Message-ID: <1afaf6160805291831i1110f3dfy3f57a93e0a9f4a09@mail.gmail.com> On Thu, May 29, 2008 at 8:28 PM, Aahz wrote: > On Thu, May 29, 2008, Benjamin Peterson wrote: >> >> Is any of the mail spam? > > Very yes -- more than half. (What gets through SpamBayes) In that case, I'm -1. -- Cheers, Benjamin Peterson "There's no place like 127.0.0.1." From brett at python.org Fri May 30 03:32:17 2008 From: brett at python.org (Brett Cannon) Date: Thu, 29 May 2008 18:32:17 -0700 Subject: [Python-Dev] Obtaining short file path In-Reply-To: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int> References: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int> Message-ID: On Tue, May 27, 2008 at 6:00 AM, Hartwell Bryan wrote: > Hi, > > Purpose: obtaining the system ("short") path from a full path > Best thing to do is to open an issue with the patch, otherwise this will just be ignored as we don't accept code to this mailing list. -Brett From barry at python.org Fri May 30 03:59:50 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 29 May 2008 21:59:50 -0400 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting In-Reply-To: <20080530011448.GA2964@panix.com> References: <20080530011448.GA2964@panix.com> Message-ID: <41278E9C-E27F-464D-BC00-8F7146E4483F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On May 29, 2008, at 9:14 PM, Aahz wrote: > I just processed two weeks of backlog on python-dev (I'm only a backup > moderator). Would anyone object to changing the list policy so that > non-member posts get discarded instead of held? How about we recruit additional moderators? Any volunteers? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSD9fl3EjvBPtnXfVAQK+cQP/Xwx5+k9gzBcDLG4QQGvAdMSXTfXjayxL OxDnWWzDNm+CfzDoY76GHhTPk/Jc3Rx80aTxBmv2DyElq+Z5r0TrqbYp25EFV4Ia J0P4q2WGkBq71LFYuTrmcYp7Z2oF47M/vm3F1zpErgXK+zEiOfKlQO9V1FMKNnRu sG4vOFq+tbU= =gWHk -----END PGP SIGNATURE----- From tjreedy at udel.edu Fri May 30 05:34:01 2008 From: tjreedy at udel.edu (tjreedy) Date: Thu, 29 May 2008 23:34:01 -0400 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting References: <20080530011448.GA2964@panix.com> <41278E9C-E27F-464D-BC00-8F7146E4483F@python.org> Message-ID: "Barry Warsaw" wrote in message news:41278E9C-E27F-464D-BC00-8F7146E4483F at python.org... > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On May 29, 2008, at 9:14 PM, Aahz wrote: > >> I just processed two weeks of backlog on python-dev (I'm only a backup >> moderator). Would anyone object to changing the list policy so that >> non-member posts get discarded instead of held? By 'non-member' I hope you mean 'never approved before' as opposed to 'non-subscriber' like me who participates through gmane. > > How about we recruit additional moderators? Any volunteers? Yes From tim.peters at gmail.com Fri May 30 05:37:13 2008 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 29 May 2008 23:37:13 -0400 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting In-Reply-To: <41278E9C-E27F-464D-BC00-8F7146E4483F@python.org> References: <20080530011448.GA2964@panix.com> <41278E9C-E27F-464D-BC00-8F7146E4483F@python.org> Message-ID: <1f7befae0805292037s27db6bck686e6db42a461f8@mail.gmail.com> [Barry] > ... > How about we recruit additional moderators? Any volunteers? Sure -- add me as a python-dev admin, send me the password, and go back to eating in peace :-) From aahz at pythoncraft.com Fri May 30 07:36:25 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 29 May 2008 22:36:25 -0700 Subject: [Python-Dev] FWD: 26 Python-Dev moderator request(s) waiting In-Reply-To: References: <20080530011448.GA2964@panix.com> <41278E9C-E27F-464D-BC00-8F7146E4483F@python.org> Message-ID: <20080530053624.GA6852@panix.com> On Thu, May 29, 2008, tjreedy wrote: >> On May 29, 2008, at 9:14 PM, Aahz wrote: >>> >>> I just processed two weeks of backlog on python-dev (I'm only a backup >>> moderator). Would anyone object to changing the list policy so that >>> non-member posts get discarded instead of held? > > By 'non-member' I hope you mean 'never approved before' as opposed to > 'non-subscriber' like me who participates through gmane. This seems to be a settled issue now, but IMO the proper way to handle your situation is to subscribe from the e-mail address you use as your From: line, setting the subscription to "no mail". -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From flashk at gmail.com Fri May 30 08:19:21 2008 From: flashk at gmail.com (Farshid Lashkari) Date: Thu, 29 May 2008 23:19:21 -0700 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> Message-ID: <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> On Wed, May 28, 2008 at 10:03 AM, Jesse Noller wrote: > I would like to renew the discussion now that "there is a PEP" to see > if there are any outstanding things people would like to get resolved. > I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback > both here and elsewhere that people would rather see this in sooner in > some form, rather than later (i.e.: 2.7/3.1). I'm not sure if this is a big issue, but I just tested the module with an application that embeds Python and it doesn't work properly. Instead of spawning worker threads using python.exe, it attempts to use the application. Does the processing module allow specifying the exe to use for spawning worker threads? I would definitely like to see this included in the next release, and having support for embedded Python interpreters would be a big plus. -Farshid From g.brandl at gmx.net Fri May 30 08:21:44 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 30 May 2008 08:21:44 +0200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <483F312E.1020302@canterbury.ac.nz> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Georg Brandl wrote: >> Greg Ewing schrieb: > > >>> Doesn't isinstance(x, basestring) already cover that? >> >> That doesn't cover UserString, for example. > > A better solution to that might be to have UserString > inherit from basestring. But with that argument you could throw out the whole ABC machinery, just let all lists inherit from list, all dicts from dict, etc. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ulrich.berning at denviso.de Fri May 30 09:10:51 2008 From: ulrich.berning at denviso.de (Ulrich Berning) Date: Fri, 30 May 2008 09:10:51 +0200 Subject: [Python-Dev] Obtaining short file path In-Reply-To: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int> References: <2EDCAD28478FC346B0C7A59FA49448B507002C96@MSX102V1.corp.int> Message-ID: <483FA87B.3010106@denviso.de> Hartwell Bryan wrote: > Hi, > > Purpose: obtaining the system (?short?) path from a full path > > Background: File dialogs (visual studio) return a full path (e.g. > f=?C:\this path has spaces\thisfilenameislongerthan8char.txt?). If > this value is provided to Python, it will not recongize this as a > file. In fact os.path.isfile(f) doesn?t return false, it crashes. > Likewise, when calling executables (from Python) with files as > arguments a short path is required. VB FileSystemObject has the > ShortPath method, while os.path and path (www.jorendorff.com) modules > do not (at least as far as my googling could determine). Why bother > creating a COM interface when you?re just going to pass as shell > run-time arguments all the values the server is better at computing? > > System: Python 2.3; Windows XP > > Sample Code: > > import win32com.client > > import time > > import os,sys > > import os.path > > #------------------------------------------------------------- > > def shortpath(x): > > z='' > > for y in x.split('\\'): > > if len(y.split('.')[0])>8: > > if ('.' in y): > > z=z+'\\'+y.split('.')[0][:6].upper()+'~1'+'.'+y.split('.')[1] > > else: > > z=z+'\\'+y[:6].upper()+'~1' > > else: > > z=z+'\\'+y > > return z[1:] > > #------------------------------------------------------------- > > xlApp = win32com.client.Dispatch("Excel.Application") > > xlBook = xlApp.ActiveWorkbook > > savFile = str(sys.argv[1]) > > rawFile = str(xlBook.Sheets("Timestamp").TextBox2) > > #print os.path.isfile(savFile) > > r=shortpath(rawFile) > > print r > > try: > > print os.path.isfile(r) > > except: > > print 'something rude' > > time.sleep(7) > > Notes: This code does not account for peer paths or files that share > the first 8 characters (and file extension). I?m also aware that this > is not the normal means for submitting a ?patch?, but in my job > function I don?t see myself regularly participating in python > development (and I?m probably not savvy enough) so the effort wasn?t > worth it. However I still thought others might benefit from what seems > to be (to me) a fundamental path function. Do with it, or ignore it, > as you please. > > Cheers, > > Bryan Hartwell > > ------------------------------------------------------------------------ > > This message is intended only for the use of the intended recipients, > and it may be privileged and confidential. If you are not the intended > recipient, you are hereby notified that any review, retransmission, > conversion to hard copy, copying, circulation or other use of this > message is strictly prohibited. If you are not the intended recipient, > please notify me immediately by return e-mail, and delete this message > from your system. > >------------------------------------------------------------------------ > >_______________________________________________ >Python-Dev mailing list >Python-Dev at python.org >http://mail.python.org/mailman/listinfo/python-dev >Unsubscribe: http://mail.python.org/mailman/options/python-dev/ulrich.berning%40denviso.de > > Why not win32api.GetShortPathName() and win32api.GetFullPathName()? >>> import os, win32api >>> path = "C:\\this path has spaces\\thisfilehasmorethan8char.txt" >>> short_path = win32api.GetShortPathName(path) >>> short_path 'C:\\THISPA~1\\THISFI~1.TXT' >>> os.path.isfile(short_path) True >>> full_path = win32api.GetLongPathName(short_path) >>> full_path 'C:\\this path has spaces\\thisfilehasmorethan8char.txt' >>> os.path.isfile(full_path) True >>> path == full_path True >>> Ulli From greg at krypto.org Fri May 30 09:45:25 2008 From: greg at krypto.org (Gregory P. Smith) Date: Fri, 30 May 2008 00:45:25 -0700 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483F34C3.3050402@gmail.com> References: <48397ECC.9070805@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> Message-ID: <52dc1c820805300045g5c37256em31de5f5d76dc365b@mail.gmail.com> On Thu, May 29, 2008 at 3:57 PM, Nick Coghlan wrote: > M.-A. Lemburg wrote: >> * Why should the 2.x code base turn to hacks, just because 3.x wants >> to restructure itself ? > > With the better explanation from Greg of what the checked in approach > achieves (i.e. preserving exact ABI compatibility for PyString_*, while > allowing PyBytes_* to be used at the source code level), I don't see what > has been done as being any more of a hack than the possibly more common > "#define " (which *would* break binary compatibility). > > The only things that I think would tidy it up further would be to: > - include an explanation of the approach and its effects on API and ABI > backward and forward compatibility within 2.x and between 2.x and 3.x in > stringobject.h > - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0 Yes that is the only complaint I believe I really see left at this point. It is easy enough to fix. Change the current stringobject.h "#define PyBytes_Foo PyString_Foo" approach into a .c file that defines one line stub functions for all PyString_Foo() functions to call actual PyBytes_Foo() functions. I'd even go so far as to put the one line alternate name stubs in the Objects/bytesobject.c and .h file right next to the PyBytes_Foo() method definitions so that its clear from reading a single file that they are the same thing. The performance implications of this are minor all things considered (a single absolute jmp given a good compiler) and regardless of what we do should only apply to extension modules, not the core. If we do the above in trunk will this thread end? I'm personally not really clear on why we need PyBytes_Foo to show up in the -binary- ABI in 2.6. The #define's are enough for me but I'm happy to make this compromise. No 2.x books, documentation or literature will be invalidated by the changes regardless. -gps From greg.ewing at canterbury.ac.nz Fri May 30 10:21:10 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 30 May 2008 20:21:10 +1200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> Message-ID: <483FB8F6.4060504@canterbury.ac.nz> Georg Brandl wrote: > Greg Ewing schrieb: > >> A better solution to that might be to have UserString >> inherit from basestring. > > But with that argument you could throw out the whole ABC machinery, > just let all lists inherit from list, all dicts from dict, etc. Well, I'm skeptical about the whole ABC thing in the first place -- it all seems very unpythonic to me. But another way of thinking about it is that we already have an ABC of sorts for strings, and it's called basestring. It might be better to enhance that with whatever's considered missing than introducing another one. -- Greg From mal at egenix.com Fri May 30 10:37:08 2008 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 30 May 2008 10:37:08 +0200 Subject: [Python-Dev] [Python-3000] Stabilizing the C API of 2.6 and 3.0 In-Reply-To: <483F34C3.3050402@gmail.com> References: <48397ECC.9070805@cheimes.de> <483ABB23.6050900@egenix.com> <483ABDCF.8000105@cheimes.de> <483AD138.7000804@egenix.com> <483B2D02.8040400@cheimes.de> <483BDE11.509@egenix.com> <483D300B.5090309@egenix.com> <52dc1c820805281347n75a4baax9222b75c8fa09ec5@mail.gmail.com> <483ECA52.6040000@egenix.com> <483ECF94.7060607@cheimes.de> <483EF139.8000606@egenix.com> <483F34C3.3050402@gmail.com> Message-ID: <483FBCB4.5020007@egenix.com> On 2008-05-30 00:57, Nick Coghlan wrote: > M.-A. Lemburg wrote: >> * Why can't we have both PyString *and* PyBytes exposed in 2.x, >> with one redirecting to the other ? > > We do have that - the PyString_* names still work perfectly fine in 2.x. > They just won't be used in the Python core codebase anymore - everything > in the Python core will use either PyBytes_* or PyUnicode_* regardless > of which branch (2.x or 3.x) you're working on. I think that's a good > thing for ease of maintenance in the future, even if it takes people a > while to get their heads around it right now. Sorry, I probably wasn't clear enough: Why can't we have both PyString *and* PyBytes exposed as C APIs (ie. visible in code and in the linker) in 2.x, with one redirecting to the other ? >> * Why should the 2.x code base turn to hacks, just because 3.x wants >> to restructure itself ? > > With the better explanation from Greg of what the checked in approach > achieves (i.e. preserving exact ABI compatibility for PyString_*, while > allowing PyBytes_* to be used at the source code level), I don't see > what has been done as being any more of a hack than the possibly more > common "#define " (which *would* break binary > compatibility). > > The only things that I think would tidy it up further would be to: > - include an explanation of the approach and its effects on API and ABI > backward and forward compatibility within 2.x and between 2.x and 3.x in > stringobject.h > - expose the PyBytes_* functions to the linker in 2.6 as well as 3.0 Which is what I was suggesting all along; sorry if I wasn't clear enough on that. The standard approach is that you provide #define redirects from the old APIs to the new ones (which are then picked up by the compiler) *and* add function wrappers to the same affect (to make linkers, dynamic load APIs such ctypes and debuggers happy). Example from pythonrun.h|c: --------------------------- /* Use macros for a bunch of old variants */ #define PyRun_String(str, s, g, l) PyRun_StringFlags(str, s, g, l, NULL) /* Deprecated C API functions still provided for binary compatiblity */ #undef PyRun_String PyAPI_FUNC(PyObject *) PyRun_String(const char *str, int s, PyObject *g, PyObject *l) { return PyRun_StringFlags(str, s, g, l, NULL); } I still believe that we should *not* make "easy of merging" the primary motivation for backporting changes in 3.x to 2.x. Software design should not be guided by restrictions in the tool chain, if not absolutely necessary. The main argument for a backport needs to be general usefulness to the 2.x users, IMHO... just like any other feature that makes it into 2.x. If merging is difficult then this needs to be addressed, but there are more options to that than always going back to the original 2.x trunk code. I've given a few suggestions on how this could be approached in other emails on this thread. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 30 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2008-07-07: EuroPython 2008, Vilnius, Lithuania 37 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From ncoghlan at gmail.com Fri May 30 11:47:35 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 30 May 2008 19:47:35 +1000 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: References: Message-ID: <483FCD37.30104@gmail.com> Simon Cross wrote: > It struct me as odd that this one case shows such a big difference > while the others show less of one. I believe the %-formatting code has some optimisations in place where it realises it can just increment the reference count of the passed in string and return that, rather than having to build a new string object. As for why you didn't see any differences in a couple of your tests: the .format() call wasn't actually substituting anything. Unfortunately, the reasons why .format() doesn't complain about extra arguments never made it into the PEP. They can be found here: http://mail.python.org/pipermail/python-dev/2006-May/065062.html Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jnoller at gmail.com Fri May 30 13:29:52 2008 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 30 May 2008 07:29:52 -0400 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> Message-ID: <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> On Fri, May 30, 2008 at 2:19 AM, Farshid Lashkari wrote: > On Wed, May 28, 2008 at 10:03 AM, Jesse Noller wrote: >> I would like to renew the discussion now that "there is a PEP" to see >> if there are any outstanding things people would like to get resolved. >> I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback >> both here and elsewhere that people would rather see this in sooner in >> some form, rather than later (i.e.: 2.7/3.1). > > I'm not sure if this is a big issue, but I just tested the module with > an application that embeds Python and it doesn't work properly. > Instead of spawning worker threads using python.exe, it attempts to > use the application. Does the processing module allow specifying the > exe to use for spawning worker threads? I would definitely like to see > this included in the next release, and having support for embedded > Python interpreters would be a big plus. > > -Farshid > Let's take this off-list Farshid - but I would like to know more about your application (I'm not up to speed with embedding python in other applications) so I can either add it to the PEP or discuss how best to address this within the module itself. -jesse From theller at ctypes.org Fri May 30 14:45:15 2008 From: theller at ctypes.org (Thomas Heller) Date: Fri, 30 May 2008 14:45:15 +0200 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> Message-ID: Jesse Noller schrieb: > On Fri, May 30, 2008 at 2:19 AM, Farshid Lashkari wrote: >> On Wed, May 28, 2008 at 10:03 AM, Jesse Noller wrote: >>> I would like to renew the discussion now that "there is a PEP" to see >>> if there are any outstanding things people would like to get resolved. >>> I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback >>> both here and elsewhere that people would rather see this in sooner in >>> some form, rather than later (i.e.: 2.7/3.1). >> >> I'm not sure if this is a big issue, but I just tested the module with >> an application that embeds Python and it doesn't work properly. >> Instead of spawning worker threads using python.exe, it attempts to >> use the application. Does the processing module allow specifying the >> exe to use for spawning worker threads? I would definitely like to see >> this included in the next release, and having support for embedded >> Python interpreters would be a big plus. >> >> -Farshid >> > > Let's take this off-list Farshid - but I would like to know more about > your application (I'm not up to speed with embedding python in other > applications) so I can either add it to the PEP or discuss how best to > address this within the module itself. This may be an issue for py2exe users as well. Please keep me informed (or is there a pyprocessing mailing list where this is discussed?). Thanks, Thomas From armin.ronacher at active-4.com Fri May 30 16:48:41 2008 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Fri, 30 May 2008 14:48:41 +0000 (UTC) Subject: [Python-Dev] Iterable String Redux (aka String ABC) References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> <483FB8F6.4060504@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > Well, I'm skeptical about the whole ABC thing in the > first place -- it all seems very unpythonic to me. I think it's very pythonic and the very best solution to interfaces *and* duck typing. Not only does it extend duck-typing in a very, very cool way but also does it provide a very cool way to get custom sets or lists going with few extra work. Subclassing builtins was always very painful in the past and many used the User* objects which however often broke because some code did something like isinstance(x, (tuple, list)). Of course one could argue that instance checking is the root of all evil but there are situations where you have to do instance checking. And ABCs are the perfect solution for that as they combine duck-typing and instance checking. In my oppinion ABCs are the best feature of 2.6 and 3.0. > But another way of thinking about it is that we > already have an ABC of sorts for strings, and it's > called basestring. It might be better to enhance > that with whatever's considered missing than > introducing another one. basestring is not subclassable for example. Also it requires subclassing which ABCs do not. Regards, Armin From rrr at ronadam.com Fri May 30 17:50:08 2008 From: rrr at ronadam.com (Ron Adam) Date: Fri, 30 May 2008 10:50:08 -0500 Subject: [Python-Dev] Iterable String Redux (aka StringABC) In-Reply-To: <1791A55949334749A9DF448461C9B11A@RaymondLaptop1> References: <1791A55949334749A9DF448461C9B11A@RaymondLaptop1> Message-ID: <48402230.3020505@ronadam.com> Raymond Hettinger wrote: > "Jim Jewett" >> It isn't really stringiness that matters, it is that you have to >> terminate even though you still have an iterable container. > > Well said. > > >> Guido had at least a start in Searchable, back when ABC >> were still in the sandbox: > > Have to disagree here. An object cannot know in general > whether a flattener wants to split it or not. That is an application > dependent decision. A better answer is be able to tell the > flattener what should be considered atomic in a given circumstance. > > > Raymond A while back (a couple of years I think), we had a discussion on python-list about flatten in which I posted the following version of a flatten function. It turned out to be nearly twice as fast as any other version. def flatten(L): """ Flatten a list in place. """ i = 0 while i < len(L): while type(L[i]) is list: L[i:i+1] = L[i] i += 1 return L For this to work the object to be flattened needs to be both mutable and list like. At the moment I can't think of any reason I would want to flatten anything that was not list like. To make it a bit more flexible it could be changed just a bit. def flatten(L): """ Flatten a list in place. """ objtype = type(L) i = 0 while i < len(L): while type(L[i]) is objtype: L[i:i+1] = L[i] i += 1 return L Generally, I don't think you would want to flatten dissimilar objects. Cheers, Ron From rrr at ronadam.com Fri May 30 17:50:08 2008 From: rrr at ronadam.com (Ron Adam) Date: Fri, 30 May 2008 10:50:08 -0500 Subject: [Python-Dev] Iterable String Redux (aka StringABC) In-Reply-To: <1791A55949334749A9DF448461C9B11A@RaymondLaptop1> References: <1791A55949334749A9DF448461C9B11A@RaymondLaptop1> Message-ID: <48402230.3020505@ronadam.com> Raymond Hettinger wrote: > "Jim Jewett" >> It isn't really stringiness that matters, it is that you have to >> terminate even though you still have an iterable container. > > Well said. > > >> Guido had at least a start in Searchable, back when ABC >> were still in the sandbox: > > Have to disagree here. An object cannot know in general > whether a flattener wants to split it or not. That is an application > dependent decision. A better answer is be able to tell the > flattener what should be considered atomic in a given circumstance. > > > Raymond A while back (a couple of years I think), we had a discussion on python-list about flatten in which I posted the following version of a flatten function. It turned out to be nearly twice as fast as any other version. def flatten(L): """ Flatten a list in place. """ i = 0 while i < len(L): while type(L[i]) is list: L[i:i+1] = L[i] i += 1 return L For this to work the object to be flattened needs to be both mutable and list like. At the moment I can't think of any reason I would want to flatten anything that was not list like. To make it a bit more flexible it could be changed just a bit. def flatten(L): """ Flatten a list in place. """ objtype = type(L) i = 0 while i < len(L): while type(L[i]) is objtype: L[i:i+1] = L[i] i += 1 return L Generally, I don't think you would want to flatten dissimilar objects. Cheers, Ron From status at bugs.python.org Fri May 30 18:06:34 2008 From: status at bugs.python.org (Python tracker) Date: Fri, 30 May 2008 18:06:34 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20080530160634.32170782E1@psf.upfronthosting.co.za> ACTIVITY SUMMARY (05/23/08 - 05/30/08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 1889 open (+35) / 12944 closed (+25) / 14833 total (+60) Open issues with patches: 566 Average duration of open issues: 706 days. Median duration of open issues: 1437 days. Open Issues Breakdown open 1866 (+35) pending 23 ( +0) Issues Created Or Reopened (63) _______________________________ Cant open python gui using VISTA 05/28/08 CLOSED http://bugs.python.org/issue1948 reopened amaury.forgeotdarc Remove commands for PEP 3108 05/26/08 CLOSED http://bugs.python.org/issue2872 reopened brett.cannon patch ElementTree parsing bus error (but only from mod_python) 05/23/08 http://bugs.python.org/issue2951 created kathyvs List comprehensions are leaking variables 05/23/08 CLOSED http://bugs.python.org/issue2952 created cartman _zip_directory_cache untested and undocumented 05/23/08 http://bugs.python.org/issue2953 created fijal [PATCH] Make bisect module functions accept an optional comparis 05/23/08 CLOSED http://bugs.python.org/issue2954 created llucax patch Python 2.5 Documentation error in Tutorial section 8.3 05/24/08 CLOSED http://bugs.python.org/issue2955 created dcw303 2to3 should have a way to disable some fixers 05/24/08 http://bugs.python.org/issue2956 created bhy patch recursion limit exceeded when importing .pyc module 05/24/08 CLOSED http://bugs.python.org/issue2957 created bhy update Lib/test/README 05/24/08 http://bugs.python.org/issue2958 created benjamin.peterson easy calling GzipFile close() more than once causes exception 05/24/08 CLOSED http://bugs.python.org/issue2959 created mmagin patch bsddb/test/test_replication.py bus error, segfault, assertion er 05/25/08 http://bugs.python.org/issue2960 created gregory.p.smith Two error messages inconsistent 05/25/08 CLOSED http://bugs.python.org/issue2961 created chester Goodbye, 'global' statement! 05/25/08 CLOSED http://bugs.python.org/issue2962 created chester Method cache is broken in Py3 05/25/08 CLOSED http://bugs.python.org/issue2963 created scoder patch instancemethod_descr_get() lacks an INCREF 05/25/08 CLOSED http://bugs.python.org/issue2964 created scoder patch Update interface of weakref dictionaries 05/25/08 http://bugs.python.org/issue2965 created georg.brandl patch pydoc doesnt show 'from module import identifier' in the docs 05/25/08 http://bugs.python.org/issue2966 created peter.puk [PATCH] Rename Tkinter to tkinter in test_tcl 05/25/08 CLOSED http://bugs.python.org/issue2967 created cartman patch Test_imports takes a long time to run 05/25/08 http://bugs.python.org/issue2968 created loewis Test_imports fails in 2.6 05/25/08 http://bugs.python.org/issue2969 created loewis test_distutils fails on Linux 05/26/08 CLOSED http://bugs.python.org/issue2970 created cartman test_zipfile64 fails 05/26/08 http://bugs.python.org/issue2971 created cartman arguments and default path not set in site.py and sitecustomize. 05/26/08 http://bugs.python.org/issue2972 created hvendelbo _ssl compiler warnings 05/26/08 http://bugs.python.org/issue2973 created benjamin.peterson test_subprocess fails on Darwin 05/26/08 CLOSED http://bugs.python.org/issue2974 created cartman VS8 include dirs grow without bound 05/26/08 http://bugs.python.org/issue2975 created jkleckner patch test_pydoc fails in trunk 05/26/08 http://bugs.python.org/issue2976 created theller truncation of text in tables in Library Reference PDF 05/26/08 http://bugs.python.org/issue2977 created hsmyers typo in text of page 80 of Library Reference PDF 05/26/08 CLOSED http://bugs.python.org/issue2978 created hsmyers use_datetime in SimpleXMLRPCServer 05/27/08 http://bugs.python.org/issue2979 created vdupras patch Pickle stream for unicode object may contain non-ASCII character 05/27/08 http://bugs.python.org/issue2980 created Mawbid confusing action of struct.pack and struct.unpack with fmt 'p' 05/27/08 http://bugs.python.org/issue2981 created liyu more tests for pyexpat 05/27/08 http://bugs.python.org/issue2982 created fijal Ttk support for Tkinter 05/27/08 http://bugs.python.org/issue2983 created gpolo test_dbm fails on Darwin 05/27/08 CLOSED http://bugs.python.org/issue2984 created cartman xmlrpclib doesn't support 64bit integer replies 05/27/08 CLOSED http://bugs.python.org/issue2985 created shrike patch difflib.SequenceMatcher not matching long sequences 05/27/08 http://bugs.python.org/issue2986 created hagna RFC2732 support for urlparse (e.g. http://[::1]:80/) 05/27/08 http://bugs.python.org/issue2987 created ndim patch Invalid cookies crash web applications 05/28/08 CLOSED http://bugs.python.org/issue2988 created techtonik type_modified() in typeobject.c should be public 05/28/08 CLOSED http://bugs.python.org/issue2989 created scoder patch type cache updates might run cleanup code in an inconsistent sta 05/28/08 CLOSED http://bugs.python.org/issue2990 created scoder patch Bad behavior in PythonWin 05/28/08 CLOSED http://bugs.python.org/issue2991 created hsmyers Bad behavior in PythonWin 05/28/08 CLOSED http://bugs.python.org/issue2992 created hsmyers Bad behavior in PythonWin 05/28/08 CLOSED http://bugs.python.org/issue2993 created hsmyers Bad behavior in PythonWin 05/28/08 CLOSED http://bugs.python.org/issue2994 created hsmyers Idle, some Linuxes, cannot position Cursor by mouseclick 05/28/08 http://bugs.python.org/issue2995 created Sunny0815 IDLE "find in files" output not formatted optimally 05/28/08 http://bugs.python.org/issue2996 created russell.blau patch PyNumberMethods has left-over fields in Py3 05/29/08 http://bugs.python.org/issue2997 created scoder patch UnicodeEncodeError: 'ascii' codec can't encode character 05/29/08 CLOSED http://bugs.python.org/issue2998 created Muallim-i ??li Py30a5: str.replace() tiny doc error 05/29/08 CLOSED http://bugs.python.org/issue2999 created mark 2to3 doesn't handle print(whatever); print nor string.* function 05/29/08 http://bugs.python.org/issue3000 created mark RLock's are SLOW 05/29/08 http://bugs.python.org/issue3001 created jcea shutil.copyfile blocks indefinitely on named pipes 05/29/08 http://bugs.python.org/issue3002 created aioryi sys.stdin.fileno() gives attribute error in IDLE 05/29/08 http://bugs.python.org/issue3003 created markveldhuis Bug in slice.indices() 05/29/08 http://bugs.python.org/issue3004 created anakha patch EasyDialogs - documentation enhancement 05/29/08 http://bugs.python.org/issue3005 created keenethery subprocess.Popen causes socket to remain open after close 05/29/08 http://bugs.python.org/issue3006 created mamulsow Remove old StringIO docs 05/29/08 CLOSED http://bugs.python.org/issue3007 created hdiogenes patch Let bin() show floats 05/30/08 http://bugs.python.org/issue3008 created rhettinger 3.0a5 tarballs contain the code twice 05/30/08 http://bugs.python.org/issue3009 created barry Module cmd documentation enhancement 05/30/08 http://bugs.python.org/issue3010 created deets patch TelnetPopen3, TelnetBase, Expect split 05/30/08 http://bugs.python.org/issue708007 reopened benjamin.peterson patch Issues Now Closed (58) ______________________ Make .pypirc handle multiple servers 128 days http://bugs.python.org/issue1858 tarek patch Cant open python gui using VISTA 2 days http://bugs.python.org/issue1948 loewis stat.ST_CTIME and stat.ST_ATIME problem 64 days http://bugs.python.org/issue2455 gregory.p.smith inconsistency with bare * in parameter list 46 days http://bugs.python.org/issue2613 bgolemon bsddb iterkeys sliently fails when database modified during iter 33 days http://bugs.python.org/issue2669 gregory.p.smith fast builtin sum may leak 22 days http://bugs.python.org/issue2784 rhettinger patch Py30a5: Tk Menu Alt-f behaves differently Linux vs Windows 15 days http://bugs.python.org/issue2806 gpolo sidebar directive fails 10 days http://bugs.python.org/issue2852 georg.brandl Add gestalt back into Python 3.0 15 days http://bugs.python.org/issue2854 benjamin.peterson patch lookkey should INCREF/DECREF startkey around PyObject_RichCompar 15 days http://bugs.python.org/issue2855 rhettinger patch Remove commands for PEP 3108 0 days http://bugs.python.org/issue2872 benjamin.peterson patch Rename the thread module to _thread 11 days http://bugs.python.org/issue2875 brett.cannon Backport UserString move from 3.0 13 days http://bugs.python.org/issue2877 brett.cannon patch Backport UserList move in 3.0 14 days http://bugs.python.org/issue2878 brett.cannon patch Rename _winreg to winreg 9 days http://bugs.python.org/issue2879 georg.brandl Create the dbm package 10 days http://bugs.python.org/issue2881 georg.brandl Create the http package 10 days http://bugs.python.org/issue2883 georg.brandl patch Create the xmlrpc package 10 days http://bugs.python.org/issue2886 georg.brandl urllib not handling ftp servers that do not support REST 9 days http://bugs.python.org/issue2891 gregory.p.smith Add __name__ in globals of generated namedtuple namespace 12 days http://bugs.python.org/issue2903 rhettinger patch Cross-referencing to a method using an alternate title don't wor 6 days http://bugs.python.org/issue2904 georg.brandl tkinter, assorted fixes 11 days http://bugs.python.org/issue2906 georg.brandl patch, patch struct.Struct.unpack to return a namedtuple for easier attribute 5 days http://bugs.python.org/issue2909 rhettinger patch Revert ConfigParser rename in 2.6 5 days http://bugs.python.org/issue2923 georg.brandl patch Revert Queue rename in 2.6 5 days http://bugs.python.org/issue2925 georg.brandl patch Revert SocketServer rename in 2.6 5 days http://bugs.python.org/issue2926 georg.brandl patch Interactive help writes to the python install destination direct 4 days http://bugs.python.org/issue2938 loewis Apache mod_python python-func strftime 5 days http://bugs.python.org/issue2939 gregory.p.smith Building Python fails on SunOS 5 days http://bugs.python.org/issue2940 loewis setuptools: bdist_wininst adds duplicate entry to dist_files 1 days http://bugs.python.org/issue2946 loewis Windows installer doesn't include OpenSSL license and notice 2 days http://bugs.python.org/issue2949 loewis silly readline module problem 1 days http://bugs.python.org/issue2950 gpolo List comprehensions are leaking variables 0 days http://bugs.python.org/issue2952 loewis [PATCH] Make bisect module functions accept an optional comparis 7 days http://bugs.python.org/issue2954 llucax patch Python 2.5 Documentation error in Tutorial section 8.3 0 days http://bugs.python.org/issue2955 georg.brandl recursion limit exceeded when importing .pyc module 2 days http://bugs.python.org/issue2957 amaury.forgeotdarc calling GzipFile close() more than once causes exception 0 days http://bugs.python.org/issue2959 georg.brandl patch Two error messages inconsistent 0 days http://bugs.python.org/issue2961 georg.brandl Goodbye, 'global' statement! 0 days http://bugs.python.org/issue2962 georg.brandl Method cache is broken in Py3 0 days http://bugs.python.org/issue2963 georg.brandl patch instancemethod_descr_get() lacks an INCREF 0 days http://bugs.python.org/issue2964 georg.brandl patch [PATCH] Rename Tkinter to tkinter in test_tcl 0 days http://bugs.python.org/issue2967 georg.brandl patch test_distutils fails on Linux 0 days http://bugs.python.org/issue2970 benjamin.peterson test_subprocess fails on Darwin 0 days http://bugs.python.org/issue2974 benjamin.peterson typo in text of page 80 of Library Reference PDF 0 days http://bugs.python.org/issue2978 benjamin.peterson test_dbm fails on Darwin 1 days http://bugs.python.org/issue2984 brett.cannon xmlrpclib doesn't support 64bit integer replies 3 days http://bugs.python.org/issue2985 georg.brandl patch Invalid cookies crash web applications 1 days http://bugs.python.org/issue2988 georg.brandl type_modified() in typeobject.c should be public 0 days http://bugs.python.org/issue2989 georg.brandl patch type cache updates might run cleanup code in an inconsistent sta 1 days http://bugs.python.org/issue2990 georg.brandl patch Bad behavior in PythonWin 0 days http://bugs.python.org/issue2991 facundobatista Bad behavior in PythonWin 0 days http://bugs.python.org/issue2992 facundobatista Bad behavior in PythonWin 0 days http://bugs.python.org/issue2993 facundobatista Bad behavior in PythonWin 0 days http://bugs.python.org/issue2994 facundobatista UnicodeEncodeError: 'ascii' codec can't encode character 1 days http://bugs.python.org/issue2998 Muallim-i ??li Py30a5: str.replace() tiny doc error 1 days http://bugs.python.org/issue2999 georg.brandl Remove old StringIO docs 0 days http://bugs.python.org/issue3007 georg.brandl patch site enhancements 1152 days http://bugs.python.org/issue1174614 georg.brandl patch Top Issues Most Discussed (10) ______________________________ 8 pprint produces different output in 2.6 and 3.0 14 days open http://bugs.python.org/issue2888 8 Add ctypes calling convention that allows safe access of errno 140 days open http://bugs.python.org/issue1798 7 xmlrpclib doesn't support 64bit integer replies 3 days closed http://bugs.python.org/issue2985 7 Pickle stream for unicode object may contain non-ASCII characte 3 days open http://bugs.python.org/issue2980 7 Backport UserString move from 3.0 13 days closed http://bugs.python.org/issue2877 6 VS8 include dirs grow without bound 4 days open http://bugs.python.org/issue2975 5 ctypes.util.find_library() doesn't consult LD_LIBRARY_PATH 9 days open http://bugs.python.org/issue2936 5 Revert Queue rename in 2.6 5 days closed http://bugs.python.org/issue2925 5 Add memory footprint query 13 days open http://bugs.python.org/issue2898 5 Rename the thread module to _thread 11 days closed http://bugs.python.org/issue2875 From fuzzyman at voidspace.org.uk Fri May 30 18:17:02 2008 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 30 May 2008 17:17:02 +0100 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> <483FB8F6.4060504@canterbury.ac.nz> Message-ID: <4840287E.5000505@voidspace.org.uk> Armin Ronacher wrote: > Greg Ewing canterbury.ac.nz> writes: > > >> Well, I'm skeptical about the whole ABC thing in the >> first place -- it all seems very unpythonic to me. >> > I think it's very pythonic and the very best solution to interfaces *and* > duck typing. Not only does it extend duck-typing in a very, very cool way > but also does it provide a very cool way to get custom sets or lists going > with few extra work. Subclassing builtins was always very painful in the > past and many used the User* objects which however often broke because some > code did something like isinstance(x, (tuple, list)). Of course one could > argue that instance checking is the root of all evil but there are > situations where you have to do instance checking. And ABCs are the > perfect solution for that as they combine duck-typing and instance > checking. > > In my oppinion ABCs are the best feature of 2.6 and 3.0. > > >> But another way of thinking about it is that we >> already have an ABC of sorts for strings, and it's >> called basestring. It might be better to enhance >> that with whatever's considered missing than >> introducing another one. >> > basestring is not subclassable for example. Also it requires subclassing > which ABCs do not. > > I would be strongly +1 on a string ABC. Currently (to my knowledge) there is no way of using duck typing for built-in APIs that expect a string. How do I pass in an object to 'open' for example that isn't actually a string or subclass? >>> class X(object): ... def __unicode__(self): ... return 'fish' ... __str__ = __repr__ = __unicode__ ... >>> x = X() >>> open(x) Traceback (most recent call last): File "", line 1, in TypeError: coercing to Unicode: need string or buffer, X found >>> unicode(x) u'fish' Michael Foord > Regards, > Armin > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.theotherdelia.co.uk/ http://www.voidspace.org.uk/ http://www.ironpython.info/ http://www.resolverhacks.net/ From ncoghlan at gmail.com Fri May 30 18:29:04 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 31 May 2008 02:29:04 +1000 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> Message-ID: <48402B50.6040809@gmail.com> Thomas Heller wrote: > Jesse Noller schrieb: >> On Fri, May 30, 2008 at 2:19 AM, Farshid Lashkari wrote: >>> On Wed, May 28, 2008 at 10:03 AM, Jesse Noller wrote: >>>> I would like to renew the discussion now that "there is a PEP" to see >>>> if there are any outstanding things people would like to get resolved. >>>> I chose to continue to push it for 2.6 / 3.0 inclusion due to feedback >>>> both here and elsewhere that people would rather see this in sooner in >>>> some form, rather than later (i.e.: 2.7/3.1). >>> I'm not sure if this is a big issue, but I just tested the module with >>> an application that embeds Python and it doesn't work properly. >>> Instead of spawning worker threads using python.exe, it attempts to >>> use the application. Does the processing module allow specifying the >>> exe to use for spawning worker threads? I would definitely like to see >>> this included in the next release, and having support for embedded >>> Python interpreters would be a big plus. >>> >>> -Farshid >>> >> Let's take this off-list Farshid - but I would like to know more about >> your application (I'm not up to speed with embedding python in other >> applications) so I can either add it to the PEP or discuss how best to >> address this within the module itself. > > This may be an issue for py2exe users as well. Please keep me informed (or > is there a pyprocessing mailing list where this is discussed?). py2exe is explicitly supported (via the freezeSupport() call). That may work for the embedded case as well, or it may be something that can be addressed by modifying sys.executable. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Fri May 30 18:40:18 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 31 May 2008 02:40:18 +1000 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <4840287E.5000505@voidspace.org.uk> References: <5D04DEB084364145ADFC45986EA548CA@RaymondLaptop1> <08May28.143320pdt.58698@synergy1.parc.xerox.com> <483DFB8B.5050909@canterbury.ac.nz> <483F312E.1020302@canterbury.ac.nz> <483FB8F6.4060504@canterbury.ac.nz> <4840287E.5000505@voidspace.org.uk> Message-ID: <48402DF2.7060906@gmail.com> Michael Foord wrote: > I would be strongly +1 on a string ABC. Currently (to my knowledge) > there is no way of using duck typing for built-in APIs that expect a > string. How do I pass in an object to 'open' for example that isn't > actually a string or subclass? Implement the character buffer API, which you can't actually do from Python code :P Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From flashk at gmail.com Fri May 30 19:21:24 2008 From: flashk at gmail.com (Farshid Lashkari) Date: Fri, 30 May 2008 10:21:24 -0700 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <48402B50.6040809@gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> Message-ID: <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> On Fri, May 30, 2008 at 9:29 AM, Nick Coghlan wrote: > py2exe is explicitly supported (via the freezeSupport() call). That may work > for the embedded case as well, or it may be something that can be addressed > by modifying sys.executable. Thanks for the tip Nick. Adding the following line before using the module works for me: sys.executable = os.path.join(sys.exec_prefix,'pythonw.exe') I'm not sure if there will be any side affects to modifying sys.executable though. Should this be the official way of supporting embedded interpreters or should there be a multiprocessing.setExecutable() method? -Farshid From jgennis at gmail.com Fri May 30 11:13:08 2008 From: jgennis at gmail.com (Jamie Gennis) Date: Fri, 30 May 2008 02:13:08 -0700 Subject: [Python-Dev] [Python-3000] Iterable String Redux (aka StringABC) In-Reply-To: <1791A55949334749A9DF448461C9B11A@RaymondLaptop1> References: <1791A55949334749A9DF448461C9B11A@RaymondLaptop1> Message-ID: Perhaps drawing a distinction between containers (or maybe "collections"?), and non-container iterables is appropriate? I would define containers as objects that can be iterated over multiple times and for which iteration does not instantiate new objects. By this definition generators would not be considered containers (but views would), and for practicality it may be worth also having an ABC for containers-and-generators (no idea what to name it). This would result in the following hierarchy: iterables - strings, bytes, etc. - containers-and-generators - - containers - - - tuple, list, set, dict views, etc. - - generators I don't think there needs to be different operations defined for the different ABCs. They're all just iterables with different iteration semantics. Jamie On Tue, May 27, 2008 at 3:54 PM, Raymond Hettinger wrote: > "Jim Jewett" > >> It isn't really stringiness that matters, it is that you have to >> terminate even though you still have an iterable container. >> > > Well said. > > > Guido had at least a start in Searchable, back when ABC >> were still in the sandbox: >> > > Have to disagree here. An object cannot know in general > whether a flattener wants to split it or not. That is an application > dependent decision. A better answer is be able to tell the > flattener what should be considered atomic in a given circumstance. > > > Raymond > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/jgennis%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ntoronto at cs.byu.edu Sat May 31 02:23:22 2008 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Fri, 30 May 2008 18:23:22 -0600 Subject: [Python-Dev] A thought on generic functions In-Reply-To: <483DF5FF.7010201@canterbury.ac.nz> References: <2D3FC96F19A04F29BE9B8086EB1971C0@RaymondLaptop1> <79990c6b0805280517w30e04673ma5404a3cdb2af6c5@mail.gmail.com> <483DF5FF.7010201@canterbury.ac.nz> Message-ID: <48409A7A.3000306@cs.byu.edu> Greg Ewing wrote: > Paul Moore wrote: >> I'd rather see a solution which addressed the >> wider visitor use case (I think I just sprained my back bending over >> backwards to avoid mentioning generic functions :-)) > > Speaking of generic functions, while thinking about the > recent discussion on proxy objects, it occurred to me > that this is something you can do with an OO system > that you can't do so easily with a generic function > system. If the operations being proxied were generic > functions rather than methods, you'd have to override > them all individually instead of having a central point > to catch them all. It depends on your dispatch rules. Say the implementation orders the candidates lexically (like default CLOS). This is equivalent to choosing as first candidates the set of functions with the most specific first argument. Resolution for a generic function call and generic method call are semantically the same, so there's no reason not to have the latter, and proxying by __getattr__ tricks becomes doable again. Neil From eric+python-dev at trueblade.com Sat May 31 03:48:30 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 30 May 2008 21:48:30 -0400 Subject: [Python-Dev] optimization required: .format() is much slower than % In-Reply-To: <483C2535.2040306@trueblade.com> References: <483BD10E.7050600@gmail.com> <483C14D1.1050007@trueblade.com> <483C2535.2040306@trueblade.com> Message-ID: <4840AE6E.40900@trueblade.com> Eric Smith wrote: > Eric Smith wrote: >> Nick Coghlan wrote: >>> Secondly, the string % operator appears to have an explicit >>> optimisation for the 'just return str(self)' case. This optimisation >>> is missing from the new string format method. >> >> I'll see if I can optimize this case. > > 3.0, from svn: > $ ./python.exe -m timeit "'some text with {0}'.format('nothing')" > 100000 loops, best of 3: 3.14 usec per loop > > 3.0, with PyUnicode_ExactCheck, skipping __format__ lookup: > $ ./python.exe -m timeit "'some text with {0}'.format('nothing')" > 100000 loops, best of 3: 2.32 usec per loop > > I could probably improve this some more, by skipping the creation of the > object used to describe the format_spec. > > Compare with: > $ ./python.exe -m timeit "'some text with %s' % 'nothing'" > 1000000 loops, best of 3: 1.37 usec per loop r63826 has an optimization when formatting types that are exactly unicode, long, or float (not subclasses). Unfortunately it's only for 3.0. For 2.6 it's messier. I'm still pondering what to do with that. I'll get to it sometime, but maybe after the beta. There's much more that can be done, but I'd like to profile it before I just start guessing. From steve at pearwood.info Sat May 31 05:13:10 2008 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 31 May 2008 13:13:10 +1000 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: References: <483FB8F6.4060504@canterbury.ac.nz> Message-ID: <200805311313.11501.steve@pearwood.info> On Sat, 31 May 2008 12:48:41 am Armin Ronacher wrote: > Greg Ewing canterbury.ac.nz> writes: > > Well, I'm skeptical about the whole ABC thing in the > > first place -- it all seems very unpythonic to me. > > I think it's very pythonic and the very best solution to interfaces > *and* duck typing. Not only does it extend duck-typing in a very, > very cool way I'm with Greg on this one: despite the assertions made in the PEP, I don't see how ABC can fail to be anything but anti-duck-typing. How does it extend duck-typing? Can you give an example? > but also does it provide a very cool way to get custom > sets or lists going with few extra work. Subclassing builtins was > always very painful in the past "Always" very painful? class ListWithClear(list): def clear(self): self[:] = self.__class__() Not so very painful to me. Maybe I just have more pain-tolerance than some people. > and many used the User* objects which > however often broke because some code did something like > isinstance(x, (tuple, list)). Of course one could argue that > instance checking is the root of all evil Perhaps not the root of *all* evil but it is certainly the root of much evil, and the treatment of delegation-based classes like UserString as second-class objects is a good example of why isinstance checking should be avoided as much as possible. -- Steven From p.f.moore at gmail.com Sat May 31 11:18:41 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 31 May 2008 10:18:41 +0100 Subject: [Python-Dev] PEP 371 Discussion (pyProcessing Module) In-Reply-To: <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> References: <4222a8490805281003s1b78ba12y999726187d5b57f7@mail.gmail.com> <978d1eac0805292319k14e50339y193609438e3dcd5a@mail.gmail.com> <4222a8490805300429p3a926e41g8bb5b1421ff43f67@mail.gmail.com> <48402B50.6040809@gmail.com> <978d1eac0805301021h6ab6b38cm57aabdf28b4bc6ae@mail.gmail.com> Message-ID: <79990c6b0805310218k36a4342bkcd5830ceb6af791d@mail.gmail.com> 2008/5/30 Farshid Lashkari : > I'm not sure if there will be any side affects to modifying > sys.executable though. Should this be the official way of supporting > embedded interpreters or should there be a > multiprocessing.setExecutable() method? +1 for setExecutable (I'd prefer set_executable, to be PEP 8 compliant). Make it explicit, rather than fiddling with stuff in sys manually. Paul. From g.brandl at gmx.net Sat May 31 13:36:29 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 31 May 2008 13:36:29 +0200 Subject: [Python-Dev] Iterable String Redux (aka String ABC) In-Reply-To: <200805311313.11501.steve@pearwood.info> References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> Message-ID: Steven D'Aprano schrieb: >> but also does it provide a very cool way to get custom >> sets or lists going with few extra work. Subclassing builtins was >> always very painful in the past > > "Always" very painful? > > class ListWithClear(list): > def clear(self): > self[:] = self.__class__() > > Not so very painful to me. Maybe I just have more pain-tolerance than > some people. Sure, nobody said that adding another method is a problem. But overriding methods like __getitem__() and having them used in other methods that derive from it (like get()) is impossible without resorting to UserDict, which in turn doesn't inherit from dict. ABCs unify these possibilities. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From python at rcn.com Sat May 31 14:25:11 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 31 May 2008 05:25:11 -0700 Subject: [Python-Dev] Alternative to more ABCs [was:] Iterable String Redux (aka String ABC) References: <483FB8F6.4060504@canterbury.ac.nz> <200805311313.11501.steve@pearwood.info> Message-ID: <1BE7988EAD8748B9B909A29DA4BC2FF4@RaymondLaptop1> ISTM, the whole reason people are asking for a String ABC is so you can write isinstance(obj, String) and allow registered string-like objects to be accepted. The downside is that everytime you want this for a concrete class or type, it is necessary to write a whole new ABC listing all of the required methods. Also, you have to change all of the client code's isinstance tests from concrete to abstract. I propose a simpler approach. Provide an alternative registration function that overrides isinstance() so that objects can explicitly fake any concrete type: s = UserString('whiffleball') print isinstance(s, str) --> False register_lookalike(UserString, str) print isinstance(s, str) --> True Besides saving us from writing tons of new ABCs, the approach works with existing code that already uses isinstance() with concrete types. The ABCs that would remain are ones that are truly abstract, that define a generic interface (like mappings and sequences) and ones that offer some useful mixin behavior. The remaining ABCs are ones where you have a fighting chance of actually being able to implement the interface (unlike String where it would be darned tricky to fully emulate encode(), split(), etc.) This would completely eliminate the need for numbers.Integral for example. AFAICT, its sole use case is to provide a way for numeric's integral types (like int8, int32) to pass themselves off as having the same API as regular ints. Unfortunately, the current approach requires all consumer code to switch from isinstance(x,int) to isinstance(x,Integral). It would be more useful if we could simply write register_lookalike(x,int) and be done with it (no need for numbers.py and its attendant abc machinery). If we don't do this, then String won't be the last request. People will want Datetime for example. Pretty much any concrete type could have a look-a-like that wanted its own ABC and for all client code to switch from testing concrete types. Raymond From steve at holdenweb.com Sat May 31 17:42:24 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 31 May 2008 11:42:24 -0400 Subject: [Python-Dev] Finishing up PEP 3108 In-Reply-To: References: Message-ID: <484171E0.4050204@holdenweb.com> Georg Brandl wrote: > Brett Cannon schrieb: > >>>> Issue 2873 - htmllib is slated to go, but pydoc still uses it. Then >>>> again, pydoc is busted thanks to the new doc format. >>> >>> I will try to handle this in the coming week. >>> >> >> Fred had the interesting suggestion of removing pydoc in Py3K based on >> the thinking that documentation tools like pydoc should be external to >> Python. With the docs now so easy to generate directly, should pydoc >> perhaps just be gutted to only what is needed for help() to work? > > pydoc is fine for displaying docstring help, and interactive help. > This should stay. > > Of course, it would also be nice for ``help("if")`` to work effortlessly, > which it currently only does if the generated HTML documentation is > available somewhere, which it typically isn't -- on Unix most distributions > put it in a separate package (from which pydoc won't always find it > of its own), on Windows only the CHM file is distributed and must be > decompiled to get single HTML files. > > Now that the docs are reST, the source is almost pretty enough to display > it raw, but I could also imagine a "text" writer that removes the more > obscure markup to present a casual-reader-friendly text version. > > The needed sources could then be distributed with Python -- it shouldn't > be more than about 200 kb. The versioned documentation will sometimes be available from the Internet if you want to think about using that as a fallback source. It *would* be nice if help("if") worked. It would be even handier if help(if) worked, but that's a syntax problem, and it would be a horrendous one to overcome, I suspect. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Sat May 31 17:42:24 2008 From: steve at holdenweb.com (Steve Holden) Date: Sat, 31 May 2008 11:42:24 -0400 Subject: [Python-Dev] Finishing up PEP 3108 In-Reply-To: References: Message-ID: <484171E0.4050204@holdenweb.com> Georg Brandl wrote: > Brett Cannon schrieb: > >>>> Issue 2873 - htmllib is slated to go, but pydoc still uses it. Then >>>> again, pydoc is busted thanks to the new doc format. >>> >>> I will try to handle this in the coming week. >>> >> >> Fred had the interesting suggestion of removing pydoc in Py3K based on >> the thinking that documentation tools like pydoc should be external to >> Python. With the docs now so easy to generate directly, should pydoc >> perhaps just be gutted to only what is needed for help() to work? > > pydoc is fine for displaying docstring help, and interactive help. > This should stay. > > Of course, it would also be nice for ``help("if")`` to work effortlessly, > which it currently only does if the generated HTML documentation is > available somewhere, which it typically isn't -- on Unix most distributions > put it in a separate package (from which pydoc won't always find it > of its own), on Windows only the CHM file is distributed and must be > decompiled to get single HTML files. > > Now that the docs are reST, the source is almost pretty enough to display > it raw, but I could also imagine a "text" writer that removes the more > obscure markup to present a casual-reader-friendly text version. > > The needed sources could then be distributed with Python -- it shouldn't > be more than about 200 kb. The versioned documentation will sometimes be available from the Internet if you want to think about using that as a fallback source. It *would* be nice if help("if") worked. It would be even handier if help(if) worked, but that's a syntax problem, and it would be a horrendous one to overcome, I suspect. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From dickinsm at gmail.com Sat May 31 23:52:10 2008 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 31 May 2008 17:52:10 -0400 Subject: [Python-Dev] [Python-3000] Finishing up PEP 3108 In-Reply-To: References: Message-ID: <5c6f2a5d0805311452v15a87d06g899fa8182dbd9d2a@mail.gmail.com> On Sat, May 31, 2008 at 11:33 AM, Georg Brandl wrote: > > Now that the docs are reST, the source is almost pretty enough to display > it raw, but I could also imagine a "text" writer that removes the more > obscure markup to present a casual-reader-friendly text version. > > The needed sources could then be distributed with Python -- it shouldn't > be more than about 200 kb. > +1 from me. Would this mean that htmllib and sgmllib could be removed without further ado. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: