From nicko at nicko.org Fri Feb 1 16:26:10 2008 From: nicko at nicko.org (Nicko van Someren) Date: Fri, 1 Feb 2008 15:26:10 +0000 Subject: [Python-3000] Set literal In-Reply-To: <47A10593.7080805@canterbury.ac.nz> References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> Message-ID: On 30 Jan 2008, at 23:17, Greg Ewing wrote: > Nicko van Someren wrote: >> Personally I'd like set comprehensions to give me mutable sets, >> and so I feel set literals should do the same. > > Do you really want set comprehensions at all, given that > set(genexp) exists? Do we really want set literals at all, given that set(...) exists? If we are going to have one then, it seems to make sense to have both. If we are going to have both, I would rather that they generate the same type of set. I appreciate that set literals can get optimised at compile time whereas comprehensions can not be optimised in the same way, and in the case of expressions in round brackets we get different types when we enclose lists vs. genexps that way, but I would rather we kept these things as consistent as possible. Nicko From python at rcn.com Sat Feb 2 00:56:38 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 1 Feb 2008 18:56:38 -0500 (EST) Subject: [Python-3000] Comparing to None Message-ID: <20080201185638.AGE12210@ms19.lnh.mail.rcn.net> It was very convenient in Py2.x to have None comparable to other types and always rank less than any other type. I would like to restore that ability. One example, it helps with heaps to be able to turn an entry into None and have it sift to the top where the null entries can be popped-off. Raymond From greg.ewing at canterbury.ac.nz Sat Feb 2 01:07:26 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 02 Feb 2008 13:07:26 +1300 Subject: [Python-3000] Set literal In-Reply-To: References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> Message-ID: <47A3B43E.1060704@canterbury.ac.nz> Nicko van Someren wrote: > Do we really want set literals at all, given that set(...) exists? > > If we are going to have one then, it seems to make sense to have both. Not necessarily. We don't have dict comprehensions, and I don't remember there being many complaints about that. Also, if generator expressions had come first, we would probably never have gotten list comprehensions. -- Greg From guido at python.org Sat Feb 2 01:44:17 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Feb 2008 16:44:17 -0800 Subject: [Python-3000] Comparing to None In-Reply-To: <20080201185638.AGE12210@ms19.lnh.mail.rcn.net> References: <20080201185638.AGE12210@ms19.lnh.mail.rcn.net> Message-ID: You should have been there when this was decided about two years ago. I really don't want to be tweaking the language spec continuously between now and the planned release date in ~August. On Feb 1, 2008 3:56 PM, Raymond Hettinger wrote: > It was very convenient in Py2.x to have None comparable to other types and always rank less than any other type. > > I would like to restore that ability. > > One example, it helps with heaps to be able to turn an entry into None and have it sift to the top where the null entries can be popped-off. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Feb 2 01:45:33 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Feb 2008 16:45:33 -0800 Subject: [Python-3000] Set literal In-Reply-To: <47A3B43E.1060704@canterbury.ac.nz> References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A3B43E.1060704@canterbury.ac.nz> Message-ID: On Feb 1, 2008 4:07 PM, Greg Ewing wrote: > Not necessarily. We don't have dict comprehensions, and We don't? bash-3.2$ ./python.exe Python 3.0a2+ (py3k:60207, Jan 22 2008, 16:58:59) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> {x:x**2 for x in range(10)} {0: 0, 1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64, 9: 81} >>> Where have you been? :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sat Feb 2 02:18:23 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 1 Feb 2008 20:18:23 -0500 (EST) Subject: [Python-3000] Comparing to None Message-ID: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net> > You should have been there when this was decided about two years ago. IIRC, the decision was a general one about cross-type comparisons not being turned-on the default. I do not recall a specific discussion about None. Also, the list at the time was flooded with propositions ranging from the reasonable to the insane. It was not always possible to know what to respond to or the implications of each choice. I sure wasn't aware that those conversations were to be immediately frozen in stone. I had thought one of the purposes of the Py3.0 was so that we could download it and explore the implications of all of these choices. I've done so and bumped into the None comparability issue several times. If you no longer want feedback, that's fine. I can bring my experimentation with the 3.0 alpha to a close. Raymond From guido at python.org Sat Feb 2 02:44:38 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Feb 2008 17:44:38 -0800 Subject: [Python-3000] Comparing to None In-Reply-To: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net> References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net> Message-ID: On Feb 1, 2008 5:18 PM, Raymond Hettinger wrote: > > You should have been there when this was decided about two years ago. > > IIRC, the decision was a general one about cross-type > comparisons not being turned-on the default. I do not > recall a specific discussion about None. I vaguely recall it was discussed but not considered an important enough use case. > Also, the list at the time was flooded with propositions > ranging from the reasonable to the insane. It was not always > possible to know what to respond to or the implications of each > choice. I sure wasn't aware that those conversations were to be > immediately frozen in stone. Not immediately. But by now we've had two alpha release already. Some stuff *does* change between alphas. But in general we need to stem the flow of change proposals. > I had thought one of the purposes > of the Py3.0 was so that we could download it and explore the > implications of all of these choices. I've done so and bumped > into the None comparability issue several times. I think you need to provide a lot more motivation than a 5-line email to change something this fundamental at this point. > If you no > longer want feedback, that's fine. I can bring my > experimentation with the 3.0 alpha to a close. To be brutally honest, your recent attempts to open up various closed issues have wasted a lot of cycles everywhere. I'd rather focus on moving forward with things like UserDict. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From stephen at xemacs.org Sat Feb 2 03:16:16 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 02 Feb 2008 11:16:16 +0900 Subject: [Python-3000] Set literal In-Reply-To: <47A3B43E.1060704@canterbury.ac.nz> References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A3B43E.1060704@canterbury.ac.nz> Message-ID: <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Also, if generator expressions had come first, we > would probably never have gotten list comprehensions. I don't understand. A list comprehension is for when you know you're going to want to work with the whole list. Similarly (even more so) for dictionary comprehensions. List comprehension can be defined in terms of a genexp, of course, but ISTM it's a worthwhile addition to the language in its own right. From talin at acm.org Sat Feb 2 03:04:18 2008 From: talin at acm.org (Talin) Date: Fri, 01 Feb 2008 18:04:18 -0800 Subject: [Python-3000] Comparing to None In-Reply-To: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net> References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net> Message-ID: <47A3CFA2.4000704@acm.org> I would like to present an argument both for and against the proposal. Against: I've never personally needed this capability. For: In many languages, the 'null' value has a special polymorphic behavior in that it is considered a subtype of all reference types. Thus in C, you can assign NULL to a Foo * or a Bar * even though Foo and Bar aren't related. So making NULL special seems in line with the general trend. Raymond Hettinger wrote: >> You should have been there when this was decided about two years ago. > > IIRC, the decision was a general one about cross-type > comparisons not being turned-on the default. I do not > recall a specific discussion about None. > > Also, the list at the time was flooded with propositions > ranging from the reasonable to the insane. It was not always > possible to know what to respond to or the implications of each > choice. I sure wasn't aware that those conversations were to be > immediately frozen in stone. I had thought one of the purposes > of the Py3.0 was so that we could download it and explore the > implications of all of these choices. I've done so and bumped > into the None comparability issue several times. If you no > longer want feedback, that's fine. I can bring my > experimentation with the 3.0 alpha to a close. > > Raymond > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org > From guido at python.org Sat Feb 2 06:21:34 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Feb 2008 21:21:34 -0800 Subject: [Python-3000] Comparing to None In-Reply-To: <47A3CFA2.4000704@acm.org> References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net> <47A3CFA2.4000704@acm.org> Message-ID: On Feb 1, 2008 6:04 PM, Talin wrote: > I would like to present an argument both for and against the proposal. > > Against: I've never personally needed this capability. > > For: In many languages, the 'null' value has a special polymorphic > behavior in that it is considered a subtype of all reference types. Thus > in C, you can assign NULL to a Foo * or a Bar * even though Foo and Bar > aren't related. > > So making NULL special seems in line with the general trend. Hardly. In all languages I know of besides Python, doing *anything* with such a NULL operand apart from passing it around or doing a pointer equality comparison on it immediately causes a segfault or a NullPointerException. In Python, you can already assign None to any variable and do a pointer comparison ('is'); you can even do a value equality ('==') without being punished, which is already better than most languages. Raymond is asking for *ordering* ('<', '<=' etc.) to be well-defined when one of the arguments is None. Python 2 indeed did that, because in Python 2 *all* values are orderable (well, *almost* all -- comparing str to unicode can sometimes raise an exception, complex numbers can't be ordered, and there are a few other odd exceptions). Python 3 does away with comparisons between heterogeneous types, unless at least one of the types take special measures to be comparable to the other -- this makes much more sense than the arbitrary decisions in Python 2 about the relative ordering of e.g. () and [], or 0 and "". (Ironically, Raymond's pet type, set(), cannot be compared to None even in Python 2.) As long as you're comparing real numbers, negative infinity can fulfill the role that Raymond is asking for. I suspect that for all collection types, the "empty" value of that type is always naturally the smallest value. If you really want to sort lists of completely heterogeneous types, you can always use (tag, value) pairs where the tags are small ints or strings representing type classifications (I guess the type name would do in most cases). I expect that, given the general trend that different types are not comparable, making an exception for None is more likely a source of errors than a source of joy. --Guido > Raymond Hettinger wrote: > >> You should have been there when this was decided about two years ago. > > > > IIRC, the decision was a general one about cross-type > > comparisons not being turned-on the default. I do not > > recall a specific discussion about None. > > > > Also, the list at the time was flooded with propositions > > ranging from the reasonable to the insane. It was not always > > possible to know what to respond to or the implications of each > > choice. I sure wasn't aware that those conversations were to be > > immediately frozen in stone. I had thought one of the purposes > > of the Py3.0 was so that we could download it and explore the > > implications of all of these choices. I've done so and bumped > > into the None comparability issue several times. If you no > > longer want feedback, that's fine. I can bring my > > experimentation with the 3.0 alpha to a close. > > > > Raymond > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sat Feb 2 06:29:05 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 1 Feb 2008 21:29:05 -0800 Subject: [Python-3000] Comparing to None References: <20080201201823.AGE21292@ms19.lnh.mail.rcn.net> <47A3CFA2.4000704@acm.org> Message-ID: <004201c8655c$87d6d860$6800a8c0@RaymondLaptop1> [GvR] > I expect that, given the general trend that different types are not > comparable, making an exception for None is more likely a source of > errors than a source of joy. That is reasonable. Raymond From guido at python.org Sat Feb 2 12:48:57 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 2 Feb 2008 03:48:57 -0800 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <994938.11662.qm@web31111.mail.mud.yahoo.com> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: This should be brought up on the python-3000 list; I'm moving it there using a Bcc to python-ideas. To some extent it is up to the vendors who distribute binaries -- they decide what to call it. Perhaps we should only install "python3.0" and not "python". That is a valid choice already and always has been (python2.1, python2.2, etc. are always installed by default, "python" is just a convenient alias). I think that worries about Python becoming the laughingstock of the language world are highly exaggerated. The post you refer to sounds to me like the typical cynical one-liner from someone who doesn't really care, not about a position of someone who is influential in the world of language users. --Guido PS. Java is the VB of this decare. On Feb 1, 2008 10:41 PM, Ralf W. Grosse-Kunstleve wrote: > This is regarding the transition to Python 3, and the confusion caused > by using the same executable name for two incompatible interpreters. > The consequences of this struck me today when I saw this message > appear on an important bulletin board for the field I'm working in: > > http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0802&L=ccp4bb&T=0&F=&S=&P=1562 > > This posting is from a heavy-weight in the field (lookup his name with > Google or better The Web of Science). Basically he's suggesting to forget > about Python because it is an unreliable environment. > > There is also this reply calling the original posting "alarmist": > > http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0802&L=ccp4bb&T=0&F=&S=&P=1464 > > And this one making fun of the whole thing: > > http://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0802&L=ccp4bb&T=0&F=&S=&P=1675 > > I interpret these messages as early signs of an imminent decade of confusion. > Therefore my plea: > > PLEASE GIVE THE PYTHON 3 EXECUTABLE A DIFFERENT NAME AND THE SCRIPTS > A DIFFERENT EXTENSION. > > It will be a terrible political setback otherwise. Many people don't > *want* to understand even if they could. Mixing up two incompatible > interpreters under one name will give them plenty of ammunition for > cheap jokes and for defending their not-invented-here-or-before-I-was-16 > attitude. > > Ralf > > P.S.: My wife is from Hungary. It seems all men in Hungary have to > have one of exactly three names: Andras, Laszlo, or Balazs. I never > know whom she's talking about... > > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Sat Feb 2 14:42:44 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 02 Feb 2008 14:42:44 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: Guido van Rossum schrieb: > This should be brought up on the python-3000 list; I'm moving it there > using a Bcc to python-ideas. > > To some extent it is up to the vendors who distribute binaries -- they > decide what to call it. > > Perhaps we should only install "python3.0" and not "python". That is a > valid choice already and always has been (python2.1, python2.2, etc. > are always installed by default, "python" is just a convenient alias). > > I think that worries about Python becoming the laughingstock of the > language world are highly exaggerated. The post you refer to sounds to > me like the typical cynical one-liner from someone who doesn't really > care, not about a position of someone who is influential in the world > of language users. I would prefer a "python3" alias analogous to the current "python" alias, at least as long as Python 2.x is still in wide use. Having to type "python3.0" explicitly strikes me as unnecessarily verbose. As for module file name extensions, what speaks against allowing (not mandating) "py3" for Python 3.x modules? > PS. Java is the VB of this decare. Had a close encounter with Java recently? :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From skip at pobox.com Sat Feb 2 15:55:10 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 2 Feb 2008 08:55:10 -0600 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: <18340.33870.160673.135647@montanaro-dyndns-org.local> Georg> I would prefer a "python3" alias analogous to the current Georg> "python" alias, at least as long as Python 2.x is still in wide Georg> use. So, "make install" would install a python3.0 executable and make python3 a symlink. Makes sense to me. "make bdfl-install" could also create a symlink to python3 called "python". ;-) Anybody who installed multiple versions of Python on their computer today - at least in the Unixoid world - already has to know about "make altinstall" so they don't accidentally obliterate the python symlink. I don't understand what all the fuss is. Skip From guido at python.org Sat Feb 2 17:08:46 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 2 Feb 2008 08:08:46 -0800 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <18340.33870.160673.135647@montanaro-dyndns-org.local> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <18340.33870.160673.135647@montanaro-dyndns-org.local> Message-ID: On Feb 2, 2008 6:55 AM, wrote: > > Georg> I would prefer a "python3" alias analogous to the current > Georg> "python" alias, at least as long as Python 2.x is still in wide > Georg> use. > > So, "make install" would install a python3.0 executable and make python3 a > symlink. Makes sense to me. "make bdfl-install" could also create a symlink > to python3 called "python". ;-) > > Anybody who installed multiple versions of Python on their computer today - > at least in the Unixoid world - already has to know about "make altinstall" > so they don't accidentally obliterate the python symlink. I don't > understand what all the fuss is. Me neither. If you don't want to type "python3.0" why not create a shell alias? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Sat Feb 2 17:31:41 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 02 Feb 2008 17:31:41 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: Georg Brandl wrote: > I would prefer a "python3" alias analogous to the current "python" alias, > at least as long as Python 2.x is still in wide use. > > Having to type "python3.0" explicitly strikes me as unnecessarily verbose. +1 > As for module file name extensions, what speaks against allowing (not > mandating) "py3" for Python 3.x modules? File extensions are only important on Windows. Or does anybody know a system which starts scripts based on the extension like Windows does? I see two ways to solve the program for Windows: * register additional file extensions for each Python version like .py26, .pyw26 (for pythonw.exe), .py30 etc. * Write a wrapper for Windows which inspects the first line of a script, searches for "#! ... python2.6" or "#!python26" and starts the right interpreter. Christian From fumanchu at aminus.org Sat Feb 2 18:00:31 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Sat, 2 Feb 2008 09:00:31 -0800 Subject: [Python-3000] [Python-ideas] Namespaces are one honking greatidea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com><18340.33870.160673.135647@montanaro-dyndns-org.local> Message-ID: Guido van Rossum wrote: > On Feb 2, 2008 6:55 AM, wrote: > > > > Georg> I would prefer a "python3" alias analogous to the current > > Georg> "python" alias, at least as long as Python 2.x is still in > > Georg> wide use. > > > > So, "make install" would install a python3.0 executable and make > > python3 a symlink. Makes sense to me. "make bdfl-install" could > > also create a symlink to python3 called "python". ;-) > > > > Anybody who installed multiple versions of Python on their computer > > today - at least in the Unixoid world - already has to know about > > "make altinstall" so they don't accidentally obliterate the python > > symlink. I don't understand what all the fuss is. > > Me neither. If you don't want to type "python3.0" why not create a > shell alias? We just went through this with CherryPy 3, and I wish we had used "cherrypy3" as the package name. Granted, that's a library and not an executable, so the biggest benefit (fewer import conflicts with a new name) doesn't really apply to Python. But there are other benefits to "python3". You can read code examples and work with users much more easily, saving a lot of emails in the cycle: "what version are you using?"; "dunno--how do I find out?". We're looking at several years of people mistakenly trying to run 2.x code on 3.x and vice-versa. That doesn't vanish completely; there will still be minor releases, but those are supposed to be more backward-compatible. I'd like to hear from e.g. the sqlite3 folks about similar experiences. Robert Brewer fumanchu at aminus.org From g.brandl at gmx.net Sat Feb 2 18:52:11 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 02 Feb 2008 18:52:11 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: Christian Heimes schrieb: > Georg Brandl wrote: >> I would prefer a "python3" alias analogous to the current "python" alias, >> at least as long as Python 2.x is still in wide use. >> >> Having to type "python3.0" explicitly strikes me as unnecessarily verbose. > > +1 > >> As for module file name extensions, what speaks against allowing (not >> mandating) "py3" for Python 3.x modules? > > File extensions are only important on Windows. Or does anybody know a > system which starts scripts based on the extension like Windows does? No, but Python looks for modules with the extension ".py" and not ".py3". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From amcnabb at mcnabbs.org Sat Feb 2 18:36:38 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Sat, 2 Feb 2008 10:36:38 -0700 Subject: [Python-3000] Set literal In-Reply-To: <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> References: <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A3B43E.1060704@canterbury.ac.nz> <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080202173638.GH2248@mcnabbs.org> On Sat, Feb 02, 2008 at 11:16:16AM +0900, Stephen J. Turnbull wrote: > Greg Ewing writes: > > > Also, if generator expressions had come first, we > > would probably never have gotten list comprehensions. > > I don't understand. A list comprehension is for when you know you're > going to want to work with the whole list. Similarly (even more so) > for dictionary comprehensions. List comprehension can be defined in > terms of a genexp, of course, but ISTM it's a worthwhile addition to > the language in its own right. I think Greg was just pointing out that with generator expressions, you can write: >>> lst = list(x**2 for x in xrange(10)) >>> which would make it less necessary to the equivalent syntax: >>> lst = [x**2 for x in xrange(10)] >>> -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20080202/90433ca9/attachment.pgp From mwm at mired.org Sat Feb 2 17:46:57 2008 From: mwm at mired.org (Mike Meyer) Date: Sat, 2 Feb 2008 11:46:57 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <18340.33870.160673.135647@montanaro-dyndns-org.local> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <18340.33870.160673.135647@montanaro-dyndns-org.local> Message-ID: <20080202114657.60185e56@bhuda.mired.org> On Sat, 2 Feb 2008 08:55:10 -0600 skip at pobox.com wrote: > Anybody who installed multiple versions of Python on their computer today - > at least in the Unixoid world - already has to know about "make altinstall" > so they don't accidentally obliterate the python symlink. I don't > understand what all the fuss is. And - at least in my experience - chances are good that nothing but the simplest scripts will work on multiple versions anyway, because they don't have the same set of third party - or possibly even builtin - modules installed, unless someone took care to make sure that exactly that happened, and even then the scripts generally only moved one direction. And it's not like backwards compatibility problems are unheard of, either with Python and pretty much every other language community. I think the fuss is because the Python community has generally taken more care with this than other languages, and in this case we're explicitly ignoring it. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From stephen at xemacs.org Sat Feb 2 22:04:30 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 03 Feb 2008 06:04:30 +0900 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> Christian Heimes writes: > > Having to type "python3.0" explicitly strikes me as unnecessarily verbose. > > +1 It won't once python 3.1 is released. > > As for module file name extensions, what speaks against allowing (not > > mandating) "py3" for Python 3.x modules? Namespace pollution, of various kinds. Python 3.x modules will already be placed in python3.x/site-packages anyway. I know that in Emacs we have a registry for which mode should be used by default for files with names matching a given regexp. The initial registry basically stabilized a few years back, because new applications were bumping into extensions (the regexps are 95% of the form "\\.abc$") previously claimed. I think this is precisely analogous to recognizing which interpreter should be used by extension, as well as being a practical case where you're likely to get immediate pushback. > File extensions are only important on Windows. Or does anybody know a > system which starts scripts based on the extension like Windows does? Mac OS X's open command does something according to extension: chibi:tmp steve$ echo 'print "hello, world"' >> hello.py chibi:tmp steve$ python hello.py hello, world chibi:tmp steve$ open hello.py chibi:tmp steve$ I don't know what, though. The disk spun, I waited a second, and then the prompt returned. :-) > I see two ways to solve the program for Windows: > > * register additional file extensions for each Python version > like .py26, .pyw26 (for pythonw.exe), .py30 etc. FWIW, I can tell you that Skip will get pushback from me if he proposes registering those extensions in XEmacs's auto-mode-alist. I won't absolutely veto it, but I'll certainly suggest to the other reviewers that this is a bad idea. > * Write a wrapper for Windows which inspects the first line of > a script, searches for "#! ... python2.6" or "#!python26" and > starts the right interpreter. How about a wrapper which (like the Unix shells) starts the interpreter from the file named in the shebang if any, and provide a tool for updating shebangs conveniently (for the user) and automatically (for installers). Of course you need to make sure that this wrapper doesn't use any Python 3000 features. From lists at cheimes.de Sat Feb 2 22:15:59 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 02 Feb 2008 22:15:59 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: Georg Brandl wrote: > No, but Python looks for modules with the extension ".py" and not ".py3". Python should still use the .py, .pyc and .pyo extensions for Python 3.0. I'm proposing a different extension for entry points to programs - the Python file that starts a program and contains the if __name__ == "__main__" magic. Do you suggest to drop the py extension and use py3 for all python files? Christian From lists at cheimes.de Sat Feb 2 22:33:07 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 02 Feb 2008 22:33:07 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47A4E193.6090503@cheimes.de> Stephen J. Turnbull wrote: > Mac OS X's open command does something according to extension: > > chibi:tmp steve$ echo 'print "hello, world"' >> hello.py > chibi:tmp steve$ python hello.py > hello, world > chibi:tmp steve$ open hello.py > chibi:tmp steve$ > > I don't know what, though. The disk spun, I waited a second, and then > the prompt returned. :-) I don't know how Mac OS X's open wrapper works. It should open the same program as finder does when you clock on a file. > How about a wrapper which (like the Unix shells) starts the > interpreter from the file named in the shebang if any, and provide a > tool for updating shebangs conveniently (for the user) and > automatically (for installers). > > Of course you need to make sure that this wrapper doesn't use any > Python 3000 features. I'm not interested in writing a replacement for the shebang magic. It may lead to more problems than it's worth the additional work. The wrapper must be implemented in C. Christian From adlaiff6 at gmail.com Sat Feb 2 22:33:30 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sat, 2 Feb 2008 16:33:30 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: On Feb 2, 2008 4:15 PM, Christian Heimes wrote: > Python should still use the .py, .pyc and .pyo extensions for Python > 3.0. I'm proposing a different extension for entry points to programs - > the Python file that starts a program and contains the if __name__ == > "__main__" magic. -1 It doesn't make any sense to me why we would want a different extension just because some code is the "part to run first". Isn't it all the same code? -- Cheers, Leif From skip at pobox.com Sat Feb 2 22:52:31 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 2 Feb 2008 15:52:31 -0600 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <18340.58911.137440.123488@montanaro.dyndns.org> Stephen> FWIW, I can tell you that Skip will get pushback from me if he Stephen> proposes registering those extensions in XEmacs's Stephen> auto-mode-alist. I won't absolutely veto it, but I'll Stephen> certainly suggest to the other reviewers that this is a bad Stephen> idea. I would never ask for such a thing. "xyz.py" is fine by me for Python3 source files. Skip From greg.ewing at canterbury.ac.nz Sat Feb 2 23:10:25 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Feb 2008 11:10:25 +1300 Subject: [Python-3000] Set literal In-Reply-To: <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A3B43E.1060704@canterbury.ac.nz> <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47A4EA51.3050506@canterbury.ac.nz> Stephen J. Turnbull wrote: > Greg Ewing writes: > > > Also, if generator expressions had come first, we > > would probably never have gotten list comprehensions. > > I don't understand. A list comprehension is for when you know you're > going to want to work with the whole list. But list(genexp) gives you very nearly the same thing -- so near that I find it hard to imagine anyone would have thought to suggest a dedicated LC syntax if genexps had already been present in the language. And if anyone had, Guido would have said "-1, too many ways to do it". -- Greg From g.brandl at gmx.net Sat Feb 2 23:27:02 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 02 Feb 2008 23:27:02 +0100 Subject: [Python-3000] Set literal In-Reply-To: <47A4EA51.3050506@canterbury.ac.nz> References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A3B43E.1060704@canterbury.ac.nz> <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4EA51.3050506@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Stephen J. Turnbull wrote: >> Greg Ewing writes: >> >> > Also, if generator expressions had come first, we >> > would probably never have gotten list comprehensions. >> >> I don't understand. A list comprehension is for when you know you're >> going to want to work with the whole list. > > But list(genexp) gives you very nearly the same thing -- > so near that I find it hard to imagine anyone would have > thought to suggest a dedicated LC syntax if genexps had > already been present in the language. > > And if anyone had, Guido would have said "-1, too many > ways to do it". Hm, but wouldn't he then have removed list comprehensions by now using the time machine? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From greg.ewing at canterbury.ac.nz Sat Feb 2 23:45:38 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Feb 2008 11:45:38 +1300 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080202114657.60185e56@bhuda.mired.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <18340.33870.160673.135647@montanaro-dyndns-org.local> <20080202114657.60185e56@bhuda.mired.org> Message-ID: <47A4F292.6080707@canterbury.ac.nz> Mike Meyer wrote: > And - at least in my experience - chances are good that nothing but > the simplest scripts will work on multiple versions anyway, because > they don't have the same set of third party - or possibly even builtin > - modules installed, unless someone took care to make sure that > exactly that happened, and even then the scripts generally only moved > one direction. But at least you *can* set things up and write scripts so that the same code works across versions. It's going to be impossible, or at least very difficult, to write code that works unchanged in both 2.x and 3.x. -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 2 23:59:47 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Feb 2008 11:59:47 +1300 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47A4F5E3.7030609@canterbury.ac.nz> Stephen J. Turnbull wrote: > Mac OS X's open command does something according to extension: > > I don't know what, though. The disk spun, I waited a second, and then > the prompt returned. :-) I just tried an experiment on my system, and it appears to want to launch IDLE -- but IDLE seems to be broken for me, so it doesn't work properly. But MacOSX certainly *can* launch things based on filename extension, so it would be possible to set it up to work the way Windows does, I think. Personally I never use it that way -- I launch all my text-mode Python programs from a shell, and make application bundles of anything I want to launch from the Finder. -- Greg From greg.ewing at canterbury.ac.nz Sun Feb 3 00:02:30 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Feb 2008 12:02:30 +1300 Subject: [Python-3000] Set literal In-Reply-To: References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A3B43E.1060704@canterbury.ac.nz> <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4EA51.3050506@canterbury.ac.nz> Message-ID: <47A4F686.2050800@canterbury.ac.nz> Georg Brandl wrote: > Hm, but wouldn't he then have removed list comprehensions > by now using the time machine? I seem to remember that LCs were being seriously considered for removal in 3.x at one point. I'm not sure of all the reasons why they were kept. -- Greg From lists at cheimes.de Sun Feb 3 00:58:10 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 03 Feb 2008 00:58:10 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: Leif Walsh wrote: > It doesn't make any sense to me why we would want a different > extension just because some code is the "part to run first". Isn't it > all the same code? It makes perfectly sense for some OSes. On Windows there is already a second file extension for Python scripts called "pyw". From stephen at xemacs.org Sun Feb 3 01:55:08 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 03 Feb 2008 09:55:08 +0900 Subject: [Python-3000] Set literal In-Reply-To: <47A4EA51.3050506@canterbury.ac.nz> References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A3B43E.1060704@canterbury.ac.nz> <87d4rg9k7j.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4EA51.3050506@canterbury.ac.nz> Message-ID: <87tzkq97v7.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > But list(genexp) gives you very nearly the same thing [as a list > comprehension] -- so near that I find it hard to imagine anyone > would have thought to suggest a dedicated LC syntax if genexps had > already been present in the language. Point taken, except that I find your faith in human nature touching. *Somebody* would have suggested it! From stephen at xemacs.org Sun Feb 3 02:05:01 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 03 Feb 2008 10:05:01 +0900 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <18340.58911.137440.123488@montanaro.dyndns.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <18340.58911.137440.123488@montanaro.dyndns.org> Message-ID: <87sl0a97eq.fsf@uwakimon.sk.tsukuba.ac.jp> skip at pobox.com writes: > > Stephen> FWIW, I can tell you that Skip will get pushback from me if he > Stephen> proposes registering those extensions in XEmacs's > Stephen> auto-mode-alist. I won't absolutely veto it, but I'll > Stephen> certainly suggest to the other reviewers that this is a bad > Stephen> idea. > > I would never ask for such a thing. "xyz.py" is fine by me for Python3 > source files. You personally wouldn't. If users of Python-mode asked for it, you would pass on the requests, though, I suppose. And they will if those extensions get registered in Windows as executable. I've long thought what the Emacsen should do is learn to recognize shebangs and other content-based file magic. XEmacs *can*, I think GNU can too, but we haven't worked out how to do it in a way that interacts well with the extension recognition feature (which is what Ye Olde Guard favors). From stephen at xemacs.org Sun Feb 3 02:24:31 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 03 Feb 2008 10:24:31 +0900 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A4F1D5.5080405@canterbury.ac.nz> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> Message-ID: <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> Guido asked that this thread be moved to python-3000. Reply-To set. Greg Ewing writes: > Adam Olsen wrote: > > They also have often broken python code that depended on one or two > > small details, and we haven't needed a new extension before. > > But we haven't had such major incompatibilities > between versions before. > > At the least, I think the executable should only > be installed as "python3", and never just "python" > (unless you know what you're doing and ask for it). I don't see any need to shorten "python3.0" to "python3". Also, another exception should be if there is no other Python installed. Then installing a link at "python" should be OK. From rrr at ronadam.com Sun Feb 3 02:38:27 2008 From: rrr at ronadam.com (Ron Adam) Date: Sat, 02 Feb 2008 19:38:27 -0600 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A4E193.6090503@cheimes.de> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> Message-ID: <47A51B13.4020500@ronadam.com> Christian Heimes wrote: > Stephen J. Turnbull wrote: >> Mac OS X's open command does something according to extension: >> >> chibi:tmp steve$ echo 'print "hello, world"' >> hello.py >> chibi:tmp steve$ python hello.py >> hello, world >> chibi:tmp steve$ open hello.py >> chibi:tmp steve$ >> >> I don't know what, though. The disk spun, I waited a second, and then >> the prompt returned. :-) > > I don't know how Mac OS X's open wrapper works. It should open the same > program as finder does when you clock on a file. > >> How about a wrapper which (like the Unix shells) starts the >> interpreter from the file named in the shebang if any, and provide a >> tool for updating shebangs conveniently (for the user) and >> automatically (for installers). >> >> Of course you need to make sure that this wrapper doesn't use any >> Python 3000 features. > > I'm not interested in writing a replacement for the shebang magic. It > may lead to more problems than it's worth the additional work. The > wrapper must be implemented in C. > > Christian Instead of a shebang which depends on the shell, maybe a version specifier of some sort could be used? # -*- pyversions: 2.5, 2.6 -*- So if a python 3.x detects a too low a version, maybe it can try to restart the program with the highest installed version specified. (Or some variation of this.) Ron From adlaiff6 at gmail.com Sun Feb 3 02:52:14 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sat, 2 Feb 2008 20:52:14 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A51B13.4020500@ronadam.com> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> Message-ID: On Feb 2, 2008 8:38 PM, Ron Adam wrote: > Instead of a shebang which depends on the shell, maybe a version specifier > of some sort could be used? > > # -*- pyversions: 2.5, 2.6 -*- > > So if a python 3.x detects a too low a version, maybe it can try to restart > the program with the highest installed version specified. (Or some > variation of this.) There is already an idiom with other packages (pygtk etc.) that suggests syntax like try: python.require('3.1') except: print('Some warning about version incompatibility') exit(1) This concern seems better addressed within the language itself, rather than in the shebang (which would make the decision depend on the shell and the packaging involved). -- Cheers, Leif From skip at pobox.com Sun Feb 3 04:58:48 2008 From: skip at pobox.com (skip at pobox.com) Date: Sat, 2 Feb 2008 21:58:48 -0600 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> Message-ID: <18341.15352.788886.98828@montanaro.dyndns.org> Leif> There is already an idiom with other packages (pygtk etc.) that Leif> suggests syntax like Leif> try: Leif> python.require('3.1') Leif> except: Leif> print('Some warning about version incompatibility') Leif> exit(1) It won't work in this case. Compilation of the module might well fail because of Python2/Python3 syntax differences. Skip From greg.ewing at canterbury.ac.nz Sun Feb 3 08:26:26 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Feb 2008 20:26:26 +1300 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47A56CA2.1010701@canterbury.ac.nz> Stephen J. Turnbull wrote: > I don't see any need to shorten "python3.0" to "python3". I was thinking that "python3" would be the 3.x series equivalent of what "python" is now, i.e. installation of 3.0 would create links called "python3.0" and (unless you're doing altinstall) also "python3". > Also, another exception should be if there is no other Python > installed. Then installing a link at "python" should be OK. Until you subsequently go to install one of the 2.x versions. -- Greg From greg at krypto.org Sun Feb 3 08:47:50 2008 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 2 Feb 2008 23:47:50 -0800 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <18341.15352.788886.98828@montanaro.dyndns.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> Message-ID: <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> yep we've already been through that problem in the past when list comprehensions, generators and with were added to name a few. since python 3 code is highly unlikely to even parse with a 2.x interpreter much of the time thats a reason to consider a .py3 extension if this precedent of not caring is to be broken. -0 on using .py3 instead of .py for python 3. i think its slightly silly but i won't object. +1 on devising a way (to appear starting in 2.6 and 3.0) of marking the minimum language version required by a file near the top so that the parser can bail with a useful error message instead of one confusing to the lay person. similar to a from future import type of thing perhaps? or should python parse a #! line and magically notice a required major version number in the python binary name (evil but it would work and magically annotate a bunch of existing code as 2 vs 3)? also yes python 3 should build and install as 'python3' as the short form of its name though realistically anyone sane will write their code with #! lines specifying the actual version in the executable name and have shell aliases setup if they want an even shorter name. On 2/2/08, skip at pobox.com wrote: > > > Leif> There is already an idiom with other packages (pygtk etc.) that > Leif> suggests syntax like > > Leif> try: > Leif> python.require('3.1') > Leif> except: > Leif> print('Some warning about version incompatibility') > Leif> exit(1) > > It won't work in this case. Compilation of the module might well fail > because of Python2/Python3 syntax differences. > > Skip > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/greg%40krypto.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080202/4bbec1cf/attachment.htm From tom at vector-seven.com Sun Feb 3 09:22:08 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sun, 03 Feb 2008 19:22:08 +1100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! Message-ID: <47A579B0.60208@vector-seven.com> Leif Walsh wrote: > On Feb 2, 2008 8:38 PM, Ron Adam wrote: > >> Instead of a shebang which depends on the shell, maybe a version >> specifier >> of some sort could be used? >> >> # -*- pyversions: 2.5, 2.6 -*- >> >> So if a python 3.x detects a too low a version, maybe it can try to >> restart >> the program with the highest installed version specified. (Or some >> variation of this.) >> > > There is already an idiom with other packages (pygtk etc.) that > suggests syntax like > > try: > python.require('3.1') > except: > print('Some warning about version incompatibility') > exit(1) > > This concern seems better addressed within the language itself, rather > than in the shebang (which would make the decision depend on the shell > and the packaging involved). > > Great idea. Since we've already got everything we need in version_info, this would be trivial and could address most of the issues raised ... something like this? >>> def checkversion(maj, min, rel): ... version = sys.version_info ... if version[0] < maj or version[1] < min or version[2] < rel: ... raise Exception('Version %d.%d.%d of Python is required for this program' % (maj, min, rel)) ... >>> checkversion(2, 5, 0) >>> checkversion(3, 0, 0) Traceback (most recent call last): File "", line 1, in File "", line 4, in checkversion Exception: Version 3.0.0 of Python is required for this program >>> Cheers, T From tom at vector-seven.com Sun Feb 3 09:23:20 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sun, 03 Feb 2008 19:23:20 +1100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <18341.15352.788886.98828@montanaro.dyndns.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> Message-ID: <47A579F8.5080302@vector-seven.com> skip at pobox.com wrote: > Leif> There is already an idiom with other packages (pygtk etc.) that > Leif> suggests syntax like > > Leif> try: > Leif> python.require('3.1') > Leif> except: > Leif> print('Some warning about version incompatibility') > Leif> exit(1) > > It won't work in this case. Compilation of the module might well fail > because of Python2/Python3 syntax differences. > > Ah, of course. Scratch my last email. :) Cheers, T From tom at vector-seven.com Sun Feb 3 09:02:02 2008 From: tom at vector-seven.com (Thomas Lee) Date: Sun, 03 Feb 2008 19:02:02 +1100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> Message-ID: <47A574FA.9010308@vector-seven.com> Leif Walsh wrote: > On Feb 2, 2008 8:38 PM, Ron Adam wrote: > >> Instead of a shebang which depends on the shell, maybe a version specifier >> of some sort could be used? >> >> # -*- pyversions: 2.5, 2.6 -*- >> >> So if a python 3.x detects a too low a version, maybe it can try to restart >> the program with the highest installed version specified. (Or some >> variation of this.) >> > > There is already an idiom with other packages (pygtk etc.) that > suggests syntax like > > try: > python.require('3.1') > except: > print('Some warning about version incompatibility') > exit(1) > > This concern seems better addressed within the language itself, rather > than in the shebang (which would make the decision depend on the shell > and the packaging involved). > > Great idea. Since we've already got everything we need in version_info, this would be trivial and could address most of the issues raised ... something like this? >>> def checkversion(maj, min, rel): ... version = sys.version_info ... if version[0] < maj or version[1] < min or version[2] < rel: ... raise Exception('Version %d.%d.%d of Python is required for this program' % (maj, min, rel)) ... >>> checkversion(2, 5, 0) >>> checkversion(3, 0, 0) Traceback (most recent call last): File "", line 1, in File "", line 4, in checkversion Exception: Version 3.0.0 of Python is required for this program >>> Cheers, T From stephen at xemacs.org Sun Feb 3 10:15:33 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 03 Feb 2008 18:15:33 +0900 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A56CA2.1010701@canterbury.ac.nz> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <47A56CA2.1010701@canterbury.ac.nz> Message-ID: <87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Stephen J. Turnbull wrote: > > > I don't see any need to shorten "python3.0" to "python3". > > I was thinking that "python3" would be the 3.x series > equivalent of what "python" is now, i.e. installation > of 3.0 would create links called "python3.0" and (unless > you're doing altinstall) also "python3". Well, yes, I understood that. I just don't see a need for it. At any given time I have a preferred Python, and I alias python to that. When I need one of the others, I invoke them as "pythonX.Y". Even on a shared system, most people will be allowed to have a private ~/bin directory or shell aliases. > > Also, another exception should be if there is no other Python > > installed. Then installing a link at "python" should be OK. > > Until you subsequently go to install one of the 2.x > versions. If somebody's introduction to Python is Python 3, I don't see why they'd want to go back except for a specific app. It *is* a better language than Python 2 or Python 1. Such an app will have an appropriate shebang or wrapper script. That doesn't mean that it would be *bad* if they want to go back, that's entirely up to them. I just think it will be rare enough that they can ask somebody how to create a symlink if they need to. From larry at hastings.org Sun Feb 3 12:21:22 2008 From: larry at hastings.org (Larry Hastings) Date: Sun, 03 Feb 2008 03:21:22 -0800 Subject: [Python-3000] Set literal In-Reply-To: References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> Message-ID: <47A5A3B2.5090805@hastings.org> Nicko van Someren wrote: > Do we really want set literals at all, given that set(...) exists? > > If we are going to have one then, it seems to make sense to have > both. If we are going to have both, I would rather that they generate > the same type of set. Maybe the postings crossed in the ether, but Guido said as much on the 30th; either they'd both change or neither would change. Personally I like the idea of changing them to frozensets; a) it's easy to cast to set() if you want mutability, b) it makes the if x in { 1, 2, 5 }: # three, sir! idiom faster. Yeah, it's a bit ticklish wrt dict literals/comprehensions returning mutable types, but at least tuple() would have company in the immutable constants department. So +1 from me... for what little that is worth. When compiling a mutable type with an immutable equivalent, does Python generate create-empty-object/insert-each-item bytecodes, or does it create an immutable constant then cast it to the appropriate type? My intuition is that the latter would be faster; given the professionalism of the Python development community, I assume both approaches have been tried, and the faster one was switched to long ago? A coworker of mine got a tiny--but measurable--speedup by applying that style of optimization to PHP arrays. /larry/ p.s. or maybe the create-empty-object takes an optional parameter of "and reserve this much space inside"? That would probably even the playing field. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080203/e3d481cf/attachment.htm From janssen at parc.com Sun Feb 3 19:31:57 2008 From: janssen at parc.com (Bill Janssen) Date: Sun, 3 Feb 2008 10:31:57 PST Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A579F8.5080302@vector-seven.com> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> <47A579F8.5080302@vector-seven.com> Message-ID: <08Feb3.103157pst."58696"@synergy1.parc.xerox.com> > skip at pobox.com wrote: > > Leif> There is already an idiom with other packages (pygtk etc.) that > > Leif> suggests syntax like > > > > Leif> try: > > Leif> python.require('3.1') > > Leif> except: > > Leif> print('Some warning about version incompatibility') > > Leif> exit(1) > > > > It won't work in this case. Compilation of the module might well fail > > because of Python2/Python3 syntax differences. > > > > > Ah, of course. Scratch my last email. :) > > Cheers, > T Does that mean you'd need a new keyword, something like requireversion 3.1 Bill From adlaiff6 at gmail.com Sun Feb 3 19:45:27 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Sun, 3 Feb 2008 13:45:27 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <18341.15352.788886.98828@montanaro.dyndns.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> Message-ID: On Feb 2, 2008 10:58 PM, wrote: > > Leif> There is already an idiom with other packages (pygtk etc.) that > Leif> suggests syntax like > > Leif> try: > Leif> python.require('3.1') > Leif> except: > Leif> print('Some warning about version incompatibility') > Leif> exit(1) > > It won't work in this case. Compilation of the module might well fail > because of Python2/Python3 syntax differences. Well, then this isn't something to be implemented for python 2, it's something to consider for python 3 and greater. I see what you're saying though, and perhaps in the interim, a shebang fix will suffice. -- Cheers, Leif From skip at pobox.com Sun Feb 3 21:10:26 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 3 Feb 2008 14:10:26 -0600 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> Message-ID: <18342.8114.345326.669792@montanaro-dyndns-org.local> >> It won't work in this case. Compilation of the module might well >> fail because of Python2/Python3 syntax differences. Leif> Well, then this isn't something to be implemented for python 2, Leif> it's something to consider for python 3 and greater. It won't work there either. It's quite possible that a module written for Python 2 won't compile under Python 3. Compilation is currently the only route to execution. You'd have to come up with some other scheme. Someone suggested something akin to a "from __future__" facility or the #! trick you and others mentioned, but that would only work for Python 2.6 and above. Skip From greg.ewing at canterbury.ac.nz Sun Feb 3 21:41:35 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 04 Feb 2008 09:41:35 +1300 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A574FA.9010308@vector-seven.com> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <47A574FA.9010308@vector-seven.com> Message-ID: <47A626FF.9050008@canterbury.ac.nz> > Leif Walsh wrote: >>try: >> python.require('3.1') A run-time test isn't sufficient, because not all 2.x code will even parse in 3.x. Something syntactic is needed. -- Greg From greg.ewing at canterbury.ac.nz Sun Feb 3 21:47:27 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 04 Feb 2008 09:47:27 +1300 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <47A56CA2.1010701@canterbury.ac.nz> <87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <47A6285F.90008@canterbury.ac.nz> Stephen J. Turnbull wrote: > If somebody's introduction to Python is Python 3, I don't see why > they'd want to go back except for a specific app. A specified app seems quite a likely reason to me. And if they've gotten used to "python" taking them to python3.x, they'll get a surprise when it suddenly starts pointing to 2.x after they installed an older Python for what to them was meant to be a restricted purpose. I just think the python2 and python3 worlds should be kept separate by default, seeing as there are both forwards and backwards incompatibilities. -- Greg From jim at zope.com Sun Feb 3 22:27:56 2008 From: jim at zope.com (Jim Fulton) Date: Sun, 3 Feb 2008 16:27:56 -0500 Subject: [Python-3000] [Python-Dev] inst_persistent_id In-Reply-To: References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> Message-ID: <832CE438-9760-4DAE-95FD-34DF1ACF8871@zope.com> On Jan 24, 2008, at 9:25 PM, Alexandre Vassalotti wrote: > On Jan 24, 2008 9:47 AM, Jim Fulton wrote: >> >> On Jan 23, 2008, at 4:30 PM, Alexandre Vassalotti wrote: >>> I am not sure what you mean by "cPickle.Pickler and >>> cPickle.Unpickler >>> subclassible in the same way as the pickle classes." It is >>> possible to >>> subclass the C implementation. However, the C implementation does >>> not >>> expose pickle "private" methods, such as Pickler.save_int and >>> Unpickler.load_tuple. This is fine because these were never >>> documented >>> as part of the interface of pickle. >> >> That doesn't mean that they aren't overridden. I've overridden them >> in specialized applications. I suspect that others have as well. >> > > Well even in pickle.py, you can't override the save_* and load_* > methods (at least, directly), since they are called via a dispatch > dictionary. You have to override the dictionary too. :) >> I'm not really looking for anything that isn't there already. The >> python pickle version is more extensible that cPickle and I find that >> valuable. I don't want to make cPickle more flexible. I'd willing >> to >> trade off speed and flexibility depending on the application. I >> don't >> want to lose either in the interest of unification. > > How much of cPickle is really speed critical? cPickle is all about speed. > Personally, I think > built-in types pickling should be optimized as much as possible and > other things, such as user-defined classes and extension types > pickling, could be done in Python. The whole reduce dance should be in C, after all, it already is. Most user-defined types inherit the reduce method(s) provided by object, which is already written in C. >> Having said that, I would agree that there are certain aspects of >> their external APIs that ought to be unified. A good example is >> handling of globals. > > Could you elaborate about this, please; how the handling of globals by > cPickle is different from pickle? pickle.Unpickler has a find_class method that loads globals. This can be overridden in a subclass to provide alternate behavior. cPickle.Unpickler has a find_global attribute that can be assigned to provide alternate behavior. (Internally, a find_class C member is used for hysterical reasons. :) Jim -- Jim Fulton Zope Corporation From rrr at ronadam.com Sun Feb 3 22:09:29 2008 From: rrr at ronadam.com (Ron Adam) Date: Sun, 03 Feb 2008 15:09:29 -0600 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <18342.8114.345326.669792@montanaro-dyndns-org.local> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> <18342.8114.345326.669792@montanaro-dyndns-org.local> Message-ID: <47A62D89.6020202@ronadam.com> skip at pobox.com wrote: > >> It won't work in this case. Compilation of the module might well > >> fail because of Python2/Python3 syntax differences. > > Leif> Well, then this isn't something to be implemented for python 2, > Leif> it's something to consider for python 3 and greater. > > It won't work there either. It's quite possible that a module written for > Python 2 won't compile under Python 3. Compilation is currently the only > route to execution. You'd have to come up with some other scheme. Someone > suggested something akin to a "from __future__" facility or the #! trick you > and others mentioned, but that would only work for Python 2.6 and above. > > Skip The tokenizer.c file has the routines to detect encoding specifiers before the file is compiled. So it could be done there right after the file encoding is detected. Ron From jim at zope.com Sun Feb 3 22:37:18 2008 From: jim at zope.com (Jim Fulton) Date: Sun, 3 Feb 2008 16:37:18 -0500 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> Message-ID: <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> On Jan 25, 2008, at 12:07 AM, Guido van Rossum wrote: > On Jan 24, 2008 8:59 PM, Aahz wrote: >> On Thu, Jan 24, 2008, Jim Fulton wrote: >>> >>> I don't know what that means. I don't know that Jython or >>> IronPython >>> need to support cPickle. Honestly, I'd be happy to see a *much* >>> smaller standard library and, IMO, the standard library doesn't need >>> to include pickle or cPickle. >> >> While it's easy enough to work around, my company would be unhappy if >> pickle were removed, and so would my previous company. > > I don't know why Jim is saying this (*), I think the standard library is bloated. I'd much prefer to see a leaner standard library that really provides features that are close to the language and provide a packaging system that make it easy to install other packages as needed. I think setuptools is a pretty good start at this. > but it's not going to happen. > pickle is here to stay. cPickle is a different story; 99% of programs > shouldn't be bothered with the choice between the two. IMO, the programs the people who care about cPickle don't want to sacrifice speed to get the same flexibility as pickle. Similarly, I don't want to lose pickles flexibility in the interest of unification. > (*) I can only speculate that it's because Jim, whose ZODB is probably > pickle's most intensive user, Maybe, but I'm not sure. > would like to "own" it Not particularly. > and would like to > evolve it much faster than the nearly frozen version in the standard > library. Fortunately, that's not the case anymore. (Except maybe for settling the inst_persistent_id optimization.) > I would like to suggest that Jim can evolve pickle without > removing it from the standard library -- he could just include a fork > in the Zope package. Of course. No need to suggest it. :) As I said earlier, I have a general desire to see a smaller standard library. Jim -- Jim Fulton Zope Corporation From brett at python.org Sun Feb 3 23:33:51 2008 From: brett at python.org (Brett Cannon) Date: Sun, 3 Feb 2008 14:33:51 -0800 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> Message-ID: On Feb 3, 2008 1:37 PM, Jim Fulton wrote: > > On Jan 25, 2008, at 12:07 AM, Guido van Rossum wrote: > > > On Jan 24, 2008 8:59 PM, Aahz wrote: > >> On Thu, Jan 24, 2008, Jim Fulton wrote: > >>> > >>> I don't know what that means. I don't know that Jython or > >>> IronPython > >>> need to support cPickle. Honestly, I'd be happy to see a *much* > >>> smaller standard library and, IMO, the standard library doesn't need > >>> to include pickle or cPickle. > >> > >> While it's easy enough to work around, my company would be unhappy if > >> pickle were removed, and so would my previous company. > > > > I don't know why Jim is saying this (*), > > I think the standard library is bloated. I'd much prefer to see a > leaner standard library that really provides features that are close > to the language and provide a packaging system that make it easy to > install other packages as needed. I think setuptools is a pretty good > start at this. > But we need a solution, not a start. While the stdlib is bloated, it is being trimmed down in Py3K already. If you want to trim more then push for stuff to be removed on a module-to-module basis. But going from "batteries included" to "batteries easily downloaded" is quite a shift for Python. And as of right now I really don't see how that level of a shift can be coded, discussed, and put into Py3K before it is released. Let alone how difficult it would be to get the manpower lined up to maintain a server that is going to get hammered by module downloads (PyPI still goes down on occasion and this would just increase the strain on the server by a good amount). > > but it's not going to happen. > > pickle is here to stay. cPickle is a different story; 99% of programs > > shouldn't be bothered with the choice between the two. > > IMO, the programs the people who care about cPickle don't want to > sacrifice speed to get the same flexibility as pickle. Similarly, I > don't want to lose pickles flexibility in the interest of unification. > Flexibility is not going anywhere; the C implementation is the one that needs to change. If the speed is not fast enough then patches are welcome. But I know I am tired of maintaining two test suites, two sets of docs, explaining to users why they can't subclass stuff from cPickle, etc. Plus alternative Python implementations should not have to say "we are incompatible with the stdlib because we don't have cPickle and its quirks". -Brett From p.f.moore at gmail.com Sun Feb 3 23:55:43 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 3 Feb 2008 22:55:43 +0000 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: <79990c6b0802031454s79fffcbam14379225c9c11568@mail.gmail.com> References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> <79990c6b0802031454s79fffcbam14379225c9c11568@mail.gmail.com> Message-ID: <79990c6b0802031455i75eeabf1jfd916a9c71561a41@mail.gmail.com> Whoops, that was meant to go to the list. Sorry. On 03/02/2008, Brett Cannon wrote: > But we need a solution, not a start. While the stdlib is bloated, it > is being trimmed down in Py3K already. If you want to trim more then > push for stuff to be removed on a module-to-module basis. But going > from "batteries included" to "batteries easily downloaded" is quite a > shift for Python. Absolutely. For a nice example of a "worst case" scenario, I'm behind a corporate filewall which doesn't even let out HTTP except via the Windows APIs (Windows ISA proxy, or some such). I know there are workarounds for this, but that's not the point. Add to that no compiler available plus a restrictive corporate policy, and there's pretty much nothing that isn't in the core that would ever be "easily downloaded". I've no idea how representative I am, or what I might be representative of (:-)) but for some of us, a large core is a real benefit. Paul. From greg.ewing at canterbury.ac.nz Mon Feb 4 00:15:22 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 04 Feb 2008 12:15:22 +1300 Subject: [Python-3000] Set literal In-Reply-To: <47A5A3B2.5090805@hastings.org> References: <20080124225636.AFR34133@ms19.lnh.mail.rcn.net> <00b501c8609e$9f5f95c0$6800a8c0@RaymondLaptop1> <479D6A45.6090607@canterbury.ac.nz> <479E5B42.1010908@canterbury.ac.nz> <479E6916.3050208@canterbury.ac.nz> <53C77886-694D-409A-B3F9-713B5EF6CAAA@nicko.org> <47A10593.7080805@canterbury.ac.nz> <47A5A3B2.5090805@hastings.org> Message-ID: <47A64B0A.6050306@canterbury.ac.nz> Larry Hastings wrote: > When compiling a mutable type with an immutable equivalent, does Python > generate create-empty-object/insert-each-item bytecodes, or does it > create an immutable constant then cast it to the appropriate type? Last time I looked, for lists and dicts it creates an empty object and adds items to it one at a time, and for tuples it pushes all the items onto the stack and then uses a special create-tuple bytecode. For frozenset constructors it could use a similar technique as for tuples. If set comprehensions were to produce frozen sets, something else would need to be done, such as creating it mutable initially and freezing it after construction, or having a special bytecode that's allowed to add an item to a frozenset. > p.s. or maybe the create-empty-object takes an optional parameter of > "and reserve this much space inside"? That wouldn't work for set comprehensions, since there's no way of telling ahead of time how many items it will contain. It mightn't help much anyway, given Python's proportional reallocation strategy. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From stephen at xemacs.org Mon Feb 4 04:55:29 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 04 Feb 2008 12:55:29 +0900 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A6285F.90008@canterbury.ac.nz> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <47A56CA2.1010701@canterbury.ac.nz> <87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp> <47A6285F.90008@canterbury.ac.nz> Message-ID: <873as98jf2.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > I just think the python2 and python3 worlds should be > kept separate by default, seeing as there are both forwards > and backwards incompatibilities. I'm afraid that's exactly what will happen: the two worlds will tend to be *kept* separate despite the common ground. > A specified app seems quite a likely reason to me. My point is that they'll install that Python for use only by the app. They won't want to change their default Python. > And if they've gotten used to "python" taking them to python3.x, > they'll get a surprise when it suddenly starts pointing to 2.x > after they installed an older Python for what to them was meant to > be a restricted purpose. I wonder how many people out there will be "surprised" by sudden changes in the version of Python installed. On Mac OS X 10.4 + MacPorts, MacPorts does not install an /opt/local/bin/python at all, so that although /opt/local/bin takes precedence over /usr/bin, you get the default Mac OS X install of Python 2.3 as "python". Linux distributions (at least Debian and Gentoo) "anoint" a particular Python version as the one that infrastructure software uses and that is the default unless the user explicitly sets a different version as "python". Installing multiple versions of Python does not reset the "python" link. So it seems to me that this particular issue can easily be handled if Python source builds by default only install a "python" link if (a) there is none or (b) the newly installed version is an upgrade of the current "python". From stephen at xemacs.org Mon Feb 4 05:16:56 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 04 Feb 2008 13:16:56 +0900 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> Message-ID: <871w7t8ifb.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > On Feb 3, 2008 1:37 PM, Jim Fulton wrote: > > I think the standard library is bloated. I'd much prefer to see a > > leaner standard library that really provides features that are close > > to the language and provide a packaging system that make it easy to > > install other packages as needed. I think setuptools is a pretty good > > start at this. > > But we need a solution, not a start. While the stdlib is bloated, it > is being trimmed down in Py3K already. If you want to trim more then > push for stuff to be removed on a module-to-module basis. But going > from "batteries included" to "batteries easily downloaded" is quite a > shift for Python. Trimming the standard library does not necessarily mean "batteries not included". It could mean that instead of getting form-fitting super- high-output custom , you get a pair of AA Duracell alkalines that you can buy by the gross at CostCo. In other words, take whatever the module maintainer is recommending, run the full test suite, and if everything outside of the "official stdlib" passes its own unit tests and any random cross-module tests that happen to touch it, ship it with the main distribution. This is conceptually a substantial step down in quality control. But in practice, is it really so different from what happens now? From ncoghlan at gmail.com Mon Feb 4 10:28:38 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 04 Feb 2008 19:28:38 +1000 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> Message-ID: <47A6DAC6.3000301@gmail.com> Brett Cannon wrote: > If you want to trim more then > push for stuff to be removed on a module-to-module basis. But going > from "batteries included" to "batteries easily downloaded" is quite a > shift for Python. Not to mention it being a serious pain in the rear for those of us who have to get licenses vetted by our contracts department before we're allowed to copy open source code across to isolated networks with no direct access to stuff that is "in the cloud". A batteries-included standard library is a *huge* boon in that kind of environment - one license to vet and one installer to copy across, instead of suffering a death of a thousand cuts as we have to retrieve different bits and pieces from PyPI (or rewrite our own versions from scratch). We-don't-all-work-in-web-app-development-ly, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From qrczak at knm.org.pl Mon Feb 4 11:00:19 2008 From: qrczak at knm.org.pl (Marcin =?UTF-8?Q?=E2=80=98Qrczak=E2=80=99?= Kowalczyk) Date: Mon, 04 Feb 2008 11:00:19 +0100 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1202119219.21638.4.camel@qrnik> Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze: > I don't see any need to shorten "python3.0" to "python3". There is a need. Using #!/usr/bin/python3.0 would break as soon as python3.1 is released, while #!/usr/bin/python3 would be fine, at least in the next few years. Using #!/usr/bin/python is not fine in an environment with both Python 2 and Python 3 installed, and such an environment is reasonable if one has Python 2 programs running in production which have not been updated to Python 3 yet, while he wants to start using Python 3 for other programs. -- __("< Marcin Kowalczyk \__/ qrczak at knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/ From stephen at xemacs.org Mon Feb 4 12:22:04 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 04 Feb 2008 20:22:04 +0900 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <1202119219.21638.4.camel@qrnik> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> Message-ID: <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> Marcin ?Qrczak? Kowalczyk writes: > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze: > > > I don't see any need to shorten "python3.0" to "python3". > > There is a need. Using #!/usr/bin/python3.0 would break as soon as > python3.1 is released, while #!/usr/bin/python3 would be fine, at > least in the next few years. First of all, under current policy, installing Python 3.1 would not uninstall or overwrite Python 3.0, so nothing "breaks" when the user uses #!/usr/bin/python3.0. On the contrary, Python 3.1 itself may be buggy. I do not think is it appropriate for the Python installer to make that decision for the user by creating a link to the most recent python3.x. Second, the point here is not whether any given user has such a need. I may disagree with your reasoning above, but you apparently feel such a need, and that's good enough for me. I have no objection if you make a link python3 -> python3.0. What I see no need for is a policy statement *by Python* that there "should" be a "python3" link, or which python3.x is should point to. I don't think that Python providing a simple tool to help the user do it is a good idea, either, because most OS distributions already provide them, and they tend to be rather obnoxious about "rogue" users who do it by hand or use 3d party (including upstream) tools to do it. From mwm at mired.org Sun Feb 3 02:20:44 2008 From: mwm at mired.org (Mike Meyer) Date: Sat, 2 Feb 2008 20:20:44 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A4F292.6080707@canterbury.ac.nz> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <18340.33870.160673.135647@montanaro-dyndns-org.local> <20080202114657.60185e56@bhuda.mired.org> <47A4F292.6080707@canterbury.ac.nz> Message-ID: <20080202202044.5177a6e2@bhuda.mired.org> On Sun, 03 Feb 2008 11:45:38 +1300 Greg Ewing wrote: > Mike Meyer wrote: > > > And - at least in my experience - chances are good that nothing but > > the simplest scripts will work on multiple versions anyway, because > > they don't have the same set of third party - or possibly even builtin > > - modules installed, unless someone took care to make sure that > > exactly that happened, and even then the scripts generally only moved > > one direction. > But at least you *can* set things up and write scripts > so that the same code works across versions. Upwards, anyway. But I find that, unless one organization is maintaining all of them (meaning either your system has no stock python install, or you're using it), you wind up with multiple python installations and scripts that only work on one as a matter of course, because that's easier than trying to make real applications work on all of them. > It's going to be impossible, or at least very difficult, to write > code that works unchanged in both 2.x and 3.x. Which in practice - at least on the applications I work on - is more a matter of semantics than anything else. When we decide to switch to 3.x, we'll do exactly what we do for going to a new 2.Y, or a new version of some third party library we use: create a new python build with the libraries and tools we need, and a new application distribution to go with it, then run them both through the qa cycle until they pass, and we can put them in production. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From mwm at mired.org Sun Feb 3 19:27:51 2008 From: mwm at mired.org (Mike Meyer) Date: Sun, 3 Feb 2008 13:27:51 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> Message-ID: <20080203132751.79169d2c@bhuda.mired.org> On Sat, 2 Feb 2008 23:47:50 -0800 "Gregory P. Smith" wrote: > +1 on devising a way (to appear starting in 2.6 and 3.0) of marking the > minimum language version required by a file near the top so that the parser > can bail with a useful error message instead of one confusing to the lay > person. similar to a from future import type of thing perhaps? -1 on any such programmer-maintained marking. The key phrase is "programmer-maintained." I think the confusion caused by this being *wrong*, while less common than simply trying to use the wrong version, will be much more painful because the user will start with the assumption that they have a good enough version. If the goal is to lower user confusion, how about having the 2.X line issue better warnings if it sees 3.0 features? Just checking syntax errors against that list and tweaking the warning? > or should > python parse a #! line and magically notice a required major version number > in the python binary name (evil but it would work and magically annotate a > bunch of existing code as 2 vs 3)? -2 on using the shebang line for this. The 2.X version-specific binaries are python2.X, so to use that, you'd have to make the shebang line invoke python2.X. Right now, if you upgrade an installation from 2.X to 2.X+1, you can expect most of the scripts to work assuming you take do care to install the same module set. Requiring explicit minor versions breaks that. > also yes python 3 should build and install as 'python3' as the short form of > its name though realistically anyone sane will write their code with #! > lines specifying the actual version in the executable name and have shell > aliases setup if they want an even shorter name. +1, so long as we keep the python alias as well (maybe with the altinstall target only creating python3 if it doesn't exist). This will help with the transition from 2 to 3, and then later from 3 to 4, as we can use python3 vs. python4 in scripts where we now can't use python2, because it isn't setup by default. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From mwm at mired.org Mon Feb 4 06:54:13 2008 From: mwm at mired.org (Mike Meyer) Date: Mon, 4 Feb 2008 00:54:13 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <873as98jf2.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <47A56CA2.1010701@canterbury.ac.nz> <87hcgq8kp6.fsf@uwakimon.sk.tsukuba.ac.jp> <47A6285F.90008@canterbury.ac.nz> <873as98jf2.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080204005413.5ba02566@bhuda.mired.org> On Mon, 04 Feb 2008 12:55:29 +0900 "Stephen J. Turnbull" wrote: > Greg Ewing writes: > I wonder how many people out there will be "surprised" by sudden > changes in the version of Python installed. It's happened in the past. I've been dealing with supporting multiple incompatible versions on the same hosts for the last two years, and users running the "wrong" python was a regular occurrence until I convinced people that it really mattered. > On Mac OS X 10.4 + MacPorts, MacPorts does not install an > /opt/local/bin/python at all, so that although /opt/local/bin takes > precedence over /usr/bin, you get the default Mac OS X install of > Python 2.3 as "python". Actually, Leopard comes with 2.5.1. Kudos to jkh & his team for the upgrade. Also, /opt/local/bin should only take precedence for users; commands run via sudo and from OS X applications that didn't come from MacPorts should ignore /opt/local/bin. > Linux distributions (at least Debian and Gentoo) "anoint" a > particular Python version as the one that infrastructure software > uses and that is the default unless the user explicitly sets a > different version as "python". And Red Hat. And - since all three provide (well, I haven't checked Gentoo in a while) system tools that use their "anointed" python - any distribution that uses those tools (most notably Ubuntu). Further, those tools depend on third party modules that may only be available as part of that OS release, so replacing the system python is a seriously bad idea. > Installing multiple versions of Python does not reset the "python" > link. Depends on how you do the install. The standard python source distribution will install the python link with "make install", but installs things in by default /usr/local, so won't break the system python, or most of the system tools (some do use #!/usr/bin/env python instead of #!/usr/bin/python). If you do "make altinstall", it won't install that symlink. If you configure it to install in /usr instead of /usr/local and do "make install", the python symlink will get reset (and if you're on one of those Linux systems, your life is about to get really, really unpleasant). I haven't looked at the macports version in a while, but suspect that it used the altinstall target to avoid surprises. On the other hand, I had no problems on Tiger adding the symlink to /opt/local/bin, and making sure it was first on my path (but not roots!). > So it seems to me that this particular issue can easily be handled if > Python source builds by default only install a "python" link if (a) > there is none or (b) the newly installed version is an upgrade of the > current "python". Neither of those is really good enough. How do you decide that "there is none"? leaving it in the hands of the installers is the only sure solution. For the second part, if you upgrade the python without making sure you've upgraded *all* the third party modules in the python library, you'll break scripts that depend on the modules you didn't upgrade. Living with multiple python installs isn't really a problem. Upgrading an existing installation isn't really a problem, either. Both work fine if done with a bit of care. It's up to the person building the distribution (or doing the install) to make provide that care. But we've been doing it for a while now, and it's not a serious issue. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From ncoghlan at gmail.com Mon Feb 4 15:09:47 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 05 Feb 2008 00:09:47 +1000 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> Message-ID: <47A71CAB.8040303@gmail.com> Guido van Rossum wrote: > This should be brought up on the python-3000 list; I'm moving it there > using a Bcc to python-ideas. > > To some extent it is up to the vendors who distribute binaries -- they > decide what to call it. > > Perhaps we should only install "python3.0" and not "python". That is a > valid choice already and always has been (python2.1, python2.2, etc. > are always installed by default, "python" is just a convenient alias). > > I think that worries about Python becoming the laughingstock of the > language world are highly exaggerated. The post you refer to sounds to > me like the typical cynical one-liner from someone who doesn't really > care, not about a position of someone who is influential in the world > of language users. I personally haven't seen anything to convince me that the 2.x -> 3.0 upgrade cycle is going to be significantly worse from a deployment point of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also possible (e.g. code using 'with' or 'as' as identifiers runs just fine on 2.4 or 2.5, but that code is going to break in 2.6). Yes, the breakages in 3.0 are more significant and more widespread, but the tools to support migration and the time allowed for migration are correspondingly increased. I would also expect nearly all systems to stay with a 2.x release as the default python at least until after 3.1 comes out. The only question is what to do about applications that want to declare themselves as python 3 applications so that the platform knows how to handle them correctly, but also want to fail gracefully if someone tries to run them with the wrong version. And really, that problem already exists for 2.x applications that require a certain minimum version (e.g. if you use with statements or conditional expressions in your main module, even with a future statement, then it will fail to compile on all versions prior to 2.5). I think the general solution is the same in all such cases: the affected application or library (not the language) should include a short and simple entry point module that uses the bare minimum amount of code needed to ensure the correct runtime environment. Something like: import sys REQUIRED_VERSION = (3, 0, 0) if sys.version_info < REQUIRED_VERSION: sys.exit("Require Python version %d.%d.%d, using version %d.%d.%d" % (REQUIRED_VERSION + sys.version_info[:3])) import real_main real_main.main() For environments where the default version of Python doesn't meet the application's requirements, then it will be up to the owner of that environment to work out how to get that specific application to run with the correct version (e.g. add a python3 alias and modify the main script's shebang line or use a wrapper script or GUI shortcut that explicitly invokes the correct python version). Actually adding a python3 alias as part of the default py3k installation would bother me a little, as we'd then be stuck with providing it for the life of the 3.x series - adding cruft while trying to remove it seems like a backwards step. And for in-house development, web-based applications or embedded devices where the developers have control of the run-time environment as well as the application code, then there is absolutely no problem at all, as the system will be configured to use the right version of the interpreter no matter which version it happens to be. Cheers, Nick. P.S. Good thing we decided to keep string mod-formatting despite the addition of the format method ;) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Mon Feb 4 14:52:01 2008 From: skip at pobox.com (skip at pobox.com) Date: Mon, 4 Feb 2008 07:52:01 -0600 Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco? Message-ID: <18343.6273.636210.6919@montanaro-dyndns-org.local> This discussion of what to install Python 3.0 as reminded me that we've already encountered this problem before. As I recall, when Python 2.0 was released Red Hat steadfastly refused to ship it with their then current version of Linux. It wasn't until the next RH product came out that they shipped Python 2.x, and then the executable was called (I think) python2. I no longer remember the details very well, but I wonder if there's a lesson to be learned from that experience as we decide how to install Python 3.0. Skip From lists at cheimes.de Mon Feb 4 16:21:46 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 04 Feb 2008 16:21:46 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080203132751.79169d2c@bhuda.mired.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> <20080203132751.79169d2c@bhuda.mired.org> Message-ID: <47A72D8A.3030700@cheimes.de> Mike Meyer wrote: > -2 on using the shebang line for this. The 2.X version-specific > binaries are python2.X, so to use that, you'd have to make the shebang > line invoke python2.X. Right now, if you upgrade an installation from > 2.X to 2.X+1, you can expect most of the scripts to work assuming you > take do care to install the same module set. Requiring explicit minor > versions breaks that. I'm on Mike's side here. There is no need to fix the Unix (and Mac OS X) side of the problem because it was never broken in the first place. The only problematic environment is (as usual) Windows. Windows has no shebang and python.exe is most likely not in PATH. One has to go through the registry to find Python 2.x and its executable. For Windows the Python stdlib should get some means of starting a script with a well defined Python version without relying on the file extension. One possible way is to assign the .py and .pyw extension with a small wrapper program written in C which parses the #! shebang line, gets the right python.exe from the registry and starts the script with it. Another way is a new Python extension (maybe .pys for python start) for ini file like startup files: --- [Python] version=2.5,2.6 module=myprogram.main gui=True [Env] pythonpath=.. --- The file causes the startup program to search for 2.5 and 2.6 in the registry. It sets the env var PYTHONPATH to .. (relative to the location of the file) and starts the program with pythonw.exe -m "myprogram.main". Christian From janssen at parc.com Mon Feb 4 16:39:21 2008 From: janssen at parc.com (Bill Janssen) Date: Mon, 4 Feb 2008 07:39:21 PST Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <08Feb4.073927pst."58696"@synergy1.parc.xerox.com> > First of all, under current policy, installing Python 3.1 would not > uninstall or overwrite Python 3.0, so nothing "breaks" when the user What policy? Bill From rrr at ronadam.com Mon Feb 4 16:45:53 2008 From: rrr at ronadam.com (Ron Adam) Date: Mon, 04 Feb 2008 09:45:53 -0600 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A72D8A.3030700@cheimes.de> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> <20080203132751.79169d2c@bhuda.mired.org> <47A72D8A.3030700@cheimes.de> Message-ID: <47A73331.4050000@ronadam.com> Christian Heimes wrote: > Mike Meyer wrote: >> -2 on using the shebang line for this. The 2.X version-specific >> binaries are python2.X, so to use that, you'd have to make the shebang >> line invoke python2.X. Right now, if you upgrade an installation from >> 2.X to 2.X+1, you can expect most of the scripts to work assuming you >> take do care to install the same module set. Requiring explicit minor >> versions breaks that. > > I'm on Mike's side here. There is no need to fix the Unix (and Mac OS X) > side of the problem because it was never broken in the first place. > > The only problematic environment is (as usual) Windows. Windows has no > shebang and python.exe is most likely not in PATH. One has to go through > the registry to find Python 2.x and its executable. > > For Windows the Python stdlib should get some means of starting a script > with a well defined Python version without relying on the file > extension. One possible way is to assign the .py and .pyw extension with > a small wrapper program written in C which parses the #! shebang line, > gets the right python.exe from the registry and starts the script with it. > > Another way is a new Python extension (maybe .pys for python start) for > ini file like startup files: I like .pst better, or maybe reuse .pth files? Ron > --- > [Python] > version=2.5,2.6 > module=myprogram.main > gui=True > > [Env] > pythonpath=.. > --- > > The file causes the startup program to search for 2.5 and 2.6 in the > registry. It sets the env var PYTHONPATH to .. (relative to the location > of the file) and starts the program with pythonw.exe -m "myprogram.main". > > Christian > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/rrr%40ronadam.com > From lists at cheimes.de Mon Feb 4 16:53:42 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 04 Feb 2008 16:53:42 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A73331.4050000@ronadam.com> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> <20080203132751.79169d2c@bhuda.mired.org> <47A72D8A.3030700@cheimes.de> <47A73331.4050000@ronadam.com> Message-ID: <47A73506.60401@cheimes.de> Ron Adam wrote: > I like .pst better, or maybe reuse .pth files? MS Outlook uses the file extension .pst. Python has a history to "claim" the .py? namespace. I like to stick to the tradition. Christian From guido at python.org Mon Feb 4 17:20:15 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 4 Feb 2008 08:20:15 -0800 Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco? In-Reply-To: <18343.6273.636210.6919@montanaro-dyndns-org.local> References: <18343.6273.636210.6919@montanaro-dyndns-org.local> Message-ID: On Feb 4, 2008 5:52 AM, wrote: > This discussion of what to install Python 3.0 as reminded me that we've > already encountered this problem before. As I recall, when Python 2.0 was > released Red Hat steadfastly refused to ship it with their then current > version of Linux. It wasn't until the next RH product came out that they > shipped Python 2.x, and then the executable was called (I think) python2. > > I no longer remember the details very well, but I wonder if there's > a lesson to be learned from that experience as we decide how to install > Python 3.0. IMO the lesson to be learned is that that was a bad idea -- other vendors have not followed suit. It also gave Python a bad name because for years, typing "python" on Red Hat systems gave you an ancient version. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From charles.merriam at gmail.com Mon Feb 4 19:04:27 2008 From: charles.merriam at gmail.com (Charles Merriam) Date: Mon, 4 Feb 2008 10:04:27 -0800 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <87ve5783z5.fsf@uwakimon.sk.tsukuba.ac.jp> <47A4E193.6090503@cheimes.de> <47A51B13.4020500@ronadam.com> <18341.15352.788886.98828@montanaro.dyndns.org> <52dc1c820802022347n45d830dag6964f299a3a5617d@mail.gmail.com> Message-ID: How about the ellipses as a Python 3 marker? >From a previous discussion, the error messages on this null-op should be made better in Python 3.0 anyway. If ellipses are allowed as a stand-alone statement, much like pass, then we could just have these messages: Python 2.6: "Unexpected '...'. Ellipses not supported until Python 3.0. Another cause might be pasting in code from within the IDLE interpreter. Python 3.0: 'Unexpected '...'. Might be caused by pasting in code from within the IDLE interpreter. An idiom that might be used during the early transition period: ... # Require python 3.x print ("I'm Pythonic!") The idiom would disappear with time, not add special cruft to the language, and would give a reasonable compile-time error message. Just a thought, Charles --- Charles Merriam e: charles.merriam at gmail.com u: http://www.charlesmerriam.com q: "Crufty solutions make crufty curmudgeons." On Feb 2, 2008 11:47 PM, Gregory P. Smith wrote: > yep we've already been through that problem in the past when list > comprehensions, generators and with were added to name a few. since python > 3 code is highly unlikely to even parse with a 2.x interpreter much of the > time thats a reason to consider a .py3 extension if this precedent of not > caring is to be broken. > > -0 on using .py3 instead of .py for python 3. i think its slightly silly > but i won't object. > > +1 on devising a way (to appear starting in 2.6 and 3.0) of marking the > minimum language version required by a file near the top so that the parser > can bail with a useful error message instead of one confusing to the lay > person. similar to a from future import type of thing perhaps? or should > python parse a #! line and magically notice a required major version number > in the python binary name (evil but it would work and magically annotate a > bunch of existing code as 2 vs 3)? > > also yes python 3 should build and install as 'python3' as the short form of > its name though realistically anyone sane will write their code with #! > lines specifying the actual version in the executable name and have shell > aliases setup if they want an even shorter name. > > > > On 2/2/08, skip at pobox.com wrote: > > > > > > > > > > Leif> There is already an idiom with other packages (pygtk etc.) that > > Leif> suggests syntax like > > > > Leif> try: > > Leif> python.require('3.1') > > Leif> except: > > Leif> print('Some warning about version incompatibility') > > Leif> exit(1) > > > > It won't work in this case. Compilation of the module might well fail > > because of Python2/Python3 syntax differences. > > > > Skip From charles.merriam at gmail.com Mon Feb 4 20:24:55 2008 From: charles.merriam at gmail.com (Charles Merriam) Date: Mon, 4 Feb 2008 11:24:55 -0800 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: <47A6DAC6.3000301@gmail.com> References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> <47A6DAC6.3000301@gmail.com> Message-ID: What about a formal dependency plan instead? Python would remain 'batteries included', perhaps verging on 'kitchen sink included'. Those users working to embed Python or work in other situations where a large set of available libraries is a problem would have some help in cutting down the footprint in an particular installation. That is, one could freely delete some set of libraries and be fairly sure the resulting library set would still run. What was left would be a smaller footprint interpreted language, just not 'Python'. Under what circumstances is this a poor idea? "Hard drive space is expensive" is not an acceptable argument. Charles --------- Charles Merriam e: charles.merriam at gmail.com u: http://charlesmerriam.com q: "Toolboxes have more tools than just a hammer." On Feb 4, 2008 1:28 AM, Nick Coghlan wrote: > Brett Cannon wrote: > > If you want to trim more then > > push for stuff to be removed on a module-to-module basis. But going > > from "batteries included" to "batteries easily downloaded" is quite a > > shift for Python. > > Not to mention it being a serious pain in the rear for those of us who > have to get licenses vetted by our contracts department before we're > allowed to copy open source code across to isolated networks with no > direct access to stuff that is "in the cloud". > > A batteries-included standard library is a *huge* boon in that kind of > environment - one license to vet and one installer to copy across, > instead of suffering a death of a thousand cuts as we have to retrieve > different bits and pieces from PyPI (or rewrite our own versions from > scratch). > > We-don't-all-work-in-web-app-development-ly, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/charles.merriam%40gmail.com > From ndbecker2 at gmail.com Mon Feb 4 20:34:31 2008 From: ndbecker2 at gmail.com (Neal Becker) Date: Mon, 04 Feb 2008 14:34:31 -0500 Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco? References: <18343.6273.636210.6919@montanaro-dyndns-org.local> Message-ID: Guido van Rossum wrote: > On Feb 4, 2008 5:52 AM, wrote: >> This discussion of what to install Python 3.0 as reminded me that we've >> already encountered this problem before. As I recall, when Python 2.0 >> was released Red Hat steadfastly refused to ship it with their then >> current >> version of Linux. It wasn't until the next RH product came out that they >> shipped Python 2.x, and then the executable was called (I think) python2. >> >> I no longer remember the details very well, but I wonder if there's >> a lesson to be learned from that experience as we decide how to install >> Python 3.0. > > IMO the lesson to be learned is that that was a bad idea -- other > vendors have not followed suit. It also gave Python a bad name because > for years, typing "python" on Red Hat systems gave you an ancient > version. > That was unfortunate. What they should have done, IMO, is install python as a link to the latest version. Where they had tools that had not been ported, those tools should have explicitly said #!/usr/bin/python1.5. Would be even nicer if python could say: require python_version >= x.y or some such. From adlaiff6 at gmail.com Mon Feb 4 22:12:42 2008 From: adlaiff6 at gmail.com (Leif Walsh) Date: Mon, 4 Feb 2008 16:12:42 -0500 Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco? In-Reply-To: References: <18343.6273.636210.6919@montanaro-dyndns-org.local> Message-ID: On Feb 4, 2008 2:34 PM, Neal Becker wrote: > That was unfortunate. What they should have done, IMO, is install python as > a link to the latest version. Where they had tools that had not been > ported, those tools should have explicitly said #!/usr/bin/python1.5. > > Would be even nicer if python could say: > require python_version >= x.y > or some such. I already had this suggestion shot down in the "Namespaces are etc." thread. The argument goes a bit like "if the os calls the wrong version of python to compile and run the code, it might not even compile, in which case the 'require version whatever' line of python code will never be executed". -- Cheers, Leif From ncoghlan at gmail.com Mon Feb 4 23:01:18 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 05 Feb 2008 08:01:18 +1000 Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco? In-Reply-To: References: <18343.6273.636210.6919@montanaro-dyndns-org.local> Message-ID: <47A78B2E.70706@gmail.com> Leif Walsh wrote: > On Feb 4, 2008 2:34 PM, Neal Becker wrote: >> Would be even nicer if python could say: >> require python_version >= x.y >> or some such. > > I already had this suggestion shot down in the "Namespaces are etc." > thread. The argument goes a bit like "if the os calls the wrong > version of python to compile and run the code, it might not even > compile, in which case the 'require version whatever' line of python > code will never be executed". Not only that, but applications which wish to provide a user-friendly error message when run with an incompatible version of the interpreter can already do so by checking sys.version_info in a main module that steps around the compilation issue by only using very limited constructs (more on that in my contribution to the namespaces thread). This approach has the virtue of being able to give a nice error message with versions of the interpreter all the way back to 2.0 (when version_info was added) instead of relying on something that would at best be added in 2.6. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From stephen at xemacs.org Tue Feb 5 00:20:00 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 05 Feb 2008 08:20:00 +0900 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080204123057.1004e485@mbook-fbsd> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> Message-ID: <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> Mike Meyer writes: > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" wrote: > > Marcin ___Qrczak___ Kowalczyk writes: > > > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze: > > > > I don't see any need to shorten "python3.0" to "python3". > > > There is a need. Using #!/usr/bin/python3.0 would break as soon as > > > python3.1 is released, while #!/usr/bin/python3 would be fine, at > > > least in the next few years. > > First of all, under current policy, installing Python 3.1 would not > > uninstall or overwrite Python 3.0 > > Wrong. The recommend "make install" overwrites the "python" link. Irrelevant. That "python" link is *not involved* in the shebang quoted above, and it is not part of any Python installation, precisely because it is designed (nowadays) to point to a real installed python somewhere else. (What I mean is that even though setup.py does make one, that doesn't mean that version of Python has any ownership rights to it.) > > so nothing "breaks" when the user uses #!/usr/bin/python3.0. > > Unless, of course, the user used their systems packaging software > to upgrade the default python install, and that's been moved to the > newer version. In that case, every packaging system I've ever used > will remove the old version, Making that decision is the distro's prerogative, nay, its sacred duty. But here, we're discussing what Python should do, not what distros should do. > > On the contrary, Python 3.1 itself may be buggy. I do not think is > > it appropriate for the Python installer to make that decision for > > the user by creating a link to the most recent python3.x. > > Except it currently does, both during the install from source Then altinstall should be made the default. > *and* as installed by most packaging systems. Different issue. That *is* a packaging system's prerogative IMO. > > What I see no need for is a policy statement *by Python* that there > > "should" be a "python3" link, or which python3.x is should point to. > > So do you think we should do away with the "python" link as well? Yes. The decision to have one should be made by the user or the distro. This doesn't mean that Python shouldn't provide an install target that makes one, but it should require more typing than the default install (and thus an explicit decision). > The reason OS distributions are hard on "rogue" users who change the > meaning of "python" is because their system tools quit working when > you do that, as they all use "python". No, it's a more general policy. That is an important justification in the case of python, but the tools generally do blow up if users mess with other "virtualized" files, too, because the metadata is incorrect. > Having the python install provide the major version link as well as > the versionless "python" link means it becomes reasonable to > distribute scripts that use link, because most installs will have > it. It encourages vendors to use that link in their system-specific > scripts, so they can provide a backwards-incompatible version of > python, and make "python" invoke it for users. Both of these are good > things. Sure, both are good things. But having setup.py provide them is not. If the user and/or the distro is providing the link, setup.py should assume that they know what they are doing. "How can setup.py know that?" you ask. Yes, *that is precisely the problem.* setup.py doesn't and can't know whether the user, the distro, or a previous avatar of setup.py created that link. Odds are very good that the distro knows far more about what the user wants than the user does (ie, it knows the dependency graph in detail, and the user just wants that graph satisfied), but setup.py does not know. And should not know. That's the distro's job. From barry at python.org Tue Feb 5 00:20:28 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Feb 2008 18:20:28 -0500 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 4, 2008, at 6:20 PM, Stephen J. Turnbull wrote: >>> On the contrary, Python 3.1 itself may be buggy. I do not think is >>> it appropriate for the Python installer to make that decision for >>> the user by creating a link to the most recent python3.x. >> >> Except it currently does, both during the install from source > > Then altinstall should be made the default. I think this is the best solution. +1 - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iQCVAwUBR6edvXEjvBPtnXfVAQLaaAQApF4obEK27TTEy71pigvMnNdd4rhd5Vnp AV+kvyPGaaFY0u5Eq4Q1HZlxyoQ32FEvCKqwpFLl9OQ+Bb16qcLVbXfycAxQFONA ITUuxYPBDSeyBV8pQnggQkJXEi1ziuSiYk/KgabZi5fOV+kGWoRF+wZSdDbgMS1M M0miCaXtFog= =S7kE -----END PGP SIGNATURE----- From rwgk at yahoo.com Tue Feb 5 00:34:56 2008 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Mon, 4 Feb 2008 15:34:56 -0800 (PST) Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! Message-ID: <448576.20652.qm@web31111.mail.mud.yahoo.com> Nick Coghlan wrote: > I personally haven't seen anything to convince me that the 2.x -> 3.0 > upgrade cycle is going to be significantly worse from a deployment point > of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also > possible (e.g. code using 'with' or 'as' as identifiers runs just fine > on 2.4 or 2.5, but that code is going to break in 2.6). The 2.x -> 3.0 transition will be *very* different. We're using Python since 2000, version 1.5.2. I've always upgraded to the latest release and got it working with a few fairly minor tweaks. Importantly, I could do this without breaking backward compatibility if I wanted to. At some point we had to give up on 1.5.2 compatibility, but that was just because of Boost.Python requirements (only works with new-style classes). IIUC, there is no conservative upgrade path for Python 3.0. Once the transition is done, the code will only work with 3.0. The bridge to older Python versions is completely broken. Python 2.x will not even parse most Python 3 scripts and vice versa. That's a situation we never had before. Even though the scripts look very similar at face value, at the parsing level they are as incompatible as e.g. perl and Python. It is a given that you have to plan for a transition period that is measured in years: if you have dependencies on 3rd-party packages (e.g. wxPython) you cannot start using Python 3 until all 3rd-party packages have gone through the transition. We are using a ton of 3rd-party packages. That's what open source and object-orientation is all about. The millions of lines of sources that have accumulated over the years cannot be converted over night. My *optimistic* time horizon for our project is five years before I can realistically think of giving up on Python 2, and start *a few weeks of work* to push all our stuff over to Python 3. During that time, all serious development work will have to be postponed. My pessimistic time horizon is "forever". Python 3 has the potential to become the software version of the Itanium if the deployment isn't handled carefully. Worse, it could even reflect badly on Python 2 if the naive user just knows that "python script" usually ends in a syntax error unless he thinks twice about what machine he is using to run which script. As Python developers you have to realize that for many people Python is "just" a core around which they build much bigger applications. If the success of Python is to continue, you have to give your large user base a clear path to working with two co-existing Python versions, so that installing a Python 3 application doesn't break all Python 2 applications (some of which may never be converted since the original developers have moved on). For example, I really need to be able to use wxPython based on Python 2 and wxPython based on Python 3 simultaneously on my Windows machine. I'm thinking this is only possible if Python 3 uses a new file extension. Ideally, I'd even want to have both Python 2 and Python 3 scripts in the same directory, while incrementally converting our applications. I'd want to cp script.py script.py3, then edit the .py3 version until it works, while still being able to run the old version during the migration period. Of course, I could also start a new Python 3 specific directory structure, but then I'd have to jump back and forth between two complex trees all the time, which is very annoying and time consuming; and I may have to copy many non-Python files that I wouldn't have to maintain in two copies otherwise. I think it will be essential to give the Python 3 interpreter a new name and the scripts a new extension. I suggest using "py3" for both. Ralf From stephen at xemacs.org Tue Feb 5 01:04:47 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 05 Feb 2008 09:04:47 +0900 Subject: [Python-3000] Does anyone remember the Red Hat 1.5->2.x fiasco? In-Reply-To: References: <18343.6273.636210.6919@montanaro-dyndns-org.local> Message-ID: <87ve54w9nk.fsf@uwakimon.sk.tsukuba.ac.jp> Neal Becker writes: > Would be even nicer if python could say: require python_version >= > x.y or some such. The starting point of this discussion was that Python 3 proves this to be impossible. The app cannot be sure at write time that no Python released in the future will make the app illegal. The sensible thing for the app to say is "I require " and the sensible reply for the python executable is one of "Yo! I can do that!" or "Take this note over to X.Y, and he'll take care of it for you" or "sorry, I dunno nobody that can do that for ya." A way to implement this would be for apps to use #!/usr/bin/pythonX.Y shebangs, and for Python W.Z to install a link pythonX.Y -> pythonW.Z if (a) pythonX.Y isn't already a real file and (b) W.Z does provide the full X.Y API. Crude, but it can be done *now* *without* support from the Python language itself. It would take care of breaks in backward compatibility nicely. Eg, 3.0 simply wouldn't install any python2.X -> python3.0 links. (Yes, I know users won't retrofit, and it's even worse for the distros. But that's true of any scheme like this.) Dunno if this can be applied to Windows, unfortunately. From jimjjewett at gmail.com Tue Feb 5 01:34:21 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 4 Feb 2008 19:34:21 -0500 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> <47A6DAC6.3000301@gmail.com> Message-ID: On 2/4/08, Charles Merriam wrote: > What about a formal dependency plan instead? > Python would remain 'batteries included', perhaps verging on 'kitchen > sink included'. > Those users working to embed Python or work in other situations where > a large set of available libraries is a problem would have some help > in cutting down the footprint in an particular installation. That is, > one could freely delete some set of libraries and be fairly sure the > resulting library set would still run. What was left would be a > smaller footprint interpreted language, just not 'Python'. > Under what circumstances is this a poor idea? "Hard drive space is > expensive" is not an acceptable argument. (1) Many library modules *use* other modules, but don't really *rely* on them. Only a little functionality would be lost. That sort of roadmap would encourage the functionality to get left out entirely, and would certainly increase the amount of fallback boilerplate. (2) In bureaucratic environments, the libraries are OK because they are standard; if a more minimal distribution becomes equally standard, that will cause problems. (It wouldn't be nearly as bad as taking them out and requiring downloads, but it would still be a problem, because of the need to justify the non-minimal version.) (3) If storage footprint is important, then it is also important for deployed applications -- and it becomes awkward if you have to sprinkle your own code with fallbacks and workarounds in case the standard library was clipped. That said, it might be useful if it were kept at unofficial status, such as a wiki page. -jJ From amcnabb at mcnabbs.org Tue Feb 5 04:06:34 2008 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Mon, 4 Feb 2008 20:06:34 -0700 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <448576.20652.qm@web31111.mail.mud.yahoo.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> Message-ID: <20080205030634.GA2267@mcnabbs.org> I really like your point about how Python should give a better error than a SyntaxError when a Python 2 script is run on a Python 3 interpreter. I agree that the cause of problems should be pointed out clearly to end users. However, I disagree that adding a new file extension is necessary. On Mon, Feb 04, 2008 at 03:34:56PM -0800, Ralf W. Grosse-Kunstleve wrote: > > If the success of Python is to continue, you have to give your large > user base a clear path to working with two co-existing Python > versions, so that installing a Python 3 application doesn't break all > Python 2 applications (some of which may never be converted since the > original developers have moved on). For example, I really need to be > able to use wxPython based on Python 2 and wxPython based on Python 3 > simultaneously on my Windows machine. I'm thinking this is only > possible if Python 3 uses a new file extension. In Python 2, "import wxPython" will import wxPython based on Python 2, and on Python 3, it will import wxPython based on Python 3. There wouldn't be a Python 2 version of wxPython in the python3.0 site-packages. > Ideally, I'd even want to have both Python 2 and Python 3 scripts in > the same directory, while incrementally converting our applications. > I'd want to cp script.py script.py3, then edit the .py3 version until > it works, while still being able to run the old version during the > migration period. I kind of like the idea of doing the following: import sys major_version = sys.version_info[0] if major_version == 2: import script2 script2.main() elif major_version == 3: import script3 script3.main() It really shouldn't be too bad, I think. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20080204/e1a27d8b/attachment.pgp From foom at fuhm.net Tue Feb 5 07:15:42 2008 From: foom at fuhm.net (James Y Knight) Date: Tue, 5 Feb 2008 01:15:42 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080205030634.GA2267@mcnabbs.org> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080205030634.GA2267@mcnabbs.org> Message-ID: <7AACD2AB-5732-4EED-A2B2-60A9C5C7C3C8@fuhm.net> On Feb 4, 2008, at 10:06 PM, Andrew McNabb wrote: > In Python 2, "import wxPython" will import wxPython based on Python 2, > and on Python 3, it will import wxPython based on Python 3. There > wouldn't be a Python 2 version of wxPython in the python3.0 > site-packages. Not much fun for users of the PYTHONPATH env var though, eh? James From ncoghlan at gmail.com Tue Feb 5 16:01:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 06 Feb 2008 01:01:05 +1000 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <448576.20652.qm@web31111.mail.mud.yahoo.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> Message-ID: <47A87A31.8070208@gmail.com> Ralf W. Grosse-Kunstleve wrote: > Nick Coghlan wrote: > >> I personally haven't seen anything to convince me that the 2.x -> 3.0 >> upgrade cycle is going to be significantly worse from a deployment point >> of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also >> possible (e.g. code using 'with' or 'as' as identifiers runs just fine >> on 2.4 or 2.5, but that code is going to break in 2.6). > > The 2.x -> 3.0 transition will be *very* different. We're using Python > since 2000, version 1.5.2. I've always upgraded to the latest release > and got it working with a few fairly minor tweaks. Importantly, I > could do this without breaking backward compatibility if I wanted to. > At some point we had to give up on 1.5.2 compatibility, but that was > just because of Boost.Python requirements (only works with new-style > classes). > > IIUC, there is no conservative upgrade path for Python 3.0. Once the > transition is done, the code will only work with 3.0. The bridge to > older Python versions is completely broken. Python 2.x will not even > parse most Python 3 scripts and vice versa. That's a situation we never > had before. I still see this as a difference in degree rather than a difference in kind. As soon as a developer puts "from __future__ import absolute_imports" or "from __future__ import with_statement" in their script every version of Python prior to 2.5 is going to refuse to even compile it. If they provide an old script which uses an identifier which is now a keyword, it is the more recent versions which are going to object violently. Either way, the solution is the same: run that particular script with a version of the interpreter that can handle it correctly. Make that happen by whatever means your platform makes available (on Windows it usually means batch files, on Unix'y systems a version specific shebang line or a shell script). > Even though the scripts look very similar at face value, > at the parsing level they are as incompatible as e.g. perl and Python. While there is almost certainly going to be incompatibility at the module level, it is perfectly possible to have compatibility at the application level. Granted, obtaining it won't be anywhere near as easy as it is when staying within the 2.x series, but it is far from a lost cause. Specifically, the application may need a launcher module that fiddles with sys.path or package __path__ entries based on the running Python version. Alternatively, the application may be installed into site-packages and executed using the -m switch so that the interpreter automatically picks up the correct version of the application for the interpreter which is running. (Note again that all of these issues are only particularly difficult to deal with in the case of applications which are written in Python and intended to be used on a system someone else controls. When the same entity controls both the application and the runtime environment - such as web services, embedded systems, or applications that bundle their own Python interpreter - it is merely necessary that the environment is kept in sync as the needs of the code change) > As Python developers you have to realize that for many people Python is > "just" a core around which they build much bigger applications. Strange as it may seem, at least some of us do exactly that as part of our day jobs ;) > If the > success of Python is to continue, you have to give your large user > base a clear path to working with two co-existing Python versions, > so that installing a Python 3 application doesn't break all Python 2 > applications (some of which may never be converted since the original > developers have moved on). You mean the way you can already have Python 1.5, 2.0, 2.1, 2.2, 2.3, 2.4 and 2.5 all installed on the same machine, and invoke different versions for different applications? Yes, only one of them can be the 'default Python' for the machine, and changing that can be an interesting exercise, but that is hardly a problem that is limited to Python - it's the case for *any* scripting language interpreter. To choose an example which affected me relatively recently, how many shell scripts did the Ubuntu folks break when they decided to change their default shell from bash to dash "because it is faster" instead of explicitly invoking dash for the particular scripts they wanted to speed up? That was one configuration change I reverted pretty darn quickly, because there are a hell of a lot of shell scripts out there which rely on bash extensions, even if the developers didn't realise it when they wrote the scripts. While I'm +1 on the idea of making altinstall the default for setup.py and the Windows installer (forcing the administrator to explicitly request that the new Python version be made the default Python installation for the machine), I'm a pretty strong -1 on officially blessing the idea of a python3 alias or a special py3 extension on Windows (end users are obviously free to create those if they want, of course). Who knows, with the rate at which virtual machine technology is improving, this whole thing may become a non-issue by the time the 3.x series is even ready for production use. One VM with a 2.x default Python, another VM with a 3.x default Python, various applications associated with each, and seamless integration into the underlying platform so the user never even notices a VM is involved. (OK, I don't actually hold out much hope of that particular scenario coming to pass in the next few years, but I'm trying to point out here that you're pushing for a language level solution to a system level problem: how do we reliably associate files with an application that understands them? What do we do when different versions of that application permit the files to contain different things? Yes, these issues matter for the 2.x->3.x transition, but moving to a new interpreter name and a new file extension is nothing more than a bandaid, and a pretty ugly one that we would have to live with for a long time) > For example, I really need to be able > to use wxPython based on Python 2 and wxPython based on Python 3 > simultaneously on my Windows machine. I'm thinking this is only > possible if Python 3 uses a new file extension. And how much trouble do you have using wxPython with 2.4 and 2.5 on the same machine? They're completely different extension modules, and they can live side by side on one machine quite happily (it's one of the main benefits of installing things to site-packages). That behaviour won't change just because one of those Python version numbers now has a 3 at the start. Making heavy use of PYTHONPATH can cause problems, but again that's the result of a platform limitation, where it isn't easy to get different applications to see different values for environment variables without the use of launch scripts. > Ideally, I'd even want to have both Python 2 and Python 3 scripts in > the same directory, while incrementally converting our applications. > I'd want to cp script.py script.py3, then edit the .py3 version > until it works, while still being able to run the old version during > the migration period. Sure, you can do it that way if you really want to, but that isn't the recommended way. And invoking Python 3 for the scripts that need it would be the same as invoking any other specific Python version (batch file or shortcut on Windows, version-specific shebang line or shell script on POSIX). > Of course, I could also start a new Python 3 > specific directory structure, but then I'd have to jump back and > forth between two complex trees all the time, which is very annoying > and time consuming; and I may have to copy many non-Python files that > I wouldn't have to maintain in two copies otherwise. The intent is that you should never need to edit files directly in both trees - instead, it should be possible to work almost entirely in the Python 2 tree, with the 2to3 tool automatically creating the Python 3 tree for you (kind of like a cross-compilation step that accepts valid Python 2 code and emits the corresponding Python 3 code). Any test failures (or other problems) in the Python 3 version would be fixed by modifying the Python 2 source and reconverting. Having an option to ask the 2to3 tool to automatically copy any non-Python files it finds might be an interesting idea, although using an existing file synchronisation tool like rsync or a version control branch (and then running 2to3 to make modifications after the other files are in sync) would probably be a better one. > I think it will be essential to give the Python 3 interpreter a new > name and the scripts a new extension. I suggest using "py3" for both. And I think that would be an ugly hack that we would then be stuck with for the life of the 3.x series. Would the renamed python3 still be looking at all the same environment variables (PYTHONPATH, PYTHONHOME, etc)? Or would all those have to be renamed as well? Should import change to accept .py3 files? If not, why not (after all, the line between execution and import gets pretty blurry where the -m switch and the runpy module are concerned)? If yes, should import be changed to not accept .py files? When python3 was asked to execute a directory or zipfile, would it look for __main__.py3, __main__.py or either? Also note that while it currently will be feasible to write quite substantial programs that run on both Python 2 and Python 3 without modification (albeit only with a total avoidance of the print statement and function, as well as taking a great deal of care in remaining str/unicode agnostic), some of the possible answers to the questions posed in the previous paragraph would make it downright impossible. The mechanisms that have been used to date to support multiple Python versions on a single system have worked pretty well, and I don't believe that they're suddenly going to start failing horribly just because the permitted degree of backward incompatibility between 2.x and 3.0 is higher than that between 2.x and 2.(x+2). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From skip at pobox.com Tue Feb 5 14:37:34 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 5 Feb 2008 07:37:34 -0600 Subject: [Python-3000] PYTHONPATH in Python 3.0 (was: Namespaces are one honking ...) In-Reply-To: <7AACD2AB-5732-4EED-A2B2-60A9C5C7C3C8@fuhm.net> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080205030634.GA2267@mcnabbs.org> <7AACD2AB-5732-4EED-A2B2-60A9C5C7C3C8@fuhm.net> Message-ID: <18344.26270.730092.20238@montanaro-dyndns-org.local> James> Not much fun for users of the PYTHONPATH env var though, eh? PYTHONPATH3 anyone? Or maybe .pth files are sufficient. So far I've been able to mangle my PYTHONSTARTUP file to work with both Python 2 and Python 3. That's not very sophisticated code though. Skip From mwm at mired.org Mon Feb 4 18:30:57 2008 From: mwm at mired.org (Mike Meyer) Date: Mon, 4 Feb 2008 12:30:57 -0500 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080204123057.1004e485@mbook-fbsd> On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" wrote: > Marcin ___Qrczak___ Kowalczyk writes: > > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze: > > > I don't see any need to shorten "python3.0" to "python3". > > There is a need. Using #!/usr/bin/python3.0 would break as soon as > > python3.1 is released, while #!/usr/bin/python3 would be fine, at > > least in the next few years. > First of all, under current policy, installing Python 3.1 would not > uninstall or overwrite Python 3.0 Wrong. The recommend "make install" overwrites the "python" link. > so nothing "breaks" when the user uses #!/usr/bin/python3.0. Unless, of course, the user used their systems packaging software to upgrade the default python install, and that's been moved to the newer version. In that case, every packaging system I've ever used will remove the old version, thus breaking any scripts that were foolish enough to use that form. Which is why those packaging systems pretty much all install python scripts using "python" for the executable. > On the contrary, Python 3.1 itself may be buggy. I do not think is > it appropriate for the Python installer to make that decision for > the user by creating a link to the most recent python3.x. Except it currently does, both during the install from source *and* as installed by most packaging systems. If the packaging system does things right, this isn't really a problem - until you get to 3.0, because it breaks backwards compatibility. > What I see no need for is a policy statement *by Python* that there > "should" be a "python3" link, or which python3.x is should point to. So do you think we should do away with the "python" link as well? > I don't think that Python providing a simple tool to help the user do > it is a good idea, either, because most OS distributions already > provide them, and they tend to be rather obnoxious about "rogue" users > who do it by hand or use 3d party (including upstream) tools to do it. The reason OS distributions are hard on "rogue" users who change the meaning of "python" is because their system tools quit working when you do that, as they all use "python". So when they start shipping python3, they'll tend to make "python" invoke the old version until they convert all their scrips, as RH did with 1 and 2. Except RH still has a python2 alias, so they can easily switch their scripts to python2 if they chose to do so. Personally, I'd rather they do the latter than the former, so people will get a modern version of python when they ask for python on that OS. Most of the time, having a default for both python and python of a specific major version is pointless, because you're only going to have one major version installed, even if you have multiple minor versions installed, so they're both the same. The exception is when a new major version arrives breaking backwards compatibility. At that time, it's nice to have a default for the major versions so scripts can pick the major version if they're not picky about minor versions (and given how much work we do making sure the language is backwards compatible at that level, there are a lot of those) at the same time as users pick what they want to get when they say "python". Having the python install provide the major version link as well as the versionless "python" link means it becomes reasonable to distribute scripts that use link, because most installs will have it. It encourages vendors to use that link in their system-specific scripts, so they can provide a backwards-incompatible version of python, and make "python" invoke it for users. Both of these are good things. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From mwm at mired.org Tue Feb 5 00:38:07 2008 From: mwm at mired.org (Mike Meyer) Date: Mon, 4 Feb 2008 18:38:07 -0500 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080204183807.412b14d8@bhuda.mired.org> On Tue, 05 Feb 2008 08:20:00 +0900 "Stephen J. Turnbull" wrote: > Mike Meyer writes: > > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" wrote: > > > Marcin ___Qrczak___ Kowalczyk writes: > > > > Dnia 03-02-2008, N o godzinie 10:24 +0900, Stephen J. Turnbull pisze: > > > > > I don't see any need to shorten "python3.0" to "python3". > > > > There is a need. Using #!/usr/bin/python3.0 would break as soon as > > > > python3.1 is released, while #!/usr/bin/python3 would be fine, at > > > > least in the next few years. > > > First of all, under current policy, installing Python 3.1 would not > > > uninstall or overwrite Python 3.0 > > Wrong. The recommend "make install" overwrites the "python" link. > Irrelevant. That "python" link is *not involved* in the shebang > quoted above, and it is not part of any Python installation, precisely The python link is part of the python install on every system I've dealt with - including from the source tarballs - as they all install it, and the vast majority of software that needs python to run expect to find it there. > > > so nothing "breaks" when the user uses #!/usr/bin/python3.0. > > Unless, of course, the user used their systems packaging software > Making that decision is the distro's prerogative, nay, its sacred > duty. But here, we're discussing what Python should do, not what > distros should do. [...] > > *and* as installed by most packaging systems. > Different issue. That *is* a packaging system's prerogative IMO. Right. And the packaging system is what most people deal with in the real world. Ignoring the real world is a good way to become irrelevant. And while we can't control them (GNU/Linux packagers have a really nasty habit of installing software configured in ways the author doesn't want to support), we can at least nudge them in the right direction by making it easy to do things in what we consider to be the best way. > > > What I see no need for is a policy statement *by Python* that there > > > "should" be a "python3" link, or which python3.x is should point to. > > So do you think we should do away with the "python" link as well? > Yes. The decision to have one should be made by the user or the > distro. This doesn't mean that Python shouldn't provide an install > target that makes one, but it should require more typing than the > default install (and thus an explicit decision). So you want the default behavior after installing python to be that the "python" command not be found? So that people who see software that "needs python" will grab and install python, and then have the software not work because they it no longer installs that link by default? Or do you expect everyone who installs python software to fix the executables to refer to the version of python they have installed? Or maybe just install every version some software author was using when they wrote the software? > > Having the python install provide the major version link as well as > > the versionless "python" link means it becomes reasonable to > > distribute scripts that use link, because most installs will have > > it. It encourages vendors to use that link in their system-specific > > scripts, so they can provide a backwards-incompatible version of > > python, and make "python" invoke it for users. Both of these are good > > things. > Sure, both are good things. But having setup.py provide them is not. > If the user and/or the distro is providing the link, setup.py should > assume that they know what they are doing. If the users are using a packaging system, then what setup.py does - or does not do - is largely irrelevant. Users will get what the packagers want them to have, and there's not jack that the people actually writing the software can do about it. If the users get what setup.py provides, then they built from source, so they at least know how to install and deal with a build environment, and it's reasonable to assume that they may know a little bit more than that. We can either determine a good way to deal with the issues this raises, and nudge package builders and people doing installs from source in that direction by making those things the default, or at least easy; or we can do what was done in the 1->2 transition and let them all work it out for themselves, and risk once again having some popular distro on which "python" is obsolete for the next N years. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From mwm at mired.org Tue Feb 5 00:56:42 2008 From: mwm at mired.org (Mike Meyer) Date: Mon, 4 Feb 2008 18:56:42 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <448576.20652.qm@web31111.mail.mud.yahoo.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> Message-ID: <20080204185642.73659254@bhuda.mired.org> On Mon, 4 Feb 2008 15:34:56 -0800 (PST) "Ralf W. Grosse-Kunstleve" wrote: > As Python developers you have to realize that for many people Python is > "just" a core around which they build much bigger applications. If the > success of Python is to continue, you have to give your large user > base a clear path to working with two co-existing Python versions, > so that installing a Python 3 application doesn't break all Python 2 > applications (some of which may never be converted since the original > developers have moved on). For example, I really need to be able > to use wxPython based on Python 2 and wxPython based on Python 3 > simultaneously on my Windows machine. I'm thinking this is only > possible if Python 3 uses a new file extension. I think the key word here is "Windows". I have no trouble doing all of this on a Unix system without changing the extensions, but it's a lot less dependent on file extensions. > Ideally, I'd even want to have both Python 2 and Python 3 scripts in > the same directory, while incrementally converting our applications. > I'd want to cp script.py script.py3, then edit the .py3 version > until it works, while still being able to run the old version during > the migration period. Of course, I could also start a new Python 3 > specific directory structure, but then I'd have to jump back and > forth between two complex trees all the time, which is very annoying > and time consuming; and I may have to copy many non-Python files that > I wouldn't have to maintain in two copies otherwise. Now, let me throw in my requirements. Most modern packaging systems can handle the upgrade from 2.x to 2.y; they'll automatically upgrade all the packages that depend on python, meaning you'll get versions installed for 2.y after the upgrade process is done, and anything that uses only packages from the packaging system will keep on working. I don't want the solution for 3.0 to break the ability to do 2.x upgrades that way. Further, I want the same solution to work for 3.0 to 3.1. > I think it will be essential to give the Python 3 interpreter a new > name and the scripts a new extension. I suggest using "py3" for both. I really, really hate this idea. But if it's the only workable solution on Windows, then it's the only workable solution. Question: how many of the editor packages for editing python files will work properly on both python and python 3.0 files? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From mwm at mired.org Tue Feb 5 17:22:32 2008 From: mwm at mired.org (Mike Meyer) Date: Tue, 5 Feb 2008 11:22:32 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A87A31.8070208@gmail.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <47A87A31.8070208@gmail.com> Message-ID: <20080205112232.34bc5c80@mbook-fbsd> On Wed, 06 Feb 2008 01:01:05 +1000 Nick Coghlan wrote: > Ralf W. Grosse-Kunstleve wrote: > > Nick Coghlan wrote: > > > >> I personally haven't seen anything to convince me that the 2.x -> 3.0 > >> upgrade cycle is going to be significantly worse from a deployment point > >> of view than a 2.x -> 2.(x+2) upgrade cycle where breakages are also > >> possible (e.g. code using 'with' or 'as' as identifiers runs just fine > >> on 2.4 or 2.5, but that code is going to break in 2.6). > > > > The 2.x -> 3.0 transition will be *very* different. We're using Python > > since 2000, version 1.5.2. I've always upgraded to the latest release > > and got it working with a few fairly minor tweaks. Importantly, I > > could do this without breaking backward compatibility if I wanted to. > > At some point we had to give up on 1.5.2 compatibility, but that was > > just because of Boost.Python requirements (only works with new-style > > classes). > > > > IIUC, there is no conservative upgrade path for Python 3.0. Once the > > transition is done, the code will only work with 3.0. The bridge to > > older Python versions is completely broken. Python 2.x will not even > > parse most Python 3 scripts and vice versa. That's a situation we never > > had before. > > I still see this as a difference in degree rather than a difference in > kind. As soon as a developer puts "from __future__ import > absolute_imports" or "from __future__ import with_statement" in their > script every version of Python prior to 2.5 is going to refuse to even > compile it. If they provide an old script which uses an identifier which > is now a keyword, it is the more recent versions which are going to > object violently. The one real difference is that you can generally go from 2.X to 2.Y>X without mucking with your scripts so long as you update all the modules you use as well. In an environment managed with package systems, this is usually easy. > Either way, the solution is the same: run that particular script with a > version of the interpreter that can handle it correctly. Make that > happen by whatever means your platform makes available (on Windows it > usually means batch files, on Unix'y systems a version specific shebang > line or a shell script). Version-specific shebangs break the above property. It's possible to have multiple versions installed and have everything work without going to that. But your system vendor probably won't support it :-(. > (Note again that all of these issues are only particularly difficult to > deal with in the case of applications which are written in Python and > intended to be used on a system someone else controls. When the same > entity controls both the application and the runtime environment - such > as web services, embedded systems, or applications that bundle their own > Python interpreter - it is merely necessary that the environment is kept > in sync as the needs of the code change) Right. I live in the last world, because I've found that it's nearly impossible to get "someone else" to provide adequate python builds. > > As Python developers you have to realize that for many people Python is > > "just" a core around which they build much bigger applications. > > Strange as it may seem, at least some of us do exactly that as part of > our day jobs ;) Yup. > > If the > > success of Python is to continue, you have to give your large user > > base a clear path to working with two co-existing Python versions, > > so that installing a Python 3 application doesn't break all Python 2 > > applications (some of which may never be converted since the original > > developers have moved on). > > You mean the way you can already have Python 1.5, 2.0, 2.1, 2.2, 2.3, > 2.4 and 2.5 all installed on the same machine, and invoke different > versions for different applications? > > Yes, only one of them can be the 'default Python' for the machine, Actually, that's not *quite* true. One of the reasons you can install Python from source on most systems - even using the default install - without breaking all the tools that depend on the system's anointed version is because it install /usr/local/bin/python. A system required version almost always lives in /usr/bin, and is invoked with that path. Even systems that don't have Python in the default install generally put packages somewhere other than /usr/local (FreeBSD is the one exception I know of). For one client, we had /usr/bin/python, a 64-bit 2.3 python; /oap/bin/python, a 32-bit 2.4 python without ssl support (from the IT support group, because they wanted the same language "everywhere"), and /home//bin/python, a 64-bit 2.4 python with the ssl support we required. The scripts all invoked the python they needed by absolute path. The biggest problem we had was getting the developers environment set up so they invoked the right python at the command line for testing. > language interpreter. To choose an example which affected me relatively > recently, how many shell scripts did the Ubuntu folks break when they > decided to change their default shell from bash to dash "because it is > faster" instead of explicitly invoking dash for the particular scripts > they wanted to speed up? That was one configuration change I reverted > pretty darn quickly, because there are a hell of a lot of shell scripts > out there which rely on bash extensions, even if the developers didn't > realise it when they wrote the scripts. And those of use who use systems where bash *isn't* the default shell, and never has been, wish all those developers would get a clue, and write "#!/usr/bin/env bash" (or at least "#!/bin/bash") instead of "#!/bin/sh", and reserve the latter for scripts that really are *shell* scripts. Sorry, that's a sore spot... > While I'm +1 on the idea of making altinstall the default for setup.py > and the Windows installer (forcing the administrator to explicitly > request that the new Python version be made the default Python > installation for the machine), I'm a pretty strong -1 on officially > blessing the idea of a python3 alias or a special py3 extension on > Windows (end users are obviously free to create those if they want, of > course). I like the python3 alias (and python2 in 2.X), because it moves us from a single default python to having a default python2 and python3, which means scripts that care can ask for that without breaking the ability to upgrade python2 or python3 without breaking those scripts, while still allowing the default python to be either python2 or python3. > Who knows, with the rate at which virtual machine technology is > improving, this whole thing may become a non-issue by the time the 3.x > series is even ready for production use. One VM with a 2.x default > Python, another VM with a 3.x default Python, various applications > associated with each, and seamless integration into the underlying > platform so the user never even notices a VM is involved. That just changes the environment from one where you have to find the right python to one where you have to find the right VM. But you don't have to go to that extreme; just keeping the different pythons separated by directory, and making sure the scripts specify the right directory as part of the shebang works (at lest for Unix). http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From rhamph at gmail.com Tue Feb 5 18:43:40 2008 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 5 Feb 2008 10:43:40 -0700 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080204185642.73659254@bhuda.mired.org> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> Message-ID: On Feb 4, 2008 4:56 PM, Mike Meyer wrote: > On Mon, 4 Feb 2008 15:34:56 -0800 (PST) "Ralf W. Grosse-Kunstleve" wrote: > > As Python developers you have to realize that for many people Python is > > "just" a core around which they build much bigger applications. If the > > success of Python is to continue, you have to give your large user > > base a clear path to working with two co-existing Python versions, > > so that installing a Python 3 application doesn't break all Python 2 > > applications (some of which may never be converted since the original > > developers have moved on). For example, I really need to be able > > to use wxPython based on Python 2 and wxPython based on Python 3 > > simultaneously on my Windows machine. I'm thinking this is only > > possible if Python 3 uses a new file extension. > > I think the key word here is "Windows". I have no trouble doing all of > this on a Unix system without changing the extensions, but it's a lot > less dependent on file extensions. So why don't we add a windows equivalent of the shebang? Files could then start like this: #!/usr/bin/python2.3 #?C:/python23/python Of course, something better than ? needs to be chosen. A problem with this approach is that, if you only had 2.3 and 2.5 installed, both it'd be completely ignored. You'd need to install a newer version (2.6 or 3.0) to force the use of 2.3. -- Adam Olsen, aka Rhamphoryncus From pje at telecommunity.com Tue Feb 5 18:57:58 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 05 Feb 2008 12:57:58 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> Message-ID: <20080205175802.3FBF93A409E@sparrow.telecommunity.com> At 10:43 AM 2/5/2008 -0700, Adam Olsen wrote: >So why don't we add a windows equivalent of the shebang? Files could >then start like this: > >#!/usr/bin/python2.3 >#?C:/python23/python FYI, setuptools uses and supports #! lines on Windows, with the executable path in quotes if it contains spaces. For a given script "foo", it generates two files: foo.exe foo-script.py And the .exe is just a standardized .exe file that looks in an adjacent -script.py file of the same name for the #! line. (This is of course only for scripts generated using setuptools-specific features.) >A problem with this approach is that, if you only had 2.3 and 2.5 >installed, both it'd be completely ignored. You'd need to install a >newer version (2.6 or 3.0) to force the use of 2.3. Setuptools works with versions 2.3 through 2.5, but of course that's because of the .exe wrappers. From rhamph at gmail.com Tue Feb 5 19:13:11 2008 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 5 Feb 2008 11:13:11 -0700 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080205175802.3FBF93A409E@sparrow.telecommunity.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> Message-ID: On Feb 5, 2008 10:57 AM, Phillip J. Eby wrote: > At 10:43 AM 2/5/2008 -0700, Adam Olsen wrote: > >So why don't we add a windows equivalent of the shebang? Files could > >then start like this: > > > >#!/usr/bin/python2.3 > >#?C:/python23/python > > FYI, setuptools uses and supports #! lines on Windows, with the > executable path in quotes if it contains spaces. For a given script > "foo", it generates two files: > > foo.exe > foo-script.py > > And the .exe is just a standardized .exe file that looks in an > adjacent -script.py file of the same name for the #! line. > > (This is of course only for scripts generated using > setuptools-specific features.) > > > >A problem with this approach is that, if you only had 2.3 and 2.5 > >installed, both it'd be completely ignored. You'd need to install a > >newer version (2.6 or 3.0) to force the use of 2.3. > > Setuptools works with versions 2.3 through 2.5, but of course that's > because of the .exe wrappers. Sounds good enough to me. Maybe the only thing "wrong" is people aren't sufficiently informed of how to be version-specific? -- Adam Olsen, aka Rhamphoryncus From fumanchu at aminus.org Tue Feb 5 19:34:22 2008 From: fumanchu at aminus.org (Robert Brewer) Date: Tue, 5 Feb 2008 10:34:22 -0800 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! Message-ID: I took the liberty of asking DRH's advice based on his experiences with sqlite/2/3, and he graciously replied: > -----Original Message----- > From: drh at hwaci.com [mailto:drh at hwaci.com] > Sent: Tuesday, February 05, 2008 10:02 AM > To: Robert Brewer > Subject: Re: Python 3 needs your help > > "Robert Brewer" wrote: > > Hello, > > > > There has been some discussion [1] lately on whether to rename the > > "python" interpreter to "python3" for Python 3.0. I immediately > > thought of "sqlite3" of course, and wondered if you had any advice > > for the Python developers about the benefits or hardships of your > > naming choices for sqlite. If you had just a couple minutes to add > > your experience to the discussion, it'd be a big help. > > The primary reason from going from "sqlite" to "sqlite3" was to > allow the same application to link against both libraries at the > same time. This was important during the transition period since > some programs needed to be able to read an SQLite v2 database then > write the content over to an SQLite v3 database as part of the > upgrade process. Even today, three years into version 3, there > still exist important programs (ex: PHP) which link against both > libraries by default. > > I do not recall encountering any problems with the "sqlite3" name. > Nobody has complained. And we have not had any negative experiences. > Everything has worked out well. > > If you are referring to the name of the SQLite CLI program as opposed > to the library, it is convenient to have a separate name for the > version 3 CLI since we often need to both the older version 2 and > new version 3 programs at the same time. For example, to upgrade a > legacy database: > > sqlite olddatabase .dump | sqlite3 newdatbase > > There are still many sqlite version 2 databases around, and so it > is important to be able to have both older "sqlite" and newer "sqlite3" > binaries in your PATH. I can imagine that it will similarly be > convenient to have both "python" and "python3" in your PATH at the > same time. > > I think calling it "python3" is probably a good idea. > > -- > D. Richard Hipp From lists at cheimes.de Tue Feb 5 21:04:25 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 05 Feb 2008 21:04:25 +0100 Subject: [Python-3000] Free list for small longs Message-ID: <47A8C149.1020407@cheimes.de> I've implemented a free list for small long objects with a size of 1 or -1. I wanted to test how large the malloc overhead is. The result was astonishing. The free list quadrupled the speed of a simple test: $ ./python -m timeit "for i in range(100): list(range(1000))" Without patch: 10 loops, best of 3: 79 msec per loop With patch: 10 loops, best of 3: 20.8 msec per loop Since the free list is limited to small longs, it will consume less than 1,5 MB on a 64bit OS and less than 900kb on a 32bit OS in a worst case scenario (2 * (1<<15) ~ 65k objects with a size of 14 / 22 byte each). http://bugs.python.org/issue2013 Christian From skip at pobox.com Tue Feb 5 21:13:29 2008 From: skip at pobox.com (skip at pobox.com) Date: Tue, 5 Feb 2008 14:13:29 -0600 Subject: [Python-3000] Free list for small longs In-Reply-To: <47A8C149.1020407@cheimes.de> References: <47A8C149.1020407@cheimes.de> Message-ID: <18344.50025.175774.601181@montanaro-dyndns-org.local> Christian> I've implemented a free list for small long objects with a Christian> size of 1 or -1. I'm not sure what you mean by "size of 1 or -1". Do you mean you only keep the numbers 1 and -1 on the free list? It's not obvious to me what a size of -1 is. Do you mean positive and negative numbers which fit in one byte or one long word? Skip From brett at python.org Tue Feb 5 21:23:07 2008 From: brett at python.org (Brett Cannon) Date: Tue, 5 Feb 2008 12:23:07 -0800 Subject: [Python-3000] Free list for small longs In-Reply-To: <18344.50025.175774.601181@montanaro-dyndns-org.local> References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> Message-ID: On Feb 5, 2008 12:13 PM, wrote: > > Christian> I've implemented a free list for small long objects with a > Christian> size of 1 or -1. > > I'm not sure what you mean by "size of 1 or -1". Do you mean you only keep > the numbers 1 and -1 on the free list? It's not obvious to me what a size > of -1 is. Do you mean positive and negative numbers which fit in one byte > or one long word? I think Christian means single digit integers. But I thought we already did this for positive numbers? So is the proposal to also cover negative numbers? And obviously having a single loop that explicitly tests the optimization is not exactly indicative of real-world use. =) -Brett From lists at cheimes.de Tue Feb 5 21:26:44 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 05 Feb 2008 21:26:44 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: <18344.50025.175774.601181@montanaro-dyndns-org.local> References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> Message-ID: <47A8C684.20002@cheimes.de> skip at pobox.com wrote: > I'm not sure what you mean by "size of 1 or -1". Do you mean you only keep > the numbers 1 and -1 on the free list? It's not obvious to me what a size > of -1 is. Do you mean positive and negative numbers which fit in one byte > or one long word? I should have explained how longs use the object size. The absolute value of Python long is stored is an array of digits (unsigned short). For Negative values Python longs set the ob_size to a negative value. >From Include/longintpr.h: /* Long integer representation. The absolute value of a number is equal to SUM(for i=0 through abs(ob_size)-1) ob_digit[i] * 2**(SHIFT*i) Negative numbers are represented with ob_size < 0; zero is represented by ob_size == 0. In a normalized number, ob_digit[abs(ob_size)-1] (the most significant digit) is never zero. Also, in all cases, for all valid i, 0 <= ob_digit[i] <= MASK. The allocation function takes care of allocating extra memory so that ob_digit[0] ... ob_digit[abs(ob_size)-1] are actually available. CAUTION: Generic code manipulating subtypes of PyVarObject has to aware that longs abuse ob_size's sign bit. */ Christian From lists at cheimes.de Tue Feb 5 21:31:17 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 05 Feb 2008 21:31:17 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> Message-ID: <47A8C795.10001@cheimes.de> Brett Cannon wrote: > I think Christian means single digit integers. But I thought we > already did this for positive numbers? So is the proposal to also > cover negative numbers? Ne, we don't use a free list for longs. In the 2.x series floats and ints are allocated in blocks of 1000 objects. The 3.x series uses the approach for floats only. Longs are always allocated with a free list unless the value is in between -5 and +256. These longs are statically allocated using an array. > And obviously having a single loop that explicitly tests the > optimization is not exactly indicative of real-world use. =) Damn, you got me ;) Christian From martin at v.loewis.de Tue Feb 5 21:44:50 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 05 Feb 2008 21:44:50 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: <47A8C149.1020407@cheimes.de> References: <47A8C149.1020407@cheimes.de> Message-ID: <47A8CAC2.2070609@v.loewis.de> > Since the free list is limited to small longs, it will consume less than > 1,5 MB on a 64bit OS and less than 900kb on a 32bit OS in a worst case > scenario (2 * (1<<15) ~ 65k objects with a size of 14 / 22 byte each). I'm puzzled as to how you arrive at this upper bound. You can surely have more than 2**16 integer objects whose abs values are all below 2**15, no? smallints=[] for i in range(2**20): smallints.append(i % 30000) Regards, Martin From lists at cheimes.de Tue Feb 5 22:47:11 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 05 Feb 2008 22:47:11 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: <47A8CAC2.2070609@v.loewis.de> References: <47A8C149.1020407@cheimes.de> <47A8CAC2.2070609@v.loewis.de> Message-ID: Martin v. L?wis wrote: > I'm puzzled as to how you arrive at this upper bound. You can surely > have more than 2**16 integer objects whose abs values are all > below 2**15, no? Oh ... I see. You are right and I did a mistake by confusing the amount of objects with the range. I'd better add an upper bound check. From jcea at argo.es Tue Feb 5 22:58:20 2008 From: jcea at argo.es (Jesus Cea) Date: Tue, 05 Feb 2008 22:58:20 +0100 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> Message-ID: <47A8DBFC.5090704@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote: | On Jan 25, 2008, at 12:07 AM, Guido van Rossum wrote: |> (*) I can only speculate that it's because Jim, whose ZODB is probably |> pickle's most intensive user, | | Maybe, but I'm not sure. Any persistence system is pickle intensive. Aside ZODB, we have Durus also. Do not forget remote call systems, like Pyro or most instances of parallel python attempts. pickle/cpickle performance is important. In fact, I find "disturbing" that "bencode" (from BitTorrent), done 100% in python, is faster that cPickle. Yes, "bencode" can't encode arbitrary objects, but still... - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ ~ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBR6jb+5lgi5GaxT1NAQLhnAP8COAFON7zALBX+4peFKH6LwlxA65IptqW Xg40Z3UZUo5bi67vkCvI/SuNLYpcvez2S5LTBrrlCHTWu4DSdI7/K9KPzQ8rbIdn EfCwuvSWOAKJ/mvTFA1LP0RixNd/+MTllFq+9TbOIe8EYZ/UbNLj4StqQBNGydjY KAmntN4VBnE= =C+TR -----END PGP SIGNATURE----- From martin at v.loewis.de Tue Feb 5 23:30:14 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 05 Feb 2008 23:30:14 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> Message-ID: <47A8E376.8000308@v.loewis.de> > I think Christian means single digit integers. But I thought we > already did this for positive numbers? So is the proposal to also > cover negative numbers? We don't have a free list, but a cache for them, and only for numbers between -5 and +256. Christian proposes a freelist, similar to the special allocate in 2.x. Regards, Martin From charles.merriam at gmail.com Tue Feb 5 23:39:36 2008 From: charles.merriam at gmail.com (Charles Merriam) Date: Tue, 5 Feb 2008 14:39:36 -0800 Subject: [Python-3000] pickle, cPickle, and the standard library (was Re: [Python-Dev] inst_persistent_id) In-Reply-To: References: <19ECF92A-31C8-48EE-AFEB-62055157EF96@zope.com> <20080125045906.GA21722@panix.com> <2939553A-422C-4222-8DFB-4734BB31CB8E@zope.com> <47A6DAC6.3000301@gmail.com> Message-ID: I agree. A wiki page for "leaving just the head of the snake" would be the correct solution. On Feb 4, 2008 4:34 PM, Jim Jewett wrote: > On 2/4/08, Charles Merriam wrote: > > What about a formal dependency plan instead? > > > Python would remain 'batteries included', perhaps verging on 'kitchen > > sink included'. > > > Those users working to embed Python or work in other situations where > > a large set of available libraries is a problem would have some help > > in cutting down the footprint in an particular installation. That is, > > one could freely delete some set of libraries and be fairly sure the > > resulting library set would still run. What was left would be a > > smaller footprint interpreted language, just not 'Python'. > > > Under what circumstances is this a poor idea? "Hard drive space is > > expensive" is not an acceptable argument. > > (1) Many library modules *use* other modules, but don't really > *rely* on them. Only a little functionality would be lost. That sort > of roadmap would encourage the functionality to get left out entirely, > and would certainly increase the amount of fallback boilerplate. > > (2) In bureaucratic environments, the libraries are OK because they > are standard; if a more minimal distribution becomes equally standard, > that will cause problems. (It wouldn't be nearly as bad as taking > them out and requiring downloads, but it would still be a problem, > because of the need to justify the non-minimal version.) > > (3) If storage footprint is important, then it is also important for > deployed applications -- and it becomes awkward if you have to > sprinkle your own code with fallbacks and workarounds in case the > standard library was clipped. > > That said, it might be useful if it were kept at unofficial status, > such as a wiki page. > > -jJ > From brett at python.org Wed Feb 6 00:05:29 2008 From: brett at python.org (Brett Cannon) Date: Tue, 5 Feb 2008 15:05:29 -0800 Subject: [Python-3000] Free list for small longs In-Reply-To: <47A8E376.8000308@v.loewis.de> References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> Message-ID: On Feb 5, 2008 2:30 PM, "Martin v. L?wis" wrote: > > I think Christian means single digit integers. But I thought we > > already did this for positive numbers? So is the proposal to also > > cover negative numbers? > > We don't have a free list, but a cache for them, and only for numbers > between -5 and +256. Christian proposes a freelist, similar to the > special allocate in 2.x. Gotcha. Well, I am a little leery of another free list. I personally don't love the fact that there are various caches and free lists laying about in the code without a central place to turn them off or clear them (or at least list them in a comment somewhere). And the problem with the free lists that the caches don't have is that they assume everyone gets a performance perk from them when there is a chance no one will ever touch a thing in the free lists. -Brett From lists at cheimes.de Wed Feb 6 00:51:20 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 06 Feb 2008 00:51:20 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> Message-ID: Brett Cannon wrote: > Well, I am a little leery of another free list. I personally don't > love the fact that there are various caches and free lists laying > about in the code without a central place to turn them off or clear > them (or at least list them in a comment somewhere). And the problem > with the free lists that the caches don't have is that they assume > everyone gets a performance perk from them when there is a chance no > one will ever touch a thing in the free lists. I understand your objection. Most free lists and caches are limited but some are not. I'm worried about the float and int blocks. Python's memory management never frees an int or float block. Code like >>> x = [float(i) for i in list(range(1000000))] >>> del x creates a million float and int objects but *never* frees the memory until Python exists. It's a pathologic case but it shows a problem. I've started to implement an API to compact the free lists, PyNAME_CompactFreelist() and sys._compact_freelists(). My new sys._compact_freelists() deallocates the float and int blocks which do not contain a referenced object. Most free lists are limited except the int (2.x only ) and float blocks. The overview may not be complete and 2.x may have more free lists. bytesobject ----------- no free list, only a single nullbyte classobject ----------- unlimited *free_list of PyMethodObject dictobject ---------- *free_dicts[] limited to 80 entries frameobject ----------- a zombie frame on each code object limited *free_list (linked list) with 200 entries max floatobject ----------- unlimited blocks of allocated floats with 1000 entries each The blocks are a single linked list and the free floats are linked by ob_type (ob_type points to the next free float). intobject (2.x) --------------- same as floatobject listobject ---------- *free_lists[] limited to 80 entries longobject ---------- cache for small number between -5 and +256 my patch for a limited free list methodobject ------------ unlimited *free_list of PyCFunctionObject setobject --------- *free_sets[] limited to 80 entries stringobject ------------ cache for nullstring and strings with one entry interned strings (2.x only) tupleobject ----------- 20 free lists for tuples with up to 20 entries. Each free list is limited to 2000 entries. unicodeobject ------------- *unicode_freelist limited to 1024 entries Christian From stephen at xemacs.org Wed Feb 6 01:32:59 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 06 Feb 2008 09:32:59 +0900 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> Message-ID: <87sl06udok.fsf@uwakimon.sk.tsukuba.ac.jp> Adam Olsen writes: > > Setuptools [pybang] works with versions 2.3 through 2.5, but of > > course that's because of the .exe wrappers. As independent corroboration, this is basically the same way that DJGPP (the DOS-extended version of GCC) provides Unix-y features like access to the environment variables and command line parsing (including redirection and pipes). Of course since DJGPP produces .exes, all it requires to use the feature is an #include. > Sounds good enough to me. Maybe the only thing "wrong" is people > aren't sufficiently informed of how to be version-specific? I think it's worse than that. People don't *want* to be version- specific (rather, they feel it is being "imposed" on them), and they generally believe that their particular feature dependencies are quite general and deserve conservation across version changes. Some, like the "field heavyweight" quoted by the OP, are refreshingly pragmatic about it. They're quite happy to use a language that is pretty crappy for most purposes today considered practical because it does a great job of continuing to run the programs written to address the "practical purposes" of three decades ago (if I interpret the "76" in "SHELX-76" correctly). Others, like the OP, want to freeze Python and request that updated versions be considered an internal fork in the project rather than evolutionary[1] progress whenever their inconvenience tolerance (which is clearly high in the OP's case, let's not belittle that!) is exceeded. Footnotes: [1] Cf. "punctuated equilibrium" for my notion of "evolution." From stephen at xemacs.org Wed Feb 6 02:32:53 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 06 Feb 2008 10:32:53 +0900 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080204183807.412b14d8@bhuda.mired.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> Message-ID: <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> Mike Meyer writes: > On Tue, 05 Feb 2008 08:20:00 +0900 "Stephen J. Turnbull" wrote: > > Mike Meyer writes: > > > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" wrote: > > > Wrong. The recommend "make install" overwrites the "python" link. > > Irrelevant. That "python" link is *not involved* in the shebang > > quoted above, and it is not part of any Python installation, precisely > > The python link is part of the python install on every system I've > dealt with - including from the source tarballs - as they all install > it, and the vast majority of software that needs python to run expect > to find it there. And your point might be? Did you snip the part where I explained that that is irrelevant to my point before reading it? If not, please explain; I've already explained (better expressed as "setup.py installdefault", see below) how I propose that practical issue be dealt with both sanely and compatibly. > Right. And the packaging system is what most people deal with in the > real world. Ignoring the real world is a good way to become > irrelevant. And while we can't control them (GNU/Linux packagers have > a really nasty habit of installing software configured in ways the > author doesn't want to support), we can at least nudge them in the > right direction by making it easy to do things in what we consider to > be the best way. GNUbies *will* ignore you. "GNU maintainers" do not care, by Stallman's definition, about the "right way" for any system but the GNU System. Unfortunately, there is no such thing as The GNU System, so each packager feels free to define what's right for the GNU System in terms of their own practices. This is paradigmatic for why "python3" is a nonsolution. To paraphrase the good Samuel Clemens, I think the reports of impending Python irrelvancy are vastly overblown. As my friend Tom Lord puts it, the vast majority of software doesn't exist yet. The battleground for relevancy lies in the future. Finally, I have been arguing that the right way is precisely to leave it up to the packagers because they have the knowledge and the technology to deal with dependencies. Python (the project) does not. As Nick Coghlan put it, "you're pushing for a language level solution to a system level problem: how do we reliably associate files with an application that understands them?" > So you want the default behavior after installing python to be that > the "python" command not be found? Maybe. It wouldn't be hard to detect that there is no "python" on $PATH and finish either the configuration stage or the install stage with the warning, **** There is no 'python' on $PATH. **** If you want this version of Python to become your default "python", run "setup.py installdefault" now. (You may need Administrator or root privileges to do so.) > > Sure, both are good things. But having setup.py provide them is not. > > If the user and/or the distro is providing the link, setup.py should > > assume that they know what they are doing. > > If the users are using a packaging system, then what setup.py does - > or does not do - is largely irrelevant. Users will get what the > packagers want them to have, and there's not jack that the people > actually writing the software can do about it. No. If the user is root installing Python as support for system software, then what setup.py does can mess up the packaging system, and this is the most likely result of having "python" and/or "python2" and/or "python3" links made by default. OTOH, if they're installing into a non-system location, the text above is more than sufficient warning to those with the ability and courage to successfully run "setup.py install" from tarballs. > If the users get what setup.py provides, then they built from source, > so they at least know how to install and deal with a build > environment, and it's reasonable to assume that they may know a little > bit more than that. That's exactly what I just wrote. I don't understand why you draw the conclusion you do, that a "python3" link somehow becomes a good idea (rather than a crock pandering to lazy developers downstream). > We can either determine a good way to deal with the issues this > raises, Nick's point, and my point, is that all the good ways require either downstream developer discipline in use of shebangs that are adequate, or system-level support. > and nudge package builders and people doing installs from > source in that direction by making those things the default, or at > least easy; Although we disagree on what the semantics should be, what could be easier than "setup.py installdefault", and still retain some semblance of providing minimal opportunity for exercise of responsibility by the sysadmin? > or we can do what was done in the 1->2 transition and let > them all work it out for themselves, The whole problem is that the RightThang[tm] has nothing to do with Python package builders and people who install Python from source. The problem is entirely downstream from Python: it is that app developers target a specific feature set but are unwilling to make the miniscule effort to announce it with a properly specific shebang. If in fact the Python 2.x for x in 0, ..., 6 are nearly always backward compatible, then a better way (but still not perfect) for the system or user to solve the problem is for the system to create a "python2.y" alias for python2.x, for y in 0, ..., x-1. If you detect a bug (eg, the one that came in with the introduction of boolean True), then you can simply override the alias with an installation of the needed version. *This doesn't work with "python2" shebangs.* You must also fix the app's shebangs. If you upgrade the app, you must remember to fix the shebangs again. ("Remember" includes maintaining local and vendor branches in a SCM of course.) > and risk once again having some popular distro on which "python" is > obsolete for the next N years. Huh? Python 1.5.2 was then, and remains now, a better language for my purposes than any version of awk, FORTRAN, or Perl I've ever used. :-) I only wrote about 10 Python scripts on Red Hat systems, but none of them barfed because "python" was "obsolete." Red Hat had *lots* of other problems, which is why I abandoned it. Having to type "python2.2" occasionally was the least of my complaints about RHL (which I have been pronouncing "are hell" since the last millennium: I was shocked, SHOCKED, I tell you, when they chose to make that pronunciation the official name of their flagship certified product!) From rwgk at yahoo.com Wed Feb 6 04:53:25 2008 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Tue, 5 Feb 2008 19:53:25 -0800 (PST) Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! Message-ID: <865955.18383.qm@web31106.mail.mud.yahoo.com> Stephen J. Turnbull wrote: > Some, like the "field heavyweight" quoted by the OP, are refreshingly > pragmatic about it. They're quite happy to use a language that is > pretty crappy for most purposes today considered practical because it > does a great job of continuing to run the programs written to address > the "practical purposes" of three decades ago (if I interpret the "76" > in "SHELX-76" correctly). He is one of the brightest people you'll find on this planet. His set of programs still is a de-facto standard, used to solve the vast majority of (small molecule) crystal structures. To know what that means, consider that without his work, you wouldn't be sitting in front of *that* computer. It would be a different computer in a different world. > Others, like the OP, want to freeze Python and request that updated > versions be considered an internal fork in the project rather than > evolutionary[1] progress whenever their inconvenience tolerance (which > is clearly high in the OP's case, let's not belittle that!) is > exceeded. It remains to be seen if your idea of "evolution" will still be remembered in 30 years. From all I've seen, arbitrarily introducing hardships in the name of progress doesn't work out in practice. Look around. C++ is dirty but got big because it never seriously broke with the C heritage. Microsoft is the biggest software company in the world even though the OS is dirty, but my DOS Turbo C compiler from 1990 probably still runs on a Windows XP system. Apple's market share in the OS market is still tiny in comparison, I'd argue to a significant degree because they were constantly "innovating". I'm not at all convinced Python 3 will succeed, although I'm hoping for it. Introducing a break like that without at the same time introducing new technologies is in stark violation of "practicality beats purity." My personal battle is to not see my own work die a slow death because everybody in my field thinks I must be crazy to base my work on something that is one day this and the next day something incompatible. If you give it a new name at least, it will be much easier for me to explain and stand my ground. Also, look at the sqlite experience: http://mail.python.org/pipermail/python-3000/2008-February/011991.html You really need to be able to have both the old and the new in the same environment indefinitely, and it has to be easily predictable what you get when you run "python". Ralf From stephen at xemacs.org Wed Feb 6 07:39:50 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 06 Feb 2008 15:39:50 +0900 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <865955.18383.qm@web31106.mail.mud.yahoo.com> References: <865955.18383.qm@web31106.mail.mud.yahoo.com> Message-ID: <87lk5ytwp5.fsf@uwakimon.sk.tsukuba.ac.jp> Ralf W. Grosse-Kunstleve writes: > Stephen J. Turnbull wrote: [really butchered quotes omitted] > [Prof. George Sheldrick] is one of the brightest people you'll find > on this planet. Ad hominem argument. My point is simply that he knows what he needs and said so. He doesn't need a language able to evolve with the times. Good for him. > It remains to be seen if your idea of "evolution" will still be > remembered in 30 years. It will, if only because it's not mine, but rather Stephen Jay Gould's. > From all I've seen, arbitrarily introducing hardships in the name > of progress doesn't work out in practice. Who's introducing hardships? The hardships are inherent in progress in a language. All I've argued is that the way you want to deal with these hardships is fundamentally broken in the face of a rapidly evolving language. The right way is for the app to declare the API it needs, and for the system to point to a version that provides that API. A quick and dirty but basically right solution is available for systems that understand the shebang: put the fully-versioned name in the shebang, and make links from the "best" version available to all versions it can support. > My personal battle is to not see my own work die a slow death because > everybody in my field thinks I must be crazy to base my work on something > that is one day this and the next day something incompatible. If so, I feel for you. Nevertheless, even if they do, that's not Python's problem to solve. > If you give it a new name at least, it will be much easier for me > to explain and stand my ground. What's wrong with "Python 2.4" (or whatever)? It's not like the vendor is going to go out of business or strategically stop distributing it. It's not like all the programmers familiar with Python 2 idioms are going to get their brains wiped. It's not like all the familiar modules will suddenly stop working with Python 2. It's not like the community will suddenly refuse to answer your questions because you're working with Python 2. And it's not like Python 2 will stop being a language far more suited to rapid evolution of your application than FORTRAN. > You really need to be able to have both the old and the new in the > same environment indefinitely, and it has to be easily predictable > what you get when you run "python". If you think you depend on predictability of "what you get when you run 'python'", I hope you're wrong, because it's not dependable unless you "own" the OS distribution that your users run on and all of their other applications, too. From ncoghlan at gmail.com Wed Feb 6 12:09:11 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 06 Feb 2008 21:09:11 +1000 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080205112232.34bc5c80@mbook-fbsd> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <47A87A31.8070208@gmail.com> <20080205112232.34bc5c80@mbook-fbsd> Message-ID: <47A99557.2000303@gmail.com> Mike Meyer wrote: > On Wed, 06 Feb 2008 01:01:05 +1000 Nick Coghlan wrote: >> While I'm +1 on the idea of making altinstall the default for setup.py >> and the Windows installer (forcing the administrator to explicitly >> request that the new Python version be made the default Python >> installation for the machine), I'm a pretty strong -1 on officially >> blessing the idea of a python3 alias or a special py3 extension on >> Windows (end users are obviously free to create those if they want, of >> course). > > I like the python3 alias (and python2 in 2.X), because it moves us > from a single default python to having a default python2 and python3, > which means scripts that care can ask for that without breaking the > ability to upgrade python2 or python3 without breaking those scripts, > while still allowing the default python to be either python2 or > python3. The main objection I have is that the executable name is only part of the story - all of the PYTHON* environment variables may matter as well (there are 9 listed in python -h, and I would have to check the code to say with complete confidence that there aren't any others). That said, while as a matter of purity I still think this is a platform problem (one which all current platforms I know of handle poorly ;), as a matter of practicality I'm seeing some benefits allowing a machine to have full fledged "Python 2" and "Python 3" setups side by side. If Guido does decide to go the route of changing the interpreter binary to python3 (ala sqlite vs sqlite3), I would also suggest that we go the whole way and insert a '3' into all of the environment variable names that python3 looks for (PYTHON3PATH, PYTHON3STARTUP, etc). I'd even concede the utility of accepting the py3 extension on modules as a concession to extension based dispatching on Windows. I still think it would be a bit of an ugly hack, but if it was followed through to the point of being able to define separate Python 2 & 3 environments without the use of launch scripts (i.e. by changing the environment variable names as described above), maybe it would still be a worthwhile thing to do. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From mwm at mired.org Wed Feb 6 07:24:35 2008 From: mwm at mired.org (Mike Meyer) Date: Wed, 6 Feb 2008 01:24:35 -0500 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080206012435.44bd22b7@bhuda.mired.org> On Wed, 06 Feb 2008 10:32:53 +0900 "Stephen J. Turnbull" wrote: > Mike Meyer writes: > > On Tue, 05 Feb 2008 08:20:00 +0900 "Stephen J. Turnbull" wrote: > > > Mike Meyer writes: > > > > On Mon, 04 Feb 2008 20:22:04 +0900 "Stephen J. Turnbull" wrote: > > > > Wrong. The recommend "make install" overwrites the "python" link. > > > Irrelevant. That "python" link is *not involved* in the shebang > > > quoted above, and it is not part of any Python installation, precisely > > The python link is part of the python install on every system I've > > dealt with - including from the source tarballs - as they all install > > it, and the vast majority of software that needs python to run expect > > to find it there. > And your point might be? Did you snip the part where I explained that > that is irrelevant to my point before reading it? If not, please It's not irrelevant, because you're basic assumption is wrong. Or maybe you're just engaged in wishful thinking. The default python install includes the python link, and overwrites, much as you might wish it otherwise. > To paraphrase the good Samuel Clemens, I think the reports of > impending Python irrelvancy are vastly overblown. So do I. That doesn't mean the situation can't be improved. > Finally, I have been arguing that the right way is precisely to leave > it up to the packagers because they have the knowledge and the > technology to deal with dependencies. You mean the people you described this way: > GNUbies *will* ignore you. "GNU maintainers" do not care, by > Stallman's definition, about the "right way" for any system but the > GNU System. Unfortunately, there is no such thing as The GNU System, > so each packager feels free to define what's right for the GNU System > in terms of their own practices. Right - they all do it however they feel like, and there's not much we can do about it. That's not *quite* the same as nothing we can do about it. > Python (the project) does not. Correct. On the other hand, the collective of people here certainly have more experience at dealing with multiple python installations on one system than any other group in the world. And you suggest that the advice from that collective experience that setup.py should offer packagers on how to deal with this situation is - none? > As Nick Coghlan put it, "you're pushing for a language level solution > to a system level problem: how do we reliably associate files with an > application that understands them?" I'm not suggesting any changes to the language. Like you, I'm suggesting a change to the default install. I happen to believe it will make most peoples lives easier. > This is paradigmatic for why "python3" is a nonsolution. Just because it doesn't solve your solutions doesn't mean it doesn't solve anyones. And it's sure as hell better than doing nothing. > > So you want the default behavior after installing python to be that > > the "python" command not be found? > Maybe. It wouldn't be hard to detect that there is no "python" on > $PATH and finish either the configuration stage or the install stage > with the warning, > **** There is no 'python' on $PATH. **** > If you want this version of Python to become your default "python", > run "setup.py installdefault" now. > (You may need Administrator or root privileges to do so.) Yup. So what's the problem with doing the same thing for python at the same time? Personally, I'd rather it just check the install directory and not $PATH, because you pretty much have to do that if you need different installs of the same X.Y version. I'd also favor more automation, and have it offer to run that script rather than just suggest it. > > > Sure, both are good things. But having setup.py provide them is not. > > > If the user and/or the distro is providing the link, setup.py should > > > assume that they know what they are doing. > > If the users are using a packaging system, then what setup.py does - > > or does not do - is largely irrelevant. Users will get what the > > packagers want them to have, and there's not jack that the people > > actually writing the software can do about it. > No. If the user is root installing Python as support for system > software, then what setup.py does can mess up the packaging system, > and this is the most likely result of having "python" and/or "python2" > and/or "python3" links made by default. Sure, there are *lots* of ways the user can screw up the system as root. Doing a default python install from source is *not* one of them, no matter what it does with those symlinks. If you know of a system where things are otherwise, please warn me about it now. > > If the users get what setup.py provides, then they built from source, > > so they at least know how to install and deal with a build > > environment, and it's reasonable to assume that they may know a little > > bit more than that. > That's exactly what I just wrote. I don't understand why you draw the > conclusion you do, that a "python3" link somehow becomes a good idea I draw that conclusion based on a decade and a half of installing, maintaining and upgrading python software. > (rather than a crock pandering to lazy developers downstream). Damn straight I'm lazy. Computers - all of them - suck bad enough as it is. Anything I can do to make my life easier is a win. > > We can either determine a good way to deal with the issues this > > raises, > Nick's point, and my point, is that all the good ways require either > downstream developer discipline in use of shebangs that are adequate, > or system-level support. True. But you act like making it *easy* for people is somehow criminal. > > and nudge package builders and people doing installs from > > source in that direction by making those things the default, or at > > least easy; > Although we disagree on what the semantics should be, what could be > easier than "setup.py installdefault", and still retain some semblance > of providing minimal opportunity for exercise of responsibility by the > sysadmin? I don't really have a problem with that; I just think we should be looking beyond the end of our noses - uh, 3.0 - and trying to set things up to make things easier for the system administrator. > > or we can do what was done in the 1->2 transition and let > > them all work it out for themselves, > The whole problem is that the RightThang[tm] has nothing to do with > Python package builders and people who install Python from source. > The problem is entirely downstream from Python: it is that app > developers target a specific feature set but are unwilling to make the > miniscule effort to announce it with a properly specific shebang. But of course they don't. Your "properly specific shebang" will break on every version of python *after* the one they target, even though the script would otherwise work. No developer in their right mind wants to deal with the extra support work that would generate. > If in fact the Python 2.x for x in 0, ..., 6 are nearly always > backward compatible, then a better way (but still not perfect) for the > system or user to solve the problem is for the system to create a > "python2.y" alias for python2.x, for y in 0, ..., x-1. Personally, I think *this* is a crock. But if you're going to advocate that people lie about what versions they have installed, then please advocate having setup.py do something to encourage people to create systems that look like this, so that app developers have a reason to use the shebang line you want them to. > If you detect a bug (eg, the one that came in with the introduction > of boolean True), then you can simply override the alias with an > installation of the needed version. *This doesn't work with > "python2" shebangs.* Right. It's not identical, so you have to do something *different* to solve this problem. With python2, you set the python2 link back to the previous version. Yeah, you may have to fix the apps that require the new version, but oddly enough, the more of those scripts there are, the less likely it is that one of your old apps will barf on that new version. Personally, I think that's a *lot* easier than running around creating or re-creating a bunch of symlinks every time I install or upgrade python, much less trying to figure out when I really can declare that I no longer need some version under those conditions. On the other hand, if setup.py made setting up the symlinks easy, maybe I'd change my mind. > > and risk once again having some popular distro on which "python" is > > obsolete for the next N years. > Huh? Python 1.5.2 was then, and remains now, a better language for my > purposes than any version of awk, FORTRAN, or Perl I've ever used. :-) > I only wrote about 10 Python scripts on Red Hat systems, but none of > them barfed because "python" was "obsolete." Red Hat had *lots* of > other problems, which is why I abandoned it. Having to type > "python2.2" occasionally was the least of my complaints about RHL > (which I have been pronouncing "are hell" since the last millennium: I > was shocked, SHOCKED, I tell you, when they chose to make that > pronunciation the official name of their flagship certified product!) Much as RHEL may work to remind you that GNU means GNUs Not Unix, it's still one of the most popular distributions around, and hence liable to be many people's first exposure to python in a Unix-oid environment. The more we can do to make that part of their experience not suck, the more likely they are to consider doing more work in Python. Not to mention making life better for those of us who have clients that won't let us include the depenguinator in our list of required software packages. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From nate at binkert.org Wed Feb 6 18:40:25 2008 From: nate at binkert.org (nathan binkert) Date: Wed, 6 Feb 2008 09:40:25 -0800 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080206012435.44bd22b7@bhuda.mired.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> <20080206012435.44bd22b7@bhuda.mired.org> Message-ID: <217accd40802060940t627548e2i2b2a32b0a7b42656@mail.gmail.com> This is very similar to the #! idea and is really just an extension. Python 2 clearly cannot execute python 3 code, but how hard would it be to require something like a __pyversion__ = 3 statement at the top of the file. If the line does not exist, the python3 binary would dlopen libpython2.x and execute the code. I understand that the syntax of the two versions are not compatible, but couldn't the tokenizers be compatible? Imagine the python3 binary tokenizing the code, searching for the __pyversion__ line and if it does not exist, doing the dlopen and passing the tokenized code to the python2.x binary (with some potential translation to fix any token incompatibility). if the libpython2.x.so doesn't exist, then a VersionError could be raised. If the initial file does have a __pyversion__ and subsequent files do, then a VersionError exception could be raised. This solution could help deal with code that requires new features. For example, say I want to use a with clause, I could require 2.6 or 3.0. Sorry to prolong an already long conversation. Nate From lists at cheimes.de Wed Feb 6 19:37:47 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 06 Feb 2008 19:37:47 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080205175802.3FBF93A409E@sparrow.telecommunity.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> Message-ID: <47A9FE7B.7090405@cheimes.de> Phillip J. Eby wrote: > FYI, setuptools uses and supports #! lines on Windows, with the > executable path in quotes if it contains spaces. For a given script > "foo", it generates two files: > > foo.exe > foo-script.py > > And the .exe is just a standardized .exe file that looks in an > adjacent -script.py file of the same name for the #! line. Nice, it's an elegant and easy solution to the problem. I like to get a quite similar solution into the core but I propose a slightly different path. Like your launcher.c, spam.exe looks for spam.py. It additionally looks for spam.pyw first giving .pyw files a higher priority. The first line of the file is read and parsed. The line must start with a shebang (#!). Contrary to your script it doesn't look for a complete path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5" both work. Then the launcher executable queries the installation path of Python X.Y from the registry (first HKCU, then HKLM). At last the script is executed with the Python binary: python.exe for .py and pythonw.exe for .pyw. The working directory is the absolute path of spam.exe. This way we neither have to change the paths for Windows scripts nor quote the path name. Comments? Christian From lists at cheimes.de Wed Feb 6 19:38:49 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 06 Feb 2008 19:38:49 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080205175802.3FBF93A409E@sparrow.telecommunity.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> Message-ID: <47A9FEB9.70701@cheimes.de> Phillip J. Eby wrote: > FYI, setuptools uses and supports #! lines on Windows, with the > executable path in quotes if it contains spaces. For a given script > "foo", it generates two files: > > foo.exe > foo-script.py > > And the .exe is just a standardized .exe file that looks in an > adjacent -script.py file of the same name for the #! line. Nice, it's an elegant and easy solution to the problem. I like to get a quite similar solution into the core but I propose a slightly different path. Like your launcher.c, spam.exe looks for spam.py. It additionally looks for spam.pyw first giving .pyw files a higher priority. The first line of the file is read and parsed. The line must start with a shebang (#!). Contrary to your script it doesn't look for a complete path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5" both work. Then the launcher executable queries the installation path of Python X.Y from the registry (first HKCU, then HKLM). At last the script is executed with the Python binary: python.exe for .py and pythonw.exe for .pyw. The working directory is the absolute path of spam.exe. This way we neither have to change the paths for Windows scripts nor quote the path name. Comments? Christian From martin at v.loewis.de Wed Feb 6 20:20:13 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 06 Feb 2008 20:20:13 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> Message-ID: <47AA086D.60904@v.loewis.de> > My new > sys._compact_freelists() deallocates the float and int blocks which do > not contain a referenced object. I wonder whether such a thing should run as a side effect of a full garbage collection, also. GC has a chance to free up arenas for good, and allocated-but-unused blocks on arenas prevent Python from returning them to the OS. Regards, Martin From theller at ctypes.org Wed Feb 6 21:04:16 2008 From: theller at ctypes.org (Thomas Heller) Date: Wed, 06 Feb 2008 21:04:16 +0100 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A9FE7B.7090405@cheimes.de> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> Message-ID: Christian Heimes schrieb: > Phillip J. Eby wrote: >> FYI, setuptools uses and supports #! lines on Windows, with the >> executable path in quotes if it contains spaces. For a given script >> "foo", it generates two files: >> >> foo.exe >> foo-script.py >> >> And the .exe is just a standardized .exe file that looks in an >> adjacent -script.py file of the same name for the #! line. > > Nice, it's an elegant and easy solution to the problem. I like to get a > quite similar solution into the core but I propose a slightly different > path. > > Like your launcher.c, spam.exe looks for spam.py. It additionally looks > for spam.pyw first giving .pyw files a higher priority. Why give a higher priority to .pyw if the .py also exists? Shouldn't there be a way to select one or the other, maybe with a command line flag or whatever? And, while we're at it: What IMO needs to be fixed is that there is no way for a .pyw file to show exceptions to the user. Maybe the console window is not such a bad idea. Maybe it could be opened on demand, as soon as there is something printed to stderr? > The first line of the file is read and parsed. The line must start with > a shebang (#!). Contrary to your script it doesn't look for a complete > path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5" > both work. You mean you parse this string to get the version number? > Then the launcher executable queries the installation path of Python X.Y > from the registry (first HKCU, then HKLM). At last the script is > executed with the Python binary: python.exe for .py and pythonw.exe for > .pyw. The working directory is the absolute path of spam.exe. This way > we neither have to change the paths for Windows scripts nor quote the > path name. What if there are no registry entries (svn checkout)? Thomas From pje at telecommunity.com Wed Feb 6 21:28:25 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 06 Feb 2008 15:28:25 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A9FEB9.70701@cheimes.de> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FEB9.70701@cheimes.de> Message-ID: <20080206202817.2950F3A40AF@sparrow.telecommunity.com> At 07:38 PM 2/6/2008 +0100, Christian Heimes wrote: >The first line of the file is read and parsed. The line must start with >a shebang (#!). Contrary to your script it doesn't look for a complete >path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5" >both work. > >Then the launcher executable queries the installation path of Python X.Y >from the registry (first HKCU, then HKLM). At last the script is >executed with the Python binary: python.exe for .py and pythonw.exe for >.pyw. The working directory is the absolute path of spam.exe. This way >we neither have to change the paths for Windows scripts nor quote the >path name. > >Comments? I intentionally didn't do it that way because it won't work right for Python applications that bundle their own not-in-the-registry Python. From lists at cheimes.de Wed Feb 6 22:35:28 2008 From: lists at cheimes.de (Christian Heimes) Date: Wed, 06 Feb 2008 22:35:28 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: <47AA086D.60904@v.loewis.de> References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> <47AA086D.60904@v.loewis.de> Message-ID: Martin v. L?wis wrote: > I wonder whether such a thing should run as a side effect of a full > garbage collection, also. GC has a chance to free up arenas for good, > and allocated-but-unused blocks on arenas prevent Python from returning > them to the OS. Under which condition does a full garbage collection take place? Does the interpreter run a full gc once in a while or is it started by gc.collect()? I'm more concerned about the int and float memory blocks. They should get freed when more than a couple of hundred blocks are allocated. The other free lists are much smaller. But memory fragmentation is an issue for them, too. Christian From python at rcn.com Wed Feb 6 23:29:39 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 6 Feb 2008 17:29:39 -0500 (EST) Subject: [Python-3000] ABC method mismatch Message-ID: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net> The tuple class is registered to the collections.Sequence ABC, but it does not support count() and index() as required by the ABC. Do you want to drop those methods from the ABC or add them to tuple? Raymond From guido at python.org Wed Feb 6 23:40:39 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 6 Feb 2008 14:40:39 -0800 Subject: [Python-3000] ABC method mismatch In-Reply-To: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net> References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net> Message-ID: On Feb 6, 2008 2:29 PM, Raymond Hettinger wrote: > The tuple class is registered to the collections.Sequence ABC, but it does not support count() and index() as required by the ABC. Good catch! > Do you want to drop those methods from the ABC or add them to tuple? I'm not sure, but I'm tempted to add them to the ABC. What's your hunch? Do others have an opinion? They shouldn't be a burden for implementers who use real inheritance from Sequence, as they are concrete methods there. And it doesn't make sense to move them to MutableSequence, because there's nothing in them that requires the sequence to be mutable. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Wed Feb 6 23:55:36 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 6 Feb 2008 17:55:36 -0500 (EST) Subject: [Python-3000] ABC method mismatch Message-ID: <20080206175536.AGL69941@ms19.lnh.mail.rcn.net> >> Do you want to drop those methods from the ABC or add them to tuple? > I'm not sure, but I'm tempted to add them to the ABC. > What's your hunch? My hunch is that adding count() and index() to tuple is at harmless at worst and sometimes helpful at best (I've occasionally needed them and felt mild annoyance at having coerce tuples to lists just to get them). Also, if you add them now, you won't spend the rest of your life turning down requests to add them as people periodically discover the mismatch with strings and lists. Raymond From fdrake at acm.org Wed Feb 6 23:58:24 2008 From: fdrake at acm.org (Fred Drake) Date: Wed, 6 Feb 2008 17:58:24 -0500 Subject: [Python-3000] ABC method mismatch In-Reply-To: References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net> Message-ID: <0F448ACD-8670-40E2-B823-638143B75E49@acm.org> On Feb 6, 2008 2:29 PM, Raymond Hettinger wrote: > Do you want to drop those methods from the ABC or add them to tuple? On Feb 6, 2008, at 5:40 PM, Guido van Rossum wrote: > I'm not sure, but I'm tempted to add them to the ABC. What's your > Is Guido having a bad typing day? ;-) I'd be happy seeing these methods added to tuple; there's no reason that they would only be useful on mutable sequences. -Fred -- Fred Drake From guido at python.org Thu Feb 7 00:01:41 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 6 Feb 2008 15:01:41 -0800 Subject: [Python-3000] ABC method mismatch In-Reply-To: <0F448ACD-8670-40E2-B823-638143B75E49@acm.org> References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net> <0F448ACD-8670-40E2-B823-638143B75E49@acm.org> Message-ID: On Feb 6, 2008 2:58 PM, Fred Drake wrote: > On Feb 6, 2008 2:29 PM, Raymond Hettinger wrote: > > Do you want to drop those methods from the ABC or add them to tuple? > > > On Feb 6, 2008, at 5:40 PM, Guido van Rossum wrote: > > I'm not sure, but I'm tempted to add them to the ABC. What's your > Is Guido having a bad typing day? ;-) Ever since I turned 50, every day seems to be a bad typing day. :-( > I'd be happy seeing these methods added to tuple; there's no reason > that they would only be useful on mutable sequences. Sounds like this is the consensus. Go for it, Raymond! --Guido > -Fred > > -- > Fred Drake > > > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Feb 7 00:04:01 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 07 Feb 2008 00:04:01 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> <47AA086D.60904@v.loewis.de> Message-ID: <47AA3CE1.40501@v.loewis.de> Christian Heimes wrote: > Martin v. L?wis wrote: >> I wonder whether such a thing should run as a side effect of a full >> garbage collection, also. GC has a chance to free up arenas for good, >> and allocated-but-unused blocks on arenas prevent Python from returning >> them to the OS. > > Under which condition does a full garbage collection take place? Does > the interpreter run a full gc once in a while or is it started by > gc.collect()? Both. Let (a,b,c) equal gc.get_threshold(), then a full gc happens after a*b*c object allocations (roughly; early object deallocations are deducted from that number). Currently, this product is 70000. > I'm more concerned about the int and float memory blocks. They should > get freed when more than a couple of hundred blocks are allocated. The > other free lists are much smaller. But memory fragmentation is an issue > for them, too. Ain't we talking about Python 3? There aren't any int memory blocks around. As for float objects - sure, we should do something about them. I propose to change them into a freelist as well, and allocate float objects through pymalloc. I'd be curious whether any real-world application allocates tons of float objects temporarily, and whether such an application would improve by using the Numeric extensions. Regards, Martin From python at rcn.com Thu Feb 7 00:05:29 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 6 Feb 2008 18:05:29 -0500 (EST) Subject: [Python-3000] ABC method mismatch Message-ID: <20080206180529.AGL71523@ms19.lnh.mail.rcn.net> [Fred Drake] > I'd be happy seeing these methods added to tuple; there's > no reason that they would only be useful on mutable sequences. That reminds me, I've always thought Sequence.__contains__(). should always be accompanied by Sequence.index(). Their use cases are strongly related: Good news: I searched your house and found your keys Bad news: I don't have method to tell you where I found them Raymond From lists at cheimes.de Thu Feb 7 00:46:02 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 07 Feb 2008 00:46:02 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: <47AA3CE1.40501@v.loewis.de> References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> <47AA086D.60904@v.loewis.de> <47AA3CE1.40501@v.loewis.de> Message-ID: <47AA46BA.6080900@cheimes.de> Martin v. L?wis wrote: > Both. Let (a,b,c) equal gc.get_threshold(), then a full gc happens > after a*b*c object allocations (roughly; early object deallocations > are deducted from that number). Currently, this product is 70000. I see. 70k is a large number. Do you have some real world experience how often a a full gc takes place? > Ain't we talking about Python 3? There aren't any int memory blocks > around. As for float objects - sure, we should do something about them. > I propose to change them into a freelist as well, and allocate float > objects through pymalloc. I was referring to Python 2.x as well. Your proposed solution would make it easier to maintain a small free list but it may also make allocation slower. How old are the int and float blocks? Where they implemented before Python got pymalloc? I found another place that may gain some speedup from free lists. Currently list and dict aren't using a real free list. The code pre-allocates 80 objects each. I've modified the code to free list without pre-allocated objects. I also added some debug code to measure how many objects are reused or allocated: ./python setup build List allocations: 1046 List reuse through freelist: 15746 Dict allocations: 770 Dict reuse through freelist: 1307 A simple startup of Python List allocations: 59 List reuse through freelist: 644 Dict allocations: 239 Dict reuse through freelist: 223 Christian From python at rcn.com Thu Feb 7 02:17:12 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 6 Feb 2008 20:17:12 -0500 (EST) Subject: [Python-3000] Should ABCMeta.register() do validation and emit warnings Message-ID: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net> It's interesting that tuples could be registered as Sequences without having all the requisite methods. Should the register() method to some sort of validation that all public methods in the ABC are in the class/type being registered? Raymond From aahz at pythoncraft.com Thu Feb 7 02:21:16 2008 From: aahz at pythoncraft.com (Aahz) Date: Wed, 6 Feb 2008 17:21:16 -0800 Subject: [Python-3000] ABC method mismatch In-Reply-To: References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net> <0F448ACD-8670-40E2-B823-638143B75E49@acm.org> Message-ID: <20080207012116.GB11221@panix.com> On Wed, Feb 06, 2008, Guido van Rossum wrote: > On Feb 6, 2008 2:58 PM, Fred Drake wrote: >> >> I'd be happy seeing these methods added to tuple; there's no reason >> that they would only be useful on mutable sequences. > > Sounds like this is the consensus. Go for it, Raymond! Far be it from me to act the grinch (and I honestly don't care), but I'm curious what caused you to change your mind at this point. You have argued against this for years: http://www.python.org/search/hypermail/python-1992/0285.html More recently, your argument has revolved around the fact that you don't want tuples used as a "poor man's list" -- they are intended to be used for heterogeneous data. (Assuming I'm rephrasing you correctly.) (My purpose in poking you is to get better at channeling you rather than because this particular decision has much significance.) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From guido at python.org Thu Feb 7 03:28:05 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 6 Feb 2008 18:28:05 -0800 Subject: [Python-3000] ABC method mismatch In-Reply-To: <20080207012116.GB11221@panix.com> References: <20080206172939.AGL65745@ms19.lnh.mail.rcn.net> <0F448ACD-8670-40E2-B823-638143B75E49@acm.org> <20080207012116.GB11221@panix.com> Message-ID: On Feb 6, 2008 5:21 PM, Aahz wrote: > On Wed, Feb 06, 2008, Guido van Rossum wrote: > > On Feb 6, 2008 2:58 PM, Fred Drake wrote: > >> > >> I'd be happy seeing these methods added to tuple; there's no reason > >> that they would only be useful on mutable sequences. > > > > Sounds like this is the consensus. Go for it, Raymond! > > Far be it from me to act the grinch (and I honestly don't care), but I'm > curious what caused you to change your mind at this point. You have > argued against this for years: > > http://www.python.org/search/hypermail/python-1992/0285.html > > More recently, your argument has revolved around the fact that you don't > want tuples used as a "poor man's list" -- they are intended to be used > for heterogeneous data. (Assuming I'm rephrasing you correctly.) > > (My purpose in poking you is to get better at channeling you rather than > because this particular decision has much significance.) Well, I guess I changed my mind when I made tuple a virtual subclass of Sequence. That meant basically accepting it is *also* usable as an immutable sequence. It is accepting what everybody has been saying for a long time. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Feb 7 03:29:59 2008 From: guido at python.org (Guido van Rossum) Date: Wed, 6 Feb 2008 18:29:59 -0800 Subject: [Python-3000] Should ABCMeta.register() do validation and emit warnings In-Reply-To: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net> References: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net> Message-ID: On Feb 6, 2008 5:17 PM, Raymond Hettinger wrote: > It's interesting that tuples could be registered as Sequences without having all the requisite methods. Should the register() method to some sort of validation that all public methods in the ABC are in the class/type being registered? I think the Zope/Twisted interfaces have gone here before. I believe they have a separate validation call which is typically only invoked in unittests. The validation can be pretty expensive, so running the validation each time the interpreter is started would just slow things down. I've just not yet written the validation code. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Feb 7 03:35:37 2008 From: python at rcn.com (Raymond Hettinger) Date: Wed, 6 Feb 2008 21:35:37 -0500 (EST) Subject: [Python-3000] Should ABCMeta.register() do validation and emit warnings Message-ID: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net> > I think the Zope/Twisted interfaces have gone here before. > I believe they have a separate validation call which is > typically only invoked in unittests. +1 That seems like a good idea. Raymond From stephen at xemacs.org Thu Feb 7 03:48:08 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 07 Feb 2008 11:48:08 +0900 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080206012025.00a73dff@bhuda.mired.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> <20080206012025.00a73dff@bhuda.mired.org> Message-ID: <87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp> Mike Meyer writes: > Personally, I think *this* [proposal to link 2.x to 2.y for y < x] > is a crock. But if you're going to advocate that people lie about > what versions they have installed, If you insist on that interpretation, consider that it's no different from the common cc -> gcc link. Anyway, your favored solution of "python2" tells exactly the same "lie" ("Python 2" is not installed!), but in a way that cannot be interpreted satisfactorily for as many apps that have specific version preferences without changing the apps. Also, it doesn't tell what version of Python is actually recommended by the developers, which is a potential support issue. Finally, the developers who use "python2" shebangs *are* lying (no quotes!) unless they use only features present in Python 2.0. You asked whether I oppose making things easy. Not everything, but I sure do oppose making such inaccuracy easy! As I interpret it, in my proposal you aren't telling anyone that you have that version of Python installed; you are telling applications that you have installed *some* version of Python that can handle that version of the API. Eg, an admin can determine which Python version will deal with "python2.2" simply by doing "ls -l `which python2.2`". Again, it's not correct to interpret "python2" that way; "python2" is the sysadmin's preferred version of Python 2 at the moment, which in my experience is unlikly to be 2.x.0 for any x, and might be substantially older if the mission-critical apps run satisfactorily on that older version. Implying that "'python2' can handle any Python 2 script" is likely to remain a lie until the release of 2.x.1 at least, and possibly indefinitely. > With python2, [if there's a problem with 2.x's initial release] you > set the python2 link back to the previous version. Yeah, you may > have to fix the apps that require the new version, Which is (a) unnecessary in my scheme and (b) a long-term maintenance burden when upgrading the apps that require the new version. The second is a serious annoyance in my experience. > Personally, I think that's a *lot* easier than running around creating > or re-creating a bunch of symlinks every time I install or upgrade > python, Huh? Eminently automatable. Why do you think it would be hard? Also, do you really install or upgrade Python more often than you install or upgrade apps that run on Python, each time risking annoying breakage that requires hacking the app to fix? And since the app may not even tell you which versions you already have available that might be satisfactory, you have to test them yourself, and possibly install and uninstall several versions! Finally, do you really uninstall old versions that often? It seems likely to me that there probably would be no need at all to fiddle with symlinks based on upgrading Python 2.4.3 to Python 2.4.4, and no *pressing* need to update the symlinks upon installing Python 2.5.1 alongside Python 2.4.4. Rather you'd do that at a later time, after Python 2.5 "shakes down" and you've tested all your important apps with 2.5. Yes, it's a matter of balance, and YMMV. But between the theoretical deficiencies of "python2" and the degree to which I believe you're exaggerating the frequency (or lack thereof) that you would encounter the relevant situations under the two plans, I really don't see a good argument for "python2". > much less trying to figure out when I really can declare that > I no longer need some version under those conditions. This is a real issue with the scheme I proposed, although modern versions of file will tell you the whole shebang, I think. It's also a problem that modern PMSes will help you with, as well as being one faced only by those who run an unusually tight ship in my experience. Again, I would imagine it's not hard to automate. > On the other hand, if setup.py made setting up the symlinks easy, > maybe I'd change my mind. I think this should be a separate tool, but either way I'd not object to making it easy to do, as long as it was not default. From martin at v.loewis.de Thu Feb 7 06:04:12 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 07 Feb 2008 06:04:12 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: <47AA46BA.6080900@cheimes.de> References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> <47AA086D.60904@v.loewis.de> <47AA3CE1.40501@v.loewis.de> <47AA46BA.6080900@cheimes.de> Message-ID: <47AA914C.8020402@v.loewis.de> >> Both. Let (a,b,c) equal gc.get_threshold(), then a full gc happens >> after a*b*c object allocations (roughly; early object deallocations >> are deducted from that number). Currently, this product is 70000. > > I see. 70k is a large number. Do you have some real world experience how > often a a full gc takes place? Not from a real world application, no. If you run the program below with python /tmp/a.py 2>&1|grep "generation 2" you'll see it does one "regular" generation-2-collection; on my machine, it takes 0.23s for that to happen. Notice that this application doesn't use reference counting at all (i.e. objects are released here only through GC, not because their refcount goes zero before the GC triggers). What are you aiming at? Is it a good thing or a bad thing that 70k is a large number? > I was referring to Python 2.x as well. Your proposed solution would make > it easier to maintain a small free list but it may also make allocation > slower. How old are the int and float blocks? Where they implemented > before Python got pymalloc? I don't remember; try to find out for yourself. I think it will be very difficult to demonstrate that releasing the freelists has a significant effect when it's run along with GC. The time for running the GC should dominate the time spend in releasing the freelists, and any application for which the freelists help (i.e. which periodically allocate many similar object and then return them) will rebuild the freelist to the required size quickly. Regards, Martin From martin at v.loewis.de Thu Feb 7 06:44:12 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 07 Feb 2008 06:44:12 +0100 Subject: [Python-3000] Free list for small longs In-Reply-To: <47AA914C.8020402@v.loewis.de> References: <47A8C149.1020407@cheimes.de> <18344.50025.175774.601181@montanaro-dyndns-org.local> <47A8E376.8000308@v.loewis.de> <47AA086D.60904@v.loewis.de> <47AA3CE1.40501@v.loewis.de> <47AA46BA.6080900@cheimes.de> <47AA914C.8020402@v.loewis.de> Message-ID: <47AA9AAC.8000106@v.loewis.de> > Not from a real world application, no. If you run the program below with And now for the program below... Regards, Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: a.py Type: text/x-python Size: 211 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20080207/1023dad9/attachment.py From p.f.moore at gmail.com Thu Feb 7 10:15:16 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 7 Feb 2008 09:15:16 +0000 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A9FE7B.7090405@cheimes.de> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> Message-ID: <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> On 06/02/2008, Christian Heimes wrote: > Nice, it's an elegant and easy solution to the problem. I like to get a > quite similar solution into the core but I propose a slightly different > path. > > Like your launcher.c, spam.exe looks for spam.py. It additionally looks > for spam.pyw first giving .pyw files a higher priority. > > The first line of the file is read and parsed. The line must start with > a shebang (#!). Contrary to your script it doesn't look for a complete > path but searches for pythonX.Y. "#!python2.5" or "#!/usr/bin/python2.5" > both work. > > Then the launcher executable queries the installation path of Python X.Y > from the registry (first HKCU, then HKLM). At last the script is > executed with the Python binary: python.exe for .py and pythonw.exe for > .pyw. The working directory is the absolute path of spam.exe. This way > we neither have to change the paths for Windows scripts nor quote the > path name. > > Comments? One major issue: The same launcher cannot be used for py and pyw files. The launcher for py files needs to be compiled as a console app, whereas the launcher for pyw (GUI) files needs to be compiled as a GUI app. On Windows, GUI vs Console is a property of the EXE file, and the differing behaviour is built into the OS loader. You may not like the fact (at times, I certainly don't) but you can't fight it. Trying to avoid this is what makes GUI apps with consoles that flash up quickly before disappearing, and other similar nastiness. Also, be aware that Python does not *have* to register itself in the registry (AIUI). But other things (like bdist_wininst IIRC) expect it, so as long as you don't fail too horribly if there's no registry entries, you should be OK. Finally, you don't want to change the current directory. Suppose I had a simple program listdir.py that displayed the contents of the current directory. Using your wrapper would break it. Why do you think changing directory is needed, as opposed to making sys.path look like you want - oh, hang on, you're running python.exe so you have no access to sys.path. Hmm, your proposal may have a more serious problem than I realised. Paul. From alexander.belopolsky at gmail.com Thu Feb 7 16:18:05 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 7 Feb 2008 15:18:05 +0000 (UTC) Subject: [Python-3000] ABC method mismatch References: <20080206180529.AGL71523@ms19.lnh.mail.rcn.net> Message-ID: Raymond Hettinger rcn.com> writes: > > [Fred Drake] > > I'd be happy seeing these methods added to tuple; there's > > no reason that they would only be useful on mutable sequences. > Note that adding index and count to tuple will conflict with using these names as attributes in classes generated by collections.namedtuple. There is certain elegance in tuples and numbers having no non- special methods. What is the advantage of having count and index in Sequence ABC? These methods are easily implementable as functions that take an iterable (in fact, itertools.count already exists) and having them in Sequence ABC puts an extra burden on implementors of concrete classes. From lists at cheimes.de Thu Feb 7 17:27:42 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 07 Feb 2008 17:27:42 +0100 Subject: [Python-3000] Should ABCMeta.register() do validation and emit warnings In-Reply-To: References: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net> Message-ID: Guido van Rossum wrote: > I think the Zope/Twisted interfaces have gone here before. I believe > they have a separate validation call which is typically only invoked > in unittests. The validation can be pretty expensive, so running the > validation each time the interpreter is started would just slow things > down. I've just not yet written the validation code. :-) You are correct. Normally interfaces are only validated in unit tests. The validation function of zope.interface checks if a class implements or object provides the promised methods and attributes. It also compares the method signatures. Christian From fdrake at acm.org Thu Feb 7 17:36:51 2008 From: fdrake at acm.org (Fred Drake) Date: Thu, 7 Feb 2008 11:36:51 -0500 Subject: [Python-3000] Should ABCMeta.register() do validation and emit warnings In-Reply-To: References: <20080206201712.AGL89528@ms19.lnh.mail.rcn.net> Message-ID: <0E557A86-41B2-495F-B610-68338F9FE47D@acm.org> On Feb 7, 2008, at 11:27 AM, Christian Heimes wrote: > You are correct. Normally interfaces are only validated in unit tests. > The validation function of zope.interface checks if a class implements > or object provides the promised methods and attributes. It also > compares > the method signatures. Note also that checking the "class implements" is frequently problematic since the verification doesn't know what the factory does to the object (where the factory is the constructor provided by the class in most cases). We frequently test instances this way in unit tests. -Fred -- Fred Drake From greg.ewing at canterbury.ac.nz Thu Feb 7 21:34:01 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 08 Feb 2008 09:34:01 +1300 Subject: [Python-3000] Should ABCMeta.register() do validation and emit warnings In-Reply-To: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net> References: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net> Message-ID: <47AB6B39.4090300@canterbury.ac.nz> Raymond Hettinger wrote: >>I think the Zope/Twisted interfaces have gone here before. >>I believe they have a separate validation call which is >>typically only invoked in unittests. > > +1 That seems like a good idea. Or maybe this is a task for pylint? -- Greg From fdrake at acm.org Thu Feb 7 21:46:34 2008 From: fdrake at acm.org (Fred Drake) Date: Thu, 7 Feb 2008 15:46:34 -0500 Subject: [Python-3000] Should ABCMeta.register() do validation and emit warnings In-Reply-To: <47AB6B39.4090300@canterbury.ac.nz> References: <20080206213537.AGL98594@ms19.lnh.mail.rcn.net> <47AB6B39.4090300@canterbury.ac.nz> Message-ID: <842C9510-415B-4959-A036-5C08C31BA4BC@acm.org> On Feb 7, 2008, at 3:34 PM, Greg Ewing wrote: > Or maybe this is a task for pylint? Maybe a lint tool could help, but it's not unusual for this to be relevant only for specific instances, not necessarily all instances. Certainly with zope.interface, you can create objects that provide a particular interface (and are recognized as providing it) without it being tied to their class. This is an important strength of the zope.interface module. Since it's entirely dependent on the dynamic behavior of the application, a lint tool will probably have a hard time with that. -Fred -- Fred Drake From greg.ewing at canterbury.ac.nz Thu Feb 7 21:52:18 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 08 Feb 2008 09:52:18 +1300 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> Message-ID: <47AB6F82.4070308@canterbury.ac.nz> Paul Moore wrote: > On Windows, GUI vs Console is a property of the EXE file, > and the differing behaviour is built into the OS loader. Would it be feasible for it to always start up as a gui app, and then create its own console window (a fake one if necessary) if it decides it needs one? Doing things this way might make it possible to be a bit smarter and not have the console window go away the moment the script ends. That tends to make console-mode scripts rather useless... Also, it would be very handy if in all cases it would display a traceback somehow in the event of an uncaught exception. It's frustrating when a gui app fails to start up and just vanishes into a black hole leaving no clue what went wrong. -- Greg From pje at telecommunity.com Thu Feb 7 23:10:36 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 07 Feb 2008 17:10:36 -0500 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <47AB6F82.4070308@canterbury.ac.nz> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> Message-ID: <20080207221027.261D33A40D7@sparrow.telecommunity.com> At 09:52 AM 2/8/2008 +1300, Greg Ewing wrote: >Would it be feasible for it to always start up as a gui >app, and then create its own console window (a fake one >if necessary) if it decides it needs one? No, because that would make it impossible to write a sane command-line app. Imagine if running 'ls' always created a *new* console window into which its results would be written... following which that console window would immediately close. That's what happens if a Windows GUI app creates its own console later. From p.f.moore at gmail.com Thu Feb 7 23:21:37 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 7 Feb 2008 22:21:37 +0000 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <20080207221027.261D33A40D7@sparrow.telecommunity.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> Message-ID: <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> On 07/02/2008, Phillip J. Eby wrote: > At 09:52 AM 2/8/2008 +1300, Greg Ewing wrote: > >Would it be feasible for it to always start up as a gui > >app, and then create its own console window (a fake one > >if necessary) if it decides it needs one? > > No, because that would make it impossible to write a sane > command-line app. Imagine if running 'ls' always created a *new* > console window into which its results would be written... following > which that console window would immediately close. > > That's what happens if a Windows GUI app creates its own console later. As Phillip says. Windows console and GUI executables are completely different in behaviour. Don't try to fight it. You need two distinct EXEs. (I can argue the pros and cons of Unix vs Windows behaviour all night. I'll take whichever side you prefer. The only thing that I am sure of is that trying to ignore the differences is always going to give you problems.) Paul. From greg.ewing at canterbury.ac.nz Fri Feb 8 03:59:23 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 08 Feb 2008 15:59:23 +1300 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> Message-ID: <47ABC58B.7010608@canterbury.ac.nz> Paul Moore wrote: > As Phillip says. Windows console and GUI executables are completely > different in behaviour. Don't try to fight it. You need two distinct > EXEs. Hmmm. So maybe what's needed is a *third* kind of exe that gets launched when you double-click a .py file, that keeps its console open after the script finishes? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From agthorr at barsoom.org Fri Feb 8 04:19:09 2008 From: agthorr at barsoom.org (Daniel Stutzbach) Date: Thu, 7 Feb 2008 21:19:09 -0600 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <47ABC58B.7010608@canterbury.ac.nz> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> Message-ID: On Feb 7, 2008 8:59 PM, Greg Ewing wrote: > Paul Moore wrote: > > As Phillip says. Windows console and GUI executables are completely > > different in behaviour. Don't try to fight it. You need two distinct > > EXEs. > > Hmmm. So maybe what's needed is a *third* kind of exe that > gets launched when you double-click a .py file, that keeps > its console open after the script finishes? > Picture command-line usage of python. You're sitting at your prompt, and you run a python script. It pops up a *new* window and you have to interact with that. Not a great user experience. Also, piping data to or from the script is impossible because stdin/stdout are tied to the new window. (I use Python for windows and pipe data to scripts everyday, so this is pretty important to me :-) ) I agree with Paul and Phillip that having two EXEs is necessary. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080207/b8e5c46d/attachment-0001.htm From pje at telecommunity.com Fri Feb 8 05:19:24 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 07 Feb 2008 23:19:24 -0500 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <47ABC58B.7010608@canterbury.ac.nz> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> Message-ID: <20080208041917.4390C3A4076@sparrow.telecommunity.com> At 03:59 PM 2/8/2008 +1300, Greg Ewing wrote: >Paul Moore wrote: > > As Phillip says. Windows console and GUI executables are completely > > different in behaviour. Don't try to fight it. You need two distinct > > EXEs. > >Hmmm. So maybe what's needed is a *third* kind of exe that >gets launched when you double-click a .py file, that keeps >its console open after the script finishes? That's actually a separate problem from the one Paul and I are talking about, one that I'm not even going to *try* to solve. (Being as I already have, and failed miserably.) Hell, I'm not sure Windows even has a way to *tell* whether your app was launched from the command line or otherwise. I couldn't find one for Win2K, anyway; there might be a way to do it in XP and Vista. From facundobatista at gmail.com Fri Feb 8 13:02:55 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 8 Feb 2008 10:02:55 -0200 Subject: [Python-3000] Windows gui vs. console In-Reply-To: References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> Message-ID: 2008/2/8, Daniel Stutzbach : > Picture command-line usage of python. You're sitting at your prompt, and > you run a python script. It pops up a *new* window and you have to interact > with that. Not a great user experience. Also, piping data to or from the > script is impossible because stdin/stdout are tied to the new window. (I > use Python for windows and pipe data to scripts everyday, so this is pretty > important to me :-) ) Not really following this discussion, but a detail regarding stdin/stdout in windows. When the process is issued as a Service, it don't even HAS stdin/stdout (not that is redirected to NUL). It don't even exist. Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From mwm at mired.org Wed Feb 6 20:07:43 2008 From: mwm at mired.org (Mike Meyer) Date: Wed, 6 Feb 2008 14:07:43 -0500 Subject: [Python-3000] [Python-ideas] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <47A99557.2000303@gmail.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <47A87A31.8070208@gmail.com> <20080205112232.34bc5c80@mbook-fbsd> <47A99557.2000303@gmail.com> Message-ID: <20080206140743.7aafa84b@mbook-fbsd> On Wed, 06 Feb 2008 21:09:11 +1000 Nick Coghlan wrote: > Mike Meyer wrote: > > On Wed, 06 Feb 2008 01:01:05 +1000 Nick Coghlan wrote: > >> While I'm +1 on the idea of making altinstall the default for setup.py > >> and the Windows installer (forcing the administrator to explicitly > >> request that the new Python version be made the default Python > >> installation for the machine), I'm a pretty strong -1 on officially > >> blessing the idea of a python3 alias or a special py3 extension on > >> Windows (end users are obviously free to create those if they want, of > >> course). > > I like the python3 alias (and python2 in 2.X), because it moves us > > from a single default python to having a default python2 and python3, > > which means scripts that care can ask for that without breaking the > > ability to upgrade python2 or python3 without breaking those scripts, > > while still allowing the default python to be either python2 or > > python3. > The main objection I have is that the executable name is only part of > the story - all of the PYTHON* environment variables may matter as well > (there are 9 listed in python -h, and I would have to check the code to > say with complete confidence that there aren't any others). True, there are 9 of them, but most of them are version-neutral, in that they change the behavior of python in some way, but *not* what scripts it finds. While you may want different versions of them for different python versions, 2 vs. 3 doesn't seem to be a factor (unless it ignores them and uses others): PYTHONOPTIMIZE, PYTHONDEBUG, PYTHONINSPECT, PYTHONUNBUFFERED, PYTHONVERBOSE and PYTHONY2K For that matter, all but PYTHON2K are an alternative to a command line option. I think we can safely ignore all of them. That leaves: PYTHONHOME - this tells python where to find it's standard libraries. Setting it is bad for your health and sanity unless you *really* know what you're doing. I think we can safely ignore it. PYTHONSTARTUP - generally only set per user, and only for interactive pythons. Anyone with the chops to set this up can make it work for both by checking the version and importing an appropriate version-specific module. PYTHONPATH - this one is the real issue. It's used by people who can't write to python library to point add third party modules to the standard path. Personally, I tend to install Python (after all, if I can install the modules, I can almost certainly install python) in that case, but that doesn't help them. If they have both installed on the system and can't write to their libraries, something has to be done to deal with it. > That said, while as a matter of purity I still think this is a platform > problem (one which all current platforms I know of handle poorly ;), as > a matter of practicality I'm seeing some benefits allowing a machine to > have full fledged "Python 2" and "Python 3" setups side by side. > > If Guido does decide to go the route of changing the interpreter binary > to python3 (ala sqlite vs sqlite3), I would also suggest that we go the > whole way and insert a '3' into all of the environment variable names > that python3 looks for (PYTHON3PATH, PYTHON3STARTUP, etc). I'd even > concede the utility of accepting the py3 extension on modules as a > concession to extension based dispatching on Windows. Actually, you can get that kind of behavior by using shell scripts instead of symlinks for python2 and python3 commands: #!/bin/sh PYTHONPATH=${PYTHON2PATH:-$PYTHONPATH} /usr/bin/python2.6 and similarly #!/bin/sh PYTHONPATH=${PYTHON3PATH:-$PYTHONPATH} /usr/bin/python3.0 Except you can't use #!/usr/bin/python2 in this case; you have to use #!/usr/bin/env python2, which might be a problem. > I still think it would be a bit of an ugly hack, but if it was followed > through to the point of being able to define separate Python 2 & 3 > environments without the use of launch scripts (i.e. by changing the > environment variable names as described above), maybe it would still be > a worthwhile thing to do. How about if we hide the "launch scripts" in the python2 and python3 commands, as above? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From mwm at mired.org Thu Feb 7 07:30:54 2008 From: mwm at mired.org (Mike Meyer) Date: Thu, 7 Feb 2008 01:30:54 -0500 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> <20080206012025.00a73dff@bhuda.mired.org> <87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20080207013054.459233bc@bhuda.mired.org> On Thu, 07 Feb 2008 11:48:08 +0900 "Stephen J. Turnbull" wrote: > Mike Meyer writes: > > On the other hand, if setup.py made setting up the symlinks easy, > > maybe I'd change my mind. > I think this should be a separate tool, but either way I'd not object > to making it easy to do, as long as it was not default. And this is the real issue with your approach. If you're doing things on systems you control, or where you're bundling in the python install you're going to use, then we can do it however you want, and things will be fine. People building an app for distribution are going to want it to run in an many places as possible. If the only name that python makes available by default is "python", that's what they're going to use. Removing the default for "python" is a step backwards. Most packagers and installers will add it back anyway, so many app developers will use it. Systems that fail to add it will appear broken. Apps that use a more specific shebang will, at the *very* best, break on exactly the same set of systems they'd break on using "python", and more likely break on a lot more. Leaving things as they are leaves things as they are. If we're going to break backwards comparability in the language, I think we should try and improve this situation. Adding an install target or tool to create version-specific links (of any flavor) suggests a solution, but I suspect it'll be ignored by a large percentage of platforms. So packagers for platform that don't ignore it can patch scripts to take advantage of it, but they tend to handle multiple versions cleanly in any case, so it's not that big a win for them. Developers wanting to distribute cross-platform scripts are still pretty much where they are now. Making the default install provide those links gives those developers an expectation that they can use them. If some packagers breaks that - well, the problem is that python on their system is broken. Yeah, some developers will ignore the links and keep using "python". But some - me, for instance - will provide a patch to users and tell them to complain to the maintainers of that package for their system. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From lists at cheimes.de Fri Feb 8 18:25:54 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 08 Feb 2008 18:25:54 +0100 Subject: [Python-3000] Windows gui vs. console In-Reply-To: References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> Message-ID: Facundo Batista wrote: > When the process is issued as a Service, it don't even HAS > stdin/stdout (not that is redirected to NUL). It don't even exist. GUI apps don't have standard streams stdin, stdout and stderr, too. The variables are defined when the code is compiled but during runtime the streams are not connected fileno(stdout) is -1. Christian From greg.ewing at canterbury.ac.nz Sat Feb 9 00:45:45 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 09 Feb 2008 12:45:45 +1300 Subject: [Python-3000] Windows gui vs. console In-Reply-To: References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> Message-ID: <47ACE9A9.10305@canterbury.ac.nz> Daniel Stutzbach wrote: > Hmmm. So maybe what's needed is a *third* kind of exe that > gets launched when you double-click a .py file, that keeps > its console open after the script finishes? > > Picture command-line usage of python. You're sitting at your prompt, > and you run a python script. It pops up a *new* window and you have to > interact with that. That's why I just suggested a *third* exe. It wouldn't be used as the command line interpreter -- it would only get launched by double-clicking a .py file in the gui. The current behaviour of having the gui launch the normal command line interpreter for .py files is almost useless, because the window disappears before you can see the output. -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 9 00:48:19 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 09 Feb 2008 12:48:19 +1300 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <20080208041917.4390C3A4076@sparrow.telecommunity.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> <20080208041917.4390C3A4076@sparrow.telecommunity.com> Message-ID: <47ACEA43.5050907@canterbury.ac.nz> Phillip J. Eby wrote: > Hell, I'm not sure Windows even > has a way to *tell* whether your app was launched from the command line > or otherwise. It wouldn't have to, if the .py extension were bound to a different executable from the standard command line interpreter, in the same way that .pyw is bound to the pythonw executable. -- Greg From stephen at xemacs.org Sat Feb 9 01:44:41 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 09 Feb 2008 09:44:41 +0900 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080207013054.459233bc@bhuda.mired.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> <20080206012025.00a73dff@bhuda.mired.org> <87prv9iis7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080207013054.459233bc@bhuda.mired.org> Message-ID: <878x1vnekm.fsf@uwakimon.sk.tsukuba.ac.jp> Mike Meyer writes: > On Thu, 07 Feb 2008 11:48:08 +0900 "Stephen J. Turnbull" wrote: > > I think this should be a separate tool, but either way I'd not object > > to making it easy to do, as long as it was not default. > > And this is the real issue with your approach. If you're doing things > on systems you control, or where you're bundling in the python install > you're going to use, then we can do it however you want, and things > will be fine. Please, give me some credit. I'm talking about a default for systems I don't control, but do want (a) to stop giving bad advice to and (b) to stop abetting their bad practices. So don't tell me that I'm mistaking my personal practices for a global optimum. I'm not. I've thought pretty carefully about how this applies in the context of presently widespread practices. But now that you've put that fallacy on the table, let me point out that it seems to apply to *you*. It's just that lots of other people have adopted the same global suboptimum, which makes it workable. As Opus the penguin says, "If a million people do the same stupid thing, it is still a stupid thing." Now, the "python" link is not *stupid* in the context of the economics of networks, but we can do better, and it won't cost much. > People building an app for distribution are going to want it to run in > an many places as possible. If the only name that python makes > available by default is "python", that's what they're going to use. First, you're wrong, as you immediately pointed out yourself. It's not *Python-provided* defaults that matter. It's the defaults provided by common distro vendors, which for the market in question dwarf installs from source as distributed by Python. Second, my point is to change that default *and the bug-prone behavior enabled by it*, so I'm not at all bothered by the fact that I'm changing the default and thus am incompatible with bug-prone behavior. > Leaving things as they are leaves things as they are. If we're going > to break backwards comparability in the language, I think we should > try and improve this situation. AFAICS my proposal *is* a technical improvement, in the sense that it will work substantially better and at lower total cost in breakage, debugging, and special-casing within communities that adopt it. You have made no argument against it except that it is a change and you don't like change, and then you vastly exaggerate the likely annoyance the removal of the "python" link creation step will cause, as well as the potential for actual breakage. True, it may get no uptake, and then the small annoyances that will occasionally happen are for nothing. But the upside is quite large compared to the small annoyances. > Adding an install target or tool to create version-specific links (of > any flavor) suggests a solution, but I suspect it'll be ignored by a > large percentage of platforms. I doubt they'll ignore it. They may decide not to implement it for the reasons you gave (stipulated and omitted). More likely, platforms like Debian with its /etc/alternatives mechanism will find a way to fit it into their existing alternatives mechanism. Either way, if Python adopts it for from-source installs, my proposal is compatible with such systems. If it works well, and as you suggest below app vendors who use it press the platform maintainers (directly or through customers as you suggest), platforms will adopt it -- we're talking a couple dozen people who do watch each other and diffuse innovations rapidly in the open source sector, and the proprietary vendors are rapidly becoming more nimble (and watching the open source sector) themselves. > Making the default install provide those links gives those developers > an expectation that they can use them. If some packagers breaks that - > well, the problem is that python on their system is broken. Yeah, some > developers will ignore the links and keep using "python". But some - > me, for instance - will provide a patch to users and tell them to > complain to the maintainers of that package for their system. That's the way it's supposed to work. I don't claim that my proposal will be entirely painless, but it won't be very painful. Application developers who *do* use it can get benefits *without* any cooperation from installed systems as long as users can install appropriate links (or wrappers on Windows) on their executable search PATH. The only problem you'll face is if the systems are doing something really insane like "ln -s perl5.8 python2.2". And of course Python or third parties can provide a link analysis tool for determining what versions of Python are available and where the binaries live (eg, borrow Zope's). From python at rcn.com Sat Feb 9 01:51:43 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 8 Feb 2008 19:51:43 -0500 (EST) Subject: [Python-3000] Nix dict.copy() Message-ID: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> I recommend dropping the dict.copy() method from Py3.0. * We can already write: newd = copy.copy(d). * We can also write: newd = dict(d) * Both of those approaches also work for most other containers. * The collections.Mapping ABC does not support copy(). * copy() is not a universal feature of mapping like objects * Lists do not have a copy() method. * If we drop dict.copy(), I'll can also drop set.copy() which is unneeded. Let's make the basic APIs as clean and parallel as possible. Raymond From guido at python.org Sat Feb 9 02:03:13 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 8 Feb 2008 17:03:13 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> Message-ID: On Feb 8, 2008 4:51 PM, Raymond Hettinger wrote: > I recommend dropping the dict.copy() method from Py3.0. > > * We can already write: newd = copy.copy(d). > * We can also write: newd = dict(d) > * Both of those approaches also work for most other containers. > * The collections.Mapping ABC does not support copy(). > * copy() is not a universal feature of mapping like objects > * Lists do not have a copy() method. > * If we drop dict.copy(), I'll can also drop set.copy() which is unneeded. > > Let's make the basic APIs as clean and parallel as possible. Makes sense. I request that you also implement the transitional code: either a fixer for 2to3 or (perhaps more realistically) a warning to dict.copy() when -3 is given. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sat Feb 9 02:35:57 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 8 Feb 2008 20:35:57 -0500 (EST) Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable Message-ID: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net> I'm wondering if collections.Iterable should inherit from collections.Container"? In Python, anything that supports __iter__ automatically supports tests using the "in" operator: class A: def __iter__(self): for i in (1,2,3): yield i >>> 2 in A() True The two concepts are deeply related. Anywhere we can write "for x in s: ..." we can also write "if x in s: ...". The new definition of Iterable would look like this: class Iterable(Container): __metaclass__ = ABCMeta @abstractmethod def __iter__(self): while False: yield None @classmethod def __subclasshook__(cls, C): if cls is Iterable: if any("__iter__" in B.__dict__ for B in C.__mro__): return True return NotImplemented def __contains__(self, value): for v in self: if v == value: return True return False Raymond From guido at python.org Sat Feb 9 02:42:37 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 8 Feb 2008 17:42:37 -0800 Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable In-Reply-To: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net> References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net> Message-ID: I would need to think more about this. I'm tempted not to do this, and let these ABCs denote the *explicit* presence of __contains__ and __iter__, respectively. Something that's iterable but doesn't implement __contains__ supports the 'in' operator very inefficiently (through linear search) which we might not want to encourage. --Guido On Feb 8, 2008 5:35 PM, Raymond Hettinger wrote: > I'm wondering if collections.Iterable should inherit from collections.Container"? > > In Python, anything that supports __iter__ automatically supports tests using the "in" operator: > > class A: > def __iter__(self): > for i in (1,2,3): > yield i > >>> 2 in A() > True > > The two concepts are deeply related. Anywhere we can write "for x in s: ..." we can also write "if x in s: ...". > > The new definition of Iterable would look like this: > > class Iterable(Container): > __metaclass__ = ABCMeta > > @abstractmethod > def __iter__(self): > while False: > yield None > > @classmethod > def __subclasshook__(cls, C): > if cls is Iterable: > if any("__iter__" in B.__dict__ for B in C.__mro__): > return True > return NotImplemented > > def __contains__(self, value): > for v in self: > if v == value: > return True > return False > > > Raymond > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sat Feb 9 02:59:05 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 09 Feb 2008 11:59:05 +1000 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <47ACE9A9.10305@canterbury.ac.nz> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> <47ACE9A9.10305@canterbury.ac.nz> Message-ID: <47AD08E9.2010806@gmail.com> Greg Ewing wrote: > Daniel Stutzbach wrote: >> Hmmm. So maybe what's needed is a *third* kind of exe that >> gets launched when you double-click a .py file, that keeps >> its console open after the script finishes? >> >> Picture command-line usage of python. You're sitting at your prompt, >> and you run a python script. It pops up a *new* window and you have to >> interact with that. > > That's why I just suggested a *third* exe. It wouldn't be > used as the command line interpreter -- it would only get > launched by double-clicking a .py file in the gui. > > The current behaviour of having the gui launch the normal > command line interpreter for .py files is almost useless, > because the window disappears before you can see the output. As far as I know, double-clicking in the GUI and typing the script name at a command prompt use the same mechanism to determine the file association. I'm not sure it is possible to separate them. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Sat Feb 9 03:23:24 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 09 Feb 2008 12:23:24 +1000 Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable In-Reply-To: References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net> Message-ID: <47AD0E9C.4080805@gmail.com> Guido van Rossum wrote: > I would need to think more about this. I'm tempted not to do this, and > let these ABCs denote the *explicit* presence of __contains__ and > __iter__, respectively. Something that's iterable but doesn't > implement __contains__ supports the 'in' operator very inefficiently > (through linear search) which we might not want to encourage. I think a bigger conceptual problem with doing containment tests on arbitrary iterables is that you may end up consuming an iterator in the process of checking whether it contains the item you want (i.e. if Iterable defines __contains__, then Iterator will acquire it by inheritance). So "assert (x in obj) == (x in obj)" may well fail in some cases if we adopt the idea that being iterable implies support for containment tests. Random thought inspired by the __contains__ code in Raymond's post: does anyone else think a 'key' parameter on any() and all() would be useful? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python at rcn.com Sat Feb 9 03:31:28 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 8 Feb 2008 21:31:28 -0500 (EST) Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable Message-ID: <20080208213128.AGP67107@ms19.lnh.mail.rcn.net> [Nick] > does anyone else think a 'key' parameter on any() and all() > would be useful? Not really. The key= function is useful whenever you need to keep the original value around for some reason (i.e. sorting it or getting the min/max value). But with any() and all() you don't keep anything. Also, if you take examples of any() and all() and try them with the key= syntax, you'll find that they become less readable. Besides, for the "gurus who are in the know", the min() and max() functions can often be substituted directly for any() and all() and then you get the key= syntax for free ;-) any(x%2==0 for x in s) max(x%2==0 for x in s) Raymond From digitalxero at gmail.com Sat Feb 9 03:45:35 2008 From: digitalxero at gmail.com (Dj Gilcrease) Date: Fri, 8 Feb 2008 18:45:35 -0800 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: <20080206012435.44bd22b7@bhuda.mired.org> References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> <20080206012435.44bd22b7@bhuda.mired.org> Message-ID: On Feb 5, 2008 10:24 PM, Mike Meyer wrote: > Sure, there are *lots* of ways the user can screw up the system as > root. Doing a default python install from source is *not* one of them, > no matter what it does with those symlinks. If you know of a system > where things are otherwise, please warn me about it now. Ubuntu will screw up if you login as root and build a newer python version then 2.2. You wont be able to use it's GUI package installer and other issues. From fdrake at acm.org Sat Feb 9 03:52:06 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 8 Feb 2008 21:52:06 -0500 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> Message-ID: On Feb 8, 2008, at 7:51 PM, Raymond Hettinger wrote: > I recommend dropping the dict.copy() method from Py3.0. +1 -Fred -- Fred Drake From ntoronto at cs.byu.edu Sat Feb 9 06:04:17 2008 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Fri, 08 Feb 2008 22:04:17 -0700 Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable In-Reply-To: References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net> Message-ID: <47AD3451.6000609@cs.byu.edu> Guido van Rossum wrote: > I would need to think more about this. I'm tempted not to do this, and > let these ABCs denote the *explicit* presence of __contains__ and > __iter__, respectively. Something that's iterable but doesn't > implement __contains__ supports the 'in' operator very inefficiently > (through linear search) which we might not want to encourage. It could be worse. Is a container necessarily finite? Neil From ntoronto at cs.byu.edu Sat Feb 9 06:30:08 2008 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Fri, 08 Feb 2008 22:30:08 -0700 Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable In-Reply-To: <47AD3451.6000609@cs.byu.edu> References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net> <47AD3451.6000609@cs.byu.edu> Message-ID: <47AD3A60.40404@cs.byu.edu> Neil Toronto wrote: > Guido van Rossum wrote: >> I would need to think more about this. I'm tempted not to do this, and >> let these ABCs denote the *explicit* presence of __contains__ and >> __iter__, respectively. Something that's iterable but doesn't >> implement __contains__ supports the 'in' operator very inefficiently >> (through linear search) which we might not want to encourage. > It could be worse. Is a container necessarily finite? I meant an iterable, of course, not a container. Neil From mwm at mired.org Sat Feb 9 07:35:13 2008 From: mwm at mired.org (Mike Meyer) Date: Sat, 9 Feb 2008 01:35:13 -0500 Subject: [Python-3000] Namespaces are one honking great idea -- let's do more of those! In-Reply-To: References: <994938.11662.qm@web31111.mail.mud.yahoo.com> <47A4F1D5.5080405@canterbury.ac.nz> <87oday96i8.fsf@uwakimon.sk.tsukuba.ac.jp> <1202119219.21638.4.camel@qrnik> <87ve55m0f7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204123057.1004e485@mbook-fbsd> <87wspkwbq7.fsf@uwakimon.sk.tsukuba.ac.jp> <20080204183807.412b14d8@bhuda.mired.org> <87r6fquawq.fsf@uwakimon.sk.tsukuba.ac.jp> <20080206012435.44bd22b7@bhuda.mired.org> Message-ID: <20080209013513.63a5a621@bhuda.mired.org> On Fri, 8 Feb 2008 18:45:35 -0800 "Dj Gilcrease" wrote: > On Feb 5, 2008 10:24 PM, Mike Meyer wrote: > > Sure, there are *lots* of ways the user can screw up the system as > > root. Doing a default python install from source is *not* one of them, > > no matter what it does with those symlinks. If you know of a system > > where things are otherwise, please warn me about it now. > Ubuntu will screw up if you login as root and build a newer python > version then 2.2. You wont be able to use it's GUI package installer > and other issues. Are you sure? The default Python install prefix is /usr/local. Ubuntu (at least on the admittedly aged system I have here) puts the system python in /usr/bin. So that default python install shouldn't break the system tools, providing they run the right python. For that matter, even on this relatively old Ubuntu, they've got a 2.4 version installed. Of course, once you've got more than one python installed on the system, chances are good that some - and possibly all - of your scripts will break if you run them on the wrong python. But even if the scripts run the default python (and system scripts shouldn't), the above can be fixed by tweaking the environment - and root's environment on my box is right. Thanks, http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. From ncoghlan at gmail.com Sat Feb 9 14:08:07 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 09 Feb 2008 23:08:07 +1000 Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable In-Reply-To: <20080208213128.AGP67107@ms19.lnh.mail.rcn.net> References: <20080208213128.AGP67107@ms19.lnh.mail.rcn.net> Message-ID: <47ADA5B7.9000408@gmail.com> Raymond Hettinger wrote: > Besides, for the "gurus who are in the know", the min() and max() > functions can often be substituted directly for any() and all() > and then you get the key= syntax for free ;-) > > any(x%2==0 for x in s) > max(x%2==0 for x in s) Except that you lose the short-circuiting behaviour by using the latter form. Anyway, for some reason the whole 'these work really well with generator expressions' aspect of any/all completely slipped my mind when I asked that question about a key argument. Sorry for the pointless noise... Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From suraj at barkale.com Fri Feb 8 18:38:52 2008 From: suraj at barkale.com (Suraj Barkale) Date: Fri, 8 Feb 2008 17:38:52 +0000 (UTC) Subject: [Python-3000] Windows gui vs. console References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > Paul Moore wrote: > > On Windows, GUI vs Console is a property of the EXE file, > > and the differing behaviour is built into the OS loader. > > Would it be feasible for it to always start up as a gui > app, and then create its own console window (a fake one > if necessary) if it decides it needs one? > This is not feasible if you are running default cmd.exe as your shell. See this blog post for details http://blogs.adobe.com/simplicity/2007/04/window_and_consolefriendly_win_1.html Regards, Suraj From python at rcn.com Sat Feb 9 22:33:57 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 9 Feb 2008 13:33:57 -0800 Subject: [Python-3000] Location of UserDict, UserList, and UserString Message-ID: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> In the process of getting UserDict updated to subclass from MutableMapping, it ended-up in the collections module during the transistion. Probably, all three should end-up in the same place, either in the collections module or in their own modules. What do you guys think? Do you like them in collections or as separate modules? Also, I'm wondering if it is time for UserString to bite the dust. It isn't really useful in the same way a UserDict or UserList which have two principal use cases. One, for testing to make sure that simulated dicts and lists are substitutable for real lists and dicts. Two, for convenience in subclassing where it's easier to write "self.data[k]" than "dict.__getitem__(self, k)" throughout. It seems that wherever people could have worked with UserString for a mutable string, in practice they prefer to work with lists and follow-up with a ''.join() afterwards. Raymond From steven.bethard at gmail.com Sat Feb 9 22:52:44 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 9 Feb 2008 14:52:44 -0700 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> Message-ID: On Feb 9, 2008 2:33 PM, Raymond Hettinger wrote: > In the process of getting UserDict updated to subclass from MutableMapping, > it ended-up in the collections module during the transistion. Probably, all three > should end-up in the same place, either in the collections module or in their own > modules. What do you guys think? Do you like them in collections or as > separate modules? +1 for putting them in the collections module. > Also, I'm wondering if it is time for UserString to bite the dust. It isn't really > useful in the same way a UserDict or UserList which have two principal use > cases. One, for testing to make sure that simulated dicts and lists are > substitutable for real lists and dicts. I thought this was the point of DictMixin, not UserDict, no? > Two, for convenience in subclassing where it's easier to write "self.data[k]" > than "dict.__getitem__(self, k)" throughout. Interesting. I'd never seen this before, but a Google code search reveals that it is reasonably common. Too bad. I was hoping that the new ABCs would eliminated the need for UserDict and UserList entirely... STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Sun Feb 10 02:27:31 2008 From: brett at python.org (Brett Cannon) Date: Sat, 9 Feb 2008 17:27:31 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> Message-ID: On Feb 8, 2008 5:03 PM, Guido van Rossum wrote: > On Feb 8, 2008 4:51 PM, Raymond Hettinger wrote: > > I recommend dropping the dict.copy() method from Py3.0. > > > > * We can already write: newd = copy.copy(d). > > * We can also write: newd = dict(d) > > * Both of those approaches also work for most other containers. > > * The collections.Mapping ABC does not support copy(). > > * copy() is not a universal feature of mapping like objects > > * Lists do not have a copy() method. > > * If we drop dict.copy(), I'll can also drop set.copy() which is unneeded. > > > > Let's make the basic APIs as clean and parallel as possible. > > Makes sense. I request that you also implement the transitional code: > either a fixer for 2to3 or (perhaps more realistically) a warning to > dict.copy() when -3 is given. +1 on the warning over the fixer. Since 2to3 can't do it perfectly I think it would be better to let users deal with it directly. -Brett From brett at python.org Sun Feb 10 02:32:12 2008 From: brett at python.org (Brett Cannon) Date: Sat, 9 Feb 2008 17:32:12 -0800 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> Message-ID: On Feb 9, 2008 1:33 PM, Raymond Hettinger wrote: > In the process of getting UserDict updated to subclass from MutableMapping, it ended-up in the collections module during the > transistion. Probably, all three should end-up in the same place, either in the collections module or in their own modules. What > do you guys think? Do you like them in collections or as separate modules? > +1 on collections. -Brett From python at rcn.com Sun Feb 10 02:43:04 2008 From: python at rcn.com (Raymond Hettinger) Date: Sat, 9 Feb 2008 17:43:04 -0800 Subject: [Python-3000] Location of UserDict, UserList, and UserString References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> Message-ID: <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> [Brett] > +1 on collections. [Steven Bethard] > +1 for putting them in the collections module. How about UserString? Is there a reason for it to continue to exist? Raymond From brett at python.org Sun Feb 10 02:48:41 2008 From: brett at python.org (Brett Cannon) Date: Sat, 9 Feb 2008 17:48:41 -0800 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> Message-ID: On Feb 9, 2008 5:43 PM, Raymond Hettinger wrote: > [Brett] > > +1 on collections. > > [Steven Bethard] > > +1 for putting them in the collections module. > > How about UserString? > Is there a reason for it to continue to exist? I have never used it so I don't have an opinion on it. But you know me, I am happy to trim the stdlib down. =) -Brett From foom at fuhm.net Sun Feb 10 02:52:41 2008 From: foom at fuhm.net (James Y Knight) Date: Sat, 9 Feb 2008 20:52:41 -0500 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> Message-ID: <39FB0D54-18FF-4B5C-B7A2-C10A8E16C564@fuhm.net> On Feb 9, 2008, at 8:27 PM, Brett Cannon wrote: > On Feb 8, 2008 5:03 PM, Guido van Rossum wrote: >> On Feb 8, 2008 4:51 PM, Raymond Hettinger wrote: >>> I recommend dropping the dict.copy() method from Py3.0. >>> >>> * We can already write: newd = copy.copy(d). >>> * We can also write: newd = dict(d) >>> * Both of those approaches also work for most other containers. >>> * The collections.Mapping ABC does not support copy(). >>> * copy() is not a universal feature of mapping like objects >>> * Lists do not have a copy() method. >>> * If we drop dict.copy(), I'll can also drop set.copy() which is >>> unneeded. >>> >>> Let's make the basic APIs as clean and parallel as possible. >> >> Makes sense. I request that you also implement the transitional >> code: >> either a fixer for 2to3 or (perhaps more realistically) a warning to >> dict.copy() when -3 is given. > > +1 on the warning over the fixer. Since 2to3 can't do it perfectly I > think it would be better to let users deal with it directly. Has anyone thought of making a execution-informed converter? If you have a comprehensive test suite, it seems that it ought be possible to use the actual execution of the test suite, under a trace hook perhaps, or something of that sort, to inform the converter and allow it to correctly make changes like this. James From guido at python.org Sun Feb 10 03:35:51 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 9 Feb 2008 18:35:51 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <39FB0D54-18FF-4B5C-B7A2-C10A8E16C564@fuhm.net> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <39FB0D54-18FF-4B5C-B7A2-C10A8E16C564@fuhm.net> Message-ID: On Feb 9, 2008 5:52 PM, James Y Knight wrote: > Has anyone thought of making a execution-informed converter? If you > have a comprehensive test suite, it seems that it ought be possible to > use the actual execution of the test suite, under a trace hook > perhaps, or something of that sort, to inform the converter and allow > it to correctly make changes like this. Yes, but it seems an overwhelmingly complex task given the available number of volunteers. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Sun Feb 10 03:51:42 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 9 Feb 2008 19:51:42 -0700 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> Message-ID: On Feb 9, 2008 6:43 PM, Raymond Hettinger wrote: > [Brett] > > +1 on collections. > > [Steven Bethard] > > +1 for putting them in the collections module. > > How about UserString? > Is there a reason for it to continue to exist? Like Brett, I've never found a use for it, so I'm happy to see it go. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From pje at telecommunity.com Sun Feb 10 05:58:35 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 09 Feb 2008 23:58:35 -0500 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> Message-ID: <20080210045826.651DD3A40B0@sparrow.telecommunity.com> At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote: >I recommend dropping the dict.copy() method from Py3.0. > >* We can already write: newd = copy.copy(d). >* We can also write: newd = dict(d) >* Both of those approaches also work for most other containers. I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) isn't. copy.copy() is polymorphic, but requires an import, which means the path of least resistance will be to use dict(d)... which will then turn non-dictionaries into dictionaries. If copy.copy() were a builtin and the "one obvious way" to copy containers, +1. Otherwise, -0.5. From solipsis at pitrou.net Sun Feb 10 13:53:53 2008 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 10 Feb 2008 12:53:53 +0000 (UTC) Subject: [Python-3000] Allocation of unicode objects Message-ID: Hi, Since there are discussions going on on the topic of allocation algorithms for various built-in types, I thought I'd mention there's a patch for turning unicode objects into variable-sized objects (rather than using a separately-allocated buffer). The aim is to make allocation of those objects lighter, and relieve cache and memory pressure a bit. http://bugs.python.org/issue1943 Marc-Andr? Lemburg expressed skepticism, based on the fact that it made subclassing unicode objects as part of C extensions more difficult. And here is a microbenchmark of the thing: Splitting a small string: ./python -m timeit -s "s=open('INTBENCH', 'r').read()" "s.split()" -> Unpatched py3k: 26.4 usec per loop -> PyVarObject patch: 20.2 usec per loop Splitting a medium-sized string: ./python -m timeit -s "s=open('LICENSE', 'r').read()" "s.split()" -> Unpatched py3k: 458 usec per loop -> PyVarObject patch: 316 usec per loop Splitting a long string: ./python -m timeit -s "s=open('Misc/HISTORY', 'r').read()" "s.split()" -> Unpatched py3k: 31.3 msec per loop -> PyVarObject patch: 17.8 msec per loop Even if the patch is rejected, I think it is important to remember that implementation characteristics of the unicode type will be crucial for Py3k performance :-) Regards Antoine. From shiblon at gmail.com Sun Feb 10 16:54:35 2008 From: shiblon at gmail.com (Chris Monson) Date: Sun, 10 Feb 2008 10:54:35 -0500 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080210045826.651DD3A40B0@sparrow.telecommunity.com> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: - Original message - I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is... Speaking of polymorphic, it seems like it would make sense to allow classes to specify their own __copy__ method for cases where specialized handling of copy is needed. copy.copy is sometimes overzealous, and therefore less efficient than necessary. Copy is a pretty fundamental idea, and I routinely run up against the fact that not only is there more than one way to do it (via constructors, using copy.copy, or when supported, thing.copy), the only "standard" approach requires an import and is weirdly one size fits all. Having a __copy__ slot and a builtin copy function with current copy.copy behavior as a fallback would be great. Just my accurately-valued 2 cents. - C Sent from Gmail for mobile On 2/9/08, Phillip J. Eby wrote: > At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote: > >I recommend dropping the dict.copy() method from Py3.0. > > > >* We can already write: newd = copy.copy(d). > >* We can also write: newd = dict(d) > >* Both of those approaches also work for most other containers. > > I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) isn't. > > copy.copy() is polymorphic, but requires an import, which means the > path of least resistance will be to use dict(d)... which will then > turn non-dictionaries into dictionaries. > > If copy.copy() were a builtin and the "one obvious way" to copy > containers, +1. Otherwise, -0.5. > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/shiblon%40gmail.com > -- Sent from Gmail for mobile | mobile.google.com From guido at python.org Sun Feb 10 17:22:38 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 10 Feb 2008 08:22:38 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080210045826.651DD3A40B0@sparrow.telecommunity.com> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: On Feb 9, 2008 8:58 PM, Phillip J. Eby wrote: > At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote: > >I recommend dropping the dict.copy() method from Py3.0. > > > >* We can already write: newd = copy.copy(d). > >* We can also write: newd = dict(d) > >* Both of those approaches also work for most other containers. > > I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) isn't. > > copy.copy() is polymorphic, but requires an import, which means the > path of least resistance will be to use dict(d)... which will then > turn non-dictionaries into dictionaries. > > If copy.copy() were a builtin and the "one obvious way" to copy > containers, +1. Otherwise, -0.5. On second thought, I'm with Phillip here, for exactly the reasons he states. Let's leave copy() alone. I'm of two minds about adding it to the MutableMapping and MutableSet ABCs; requiring it might constrain implementations unnecessary, and there isn't an easy way to provide a generic implementation. Now I wish we had "optional" abstract methods. Maybe those can be added to the ABC PEP yet? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Feb 10 17:24:32 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 10 Feb 2008 08:24:32 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: On Feb 10, 2008 7:54 AM, Chris Monson wrote: > Speaking of polymorphic, it seems like it would make sense to allow > classes to specify their own __copy__ method for cases where > specialized handling of copy is needed. copy.copy is sometimes > overzealous, and therefore less efficient than necessary. Good idea! I've fired up the time machine, and fixed the world. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Feb 10 17:27:52 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 10 Feb 2008 08:27:52 -0800 Subject: [Python-3000] Allocation of unicode objects In-Reply-To: References: Message-ID: On Feb 10, 2008 4:53 AM, Antoine Pitrou wrote: > Since there are discussions going on on the topic of allocation algorithms for > various built-in types, I thought I'd mention there's a patch for turning > unicode objects into variable-sized objects (rather than using a > separately-allocated buffer). The aim is to make allocation of those objects > lighter, and relieve cache and memory pressure a bit. > > http://bugs.python.org/issue1943 > > Marc-Andr? Lemburg expressed skepticism, based on the fact that it made > subclassing unicode objects as part of C extensions more difficult. Has anybody ever tried that? The same would apply to PyString and I've never heard this complaint. I think that given the relative importance of fast strings in Py3k vs. the convenience of subclassing PyUnicode, the latter may have to suffer. > And here is a microbenchmark of the thing: > > Splitting a small string: > ./python -m timeit -s "s=open('INTBENCH', 'r').read()" "s.split()" > -> Unpatched py3k: 26.4 usec per loop > -> PyVarObject patch: 20.2 usec per loop > > Splitting a medium-sized string: > ./python -m timeit -s "s=open('LICENSE', 'r').read()" "s.split()" > -> Unpatched py3k: 458 usec per loop > -> PyVarObject patch: 316 usec per loop > > Splitting a long string: > ./python -m timeit -s "s=open('Misc/HISTORY', 'r').read()" "s.split()" > -> Unpatched py3k: 31.3 msec per loop > -> PyVarObject patch: 17.8 msec per loop > > Even if the patch is rejected, I think it is important to remember that > implementation characteristics of the unicode type will be crucial for Py3k > performance :-) Right. I haven't had enough time to review this (or any other patch), but the idea is very appealing. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Sun Feb 10 18:33:21 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 10 Feb 2008 10:33:21 -0700 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: On Feb 10, 2008 9:22 AM, Guido van Rossum wrote: > On Feb 9, 2008 8:58 PM, Phillip J. Eby wrote: > > At 07:51 PM 2/8/2008 -0500, Raymond Hettinger wrote: > > >I recommend dropping the dict.copy() method from Py3.0. > > > > > >* We can already write: newd = copy.copy(d). > > >* We can also write: newd = dict(d) > > >* Both of those approaches also work for most other containers. > > > > I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) isn't. > > > > copy.copy() is polymorphic, but requires an import, which means the > > path of least resistance will be to use dict(d)... which will then > > turn non-dictionaries into dictionaries. > > > > If copy.copy() were a builtin and the "one obvious way" to copy > > containers, +1. Otherwise, -0.5. > > On second thought, I'm with Phillip here, for exactly the reasons he > states. Let's leave copy() alone. Maybe copy.copy() could become __builtin__.copy()? I know everyone hates adding builtins, but it is commonly needed functionality, it has __special_method__ support, and putting it in builtins would allow us to simplify the set() and dict() APIs... STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From skip at pobox.com Sun Feb 10 20:42:45 2008 From: skip at pobox.com (skip at pobox.com) Date: Sun, 10 Feb 2008 13:42:45 -0600 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: <18351.21429.600838.877694@montanaro-dyndns-org.local> Steven> Maybe copy.copy() could become __builtin__.copy()? And what of copy.depcopy()? Doesn't seem to be much else public defined in the copy module: % python -c 'import copy ; print copy.__all__' ['Error', 'copy', 'deepcopy'] If copy.copy() migrates to builtin it should raise something other than copy.Error as well. Skip From python at rcn.com Sun Feb 10 20:55:45 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 10 Feb 2008 11:55:45 -0800 Subject: [Python-3000] Nix dict.copy() References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> [Phillip Eby] > I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is... Can't say dict.copy() is really polymorphic if only one other class defines the method. The __copy__ method is more common. The only wrinkle is that the copy() function isn't a builtin. Raymond From larry at hastings.org Sun Feb 10 21:11:00 2008 From: larry at hastings.org (Larry Hastings) Date: Sun, 10 Feb 2008 12:11:00 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: <47AF5A54.1020701@hastings.org> Steven Bethard wrote: > Maybe copy.copy() could become __builtin__.copy()? +1 for exactly the reasons cited. I think copy() and deepcopy() should both be "essential" built-in functions. /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/1be73f4b/attachment.htm From lists at cheimes.de Sun Feb 10 21:49:03 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 10 Feb 2008 21:49:03 +0100 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <47AF5A54.1020701@hastings.org> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> Message-ID: Larry Hastings wrote: > +1 for exactly the reasons cited. I think copy() and deepcopy() should > both be "essential" built-in functions. I'm -0 on copy and -1 on deepcopy. If you need a copy or a deepcopy of an object (except dicts, lists and sets) you are most certainly using the wrong approach. Back when I was learning Python over 5 years ago I was using copy and deepcopy often. Nowadays I don't need the copy module anymore because I've learned how to design software without the need for copies. In my humble opinion the addition of copy to builtins leaves a bad mark. Christian From martin at v.loewis.de Sun Feb 10 21:55:09 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 10 Feb 2008 21:55:09 +0100 Subject: [Python-3000] Allocation of unicode objects In-Reply-To: References: Message-ID: <47AF64AD.3040204@v.loewis.de> > Since there are discussions going on on the topic of allocation algorithms for > various built-in types, I thought I'd mention there's a patch for turning > unicode objects into variable-sized objects (rather than using a > separately-allocated buffer). The aim is to make allocation of those objects > lighter, and relieve cache and memory pressure a bit. I find that fairly reasonable, so I'm +1 for this approach (haven't studied the patch, though). I always wondered why Unicode objects didn't work the way string objects work, to begin with. Regards, Martin From martin at v.loewis.de Sun Feb 10 22:01:42 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 10 Feb 2008 22:01:42 +0100 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> Message-ID: <47AF6636.70104@v.loewis.de> Raymond Hettinger wrote: > [Phillip Eby] >> I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is... > > Can't say dict.copy() is really polymorphic if only one other class defines the method. Why do you say it's only one? I found atleast UserDict.copy, os._Environ.copy, WeakValueDictionary.copy, WeakKeyDictionary.copy, and test.test_userdict.SeqDict. Notice that UserDict.copy is also polymorphic in the sense that it may return instances of subclasses, by using copy.copy. Regards, Martin From python at rcn.com Sun Feb 10 22:10:48 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 10 Feb 2008 13:10:48 -0800 Subject: [Python-3000] Nix dict.copy() References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> Message-ID: <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> >> [Phillip Eby] >>> I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) is... [Raymond] >> Can't say dict.copy() is really polymorphic if only one other class defines the method. [Martin] > Why do you say it's only one? I found atleast UserDict.copy, os._Environ.copy, WeakValueDictionary.copy, WeakKeyDictionary.copy, It would have been better for me to say that copy() seems to appear only in variants of dicts, but __copy__() is a long-standing defined protocol that is supported across non-mapping types as well. Underneath the hood, __copy__() and copy() have the same implementation, so it's really a question of whether we want two ways to say the same thing (with one of those ways being polymorphic across many objects and the other being used mainly for dict variants). Had copy() been a builtin, we probably wouldn't be having this discussion. Raymond From python at rcn.com Sun Feb 10 22:01:00 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 10 Feb 2008 13:01:00 -0800 Subject: [Python-3000] __reduce__() and __reduce_ex__() Message-ID: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1> In Py3.0, would it be worth it to simplify this protocol by combining these two methods? Always pass in a proto argument and the method can ignore it if it wants. Raymond From rhamph at gmail.com Sun Feb 10 22:39:22 2008 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 10 Feb 2008 14:39:22 -0700 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> Message-ID: On Feb 10, 2008 1:49 PM, Christian Heimes wrote: > Larry Hastings wrote: > > +1 for exactly the reasons cited. I think copy() and deepcopy() should > > both be "essential" built-in functions. > > I'm -0 on copy and -1 on deepcopy. > > If you need a copy or a deepcopy of an object (except dicts, lists and > sets) you are most certainly using the wrong approach. Back when I was > learning Python over 5 years ago I was using copy and deepcopy often. > Nowadays I don't need the copy module anymore because I've learned how > to design software without the need for copies. > > In my humble opinion the addition of copy to builtins leaves a bad mark. -1 on both. If a dict is wrong for some input, it's probably wrong for all inputs. If you're keeping this new dict for your own uses then you clearly want to keep it as your preferred type, and if you're returning it to your caller the preference for builtins is clearly want a single standardized return type (even filter does this in 3.0). I'm having a hard time imagining cases where you *really* want to return a copy of whatever the original type was, and this makes me think we shouldn't encourage that coding style. dict(iterable) for the one, and only one, obvious way to do it. -- Adam Olsen, aka Rhamphoryncus From martin at v.loewis.de Sun Feb 10 22:51:44 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 10 Feb 2008 22:51:44 +0100 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> Message-ID: <47AF71F0.4080301@v.loewis.de> >>>> I'm not fond of this idea. dict.copy() is polymorphic -- but dict(d) >>>> is... > > [Raymond] >>> Can't say dict.copy() is really polymorphic if only one other class >>> defines the method. > > [Martin] >> Why do you say it's only one? I found atleast UserDict.copy, >> os._Environ.copy, WeakValueDictionary.copy, WeakKeyDictionary.copy, > > It would have been better for me to say that copy() seems to appear only > in variants of dicts, but __copy__() is a long-standing defined protocol > that is supported across non-mapping types as well. But that's Phillip's point: dict(d) is not polymorphic, yet d.copy() is. That's important in applications where you want to copy a dict-like object polymorphically. So indeed, your claim that .copy "is [not] really polymorphic" is quite puzzling - we now seem to agree that it *is* polymorphic. So going back to your original message, you should remove dict(d) from the list of alternatives for dict.copy() (or remark that this will be a copy-to-basetype, so not quite the same thing). > Had copy() been a builtin, we probably wouldn't be having this discussion. Interestingly enough, dict.copy was added *after* copy.copy (in r8135), which supported __copy__ right from the beginning (since r4555). Even more interesting, it was added along with dict.absorb, which then got renamed dict.update a week later. No rationale is given in the checkin message on the purpose of the copy method. Regards, Martin From python at rcn.com Sun Feb 10 23:14:51 2008 From: python at rcn.com (Raymond Hettinger) Date: Sun, 10 Feb 2008 14:14:51 -0800 Subject: [Python-3000] Nix dict.copy() References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> <47AF71F0.4080301@v.loewis.de> Message-ID: <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1> [Martin] > So indeed, your claim that .copy "is [not] > really polymorphic" is quite puzzling - we now seem to agree that it > *is* polymorphic. Yes, of course. Every method used by dict imitators could be considered polymophic. I was just observing copy.copy(obj) applies to many, many more object types than obj.copy(). The former works with lists, tuples, deques, strings, and all kinds of things. The latter shows-up mainly in dict imitators. > So going back to your original message, you should remove dict(d) > from the list of alternatives for dict.copy() (or remark that this > will be a copy-to-basetype, so not quite the same thing). That makes sense. You can never be sure of a constructor's signature. It works most of the time but not all of the time. >> Had copy() been a builtin, we probably wouldn't be having this discussion. > > Interestingly enough, dict.copy was added *after* copy.copy (in r8135), > which supported __copy__ right from the beginning (since r4555). Even > more interesting, it was added along with dict.absorb, which then > got renamed dict.update a week later. No rationale is given in the > checkin message on the purpose of the copy method. Interesting history. Now, we have one and only one chance to slim-down the mapping API, leaving the copy() function as the one, universal, preferred way to do it. I don't think this chance will come again. Raymond From p.f.moore at gmail.com Mon Feb 11 00:46:25 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 10 Feb 2008 23:46:25 +0000 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> <47AF71F0.4080301@v.loewis.de> <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1> Message-ID: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com> On 10/02/2008, Raymond Hettinger wrote: > Now, we have one and only one chance to slim-down the mapping API, leaving the copy() > function as the one, universal, preferred way to do it. I don't think this chance will come > again. This discussion confused me (largely because of the subtle differences between the various ways of copying a dictionary). If I want a builtin dictionary from an arbitrary mapping d, I'd use dict(d.items()) - even if I knew d was a real dictionary. I wouldn't use d.copy() as that would not guarantee I had a builtin dict, should someone later pass me a user defined type. If I want a copy of an arbitrary mapping (or any object) and I want to preserve type, I would use copy.copy(). This is just as polymorphic as a copy method (interesting that it is Phillip arguing for methods being the polymorphic choice, when copy.copy is a generic function, which he has previously argued as being better...) I'm not sure how often I'd want to preserve type like this. It's never come up for me in real code. So I don't mind importing the copy module to get the functionality, it reminds me I'm doing something a little unusual. (On the other hand, (shallow) copy seems fundamental enough to me to qualify as a builtin, should someone care enough to argue for that). Given the above, I see no case where I'd want to use dict.copy(). So I'm in favour of removing it. On the other hand, I'd imagine it would be difficult to write a good fixer for this. I'm not sure how much of a showstopper that might be. Paul. From guido at python.org Mon Feb 11 01:18:09 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 10 Feb 2008 16:18:09 -0800 Subject: [Python-3000] __reduce__() and __reduce_ex__() In-Reply-To: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1> References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1> Message-ID: On Feb 10, 2008 1:01 PM, Raymond Hettinger wrote: > In Py3.0, would it be worth it to simplify this protocol by combining these two methods? Always pass in a proto argument and the > method can ignore it if it wants. Yeah -- the _ex suffix on the name suggests that that's the API we wanted but we'd already released a version with a suboptimal API... :-( There are other uses of 'ex' or 'Ex' on APIs too I believe (maybe in the C API). It may well be a convention I borrowed from Microsoft. Any of these that you find are likely cleanup candidates. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Mon Feb 11 01:24:00 2008 From: aahz at pythoncraft.com (Aahz) Date: Sun, 10 Feb 2008 16:24:00 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> Message-ID: <20080211002400.GA2720@panix.com> On Sun, Feb 10, 2008, Christian Heimes wrote: > Larry Hastings wrote: >> >> +1 for exactly the reasons cited. I think copy() and deepcopy() should >> both be "essential" built-in functions. > > I'm -0 on copy and -1 on deepcopy. > > If you need a copy or a deepcopy of an object (except dicts, lists and > sets) you are most certainly using the wrong approach. Back when I was > learning Python over 5 years ago I was using copy and deepcopy often. > Nowadays I don't need the copy module anymore because I've learned how > to design software without the need for copies. Let's suppose you have an object that represents an order. And let's suppose that this object needs to be copied to create a re-order (letting the customer preserve the information from the original order). How do you suggest that the re-order be created without copy(), and especially deepcopy()? What if an attribute of orders can be a list or dict? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From digitalxero at gmail.com Mon Feb 11 01:29:12 2008 From: digitalxero at gmail.com (Dj Gilcrease) Date: Sun, 10 Feb 2008 16:29:12 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> <47AF71F0.4080301@v.loewis.de> <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1> <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com> Message-ID: On Feb 10, 2008 3:46 PM, Paul Moore wrote: > On the other hand, I'd imagine it would be difficult to write a good > fixer for this. I'm not sure how much of a showstopper that might be. Cant you just flag it as depreciated and have it actually call copy.copy() when it is run. That way you dont need to write a fixer for it and can remove it in 3.1 From greg.ewing at canterbury.ac.nz Mon Feb 11 01:54:21 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 11 Feb 2008 13:54:21 +1300 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> <47AF71F0.4080301@v.loewis.de> <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1> <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com> Message-ID: <47AF9CBD.1050504@canterbury.ac.nz> Paul Moore wrote: > Given the above, I see no case where I'd want to use dict.copy(). So > I'm in favour of removing it. Personally I don't think I would even notice if the entire contents of the copy module disappeared. I've never used either form of generic copying. In the rare cases when I do want to copy something, I know what type I'm dealing with and have specific ideas on how to copy it, so I write code accordingly. Generic copying in Python seems like an anti-pattern to me, and not something that should be encouraged. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Mon Feb 11 01:57:18 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 11 Feb 2008 13:57:18 +1300 Subject: [Python-3000] __reduce__() and __reduce_ex__() In-Reply-To: References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1> Message-ID: <47AF9D6E.5080709@canterbury.ac.nz> Guido van Rossum wrote: > Yeah -- the _ex suffix on the name suggests that that's the API we > wanted but we'd already released a version with a suboptimal API... Hmmm, that convention isn't very scalable. What happens when you update the API again -- do you use an _ex_ex suffix? :-) Also it tends to make things look unpleasanty Microsoftish. :-( -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Mon Feb 11 02:01:15 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 11 Feb 2008 14:01:15 +1300 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080211002400.GA2720@panix.com> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> <20080211002400.GA2720@panix.com> Message-ID: <47AF9E5B.8010902@canterbury.ac.nz> Aahz wrote: > Let's suppose you have an object that represents an order. And let's > suppose that this object needs to be copied to create a re-order (letting > the customer preserve the information from the original order). In that case, I would give my Order class a copy() method (or re_order() or whatever name is appropriate) that does the right thing for orders in the context of that application. There are bound to be things that you *don't* want to copy from the original order, e.g. the order ID, the date... -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From guido at python.org Mon Feb 11 02:22:01 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 10 Feb 2008 17:22:01 -0800 Subject: [Python-3000] __reduce__() and __reduce_ex__() In-Reply-To: <47AF9D6E.5080709@canterbury.ac.nz> References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1> <47AF9D6E.5080709@canterbury.ac.nz> Message-ID: On Feb 10, 2008 4:57 PM, Greg Ewing wrote: > Guido van Rossum wrote: > > Yeah -- the _ex suffix on the name suggests that that's the API we > > wanted but we'd already released a version with a suboptimal API... > > Hmmm, that convention isn't very scalable. What happens > when you update the API again -- do you use an _ex_ex > suffix? :-) > > Also it tends to make things look unpleasanty Microsoftish. :-( Whine, whine. Do you have a better idea? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Feb 11 03:02:22 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 10 Feb 2008 21:02:22 -0500 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.co m> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> <47AF71F0.4080301@v.loewis.de> <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1> <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com> Message-ID: <20080211020218.E32A23A4075@sparrow.telecommunity.com> At 11:46 PM 2/10/2008 +0000, Paul Moore wrote: >If I want a copy of an arbitrary mapping (or any object) and I want to >preserve type, I would use copy.copy(). This is just as polymorphic as >a copy method (interesting that it is Phillip arguing for methods >being the polymorphic choice, when copy.copy is a generic function, >which he has previously argued as being better...) Actually, the issue here is the import, not methods vs. functions. Otherwise, I'd be arguing for ob.len(). :) And if we were starting from square one with no existing code, I'd probably be okay with needing to import copy, as long as that was the publicized One Obvious Way to copy all things. However, since existing code has to be migrated, and lots of things have copy() methods, and 2to3 isn't going to be able to tell, practicality (IMO) seems to favor keeping the existing method. From larry at hastings.org Mon Feb 11 03:24:56 2008 From: larry at hastings.org (Larry Hastings) Date: Sun, 10 Feb 2008 18:24:56 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> Message-ID: <47AFB1F8.8050803@hastings.org> Christian Heimes wrote: > Larry Hastings wrote: > >> +1 for exactly the reasons cited. I think copy() and deepcopy() should >> both be "essential" built-in functions. >> > > I'm -0 on copy and -1 on deepcopy. > > If you need a copy or a deepcopy of an object (except dicts, lists and > sets) you are most certainly using the wrong approach. My +1 isn't based primarily on how often I would use copy/deepcopy, as I don't need them very often. (Although I do use it *ever*, unlike many builtins--complex(), divmod(), and reversed() just to name three.) Instead, it's based on the observation that copy and deepcopy seem to be a first-class part of the language. The following built-in classes implement __copy__; those that also implement __deepcopy__ are marked with an asterisk: array.array* collections.defaultdict collections.deque decimal.Decimal xml.etree.ElementTree* itertools.tee re.match* re.pattern* All of those are implemented in C except for decimal.Decimal. This list was compiled from Python 3.0a2. I pass no judgment on programmers using copy/deepcopy. Though I note that, if you are passed a merely dict-like / list-like / set-like object, I would feel more confident in using copy.copy() to copy it than using dict() / list() / set() to copy it; enforcing the type this way seems so un-duck-typing to me. I suppose I could go fishing in the object to see if it implements a copy() method (dicts and sets do, lists don't), but then the deprecation/removal of .copy() methods in favor of copy.copy was how this whole thread got started. And, needless to say, there is simply no substitute for copy.deepcopy(), /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/25f1d4f7/attachment.htm From daniel at stutzbachenterprises.com Mon Feb 11 03:38:34 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 10 Feb 2008 20:38:34 -0600 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080211020218.E32A23A4075@sparrow.telecommunity.com> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <00b101c86c1e$ed42dd20$6800a8c0@RaymondLaptop1> <47AF6636.70104@v.loewis.de> <00e701c86c29$69745630$6800a8c0@RaymondLaptop1> <47AF71F0.4080301@v.loewis.de> <000701c86c32$5c0141d0$6800a8c0@RaymondLaptop1> <79990c6b0802101546g11cceb6fn7d1208e9ef9ccbc7@mail.gmail.com> <20080211020218.E32A23A4075@sparrow.telecommunity.com> Message-ID: On Feb 10, 2008 8:02 PM, Phillip J. Eby wrote: > However, since existing code has to be migrated, and lots of things > have copy() methods, and 2to3 isn't going to be able to tell, > practicality (IMO) seems to favor keeping the existing method. > If it only converts dict and set .copy() methods, 2to3 can't tell, but it wouldn't be too hard to respell all instances of foo.copy() as copy(foo). Obviously, this is a bit more radical and has good and bad points. It'd be giving every class's .copy method a promotion to __copy__, basically. Suggested 2to3 operation: foo.copy() => copy(foo) def copy(self): => def __copy__(self): (if within a class definition) If __copy__(self) is just "self.copy()", remove it. If copy(self) is just "self.__copy__()" or "copy.copy(self)", remove it If a class still defines both __copy__ and copy(), print a warning/error (this should be rare). If copy is called with arguments or defined with more than the "self" argument, make no change. If copy is defined with more than the "self" argument, but all the arguments are optional, print a warning/error. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/f4ff421b/attachment-0001.htm From lists at cheimes.de Mon Feb 11 02:14:19 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 11 Feb 2008 02:14:19 +0100 Subject: [Python-3000] __reduce__() and __reduce_ex__() In-Reply-To: <47AF9D6E.5080709@canterbury.ac.nz> References: <00e201c86c28$0ae2b360$6800a8c0@RaymondLaptop1> <47AF9D6E.5080709@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Hmmm, that convention isn't very scalable. What happens > when you update the API again -- do you use an _ex_ex > suffix? :-) How about a _ng suffix (next generation)? *scnr* Christian From aahz at pythoncraft.com Mon Feb 11 04:44:19 2008 From: aahz at pythoncraft.com (Aahz) Date: Sun, 10 Feb 2008 19:44:19 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <47AF9E5B.8010902@canterbury.ac.nz> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> <20080211002400.GA2720@panix.com> <47AF9E5B.8010902@canterbury.ac.nz> Message-ID: <20080211034419.GA10737@panix.com> On Mon, Feb 11, 2008, Greg Ewing wrote: > Aahz wrote: >> >> Let's suppose you have an object that represents an order. And let's >> suppose that this object needs to be copied to create a re-order (letting >> the customer preserve the information from the original order). > > In that case, I would give my Order class a copy() method (or > re_order() or whatever name is appropriate) that does the right > thing for orders in the context of that application. > > There are bound to be things that you *don't* want to copy from > the original order, e.g. the order ID, the date... Certainly -- that's why __copy__() exists, right? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From greg.ewing at canterbury.ac.nz Mon Feb 11 05:23:26 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 11 Feb 2008 17:23:26 +1300 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <20080211034419.GA10737@panix.com> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> <20080211002400.GA2720@panix.com> <47AF9E5B.8010902@canterbury.ac.nz> <20080211034419.GA10737@panix.com> Message-ID: <47AFCDBE.50804@canterbury.ac.nz> Aahz wrote: > On Mon, Feb 11, 2008, Greg Ewing wrote: > >>There are bound to be things that you *don't* want to copy from >>the original order, e.g. the order ID, the date... > > Certainly -- that's why __copy__() exists, right? Yes, but then I don't see the advantage over just giving the object a copy() method and calling it directly. In other words, I see little benefit in having copy() be a generic function. -- Greg From larry at hastings.org Mon Feb 11 08:55:43 2008 From: larry at hastings.org (Larry Hastings) Date: Sun, 10 Feb 2008 23:55:43 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <47AFCDBE.50804@canterbury.ac.nz> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> <20080211002400.GA2720@panix.com> <47AF9E5B.8010902@canterbury.ac.nz> <20080211034419.GA10737@panix.com> <47AFCDBE.50804@canterbury.ac.nz> Message-ID: <47AFFF7F.6090803@hastings.org> Greg Ewing wrote: > Yes, but then I don't see the advantage over just giving the object a copy() method and calling it directly. > > In other words, I see little benefit in having copy() be a generic function. > So true! Other candidates for not being generic functions: len(), repr(), str(), int(), complex(), dict(), list(), set()... So what decides whether something should be a generic function or a commonly-defined member function? Personally I find "historical accident" an unsatisfying answer. /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080210/ae20c536/attachment.htm From pje at telecommunity.com Mon Feb 11 18:56:15 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 11 Feb 2008 12:56:15 -0500 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <47AFCDBE.50804@canterbury.ac.nz> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <47AF5A54.1020701@hastings.org> <20080211002400.GA2720@panix.com> <47AF9E5B.8010902@canterbury.ac.nz> <20080211034419.GA10737@panix.com> <47AFCDBE.50804@canterbury.ac.nz> Message-ID: <20080211175606.4AE1C3A4075@sparrow.telecommunity.com> At 05:23 PM 2/11/2008 +1300, Greg Ewing wrote: >Aahz wrote: > > On Mon, Feb 11, 2008, Greg Ewing wrote: > > > >>There are bound to be things that you *don't* want to copy from > >>the original order, e.g. the order ID, the date... > > > > Certainly -- that's why __copy__() exists, right? > >Yes, but then I don't see the advantage over just >giving the object a copy() method and calling it >directly. > >In other words, I see little benefit in having >copy() be a generic function. Well, in the current circumstance, it helps because third parties can implement copying for things that didn't implement it themselves. From dalcinl at gmail.com Tue Feb 12 00:19:54 2008 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 11 Feb 2008 20:19:54 -0300 Subject: [Python-3000] Thoughts on collections.Container and collections.Iterable In-Reply-To: <47AD3A60.40404@cs.byu.edu> References: <20080208203557.AGP62362@ms19.lnh.mail.rcn.net> <47AD3451.6000609@cs.byu.edu> <47AD3A60.40404@cs.byu.edu> Message-ID: itertools.count() maybe ? Well not really infinite, it stops with OverflowError... But then (don't try this at home)... >>> 4 in itertools.cycle(range(3)) # ups! I seems that implicitely supporting containment testing for any iterable is not always good idea. On 2/9/08, Neil Toronto wrote: > Neil Toronto wrote: > > Guido van Rossum wrote: > >> I would need to think more about this. I'm tempted not to do this, and > >> let these ABCs denote the *explicit* presence of __contains__ and > >> __iter__, respectively. Something that's iterable but doesn't > >> implement __contains__ supports the 'in' operator very inefficiently > >> (through linear search) which we might not want to encourage. > > It could be worse. Is a container necessarily finite? > > I meant an iterable, of course, not a container. > > Neil > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/dalcinl%40gmail.com > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From python at rcn.com Tue Feb 12 01:43:54 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 19:43:54 -0500 (EST) Subject: [Python-3000] ABC for Inequalities Message-ID: <20080211194354.AGT26164@ms19.lnh.mail.rcn.net> Was some thought given to providing a mixin for boolean inequalities in total orderings (define __le__ and get the rest for free)? One of the motivating examples in the ABC pep was that the presence of __getitem__ was insufficient to distinguish between a sequence and a mapping. By registering one of the collections ABCs, a class can make an affirmative declaration that it is either a mapping or a sequence. It seems that there is a similar issue with inequalities. If something defines __le__, you don't necessarily know whether it returns a boolean (it could return an array of bools for a vector comparison). Even if a bool is returned, it is not clear whether it implies sortable ordering or whether is something with a completely different meaning (i.e. set.issubset). Raymond From guido at python.org Tue Feb 12 02:02:27 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 11 Feb 2008 17:02:27 -0800 Subject: [Python-3000] ABC for Inequalities In-Reply-To: <20080211194354.AGT26164@ms19.lnh.mail.rcn.net> References: <20080211194354.AGT26164@ms19.lnh.mail.rcn.net> Message-ID: On Feb 11, 2008 4:43 PM, Raymond Hettinger wrote: > Was some thought given to providing a mixin for boolean inequalities in total orderings (define __le__ and get the rest for free)? I think it was briefly mentioned but we didn't get beyond the stage of "you can do this with a tiny metaclass". There should be a way to avoid getting the rest for free too though, as the example of sets shows. There was also the issue of what to do if a subclass defines only __le__ and a superclass already defined all 6 operators. And there's also the issue that because of the way it's mapped to a single tp_richcompare slot in C, 'object' defines all 6 operators, but 4 of these raise exceptions. IOW it's messy. > One of the motivating examples in the ABC pep was that the presence of __getitem__ was insufficient to distinguish between a sequence and a mapping. By registering one of the collections ABCs, a class can make an affirmative declaration that it is either a mapping or a sequence. > > It seems that there is a similar issue with inequalities. If something defines __le__, you don't necessarily know whether it returns a boolean (it could return an array of bools for a vector comparison). Even if a bool is returned, it is not clear whether it implies sortable ordering or whether is something with a completely different meaning (i.e. set.issubset). I'm not sure I follow the connection with the above query, but I believe that for now the best approach here is to document the requirements for the comparisons. E.g. for Real numbers we require a total ordering based on the standard ordering for mathematical reals (insofar as practical in the light of inexactness and NaNs, of course), for Sequences I believe we specify that it should order item-wise, and for Sets we redefine the comparisons as set inclusion operators. A class whose __le__ returns an array or results should probably not derive directly from one of those standard ABCs. (Possibly the numpy folks might be interested in defining a different ABC for arrays whose '+' operator does elementwise addition instead of concatenation; but that's up to them.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Feb 12 02:45:01 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 20:45:01 -0500 (EST) Subject: [Python-3000] ABC for Inequalities Message-ID: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> >> Was some thought given to providing a mixin for boolean >> inequalities in total orderings (define __le__ and get >> the rest for free) [GvR] > IOW it's messy. Would it make sense to do something in numbers.py modeled after Exact/InExact? Those don't have any behavior; they just make a statement about semantics. Raymond --------------------------- class TotalOrdering(object): '''Equality and inequality operations on this type are guaranteed to return boolean values and to imply a total ordering where the relations are transitive (aa) == (not a>=b) == (not b<=a). ''' @abstractmethod def __eq__(self, other): pass @abstractmethod def __ne__(self, other): pass @abstractmethod def __lt__(self, other): pass @abstractmethod def __le__(self, other): pass @abstractmethod def __gt__(self, other): pass @abstractmethod def __ge__(self, other): pass From python at rcn.com Tue Feb 12 03:58:27 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 21:58:27 -0500 (EST) Subject: [Python-3000] What is numbers.Integral used for? Message-ID: <20080211215827.AGT42482@ms19.lnh.mail.rcn.net> During the discussion of PEP 3141, what use cases were envisioned for the Integral type? Were there some examples of concrete implementations other than or #t and (integer? 8/4) ==> #t but in our current implementation Rational(8,4) is not Integral like it is for Scheme. Also, Scheme seems apply implement the notions of Exact/Inexact much differently than we do: "A number is exact if it was written as an exact constant or was derived from exact numbers using only exact operations." So, while all integers are exact, some floats and decimal constants could also be exact. This matches how exactness is used in the decimal module -- all constants start as exact and operations can return either exact or inexact (and the module signals when this occurs). Our module says Integrals are Exact and makes no further assertions. Raymond From guido at python.org Tue Feb 12 04:08:58 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 11 Feb 2008 19:08:58 -0800 Subject: [Python-3000] ABC for Inequalities In-Reply-To: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> Message-ID: Hold on. We went down that road quite a bit and then discovered it wasn't going to work. The problem is that if you have two instance x, y such that isinstance(x, TotalOrdering) and isinstance(y, TotalOrdering) you still don't know if x < y is defined. (For example, if x is a string and y is a number.) We ended up deciding that such an ABC was not useful. There's a useful definition in there somewhere but it's elusive. We'll eventually figure it out. On Feb 11, 2008 5:45 PM, Raymond Hettinger wrote: > >> Was some thought given to providing a mixin for boolean > >> inequalities in total orderings (define __le__ and get > >> the rest for free) > > [GvR] > > IOW it's messy. > > Would it make sense to do something in numbers.py modeled > after Exact/InExact? Those don't have any behavior; they > just make a statement about semantics. > > > Raymond > > > --------------------------- > class TotalOrdering(object): > '''Equality and inequality operations on this type are guaranteed to > return boolean values and to imply a total ordering where the relations > are transitive (a usual interrelationships: (aa) == (not a>=b) == (not b<=a). > ''' > @abstractmethod > def __eq__(self, other): pass > > @abstractmethod > def __ne__(self, other): pass > > @abstractmethod > def __lt__(self, other): pass > > @abstractmethod > def __le__(self, other): pass > > @abstractmethod > def __gt__(self, other): pass > > @abstractmethod > def __ge__(self, other): pass > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 12 04:13:40 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 11 Feb 2008 19:13:40 -0800 Subject: [Python-3000] What is numbers.Integral used for? In-Reply-To: <20080211215827.AGT42482@ms19.lnh.mail.rcn.net> References: <20080211215827.AGT42482@ms19.lnh.mail.rcn.net> Message-ID: On Feb 11, 2008 6:58 PM, Raymond Hettinger wrote: > During the discussion of PEP 3141, what use cases were envisioned for the Integral type? Were there some examples of concrete implementations other than or I'm asking because I'm unclear about what a class is stating about itself when it registers as Integral. I'm not sure that most finite fields would fit into the structure since they all Integrals support logical operations, are convertible into ints via __index__, andy the must support all of the operations defined for Reals. An earlier version of the PEP talked about fields and all sort of stuff, but we found nobody cared. > I looked at the reference for Scheme's numeric tower to find guidance, but it seemed that the notion expressed solely through the "integer?" predicate which behaves differently than our inheritance tower. For example, the spec says (rational? 6/10) ==> #t and (integer? 8/4) ==> #t but in our current implementation Rational(8,4) is not Integral like it is for Scheme. > > Also, Scheme seems apply implement the notions of Exact/Inexact much differently > than we do: "A number is exact if it was written as an exact constant or was derived from exact numbers using only exact operations." So, while all integers are exact, some floats and decimal constants could also be exact. This matches how exactness is used in the decimal module -- all constants start as exact and operations can return either exact or inexact (and the module signals when this occurs). Our module says Integrals are Exact and makes no further assertions. I expect that there are separate useful notions of exactness and inexactness, one solely based on the type, and one based on value. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Feb 12 04:31:14 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 19:31:14 -0800 Subject: [Python-3000] ABC for Inequalities References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> Message-ID: <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> [GvR] > Hold on. We went down that road quite a bit and then discovered it > wasn't going to work. I figured that it was missing for a reason. > The problem is that if you have two instance x, > y such that isinstance(x, TotalOrdering) and isinstance(y, > TotalOrdering) you still don't know if x < y is defined. (For example, > if x is a string and y is a number.) So, ordering only makes sense when "other" is known to be of the same type as "self". > There's a useful definition in there somewhere but > it's elusive. We'll eventually figure it out. I hope so. I'm less interested in the mixin and more interested in a way for a class to have a way to tell you whether it is sortable. Right now, the presence or absence of __le__ doesn't tell you much. Raymond From python at rcn.com Tue Feb 12 04:58:55 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 19:58:55 -0800 Subject: [Python-3000] Nix dict.copy() References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> Message-ID: <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> Where did we end-up on dict.copy()? Is the copy(obj) function sufficiently universal to replace the obj.copy() method used in sets and in dict imitators? Or does the need for import preclude it from consideration? To me it seems that relationship between __copy__ and the copy() function is just as universal as the relationship between __len__ and len(). Am I missing something that makes d.copy() preferable to copy(d)? I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for dict.__copy__ and dict.copy share the same implementation). Raymond From python at rcn.com Tue Feb 12 05:10:29 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 20:10:29 -0800 Subject: [Python-3000] Location of UserDict, UserList, and UserString References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> Message-ID: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> Was the concensus on this to move UserList into the collections module and to remove UserString? Any objections? Raymond --------------------------------------------------- >> [Brett] >> > +1 on collections. >> >> [Steven Bethard] >> > +1 for putting them in the collections module. >> >> How about UserString? >> Is there a reason for it to continue to exist? > > Like Brett, I've never found a use for it, so I'm happy to see it go. > > Steve > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy From guido at python.org Tue Feb 12 05:41:55 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 11 Feb 2008 20:41:55 -0800 Subject: [Python-3000] ABC for Inequalities In-Reply-To: <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> Message-ID: On Feb 11, 2008 7:31 PM, Raymond Hettinger wrote: > [GvR] > > Hold on. We went down that road quite a bit and then discovered it > > wasn't going to work. > > I figured that it was missing for a reason. > > > The problem is that if you have two instance x, > > y such that isinstance(x, TotalOrdering) and isinstance(y, > > TotalOrdering) you still don't know if x < y is defined. (For example, > > if x is a string and y is a number.) > > So, ordering only makes sense when "other" is known to be > of the same type as "self". No, that's too far in the other direction. E.g. two instances of Real but of different implementations should still be comparable. (Though I expect there's some higher math involved to prove this for all reals. :-) > > There's a useful definition in there somewhere but > > it's elusive. We'll eventually figure it out. > > I hope so. I'm less interested in the mixin and more interested in > a way for a class to have a way to tell you whether it is sortable. > Right now, the presence or absence of __le__ doesn't tell > you much. Yeah, this is one area where a leap of faith (a.k.a. duck typing :-) may still be the best approach. I think you can probably get something that is reasonable given the kind of assumptions that prevail in an ABC world by having a "Sortable" metaclass (itself a subclass of ABCMeta) with the implied semantics that objects x and y can be sorted if the nearest common ancestor of x.__class__ and y.__class__ is an instance of Sortable. E.g. class S(metaclass=Sortable): ... class S0(S): def __lt__(self, other): ... def __le__(self, other): ... ...etc... class S1(S0): ... class S2(S0): ... x = S1() y = S2() Now x References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> Message-ID: On Feb 11, 2008 7:58 PM, Raymond Hettinger wrote: > Where did we end-up on dict.copy()? > > Is the copy(obj) function sufficiently universal to replace the obj.copy() method used in sets and in dict imitators? Or does the > need for import preclude it from consideration? > > To me it seems that relationship between __copy__ and the copy() function is just as universal as the relationship between __len__ > and len(). Am I missing something that makes d.copy() preferable to copy(d)? > > I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for > dict.__copy__ and dict.copy share the same implementation). I find the use of a copy operation that purports to copy any object suspect (regardless whether it's a module or a builtin), and that I'd rather have a copy() method on the MutableMapping API. I think my objection against the fully generic copy operation is that it's type is useless -- all we know that it is supposed to return the same type as its argument, but our type system can't express that. The type of the more constrained (but still generic in the sense of PEP 3124) copy operation on mutable sequences is better defined -- it returns another mutable sequence, with the understanding that it typically returns something of the same type as its input, though not always -- e.g. if I subclass dict but don't override .copy(), the inherited .copy() method returns a dict, not my subclass. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 12 05:48:27 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 11 Feb 2008 20:48:27 -0800 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> Message-ID: I don't know about consensus, but personally I'm +1 on this. On Feb 11, 2008 8:10 PM, Raymond Hettinger wrote: > Was the concensus on this to move UserList into the collections module and to remove UserString? > Any objections? > > > Raymond > > > > --------------------------------------------------- > > >> [Brett] > >> > +1 on collections. > >> > >> [Steven Bethard] > >> > +1 for putting them in the collections module. > >> > >> How about UserString? > >> Is there a reason for it to continue to exist? > > > > Like Brett, I've never found a use for it, so I'm happy to see it go. > > > > Steve > > -- > > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > > tiny blip on the distant coast of sanity. > > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Tue Feb 12 06:41:36 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 12 Feb 2008 06:41:36 +0100 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> Message-ID: Raymond Hettinger wrote: > Was the concensus on this to move UserList into the collections module and to remove UserString? > Any objections? You have my +1 as well. Christian From brett at python.org Tue Feb 12 07:10:34 2008 From: brett at python.org (Brett Cannon) Date: Mon, 11 Feb 2008 22:10:34 -0800 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> Message-ID: On Feb 11, 2008 8:10 PM, Raymond Hettinger wrote: > Was the concensus on this to move UserList into the collections module and to remove UserString? > Any objections? I ain't going to object. -Brett From python at rcn.com Tue Feb 12 07:20:31 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 22:20:31 -0800 Subject: [Python-3000] Nix dict.copy() References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> Message-ID: <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> >> I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for >> dict.__copy__ and dict.copy share the same implementation). > > I find the use of a copy operation that purports to copy any object > suspect (regardless whether it's a module or a builtin), and that I'd > rather have a copy() method on the MutableMapping API. Should I revert the -3 warning on copy() methods that I checked-in a few days ago? > I think my objection against the fully generic copy operation is that > it's type is useless -- all we know that it is supposed to return the > same type as its argument, but our type system can't express that. I'm not sure I follow you here. The definition of Iterator.__iter__ is to return self, so it doesn't seem like much of a leap to express a signature with a return type of type(self). Or, were you saying that the actual problem is that we can't know in advnace the signature of type(self).__init__()? > The > type of the more constrained (but still generic in the sense of PEP > 3124) copy operation on mutable sequences is better defined -- it > returns another mutable sequence, with the understanding that it > typically returns something of the same type as its input, though not > always -- e.g. if I subclass dict but don't override .copy(), the > inherited .copy() method returns a dict, not my subclass. The part I'm not following in this paragraph is how a d.copy() method is any different from the d.__copy__() method that is called by the copy() function. Both share the same underlying implementation and both have the same effects when overridden. If all we want a guaranteed dict, why not write dict(obj)? This already works with all the dict lookalikes because they define obj.keys() and obj.__getitem__(). Raymond From python at rcn.com Tue Feb 12 07:38:58 2008 From: python at rcn.com (Raymond Hettinger) Date: Mon, 11 Feb 2008 22:38:58 -0800 Subject: [Python-3000] ABC for Inequalities References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> Message-ID: <00d801c86d41$f2f36b80$6800a8c0@RaymondLaptop1> [GvR] > I think you can probably get something that is reasonable given the > kind of assumptions that prevail in an ABC world by having a > "Sortable" metaclass (itself a subclass of ABCMeta) with the implied > semantics that objects x and y can be sorted if the nearest common > ancestor of x.__class__ and y.__class__ is an instance of Sortable. > E.g. > > class S(metaclass=Sortable): ... > > class S0(S): > def __lt__(self, other): ... > def __le__(self, other): ... > ...etc... > > class S1(S0): ... > > class S2(S0): ... > > x = S1() > y = S2() > > Now x which is a subclass of S, which is an instance of Sortable. (If S is > an instance of Sortable, so is S0 -- that's how metaclasses are > inherited.) > > But I'm not sure this is worth it; we don't have an easy way to add > this to an argument annotation, since the argument annotations are > (usually) assumed to be type expressions so that f(x:T) means > isinstance(x,T) -- but here we want to express isinstance(x.__class__, > Sortable). The best I can think of is to say that if the annotation is > a lambda (not just any callable, since classes are callable) it must > evaluate to true when called on the actual argument: then we'd write > f(x: (lambda x: isinstance(x.__class__, Sortable))). Very ugly, but > could of course be abbreviated with a suitable helper function to f(x: > metaclass_is(Sortable)). Wow! That's a lot of firepower for such a little problem ;-) Raymond From ntoronto at cs.byu.edu Tue Feb 12 07:40:35 2008 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Mon, 11 Feb 2008 23:40:35 -0700 Subject: [Python-3000] ABC for Inequalities In-Reply-To: References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> Message-ID: <47B13F63.9010302@cs.byu.edu> Guido van Rossum wrote: > On Feb 11, 2008 7:31 PM, Raymond Hettinger wrote: >> [GvR] >>> The problem is that if you have two instance x, >>> y such that isinstance(x, TotalOrdering) and isinstance(y, >>> TotalOrdering) you still don't know if x < y is defined. (For example, >>> if x is a string and y is a number.) >> So, ordering only makes sense when "other" is known to be >> of the same type as "self". > > No, that's too far in the other direction. E.g. two instances of Real > but of different implementations should still be comparable. (Though I > expect there's some higher math involved to prove this for all reals. > :-) The problem is that, like all the binary operators, "orderable" is only defined for pairs of types. A single declaration of orderable-ness necessarily lacks information needed for type safety. >>> There's a useful definition in there somewhere but >>> it's elusive. We'll eventually figure it out. >> I hope so. I'm less interested in the mixin and more interested in >> a way for a class to have a way to tell you whether it is sortable. >> Right now, the presence or absence of __le__ doesn't tell >> you much. > > Yeah, this is one area where a leap of faith (a.k.a. duck typing :-) > may still be the best approach. > > I think you can probably get something that is reasonable given the > kind of assumptions that prevail in an ABC world by having a > "Sortable" metaclass (itself a subclass of ABCMeta) with the implied > semantics that objects x and y can be sorted if the nearest common > ancestor of x.__class__ and y.__class__ is an instance of Sortable. Besides probably not being "worth it" (as you say), forcing the classes into an inheritance hierarchy doesn't seem Pythonic. (I may not have the Python-fu to make such declarations, but I'll call 'em as I see 'em.) Also, x objects and y objects may be orderable homogeneously, but not heterogeneously. I could impose a total order on sets of circles and another on sets of squares without imposing an order on sets of both circles and squares, and not be able to spell this under the inheritance approach. To do it properly beyond duck typing, there should be a way to ask whether (and declare that) a *pair* of types implements an interface such as Orderable. This is analogous to how the binary operators currently work, though those are decentralized: the op/rop methods jointly decide what's legal. A little voice in my head is shouting "icky!" at the prospects. IIRC, one major motivation for ABCs was to avoid calling potentially destructive methods by allowing interface queries first. Does anyone expect comparisons to be destructive? Where would an "Orderable" query happen? Not before sorting a list - you'd have to scan the list to see what types are in it. (And declaring orderability for just pairs of types may not be enough then. Is orderability transitive?) Before each comparison? If that's the case, the only reason I can see for checking for Orderable is to keep people from doing dumb things. Neil From guido at python.org Tue Feb 12 18:56:58 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 12 Feb 2008 09:56:58 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: On Feb 11, 2008 10:20 PM, Raymond Hettinger wrote: > >> I don't think we'll ever get another chance to clean-up the mapping API and to remove duplicate functionality (the code for > >> dict.__copy__ and dict.copy share the same implementation). > > > > I find the use of a copy operation that purports to copy any object > > suspect (regardless whether it's a module or a builtin), and that I'd > > rather have a copy() method on the MutableMapping API. > > Should I revert the -3 warning on copy() methods that I checked-in a few days ago? Yes, I think we jumped the gun here a bit. > > I think my objection against the fully generic copy operation is that > > it's type is useless -- all we know that it is supposed to return the > > same type as its argument, but our type system can't express that. > > I'm not sure I follow you here. The definition of Iterator.__iter__ > is to return self, so it doesn't seem like much of a leap to express > a signature with a return type of type(self). Actually, __iter__ is defined on any Iterable, and it returns an Iterator; so after x = y.__iter__() (or x = iter(y) of course) you know that x is an Iterator, even if you know nothing about y's type (though if y is not an Iterable, the expresion may raise an exception). OTOH if you see x = copy(y), you don't know anything about x unless you know something about y; without knowing anything about y, all you know is that x is an object. Here's another way to look at it, maybe it'll help. My hunch is that whenever someone writes x = copy(y), they actually know more about the type of y than just that it is an object! Because if they knew *nothing* about it, they wouldn't know whether copying was (a) pointless (as it is for immutable objects), (b) impossible (e.g. for sockets), or (c) useful (e.g. for lists). When thinking about these kind of things I often place myself in the position of a static type checker that happens to know something about the types of some variables and is trying to gather more information. Seeing x=iter(y) can provide new information about y (it must be an Iterable). Seeing x=copy(y) provides no new information about y, since all (well, most) objects can be copied. OTOH seeing x=y.copy() *does* provide new information about y; it must be a type that has a .copy() method. Now seeing x=len(y) *does* provide new information about y (since we know which types support .__len__()). I realize I'm doing a lousy job of explaining me. Please trust me that there's something here. I will eventually get to it (maybe not today). > Or, were you saying that the actual problem is that we can't know > in advance the signature of type(self).__init__()? No, the __copy__ GF takes care of that concern. > > The > > type of the more constrained (but still generic in the sense of PEP > > 3124) copy operation on mutable sequences is better defined -- it > > returns another mutable sequence, with the understanding that it > > typically returns something of the same type as its input, though not > > always -- e.g. if I subclass dict but don't override .copy(), the > > inherited .copy() method returns a dict, not my subclass. > > The part I'm not following in this paragraph is how a d.copy() > method is any different from the d.__copy__() method that > is called by the copy() function. Both share the same underlying > implementation and both have the same effects when overridden. The difference is that *every* object has a .__copy__() method (it is inherited from object, at least conceptually), but not every object has a .copy() method. > If all we want a guaranteed dict, why not write dict(obj)? > This already works with all the dict lookalikes because they > define obj.keys() and obj.__getitem__(). Well, the whole point of this discussion that dict(obj) is what you should write if you want a guaranteed dict, and obj.copy() is what you should write if you want a guaranteed mutable mapping of the same type as obj. Just like you write list(obj) if you want a list and obj[:] if you want a sequence of the same type as obj. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 12 18:57:45 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 12 Feb 2008 09:57:45 -0800 Subject: [Python-3000] ABC for Inequalities In-Reply-To: <00d801c86d41$f2f36b80$6800a8c0@RaymondLaptop1> References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> <00d801c86d41$f2f36b80$6800a8c0@RaymondLaptop1> Message-ID: On Feb 11, 2008 10:38 PM, Raymond Hettinger wrote: > That's a lot of firepower for such a little problem ;-) Indeed, which is why it was not adopted. ;-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 12 19:02:25 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 12 Feb 2008 10:02:25 -0800 Subject: [Python-3000] ABC for Inequalities In-Reply-To: <47B13F63.9010302@cs.byu.edu> References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> <47B13F63.9010302@cs.byu.edu> Message-ID: On Feb 11, 2008 10:40 PM, Neil Toronto wrote: > The problem is that, like all the binary operators, "orderable" is only > defined for pairs of types. A single declaration of orderable-ness > necessarily lacks information needed for type safety. Actually, lots of languages have ways to express that the values of a given type are orderable. E.g. in C++ you can create a template that takes a single type argument which must define a total ordering using '<'. > Besides probably not being "worth it" (as you say), forcing the classes > into an inheritance hierarchy doesn't seem Pythonic. (I may not have the > Python-fu to make such declarations, but I'll call 'em as I see 'em.) You're forgetting about virtual inheritance. > Also, x objects and y objects may be orderable homogeneously, but not > heterogeneously. I could impose a total order on sets of circles and > another on sets of squares without imposing an order on sets of both > circles and squares, and not be able to spell this under the inheritance > approach. Yes, that was what I explained at length. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 12 19:10:53 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 12 Feb 2008 10:10:53 -0800 Subject: [Python-3000] Preventing merges into Py3k Message-ID: On Feb 12, 2008 9:44 AM, thomas.heller wrote: > Author: thomas.heller > Date: Tue Feb 12 18:44:23 2008 > New Revision: 60746 > > Modified: > python/branches/py3k/Modules/_ctypes/_ctypes.c > Log: > Revert the last svnmerge (r60681) from trunk to _ctypes.c, it should > not have been merged as was noticed in the commit message. There is a way to prevent merging a particular revision by instructing svnmerge properly. I believe the syntax is svnmerge block (svnmerge help block will explain you more). Christian has been using this -- Christian, care to post a detailed example? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Tue Feb 12 19:30:30 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 12 Feb 2008 11:30:30 -0700 Subject: [Python-3000] Location of UserDict, UserList, and UserString In-Reply-To: <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> References: <000501c86b63$7ac5d920$6800a8c0@RaymondLaptop1> <000c01c86b86$48393ec0$6800a8c0@RaymondLaptop1> <008c01c86d2d$34aba840$6800a8c0@RaymondLaptop1> Message-ID: On Feb 11, 2008 9:10 PM, Raymond Hettinger wrote: > Was the concensus on this to move UserList into the collections module and to remove UserString? > Any objections? No objections here either. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From lists at cheimes.de Tue Feb 12 20:41:43 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 12 Feb 2008 20:41:43 +0100 Subject: [Python-3000] Preventing merges into Py3k In-Reply-To: References: Message-ID: Guido van Rossum wrote: > There is a way to prevent merging a particular revision by instructing > svnmerge properly. I believe the syntax is svnmerge block (svnmerge > help block will explain you more). > > Christian has been using this -- Christian, care to post a detailed example? Yes, svnmerge.py block is correct. In order to prevent a merge you have to use svnmerge.py in a local copy of the py3k branch. Don't forget to submit the change! ;) cd .../py3k svnmerge.py block -r12345 svn ci -m "block r12345" Christian From greg.ewing at canterbury.ac.nz Tue Feb 12 23:05:10 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 13 Feb 2008 11:05:10 +1300 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: <47B21816.7050505@canterbury.ac.nz> Raymond Hettinger wrote: > The part I'm not following in this paragraph is how a d.copy() > method is any different from the d.__copy__() method that > is called by the copy() function. Both share the same underlying > implementation and both have the same effects when overridden. Having it a function/special method pair appears to make a strong statement that it's a very important operation that a large number of types ought to implement. As Guido said, it's not really a generic enough operation to warrant that. -- Greg From greg.ewing at canterbury.ac.nz Tue Feb 12 23:20:54 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 13 Feb 2008 11:20:54 +1300 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: <47B21BC6.9060309@canterbury.ac.nz> Guido van Rossum wrote: > Here's another way to look at it, maybe it'll help. My hunch is that > whenever someone writes x = copy(y), they actually know more about the > type of y than just that it is an object! I don't see how this is much different than the situation with __iter__. If you know that y is a sequence of items of type Y, then you know that iter(y) probably produces an object with a next() method that returns objects of type Y. If you're talking about programmers doing type analysis in their heads, I think the type system they use is quite a sophisticated one, certainly a parametric one that has no trouble coping with things like this. > I realize I'm doing a lousy job of explaining me. What's confusing is that you're talking about static typing issues in a language that has no static type system. I'm having trouble seeing the relevance. -- Greg From guido at python.org Tue Feb 12 23:32:30 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 12 Feb 2008 14:32:30 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: <47B21BC6.9060309@canterbury.ac.nz> References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> <47B21BC6.9060309@canterbury.ac.nz> Message-ID: On Feb 12, 2008 2:20 PM, Greg Ewing wrote: > Guido van Rossum wrote: > > > Here's another way to look at it, maybe it'll help. My hunch is that > > whenever someone writes x = copy(y), they actually know more about the > > type of y than just that it is an object! > > I don't see how this is much different than the situation > with __iter__. If you know that y is a sequence of items > of type Y, then you know that iter(y) probably produces > an object with a next() method that returns objects of > type Y. > > If you're talking about programmers doing type analysis > in their heads, I think the type system they use is quite > a sophisticated one, certainly a parametric one that has > no trouble coping with things like this. It depends. I used Google Code Search to find examples of copy.copy(). Many were hard to understand because there was no clue about the argument passed it, and the copy() call doesn't help me infer its type either. Contrast this with example code using iter() -- it tells me that the argument is an iterable. That rules out lots of possibilities, like numbers. > > I realize I'm doing a lousy job of explaining me. > > What's confusing is that you're talking about static > typing issues in a language that has no static type > system. I'm having trouble seeing the relevance. Trust me, there is a relevance to it in my head! It has to do with Python's ancestry from ABC, which did static type inference. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From daniel at stutzbachenterprises.com Tue Feb 12 23:40:10 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 12 Feb 2008 16:40:10 -0600 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: On Feb 12, 2008 11:56 AM, Guido van Rossum wrote: > Here's another way to look at it, maybe it'll help. My hunch is that > whenever someone writes x = copy(y), they actually know more about the > type of y than just that it is an object! Because if they knew > *nothing* about it, they wouldn't know whether copying was (a) > pointless (as it is for immutable objects), (b) impossible (e.g. for > sockets), or (c) useful (e.g. for lists). Based on the same reasoning, when we see x = copy(y) we can tentatively assume that y is a type that can usefully be copied. Unless we don't trust the author of the code to be sensible. > Seeing x=copy(y) provides no new information about y, since > all (well, most) objects can be copied. OTOH seeing x=y.copy() *does* > provide new information about y; it must be a type that has a .copy() > method. What useful information do we get by knowing that a type has a .copy() method? If I took the last sentence out of context, I would assume that having a .copy() method informs you that the type is copyable. Except you've just said that all (well, most) objects in Python can be copied--so I'm not sure new information a .copy() method conveys. Normally I might assume that it does a special kind of copying that doesn't fit the definition of copy.copy() nor copy.deepcopy(), except that for dict and set types, .copy() and .__copy__() are identical. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080212/525d4a0e/attachment.htm From greg.ewing at canterbury.ac.nz Tue Feb 12 23:39:13 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 13 Feb 2008 11:39:13 +1300 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> <47B21BC6.9060309@canterbury.ac.nz> Message-ID: <47B22011.7070906@canterbury.ac.nz> Guido van Rossum wrote: > I used Google Code Search to find examples of copy.copy(). > Many were hard to understand because there was no clue about the > argument passed it, and the copy() call doesn't help me infer its type > either. Contrast this with example code using iter() -- it tells me > that the argument is an iterable. That rules out lots of > possibilities, like numbers. Hmmm. Personally I'm not sure that such ad-hoc piecemeal type inference is all that useful. Just knowing that something is an iterable still doesn't tell me anything about the type of the items it contains, which is likely to be of much more interest to me. Similarly, just knowing that something has a copy() method doesn't narrow things down all that much. It might be a dict, or a set, or something else the user has come up with, which could be just about anything. -- Greg From guido at python.org Tue Feb 12 23:52:04 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 12 Feb 2008 14:52:04 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: On Feb 12, 2008 2:40 PM, Daniel Stutzbach > Based on the same reasoning, when we see x = copy(y) we can tentatively > assume that y is a type that can usefully be copied. Unless we don't trust > the author of the code to be sensible. My point is that (almost) any type is copyable so we can't tell what the author is assuming. Maybe they believe they had either a list or a dict -- or maybe they know they have an arbitrary class instance. > > Seeing x=copy(y) provides no new information about y, since > > all (well, most) objects can be copied. OTOH seeing x=y.copy() *does* > > provide new information about y; it must be a type that has a .copy() > > method. > > What useful information do we get by knowing that a type has a .copy() > method? It rules out all classes that don't have one. That's nearly all types. > If I took the last sentence out of context, I would assume that having a > .copy() method informs you that the type is copyable. Except you've just > said that all (well, most) objects in Python can be copied--so I'm not sure > new information a .copy() method conveys. Only dict and a few mapping-like types (and a vanishingly small number of user-defined classes) define copy(). > Normally I might assume that it does a special kind of copying that doesn't > fit the definition of copy.copy() nor copy.deepcopy(), except that for dict > and set types, .copy() and .__copy__() are identical. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From daniel at stutzbachenterprises.com Wed Feb 13 00:11:09 2008 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 12 Feb 2008 17:11:09 -0600 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: On Feb 12, 2008 4:52 PM, Guido van Rossum wrote: > > What useful information do we get by knowing that a type has a .copy() > > method? > > It rules out all classes that don't have one. That's nearly all types. > Okay, but turn it around for a minute. Which types should have a .copy() method and why? If I'm making a new copyable type, is there a rule of thumb that helps me decide on .copy(), .__copy__(), or both? Rules of "dict and set have a .copy() method" or "mapping types have a .copy() method" seem... arbitrary. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20080212/b0f2e170/attachment.htm From guido at python.org Wed Feb 13 00:15:31 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 12 Feb 2008 15:15:31 -0800 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: On Feb 12, 2008 3:11 PM, Daniel Stutzbach wrote: > On Feb 12, 2008 4:52 PM, Guido van Rossum wrote: > > > > > > What useful information do we get by knowing that a type has a .copy() > > > method? > > > > It rules out all classes that don't have one. That's nearly all types. > > > > Okay, but turn it around for a minute. Which types should have a .copy() > method and why? > > If I'm making a new copyable type, is there a rule of thumb that helps me > decide on .copy(), .__copy__(), or both? > > Rules of "dict and set have a .copy() method" or "mapping types have a > .copy() method" seem... arbitrary. I can't answer those questions in the abstract; I'd have to see the type you are trying to create and the use case for creating copies. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Feb 14 00:34:47 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 14 Feb 2008 12:34:47 +1300 Subject: [Python-3000] ABC for Inequalities In-Reply-To: <47B13F63.9010302@cs.byu.edu> References: <20080211204501.AGT33998@ms19.lnh.mail.rcn.net> <003201c86d27$b94c5a00$6800a8c0@RaymondLaptop1> <47B13F63.9010302@cs.byu.edu> Message-ID: <47B37E97.5010409@canterbury.ac.nz> Neil Toronto wrote: > The problem is that, like all the binary operators, "orderable" is only > defined for pairs of types. I'm loathe to mention it, but it sounds like what's needed for this is generic functions on types... -- Greg From van.lindberg at gmail.com Thu Feb 14 16:13:22 2008 From: van.lindberg at gmail.com (VanL) Date: Thu, 14 Feb 2008 09:13:22 -0600 Subject: [Python-3000] Nix dict.copy() In-Reply-To: References: <20080208195143.AGP58228@ms19.lnh.mail.rcn.net> <20080210045826.651DD3A40B0@sparrow.telecommunity.com> <006d01c86d2b$975bdb60$6800a8c0@RaymondLaptop1> <00bb01c86d3f$5f610460$6800a8c0@RaymondLaptop1> Message-ID: Daniel Stutzbach wrote: > Okay, but turn it around for a minute. Which types should have a > .copy() method and why? I would argue that it's not about mappings, it's about mutability. I always thought the .copy method on dicts was to allow functions to work on a passed-in dict without causing spooky action at a distance due to mutability issues. (See the silly example at the end of the post.) Sets, being by default mutable, got the same treatment. Other types (other than lists) aren't susceptible to this sort of problem. The default idiom for lists, copy = list[:], is a workaround for the lack of a .copy() method. I would argue that if you want to make some sort of ABC, make it a 'Mutable' ABC and put the .copy method in *there*. \ - Van *For example, you can get into this problem when using dicts for string substitution: def fragment1(vars, boilerplate=''): if boilerplate: vars['extra'] = boilerplate return "%(sentence)s with %(something)s %(extra)s." % vars def fragment2(vars, boilerplate=''): if boilerplate: vars['extra'] = boilerplate return "%(another)s %(sentence)s with %(extra)s." % vars def statement(): # vars is a dict built from a database query vars = get_dict_from_db() return '%s %s' % (fragment1(vars, 'extra'), fragment2(vars)) Because of mutability, the 'extra' in fragment1 will also show up in fragment2. The solution to this is to make a .copy(). The susceptibility of newbies to this 'spooky action at a distance' problem is one of the reasons why mutability and immutability have to be addressed almost at the very start of many learning Python books. From skip at pobox.com Thu Feb 14 22:09:05 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 14 Feb 2008 15:09:05 -0600 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? Message-ID: <18356.44529.51137.327615@montanaro-dyndns-org.local> A thread on the Python Mac sig got me to wondering if there is any magic in Python 3's print function for printing Unicode. Nope, no magic: >>> print("\xef") Traceback (most recent call last): File "", line 1, in File "/Users/skip/local/lib/python3.0/io.py", line 1246, in write b = encoder.encode(s) File "/Users/skip/local/lib/python3.0/encodings/ascii.py", line 22, in encode return codecs.ascii_encode(input, self.errors)[0] UnicodeEncodeError: 'ascii' codec can't encode character '\xef' in position 0: ordinal not in range(128) Which kind of confuses me because on my system my default encoding is utf-8: >>> sys.getdefaultencoding() 'utf-8' Well, then how about an encoding arg? Nope again: >>> print("\xef", encoding="utf-8") Traceback (most recent call last): File "", line 1, in TypeError: 'encoding' is an invalid keyword argument for this function help(print) doesn't offer any suggestions either. Any chance that maybe an encoding keyword arg could make it into an upcoming 3.0aN release? (Or... Am I missing a simple solution to the problem?) It seems that printing Unicode ought to be easier in Python 3 than it is. Skip From guido at python.org Thu Feb 14 23:02:31 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 14 Feb 2008 14:02:31 -0800 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <18356.44529.51137.327615@montanaro-dyndns-org.local> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> Message-ID: The encoding is a file property, not a print parameter. The code that guesses the default stdout encoding isn't very good, especially not on OSX. Suggestions are welcome. You can hack around this for now by doing (before printing anything) sys.stdout._encoding = 'utf-8' but that ought to be a temporary hack until we've figured out the right way to set it. --Guido On Thu, Feb 14, 2008 at 1:09 PM, wrote: > A thread on the Python Mac sig got me to wondering if there is any magic in > Python 3's print function for printing Unicode. Nope, no magic: > > >>> print("\xef") > Traceback (most recent call last): > File "", line 1, in > File "/Users/skip/local/lib/python3.0/io.py", line 1246, in write > b = encoder.encode(s) > File "/Users/skip/local/lib/python3.0/encodings/ascii.py", line 22, in encode > return codecs.ascii_encode(input, self.errors)[0] > UnicodeEncodeError: 'ascii' codec can't encode character '\xef' in position 0: ordinal not in range(128) > > Which kind of confuses me because on my system my default encoding is utf-8: > > >>> sys.getdefaultencoding() > 'utf-8' > > Well, then how about an encoding arg? Nope again: > > >>> print("\xef", encoding="utf-8") > Traceback (most recent call last): > File "", line 1, in > TypeError: 'encoding' is an invalid keyword argument for this function > > help(print) doesn't offer any suggestions either. Any chance that maybe an > encoding keyword arg could make it into an upcoming 3.0aN release? > (Or... Am I missing a simple solution to the problem?) It seems that > printing Unicode ought to be easier in Python 3 than it is. > > Skip > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Thu Feb 14 23:21:40 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 14 Feb 2008 23:21:40 +0100 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> Message-ID: <47B4BEF4.5040206@v.loewis.de> > The code that guesses the default stdout encoding isn't very good, > especially not on OSX. Suggestions are welcome. Unfortunately, Apple isn't very cooperative here. There is no way of determining the encoding of a Terminal.app window, AFAIK (it's normally UTF-8, unless the user has changed it). Regards, Martin From guido at python.org Thu Feb 14 23:38:44 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 14 Feb 2008 14:38:44 -0800 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <47B4BEF4.5040206@v.loewis.de> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <47B4BEF4.5040206@v.loewis.de> Message-ID: In Leopard (OSX 10.5), it looks like LANG is set to convey this: it has the value en_US.UTF-8 for me, and I haven't done anything to change it. The Preferences dialog is completely changed from 10.4 and there's a checkbox (on by default AFAIK) to set the LANG variable. (However I've noticed that it doesn't set an encoding when I specify another encoding other than UTF-8 -- perhaps because there is interaction with the Language choice.) In 10.4 LANG isn't set at all. The effect is that in 10.5, sys.stdout.encoding defaults to UTF-8 but in 10.4 to US-ASCII. Maybe a decent rule of thumb would be: - if LANG is set, use the encoding it specifies; if there's no encoding in it, assume it's ASCII. - if LANG is *not* set, default to UTF-8 instead of to US-ASCII; UTF-8 is much more likely to be correct and useful for 10.4 and before. --Guido On Thu, Feb 14, 2008 at 2:21 PM, "Martin v. L?wis" wrote: > > The code that guesses the default stdout encoding isn't very good, > > especially not on OSX. Suggestions are welcome. > > Unfortunately, Apple isn't very cooperative here. There is no way > of determining the encoding of a Terminal.app window, AFAIK > (it's normally UTF-8, unless the user has changed it). > > Regards, > Martin > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Feb 14 23:48:12 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 14 Feb 2008 17:48:12 -0500 (EST) Subject: [Python-3000] PEP 3102 Message-ID: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> I've been exercising the new keyword-only arguments syntax. It is absolutely wonderful. I'm amazed at how many long standing problems it solves elegantly. Thanks Talin! Raymond From martin at v.loewis.de Thu Feb 14 23:59:04 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 14 Feb 2008 23:59:04 +0100 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <47B4BEF4.5040206@v.loewis.de> Message-ID: <47B4C7B8.7020009@v.loewis.de> > Maybe a decent rule of thumb would be: > > - if LANG is set, use the encoding it specifies; if there's no > encoding in it, assume it's ASCII. > > - if LANG is *not* set, default to UTF-8 instead of to US-ASCII; UTF-8 > is much more likely to be correct and useful for 10.4 and before. You mean, on OSX? Sounds fine to me. Some people will see moji-bake with this surely; they'll have to set LANG themselves. (IOW, ASCII is more likely to be correct, but UTF-8 is more likely to be useful). Regards, Martin From guido at python.org Fri Feb 15 00:13:21 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 14 Feb 2008 15:13:21 -0800 Subject: [Python-3000] PEP 3102 In-Reply-To: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> Message-ID: On Thu, Feb 14, 2008 at 2:48 PM, Raymond Hettinger wrote: > I've been exercising the new keyword-only arguments syntax. It is absolutely wonderful. I'm amazed at how many long standing problems it solves elegantly. Agreed. Now can you come up with a syntax for positional-only arguments? So far everybody has failed at that, and there are some use cases where it's useful too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Feb 15 00:14:34 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 14 Feb 2008 15:14:34 -0800 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <47B4C7B8.7020009@v.loewis.de> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <47B4BEF4.5040206@v.loewis.de> <47B4C7B8.7020009@v.loewis.de> Message-ID: On Thu, Feb 14, 2008 at 2:59 PM, "Martin v. L?wis" wrote: > > Maybe a decent rule of thumb would be: > > > > - if LANG is set, use the encoding it specifies; if there's no > > encoding in it, assume it's ASCII. > > > > - if LANG is *not* set, default to UTF-8 instead of to US-ASCII; UTF-8 > > is much more likely to be correct and useful for 10.4 and before. > > You mean, on OSX? Sounds fine to me. Right. > Some people will see moji-bake with this surely; they'll have to set > LANG themselves. > > (IOW, ASCII is more likely to be correct, but UTF-8 is more likely > to be useful). Right. You can't completely rule out moji-bake -- people can set LANG to an incorrect value too. Can anyone supply a patch to open() in io.py? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From stephen at xemacs.org Fri Feb 15 01:06:27 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 15 Feb 2008 09:06:27 +0900 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> Message-ID: <87abm36q2k.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > The code that guesses the default stdout encoding isn't very good, > especially not on OSX. Suggestions are welcome. FWIW XEmacs 21.5 (still unreleased) was changed to set all system codecs (path, stdin, and stdout) to UTF-8 at startup if it's running on Mac OS (a build-time constant) some months ago, and I haven't seen a complaint yet from the testers. From python at rcn.com Fri Feb 15 01:52:43 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 14 Feb 2008 19:52:43 -0500 (EST) Subject: [Python-3000] Drop list.reverse()? Message-ID: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> Would it be worthwhile to leave reversed(s) and s[::-1] as the two obvious ways to do it (ways that work with any Sequence, not just lists)? Raymond From steven.bethard at gmail.com Fri Feb 15 01:59:59 2008 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 14 Feb 2008 17:59:59 -0700 Subject: [Python-3000] Drop list.reverse()? In-Reply-To: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> Message-ID: On Thu, Feb 14, 2008 at 5:52 PM, Raymond Hettinger wrote: > Would it be worthwhile to leave reversed(s) and s[::-1] as the two obvious ways to do it (ways that work with any Sequence, not just lists)? But .reverse() is in-place, while the other two create new sequences, no? Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From python at rcn.com Fri Feb 15 02:06:32 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 14 Feb 2008 20:06:32 -0500 (EST) Subject: [Python-3000] Drop list.reverse()? Message-ID: <20080214200632.AGZ09351@ms19.lnh.mail.rcn.net> [Steven] > But .reverse() is in-place, while the other two create new sequences, no? reversed(s) creates an iterator and s[::-1] creates a new list. They are not the same as s.reverse() but the use cases overlap quite a bit. Raymond From skip at pobox.com Fri Feb 15 02:33:22 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 14 Feb 2008 19:33:22 -0600 Subject: [Python-3000] Drop list.reverse()? In-Reply-To: <20080214200632.AGZ09351@ms19.lnh.mail.rcn.net> References: <20080214200632.AGZ09351@ms19.lnh.mail.rcn.net> Message-ID: <18356.60386.774541.55072@montanaro-dyndns-org.local> >> But .reverse() is in-place, while the other two create new sequences, no? Raymond> reversed(s) creates an iterator and s[::-1] creates a new list. Raymond> They are not the same as s.reverse() but the use cases overlap Raymond> quite a bit. If s.reverse() is dumped shouldn't s.sort() be as well? Skip From python at rcn.com Fri Feb 15 02:44:07 2008 From: python at rcn.com (Raymond Hettinger) Date: Thu, 14 Feb 2008 20:44:07 -0500 (EST) Subject: [Python-3000] Drop list.reverse()? Message-ID: <20080214204407.AGZ13523@ms19.lnh.mail.rcn.net> [Skip] > If s.reverse() is dumped shouldn't s.sort() be as well? I don't think so. When you run reversed(s), the s argument can be any Sequence (either mutable or immutable) and the return value is an iterator that doesn't copy the whole dataset. In contrast, sorted() takes any iterable and makes a full copy of it before sorting it. So reversed(s) can be done without doubling memory consumption (and is just as fast as iterating forward), but sorted() is a much heavier operation. Since reversed() and sorted() were introduced, I've almost never had occasion to want list.reverse() but I still use list.sort(). One other reason for keeping list.sort() is that sort stability lets you make multiple passes: mylist.sort(key=attrgetter('name') mylist.sort(key=attrgetter('score', reverse=True) # Now, it's sorted by reverse score, then alphabetically. There's not a parallel use case for successive reverses. Raymond From skip at pobox.com Fri Feb 15 02:02:11 2008 From: skip at pobox.com (skip at pobox.com) Date: Thu, 14 Feb 2008 19:02:11 -0600 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> Message-ID: <18356.58515.512592.321048@montanaro-dyndns-org.local> Guido> You can hack around this for now by doing (before printing Guido> anything) Guido> sys.stdout._encoding = 'utf-8' Guido> but that ought to be a temporary hack until we've figured out the Guido> right way to set it. How about being able to reopen() an open file object, e.g.: print(..., file=sys.stdout.reopen(encoding="utf-8") ? Skip From greg.ewing at canterbury.ac.nz Fri Feb 15 03:04:55 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 15 Feb 2008 15:04:55 +1300 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> Message-ID: <47B4F347.7050102@canterbury.ac.nz> Guido van Rossum wrote: > Now can you come up with a syntax for positional-only > arguments? So far everybody has failed at that, and there are some use > cases where it's useful too. def foo(*(a, b,c)): ... i.e. catch them in a * arg which is unpacked in-place. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Fri Feb 15 03:09:21 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 15 Feb 2008 15:09:21 +1300 Subject: [Python-3000] Drop list.reverse()? In-Reply-To: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> Message-ID: <47B4F451.3070707@canterbury.ac.nz> Raymond Hettinger wrote: > Would it be worthwhile to leave reversed(s) and s[::-1] > as the two obvious ways to do it? But that would leave us without an efficient way of reversing in-place. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Fri Feb 15 03:11:21 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 15 Feb 2008 15:11:21 +1300 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <18356.58515.512592.321048@montanaro-dyndns-org.local> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> Message-ID: <47B4F4C9.3020402@canterbury.ac.nz> skip at pobox.com wrote: > How about being able to reopen() an open file object, e.g.: > > print(..., file=sys.stdout.reopen(encoding="utf-8") But reopening implies a lot more than just changing the encoding. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Fri Feb 15 03:16:14 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 15 Feb 2008 15:16:14 +1300 Subject: [Python-3000] Drop list.reverse()? In-Reply-To: <20080214204407.AGZ13523@ms19.lnh.mail.rcn.net> References: <20080214204407.AGZ13523@ms19.lnh.mail.rcn.net> Message-ID: <47B4F5EE.8000800@canterbury.ac.nz> Raymond Hettinger wrote: > When you run reversed(s), the s argument > can be any Sequence (either mutable or immutable) and the > return value is an iterator that doesn't copy the whole > dataset. If reversed() became a *view* (a mutable one) rather than an iterator, there might be more of a case for dropping list.reverse(), although I'm still not sure. It might still be a good idea to make reversed() a view (maybe mutable, maybe not). -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From digitalxero at gmail.com Fri Feb 15 03:28:28 2008 From: digitalxero at gmail.com (Dj Gilcrease) Date: Thu, 14 Feb 2008 18:28:28 -0800 Subject: [Python-3000] PEP 3102 In-Reply-To: <47B4F347.7050102@canterbury.ac.nz> References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B4F347.7050102@canterbury.ac.nz> Message-ID: On Thu, Feb 14, 2008 at 6:04 PM, Greg Ewing wrote: > Guido van Rossum wrote: > > Now can you come up with a syntax for positional-only > > arguments? So far everybody has failed at that, and there are some use > > cases where it's useful too. > > def foo(*(a, b,c)): > ... > > i.e. catch them in a * arg which is unpacked in-place. I thought the ability to do that was being removed in py3? throwing some ideas around, for all examples arg1 and arg2 would be positional only def test(, arg3, *arg4): ... It's ok, since it clearly groups the positional only args, but the parser might not be able to handle this. def test(|arg1, |arg2, arg3, *, arg4): ... Simple, but inelegant. In just looking at the code it seems wrong. def test(arg1, arg2, |, arg3, *, arg4): ... I like this idea better, use the pipe as a separator for positional only, and normal arguments. From nick at craig-wood.com Fri Feb 15 11:12:19 2008 From: nick at craig-wood.com (Nick Craig-Wood) Date: Fri, 15 Feb 2008 10:12:19 +0000 Subject: [Python-3000] weakrefs of bound methods Message-ID: <20080215101219.GA29219@craig-wood.com> I've just been bitten yet again by the fact you can't have a weakref to a bound method! I find myself wanting to do this whenever I have a registry of callback functions. Could we in py3k either 1) make weakrefs to bound methods work? A weakref to a bound method should mean hold the weakref on the instance and bind the method at the last moment. or 2) throw an exception when you attempt to take a weak reference of a bound method? Maybe only when it has a single refcount? Eg Python 3.0a2+ (py3k:60831, Feb 15 2008, 09:52:40) [GCC 4.2.3 20071123 (prerelease) (Debian 4.2.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from weakref import WeakKeyDictionary >>> d=WeakKeyDictionary() >>> class A(object): ... def f(self): print("hello") ... >>> a=A() >>> a.f() hello >>> d[a.f] = True >>> d.keys() [] >>> I know how to work around the problem. The fact that it passes completely silently uses up lots of developer time debugging it though! Thanks! -- Nick Craig-Wood -- http://www.craig-wood.com/nick From niki.spahiev at gmail.com Fri Feb 15 13:18:21 2008 From: niki.spahiev at gmail.com (Niki Spahiev) Date: Fri, 15 Feb 2008 14:18:21 +0200 Subject: [Python-3000] Windows gui vs. console In-Reply-To: <20080208041917.4390C3A4076@sparrow.telecommunity.com> References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> <20080208041917.4390C3A4076@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > That's actually a separate problem from the one Paul and I are > talking about, one that I'm not even going to *try* to solve. (Being > as I already have, and failed miserably.) Hell, I'm not sure Windows > even has a way to *tell* whether your app was launched from the > command line or otherwise. I couldn't find one for Win2K, anyway; > there might be a way to do it in XP and Vista. Checking for attached console is one possibility. AttachConsole(ATTACH_PARENT_CONSOLE) Remarks The calling application can call any of the console functions. For example, a graphical application can use the console to display debugging output. A process can be attached to one console. If the calling process is already attached to a console, AttachConsole fails with ERROR_ACCESS_DENIED. Niki Spahiev From pje at telecommunity.com Fri Feb 15 15:34:41 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 09:34:41 -0500 Subject: [Python-3000] Windows gui vs. console In-Reply-To: References: <448576.20652.qm@web31111.mail.mud.yahoo.com> <20080204185642.73659254@bhuda.mired.org> <20080205175802.3FBF93A409E@sparrow.telecommunity.com> <47A9FE7B.7090405@cheimes.de> <79990c6b0802070115g44636852hc45024da9141d845@mail.gmail.com> <47AB6F82.4070308@canterbury.ac.nz> <20080207221027.261D33A40D7@sparrow.telecommunity.com> <79990c6b0802071421r151bf89ci13cf0666c3a3edc8@mail.gmail.com> <47ABC58B.7010608@canterbury.ac.nz> <20080208041917.4390C3A4076@sparrow.telecommunity.com> Message-ID: <20080215143429.92C563A4075@sparrow.telecommunity.com> At 02:18 PM 2/15/2008 +0200, Niki Spahiev wrote: >Phillip J. Eby wrote: > > That's actually a separate problem from the one Paul and I are > > talking about, one that I'm not even going to *try* to solve. (Being > > as I already have, and failed miserably.) Hell, I'm not sure Windows > > even has a way to *tell* whether your app was launched from the > > command line or otherwise. I couldn't find one for Win2K, anyway; > > there might be a way to do it in XP and Vista. > >Checking for attached console is one possibility. >AttachConsole(ATTACH_PARENT_CONSOLE) That requires XP, but while I was looking it up I ran across an article that suggested using both .com and .exe versions of the same program, where the .com is a console version, and the .exe is the GUI version linked to by any installed shortcuts. Apparently, you can have a .com file whose contents are actually an .exe. From ronaldoussoren at mac.com Fri Feb 15 15:26:02 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 15 Feb 2008 15:26:02 +0100 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <47B4BEF4.5040206@v.loewis.de> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <47B4BEF4.5040206@v.loewis.de> Message-ID: On 14 Feb, 2008, at 23:21, Martin v. L?wis wrote: >> The code that guesses the default stdout encoding isn't very good, >> especially not on OSX. Suggestions are welcome. > > Unfortunately, Apple isn't very cooperative here. There is no way > of determining the encoding of a Terminal.app window, AFAIK > (it's normally UTF-8, unless the user has changed it). On OSX 10.5 Terminal sets the LANG environment variable, which pythonrun.c picks up to set the encoding of sys.stdout when that's connected to a tty. Printing arbitary unicode strings therefore works fine on my system (at least using python 2.5, I haven't used 3.0 yet). I haven't checked if other OS releases behave simularly. Ronald > > > Regards, > Martin > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ronaldoussoren%40mac.com From guido at python.org Fri Feb 15 18:24:40 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Feb 2008 09:24:40 -0800 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <18356.58515.512592.321048@montanaro-dyndns-org.local> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> Message-ID: On Thu, Feb 14, 2008 at 5:02 PM, wrote: > Guido> You can hack around this for now by doing (before printing > Guido> anything) > > Guido> sys.stdout._encoding = 'utf-8' > > Guido> but that ought to be a temporary hack until we've figured out the > Guido> right way to set it. > > How about being able to reopen() an open file object, e.g.: > > print(..., file=sys.stdout.reopen(encoding="utf-8") > > ? I think you're on to something. There's an API to access the underlying (buffered) binary stream: sys.stdout.buffer. It's possible to create a new TextIOWrapper on top of that: sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8") The only problem is that if the original TextIOWrapper is deleted or closed, its close() method will close the underlying buffer object, rendering sys.stdout useless. (Well, actually, the lowest-level raw I/O object refuses to be closed if the file descriptor is 0, 1 or 2, but is issues a warning and this is a close call. It would be better if TextIOWrapper didn't explicitly close() when deleted. That requires some redesign of the interaction between __del__() and close() in the IOBase class. Maybe __del__() should just flush() instead of close()? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Feb 15 18:35:42 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Feb 2008 09:35:42 -0800 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: <20080215101219.GA29219@craig-wood.com> References: <20080215101219.GA29219@craig-wood.com> Message-ID: On Fri, Feb 15, 2008 at 2:12 AM, Nick Craig-Wood wrote: > I've just been bitten yet again by the fact you can't have a weakref > to a bound method! I find myself wanting to do this whenever I have a > registry of callback functions. > > Could we in py3k either > > 1) make weakrefs to bound methods work? A weakref to a bound method > should mean hold the weakref on the instance and bind the method at > the last moment. I think you're somehow confused about how weakrefs are supposed to work. They work fine! Python 3.0a2+ (py3k:60836, Feb 15 2008, 09:21:45) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import weakref >>> class C: ... def __init__(self, x=0): self.x = x ... def f(self): return self.x ... >>> f = C(42).f >>> r = weakref.ref(f) >>> r >>> r() > >>> r()() 42 >>> del f >>> r() >>> r >>> > or > > 2) throw an exception when you attempt to take a weak reference of a > bound method? Maybe only when it has a single refcount? > > Eg > > Python 3.0a2+ (py3k:60831, Feb 15 2008, 09:52:40) > [GCC 4.2.3 20071123 (prerelease) (Debian 4.2.2-4)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> from weakref import WeakKeyDictionary > >>> d=WeakKeyDictionary() > >>> class A(object): > ... def f(self): print("hello") > ... > >>> a=A() > >>> a.f() > hello > >>> d[a.f] = True > >>> d.keys() > [] > >>> > > I know how to work around the problem. The fact that it passes > completely silently uses up lots of developer time debugging it > though! Sorry, you don't seem to understand how weak references work.The bound method object a.f goes away because you're not keeping any other reference to it. Try this instead: >>> d = weakref.WeakKeyDictionary() >>> f = C(100).f >>> d[f] = 200 >>> d >>> d.keys() [>] >>> del f >>> d.keys() [>] >>> >>> 1 1 >>> d.keys() [] >>> (The second time I printed d.keys() it wasn't gone yet because the hidden variable '_' was holding on to the last object printed, which was the list of d's keys.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Feb 15 18:37:43 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Feb 2008 09:37:43 -0800 Subject: [Python-3000] Drop list.reverse()? In-Reply-To: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> Message-ID: On Thu, Feb 14, 2008 at 4:52 PM, Raymond Hettinger wrote: > Would it be worthwhile to leave reversed(s) and s[::-1] as the two obvious ways to do it (ways that work with any Sequence, not just lists)? I think we should keep it. There's no faster way to reverse a list in place. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rhamph at gmail.com Fri Feb 15 18:42:04 2008 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 15 Feb 2008 10:42:04 -0700 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> Message-ID: On Fri, Feb 15, 2008 at 10:24 AM, Guido van Rossum wrote: > On Thu, Feb 14, 2008 at 5:02 PM, wrote: > > Guido> You can hack around this for now by doing (before printing > > Guido> anything) > > > > Guido> sys.stdout._encoding = 'utf-8' > > > > Guido> but that ought to be a temporary hack until we've figured out the > > Guido> right way to set it. > > > > How about being able to reopen() an open file object, e.g.: > > > > print(..., file=sys.stdout.reopen(encoding="utf-8") > > > > ? > > I think you're on to something. There's an API to access the > underlying (buffered) binary stream: sys.stdout.buffer. It's possible > to create a new TextIOWrapper on top of that: > > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8") > > The only problem is that if the original TextIOWrapper is deleted or > closed, its close() method will close the underlying buffer object, > rendering sys.stdout useless. (Well, actually, the lowest-level raw > I/O object refuses to be closed if the file descriptor is 0, 1 or 2, > but is issues a warning and this is a close call. It would be better > if TextIOWrapper didn't explicitly close() when deleted. That requires > some redesign of the interaction between __del__() and close() in the > IOBase class. Maybe __del__() should just flush() instead of close()? That'd rely on prompt deletion for correct behaviour. Why not an explicit detach method? buf = sys.stdout.buffer sys.stdout.detach() sys.stdout = io.TextIOWrapper(buf, "utf-8") -- Adam Olsen, aka Rhamphoryncus From digitalxero at gmail.com Fri Feb 15 18:42:39 2008 From: digitalxero at gmail.com (Dj Gilcrease) Date: Fri, 15 Feb 2008 09:42:39 -0800 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B4F347.7050102@canterbury.ac.nz> <47B5C1E4.3000808@cornell.edu> Message-ID: On Fri, Feb 15, 2008 at 9:37 AM, Dj Gilcrease wrote: > The intent is to make it so (Assuming arg1 and arg2 are positional > only) you cannot do test(arg3=9, arg1=7, arg2=8), thus you would be > required to do test(7, 8, 9) {since doing test(7, 8, arg3=9) would > raise an error of positional arg being set by position and keyword at > the same time} > Little correction, test(7, 8, arg3=9) should work and does not raise the error I though it did From digitalxero at gmail.com Fri Feb 15 18:37:50 2008 From: digitalxero at gmail.com (Dj Gilcrease) Date: Fri, 15 Feb 2008 09:37:50 -0800 Subject: [Python-3000] PEP 3102 In-Reply-To: <47B5C1E4.3000808@cornell.edu> References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B4F347.7050102@canterbury.ac.nz> <47B5C1E4.3000808@cornell.edu> Message-ID: On Fri, Feb 15, 2008 at 8:46 AM, Joel Bender wrote: > How about a BNFish notation? Use brackets around optional parameters. > > def test([arg1, [arg2,]] arg3): > ... If I understand right, positional only arguments would not be optional in any way shape or form, thats what default arguments are for. Positional only args would be arguments that you cannot specify by keyword, so having a positional only arg that has a default value, thus making it optional just seems broken to me. The notation you recommended just confuses the hell outa me when reading it, it is neither clean nor elegant for the stated intent, which is a method of defining positional only arguments. eg. def test([arg1=1, arg2=2,] arg3): return (arg1, arg2, arg3) assert test(9) == (1, 2, 9) assert test(7, 8, 9) == (7, 8, 9) test(8, 9) == Exception test(arg3=9) == ??? This example clearly shows what confuses me about this notation. arg1 and arg2 are supposed to be positional only, but your first example the first argument you set end up in the 3rd position for no reason other then you defined default arguments before the positional arguments, which should and does raise "SyntaxError: non-default argument follows default argument" The intent is to make it so (Assuming arg1 and arg2 are positional only) you cannot do test(arg3=9, arg1=7, arg2=8), thus you would be required to do test(7, 8, 9) {since doing test(7, 8, arg3=9) would raise an error of positional arg being set by position and keyword at the same time} From jjb5 at cornell.edu Fri Feb 15 17:46:28 2008 From: jjb5 at cornell.edu (Joel Bender) Date: Fri, 15 Feb 2008 11:46:28 -0500 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B4F347.7050102@canterbury.ac.nz> Message-ID: <47B5C1E4.3000808@cornell.edu> Dj Gilcrease wrote: > On Thu, Feb 14, 2008 at 6:04 PM, Greg Ewing wrote: >> Guido van Rossum wrote: >> > Now can you come up with a syntax for positional-only >> > arguments? So far everybody has failed at that, and there are some use >> > cases where it's useful too. > > > throwing some ideas around... How about a BNFish notation? Use brackets around optional parameters. def test([arg1, [arg2,]] arg3): ... I haven't found an example of what the value of an unspecified positional only argument would be, I think it should be required: def test([arg1=1,] arg2): return (arg1, arg2) This could also accommodate trailing positional only args, even though the existing syntax already has it (but keep reading): def test(arg1 [, arg2=2]): return (arg1, arg2) assert test(1) == (1, 2) assert test(1,3) == (1, 3) If both were provided: def test([arg1=1,] arg2 [, arg3=3]): ... # error I would expect it to be an "ambiguous combination of optional positional-only arguments" syntax error (as opposed to runtime error). If you wanted arg2 specified when arg1 is: def test([arg1=1, arg2=2,] arg3): return (arg1, arg2, arg3) assert test(9) == (1, 2, 9) assert test(7, 8, 9) == (7, 8, 9) test(8, 9) # error With nesting you can specify which one is more optional: def test([[arg1=1,] arg2=2,] arg3): return (arg1, arg2, arg3) assert test(9) == (1, 2, 9) assert test(8, 9) == (1, 8, 9) def test([arg1=1, [arg2=2,]] arg3): return (arg1, arg2, arg3) assert test(8, 9) == (8, 2, 9) Capturing all of the args except the last one: def test([*arg1,] arg2): return (arg1, arg2) assert test(9) == ((), 9) assert test(8, 9) == ((8,), 9) assert test(7, 8, 9) == ((7, 8), 9) First is a little more important: def test([arg1=1, *arg2,] arg3): return (arg1, arg2, arg3) assert test(9) == (1, (), 9) assert test(8, 9) == (8, (), 9) assert test(7, 8, 9) == (7, (8,), 9) Force the end of positional arguments to switch to keywords: def test([*arg1,] arg2, *, arg3=3): return (arg1, arg2, arg3) Specifying more than one dumping spot for additional positional parameters is ambiguous: def test([arg1=1, *arg2,] arg3, *arg4): ... # error Specifying default values for a positional argument may make it ambiguous: def test([arg1=1,] arg2, arg3=3): return (arg1, arg2, arg3) This is OK, but not recommended, it's more complicated than it needs to be: def test(arg1 [, arg2=1 [, arg3=2]]): return (arg1, arg2, arg3) And now for something completely different: def test(arg1 [[, arg2=2], arg3=3]): return (arg1, arg2, arg3) assert test(8) == (8, 2, 3) assert test(8, 9) == (8, 2, 9) assert test(7, 8, 9) == (7, 8, 9) Like the other dumping spot: def test(arg1 [[, *arg2], arg3=3]): return (arg1, arg2, arg3) assert test(8) == (8, (), 3) assert test(8, 9) == (8, (), 9) assert test(7, 8, 9) == (7, (8,), 9) Nested galore: def test([arg1=1, [[*arg2,] arg3=3,]] arg4): return (arg1, arg2, arg3, arg4) assert test(7) = (1, (), 3, 7) assert test(7, 8) = (7, (), 3, 8) assert test(7, 8, 9) = (7, (), 8, 9) assert test(7, 8, 9, 10) = (7, (8,), 9, 10) The notation for an optional tuple that gets ripped apart seems busy, but is still grokable: def test([(arg1, arg2)=(1, 2),] arg3): return (arg1, arg2, arg3) assert test(8) == (1, 2, 8) assert test((7, 8), 9) == (7, 8, 9) Joel From pje at telecommunity.com Fri Feb 15 19:26:21 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 13:26:21 -0500 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: References: <20080215101219.GA29219@craig-wood.com> Message-ID: <20080215182609.0251A3A4075@sparrow.telecommunity.com> At 09:35 AM 2/15/2008 -0800, Guido van Rossum wrote: >On Fri, Feb 15, 2008 at 2:12 AM, Nick Craig-Wood wrote: > > I've just been bitten yet again by the fact you can't have a weakref > > to a bound method! I find myself wanting to do this whenever I have a > > registry of callback functions. > > > > Could we in py3k either > > > > 1) make weakrefs to bound methods work? A weakref to a bound method > > should mean hold the weakref on the instance and bind the method at > > the last moment. > >I think you're somehow confused about how weakrefs are supposed to >work. They work fine! The use case here is actually to keep the callback around only so long as the object exists. There are various ways to do this, of course, that don't require changing the language or stdlib. One of the simplest (for 2.4 and up) is to create a callable weakref subclass that emulates a bound method. Another is to create a bound method whose im_self is a weakref proxy. I've never actually encountered a usecase for keeping a standard weakref to a standard bound method, though. It's pretty useless, because bound methods are nearly always immediately disposed of, and there's no logical place (or reason) to store them. So, Nick's query is (in effect), can we either: 1. make ref(method) create a "weak method" instead, or 2. make ref(method) an error instead of useless behavior that gives you the illusion of working code From guido at python.org Fri Feb 15 20:41:14 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Feb 2008 11:41:14 -0800 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: <20080215182609.0251A3A4075@sparrow.telecommunity.com> References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> Message-ID: On Fri, Feb 15, 2008 at 10:26 AM, Phillip J. Eby wrote: > At 09:35 AM 2/15/2008 -0800, Guido van Rossum wrote: > >On Fri, Feb 15, 2008 at 2:12 AM, Nick Craig-Wood wrote: > > > I've just been bitten yet again by the fact you can't have a weakref > > > to a bound method! I find myself wanting to do this whenever I have a > > > registry of callback functions. > > > > > > Could we in py3k either > > > > > > 1) make weakrefs to bound methods work? A weakref to a bound method > > > should mean hold the weakref on the instance and bind the method at > > > the last moment. > > > >I think you're somehow confused about how weakrefs are supposed to > >work. They work fine! > > The use case here is actually to keep the callback around only so > long as the object exists. Ah, I see. Nick actually wrote this but because I'm so familiar with the concept of a bound method object I didn't see what his actual issue was. Sorry Nick! > There are various ways to do this, of course, that don't require > changing the language or stdlib. One of the simplest (for 2.4 and > up) is to create a callable weakref subclass that emulates a bound > method. Another is to create a bound method whose im_self is a weakref proxy. > > I've never actually encountered a usecase for keeping a standard > weakref to a standard bound method, though. It's pretty useless, > because bound methods are nearly always immediately disposed of, and > there's no logical place (or reason) to store them. So, Nick's query > is (in effect), can we either: > > 1. make ref(method) create a "weak method" instead, or > > 2. make ref(method) an error instead of useless behavior that gives > you the illusion of working code #2 is easily implemented (just remove the machinery to support weak refs from the bound method object) but the question is, will it break anybody's code -- and if so, do we care enough to break it? (In Py3k, that is.) #1 isn't so easy in terms of how you'd implement it -- I don't think the weakref.ref() call currently has a way to customize what kind of thing you get back. I'd be open for a patch containing working code that can be generalized to other cases. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jjb5 at cornell.edu Fri Feb 15 20:55:46 2008 From: jjb5 at cornell.edu (Joel Bender) Date: Fri, 15 Feb 2008 14:55:46 -0500 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B4F347.7050102@canterbury.ac.nz> <47B5C1E4.3000808@cornell.edu> Message-ID: <47B5EE42.9000001@cornell.edu> Dj Gilcrease wrote: > If I understand right, positional only arguments would not be optional > in any way shape or form, thats what default arguments are for. I have been expecting that by providing a default value to an argument in the definition that you are making it optional, be it keyword or positional. The PEP doesn't have an example of what this means: def f(a, b, *, c): ... Is 'c' a keyword only argument that must be specified? In that case I don't consider the following confusing, there is a clear indication of which arguments are positional and which ones are keyword only. def f(a, b=2, *, c=3): ... Given this definition, which isn't in the PEP but maybe it should be: def f(x, *): I would be perfectly happy if there was an exception for f(x=3), just as there is for math.sin(x=3). > Positional only args would be arguments that you cannot specify by > keyword, so having a positional only arg that has a default value, > thus making it optional just seems broken to me. I think this is a case of differing viewpoints. When I see a definition like this: def f(a, b=3): ... I think of 'b' being an optional argument, you seem to see it differently. I don't use keywords when I specify the values of positional arguments, so if I see this in documentation: min(iterable[, key=func]) -> value I don't call min(iterable=[3]) because it returns a TypeError. So when I see something like this: def f(x, y): I've never gotten in the habit of even trying f(y=2, x=3), and I would consider it confusing even if and when it works. Then again, perhaps it's because I've never felt compelled to reorder the sequence of values of positional parameters when I call a function. Joel From g.brandl at gmx.net Fri Feb 15 21:25:33 2008 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 15 Feb 2008 21:25:33 +0100 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B4F347.7050102@canterbury.ac.nz> <47B5C1E4.3000808@cornell.edu> Message-ID: Dj Gilcrease schrieb: > On Fri, Feb 15, 2008 at 8:46 AM, Joel Bender wrote: >> How about a BNFish notation? Use brackets around optional parameters. >> >> def test([arg1, [arg2,]] arg3): >> ... > > If I understand right, positional only arguments would not be optional > in any way shape or form, thats what default arguments are for. > Positional only args would be arguments that you cannot specify by > keyword, so having a positional only arg that has a default value, > thus making it optional just seems broken to me. Being positional-only and having a default value does not conflict. There's nothing at all broken with def foo(!x, !y='bar'): pass where !x and !y would be positional-only arguments. You can't of course have something like def foo(!x, !y='bar', z): pass but you can't have that today either. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From pje at telecommunity.com Fri Feb 15 21:52:43 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 15 Feb 2008 15:52:43 -0500 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> Message-ID: <20080215205231.93DD43A4075@sparrow.telecommunity.com> At 11:41 AM 2/15/2008 -0800, Guido van Rossum wrote: >On Fri, Feb 15, 2008 at 10:26 AM, Phillip J. Eby > wrote: > > I've never actually encountered a usecase for keeping a standard > > weakref to a standard bound method, though. It's pretty useless, > > because bound methods are nearly always immediately disposed of, and > > there's no logical place (or reason) to store them. So, Nick's query > > is (in effect), can we either: > > > > 1. make ref(method) create a "weak method" instead, or > > > > 2. make ref(method) an error instead of useless behavior that gives > > you the illusion of working code > >#2 is easily implemented (just remove the machinery to support weak >refs from the bound method object) but the question is, will it break >anybody's code -- and if so, do we care enough to break it? (In Py3k, >that is.) It's hard to imagine what could be using it, but I guess it's possible. >#1 isn't so easy in terms of how you'd implement it -- I don't think >the weakref.ref() call currently has a way to customize what kind of >thing you get back. I'd be open for a patch containing working code >that can be generalized to other cases. Well, one could allow for a __weaken__ method that would be called if present, instead of the default constructor. The part I'm not immediately sure of is where to call it *from*, so that you can still subclass weakref.ref and not end up in a strange delegation loop. I guess you could call it if and only if weakref.ref.__new__ was passed weakref.ref as the class to construct. From guido at python.org Fri Feb 15 22:20:03 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Feb 2008 13:20:03 -0800 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: <20080215205231.93DD43A4075@sparrow.telecommunity.com> References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> <20080215205231.93DD43A4075@sparrow.telecommunity.com> Message-ID: On Fri, Feb 15, 2008 at 12:52 PM, Phillip J. Eby wrote: > At 11:41 AM 2/15/2008 -0800, Guido van Rossum wrote: > >On Fri, Feb 15, 2008 at 10:26 AM, Phillip J. Eby > > wrote: > > > > I've never actually encountered a usecase for keeping a standard > > > weakref to a standard bound method, though. It's pretty useless, > > > because bound methods are nearly always immediately disposed of, and > > > there's no logical place (or reason) to store them. So, Nick's query > > > is (in effect), can we either: > > > > > > 1. make ref(method) create a "weak method" instead, or > > > > > > 2. make ref(method) an error instead of useless behavior that gives > > > you the illusion of working code > > > >#2 is easily implemented (just remove the machinery to support weak > >refs from the bound method object) but the question is, will it break > >anybody's code -- and if so, do we care enough to break it? (In Py3k, > >that is.) > > It's hard to imagine what could be using it, but I guess it's possible. > > > > >#1 isn't so easy in terms of how you'd implement it -- I don't think > >the weakref.ref() call currently has a way to customize what kind of > >thing you get back. I'd be open for a patch containing working code > >that can be generalized to other cases. > > Well, one could allow for a __weaken__ method that would be called if > present, instead of the default constructor. The part I'm not > immediately sure of is where to call it *from*, so that you can still > subclass weakref.ref and not end up in a strange delegation loop. I > guess you could call it if and only if weakref.ref.__new__ was passed > weakref.ref as the class to construct. I think str() does this. But maybe given how rare the use case is, it would be easier to just create a custom class in weakref.py that does what Nick requested. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nicko at nicko.org Fri Feb 15 23:53:18 2008 From: nicko at nicko.org (Nicko van Someren) Date: Fri, 15 Feb 2008 22:53:18 +0000 Subject: [Python-3000] Drop list.reverse()? In-Reply-To: References: <20080214195243.AGZ07721@ms19.lnh.mail.rcn.net> Message-ID: On 15 Feb 2008, at 17:37, Guido van Rossum wrote: > On Thu, Feb 14, 2008 at 4:52 PM, Raymond Hettinger > wrote: >> Would it be worthwhile to leave reversed(s) and s[::-1] as the two >> obvious ways to do it (ways that work with any Sequence, not just >> lists)? > > I think we should keep it. There's no faster way to reverse a list > in place. Out of curiosity, I checked just how much faster it really is. The answer is that for in place reversal it's more than an order of magnitude faster than using either of the suggested alternatives: >>> timeit.Timer('s.reverse()', setup='s=range(1000)').timeit() 0.80964398384094238 >>> timeit.Timer('s[:] = s[::-1]', setup='s=range(1000)').timeit() 9.2884359359741211 >>> timeit.Timer('s[:] = reversed(s)', setup='s=range(1000)').timeit() 16.757926940917969 Not surprising, given that no objects were created or harmed in the reversal of this list... Interestingly, it's not the fastest way to make a reverse copy: >>> timeit.Timer('s2 = list(s); s2.reverse()', setup='s=range(1000)').timeit() 6.4023230075836182 >>> timeit.Timer('s2 = s[::-1]', setup='s=range(1000)').timeit() 5.3177969455718994 >>> timeit.Timer('s2 = list(reversed(s))', setup='s=range(1000)').timeit() 12.906921148300171 (Times take on an Intel Mac, Python 2.5.1) Nicko From greg.ewing at canterbury.ac.nz Sat Feb 16 01:13:25 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 16 Feb 2008 13:13:25 +1300 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> Message-ID: <47B62AA5.20508@canterbury.ac.nz> Guido van Rossum wrote: > It would be better > if TextIOWrapper didn't explicitly close() when deleted. > Maybe __del__() should just flush() instead of close()? Sounds good to me. -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 16 01:19:01 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 16 Feb 2008 13:19:01 +1300 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B4F347.7050102@canterbury.ac.nz> <47B5C1E4.3000808@cornell.edu> Message-ID: <47B62BF5.90501@canterbury.ac.nz> Dj Gilcrease wrote: > Positional only args would be arguments that you cannot specify by > keyword, so having a positional only arg that has a default value, > thus making it optional just seems broken to me. There's no logical problem with optional positional-only args -- it just means you have to supply all the previous args as well. E.g. if the third arg of foo is positional-only and optional, you can call it as either foo(a, b) foo(a, b, c) -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 16 01:21:12 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 16 Feb 2008 13:21:12 +1300 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> Message-ID: <47B62C78.3050609@canterbury.ac.nz> Adam Olsen wrote: > That'd rely on prompt deletion for correct behaviour. Why not an > explicit detach method? > > buf = sys.stdout.buffer > sys.stdout.detach() > sys.stdout = io.TextIOWrapper(buf, "utf-8") Or keep an explicit count of the number of unclosed wrappers that are referencing a buffer. -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 16 01:26:41 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 16 Feb 2008 13:26:41 +1300 Subject: [Python-3000] Weakref protocol? (Re: weakrefs of bound methods) In-Reply-To: <20080215182609.0251A3A4075@sparrow.telecommunity.com> References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> Message-ID: <47B62DC1.4070606@canterbury.ac.nz> Phillip J. Eby wrote: > So, Nick's query is (in effect), can we either: > > 1. make ref(method) create a "weak method" instead, or > > 2. make ref(method) an error instead of useless behavior that gives > you the illusion of working code Maybe what we want here is a weakref protocol. So instead of weakref() directly creating a weak ref object, it would call __weakref__ on its argument. Most objects would respond by creating a standard weak ref object, but special cases such as bound methods could do something different. -- Greg From guido at python.org Sat Feb 16 01:36:34 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Feb 2008 16:36:34 -0800 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <47B62C78.3050609@canterbury.ac.nz> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> <47B62C78.3050609@canterbury.ac.nz> Message-ID: On Fri, Feb 15, 2008 at 4:21 PM, Greg Ewing wrote: > Adam Olsen wrote: > > That'd rely on prompt deletion for correct behaviour. Why not an > > explicit detach method? > > > > buf = sys.stdout.buffer > > sys.stdout.detach() > > sys.stdout = io.TextIOWrapper(buf, "utf-8") > > Or keep an explicit count of the number of unclosed > wrappers that are referencing a buffer. Yuck. We have to do this for sockets and it's a mess. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Sat Feb 16 01:34:07 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 16 Feb 2008 13:34:07 +1300 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: <20080215205231.93DD43A4075@sparrow.telecommunity.com> References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> <20080215205231.93DD43A4075@sparrow.telecommunity.com> Message-ID: <47B62F7F.2070906@canterbury.ac.nz> Phillip J. Eby wrote: > The part I'm not > immediately sure of is where to call it *from*, so that you can still > subclass weakref.ref and not end up in a strange delegation loop. The function for creating a weak ref would need to be different from the weakref class itself. One of them would have to be renamed (unless you want to do something magical and potentially confusing). -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 16 01:35:35 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 16 Feb 2008 13:35:35 +1300 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> <20080215205231.93DD43A4075@sparrow.telecommunity.com> Message-ID: <47B62FD7.5060409@canterbury.ac.nz> Guido van Rossum wrote: > But maybe given how rare the use case is, it would be easier to just > create a custom class in weakref.py that does what Nick requested. Nick would still forget to use it and get strange bugs in his code, though! -- Greg From martin at v.loewis.de Sat Feb 16 02:03:33 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 16 Feb 2008 02:03:33 +0100 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> Message-ID: <47B63665.6020808@v.loewis.de> > I think you're on to something. There's an API to access the > underlying (buffered) binary stream: sys.stdout.buffer. It's possible > to create a new TextIOWrapper on top of that: > > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8") While it might be possible to switch an output stream that way, it's fairly difficult to do that correctly for input, no? The wrapper may have already read data from the underlying stream which have not been given to the application. If you don't want to lose these data, you either need to copy them over to the new stream, or unread them from the underlying buffer. That's tricky as some of the data might be in the codec. Regards, Martin From guido at python.org Sat Feb 16 03:11:39 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Feb 2008 18:11:39 -0800 Subject: [Python-3000] Does Py3k's print offer any unicode encoding help? In-Reply-To: <47B63665.6020808@v.loewis.de> References: <18356.44529.51137.327615@montanaro-dyndns-org.local> <18356.58515.512592.321048@montanaro-dyndns-org.local> <47B63665.6020808@v.loewis.de> Message-ID: In general these things are only safe when done before any I/O happens. But even so it's only a stop-gap measure; the app shouldn't have to do this, we should fix the guessing algorithm. On Fri, Feb 15, 2008 at 5:03 PM, "Martin v. L?wis" wrote: > > I think you're on to something. There's an API to access the > > underlying (buffered) binary stream: sys.stdout.buffer. It's possible > > to create a new TextIOWrapper on top of that: > > > > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, "utf-8") > > While it might be possible to switch an output stream that way, it's > fairly difficult to do that correctly for input, no? The wrapper > may have already read data from the underlying stream which have not > been given to the application. If you don't want to lose these > data, you either need to copy them over to the new stream, or unread > them from the underlying buffer. That's tricky as some of the data might > be in the codec. > > Regards, > Martin > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Sat Feb 16 05:32:57 2008 From: talin at acm.org (Talin) Date: Fri, 15 Feb 2008 20:32:57 -0800 Subject: [Python-3000] PEP 3102 In-Reply-To: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> Message-ID: <47B66779.8000905@acm.org> Raymond Hettinger wrote: > I've been exercising the new keyword-only arguments syntax. It is absolutely wonderful. I'm amazed at how many long standing problems it solves elegantly. > > Thanks Talin! Well, that made my day :) -- Talin From rasky at develer.com Sat Feb 16 16:20:28 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 16 Feb 2008 15:20:28 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode Message-ID: Hello, CPython 2.x (and 3.x) under Win32 has an issue with sys.argv. The list is computed using the ANSI version of the windows APIs[*]. The problem is apparent when you have a file/directory which can't be represented in the system encoding (eg: a japanese-named file or directory on a Western Windows), because the Windows ANSI API will encode the filename to the system encoding using what we call the "replace" policy, and sys.argv[] will contain an entry like "c:\\foo\\??????????????.dat". At the moment, there's simply no way of passing such a file to a Python script/application as an argument (eg: if you double-click on that file, and the file is associated to a Python application). This is a wide- spread problem among Python applications; eg. if you click on a Japanese .torrent file, ABC (a Bittorent client written in Python) won't be able to open it and will complain "cannot access file ??????????.torrent". I understand that fixing this properly in the 2.x serie might have backward compatibility issues, but I propose that this be fixed at least in the Python 3.x serie, and I volunteer to write a patch. I would be glad if someone expert with ANSI/Unicode/Windows (MvL?) would show me what he believes being the correct way of approaching this problem. My suggestion is that: * At the Python level, we still expose a single sys.argv[], which will contain unicode strings. I think this exactly matches what Py3k does now. (Back in the time, there were proposals to add a sys.argvu, but I guess it does not make sense right now). * At the C level, I believe it involves using GetCommandLineW() and CommandLineToArgvW() in WinMain.c, but should Py_Main/PySys_SetArgv() be changed to also accept wchar_t** arguments? Or is it better to allow for NULL to be passed (under Windows at least), so that the Windows code-path in there can use GetCommandLineW()/CommandLineToArgvW() to get the current process' arguments? Thanks! [*] In detail: it actually comes from __argc/__argv (see WinMain.c), which in turn are computed by the CRT startup code, which would adapt to user's choice but Python is being compiled in ANSI mode. -- Giovanni Bajo From guido at python.org Sat Feb 16 16:49:22 2008 From: guido at python.org (Guido van Rossum) Date: Sat, 16 Feb 2008 07:49:22 -0800 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: Message-ID: Thanks for reminding us of this! Is there any chance that you can submit a patch (or even the beginnings of one) for this? At the very least, can you add the issue to the tracker so we'll have a permanent reminder until it's resolved? On Feb 16, 2008 7:20 AM, Giovanni Bajo wrote: > Hello, > > CPython 2.x (and 3.x) under Win32 has an issue with sys.argv. The list is > computed using the ANSI version of the windows APIs[*]. The problem is > apparent when you have a file/directory which can't be represented in the > system encoding (eg: a japanese-named file or directory on a Western > Windows), because the Windows ANSI API will encode the filename to the > system encoding using what we call the "replace" policy, and sys.argv[] > will contain an entry like "c:\\foo\\??????????????.dat". > > At the moment, there's simply no way of passing such a file to a Python > script/application as an argument (eg: if you double-click on that file, > and the file is associated to a Python application). This is a wide- > spread problem among Python applications; eg. if you click on a > Japanese .torrent file, ABC (a Bittorent client written in Python) won't > be able to open it and will complain "cannot access > file ??????????.torrent". > > I understand that fixing this properly in the 2.x serie might have > backward compatibility issues, but I propose that this be fixed at least > in the Python 3.x serie, and I volunteer to write a patch. I would be > glad if someone expert with ANSI/Unicode/Windows (MvL?) would show me > what he believes being the correct way of approaching this problem. > > My suggestion is that: > > * At the Python level, we still expose a single sys.argv[], which will > contain unicode strings. I think this exactly matches what Py3k does now. > (Back in the time, there were proposals to add a sys.argvu, but I guess > it does not make sense right now). > * At the C level, I believe it involves using GetCommandLineW() and > CommandLineToArgvW() in WinMain.c, but should Py_Main/PySys_SetArgv() be > changed to also accept wchar_t** arguments? Or is it better to allow for > NULL to be passed (under Windows at least), so that the Windows code-path > in there can use GetCommandLineW()/CommandLineToArgvW() to get the > current process' arguments? > > Thanks! > > [*] In detail: it actually comes from __argc/__argv (see WinMain.c), > which in turn are computed by the CRT startup code, which would adapt to > user's choice but Python is being compiled in ANSI mode. > -- > Giovanni Bajo > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rasky at develer.com Sat Feb 16 17:32:43 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 16 Feb 2008 16:32:43 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: Message-ID: On Sat, 16 Feb 2008 07:49:22 -0800, Guido van Rossum wrote: > Thanks for reminding us of this! > > Is there any chance that you can submit a patch (or even the beginnings > of one) for this? Sure, during next week. I'd appreciate if, meanwhile, a core developer commented on my proposal at the C level, so that I don't write a totally wrong patch to begin with. I'm not really familiar with CPython hacking. > At the very least, can you add the issue to the tracker so we'll have a > permanent reminder until it's resolved? Done: issue 2128. I didn't find a way to mark it Windows-only in the metadata (I guess there is not one). -- Giovanni Bajo From lists at cheimes.de Sat Feb 16 17:51:08 2008 From: lists at cheimes.de (Christian Heimes) Date: Sat, 16 Feb 2008 17:51:08 +0100 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: Message-ID: <47B7147C.10506@cheimes.de> Giovanni Bajo wrote: > Sure, during next week. I'd appreciate if, meanwhile, a core developer > commented on my proposal at the C level, so that I don't write a totally > wrong patch to begin with. I'm not really familiar with CPython hacking. We have dropped support for 9x, ME and NT4 for the 2.6 and 3.0 series. This allows us to use wide API methods without restrictions. IIRC Win2k and newer versions of CE have full support for wide unicode. Several places need an update to support unicode on Windows. This includes argv, the environment variables and file names / path handling. See http://bugs.python.org/issue1342 for another example. In the last months Amaury, Martin von L?wis and I have done most of the Windows work. Any help from an experienced Windows developer is greatly appreciated! We *need* an additional Windows developer. So if you are interested in getting the problems fixed I welcome you on board. I suggest that you start by writing down a list of required changes (file names, function names) and a proposal how to pursue the issues. Christian From rasky at develer.com Sat Feb 16 18:06:45 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 16 Feb 2008 17:06:45 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: <47B7147C.10506@cheimes.de> Message-ID: On Sat, 16 Feb 2008 17:51:08 +0100, Christian Heimes wrote: > In the last months Amaury, Martin von L?wis and I have done most of the > Windows work. Any help from an experienced Windows developer is greatly > appreciated! We *need* an additional Windows developer. I would really like to help you out on these wide-API tasks, but it is unlikely I will ever have much time in the next weeks (till May, let's say). Is there a todo list, a wiki, a keyword to use when searching for bugs, etc.? Anything to see what still needs to be done, who's doing it, etc. BTW: is there a long-time plan to make the Python core *not* link against msvcrt dll anymore but only rely on Windows APIs (or maybe also the static C runtime, I don't really care)? > So if you are interested in getting the problems fixed I welcome you on > board. I suggest that you start by writing down a list of required > changes (file names, function names) and a proposal how to pursue the > issues. OK. I'll have a look at issue 2128 during next week and come back with something. -- Giovanni Bajo From pje at telecommunity.com Sat Feb 16 18:40:52 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 16 Feb 2008 12:40:52 -0500 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: <47B7147C.10506@cheimes.de> Message-ID: <20080216174039.826583A4075@sparrow.telecommunity.com> At 05:06 PM 2/16/2008 +0000, Giovanni Bajo wrote: >BTW: is there a long-time plan to make the Python core *not* link >against msvcrt dll anymore but only rely on Windows APIs (or maybe >also the static C runtime, I don't really care)? How would this affect using MinGW to build Python extensions? From rasky at develer.com Sat Feb 16 18:55:00 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 16 Feb 2008 17:55:00 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> Message-ID: On Sat, 16 Feb 2008 12:40:52 -0500, Phillip J. Eby wrote: >>BTW: is there a long-time plan to make the Python core *not* link >>against msvcrt dll anymore but only rely on Windows APIs (or maybe also >>the static C runtime, I don't really care)? > > How would this affect using MinGW to build Python extensions? AFAICT, it would make python.dll not depend on any msvcrt.dll, and thus being compatible with any extension, compiled with any compiler (with any version of msvcrt, or without msvcrt at all). Putting an end to this annoying "what version of Visual Studio / msvcrt do I need for this version of Python?" is a sensible goal, IMHO. -- Giovanni Bajo From p.f.moore at gmail.com Sat Feb 16 19:29:07 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 16 Feb 2008 18:29:07 +0000 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> Message-ID: <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> On 16/02/2008, Giovanni Bajo wrote: > On Sat, 16 Feb 2008 12:40:52 -0500, Phillip J. Eby wrote: > > >>BTW: is there a long-time plan to make the Python core *not* link > >>against msvcrt dll anymore but only rely on Windows APIs (or maybe also > >>the static C runtime, I don't really care)? > > > > How would this affect using MinGW to build Python extensions? > > AFAICT, it would make python.dll not depend on any msvcrt.dll, and thus > being compatible with any extension, compiled with any compiler (with any > version of msvcrt, or without msvcrt at all). > > Putting an end to this annoying "what version of Visual Studio / msvcrt > do I need for this version of Python?" is a sensible goal, IMHO. Actually, it would mean that there was no way of building extensions, as Python would be using a static C runtime, which extensions wouldn't have access to (so a FILE* from an extension would be totally incompatible with the core CRT, a memory block malloc'd in an extension couldn't be freed by the core, etc...) In practice, it's possible to take a more relaxed view (differing CRTs can be used, but only as long as nothing CRT-dependent gets passed across the boundaries) but it's not completely clear what is affected, and so the official policy is strict. If you avoided the static CRT, and only used Windows APIs, that would avoid this issue, but that means reimplementing everything - malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone is contemplating that. Paul. From p.f.moore at gmail.com Sat Feb 16 19:37:45 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 16 Feb 2008 18:37:45 +0000 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: <47B7147C.10506@cheimes.de> References: <47B7147C.10506@cheimes.de> Message-ID: <79990c6b0802161037n3f2f8462vaff0b7484ab4afdb@mail.gmail.com> On 16/02/2008, Christian Heimes wrote: > In the last months Amaury, Martin von L?wis and I have done most of the > Windows work. Any help from an experienced Windows developer is greatly > appreciated! We *need* an additional Windows developer. Is there a list anywhere of tasks that need a Windows C developer? I may be able to have a go at picking a few up, but unfortunately, I don't have the time to find what needs to be done. If you point me at something, I'll see what I can pick up. Paul. From rasky at develer.com Sat Feb 16 20:08:54 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sat, 16 Feb 2008 19:08:54 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> Message-ID: On Sat, 16 Feb 2008 18:29:07 +0000, Paul Moore wrote: >> >>BTW: is there a long-time plan to make the Python core *not* link >> >>against msvcrt dll anymore but only rely on Windows APIs (or maybe >> >>also the static C runtime, I don't really care)? >> > >> > How would this affect using MinGW to build Python extensions? >> >> AFAICT, it would make python.dll not depend on any msvcrt.dll, and thus >> being compatible with any extension, compiled with any compiler (with >> any version of msvcrt, or without msvcrt at all). >> >> Putting an end to this annoying "what version of Visual Studio / msvcrt >> do I need for this version of Python?" is a sensible goal, IMHO. > > Actually, it would mean that there was no way of building extensions, as > Python would be using a static C runtime, which extensions wouldn't have > access to (so a FILE* from an extension would be totally incompatible > with the core CRT, a memory block malloc'd in an extension couldn't be > freed by the core, etc...) Yes. In fact, what I really meant was to get rid of any C-ism from the Python API, so that these problems you are listing simply do not arise. For instance, memory allocation is not a problem as far as I can see, as extensions always use core calls to allocate objects (and those objects get freed through core calls too). But yes, one would have to get rid of the APIs with FILE* and a few others (I wonder if anybody has already made a list). It does not seem like a terrible big deal to me in practice, but I don't know how hard it would be implementation-wise. Consider for instance that it was even proposed to rewrite fileobject.c to use OS native file support (eg: CreateFile on Windows), as the theoretically-portable C API has too many quirks in small details which makes it hard to really provide a Python fileobject consistent across operating systems. In such a scenario, an API like PyFile_FromFile() does not make much sense (but a PyFile_FromDescriptor() might still be useful of course). -- Giovanni Bajo From martin at v.loewis.de Sat Feb 16 20:58:51 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 16 Feb 2008 20:58:51 +0100 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: <47B7147C.10506@cheimes.de> Message-ID: <47B7407B.2080006@v.loewis.de> > BTW: is there a long-time plan to make the Python core *not* link against > msvcrt dll anymore but only rely on Windows APIs (or maybe also the > static C runtime, I don't really care)? I have plans to reduce usage of the CRT as much as possible. I don't believe that linking statically would actually work very well. Regards, Martin From martin at v.loewis.de Sat Feb 16 21:01:10 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 16 Feb 2008 21:01:10 +0100 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> Message-ID: <47B74106.1060102@v.loewis.de> > If you avoided the static CRT, and only used Windows APIs, that would > avoid this issue, but that means reimplementing everything - > malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone > is contemplating that. Actually, I am. Py3k will already remove usage of stdio, for file input and output. I would also like to remove CRT file handles, and replace them with platform file handles. As for malloc and free, using the Heap API might be an option. Regards, Martin From p.f.moore at gmail.com Sat Feb 16 21:20:28 2008 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 16 Feb 2008 20:20:28 +0000 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: <47B74106.1060102@v.loewis.de> References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> <47B74106.1060102@v.loewis.de> Message-ID: <79990c6b0802161220k6c3e5e6u56fd0d828ac1b9a7@mail.gmail.com> On 16/02/2008, "Martin v. L?wis" wrote: > > If you avoided the static CRT, and only used Windows APIs, that would > > avoid this issue, but that means reimplementing everything - > > malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone > > is contemplating that. > > Actually, I am. I stand corrected :-) If you need any help, I'd be willing to assist. Paul. From martin at v.loewis.de Sat Feb 16 21:45:39 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 16 Feb 2008 21:45:39 +0100 Subject: [Python-3000] malloc on Windows (Was: Windows, sys.argv and unicode) In-Reply-To: <79990c6b0802161220k6c3e5e6u56fd0d828ac1b9a7@mail.gmail.com> References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> <47B74106.1060102@v.loewis.de> <79990c6b0802161220k6c3e5e6u56fd0d828ac1b9a7@mail.gmail.com> Message-ID: <47B74B73.3010300@v.loewis.de> >>> If you avoided the static CRT, and only used Windows APIs, that would >>> avoid this issue, but that means reimplementing everything - >>> malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone >>> is contemplating that. >> Actually, I am. > > I stand corrected :-) If you need any help, I'd be willing to assist. It's a long-term project. If you are curious, I would recommend you look into memory management. At a minimum, pymalloc should use VirtualAlloc instead of malloc - as that can guarantee page alignment out of the box (at the same time, mmap should be used on Unix). For variable-sized blocks, you could try HeapAlloc, probably on the process heap. My only concern would be whether HeapAlloc can compete with the CRT's malloc performance-wise. However, during the implementation, additional issues might come up. Regards, Martin From nick at craig-wood.com Sun Feb 17 10:29:56 2008 From: nick at craig-wood.com (Nick Craig-Wood) Date: Sun, 17 Feb 2008 09:29:56 +0000 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> <20080215205231.93DD43A4075@sparrow.telecommunity.com> Message-ID: <20080217092956.GA17269@craig-wood.com> On Fri, Feb 15, 2008 at 01:20:03PM -0800, Guido van Rossum wrote: > But maybe given how rare the use case is, it would be easier to just > create a custom class in weakref.py that does what Nick requested. I still don't like the silent failure, but a noisy failure and a custom class would be great. FYI here is the class I was writing which tripped me up. I created this to untangle some spaghetti code in a reasonably complex GUI application I'm writing. Instead of there being lots of hardcoded callbacks in the GUI objects I made a central registry which made the code a lot cleaner. The weakrefs avoided registration and deregistration. Since this is used from GUI code all the callback functions I needed to add are bound methods. (The last time I got tripped by this was a similar case involving 3rd parties registering interests in other classes changing.) ------------------------------------------------------------ class Callback(object): """ A holder for functions which need to be called back at some specific point Silently fails if you pass in bound methods! """ def __init__(self): self.fns = WeakKeyDictionary() def add(self, fn): """Add function to be called later.""" self.fns[fn] = True def call(self): """Call all the registered functions""" for fn in self.fns.keys(): fn() ------------------------------------------------------------ Here is my explicit (no introspection) work-around - there are obviously better solutions! class Callback(object): """ A holder for functions which need to be called back at some specific point """ def __init__(self): self.fns = WeakKeyDictionary() def add(self, fn, attribute=None): """ Add function to be called later. Either pass in a function, or an object and an attribute to call. Don't pass in bound methods since you can't take a weakref of a bound method. It holds weakrefs so doesn't need to be deregistered. """ self.fns[fn] = attribute def call(self): """Call all the registered functions""" for fn, attribute in self.fns.iteritems(): if attribute is not None: fn = getattr(fn, attribute) fn() -- Nick Craig-Wood -- http://www.craig-wood.com/nick From nick at craig-wood.com Sun Feb 17 10:33:18 2008 From: nick at craig-wood.com (Nick Craig-Wood) Date: Sun, 17 Feb 2008 09:33:18 +0000 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: <47B62FD7.5060409@canterbury.ac.nz> References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> <20080215205231.93DD43A4075@sparrow.telecommunity.com> <47B62FD7.5060409@canterbury.ac.nz> Message-ID: <20080217093318.0A59A14C62F@irishsea.home.craig-wood.com> Greg Ewing wrote: > Guido van Rossum wrote: > > But maybe given how rare the use case is, it would be easier to just > > create a custom class in weakref.py that does what Nick requested. > > Nick would still forget to use it and get strange bugs > in his code, though! More than likely ;-) Though an exception if I tried to add a bound method to a weak dictionary would certainly remind me! -- Nick Craig-Wood -- http://www.craig-wood.com/nick From rasky at develer.com Sun Feb 17 12:24:23 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 17 Feb 2008 11:24:23 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> <47B74106.1060102@v.loewis.de> Message-ID: On Sat, 16 Feb 2008 21:01:10 +0100, Martin v. L?wis wrote: >> If you avoided the static CRT, and only used Windows APIs, that would >> avoid this issue, but that means reimplementing everything - >> malloc/free, FILE*, stdin/stdout/stderr, etc etc. I don't think anyone >> is contemplating that. > > Actually, I am. Py3k will already remove usage of stdio, for file input > and output. Are you doing this work on some public branch? -- Giovanni Bajo From rasky at develer.com Sun Feb 17 12:27:55 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 17 Feb 2008 11:27:55 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: <47B7147C.10506@cheimes.de> <47B7407B.2080006@v.loewis.de> Message-ID: On Sat, 16 Feb 2008 20:58:51 +0100, Martin v. L?wis wrote: >> BTW: is there a long-time plan to make the Python core *not* link >> against msvcrt dll anymore but only rely on Windows APIs (or maybe also >> the static C runtime, I don't really care)? > > I have plans to reduce usage of the CRT as much as possible. I don't > believe that linking statically would actually work very well. Given a reduced usage of CRT, why statically linking should not work? For instance, if all else fail wrt memory allocation, I can't see why it is a problem using the statically-linked CRT's malloc (or another multi- platform malloc implementation like Google's tcmalloc -- which would of course be statically linked as well). -- Giovanni Bajo From rasky at develer.com Sun Feb 17 14:09:40 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 17 Feb 2008 13:09:40 +0000 (UTC) Subject: [Python-3000] Windows: debug mode Message-ID: Hi, an issue related to the CRT removal/reduced usage under Windows is that of debug mode, which I stand that it is mostly useless as it is currently implemented. Python's debug mode under Windows mandates an ABI change: the debug version of dynamic libraries (_d) are linked against the CRT debug runtime, which is not ABI-compatible to the release runtime. So you either have the _d version of all the extensions you need or you can't even start your program in debug mode. I believe this started back in the days where Python programs were small, had only a couple of dependencies, and most developers were also core developers (core developers are probably the only ones that can succesfully use the current debug mode). Thus, what happens is that if you get a segfault in an extension library (eg: one you're developing) within a large application, you are forced to *recompile all your dependencies* to debug mode before being able to debug your extension. This is long and tiresome: you might be using large and complex extensions (like numpy or PyOpenGL) which are really hard to compile and requires many C-level dependecies and complicated setup; in fact, it is likely that you have always used them through their binary release-only distributions, and now, simply to get a small stacktrace of a crash in your extension, you are forced to setup the whole development environment needed to recompile all those extensions! So, what does a programmer do instead, today? It simply hacks his own distutils script to compile his own extension in debug mode, but *without* relying on Python's official debug mode support, that is *without* using CRT's debug libraries; for example, adding /Zi and /DEBUG switches to Visual Studio compiler and linker is sufficient to get a *fully debuggable* extension that fully cooperates with the standard Python runtime and the standard binary extensions. So, the programmer totally ditches the official debug support (eg: --debug in distutils), and has to come up with something himself. Thus, I'm hereby proposing that we change this in Py3k so that Python's debug mode does not mandate any ABI change anymore. This means that a programmer will be able to mixmatch release-mode extensions and compile- mode extensions, thus getting debuggability only of those extensions he really cares. He will be able to compile in debug mode only his extension by simply doing "setup.py build --debug" and get useful stacktraces and debuggability (step-by-step, whatever) through it, without even needed a debug build of Python. If people agree on this goal, I'm willing to actually implement it in the Python core (there must be a change in the logic through which "_d" extensions are picked up since they would become optional rather than required) and everywhere else it might be needed (eg: distutils). -- Giovanni Bajo From lists at cheimes.de Sun Feb 17 15:23:17 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 17 Feb 2008 15:23:17 +0100 Subject: [Python-3000] Windows: debug mode In-Reply-To: References: Message-ID: Giovanni Bajo wrote: > Python's debug mode under Windows mandates an ABI change: the debug > version of dynamic libraries (_d) are linked against the CRT debug > runtime, which is not ABI-compatible to the release runtime. So you > either have the _d version of all the extensions you need or you can't > even start your program in debug mode. I believe this started back in the > days where Python programs were small, had only a couple of dependencies, > and most developers were also core developers (core developers are > probably the only ones that can succesfully use the current debug mode). [snip] Python's Py_DEBUG mode adds lots of extra checks which require an ABI change. For example in debug mode memory allocation and and reference counting do extra checks etc. In release mode those extra checks are either disabled or they are replaced by much simpler and faster macros. It *may* be possible to add the extra functions to the Python library so that a debug build can also load release libraries. It's a complex and time consuming task.If you want to invest your time I'm fine with enhancing the situation for Windows developers. But keep in mind that any change must neither slow down Python in release mode nor make the debug mode harder to use for a core developer! Windows issue must not place an extra burden on Unix users. Christian From pje at telecommunity.com Sun Feb 17 16:10:59 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 17 Feb 2008 10:10:59 -0500 Subject: [Python-3000] weakrefs of bound methods In-Reply-To: <20080217092956.GA17269@craig-wood.com> References: <20080215101219.GA29219@craig-wood.com> <20080215182609.0251A3A4075@sparrow.telecommunity.com> <20080215205231.93DD43A4075@sparrow.telecommunity.com> <20080217092956.GA17269@craig-wood.com> Message-ID: <20080217151045.C3CC93A40A5@sparrow.telecommunity.com> At 09:29 AM 2/17/2008 +0000, Nick Craig-Wood wrote: >On Fri, Feb 15, 2008 at 01:20:03PM -0800, Guido van Rossum wrote: > > But maybe given how rare the use case is, it would be easier to just > > create a custom class in weakref.py that does what Nick requested. > >I still don't like the silent failure, but a noisy failure and a >custom class would be great. > >FYI here is the class I was writing which tripped me up. I created >this to untangle some spaghetti code in a reasonably complex GUI >application I'm writing. > >Instead of there being lots of hardcoded callbacks in the GUI objects >I made a central registry which made the code a lot cleaner. The >weakrefs avoided registration and deregistration. You might want to check this out: http://pypi.python.org/pypi/PyDispatcher """Included in the package are the robustapply and saferef modules, which provide the ability to selectively apply arguments to callable objects and to reference instance methods using weak-references.""" From pje at telecommunity.com Sun Feb 17 16:15:22 2008 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 17 Feb 2008 10:15:22 -0500 Subject: [Python-3000] Windows: debug mode In-Reply-To: References: Message-ID: <20080217151509.1C7A33A40A5@sparrow.telecommunity.com> At 03:23 PM 2/17/2008 +0100, Christian Heimes wrote: >Giovanni Bajo wrote: > > Python's debug mode under Windows mandates an ABI change: the debug > > version of dynamic libraries (_d) are linked against the CRT debug > > runtime, which is not ABI-compatible to the release runtime. So you > > either have the _d version of all the extensions you need or you can't > > even start your program in debug mode. I believe this started back in the > > days where Python programs were small, had only a couple of dependencies, > > and most developers were also core developers (core developers are > > probably the only ones that can succesfully use the current debug mode). > >[snip] > >Python's Py_DEBUG mode adds lots of extra checks which require an ABI >change. For example in debug mode memory allocation and and reference >counting do extra checks etc. In release mode those extra checks are >either disabled or they are replaced by much simpler and faster macros. > >It *may* be possible to add the extra functions to the Python library so >that a debug build can also load release libraries. Actually, I think Giovanni simply wants to be able to build extension modules with debugging information, *without* needing to use a Py_DEBUG build. That is, this is about using debug extensions with a release Python, not the other way 'round. And if I understand correctly, this could be done by additions to the distutils. From martin at v.loewis.de Sun Feb 17 18:09:18 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 17 Feb 2008 18:09:18 +0100 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> <47B74106.1060102@v.loewis.de> Message-ID: <47B86A3E.2090503@v.loewis.de> > Are you doing this work on some public branch? Yes, on the Py3k branch, and the trunk. Regards, Martin From lists at cheimes.de Sun Feb 17 18:09:51 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 17 Feb 2008 18:09:51 +0100 Subject: [Python-3000] Windows: debug mode In-Reply-To: <20080217151509.1C7A33A40A5@sparrow.telecommunity.com> References: <20080217151509.1C7A33A40A5@sparrow.telecommunity.com> Message-ID: Phillip J. Eby wrote: > Actually, I think Giovanni simply wants to be able to build extension > modules with debugging information, *without* needing to use a Py_DEBUG build. > > That is, this is about using debug extensions with a release Python, > not the other way 'round. And if I understand correctly, this could > be done by additions to the distutils. Ah, I see. It's about building an extension with debug symbols but without a Py_DEBUG build. That's possible with some minor adjustments to PC/pyconfig.h and PCbuild/debug.vsprops. One has to decouple _DEBUG from Py_DEBUG. Currently _DEBUG implies Py_DEBUG. A debug build without Py_DEBUG is semi useful. Bugs and memory leaks are usually caused by reference counting issues. Christian From martin at v.loewis.de Sun Feb 17 18:13:32 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 17 Feb 2008 18:13:32 +0100 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: <47B7147C.10506@cheimes.de> <47B7407B.2080006@v.loewis.de> Message-ID: <47B86B3C.3080400@v.loewis.de> >> I have plans to reduce usage of the CRT as much as possible. I don't >> believe that linking statically would actually work very well. > > Given a reduced usage of CRT, why statically linking should not work? When this project is complete (say, in 2015), there won't be a need to link statically. Until then, various things might fail. For example, setting the locale through setlocale might not have the desired effect on a different module. In addition, there is the size increase - you would easily get multiple copies of stdio in the distribution. It's fairly tricky not to link stdio in static linking, even if you don't call any stdio functions. Regards, Martin From martin at v.loewis.de Sun Feb 17 18:19:30 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 17 Feb 2008 18:19:30 +0100 Subject: [Python-3000] Windows: debug mode In-Reply-To: References: Message-ID: <47B86CA2.7090908@v.loewis.de> > Thus, I'm hereby proposing that we change this in Py3k so that Python's > debug mode does not mandate any ABI change anymore. It seems people still debate what it is that you *actually* want. This sentence suggests Christian's interpretation, i.e. you might be unaware that the debug mode not only adds debug symbols, but also changes the layout of objects. That is *very* useful, and should not be dropped. > This means that a > programmer will be able to mixmatch release-mode extensions and compile- > mode extensions, thus getting debuggability only of those extensions he > really cares. Whereas this suggests Phillip's interpretation, i.e. you merely want debug symbols on a single DLL. You can get that today - just enable debug information for your extension module, and the compiler will emit the PDB files you need for debugging. > If people agree on this goal, I'm willing to actually implement it in the > Python core (there must be a change in the logic through which "_d" > extensions are picked up since they would become optional rather than > required) and everywhere else it might be needed (eg: distutils). See above. Don't drop what Python calls the debug mode. Instead, just learn to build your extension with debugging, for Python's release mode. Regards, Martin From arnodel at googlemail.com Sun Feb 17 18:45:48 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Sun, 17 Feb 2008 09:45:48 -0800 (PST) Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> Message-ID: On Feb 14, 11:13?pm, "Guido van Rossum" wrote: > On Thu, Feb 14, 2008 at 2:48 PM, Raymond Hettinger wrote: > > I've been exercising the new keyword-only arguments syntax. ?It is absolutely wonderful. ?I'm amazed at how many long standing problems it solves elegantly. > > Agreed. Now can you come up with a syntax for positional-only > arguments? So far everybody has failed at that, and there are some use > cases where it's useful too. The idea in PEP 3102 is that *args gobbles up all remaining positional arguments, hence what comes after can only be interpreted as keyword-only arguments. This is extended to allowing * on its own. e.g. f(*, z, t=2) => z, t are keyword-only Following the same train of thought, **kwargs gobbles up all remaining keyword arguments, hence what comes after can only interpreted as positional-only arguments. Then one could extend it to allowing ** on its own. e.g. f(**, x, y=1) => x, y are positional-only Now, how do keyword-only arguments fit in? Say x, y=1 are positional-only and z, t=2 are keyword-only. One could have: f(**, x, y=1, *, z, t=2) The general cases could look like f(x, y=1, *args, z, t=2, **kwargs): => x, y are hybrid => args: all remaining positional arguments => z, t are keyword-only => kwargs: all remaining keyword arguments f(**kwargs, x, y=1, *args, z, t=2): => x, y are positional-only => args: all remaining positional arguments => z, t are keyword-only => kwargs: all remaining keyword arguments 'args' and 'kwargs' are optional in both cases. If 'kwargs' is present in the second version, it is a bit strange as it usually only appears at the end. Alternatively, only '**' could be allowed at the start and if needed '**kwargs' would come at the end: f(**, x, y=1, *args, z, t=2, **kwargs) Note that this syntax doesn't allow for positional-only and hybrid arguments to coexist. But this is nice IMHO, as it seems to me that it would only create confusion. Unconvincingly yours, -- Arnaud From rasky at develer.com Sun Feb 17 22:25:42 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 17 Feb 2008 21:25:42 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> <47B74106.1060102@v.loewis.de> <47B86A3E.2090503@v.loewis.de> Message-ID: On Sun, 17 Feb 2008 18:09:18 +0100, Martin v. L?wis wrote: >> Actually, I am. Py3k will already remove usage of stdio, for file input >> and output. >> Are you doing this work on some public branch? > > Yes, on the Py3k branch, and the trunk. I looked at fileobject.c in py3k branch today and it looked like it was still using FILE* everywhere. I'm not sure I know where to look, though. BTW, is there a public description about how you are planning to remove usage of stdio from fileobject? For instance, will PyFile_FromFile() be deprecated/removed? Generically speaking, all the UNIX world seems to speak char* and UTF-8 lately, while the Windows world speaks either char* and MBCS or wchar_t and UTF-16[*]. How do you plan to make these two world live together? In case the question is not clear, I elaborate with an example. Today, I found out that putting Unicode strings in sys.path breaks zipimporter, because it expects a 8-byte string; I guess the whole import machinery passes char* everywhere for filenames and directories. How do you plan to make all that code Unicode-aware on Windows? [*] I'm not a Unicode guru, so I'm not 100% sure that the wide APIs in Windows formally accepts UTF-16, but at least it looks like UTF-16 to me :) It's not really relevant to the question, I reckon. -- Giovanni Bajo From lists at cheimes.de Sun Feb 17 22:31:56 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 17 Feb 2008 22:31:56 +0100 Subject: [Python-3000] Windows, sys.argv and unicode In-Reply-To: References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> <47B74106.1060102@v.loewis.de> <47B86A3E.2090503@v.loewis.de> Message-ID: Giovanni Bajo wrote: > On Sun, 17 Feb 2008 18:09:18 +0100, Martin v. L?wis wrote: > >>> Actually, I am. Py3k will already remove usage of stdio, for file input >>> and output. > >>> Are you doing this work on some public branch? >> Yes, on the Py3k branch, and the trunk. > > I looked at fileobject.c in py3k branch today and it looked like it was > still using FILE* everywhere. I'm not sure I know where to look, though. Are you looking at the correct sources? Neither fileobject nor _fileio.c are using FILE except for Py_UniversalNewlineFgets. Christian From rasky at develer.com Sun Feb 17 22:33:46 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 17 Feb 2008 21:33:46 +0000 (UTC) Subject: [Python-3000] Windows: debug mode References: <47B86CA2.7090908@v.loewis.de> Message-ID: On Sun, 17 Feb 2008 18:19:30 +0100, Martin v. L?wis wrote: >> Thus, I'm hereby proposing that we change this in Py3k so that Python's >> debug mode does not mandate any ABI change anymore. > > It seems people still debate what it is that you *actually* want. Yes, sorry for the confusion. > This > sentence suggests Christian's interpretation, i.e. you might be unaware > that the debug mode not only adds debug symbols, but also changes the > layout of objects. That is *very* useful, and should not be dropped. Yes, I know it changes the ABI, and I know it must say as it is. I believe that Py_DEBUG is mostly useful to core developers and not much to extension developer or regular Python developers, but it's immaterial in this discussion. >> This means that a >> programmer will be able to mixmatch release-mode extensions and >> compile- mode extensions, thus getting debuggability only of those >> extensions he really cares. > > Whereas this suggests Phillip's interpretation, i.e. you merely want > debug symbols on a single DLL. Yes I want this. > You can get that today - just enable debug information for your > extension module, and the compiler will emit the PDB files you need for > debugging. Yes, this is what I do everyday by manually hand-edit my setup.py script. Is there a way to make distutils do this for me with a simple command- line switch? I think my request can be resumed like this: 1) Add to distutils a way support to compile extensions with debug informations, without optimizations, and linked with the non-debug CRT. 2) Add the same compilation mode to Python itself, so that users can easily compile a Python DLL which has debug symbols but with the release ABI (since stepping into the Python's core can be useful while debugging an extension). I hope it's clear now :) -- Giovanni Bajo From rasky at develer.com Sun Feb 17 22:45:38 2008 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 17 Feb 2008 21:45:38 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: <47B7147C.10506@cheimes.de> <20080216174039.826583A4075@sparrow.telecommunity.com> <79990c6b0802161029h48c47819k3b997afb88e4aa2@mail.gmail.com> <47B74106.1060102@v.loewis.de> <47B86A3E.2090503@v.loewis.de> Message-ID: On Sun, 17 Feb 2008 22:31:56 +0100, Christian Heimes wrote: > Giovanni Bajo wrote: >> On Sun, 17 Feb 2008 18:09:18 +0100, Martin v. L?wis wrote: >> >>>> Actually, I am. Py3k will already remove usage of stdio, for file >>>> input and output. >> >>>> Are you doing this work on some public branch? >>> Yes, on the Py3k branch, and the trunk. >> >> I looked at fileobject.c in py3k branch today and it looked like it was >> still using FILE* everywhere. I'm not sure I know where to look, >> though. > > Are you looking at the correct sources? Neither fileobject nor _fileio.c > are using FILE except for Py_UniversalNewlineFgets. Doh! I don't know what I was looking at... I probably got my directory crossed. Sorry for the spam. -- Giovanni Bajo From lists at cheimes.de Sun Feb 17 23:11:24 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 17 Feb 2008 23:11:24 +0100 Subject: [Python-3000] Windows: debug mode In-Reply-To: References: <47B86CA2.7090908@v.loewis.de> Message-ID: Giovanni Bajo wrote: > Yes, I know it changes the ABI, and I know it must say as it is. I > believe that Py_DEBUG is mostly useful to core developers and not much to > extension developer or regular Python developers, but it's immaterial in > this discussion. Py_DEBUG builds are useful to extension developers, too. It's the best way to debug reference leaks. You'll need at least a Py_REF_DEBUG build for the task. > Yes, this is what I do everyday by manually hand-edit my setup.py script. > Is there a way to make distutils do this for me with a simple command- > line switch? Yes - provide a patch :) > 2) Add the same compilation mode to Python itself, so that users can > easily compile a Python DLL which has debug symbols but with the release > ABI (since stepping into the Python's core can be useful while debugging > an extension). The Windows build files were carefully design for the job. I've invested quite some time to build a project inheritance tree. All projects inherit settings from pyproject.vsprops. In order to build a release version of Python with less optimization and debug symbols you've to alter the global properties for Release|Win32 and recompile the project. Christian From martin at v.loewis.de Mon Feb 18 19:17:22 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 18 Feb 2008 19:17:22 +0100 Subject: [Python-3000] Windows: debug mode In-Reply-To: References: <47B86CA2.7090908@v.loewis.de> Message-ID: <47B9CBB2.3070105@v.loewis.de> >> Is there a way to make distutils do this for me with a simple command- >> line switch? > > Yes - provide a patch :) > >> 2) Add the same compilation mode to Python itself, so that users can >> easily compile a Python DLL which has debug symbols but with the release >> ABI (since stepping into the Python's core can be useful while debugging >> an extension). > > The Windows build files were carefully design for the job. [...] As he says. Regards, Martin From talin at acm.org Mon Feb 18 20:39:37 2008 From: talin at acm.org (Talin) Date: Mon, 18 Feb 2008 11:39:37 -0800 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> Message-ID: <47B9DEF9.2070606@acm.org> Arnaud Delobelle wrote: > > On Feb 14, 11:13 pm, "Guido van Rossum" wrote: >> On Thu, Feb 14, 2008 at 2:48 PM, Raymond Hettinger wrote: >>> I've been exercising the new keyword-only arguments syntax. It is absolutely wonderful. I'm amazed at how many long standing problems it solves elegantly. >> Agreed. Now can you come up with a syntax for positional-only >> arguments? So far everybody has failed at that, and there are some use >> cases where it's useful too. > > The idea in PEP 3102 is that *args gobbles up all remaining positional > arguments, hence what comes after can only be interpreted as > keyword-only arguments. This is extended to allowing * on its own. > > e.g. f(*, z, t=2) > => z, t are keyword-only > > Following the same train of thought, **kwargs gobbles up all remaining > keyword arguments, hence what comes after can only interpreted as > positional-only arguments. Then one could extend it to allowing ** on > its own. > > e.g. f(**, x, y=1) > => x, y are positional-only > > Now, how do keyword-only arguments fit in? Say x, y=1 are > positional-only and z, t=2 are keyword-only. One could have: > > f(**, x, y=1, *, z, t=2) > > The general cases could look like > > f(x, y=1, *args, z, t=2, **kwargs): > => x, y are hybrid > => args: all remaining positional arguments > => z, t are keyword-only > => kwargs: all remaining keyword arguments > > f(**kwargs, x, y=1, *args, z, t=2): > => x, y are positional-only > => args: all remaining positional arguments > => z, t are keyword-only > => kwargs: all remaining keyword arguments > > 'args' and 'kwargs' are optional in both cases. If 'kwargs' is present > in the second version, it is a bit strange as it usually only appears > at the end. Alternatively, only '**' could be allowed at the start > and if needed '**kwargs' would come at the end: > > f(**, x, y=1, *args, z, t=2, **kwargs) > > Note that this syntax doesn't allow for positional-only and hybrid > arguments to coexist. But this is nice IMHO, as it seems to me that > it would only create confusion. > > Unconvincingly yours, Well, that is certainly a logical continuation of the train of thought behind the 'single *' syntax. I'd be curious to know which parts of 3102 people are finding most useful. You see, the itch that I was originally attempting to scratch concerned just the narrow use case of combining a 'varargs' function with optional keywords. I've seen/written a lot of code where you have a function that takes a list of arguments, with a couple of optional keywords that control how that list is handled, and 3102 was intended to make that case much easier to handle. I was never all that interested in 'forcing' some of the arguments to a non-varargs function to be specified by keyword. I am perfectly comfortable with the idea that the caller always gets to choose whether to pass something positionally or via keyword, and limiting the caller's choice seems vaguely unpythonic to me. However, I included that part in the PEP based on the feedback from here. In any case, limiting arguments to being specified by keyword-only or positional-only was not part of my original 'itch', and I am curious as to what are the use cases for it that people are envisioning. > -- > Arnaud > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org > From arnodel at googlemail.com Mon Feb 18 21:17:41 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Mon, 18 Feb 2008 20:17:41 +0000 Subject: [Python-3000] PEP 3102 In-Reply-To: <47B9DEF9.2070606@acm.org> References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B9DEF9.2070606@acm.org> Message-ID: <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com> On 18 Feb 2008, at 19:39, Talin wrote: > Arnaud Delobelle wrote: >> [...] >> Unconvincingly yours, > > Well, that is certainly a logical continuation of the train of > thought behind the 'single *' syntax. > I'd be curious to know which parts of 3102 people are finding most > useful. You see, the itch that I was originally attempting to > scratch concerned just the narrow use case of combining a 'varargs' > function with optional keywords. I've seen/written a lot of code > where you have a function that takes a list of arguments, with a > couple of optional keywords that control how that list is handled, > and 3102 was intended to make that case much easier to handle. Another use would be allowing the '_cache trick' with a varargs function, i.e. def f(*args, _cache={}): ... Personally I don't like this trick though... > I was never all that interested in 'forcing' some of the arguments > to a non-varargs function to be specified by keyword. I am perfectly > comfortable with the idea that the caller always gets to choose > whether to pass something positionally or via keyword, and limiting > the caller's choice seems vaguely unpythonic to me. However, I > included that part in the PEP based on the feedback from here. > > In any case, limiting arguments to being specified by keyword-only > or positional-only was not part of my original 'itch', and I am > curious as to what are the use cases for it that people are > envisioning. > A typical use of positional-only arguments is with a function def f(x, y=1, **kwargs): ... where keyword arguments are potentially anything at all, including x and y. For example: dict.update(). In fact it is a fairly symmetrical itch to yours: without positional only arguments, the above must be written something like: def f(*args, **kwargs): nargs = len(args) if 1 <= nargs <= 2: x = args[0] y = args[1] if nargs==2 else 1 else: raise blah ... whereas with positional-only arguments (using the syntax I put forward) one could write simply: def f(**kwargs, x, y=1) Just like def f(*args, a, b=1): ... had to be written pre-PEP 3102 as something like: def f(*args, **kwargs): a = kwargs.pop('a') b = kwargs.pop('b', 1) if kwargs: raise blah Both itches are about not having to unpack something: **kwargs in the case of PEP 3102, *args in the case of positional-only arguments. -- Arnaud From kumar.mcmillan at gmail.com Tue Feb 19 01:32:35 2008 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Mon, 18 Feb 2008 18:32:35 -0600 Subject: [Python-3000] detect file encoding or always use the default, UTF-8? Message-ID: Hello. In Python 3, when opening a file without declaring the encoding keyword is Python going to guess the encoding of the file or just use the default? That is, assume the file is UTF-8? What led me to wonder this is that on Mac OS X 10.4.11, opening a file containing UTF-8 encoded text creates a unicode object under the assumption the text was "mac roman" encoding. This seems like a bug. It works if I set the encoding to UTF-8 explicitly. I can post the code but it sounds like there are some Mac encoding issues in flux so I thought to ask first. thanks, Kumar From rhamph at gmail.com Tue Feb 19 02:29:39 2008 From: rhamph at gmail.com (Adam Olsen) Date: Mon, 18 Feb 2008 18:29:39 -0700 Subject: [Python-3000] PEP 3102 In-Reply-To: <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com> References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B9DEF9.2070606@acm.org> <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com> Message-ID: On Feb 18, 2008 1:17 PM, Arnaud Delobelle wrote: > A typical use of positional-only arguments is with a function > > def f(x, y=1, **kwargs): > ... > > where keyword arguments are potentially anything at all, including x > and y. For example: dict.update(). In fact it is a fairly > symmetrical itch to yours: without positional only arguments, the > above must be written something like: I worry that dict is not just an example, it may be the *only* example. Does somebody have a use case that doesn't involve dict? Whereas keyword-only arguments force explicitness (at the cost of brevity), positional-only arguments do the opposite. The default already allows us to choose brevity when appropriate, and the only reason I see to force it is when there's a conflict with **kwargs. Such obscure features warrant an obscure yet well-named decorator, not an obscure syntax that's difficult to get help on. -- Adam Olsen, aka Rhamphoryncus From ncoghlan at gmail.com Tue Feb 19 11:49:28 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Feb 2008 20:49:28 +1000 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B9DEF9.2070606@acm.org> <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com> Message-ID: <47BAB438.9050504@gmail.com> Adam Olsen wrote: > Whereas keyword-only arguments force explicitness (at the cost of > brevity), positional-only arguments do the opposite. The default > already allows us to choose brevity when appropriate, and the only > reason I see to force it is when there's a conflict with **kwargs. Which for me happens when I have a method which I want to accept arbitrary keyword arguments (I'll grant that isn't very often, but it has happened). I have to choose between disallowing 'self' or '_self' as one of the no-longer-quite-so-arbitrary keyword arguments, or else I have to do the song and dance to implement manual unpacking. > Such obscure features warrant an obscure yet well-named decorator, not > an obscure syntax that's difficult to get help on. It's difficult to handle this in a decorator, as the problem that occurs is the underlying function reporting duplicate definitions for a parameter. >>> def f(self, **kwds): ... print self, kwds ... >>> f(1, self=2) Traceback (most recent call last): File "", line 1, in TypeError: f() got multiple values for keyword argument 'self' There really isn't a lot a decorator can do to prevent this, unless we modify function calls to permit direct assignment of the keywords dictionary without unpacking it. Then it would become possible to provide a decorator in functools along the lines of: def positional_only(required=1): def wrap(f): @wraps(f) def wrapper(*args, **kwds): if len(args) < required: raise TypeError("Need %d positional arguments " "(%d given)" % (required, len(args))) f(*args, **=kwds) The idea of the **= syntax would be that the supplied dictionary is used directly as the arbitrary keywords parameter for the underlying function. Other named arguments would *not* be extracted from that dictionary (obviously, the above decorator would need further work in order to correctly handle parameters which can passed as either positional or keywords parameters). Personally, I don't think the problem comes up often enough to be overly worried about it, as much as my sense of aesthetics would like to see an elegant solution (as judged by own personal definition of elegance of course ;) We've also veered fairly far off topic for the Py3k list - further ideas for positional-only argument syntax or decorators should probably be kicked around on python-ideas rather than here or python-dev. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Tue Feb 19 19:33:03 2008 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Feb 2008 10:33:03 -0800 Subject: [Python-3000] detect file encoding or always use the default, UTF-8? In-Reply-To: References: Message-ID: Well, we're basically hoping that the folks who actually uses Python to read and write text files containing non-ASCII characters on OSX tell us what they want. At least that's where I am. Since I personally still live in a nearly-ASCII world (and probably always will), my own experience just doesn't give me any guidance as to what would be the most useful. How does Apple's TextEdit do the guessing (assuming it guesses at all)? I just typed a few non-ASCII characters into a text file with TextEdit on OSX 10.5, and it seems to have written the file in MacRoman. This with the preferences for open and save set to automatic. --Guido On Feb 18, 2008 4:32 PM, Kumar McMillan wrote: > Hello. > > In Python 3, when opening a file without declaring the encoding > keyword is Python going to guess the encoding of the file or just use > the default? That is, assume the file is UTF-8? > > What led me to wonder this is that on Mac OS X 10.4.11, opening a file > containing UTF-8 encoded text creates a unicode object under the > assumption the text was "mac roman" encoding. This seems like a bug. > It works if I set the encoding to UTF-8 explicitly. I can post the > code but it sounds like there are some Mac encoding issues in flux so > I thought to ask first. > > thanks, Kumar > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From arnodel at googlemail.com Tue Feb 19 19:45:06 2008 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Tue, 19 Feb 2008 18:45:06 +0000 Subject: [Python-3000] PEP 3102 In-Reply-To: <47BAB438.9050504@gmail.com> References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B9DEF9.2070606@acm.org> <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com> <47BAB438.9050504@gmail.com> Message-ID: <2C0C17DD-7DD1-4563-BB6E-9F3DC9491EF6@gmail.com> On 19 Feb 2008, at 10:49, Nick Coghlan wrote: [...] >> Such obscure features warrant an obscure yet well-named decorator, >> not >> an obscure syntax that's difficult to get help on. > > It's difficult to handle this in a decorator, as the problem that > occurs is the underlying function reporting duplicate definitions > for a parameter. > > >>> def f(self, **kwds): > ... print self, kwds > ... > >>> f(1, self=2) > Traceback (most recent call last): > File "", line 1, in > TypeError: f() got multiple values for keyword argument 'self' > > There really isn't a lot a decorator can do to prevent this, unless > we modify function calls to permit direct assignment of the keywords > dictionary without unpacking it. Then it would become possible to > provide a decorator in functools along the lines of: > > def positional_only(required=1): > def wrap(f): > @wraps(f) > def wrapper(*args, **kwds): > if len(args) < required: > raise TypeError("Need %d positional arguments " > "(%d given)" % (required, len(args))) > f(*args, **=kwds) > [...] > We've also veered fairly far off topic for the Py3k list - further > ideas for positional-only argument syntax or decorators should > probably be kicked around on python-ideas rather than here or python- > dev. For reference, here was a long thread on this on python-ideas [1]. Several syntax options were discussed, I proposed using a decorator and several ways of achieving this were posted: * by Steven Bethard [2] * by me [3], [4] I posted a cookbook recipe based on [4]. [1] http://mail.python.org/pipermail/python-ideas/2007-May/000704.html [2] http://mail.python.org/pipermail/python-ideas/2007-May/000763.html [3] http://mail.python.org/pipermail/python-ideas/2007-May/000808.html [4] http://mail.python.org/pipermail/python-ideas/2007-May/000860.html -- Arnaud From kumar.mcmillan at gmail.com Tue Feb 19 20:35:38 2008 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Tue, 19 Feb 2008 13:35:38 -0600 Subject: [Python-3000] detect file encoding or always use the default, UTF-8? In-Reply-To: References: Message-ID: Hi. I was mainly confused about the auto-detection of the encoding. I thought I read something earlier that Python 3 will not try to guess encodings and that opening files will always use the sys encoding (defaulting to UTF-8) instead of guessing. Can't find where I read that so I might have made it up :D Now at least I know it will be guessing encodings. On Feb 19, 2008 12:33 PM, Guido van Rossum wrote: > Well, we're basically hoping that the folks who actually uses Python > to read and write text files containing non-ASCII characters on OSX > tell us what they want. At least that's where I am. Since I personally > still live in a nearly-ASCII world (and probably always will), my own > experience just doesn't give me any guidance as to what would be the > most useful. I too pretty much work entirely in ascii for day to day stuff so it's hard to say. > > How does Apple's TextEdit do the guessing (assuming it guesses at all)? > > I just typed a few non-ASCII characters into a text file with TextEdit > on OSX 10.5, and it seems to have written the file in MacRoman. This > with the preferences for open and save set to automatic. Were you saving in RTF format or plain text? TextEdit is strange in that it makes it hard to save a plain text file so if you were working with RTF files I'd think you'd have to open it from Python in binary mode anyway, no? But for plain text TextEdit doesn't guess. I typed some non-ascii and went to format -> Make plain text. Then in the save dialog it defaulted to UTF-16 and the only options (without making custom modifications) were UTF-8 and simplified Chinese. However, this was on 10.4; someone checked 10.5 for me and said the default is UTF-8 (sounds like they fixed something!). thanks, Kumar From hashcollision at gmail.com Tue Feb 19 20:54:40 2008 From: hashcollision at gmail.com (hashcollision) Date: Tue, 19 Feb 2008 14:54:40 -0500 Subject: [Python-3000] Library Change Request: change functools.partial.keywords to kwargs Message-ID: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com> I have a library change request: currently partial Objects created by functools.partial has the "keywords" attribute. However, the standard name for something like it is kwargs. I request that it be renamed to "kwargs". http://docs.python.org/lib/partial-objects.html From brett at python.org Tue Feb 19 21:03:31 2008 From: brett at python.org (Brett Cannon) Date: Tue, 19 Feb 2008 12:03:31 -0800 Subject: [Python-3000] Library Change Request: change functools.partial.keywords to kwargs In-Reply-To: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com> References: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com> Message-ID: On Feb 19, 2008 11:54 AM, hashcollision wrote: > I have a library change request: currently partial Objects created by > functools.partial has the "keywords" attribute. However, the standard > name for something like it is kwargs. I request that it be renamed to > "kwargs". -1 The change has minor benefit while breaking code in the process. If you want this to happen it is going to take at least a full release cycle of having a deprecation warning on the old attribute name and I still don't see enough benefit to bother. -Brett From collinw at gmail.com Tue Feb 19 21:03:40 2008 From: collinw at gmail.com (Collin Winter) Date: Tue, 19 Feb 2008 12:03:40 -0800 Subject: [Python-3000] Library Change Request: change functools.partial.keywords to kwargs In-Reply-To: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com> References: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com> Message-ID: <43aa6ff70802191203m71594d56gb15f2806d5a0ad22@mail.gmail.com> On Feb 19, 2008 11:54 AM, hashcollision wrote: > I have a library change request: currently partial Objects created by > functools.partial has the "keywords" attribute. However, the standard > name for something like it is kwargs. I request that it be renamed to > "kwargs". Is it conflicting with something you're doing or this just a personal aesthetic preference? Collin Winter From ncoghlan at gmail.com Wed Feb 20 00:14:56 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Feb 2008 09:14:56 +1000 Subject: [Python-3000] Library Change Request: change functools.partial.keywords to kwargs In-Reply-To: References: <37f76d50802191154r56f7c655xb1b5b91dbcba23cc@mail.gmail.com> Message-ID: <47BB62F0.5050202@gmail.com> Brett Cannon wrote: > On Feb 19, 2008 11:54 AM, hashcollision wrote: >> I have a library change request: currently partial Objects created by >> functools.partial has the "keywords" attribute. However, the standard >> name for something like it is kwargs. I request that it be renamed to >> "kwargs". > > -1 > > The change has minor benefit while breaking code in the process. If > you want this to happen it is going to take at least a full release > cycle of having a deprecation warning on the old attribute name and I > still don't see enough benefit to bother. Calling 'kwargs' a standard name is overstating the case quite a bit as well (I always use '**kwds' myself). Cheers, Nick. P.S. -1 if that wasn't obvious ;) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jimjjewett at gmail.com Wed Feb 20 02:33:48 2008 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 19 Feb 2008 20:33:48 -0500 Subject: [Python-3000] PEP 3102 In-Reply-To: References: <20080214174812.AGY90177@ms19.lnh.mail.rcn.net> <47B9DEF9.2070606@acm.org> <6AF2BB12-ECFE-4B32-A6F8-DE589C49E213@gmail.com> Message-ID: On 2/18/08, Adam Olsen wrote: > On Feb 18, 2008 1:17 PM, Arnaud Delobelle wrote: > > A typical use of positional-only arguments is with a function > > def f(x, y=1, **kwargs): > > ... > > > > where keyword arguments are potentially anything at all, including x > > and y. For example: dict.update(). > I worry that dict is not just an example, it may be the *only* > example. Does somebody have a use case that doesn't involve dict? Some existing functions are positional only, often because they expose underlying (positional-only) C library functions. Attemping to make a duck-type replacement just doesn't feel as clean if the arguments are named. That said, I don't think it does any actual damage except for the "and it takes arbitrary keywords" case. Even then, a reserved set (such as __* names) is usually acceptable. -jJ From ronaldoussoren at mac.com Wed Feb 20 15:54:30 2008 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 20 Feb 2008 15:54:30 +0100 Subject: [Python-3000] detect file encoding or always use the default, UTF-8? In-Reply-To: References: Message-ID: On 19 Feb, 2008, at 19:33, Guido van Rossum wrote: > Well, we're basically hoping that the folks who actually uses Python > to read and write text files containing non-ASCII characters on OSX > tell us what they want. At least that's where I am. Since I personally > still live in a nearly-ASCII world (and probably always will), my own > experience just doesn't give me any guidance as to what would be the > most useful. I'd prefer UTF-8 because that's a much more standard format in the wider world. > > > How does Apple's TextEdit do the guessing (assuming it guesses at > all)? TextEdit by default saves as UTF-8 on 10.5 (at encoding set to "automatic" in the preferences). Ronald > > > I just typed a few non-ASCII characters into a text file with TextEdit > on OSX 10.5, and it seems to have written the file in MacRoman. This > with the preferences for open and save set to automatic. > > --Guido > > On Feb 18, 2008 4:32 PM, Kumar McMillan > wrote: >> Hello. >> >> In Python 3, when opening a file without declaring the encoding >> keyword is Python going to guess the encoding of the file or just use >> the default? That is, assume the file is UTF-8? >> >> What led me to wonder this is that on Mac OS X 10.4.11, opening a >> file >> containing UTF-8 encoded text creates a unicode object under the >> assumption the text was "mac roman" encoding. This seems like a bug. >> It works if I set the encoding to UTF-8 explicitly. I can post the >> code but it sounds like there are some Mac encoding issues in flux so >> I thought to ask first. >> >> thanks, Kumar >> _______________________________________________ >> Python-3000 mailing list >> Python-3000 at python.org >> http://mail.python.org/mailman/listinfo/python-3000 >> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ronaldoussoren%40mac.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20080220/f294a21e/attachment.bin From kumar.mcmillan at gmail.com Wed Feb 20 16:30:57 2008 From: kumar.mcmillan at gmail.com (Kumar McMillan) Date: Wed, 20 Feb 2008 09:30:57 -0600 Subject: [Python-3000] detect file encoding or always use the default, UTF-8? In-Reply-To: References: Message-ID: On Feb 20, 2008 8:54 AM, Ronald Oussoren wrote: > > I'd prefer UTF-8 because that's a much more standard format in the > wider world. I agree. Mac OS Roman is only marginally better than ASCII. UTF-8 would be more versatile. From python3now at gmail.com Wed Feb 20 20:34:22 2008 From: python3now at gmail.com (James Thiele) Date: Wed, 20 Feb 2008 11:34:22 -0800 Subject: [Python-3000] PEP 3119 "Open issues" Message-ID: <8f01efd00802201134v96c9bccu9095ea178631e8df@mail.gmail.com> PEP 3119 has some "Open issues", three of which I list below. These have (I assume) probably been resolved but it is hard to know what search terms to use to find them in this mailing list. It would be useful to record the decisions in the PEP, or at least resolve them if they are still open issues. (in "One Trick Ponies") Open issues: Conceivably, instead of using the ABCMeta metaclass, these classes could override __instancecheck__ and __subclasscheck__ to check for the presence of the applicable special method (in Sequence) Open issues: Other candidate methods, which can all have default concrete implementations that only depend on __len__ and __getitem__ with an integer argument: __reversed__, index, count, __add__, __mul__. (in Strings) Open issues: define the base interfaces for these so alternative implementations and subclasses know what they are in for. This may be the subject of a new PEP or PEPs (PEP 358 should be co-opted for the bytes type). From barry at python.org Thu Feb 21 16:01:56 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 21 Feb 2008 10:01:56 -0500 Subject: [Python-3000] Putting pdb.set_trace() in builtins? Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I know this will get shot down like a Navy missile aimed at an errant spy satellite, but I'm going to suggest it anyway. Several of us have been talking about merging the two Emacs Python modes and this prompted a discussion of Ken Manheimer's excellent pdbtrack hack, which got me thinking about the way I use pdb. I don't know if this is typical or unique, or whether anyone else is annoyed the same way I am. Admittedly it's a minor annoyance, but I still think we can do better. So I have some particularly troublesome code that I want to set a breakpoint at and start stepping through. I usually will stick this tidbit at the place I want to break: good_code() import pdb; pdb.set_trace() call_buggy_code() In doctests it's a little ickier because I have to put that all on the line where I want to break, not before it or doctest won't do the right thing, e.g.: >>> import pdb; pdb.set_trace(); call_buggy_code() I suggest that setting a breakpoint be done with a new built-in, e.g breakpoint(). So now I would do this: good_code() breakpoint() call_buggy_code() or >>> breakpoint(); call_buggy_code() Yes, all it saves me is an import, but it also elevates debugging to a more important, and more visible location. Okay, so now that I've convinced you this is the most brilliant idea this century , I'm really going to smoke some crack. The other thing this should do is allow us to use different debuggers if we wanted. Maybe today breakpoint() would call into pdb, but maybe tomorrow we'll have a better debugger, or someone will have installed a better debugger in site-packages (e.g. pydb). I would suggest that breakpoint() consult an environment variable such as PYTHONDEBUGGER to see which debugger to invoke (the first time in a session only of course). Perhaps you could also set PYTHONDEBUGGER=ignore to noop any and all calls to breakpoint. Maybe there's a python command line switch to mirror $PYTHONDEBUGGER. trying-not-to-spray-hydrazine-all-over-you-ly y'rs, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR72SZXEjvBPtnXfVAQK7gwP7BKyIMAarMZEPzTQVfR/W8J4KzBEQFhtj 3pLd83I8oQMgP4SD0BapiJZ2BOws3i2TAZ501V8VF8iR50nyNGTYCP4pJ/sy6Cr0 Q79ahrvKq5+eS/17+Zi/c/yC028h585m1pvg9/VALu1decLvz3m6UPxJWsZJ+ldw Qj5bUtuCJ7c= =YBMp -----END PGP SIGNATURE----- From exarkun at divmod.com Thu Feb 21 16:23:48 2008 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 21 Feb 2008 10:23:48 -0500 Subject: [Python-3000] Putting pdb.set_trace() in builtins? In-Reply-To: Message-ID: <20080221152348.6859.1072137842.divmod.quotient.11578@ohm> On Thu, 21 Feb 2008 10:01:56 -0500, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >I know this will get shot down like a Navy missile aimed at an errant >spy satellite, but I'm going to suggest it anyway. > >Several of us have been talking about merging the two Emacs Python >modes and this prompted a discussion of Ken Manheimer's excellent >pdbtrack hack, which got me thinking about the way I use pdb. I don't >know if this is typical or unique, or whether anyone else is annoyed >the same way I am. Admittedly it's a minor annoyance, but I still >think we can do better. > >So I have some particularly troublesome code that I want to set a >breakpoint at and start stepping through. I usually will stick this >tidbit at the place I want to break: > > good_code() > import pdb; pdb.set_trace() > call_buggy_code() I do this all the time, too. Basically any time I need to use a debugger to fix something. > >In doctests it's a little ickier because I have to put that all on the >line where I want to break, not before it or doctest won't do the >right thing, e.g.: > > >>> import pdb; pdb.set_trace(); call_buggy_code() > >I suggest that setting a breakpoint be done with a new built-in, e.g >breakpoint(). So now I would do this: > > good_code() > breakpoint() > call_buggy_code() > >or > > >>> breakpoint(); call_buggy_code() > >Yes, all it saves me is an import, but it also elevates debugging to a >more important, and more visible location. > >Okay, so now that I've convinced you this is the most brilliant idea >this century , I'm really going to smoke some crack. > >The other thing this should do is allow us to use different debuggers >if we wanted. Maybe today breakpoint() would call into pdb, but maybe >tomorrow we'll have a better debugger, or someone will have installed >a better debugger in site-packages (e.g. pydb). I would suggest that >breakpoint() consult an environment variable such as PYTHONDEBUGGER to >see which debugger to invoke (the first time in a session only of >course). Perhaps you could also set PYTHONDEBUGGER=ignore to noop any >and all calls to breakpoint. Maybe there's a python command line >switch to mirror $PYTHONDEBUGGER. > >trying-not-to-spray-hydrazine-all-over-you-ly y'rs, >- -Barry Wouldn't a less invasive solution be to make pdb better? In particular, to make `breakpoint? actually reliably set a breakpoint? Jean-Paul From facundobatista at gmail.com Thu Feb 21 16:28:40 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 21 Feb 2008 12:28:40 -0300 Subject: [Python-3000] Putting pdb.set_trace() in builtins? In-Reply-To: References: Message-ID: 2008/2/21, Barry Warsaw : > good_code() > import pdb; pdb.set_trace() > call_buggy_code() > > ... > > good_code() > breakpoint() > call_buggy_code() "import gdb;gdb.set_trace()" is my second-most-used debugging tool (the first one is "print"), so I'm with you here! +1 -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From aahz at pythoncraft.com Thu Feb 21 16:41:32 2008 From: aahz at pythoncraft.com (Aahz) Date: Thu, 21 Feb 2008 07:41:32 -0800 Subject: [Python-3000] Putting pdb.set_trace() in builtins? In-Reply-To: References: Message-ID: <20080221154132.GA9990@panix.com> On Thu, Feb 21, 2008, Barry Warsaw wrote: > > So I have some particularly troublesome code that I want to set a > breakpoint at and start stepping through. I usually will stick this > tidbit at the place I want to break: > > good_code() > import pdb; pdb.set_trace() > call_buggy_code() > > In doctests it's a little ickier because I have to put that all on the > line where I want to break, not before it or doctest won't do the > right thing, e.g.: > > >>> import pdb; pdb.set_trace(); call_buggy_code() > > I suggest that setting a breakpoint be done with a new built-in, e.g > breakpoint(). So now I would do this: > > good_code() > breakpoint() > call_buggy_code() -0 >From my POV, this is cruft in the builtins (I mostly do web apps) -- and I don't particularly like your suggestions for modifying the result of breakpoint() (note that Facundo wants to use gdb). Why not simply inject set_trace() into the builtins in your startup code? You can control whether it gets injected with a DEBUG env var or something. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "All problems in computer science can be solved by another level of indirection." --Butler Lampson From facundobatista at gmail.com Thu Feb 21 16:51:09 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 21 Feb 2008 12:51:09 -0300 Subject: [Python-3000] Putting pdb.set_trace() in builtins? In-Reply-To: <20080221154132.GA9990@panix.com> References: <20080221154132.GA9990@panix.com> Message-ID: 2008/2/21, Aahz : > >From my POV, this is cruft in the builtins (I mostly do web apps) -- and > I don't particularly like your suggestions for modifying the result of > breakpoint() (note that Facundo wants to use gdb). Why not simply No, no, my mistake, I also use pdb. BTW, I use this even when doing PyGTK... didn't try it on web development, though. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From rasky at develer.com Thu Feb 21 21:18:19 2008 From: rasky at develer.com (Giovanni Bajo) Date: Thu, 21 Feb 2008 20:18:19 +0000 (UTC) Subject: [Python-3000] Windows, sys.argv and unicode References: Message-ID: On Sat, 16 Feb 2008 16:32:43 +0000, Giovanni Bajo wrote: >> At the very least, can you add the issue to the tracker so we'll have a >> permanent reminder until it's resolved? > > Done: issue 2128. The patch was uploaded a few days ago. Can any Windows maintainer find some time to review it? Thanks! -- Giovanni Bajo From dalke at dalkescientific.com Wed Feb 20 21:45:40 2008 From: dalke at dalkescientific.com (Andrew Dalke) Date: Wed, 20 Feb 2008 21:45:40 +0100 Subject: [Python-3000] removing destructuring del Message-ID: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> For lack of a better term, I'll call this a destructuring del del (a, (b, (c))) I propose that it should not be valid in Python 3. It's supported in 2.x and 3.0a2. It's never caused anyone problems. No one (that I can find through 10 minutes of grepping) uses it. When I figured out that I could do it, by looking at the grammar, I didn't believe it was valid. I still can't figure out why anyone would use it. Python 3 removes support for destructuring in function calls def f((a,b)): # SyntaxError in Python3 pass and I think this falls into the same category. I would even argue that del(x) should not be allowed because it's suggests and is likely based on the false belief that 'del' is a function call. But I'm not proposing getting rid of return(x). Andrew dalke at dalkescientific.com From guido at python.org Fri Feb 22 02:25:32 2008 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Feb 2008 17:25:32 -0800 Subject: [Python-3000] removing destructuring del In-Reply-To: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> Message-ID: What syntax would you allow instead? Just del variable, variable, ...? On Wed, Feb 20, 2008 at 12:45 PM, Andrew Dalke wrote: > For lack of a better term, I'll call this a destructuring del > > del (a, (b, (c))) > > I propose that it should not be valid in Python 3. > > It's supported in 2.x and 3.0a2. It's never caused anyone problems. > No one (that I can find through 10 minutes of grepping) uses it. > When I figured out that I could do it, by looking at the grammar, I > didn't believe it was valid. I still can't figure out why anyone > would use it. > > Python 3 removes support for destructuring in function calls > > def f((a,b)): # SyntaxError in Python3 > pass > > and I think this falls into the same category. > > I would even argue that > > del(x) > > should not be allowed because it's suggests and is likely based on > the false belief that 'del' is a function call. But I'm not > proposing getting rid of return(x). > > > Andrew > dalke at dalkescientific.com > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dalke at dalkescientific.com Fri Feb 22 08:27:37 2008 From: dalke at dalkescientific.com (Andrew Dalke) Date: Fri, 22 Feb 2008 08:27:37 +0100 Subject: [Python-3000] removing destructuring del In-Reply-To: References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> Message-ID: <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> On Feb 22, 2008, at 2:25 AM, Guido van Rossum wrote: > What syntax would you allow instead? Just del variable, variable, ...? Yes. These are and should be legal del a, b[1], c.d del (e+f).g[9]().h but I think this should not be legal del a, (b, c) That should raise SyntaxError: can't delete tuple and I would also like it if the following raised the same error del (a, b) but people use it now and I can see that perhaps, like with the from statement, using '()' for long expressions is useful del (some_very_large_variable_name[1], another_large_variable_name[2].with_attribute) I also think del a, should not be legal ("SyntaxError: trailing comma not allowed without surrounding parentheses"?), but that's getting into my own personal preferences. Andrew dalke at dalkescientific.com From guido at python.org Fri Feb 22 16:44:25 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 22 Feb 2008 07:44:25 -0800 Subject: [Python-3000] removing destructuring del In-Reply-To: <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> Message-ID: On Thu, Feb 21, 2008 at 11:27 PM, Andrew Dalke wrote: > On Feb 22, 2008, at 2:25 AM, Guido van Rossum wrote: > > What syntax would you allow instead? Just del variable, variable, ...? > > Yes. These are and should be legal > > del a, b[1], c.d > del (e+f).g[9]().h > > but I think this should not be legal > > del a, (b, c) > > That should raise > > SyntaxError: can't delete tuple > > and I would also like it if the following raised the same error > > del (a, b) > > but people use it now and I can see that perhaps, like with the from > statement, using '()' for long expressions is useful > > del (some_very_large_variable_name[1], > another_large_variable_name[2].with_attribute) Well, you could rewrite that as two del statements of course... The only use I can think if would be del (very_long_name. very_long_attribute) > I also think > > del a, > > should not be legal ("SyntaxError: trailing comma not allowed without > surrounding parentheses"?), but that's getting into my own personal > preferences. These all make sense to me, but at the same time it seems such a minor issue that I'm not sure we should bother. It would probably end up being more custom syntax -- right now it just uses exprlist in the grammar and is limited to assignment targets by the compiler. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From alexander.belopolsky at gmail.com Fri Feb 22 18:07:12 2008 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Fri, 22 Feb 2008 17:07:12 +0000 (UTC) Subject: [Python-3000] removing destructuring del References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> Message-ID: Guido van Rossum python.org> writes: > > On Thu, Feb 21, 2008 at 11:27 PM, Andrew Dalke > dalkescientific.com> wrote: > .. I think this should not be legal > > > > del a, (b, c) > > > > That should raise > > > > SyntaxError: can't delete tuple .. > These all make sense to me, but at the same time it seems such a minor > issue that I'm not sure we should bother. It would probably end up > being more custom syntax -- right now it just uses exprlist in the > grammar and is limited to assignment targets by the compiler. > I was always wondering why both lists and tuples are allowed as assignment targets. As far as I can tell, the only difference is in the corner cases such as >>> del [] >>> del () File "", line 1 SyntaxError: can't assign to () >>> [] = () >>> () = () File "", line 1 SyntaxError: can't assign to () Maybe Py3k should eliminate the list form, but allow empty tuples as useful in auto-generated code (deleting variable number of names.) From greg.ewing at canterbury.ac.nz Fri Feb 22 21:08:14 2008 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 23 Feb 2008 09:08:14 +1300 Subject: [Python-3000] removing destructuring del In-Reply-To: References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> Message-ID: <47BF2BAE.9090709@canterbury.ac.nz> Alexander Belopolsky wrote: > I was always wondering why both lists and tuples are allowed as > assignment targets. I seem to remember a very early version of Python when you had to use the list form for unpacking lists and the tuple form for unpacking tuples, and you'd be told off for using the wrong one. That was quickly seen to be a silly idea, but both syntaxes were kept for compatibility. -- Greg From dalke at dalkescientific.com Fri Feb 22 21:42:18 2008 From: dalke at dalkescientific.com (Andrew Dalke) Date: Fri, 22 Feb 2008 21:42:18 +0100 Subject: [Python-3000] removing destructuring del In-Reply-To: References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> Message-ID: <1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com> On Feb 22, 2008, at 6:07 PM, Alexander Belopolsky wrote: > Maybe Py3k should eliminate the list form, but allow empty tuples > as useful in auto-generated code (deleting variable number of > names.) That seems pretty theoretical. It's easy to add a "if has variables" check first. I looked at quite a few libraries grepping for unusual del statements and didn't find any, including nothing like 'del []' BTW, there's a bug in the error message for 'del ()' Python 2.5 (r25:51908, Oct 9 2006, 13:24:22) [GCC 3.4.6 [FreeBSD] 20060305] on freebsd6 Type "help", "copyright", "credits" or "license" for more information. >>> del [] >>> del () File "", line 1 SyntaxError: can't assign to () >>> Python 3.0a2 (r30a2:59382, Feb 20 2008, 12:09:57) [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> del [] >>> del () SyntaxError: can't assign to () (, line 1) >>> This is the exception I want to see when someone does a destructuring del, though with corrected wording. Andrew dalke at dalkescientific.com From guido at python.org Fri Feb 22 22:49:16 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 22 Feb 2008 13:49:16 -0800 Subject: [Python-3000] removing destructuring del In-Reply-To: <1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com> References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> <1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com> Message-ID: On Fri, Feb 22, 2008 at 12:42 PM, Andrew Dalke wrote: > On Feb 22, 2008, at 6:07 PM, Alexander Belopolsky wrote: > > Maybe Py3k should eliminate the list form, but allow empty tuples > > as useful in auto-generated code (deleting variable number of > > names.) > > That seems pretty theoretical. It's easy to add a "if has variables" > check first. I looked at quite a few libraries grepping for unusual > del statements and didn't find any, including nothing like 'del []' > > BTW, there's a bug in the error message for 'del ()' > > Python 2.5 (r25:51908, Oct 9 2006, 13:24:22) > [GCC 3.4.6 [FreeBSD] 20060305] on freebsd6 > Type "help", "copyright", "credits" or "license" for more information. > > >>> del [] > >>> del () > File "", line 1 > SyntaxError: can't assign to () > >>> The only bug I see here is that it doesn't complain about "del []". > Python 3.0a2 (r30a2:59382, Feb 20 2008, 12:09:57) > [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> del [] > >>> del () > SyntaxError: can't assign to () (, line 1) > >>> > > This is the exception I want to see when someone does a destructuring > del, though with corrected wording. I guess if you're really motivated to get this through, you'll have to submit a patch, including unittests and doc updates. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dalke at dalkescientific.com Fri Feb 22 23:00:35 2008 From: dalke at dalkescientific.com (Andrew Dalke) Date: Fri, 22 Feb 2008 23:00:35 +0100 Subject: [Python-3000] removing destructuring del In-Reply-To: References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> <1D75411D-E13B-4322-9F2F-B4A4F7C390C4@dalkescientific.com> Message-ID: <12FF10F2-E91D-47A3-B9CF-416BFFADF55F@dalkescientific.com> On Feb 22, 2008, at 10:49 PM, Guido van Rossum wrote: >>>>> del [] >>>>> del () >> File "", line 1 >> SyntaxError: can't assign to () >>>>> > > The only bug I see here is that it doesn't complain about "del []". I expected it to say "can't delete" instead of "can't assign" >>> del 1 SyntaxError: can't delete literal (, line 1) > I guess if you're really motivated to get this through, you'll have to > submit a patch, including unittests and doc updates. I figured that was a likely response - and it's better than "nope" or "too late". Mmm, _ast.c, I presume? Let me find my pith helmet. :) Andrew dalke at dalkescientific.com From barry at python.org Sat Feb 23 00:45:23 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 22 Feb 2008 18:45:23 -0500 Subject: [Python-3000] Python 2.6 and 3.0 Message-ID: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi everyone, I've volunteered to be the release manager for Python 2.6 and 3.0. It's been several years since I've RM'd a Python release, and I'm happy to do it again (he says while the medication is still working :). I would like to get the next alpha releases of both versions out before Pycon, so I propose next Friday, February 29 for both. Guido reminded me that we released Python 1.6 and 2.0 together and it makes sense to both of us to do the same for Python 2.6 and 3.0. I don't think it will be that much more work (for me at least :) to release them in lockstep, so I think we should try it. I won't try to sync their pre-release version numbers except at the milestones (e.g. first beta, release candidates, final releases). I propose to change PEP 361 to outline the release schedule for both Python 2.6 and 3.0. I'm hoping we can work out a more definite schedule at Pycon, but for now I want to at least describe the lockstep release schedule and the Feb 29 date. I'd also like for us to consider doing regular monthly releases. Several other FLOSS projects I'm involved with are doing this to very good success. The nice thing is that everyone knows well in advance when the next release is going to happen, and so all developers and users know what to expect and what is needed from them. I'd like to propose that we do a joint release the last Friday of every month. For the alphas, it's basically what's in svn. This gives us some time to experiment with the process out and see if we like it enough to keep it going through the betas and final releases. Comments? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR79ek3EjvBPtnXfVAQJrmQP+KAzy0lSzake2BZsVxErD0kpFQJM+Iij0 qN86wjH0NoqARMfYKVA6eUzEY8vZPu7sJl+SjCOmhnE/KP+l/ArOdis5smiSKwQI klL4XKd/qdorTMqQ9mWgA0DeBb0Asvln9y64nxzNqgve+36q9j6ytPuRey1GjSI5 nVWoJDb3WsM= =4SRa -----END PGP SIGNATURE----- From brett at python.org Sat Feb 23 00:54:52 2008 From: brett at python.org (Brett Cannon) Date: Fri, 22 Feb 2008 15:54:52 -0800 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> Message-ID: On Fri, Feb 22, 2008 at 3:45 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi everyone, > > I've volunteered to be the release manager for Python 2.6 and 3.0. > It's been several years since I've RM'd a Python release, and I'm > happy to do it again (he says while the medication is still > working :). Can the PSF buy you more of the meds? =) > I would like to get the next alpha releases of both > versions out before Pycon, so I propose next Friday, February 29 for > both. > Since they are just alphas, sure. Not like I am going to make any earth-shattering changes that soon. > Guido reminded me that we released Python 1.6 and 2.0 together and it > makes sense to both of us to do the same for Python 2.6 and 3.0. I > don't think it will be that much more work (for me at least :) to > release them in lockstep, so I think we should try it. I won't try to > sync their pre-release version numbers except at the milestones (e.g. > first beta, release candidates, final releases). > > I propose to change PEP 361 to outline the release schedule for both > Python 2.6 and 3.0. I'm hoping we can work out a more definite > schedule at Pycon, but for now I want to at least describe the > lockstep release schedule and the Feb 29 date. > > I'd also like for us to consider doing regular monthly releases. > Several other FLOSS projects I'm involved with are doing this to very > good success. The nice thing is that everyone knows well in advance > when the next release is going to happen, and so all developers and > users know what to expect and what is needed from them. > > I'd like to propose that we do a joint release the last Friday of > every month. For the alphas, it's basically what's in svn. This > gives us some time to experiment with the process out and see if we > like it enough to keep it going through the betas and final releases. > > Comments? If you want to do monthly alphas, go for it! But if you are going to do that frequently is a source release going to make more sense than doing binary builds? -Brett From python at rcn.com Sat Feb 23 01:29:32 2008 From: python at rcn.com (Raymond Hettinger) Date: Fri, 22 Feb 2008 19:29:32 -0500 (EST) Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 Message-ID: <20080222192932.AHK93803@ms19.lnh.mail.rcn.net> [Barry] > I'd also like for us to consider doing regular monthly releases. +1 From brett at python.org Sat Feb 23 04:46:20 2008 From: brett at python.org (Brett Cannon) Date: Fri, 22 Feb 2008 19:46:20 -0800 Subject: [Python-3000] Please sign up for the PyCon sprint if you are attending! Message-ID: With the PyCon sprint approaching I would like all attendees to be signed up on the wiki page for the sprint: http://wiki.python.org/moin/PyCon2008/SprintSignups/Python . I am going to be using that list to send out an email about what is exactly expected of people in terms of setup ahead of time, etc. And even if you do know what you are doing, I still would appreciate people adding themselves so I can have a head count. That will let the sprint coordinators make sure we have a space big enough for us. -Brett From nnorwitz at gmail.com Sun Feb 24 00:26:04 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sat, 23 Feb 2008 15:26:04 -0800 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> Message-ID: +1 to everything -- n On Fri, Feb 22, 2008 at 3:45 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi everyone, > > I've volunteered to be the release manager for Python 2.6 and 3.0. > It's been several years since I've RM'd a Python release, and I'm > happy to do it again (he says while the medication is still > working :). I would like to get the next alpha releases of both > versions out before Pycon, so I propose next Friday, February 29 for > both. > > Guido reminded me that we released Python 1.6 and 2.0 together and it > makes sense to both of us to do the same for Python 2.6 and 3.0. I > don't think it will be that much more work (for me at least :) to > release them in lockstep, so I think we should try it. I won't try to > sync their pre-release version numbers except at the milestones (e.g. > first beta, release candidates, final releases). > > I propose to change PEP 361 to outline the release schedule for both > Python 2.6 and 3.0. I'm hoping we can work out a more definite > schedule at Pycon, but for now I want to at least describe the > lockstep release schedule and the Feb 29 date. > > I'd also like for us to consider doing regular monthly releases. > Several other FLOSS projects I'm involved with are doing this to very > good success. The nice thing is that everyone knows well in advance > when the next release is going to happen, and so all developers and > users know what to expect and what is needed from them. > > I'd like to propose that we do a joint release the last Friday of > every month. For the alphas, it's basically what's in svn. This > gives us some time to experiment with the process out and see if we > like it enough to keep it going through the betas and final releases. > > Comments? > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.8 (Darwin) > > iQCVAwUBR79ek3EjvBPtnXfVAQJrmQP+KAzy0lSzake2BZsVxErD0kpFQJM+Iij0 > qN86wjH0NoqARMfYKVA6eUzEY8vZPu7sJl+SjCOmhnE/KP+l/ArOdis5smiSKwQI > klL4XKd/qdorTMqQ9mWgA0DeBb0Asvln9y64nxzNqgve+36q9j6ytPuRey1GjSI5 > nVWoJDb3WsM= > =4SRa > -----END PGP SIGNATURE----- > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/nnorwitz%40gmail.com > From barry at python.org Sun Feb 24 13:32:02 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 24 Feb 2008 07:32:02 -0500 Subject: [Python-3000] Putting pdb.set_trace() in builtins? In-Reply-To: <20080221152348.6859.1072137842.divmod.quotient.11578@ohm> References: <20080221152348.6859.1072137842.divmod.quotient.11578@ohm> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 21, 2008, at 10:23 AM, Jean-Paul Calderone wrote: > Wouldn't a less invasive solution be to make pdb better? In > particular, > to make `breakpoint? actually reliably set a breakpoint? Maybe, although I like the directness and simplicity of setting a break point explicitly in the code (maybe inside a conditional, etc.). I just want to type less. :) Well, it was probably a dumb idea anyway born of minor frustration, so I'll drop it. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8FjwnEjvBPtnXfVAQIvqQQApHsTLcflVp4TkVWNhgdRYTcIxaWCpOjS 9XCz40WKgh9tg0m4iiZng+g7/h2rtlL4ythRGJfNFXr0xEGEt62appu87XHOt00g sOwLfglpRG/pA+RCnmrb/ezNCZMMJYjCfGbbN5huqb4EYUoAH2Pml5GjrG8bJN+G gozq5Pb1Jxk= =aELO -----END PGP SIGNATURE----- From lists at cheimes.de Sun Feb 24 15:19:02 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 24 Feb 2008 15:19:02 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> Message-ID: <47C17CD6.1040707@cheimes.de> Barry Warsaw wrote: > I'd also like for us to consider doing regular monthly releases. > Several other FLOSS projects I'm involved with are doing this to very > good success. The nice thing is that everyone knows well in advance > when the next release is going to happen, and so all developers and > users know what to expect and what is needed from them. > > I'd like to propose that we do a joint release the last Friday of > every month. For the alphas, it's basically what's in svn. This > gives us some time to experiment with the process out and see if we > like it enough to keep it going through the betas and final releases. Thanks for volunteering for the job, Barry! +1 for release early, release often but I want to point out a resource issue that may speak against a monthly release cycle. The Windows binaries still require a large amount of attention and human interaction. The last Windows binaries were build by me and I spent half the night ironing out issues and testing the installers. As far as I know the team only Martin and I have the infrastructure and tools to build the Windows binaries. We could cut down the time requirements by shipping only normal binaries instead of PGO (profile guided optimization) binaries. IMHO we could also skip the AMD64 builds until we have reached beta stage. But it's still going to cost either Martin or me the better half of a Friday night. I won't have time to assist with the Windows builds next Friday. I'm more than busy with personal life and job interviews. Hopefully I'm going to find a job that allows me to work on the Python core as much as during the past few months. That's for the Windows builds. Now I have a list of bugs and features I like to see fixed / applied before the next release: http://bugs.python.org/issue1692335 Fix exception pickling: Move initial args assignment to BaseException.__new__ http://bugs.python.org/issue1792 (trivial performance patch for marshal) http://bugs.python.org/issue1569 MS VCRT redist issue. IIRC Martin had a working solution for it in his sandbox. http://bugs.python.org/issue1657 my kqueue and epoll patch, just needs another review Christian From barry at python.org Sun Feb 24 18:51:24 2008 From: barry at python.org (Barry Warsaw) Date: Sun, 24 Feb 2008 12:51:24 -0500 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> Message-ID: <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 22, 2008, at 6:54 PM, Brett Cannon wrote: >> Hi everyone, >> >> I've volunteered to be the release manager for Python 2.6 and 3.0. >> It's been several years since I've RM'd a Python release, and I'm >> happy to do it again (he says while the medication is still >> working :). > > Can the PSF buy you more of the meds? =) Depends on the jurisdiction. :) >> I would like to get the next alpha releases of both >> versions out before Pycon, so I propose next Friday, February 29 for >> both. >> > > Since they are just alphas, sure. Not like I am going to make any > earth-shattering changes that soon. Cool. >> Guido reminded me that we released Python 1.6 and 2.0 together and it >> makes sense to both of us to do the same for Python 2.6 and 3.0. I >> don't think it will be that much more work (for me at least :) to >> release them in lockstep, so I think we should try it. I won't try >> to >> sync their pre-release version numbers except at the milestones (e.g. >> first beta, release candidates, final releases). >> >> I propose to change PEP 361 to outline the release schedule for both >> Python 2.6 and 3.0. I'm hoping we can work out a more definite >> schedule at Pycon, but for now I want to at least describe the >> lockstep release schedule and the Feb 29 date. >> >> I'd also like for us to consider doing regular monthly releases. >> Several other FLOSS projects I'm involved with are doing this to very >> good success. The nice thing is that everyone knows well in advance >> when the next release is going to happen, and so all developers and >> users know what to expect and what is needed from them. >> >> I'd like to propose that we do a joint release the last Friday of >> every month. For the alphas, it's basically what's in svn. This >> gives us some time to experiment with the process out and see if we >> like it enough to keep it going through the betas and final releases. >> >> Comments? > > If you want to do monthly alphas, go for it! But if you are going to > do that frequently is a source release going to make more sense than > doing binary builds? It very well might. See Christian Heimes's follow up re: Windows builds. OTOH, I'm okay if at least for the alphas, the binary builds lag behind the source releases, though I'd like to get the process as streamlined as possible. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8GunXEjvBPtnXfVAQIP0AQAo5F2tH1vXWbMAFGARZN576xopbQXSokX uVNXbeg5yjopCx38sHb5OCbublyIDOO8/2ubuuQ6uvAOJc3Br4BuMGHoC5ymQGqf 6pZYZLf4YUGLqFlYOB/huXpJPfH98RJJnK99zA8IQh4B7pN4xg14MF22gGij3Ybt z2hoy1EbYEk= =hW7b -----END PGP SIGNATURE----- From martin at v.loewis.de Sun Feb 24 19:57:41 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 24 Feb 2008 19:57:41 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> Message-ID: <47C1BE25.9050300@v.loewis.de> > It very well might. See Christian Heimes's follow up re: Windows > builds. OTOH, I'm okay if at least for the alphas, the binary builds > lag behind the source releases, though I'd like to get the process as > streamlined as possible. I can continue to provide Windows binaries if desired. Regards, Martin From brett at python.org Sun Feb 24 22:17:55 2008 From: brett at python.org (Brett Cannon) Date: Sun, 24 Feb 2008 13:17:55 -0800 Subject: [Python-3000] Buildbots are all red Message-ID: test_types is triggering an assertion error:: Objects/stringlib/string_format.h:497: failed assertion `PyString_Check(result)' A *really* quick session with gdb would seem to suggest that 'result' is actually a str, so I don't know what is going on. Don't have time to dig any deeper right now. -Brett From eric+python-dev at trueblade.com Sun Feb 24 22:19:20 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Sun, 24 Feb 2008 16:19:20 -0500 Subject: [Python-3000] Buildbots are all red In-Reply-To: References: Message-ID: <47C1DF58.6060903@trueblade.com> Brett Cannon wrote: > test_types is triggering an assertion error:: > > Objects/stringlib/string_format.h:497: failed assertion > `PyString_Check(result)' > > A *really* quick session with gdb would seem to suggest that 'result' > is actually a str, so I don't know what is going on. Don't have time > to dig any deeper right now. I'll check. Eric. From guido at python.org Sun Feb 24 22:26:00 2008 From: guido at python.org (Guido van Rossum) Date: Sun, 24 Feb 2008 13:26:00 -0800 Subject: [Python-3000] Buildbots are all red In-Reply-To: <47C1DF58.6060903@trueblade.com> References: <47C1DF58.6060903@trueblade.com> Message-ID: If this is Py3k, perhaps the result should be a PyUnicode instead? On Sun, Feb 24, 2008 at 1:19 PM, Eric Smith wrote: > Brett Cannon wrote: > > test_types is triggering an assertion error:: > > > > Objects/stringlib/string_format.h:497: failed assertion > > `PyString_Check(result)' > > > > A *really* quick session with gdb would seem to suggest that 'result' > > is actually a str, so I don't know what is going on. Don't have time > > to dig any deeper right now. > > I'll check. > > Eric. > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Sun Feb 24 22:31:13 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Sun, 24 Feb 2008 16:31:13 -0500 Subject: [Python-3000] Buildbots are all red In-Reply-To: References: <47C1DF58.6060903@trueblade.com> Message-ID: <47C1E221.4020108@trueblade.com> Guido van Rossum wrote: > If this is Py3k, perhaps the result should be a PyUnicode instead? Probably so, yes. But I can't reproduce this on a Fedora Core 6 box. I'm compiling under Windows now to see if I can reproduce it there. I'll change it to PyUnicode_Check shortly, even if I can't reproduce it. Eric. > > On Sun, Feb 24, 2008 at 1:19 PM, Eric Smith > wrote: >> Brett Cannon wrote: >> > test_types is triggering an assertion error:: >> > >> > Objects/stringlib/string_format.h:497: failed assertion >> > `PyString_Check(result)' >> > >> > A *really* quick session with gdb would seem to suggest that 'result' >> > is actually a str, so I don't know what is going on. Don't have time >> > to dig any deeper right now. >> >> I'll check. >> >> Eric. >> >> >> _______________________________________________ >> Python-3000 mailing list >> Python-3000 at python.org >> http://mail.python.org/mailman/listinfo/python-3000 >> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org >> > > > From eric+python-dev at trueblade.com Sun Feb 24 22:46:30 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Sun, 24 Feb 2008 16:46:30 -0500 Subject: [Python-3000] Buildbots are all red In-Reply-To: <47C1E221.4020108@trueblade.com> References: <47C1DF58.6060903@trueblade.com> <47C1E221.4020108@trueblade.com> Message-ID: <47C1E5B6.7070407@trueblade.com> Eric Smith wrote: > Guido van Rossum wrote: >> If this is Py3k, perhaps the result should be a PyUnicode instead? > > Probably so, yes. > > But I can't reproduce this on a Fedora Core 6 box. I'm compiling under > Windows now to see if I can reproduce it there. I'll change it to > PyUnicode_Check shortly, even if I can't reproduce it. I reproduced it on Windows, and checked in the fix, which Guido diagnosed correctly. I'm not sure what the issue with my FC6 box is, I'll investigate. >> On Sun, Feb 24, 2008 at 1:19 PM, Eric Smith >> wrote: >>> Brett Cannon wrote: >>> > test_types is triggering an assertion error:: >>> > >>> > Objects/stringlib/string_format.h:497: failed assertion >>> > `PyString_Check(result)' >>> > >>> > A *really* quick session with gdb would seem to suggest that 'result' >>> > is actually a str, so I don't know what is going on. Don't have time >>> > to dig any deeper right now. >>> >>> I'll check. >>> >>> Eric. >>> >>> >>> _______________________________________________ >>> Python-3000 mailing list >>> Python-3000 at python.org >>> http://mail.python.org/mailman/listinfo/python-3000 >>> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org >>> >> >> > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/eric%2Bpython-dev%40trueblade.com > From lists at cheimes.de Sun Feb 24 23:35:18 2008 From: lists at cheimes.de (Christian Heimes) Date: Sun, 24 Feb 2008 23:35:18 +0100 Subject: [Python-3000] Buildbots are all red In-Reply-To: <47C1E5B6.7070407@trueblade.com> References: <47C1DF58.6060903@trueblade.com> <47C1E221.4020108@trueblade.com> <47C1E5B6.7070407@trueblade.com> Message-ID: Eric Smith wrote: > Eric Smith wrote: >> Guido van Rossum wrote: >>> If this is Py3k, perhaps the result should be a PyUnicode instead? >> Probably so, yes. >> >> But I can't reproduce this on a Fedora Core 6 box. I'm compiling under >> Windows now to see if I can reproduce it there. I'll change it to >> PyUnicode_Check shortly, even if I can't reproduce it. > > I reproduced it on Windows, and checked in the fix, which Guido > diagnosed correctly. I'm not sure what the issue with my FC6 box is, > I'll investigate. I assume it's yet another problem with the makefile. Most likely the header files aren't listed correctly. I'm working on it. Christian From mhammond at skippinet.com.au Sun Feb 24 23:58:56 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Mon, 25 Feb 2008 09:58:56 +1100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <20080224201613.GL66273@nexus.in-nomine.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> Message-ID: <01a101c87738$da592b90$8f0b82b0$@com.au> Jeroen Ruigrok van der Werven: > -On [20080224 19:57], "Martin v. Lwis" (martin at v.loewis.de) wrote: > >I can continue to provide Windows binaries if desired. > > If need be, I can help testing the build infrastructure since I have > access > to various releases of Visual Studio as well. Me too - I still regularly build the svn version of Python, and although I don't regularly turn them into .MSI files, I'm confident I could learn how to do that ;) I understand that over time the binary process will need some tweaking, but in the general case, I expect that turning the crank for a test build needn't be that much or a burden. I expect the biggest problem is that I could no longer ignore certain extensions I don't care about, such as Tk. Cheers, Mark From nnorwitz at gmail.com Mon Feb 25 06:55:11 2008 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 24 Feb 2008 21:55:11 -0800 Subject: [Python-3000] removing destructuring del In-Reply-To: References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> Message-ID: On Fri, Feb 22, 2008 at 7:44 AM, Guido van Rossum wrote: [... del (tuple, of, variables), etc deleted] > > > I also think > > > > del a, > > > > should not be legal ("SyntaxError: trailing comma not allowed without > > surrounding parentheses"?), but that's getting into my own personal > > preferences. > > These all make sense to me, but at the same time it seems such a minor > issue that I'm not sure we should bother. It would probably end up > being more custom syntax -- right now it just uses exprlist in the > grammar and is limited to assignment targets by the compiler. What bothers me more is assert of a tuple. It does the wrong thing from what newbies expect. pychecker warns about this, but the version of pylint in use at Google doesn't warn about this. I wonder if we should add a warning? The code I recently saw was: assert (something_short, 'Some long assertion string that needs its own line') The original code that pylint warned about was: assert something_short, \ 'Some long assertion string that needs its own line' Index: Python/compile.c =================================================================== --- Python/compile.c (revision 61058) +++ Python/compile.c (working copy) @@ -2056,6 +2056,9 @@ if (assertion_error == NULL) return 0; } + if (s->v.Assert.test->kind == Tuple_kind) { + printf("got a tuple\n"); + } VISIT(c, expr, s->v.Assert.test); end = compiler_new_block(c); if (end == NULL) Seems to do the trick. Maybe this should be turned into a real patch with using PyErr_Warn()? n From lists at cheimes.de Mon Feb 25 16:01:21 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 25 Feb 2008 16:01:21 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <01a101c87738$da592b90$8f0b82b0$@com.au> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> Message-ID: <47C2D841.2030501@cheimes.de> Mark Hammond wrote: > Me too - I still regularly build the svn version of Python, and although I don't regularly turn them into .MSI files, I'm confident I could learn how to do that ;) I understand that over time the binary process will need some tweaking, but in the general case, I expect that turning the crank for a test build needn't be that much or a burden. I expect the biggest problem is that I could no longer ignore certain extensions I don't care about, such as Tk. It's not too hard. Tkinter, bsddb and friends are explained in PCbuild/README.txt. You should be able to compile them in less than half an hour. For the MSI installers you also need Python 2.5, your pywin32 package and some additional tools like the help compiler and cabarc.exe. Martin: Have you solved the problem with the VS CRT redist (http://bugs.python.org/issue1569)? Maybe Mark is able to assist you. Christian From asmodai at in-nomine.org Sun Feb 24 21:16:13 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sun, 24 Feb 2008 21:16:13 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C1BE25.9050300@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> Message-ID: <20080224201613.GL66273@nexus.in-nomine.org> -On [20080224 19:57], "Martin v. L?wis" (martin at v.loewis.de) wrote: >I can continue to provide Windows binaries if desired. If need be, I can help testing the build infrastructure since I have access to various releases of Visual Studio as well. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ What's in a name? That which we call a rose by any other name would smell as sweet... From asmodai at in-nomine.org Mon Feb 25 16:44:43 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 25 Feb 2008 16:44:43 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C2D841.2030501@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> Message-ID: <20080225154443.GP66273@nexus.in-nomine.org> -On [20080225 16:02], Christian Heimes (lists at cheimes.de) wrote: >For the MSI installers you also need Python 2.5, your pywin32 package >and some additional tools like the help compiler and cabarc.exe. Have you looked at http://wix.sourceforge.net/ ? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ What good will it be for a man if he gains the whole world, yet forfeits his soul? From lists at cheimes.de Mon Feb 25 17:06:12 2008 From: lists at cheimes.de (Christian Heimes) Date: Mon, 25 Feb 2008 17:06:12 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <20080225154443.GP66273@nexus.in-nomine.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <20080225154443.GP66273@nexus.in-nomine.org> Message-ID: <47C2E774.2040205@cheimes.de> Jeroen Ruigrok van der Werven wrote: > Have you looked at http://wix.sourceforge.net/ ? WiX looks interesting but I'm neither in the position to change the installer nor do I have a strong opinion. It's Martin's area of expertise. On the one hand a XML based MSI generator could be easier to maintain. On the other hand it would introduce another dependency to a 3rd party tool and things may get complicated if we leave the road. Automation tools tend to make common things really easy but special and uncommon stuff really hard. Christian From guido at python.org Mon Feb 25 18:07:23 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Feb 2008 09:07:23 -0800 Subject: [Python-3000] removing destructuring del In-Reply-To: References: <9CB907CE-4F42-4AE9-9D28-90A6DCA83EDB@dalkescientific.com> <741DC74F-8366-471F-B0C2-9B9153F4D83F@dalkescientific.com> Message-ID: On Sun, Feb 24, 2008 at 9:55 PM, Neal Norwitz wrote: > What bothers me more is assert of a tuple. It does the wrong thing > from what newbies expect. pychecker warns about this, but the version > of pylint in use at Google doesn't warn about this. I wonder if we > should add a warning? > > The code I recently saw was: > assert (something_short, > 'Some long assertion string that needs its own line') > > The original code that pylint warned about was: > assert something_short, \ > 'Some long assertion string that needs its own line' > > Index: Python/compile.c > =================================================================== > --- Python/compile.c (revision 61058) > +++ Python/compile.c (working copy) > @@ -2056,6 +2056,9 @@ > if (assertion_error == NULL) > return 0; > } > + if (s->v.Assert.test->kind == Tuple_kind) { > + printf("got a tuple\n"); > + } > VISIT(c, expr, s->v.Assert.test); > end = compiler_new_block(c); > if (end == NULL) > > Seems to do the trick. Maybe this should be turned into a real patch > with using PyErr_Warn()? +1 -- --Guido van Rossum (home page: http://www.python.org/~guido/) From hasan.diwan at gmail.com Mon Feb 25 18:14:59 2008 From: hasan.diwan at gmail.com (Hasan Diwan) Date: Mon, 25 Feb 2008 09:14:59 -0800 Subject: [Python-3000] Print Statement => Function Message-ID: <2cda2fc90802250914x20926827nbddd0beddbab9cd3@mail.gmail.com> I attended Guido's talk in which he said that the print statement was going to become a function, but gave no sense of direction. There is a print-like statement in the file statement called "write". So, perhaps print could be reworked as follows, for starters: def print(self, fileobj, stringobj, *kwarg): fileobj.write(stringobj) -- Cheers, Hasan Diwan From guido at python.org Mon Feb 25 18:32:36 2008 From: guido at python.org (Guido van Rossum) Date: Mon, 25 Feb 2008 09:32:36 -0800 Subject: [Python-3000] Print Statement => Function In-Reply-To: <2cda2fc90802250914x20926827nbddd0beddbab9cd3@mail.gmail.com> References: <2cda2fc90802250914x20926827nbddd0beddbab9cd3@mail.gmail.com> Message-ID: PEP 3105 On Mon, Feb 25, 2008 at 9:14 AM, Hasan Diwan wrote: > I attended Guido's talk in which he said that the print statement was > going to become a function, but gave no sense of direction. There is a > print-like statement in the file statement called "write". So, perhaps > print could be reworked as follows, for starters: > > def print(self, fileobj, stringobj, *kwarg): > fileobj.write(stringobj) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Mon Feb 25 22:07:11 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Feb 2008 16:07:11 -0500 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <47C17CD6.1040707@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <47C17CD6.1040707@cheimes.de> Message-ID: <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 24, 2008, at 9:19 AM, Christian Heimes wrote: > Barry Warsaw wrote: >> I'd also like for us to consider doing regular monthly releases. >> Several other FLOSS projects I'm involved with are doing this to very >> good success. The nice thing is that everyone knows well in advance >> when the next release is going to happen, and so all developers and >> users know what to expect and what is needed from them. >> >> I'd like to propose that we do a joint release the last Friday of >> every month. For the alphas, it's basically what's in svn. This >> gives us some time to experiment with the process out and see if we >> like it enough to keep it going through the betas and final releases. > > Thanks for volunteering for the job, Barry! > > +1 for release early, release often but I want to point out a resource > issue that may speak against a monthly release cycle. The Windows > binaries still require a large amount of attention and human > interaction. The last Windows binaries were build by me and I spent > half > the night ironing out issues and testing the installers. As far as I > know the team only Martin and I have the infrastructure and tools to > build the Windows binaries. From the follow ups, it sounds like others can pitch in here. A question though: is it reasonable to hold up the monthly release because a binary build we're going to make available can't be done at the same time? My preference (at least for the alphas) is "no". If we can make a source release, and if we can build a binary release from exactly the same revision, then we should go ahead and release. I'd rather get the alpha out there and in people's hands. We'll almost certainly be stricter for the candidates, finals, and maybe betas. > We could cut down the time requirements by shipping only normal > binaries > instead of PGO (profile guided optimization) binaries. IMHO we could > also skip the AMD64 builds until we have reached beta stage. But it's > still going to cost either Martin or me the better half of a Friday > night. Dang. I actually don't know how long it's going to take me to do the source release, but I hope it's no more than 3 or 4 hours. > I won't have time to assist with the Windows builds next Friday. I'm > more than busy with personal life and job interviews. Hopefully I'm > going to find a job that allows me to work on the Python core as > much as > during the past few months. When's the PSF gonna start hiring? :) > That's for the Windows builds. Now I have a list of bugs and > features I > like to see fixed / applied before the next release: > > http://bugs.python.org/issue1692335 Fix exception pickling: Move > initial > args assignment to BaseException.__new__ > > http://bugs.python.org/issue1792 (trivial performance patch for > marshal) > > http://bugs.python.org/issue1569 MS VCRT redist issue. IIRC Martin > had a > working solution for it in his sandbox. > > http://bugs.python.org/issue1657 my kqueue and epoll patch, just needs > another review So, as I mentioned in my last reply, I'm planning to only allow critical bugs (as described in roundup) hold up the release. Right now there are no critical bugs open. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8MuAHEjvBPtnXfVAQJ4JQP9F8AijArF8KhyxC7lp0ePDBGphgZjq7h8 9vZZ13oGOCtED/6M4bGDaZWrI1LEcj3iuf61Kdk6KwaeAi3dnGHkrP1XOTxZbLcz 8euKbC8JhBHan/A4SO4+xzxx4ZI9vCMRQqe+sLOQJsE9vH+4UMU1FDrhROxYwLbb aG0+fzGPdzA= =zpPQ -----END PGP SIGNATURE----- From barry at python.org Mon Feb 25 22:07:25 2008 From: barry at python.org (Barry Warsaw) Date: Mon, 25 Feb 2008 16:07:25 -0500 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <47C1BE25.9050300@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> Message-ID: <4CE13670-0F86-444E-9F3F-B94AEA9B8295@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 24, 2008, at 1:57 PM, Martin v. L?wis wrote: >> It very well might. See Christian Heimes's follow up re: Windows >> builds. OTOH, I'm okay if at least for the alphas, the binary >> builds lag behind the source releases, though I'd like to get the >> process as streamlined as possible. > > I can continue to provide Windows binaries if desired. Great, thanks! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8MuDXEjvBPtnXfVAQJeJwP/R8MMAhOsyRtpxIISEUoqMGDJksI5EtVM PcsxNO1p0MGcHfm5lNg0YNYwxsfc/0ghkRbsidegvcyN6BZWgHSaA7I0O1cBTG1x R4eNmLJBWCOcJNmTGgxCA7G8eEHTNTxneaQ0APO+yQbbHS/eyGGMcmFldNMkDqNO ycqikt0XiWI= =M3eC -----END PGP SIGNATURE----- From martin at v.loewis.de Mon Feb 25 23:02:18 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 25 Feb 2008 23:02:18 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <47C2D841.2030501@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> Message-ID: <47C33AEA.2010600@v.loewis.de> > Have you solved the problem with the VS CRT redist > (http://bugs.python.org/issue1569)? Maybe Mark is able to assist you. No, I still haven't found a solution. I do want to use the merge module; anything else probably isn't going to work. Regards, Martin From martin at v.loewis.de Mon Feb 25 23:02:36 2008 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 25 Feb 2008 23:02:36 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <20080225154443.GP66273@nexus.in-nomine.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <20080225154443.GP66273@nexus.in-nomine.org> Message-ID: <47C33AFC.8010307@v.loewis.de> >> For the MSI installers you also need Python 2.5, your pywin32 package >> and some additional tools like the help compiler and cabarc.exe. > > Have you looked at http://wix.sourceforge.net/ ? Yes. Martin From martin at v.loewis.de Mon Feb 25 23:06:11 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 25 Feb 2008 23:06:11 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C2E774.2040205@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <20080225154443.GP66273@nexus.in-nomine.org> <47C2E774.2040205@cheimes.de> Message-ID: <47C33BD3.5040003@v.loewis.de> > On the one hand a XML based MSI generator could be easier to maintain. I've looked at it, and I seriously doubt that. In WiX, you need to specify a fixed file list (perhaps with wildcards; I'm unsure). This will be tricky for Python, where the list of files to be installed changes all the time. You *need* to have a turing-complete packing language (such as Python). > On the other hand it would introduce another dependency to a 3rd party > tool and things may get complicated if we leave the road. Automation > tools tend to make common things really easy but special and uncommon > stuff really hard. Exactly so. When I started with MSI generation, and tried what comes with Visual Studio, and found it unusable - it did not support Itanium, and could not be changed to support it. Regards, Martin From db3l.net at gmail.com Mon Feb 25 23:50:36 2008 From: db3l.net at gmail.com (David Bolen) Date: Mon, 25 Feb 2008 17:50:36 -0500 Subject: [Python-3000] Python 2.6 and 3.0 References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <4CE13670-0F86-444E-9F3F-B94AEA9B8295@python.org> Message-ID: Barry Warsaw writes: > On Feb 24, 2008, at 1:57 PM, Martin v. L?wis wrote: > >>> It very well might. See Christian Heimes's follow up re: Windows >>> builds. OTOH, I'm okay if at least for the alphas, the binary >>> builds lag behind the source releases, though I'd like to get the >>> process as streamlined as possible. >> >> I can continue to provide Windows binaries if desired. > > Great, thanks! Note that my buildbot is still also building MSIs each night based on the svn head for 2.5, 2.6 and 3.0, and uploading them back to python.org (viewable at http://www.python.org/dev/daily-msi). So at least for an alpha based on the current SVN trunk, that might be an easy place to grab a binary snapshot from, unless I'm missing something. Conversely, the machine is there to make builds upon request, I presume, depending on the master configuration. I know Martin set the current scheme up. -- David From lists at cheimes.de Tue Feb 26 00:50:19 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 26 Feb 2008 00:50:19 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <47C33AEA.2010600@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de> Message-ID: <47C3543B.8060303@cheimes.de> Martin v. L?wis wrote: > No, I still haven't found a solution. I do want to use the merge > module; anything else probably isn't going to work. Da...ng Didn't you prepare a new MSI installer for 3.0a2 that includes the VS Redist MSM for X86? I vaguely remember that you've replaced my installer with a new one. The issue should be resolved before Python 2.6 and 3.0 are reaching beta stage. Maybe we can get some help from outside? Christian From lists at cheimes.de Tue Feb 26 00:58:32 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 26 Feb 2008 00:58:32 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C33BD3.5040003@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <20080225154443.GP66273@nexus.in-nomine.org> <47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de> Message-ID: <47C35628.6060200@cheimes.de> Martin v. L?wis wrote: > I've looked at it, and I seriously doubt that. In WiX, you need to > specify a fixed file list (perhaps with wildcards; I'm unsure). This > will be tricky for Python, where the list of files to be installed > changes all the time. > > You *need* to have a turing-complete packing language (such as Python). You are most likely right. A pure XML based solution ain't going to work for Python. But how about a mixed solution? XML templates -> Python fu -> WiX XML -> MSI We take some XML templates, modify them from Python and add the files. Finalliy we let WiX create the MSI installer from the resulting XML file. What do you think? > Exactly so. When I started with MSI generation, and tried what comes > with Visual Studio, and found it unusable - it did not support > Itanium, and could not be changed to support it. Yeah, been there, got frustrated, went back :/ Christian From lists at cheimes.de Tue Feb 26 01:11:26 2008 From: lists at cheimes.de (Christian Heimes) Date: Tue, 26 Feb 2008 01:11:26 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <47C17CD6.1040707@cheimes.de> <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org> Message-ID: <47C3592E.2080704@cheimes.de> Barry Warsaw wrote: > From the follow ups, it sounds like others can pitch in here. A > question though: is it reasonable to hold up the monthly release because > a binary build we're going to make available can't be done at the same > time? > > My preference (at least for the alphas) is "no". If we can make a > source release, and if we can build a binary release from exactly the > same revision, then we should go ahead and release. I'd rather get the > alpha out there and in people's hands. > > We'll almost certainly be stricter for the candidates, finals, and maybe > betas. I agree. It's not reasonable to hold of alphas because the Windows binaries may be released a few days later. Do we need official Windows binaries for alphas at all? Martin's buildbot creates nightly Windows builds. We could point users to the nightly builds and ask them to test the version. > Dang. I actually don't know how long it's going to take me to do the > source release, but I hope it's no more than 3 or 4 hours. I guess it's less than 2 hours. You can prepare most of the work like the announcements a couple of days earlier. I (perhaps naively) assume you have to smack the whip to get everything in place, do the svn cp to tag, svn export, tar cz, tar xzf && ./configure && make && make test dance and upload the tar.gz. Am I missing something important? :] > When's the PSF gonna start hiring? :) Dunno :) But I'm probably the last in a long line of Python core developers to get hired. Don't forget I'm still fresh and a junior core developer. *jk* > So, as I mentioned in my last reply, I'm planning to only allow critical > bugs (as described in roundup) hold up the release. Right now there are > no critical bugs open. #1569 is critical but not important enough to stop an alpha. As I said in the other mail it should be fixed for the first beta and must be fixed for the first rc. Christian From mhammond at skippinet.com.au Tue Feb 26 10:09:06 2008 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue, 26 Feb 2008 20:09:06 +1100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C35628.6060200@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <20080225154443.GP66273@nexus.in-nomine.org> <47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de> <47C35628.6060200@cheimes.de> Message-ID: <02cf01c87857$4d19bf40$e74d3dc0$@com.au> > Martin v. L?wis wrote: > > I've looked at it, and I seriously doubt that. In WiX, you need to > > specify a fixed file list (perhaps with wildcards; I'm unsure). This > > will be tricky for Python, where the list of files to be installed > > changes all the time. > > > > You *need* to have a turing-complete packing language (such as > Python). > > You are most likely right. A pure XML based solution ain't going to > work > for Python. But how about a mixed solution? > > XML templates -> Python fu -> WiX XML -> MSI > > We take some XML templates, modify them from Python and add the files. > Finalliy we let WiX create the MSI installer from the resulting XML > file. > > What do you think? I'm inclined to agree with Martin that WiX doesn't offer us much value (it offers value in many places though - just not for our requirements given Martin's msilib). I believe that once we know how to solve a particular problem, it would not be significantly easier to implement using WiX than it would using the current infrastructure. My problem is still getting my head around various MSI issues at any level (eg, bdist_msi needs some tweaking to allow for different releases of the same "package" to be recognized as such, but I'm not sure what MSI concept I'm dealing with yet...) WiX is an excellent inspiration though - if a WiX example can be found for something, it should be a significant help in implementing it via msilib. Cheers, Mark From barry at python.org Tue Feb 26 13:28:20 2008 From: barry at python.org (Barry Warsaw) Date: Tue, 26 Feb 2008 07:28:20 -0500 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <47C3592E.2080704@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <47C17CD6.1040707@cheimes.de> <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org> <47C3592E.2080704@cheimes.de> Message-ID: <7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 25, 2008, at 7:11 PM, Christian Heimes wrote: > Barry Warsaw wrote: >> From the follow ups, it sounds like others can pitch in here. A >> question though: is it reasonable to hold up the monthly release >> because >> a binary build we're going to make available can't be done at the >> same >> time? >> >> My preference (at least for the alphas) is "no". If we can make a >> source release, and if we can build a binary release from exactly the >> same revision, then we should go ahead and release. I'd rather get >> the >> alpha out there and in people's hands. >> >> We'll almost certainly be stricter for the candidates, finals, and >> maybe >> betas. > > I agree. It's not reasonable to hold of alphas because the Windows > binaries may be released a few days later. Do we need official Windows > binaries for alphas at all? Martin's buildbot creates nightly Windows > builds. We could point users to the nightly builds and ask them to > test > the version. That would be find with me. Where are those Windows binaries available for download from? >> Dang. I actually don't know how long it's going to take me to do the >> source release, but I hope it's no more than 3 or 4 hours. > > I guess it's less than 2 hours. You can prepare most of the work like > the announcements a couple of days earlier. I (perhaps naively) assume > you have to smack the whip to get everything in place, do the svn cp > to > tag, svn export, tar cz, tar xzf && ./configure && make && make test > dance and upload the tar.gz. Am I missing something important? :] Dunno yet! It's been years since I did a release and I really want to check out Anthony's welease tool this time. I may not have time before Friday to look at this though. >> When's the PSF gonna start hiring? :) > > Dunno :) But I'm probably the last in a long line of Python core > developers to get hired. Don't forget I'm still fresh and a junior > core > developer. *jk* :) >> So, as I mentioned in my last reply, I'm planning to only allow >> critical >> bugs (as described in roundup) hold up the release. Right now >> there are >> no critical bugs open. > > #1569 is critical but not important enough to stop an alpha. As I said > in the other mail it should be fixed for the first beta and must be > fixed for the first rc. It's not marked critical in roundup, and that's the only thing I'm going by! But it doesn't seem critical in the sense that it should hold up the alpha release, IMO. Cheers, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8QF5nEjvBPtnXfVAQJhrgP/Xz3IQrlCB9QPGsMGIL+xG3I5t+aThNg6 4n/bMjt4DRzTCRiNBjUllyCb5+VtPfTZu2wFVdi5I7NLMDG4WI4jfDGZlhvodbHW TPG/7bN/ykx9yE1hUPI5X+Kqrg0lG7Tbp9Zev5eHJCMwParSVu+hfWqD48+1bQqw JGfzz8AlqE0= =PQgE -----END PGP SIGNATURE----- From asmodai at in-nomine.org Tue Feb 26 07:31:04 2008 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Tue, 26 Feb 2008 07:31:04 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C33AEA.2010600@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de> Message-ID: <20080226063104.GU66273@nexus.in-nomine.org> -On [20080225 23:03], "Martin v. L?wis" (martin at v.loewis.de) wrote: >No, I still haven't found a solution. I do want to use the merge >module; anything else probably isn't going to work. I updated the ticket with some links to how to approach this issue. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ To conquer fear is the beginning of wisdom... From christian at cheimes.de Tue Feb 26 15:13:54 2008 From: christian at cheimes.de (Christian Heimes) Date: Tue, 26 Feb 2008 15:13:54 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <47C17CD6.1040707@cheimes.de> <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org> <47C3592E.2080704@cheimes.de> <7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org> Message-ID: <47C41EA2.4070508@cheimes.de> Barry Warsaw wrote: > That would be find with me. Where are those Windows binaries available > for download from? The Windows builds are hidden in the development section. It took me 10 minutes to find them because I was searching in the download section and for nightly builds. The *daily* builds are available at http://www.python.org/dev/daily-msi/ Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-3000/attachments/20080226/e9756b6b/attachment.pgp From martin at v.loewis.de Tue Feb 26 21:13:14 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 26 Feb 2008 21:13:14 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <02cf01c87857$4d19bf40$e74d3dc0$@com.au> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <20080225154443.GP66273@nexus.in-nomine.org> <47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de> <47C35628.6060200@cheimes.de> <02cf01c87857$4d19bf40$e74d3dc0$@com.au> Message-ID: <47C472DA.1070708@v.loewis.de> > My problem is still getting my head > around various MSI issues at any level (eg, bdist_msi needs some tweaking to > allow for different releases of the same "package" to be recognized as such, > but I'm not sure what MSI concept I'm dealing with yet...) Don't hesitate to ask here. Not sure what problem you are talking to specifically, but I guess you are asking for "upgrade codes"; there is also upgrade codes and package codes. Each package file should have its own unique *package? code, mere rebuilding should generate a new one. msilib does that correctly. A package code code can be installed at most once on a system. Related packages for the same software should share a *product* code. Rebuilding should not change the package code (but might in msilib; I'd have to check). Again, installer enforces that a specific package code can only be installed once on a system. Python assigns a separate product code for each bug fix release. Product codes are most useful for uninstallation, e.g. to uninstall Python 2.5.1, do msiexec /x {31800004-6386-4999-a519-518f2d78d8f0} Use separate product codes if you want to allow for simultaneous versions. Subsequent versions of the same product should share an *upgrade* code. MSI will check the Upgrade table, to see whether a package with the same upgrade code (but a different product code) is already installed, and if so, whether the version range matches. If the installed product is newer, it will refuse to install the older one. If the installed product is older, it will perform an "upgrade installation", which involves uninstalling the older version (possibly on a file-by-file basis), and possibly migrating the feature selections. Python uses a single upgrade code (until 2.5.2, which introduces a separate upgrade code for Win64). It then uses version ranges to make 2.5.2 an upgrade of 2.5.1 and 2.5.0, but not of 2.4.2 (say), essentially causing only one bug fix release per 2.5.x to be installed on the system, but allowing simultaneous installation of 2.5 and 2.4 (say). With 2.5.2, simultaneous installation of Win64 and Win32 releases on a single system becomes possible - which also requires to assign separate product codes to Win64, namely 2.5.2, Win32: 6b976adf-8ae8-434e-b282-a06c7f624d2f 2.5.2, Win64: 6b976adf-8ae8-434e-b282-a06c7f624d20 > WiX is an excellent inspiration though - if a WiX example can be found for > something, it should be a significant help in implementing it via msilib. The current challenge is merge modules: How can I merge the VC msm into the Python MSI (including support for SxS). Regards, Martin From martin at v.loewis.de Tue Feb 26 21:26:20 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 26 Feb 2008 21:26:20 +0100 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <47C3543B.8060303@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de> <47C3543B.8060303@cheimes.de> Message-ID: <47C475EC.1020004@v.loewis.de> >> No, I still haven't found a solution. I do want to use the merge >> module; anything else probably isn't going to work. > > Da...ng > Didn't you prepare a new MSI installer for 3.0a2 that includes the VS > Redist MSM for X86? I vaguely remember that you've replaced my installer > with a new one. Right. I produced a package that ships the CRT, but not by using the merge module. Instead, I arranged to include sufficient copies of the manifest file to make it work in the non-admin installation (and yes, you do need to install multiple copies of it - just see the 3.0a2 MSI file). > The issue should be resolved before Python 2.6 and 3.0 are reaching beta > stage. Maybe we can get some help from outside? Perhaps. I'm confident that I can find a solution when I get the time; I'm not so confident that I can find the time, though. Of course, I would prefer if the outside help would propose something better than "switch to this completely different technology; it may work" (unless a complete working solution is presented in that other technology, and as long as that other technology still creates MSI files with free-as-in-beer tools). In any case, contributions are welcome. Regards, Martin From martin at v.loewis.de Tue Feb 26 21:35:10 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 26 Feb 2008 21:35:10 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C35628.6060200@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <20080225154443.GP66273@nexus.in-nomine.org> <47C2E774.2040205@cheimes.de> <47C33BD3.5040003@v.loewis.de> <47C35628.6060200@cheimes.de> Message-ID: <47C477FE.3000301@v.loewis.de> > What do you think? Feel free to try it out. I'm skeptical that it will be a better overall solution than the current one - the main difference would be that, rather than me being the only one who can realistically change the packaging chain, it would be you who is the only one - which, in principle, would be fine with me. I believe you need deep inside knowledge of the MSI database format for WiX, just as you do for using the automation API. I think I could learn WiX fairly quickly after all these years of learning MSI in the first place. I think the WiX designers did right in tying WiX so close to the MSI data model, but it means that WiX makes package creation not simpler - merely more productive for the experienced user (who I hesitate to call WiXers :-) In any case, when you work with WiX, I'm sure you'll gain a lot of expert knowledge on Windows packaging. Depending on your job situation, that might pay some day :-) Regards, Martin From oliphant.travis at ieee.org Tue Feb 26 21:51:59 2008 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Tue, 26 Feb 2008 14:51:59 -0600 Subject: [Python-3000] Python 2.6 and 3.0 In-Reply-To: <47C475EC.1020004@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de> <47C3543B.8060303@cheimes.de> <47C475EC.1020004@v.loewis.de> Message-ID: Martin v. L?wis wrote: >>> No, I still haven't found a solution. I do want to use the merge >>> module; anything else probably isn't going to work. >> Da...ng >> Didn't you prepare a new MSI installer for 3.0a2 that includes the VS >> Redist MSM for X86? I vaguely remember that you've replaced my installer >> with a new one. > > Right. I produced a package that ships the CRT, but not by using the > merge module. Instead, I arranged to include sufficient copies of the > manifest file to make it work in the non-admin installation (and yes, > you do need to install multiple copies of it - just see the 3.0a2 MSI > file). > >> The issue should be resolved before Python 2.6 and 3.0 are reaching beta >> stage. Maybe we can get some help from outside? > > Perhaps. I'm confident that I can find a solution when I get the time; > I'm not so confident that I can find the time, though. Of course, I > would prefer if the outside help would propose something better than > "switch to this completely different technology; it may work" (unless > a complete working solution is presented in that other technology, and > as long as that other technology still creates MSI files with > free-as-in-beer tools). > > In any case, contributions are welcome. > I've been involved in packaging MSI's at Enthought for a few months now and I have been using Martin's msi Tool extensively. We explored several other options (including WiX) before settling back to Martin's tool. It is really a fantastic thing. I've changed a few things in msilib that I will offer back (as soon as I'm happy with the results) --- mainly in the area of File-Table naming strategies to avoid conflicts, error reporting, and adding custom actions. We also have a large file schema that we are using (because our MSI has over 40,000 files in it). We are able to package the entire Enthought Python Distribution (which includes python 2.5.2 plus about 70 other packages as eggs) into an MSI using a modified version of Martin's tool. If you want to check out the beta you can get the 174 MB download here (the official beta announcement will be tomorrow so you might want to wait a bit): http://code.enthought.com/downloads/epd/enthought_python-2.5.2001.msi My main purpose in posting to this thread is to offer kudos to Martin and advertise the utility of the msi-generation tools that he has provided. Thanks for your very useful tool Martin, -Travis Oliphant From martin at v.loewis.de Tue Feb 26 22:19:00 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 26 Feb 2008 22:19:00 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C41EA2.4070508@cheimes.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <47C17CD6.1040707@cheimes.de> <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org> <47C3592E.2080704@cheimes.de> <7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org> <47C41EA2.4070508@cheimes.de> Message-ID: <47C48244.4060404@v.loewis.de> > The Windows builds are hidden in the development section. It took me 10 > minutes to find them because I was searching in the download section and > for nightly builds. The *daily* builds are available at > http://www.python.org/dev/daily-msi/ The builds occur 11:00 UTC (2.5), 12:00 UTC (2.6) and 13:00 UTC (3.0). What that's at day or at night depends on where you live :-) Regards, Martin From christian at cheimes.de Tue Feb 26 23:42:42 2008 From: christian at cheimes.de (Christian Heimes) Date: Tue, 26 Feb 2008 23:42:42 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C48244.4060404@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <47C17CD6.1040707@cheimes.de> <8E45F999-6B4B-43A4-AD9A-54D2D1570732@python.org> <47C3592E.2080704@cheimes.de> <7DC2E52B-1F40-4DFB-9680-5410DD9E854E@python.org> <47C41EA2.4070508@cheimes.de> <47C48244.4060404@v.loewis.de> Message-ID: <47C495E2.2090705@cheimes.de> Martin v. L?wis wrote: >> The Windows builds are hidden in the development section. It took me 10 >> minutes to find them because I was searching in the download section and >> for nightly builds. The *daily* builds are available at >> http://www.python.org/dev/daily-msi/ > > The builds occur 11:00 UTC (2.5), 12:00 UTC (2.6) and 13:00 UTC (3.0). > What that's at day or at night depends on where you live :-) Trust me, the day and night cycle of our sun can be unrelated to the day and night cycle of my life. I define morning as the time span after my first coffee. I sometimes work until dawn so technically speaking 11:00 UTC could fit my definition of nightly build. :) Christian From martin at v.loewis.de Wed Feb 27 21:56:12 2008 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 27 Feb 2008 21:56:12 +0100 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDC4@EXMBX04.exchhosting.com> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de> <47C3543B.8060303@cheimes.de>, <47C475EC.1020004@v.loewis.de> <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDC4@EXMBX04.exchhosting.com> Message-ID: <47C5CE6C.8020602@v.loewis.de> >> (unless a complete working solution is presented in that other >> technology, and as long as that other technology still creates MSI >> files with free-as-in-beer tools). > > Just out of interest, what's the reason for enforcing that the > installer must be an MSI? Or, rather, if I were to present an > alternative .exe installer that ticks all of the above boxes, exceeds > the capabilities of the current installer and above all is easier to > extend and maintain -- would that be a non-starter because it's not > an MSI? Not necessarily - it is just very hard to provide the same features as MSI, but with a different tool. It seems that most tools have given up the battle against MSI, and now provide just another layer on top of MSI (just as my msilib does, or WiX). Regards, Martin From tnelson at onresolve.com Wed Feb 27 21:26:19 2008 From: tnelson at onresolve.com (Trent Nelson) Date: Wed, 27 Feb 2008 12:26:19 -0800 Subject: [Python-3000] [Python-Dev] Python 2.6 and 3.0 In-Reply-To: <47C475EC.1020004@v.loewis.de> References: <54334B67-3CD3-436D-BFEE-E358CF31A270@python.org> <6D7348BC-EF35-4639-BBB5-83C8C7410EC6@python.org> <47C1BE25.9050300@v.loewis.de> <20080224201613.GL66273@nexus.in-nomine.org> <01a101c87738$da592b90$8f0b82b0$@com.au> <47C2D841.2030501@cheimes.de> <47C33AEA.2010600@v.loewis.de> <47C3543B.8060303@cheimes.de>,<47C475EC.1020004@v.loewis.de> Message-ID: <87D3F9C72FBF214DB39FA4E3FE618CDC6E160CEDC4@EXMBX04.exchhosting.com> > (unless a complete working solution is presented in that other technology, > and as long as that other technology still creates MSI files with free-as-in-beer tools). Just out of interest, what's the reason for enforcing that the installer must be an MSI? Or, rather, if I were to present an alternative .exe installer that ticks all of the above boxes, exceeds the capabilities of the current installer and above all is easier to extend and maintain -- would that be a non-starter because it's not an MSI? Trent. From lists at cheimes.de Thu Feb 28 20:49:46 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 28 Feb 2008 20:49:46 +0100 Subject: [Python-3000] Code freeze? Message-ID: <47C7105A.20908@cheimes.de> Hey Barry! When are you planing to freeze the code of the trunk and branches/py3k for the upcoming alpha releases? I'll merge the last modifications from 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking good, except for the two profile tests on 3.0. I'm going to test Windows later. Christian From arkanes at gmail.com Thu Feb 28 21:03:51 2008 From: arkanes at gmail.com (Chris Mellon) Date: Thu, 28 Feb 2008 14:03:51 -0600 Subject: [Python-3000] Code freeze? In-Reply-To: <47C7105A.20908@cheimes.de> References: <47C7105A.20908@cheimes.de> Message-ID: <4866bea60802281203p29caa500q53de0ddb6fb92a82@mail.gmail.com> On Thu, Feb 28, 2008 at 1:49 PM, Christian Heimes wrote: > Hey Barry! > > When are you planing to freeze the code of the trunk and branches/py3k > for the upcoming alpha releases? I'll merge the last modifications from > 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking good, > except for the two profile tests on 3.0. I'm going to test Windows later. > Would a call for third-party module maintainer updates be justified? pysqlite in particular has advanced a couple versions since 2.5, and it'd be nice if the latest made it into 2.6. From barry at python.org Thu Feb 28 21:51:13 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Feb 2008 15:51:13 -0500 Subject: [Python-3000] Code freeze? In-Reply-To: <47C7105A.20908@cheimes.de> References: <47C7105A.20908@cheimes.de> Message-ID: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote: > Hey Barry! Hi Christian! > When are you planing to freeze the code of the trunk and branches/py3k > for the upcoming alpha releases? I'll merge the last modifications > from > 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking > good, > except for the two profile tests on 3.0. I'm going to test Windows > later. Okay, let's go ahead and make it official. I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to that: 2200 UTC Friday 29-Feb-2008. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8cewnEjvBPtnXfVAQJSpQP/aWjpFbTAETdHGnp0IOEagaXNojwJEBBl llFAE6FQI+WjPxNDAG6Y8T0y4kdiBVubMA7yfp+wXZdn+zpO/4D5OtBVeAoGVjLj Tg1Ws1Y2uEf7Ah4lRqLya1tfgO+rnKJ38vsCit58XACYVGKWDpD0mVu+An7+6Jmj rtlEjwGpvFQ= =jIiD -----END PGP SIGNATURE----- From barry at python.org Thu Feb 28 21:55:26 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Feb 2008 15:55:26 -0500 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <4866bea60802281203p29caa500q53de0ddb6fb92a82@mail.gmail.com> References: <47C7105A.20908@cheimes.de> <4866bea60802281203p29caa500q53de0ddb6fb92a82@mail.gmail.com> Message-ID: <8C97586F-C049-4574-817C-34EB4D4C0C7A@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 28, 2008, at 3:03 PM, Chris Mellon wrote: > On Thu, Feb 28, 2008 at 1:49 PM, Christian Heimes > wrote: >> Hey Barry! >> >> When are you planing to freeze the code of the trunk and branches/ >> py3k >> for the upcoming alpha releases? I'll merge the last modifications >> from >> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking >> good, >> except for the two profile tests on 3.0. I'm going to test Windows >> later. >> > > Would a call for third-party module maintainer updates be justified? > pysqlite in particular has advanced a couple versions since 2.5, and > it'd be nice if the latest made it into 2.6. There's still plenty of time for stuff to make it into 2.6 -- this is only an alpha release! I encourage people to get their changes in now if they want them reflected in the next alphas, but I intend to cut the release from whatever's in svn at 2300 UTC tomorrow. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8cfvnEjvBPtnXfVAQJKSwP+M5RzAFsQ3fNSwwYzRa+j3EaocwLgWJ+i QVLxALoH+ha6yWQDmdQJCvm4H+DPBSnz9o/ejuJ3Wuhf9TEIWBLjuwB1rM8qm7mB blaaNVV9eOXfORjlQ3PvwGblfwmGMGFUKfTuta8BZqkxtEzNKL5tDvgVdx98rbTV VEyfj7ap3BM= =dsql -----END PGP SIGNATURE----- From lists at cheimes.de Thu Feb 28 22:15:04 2008 From: lists at cheimes.de (Christian Heimes) Date: Thu, 28 Feb 2008 22:15:04 +0100 Subject: [Python-3000] Code freeze? In-Reply-To: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> Message-ID: <47C72458.8090306@cheimes.de> Barry Warsaw wrote: > Okay, let's go ahead and make it official. > > I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern > (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to > that: 2200 UTC Friday 29-Feb-2008. Linux is looking good. I've fixed some minor Windows issue in the last 30 minutes. I found one strange behavior. Some tests were failing because iter(fileobj) where fileobj is a tempfile._TemporaryFileWrapper failed. Apparently iter() doesn't use __getattr__ to acquire the __iter__ method. Is this behavior deliberately? Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/python-3000/attachments/20080228/948cd274/attachment.pgp From eric+python-dev at trueblade.com Thu Feb 28 22:03:28 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 28 Feb 2008 16:03:28 -0500 Subject: [Python-3000] Code freeze? In-Reply-To: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> Message-ID: <47C721A0.6060802@trueblade.com> Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote: > >> Hey Barry! > > Hi Christian! > >> When are you planing to freeze the code of the trunk and branches/py3k >> for the upcoming alpha releases? I'll merge the last modifications >> from >> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking >> good, >> except for the two profile tests on 3.0. I'm going to test Windows >> later. > > Okay, let's go ahead and make it official. > > I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern > (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to > that: 2200 UTC Friday 29-Feb-2008. Argh! I was going to check the last of the PEP 3127 changes in tonight, but I won't make it by 6 pm EST. I have them finished, but no tests written, so I'm not comfortable checking them in yet. I guess it's no big deal that they slip until the next alpha. Eric. From stephen at xemacs.org Thu Feb 28 23:07:04 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 29 Feb 2008 07:07:04 +0900 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> Message-ID: <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern > (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to > that: 2200 UTC Friday 29-Feb-2008. Is that enough time for the buildbots to do their thing and for you to look at the page? Alterntaively, I guess you could just suggest that people check the buildbot page for their platforms before downloading .... From barry at python.org Thu Feb 28 23:07:03 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Feb 2008 17:07:03 -0500 Subject: [Python-3000] Code freeze? In-Reply-To: <47C721A0.6060802@trueblade.com> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C721A0.6060802@trueblade.com> Message-ID: <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 28, 2008, at 4:03 PM, Eric Smith wrote: > Barry Warsaw wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote: >>> Hey Barry! >> Hi Christian! >>> When are you planing to freeze the code of the trunk and branches/ >>> py3k >>> for the upcoming alpha releases? I'll merge the last >>> modifications from >>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking >>> good, >>> except for the two profile tests on 3.0. I'm going to test >>> Windows later. >> Okay, let's go ahead and make it official. >> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern >> (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to >> that: 2200 UTC Friday 29-Feb-2008. > > Argh! I was going to check the last of the PEP 3127 changes in > tonight, but I won't make it by 6 pm EST. I have them finished, but > no tests written, so I'm not comfortable checking them in yet. I > guess it's no big deal that they slip until the next alpha. Sorry, I notice my message might not have been clear. As of this writing, you have 23 hours and 54 minute before code freeze :). Code freeze: 2200 UTC 29-Feb-2008 Alpha making: 2300 UTC 29-Feb-2008 - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8cwh3EjvBPtnXfVAQLRSAP/XalWCb1ArFUxoa0OQlA0T7Rw7Si3AFYu mXRTlU0kqptiYDiHYbPaQwzFwI3xgUBCG5qUnFFq3PCLrxmd6ilYcWAhErSGmxOS PAWfmrxZElAxqXHviOPQw5dlnoD8JhDudArgn74zRHHluazZYR4r+fCvN8fV0gKc zRDt8Owdwng= =uCn6 -----END PGP SIGNATURE----- From barry at python.org Thu Feb 28 23:09:45 2008 From: barry at python.org (Barry Warsaw) Date: Thu, 28 Feb 2008 17:09:45 -0500 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 28, 2008, at 5:07 PM, Stephen J. Turnbull wrote: > Barry Warsaw writes: > >> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern >> (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to >> that: 2200 UTC Friday 29-Feb-2008. > > Is that enough time for the buildbots to do their thing and for you to > look at the page? I don't know, but we'll find out! The first time (in years) will be a bit of a trial-and-error for me. I can always get dinner in the middle of it. :) > Alterntaively, I guess you could just suggest that people check the > buildbot page for their platforms before downloading .... Yes, good idea. I'm only going to cut source tarballs for the alphas. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8cxKXEjvBPtnXfVAQLgqwQAiiQUL1lhQM6EmYr8bvZNW2h97ziKGtpf mVTiNOCJZ1xZ6vKkcmR3Asd+Cwm7B2eT+sVLEpq+Z6tbki+7o0wXC4V6h/ar+zMz dQcR0QdwM218hioOiIVhtZXbWUTLGndWVviKBcoOdw6qwTbWFspCMV0+FEzdiNUN V0bDUjn4zkI= =uPib -----END PGP SIGNATURE----- From eric+python-dev at trueblade.com Fri Feb 29 00:31:45 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Thu, 28 Feb 2008 18:31:45 -0500 Subject: [Python-3000] Code freeze? In-Reply-To: <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C721A0.6060802@trueblade.com> <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org> Message-ID: <47C74461.20103@trueblade.com> Barry Warsaw wrote: >>> Okay, let's go ahead and make it official. >>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern >>> (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to >>> that: 2200 UTC Friday 29-Feb-2008. >> >> Argh! I was going to check the last of the PEP 3127 changes in >> tonight, but I won't make it by 6 pm EST. I have them finished, but >> no tests written, so I'm not comfortable checking them in yet. I >> guess it's no big deal that they slip until the next alpha. > > Sorry, I notice my message might not have been clear. As of this > writing, you have 23 hours and 54 minute before code freeze :). > > Code freeze: 2200 UTC 29-Feb-2008 > Alpha making: 2300 UTC 29-Feb-2008 Your message was clear, it's my reading comprehension that is low. Now you've removed my excuse for not getting this done. To the keyboard! Eric. From ncoghlan at gmail.com Fri Feb 29 12:39:22 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Feb 2008 21:39:22 +1000 Subject: [Python-3000] Code freeze? In-Reply-To: <47C72458.8090306@cheimes.de> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C72458.8090306@cheimes.de> Message-ID: <47C7EEEA.3080405@gmail.com> Christian Heimes wrote: > Barry Warsaw wrote: >> Okay, let's go ahead and make it official. >> >> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern >> (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to >> that: 2200 UTC Friday 29-Feb-2008. > > Linux is looking good. I've fixed some minor Windows issue in the last > 30 minutes. I found one strange behavior. Some tests were failing > because iter(fileobj) where fileobj is a tempfile._TemporaryFileWrapper > failed. Apparently iter() doesn't use __getattr__ to acquire the > __iter__ method. Is this behavior deliberately? The interpreter is actually permitted to bypass the instance when looking up magic methods. Whether it does or not is implementation dependent - even CPython varies its behaviour depending on the specific circumstance (e.g. the translation of with statements to bytecode ends up using normal GET_ATTR opcodes, so those will see __enter__ and __exit__ methods on instances, but most of the magic method lookups from C code bypass the instance and go straight to the relevant tp_* slot). I'd call this behaviour a bug in _TemporaryFileWrapper, since it's delegation trick in __getattr__ isn't guaranteed to work for the magic methods. I'm actually becoming less and less enamoured of that shortcut every day - given that we've already had to spell out the file method and property delegation for SpooledTemporaryFile in 2.6, I'm tempted to start spelling it out for _TemporaryFileWrapper as well. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Fri Feb 29 13:00:44 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 07:00:44 -0500 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> <47C7C90B.5040707@scottdial.com> Message-ID: <44FA87F7-324E-4D2A-9005-820195A4F423@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 29, 2008, at 5:21 AM, Georg Brandl wrote: > Scott Dial schrieb: >> Barry Warsaw wrote: >>>> Alterntaively, I guess you could just suggest that people check the >>>> buildbot page for their platforms before downloading .... >>> >>> Yes, good idea. I'm only going to cut source tarballs for the >>> alphas. >>> >> >> I apologize for having doubt in your plan, and I can certainly >> appreciate the work you will be doing as release manager. But.. >> >> I don't understand who these alpha releases are supposed to be for, >> and >> who they will serve. > > For one thing, releases generate "news", meaning that people will be > made > aware that things are moving, that Python is well underway to its next > major versions, and maybe will be more inclined to look at what's new, > or check out a release. I completely agree. There's another very important aspect of doing alpha releases, and that is to debug the /process/ of releasing. I need to see how far we've come in the years since I RM'd last. How smoothly we can coordinate all the various players? What does it take to update the website? Where are the glitches in the process that slows everything down or blocks certain steps? What is the state of the tools, and is there anything else we can automate? I also want to see if sticking to the monthly cycle can make us all work more productively as a community, so that we know when bugs need to be fixed in order to show up in the next release, and so on. Think of it this way: the alphas are for /us/ as much as for our users. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8fz7XEjvBPtnXfVAQI0zgQAnp8nva5qxIoN4ocr7uNXeNxJ+BdP2lMA 8GN1kv3R/+9ny6h/iakfyE0lt1hvHP+uchchd4zo1CWQ390h1W8eR0SpzbubRYFP Bvdnup7K1zSSGe0ILGoa6Zv8VCYI4vET0PsxM+bkhj8NblwQQu1xUcc6CnYBtapM c9TwZum/ODg= =A2cs -----END PGP SIGNATURE----- From ncoghlan at gmail.com Fri Feb 29 13:39:42 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 29 Feb 2008 22:39:42 +1000 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <44FA87F7-324E-4D2A-9005-820195A4F423@python.org> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> <47C7C90B.5040707@scottdial.com> <44FA87F7-324E-4D2A-9005-820195A4F423@python.org> Message-ID: <47C7FD0E.3090306@gmail.com> Barry Warsaw wrote: > Think of it this way: the alphas are for /us/ as much as for our users. In that vein, I think the monthly alphas may also help as a means for setting personal deadlines for things we're working on. "Implement for 2.6" is a bit of a wishy-washy deadline at some vague point in the future - it doesn't provide much impetus to sit down and spend an afternoon or evening getting it done. "Implement for next 2.6 alpha, which will be cut in 3 weeks, 2 days and 16 hours" is a real concrete target that can be aimed for (even if I'm the only person that knows I'm aiming for it for a given bug fix or feature). It should also motivate at least a monthly review of the buildbot status, and is already motivating a push towards making the buildbots a more reliable indicator of the health of the source tree. Sure, they may mostly be internal milestones (with third parties waiting for a feature-complete alpha or the first beta), but I think there is a decent chance the approach will turn out to be beneficial. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Fri Feb 29 13:57:13 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 07:57:13 -0500 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <47C7FD0E.3090306@gmail.com> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> <47C7C90B.5040707@scottdial.com> <44FA87F7-324E-4D2A-9005-820195A4F423@python.org> <47C7FD0E.3090306@gmail.com> Message-ID: <7FF2D914-8120-487E-A962-D306CA53F888@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 29, 2008, at 7:39 AM, Nick Coghlan wrote: > Barry Warsaw wrote: >> Think of it this way: the alphas are for /us/ as much as for our >> users. > > In that vein, I think the monthly alphas may also help as a means > for setting personal deadlines for things we're working on. > "Implement for 2.6" is a bit of a wishy-washy deadline at some vague > point in the future - it doesn't provide much impetus to sit down > and spend an afternoon or evening getting it done. "Implement for > next 2.6 alpha, which will be cut in 3 weeks, 2 days and 16 hours" > is a real concrete target that can be aimed for (even if I'm the > only person that knows I'm aiming for it for a given bug fix or > feature). > > It should also motivate at least a monthly review of the buildbot > status, and is already motivating a push towards making the > buildbots a more reliable indicator of the health of the source tree. > > Sure, they may mostly be internal milestones (with third parties > waiting for a feature-complete alpha or the first beta), but I think > there is a decent chance the approach will turn out to be beneficial. That's my hope too! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8gBKnEjvBPtnXfVAQK5egP+JgrZd68JtkyhLOGhcd0aBjhQEJM4e2FO HcRION8EeI4jhg5KQWOt/KmP6CBLblGq50RFI0afsLaJVapk4U6MFTB0LQus3LgX myd0zK5egu+sblFfKFfe5mMKn5T1Mx5HCLuOE4g3uwuFtdddsGMvYwCyqjm3smq1 T38bbujy234= =xSv1 -----END PGP SIGNATURE----- From lists at cheimes.de Fri Feb 29 14:09:51 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 29 Feb 2008 14:09:51 +0100 Subject: [Python-3000] Code freeze? In-Reply-To: <47C7EEEA.3080405@gmail.com> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C72458.8090306@cheimes.de> <47C7EEEA.3080405@gmail.com> Message-ID: Nick Coghlan wrote: > The interpreter is actually permitted to bypass the instance when > looking up magic methods. Whether it does or not is implementation > dependent - even CPython varies its behaviour depending on the specific > circumstance (e.g. the translation of with statements to bytecode ends > up using normal GET_ATTR opcodes, so those will see __enter__ and > __exit__ methods on instances, but most of the magic method lookups from > C code bypass the instance and go straight to the relevant tp_* slot). I suspected the different behavior is related to C type slots as well. tp_iter is part of the PyTypeObject struct and the __enter__ and __exit__ methods are ordinary methods. I just wasn't sure if the behavior is correct. I've never encountered a problem with the dark corners of the language so far. ;) I did some further testing. iter() doesn't even go through __getattribute__. It grabs the class and does a lookup on the class. >>> class Example(object): ... def __getattribute__(self, name): ... print name ... return object.__getattribute__(self, name) >>> e = Example() >>> iter(e) __class__ Traceback (most recent call last): File "", line 1, in TypeError: 'Example' object is not iterable That's either a bug or at least a big gotcha for developers of proxy objects. At least I expected that __getattribute__ is called for *every* attribute including magic methods. Is this documented somewhere? The docs say "See the :meth:`__getattribute__` method below for a way to actually get total control over attribute access.". Christian From ncoghlan at gmail.com Fri Feb 29 15:30:39 2008 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 01 Mar 2008 00:30:39 +1000 Subject: [Python-3000] Code freeze? In-Reply-To: References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C72458.8090306@cheimes.de> <47C7EEEA.3080405@gmail.com> Message-ID: <47C8170F.2050003@gmail.com> Christian Heimes wrote: > Is this documented somewhere? The docs say "See the > :meth:`__getattribute__` method below for a way to actually get total > control over attribute access.". I just checked, and the restriction is now documented in the development docs in the section on special methods at [1]. It was backported from the Py3k docs and isn't part of the docs for 2.5 or earlier versions. The gist is that special methods *must* be defined directly on a new-style class in order for the interpreter to reliably access them, but defining them on a new-style instance *may* still affect the interpreter's behaviour in some cases (but won't necessarily do so). In composing this response, I also realised why the new tempfile tests worked for me before I checked them in: I was testing it on the trunk, where _TemporaryFileWrapper is a classic class. Special method lookup on classic classes doesn't include the 'direct-to-type' optimisation, so those tests work with the tempfile module as written on 2.x. Unfortunately, proxies like _TemporaryFileWrapper that rely on __getattr__ to provide special methods simply won't work properly when implemented as new-style classes - for that, they need to spell out the overridden special methods the way SpooledTemporaryFile does. I think we may need a -3 warning that detects situations where "__*__" attributes are retrieved via a __getattr__ implementation on a classic class - classes being used that way have a very high chance of breaking when ported to 3.0 and I can't see any possible way for 2to3 to detect them. Cheers, Nick. [1] http://docs.python.org/dev/reference/datamodel.html#special-method-names -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From lists at cheimes.de Fri Feb 29 15:54:39 2008 From: lists at cheimes.de (Christian Heimes) Date: Fri, 29 Feb 2008 15:54:39 +0100 Subject: [Python-3000] Code freeze? In-Reply-To: <47C8170F.2050003@gmail.com> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C72458.8090306@cheimes.de> <47C7EEEA.3080405@gmail.com> <47C8170F.2050003@gmail.com> Message-ID: <47C81CAF.6070004@cheimes.de> Nick Coghlan wrote: > In composing this response, I also realised why the new tempfile tests > worked for me before I checked them in: I was testing it on the trunk, > where _TemporaryFileWrapper is a classic class. Special method lookup on > classic classes doesn't include the 'direct-to-type' optimisation, so > those tests work with the tempfile module as written on 2.x. tempfile is also using different implementation for TemporaryFile on Win32. On Windows it's an alias for NamedTemporaryFile while on Unix it's using a file descriptor of an unlinked file. Christian From guido at python.org Fri Feb 29 17:16:14 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Feb 2008 08:16:14 -0800 Subject: [Python-3000] test_profile failing Message-ID: Am I the only one for whom test_profile and test_cProfile are failing with a current Py3k head checkout? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri Feb 29 17:20:07 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 11:20:07 -0500 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 29, 2008, at 11:16 AM, Guido van Rossum wrote: > Am I the only one for whom test_profile and test_cProfile are failing > with a current Py3k head checkout? No, they're failing across the board afaict. I won't have time to look at them before tonight, but it would be great if we can fix them before I cut the alphas. I think they're the only failures in the 3.0 test suite. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8gwuHEjvBPtnXfVAQJclQP/Sz90X6qj0BWxaadmzO4Kr+bqJL2yq4gK vscxkuGxt6leh/ojMgn8esSoGGZsy0GhCQgfagP93txc7MI36ZiYheZoyRFUwUCG 3Y/W+0E0yArShXhgQ/V8yfIN1tTq7iQxzIiJvHrZ9g2xOMh+j7leSxRk8j+vW7TL +ye8u8r7dvQ= =C0ir -----END PGP SIGNATURE----- From facundobatista at gmail.com Fri Feb 29 17:25:20 2008 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 29 Feb 2008 14:25:20 -0200 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: 2008/2/29, Guido van Rossum : > Am I the only one for whom test_profile and test_cProfile are failing > with a current Py3k head checkout? Both fail for me too. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From fdrake at acm.org Fri Feb 29 17:55:46 2008 From: fdrake at acm.org (Fred Drake) Date: Fri, 29 Feb 2008 11:55:46 -0500 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Feb 29, 2008, at 11:16 AM, Guido van Rossum wrote: > Am I the only one for whom test_profile and test_cProfile are failing > with a current Py3k head checkout? They were failing for me a couple of days ago; checking now, they still are, and test_itertools has been added to the list of failing tests: test test_itertools failed -- Traceback (most recent call last): File "/Users/fdrake/projects/python/py3k/Lib/test/ test_itertools.py", line 698, in test_chain self.assertRaises(TypeError, chain, N(s)) AssertionError: TypeError not raised by chain This is on Mac OS X 10.5.2 with no local changes. I did a "make clean; make; make test" and still get the error. I've no time to look into it right now. -Fred -- Fred Drake From guido at python.org Fri Feb 29 18:41:13 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Feb 2008 09:41:13 -0800 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Fri, Feb 29, 2008 at 8:55 AM, Fred Drake wrote: > On Feb 29, 2008, at 11:16 AM, Guido van Rossum wrote: > > > Am I the only one for whom test_profile and test_cProfile are failing > > with a current Py3k head checkout? > > > They were failing for me a couple of days ago; checking now, they > still are, and test_itertools has been added to the list of failing > tests: > > test test_itertools failed -- Traceback (most recent call last): > File "/Users/fdrake/projects/python/py3k/Lib/test/ > test_itertools.py", line 698, in test_chain > self.assertRaises(TypeError, chain, N(s)) > AssertionError: TypeError not raised by chain > > This is on Mac OS X 10.5.2 with no local changes. I did a "make > clean; make; make test" and still get the error. I've no time to look > into it right now. We should not release with failing tests. We've got to keep *some* standard of quality. It looks like the failures are shallow -- there's a "golden" expected output which includes some silliness like the exact number of CPU seconds used. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Feb 29 18:49:03 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Feb 2008 09:49:03 -0800 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Fri, Feb 29, 2008 at 9:41 AM, Guido van Rossum wrote: > It looks like the failures are shallow -- there's a "golden" expected > output which includes some silliness like the exact number of CPU > seconds used. Actually I take it back. The same approach works fine in the trunk, so I suspect it's a merge problem. It may be quite subtle, alas. :-( Still, we shouldn't release with this failing. It would reduce confidence in the releases and generate tons of spurious customer support calls ^W^W^W bug reports. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric+python-dev at trueblade.com Fri Feb 29 19:08:24 2008 From: eric+python-dev at trueblade.com (Eric Smith) Date: Fri, 29 Feb 2008 13:08:24 -0500 Subject: [Python-3000] Code freeze? In-Reply-To: <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C721A0.6060802@trueblade.com> <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org> Message-ID: <47C84A18.5030602@trueblade.com> Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 28, 2008, at 4:03 PM, Eric Smith wrote: > >> Barry Warsaw wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote: >>>> Hey Barry! >>> Hi Christian! >>>> When are you planing to freeze the code of the trunk and branches/py3k >>>> for the upcoming alpha releases? I'll merge the last modifications >>>> from >>>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are looking >>>> good, >>>> except for the two profile tests on 3.0. I'm going to test Windows >>>> later. >>> Okay, let's go ahead and make it official. >>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm Eastern >>> (UTC-5) time or 2300 UTC. Let's freeze the tree one hour prior to >>> that: 2200 UTC Friday 29-Feb-2008. >> >> Argh! I was going to check the last of the PEP 3127 changes in >> tonight, but I won't make it by 6 pm EST. I have them finished, but >> no tests written, so I'm not comfortable checking them in yet. I >> guess it's no big deal that they slip until the next alpha. > > Sorry, I notice my message might not have been clear. As of this > writing, you have 23 hours and 54 minute before code freeze :). > > Code freeze: 2200 UTC 29-Feb-2008 > Alpha making: 2300 UTC 29-Feb-2008 It turns out I'm not going to make this first alpha with the rest of the PEP 3127 changes. The code doesn't handle all combinations of (int, long) and (str, unicode). I'll get it finished before the next alpha. Eric. From stephen at xemacs.org Fri Feb 29 19:59:03 2008 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 01 Mar 2008 03:59:03 +0900 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <47C7C90B.5040707@scottdial.com> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> <47C7C90B.5040707@scottdial.com> Message-ID: <873arblhc8.fsf@uwakimon.sk.tsukuba.ac.jp> Not speaking for Barry or anyone else but myself. This is an explanation of how I understand the process and why I welcome it. Scott Dial writes: > I don't understand who these alpha releases are supposed to be for, and > who they will serve. An alpha test is internal. It is comprised of those tests the developers working on the product (and/or an affiliated QA department) can imagine. Similarly, an alpha release is of, by, and for the developers. > I'm not sure what developer outside of the core community would > want to work with something missing key features and released > fairly arbitrarily. Developers outside of the core community are not targeted. That said, how do you know that key features are missing? Part of the point of a release is that it says "*this stuff* is working". I want *my* key feature to be part of "this stuff", not *your* key feature. Given "my feature," I download to give the team feedback as early as possible. Lacking yours, you don't. Next alpha, presumably our roles will be reversed. > More to the point, I don't know why a developer wouldn't just checkout > from SVN in any case. Certainly if they are going to help root out bugs, > then we would like them to be using the trunk if possible. I fear that > once an alpha is 2 weeks old, we will start saying "please check if its > still a problem on the trunk." Software development in practice is a matter of "take two steps back, then three steps forward." The point of an alpha release is to checkpoint what is a regression, and what is a temporary "feature" introduced by new development. The latter is admissible on the trunk, but the goal should be none from one alpha to the next. In other words, a bug introduced by new development should be fixed by the next alpha. If it can't be, the developer misjudged the timing of the merge to mainline; it wasn't ready yet. > A mild concern is how this effects checkins with individuals either > trying to meet a deadline or even wait until after an alpha release to > checkin a large change. I'm not sure this will happen, but having > releases scheduled without feature milestones attached certainly changes > the environment for work to be done in. Scheduling feature milestones assumes that people put priority on those features at a given time. Barry can't assume that yet. Once the rhythm is established, Barry can start asking for, and some people will start volunteering, milestone commitments for given releases. I think that it probably is desirable to to put that deadline pressure on. Individuals who rush to get their work in, and cause alpha-to- alpha regressions, can be advised to wait in the future in similar circumstances. Once the rhythm is established, people can expect that alphas will be consistently increasing in features and consistently decreasing in defects. If that's not true, something's wrong with the process, and the team needs to step back and do something about it. But in the nature of software development, *monotone improvement is not something you want to, or even can, impose on the trunk at all points in time*. From barry at python.org Fri Feb 29 20:22:46 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 14:22:46 -0500 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 29, 2008, at 12:49 PM, Guido van Rossum wrote: > On Fri, Feb 29, 2008 at 9:41 AM, Guido van Rossum > wrote: >> It looks like the failures are shallow -- there's a "golden" expected >> output which includes some silliness like the exact number of CPU >> seconds used. > > Actually I take it back. The same approach works fine in the trunk, so > I suspect it's a merge problem. It may be quite subtle, alas. :-( > > Still, we shouldn't release with this failing. It would reduce > confidence in the releases and generate tons of spurious customer > support calls ^W^W^W bug reports. Of course, the /real/ bug is that a commit was allowed that broke the test suite. This failure goes back to the first rev on the buildbot page 61103 so it'll take a little more investigation to see what broke it. I'll see what can be done to fix this particular problem, and try to be more attentive to buildbot failure notices when they happen so we can nip it closer to the bud. Ultimately, it would be nice if we had a process that prevented any breaking patch from landing, but we're not there yet. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8hbhnEjvBPtnXfVAQIUxQQAhPzrDAdBxbIRqAHSK2ipj3eD27Hvq6FR vZFDwL7rV/tobvnacPAiZCoTXKg39PXoPZQHhPbheB7mxqMcHRLEb9Kf0GD0qhBL GKu6p+wgJ7PUOciVlXdykE/E+3+rT0v0MXU3RcvZW7zVRidsLZmQP26OjWgFm0RW S6VuFw7E2e0= =kOv3 -----END PGP SIGNATURE----- From barry at python.org Fri Feb 29 20:23:26 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 14:23:26 -0500 Subject: [Python-3000] Code freeze? In-Reply-To: <47C84A18.5030602@trueblade.com> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <47C721A0.6060802@trueblade.com> <59F9E6AD-3FED-498B-960A-5CD0426FB409@python.org> <47C84A18.5030602@trueblade.com> Message-ID: <5B7CA66C-E966-4308-9FD6-4AD7A47D0D19@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 29, 2008, at 1:08 PM, Eric Smith wrote: > Barry Warsaw wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> On Feb 28, 2008, at 4:03 PM, Eric Smith wrote: >>> Barry Warsaw wrote: >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> On Feb 28, 2008, at 2:49 PM, Christian Heimes wrote: >>>>> Hey Barry! >>>> Hi Christian! >>>>> When are you planing to freeze the code of the trunk and >>>>> branches/py3k >>>>> for the upcoming alpha releases? I'll merge the last >>>>> modifications from >>>>> 2.6 to 3.0 in a couple of minutes. All tests on Linux are >>>>> looking good, >>>>> except for the two profile tests on 3.0. I'm going to test >>>>> Windows later. >>>> Okay, let's go ahead and make it official. >>>> I plan on cutting the alphas for 2.6 and 3.0 at about 6pm >>>> Eastern (UTC-5) time or 2300 UTC. Let's freeze the tree one >>>> hour prior to that: 2200 UTC Friday 29-Feb-2008. >>> >>> Argh! I was going to check the last of the PEP 3127 changes in >>> tonight, but I won't make it by 6 pm EST. I have them finished, >>> but no tests written, so I'm not comfortable checking them in >>> yet. I guess it's no big deal that they slip until the next alpha. >> Sorry, I notice my message might not have been clear. As of this >> writing, you have 23 hours and 54 minute before code freeze :). >> Code freeze: 2200 UTC 29-Feb-2008 >> Alpha making: 2300 UTC 29-Feb-2008 > > It turns out I'm not going to make this first alpha with the rest of > the PEP 3127 changes. The code doesn't handle all combinations of > (int, long) and (str, unicode). I'll get it finished before the > next alpha. No problem Eric. You know when the next alpha is coming out anyway. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8hbrnEjvBPtnXfVAQKRvgP/Qe5/vCtHPX5LKklLTtcD4pnR/AtRs/V2 /VNeqkvXnr4Ooe8LwNKFMo4Yy0pdvjEQbLR/FGKojG2JIJ0kXyE2ObQXFsKCKAxV TCr8ZPeh5wqGLYuJAXtTz/UJzf5CzkgbKY9p32u5yy5qXiP7rgGpi8JvSZJKN3cP bApx/pGyNdU= =GaSZ -----END PGP SIGNATURE----- From barry at python.org Fri Feb 29 20:33:34 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 14:33:34 -0500 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <873arblhc8.fsf@uwakimon.sk.tsukuba.ac.jp> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> <47C7C90B.5040707@scottdial.com> <873arblhc8.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <079EC957-A3D3-48F9-8A77-A8BEB402E749@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 29, 2008, at 1:59 PM, Stephen J. Turnbull wrote: > I think that it probably is desirable to to put that deadline pressure > on. Individuals who rush to get their work in, and cause alpha-to- > alpha regressions, can be advised to wait in the future in similar > circumstances. Once the rhythm is established, people can expect that > alphas will be consistently increasing in features and consistently > decreasing in defects. If that's not true, something's wrong with the > process, and the team needs to step back and do something about it. I agree, and what I already think is broken about our process is that changes can land that break the buildbots. Ideally, this should never happen, but our build/test environment has two flaws. First, it's retroactive not proactive. Second, the tests themselves are not always stable. Given the wide range of platforms and the volunteer nature of the buildbot farm, I don't think there's much right now that we can do about the second point. We can do things to mitigate it though, such as take a majority-success approach, or give more weight to more stable platforms and buildbots. The second one is tougher because more work on the process is necessary, and it would change our workflow for committing changes to the tree. Even with its faults, I'm a big fan of PQM though that's not something I think we're ready for. The bigger question though is whether we as a development community would change the way we work so that nothing lands if it doesn't pass all the tests. We'd be trading some inconvenience (and administrative headaches) for better overall quality and always-releasable guarantees. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8heD3EjvBPtnXfVAQIj/gP/dyf1oavy7y4gTrKHi+j/m+0y4DJstIDf Fh2MhqExWtCX6V8M3mOn46uRwStwtz9+TUEznNEC8xsxq734GtVyi9Vw6cLpmZQ6 uQp+IBT+nkqNz3sDd8N/ewAGPBO5Ml2m+yn+rfi2XaT5Vfi5akSR/aDwJKGC71fL 8IaWHo5XKEM= =JQMb -----END PGP SIGNATURE----- From brett at python.org Fri Feb 29 20:44:05 2008 From: brett at python.org (Brett Cannon) Date: Fri, 29 Feb 2008 11:44:05 -0800 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On Feb 29, 2008, at 12:49 PM, Guido van Rossum wrote: > > > On Fri, Feb 29, 2008 at 9:41 AM, Guido van Rossum > > wrote: > >> It looks like the failures are shallow -- there's a "golden" expected > >> output which includes some silliness like the exact number of CPU > >> seconds used. > > > > Actually I take it back. The same approach works fine in the trunk, so > > I suspect it's a merge problem. It may be quite subtle, alas. :-( > > > > Still, we shouldn't release with this failing. It would reduce > > confidence in the releases and generate tons of spurious customer > > support calls ^W^W^W bug reports. > > Of course, the /real/ bug is that a commit was allowed that broke the > test suite. This failure goes back to the first rev on the buildbot > page 61103 so it'll take a little more investigation to see what broke > it. > > I'll see what can be done to fix this particular problem, and try to > be more attentive to buildbot failure notices when they happen so we > can nip it closer to the bud. Ultimately, it would be nice if we had > a process that prevented any breaking patch from landing, but we're > not there yet. A quick tweak to test_profile to print out what it got for output shows that some changes to io.py led to a different tracing. -Brett From barry at python.org Fri Feb 29 20:57:50 2008 From: barry at python.org (Barry Warsaw) Date: Fri, 29 Feb 2008 14:57:50 -0500 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote: > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw > wrote: >> >> I'll see what can be done to fix this particular problem, and try to >> be more attentive to buildbot failure notices when they happen so we >> can nip it closer to the bud. Ultimately, it would be nice if we had >> a process that prevented any breaking patch from landing, but we're >> not there yet. > > A quick tweak to test_profile to print out what it got for output > shows that some changes to io.py led to a different tracing. Thanks Brett. Do we just need to regen the expected output then? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iQCVAwUBR8hjvnEjvBPtnXfVAQJNPgP/WSq1x89laWuptso3MNYdtmwnY7L9pZbN 0IoFlqSbPQ/iB42lbE6apAdBNwUjPBAX+IpwFMKEtbDEQzBK5GZ/smksOxyw152E yMwJaLCnsAf/SS57cxxRv2s6bfB7XF/A1XeodtqlV3O+5seV/zHvX+5IycQ0HJ0R Ap4uOwC8+II= =XpFM -----END PGP SIGNATURE----- From brett at python.org Fri Feb 29 21:01:26 2008 From: brett at python.org (Brett Cannon) Date: Fri, 29 Feb 2008 12:01:26 -0800 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote: > > > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw > > wrote: > >> > > >> I'll see what can be done to fix this particular problem, and try to > >> be more attentive to buildbot failure notices when they happen so we > >> can nip it closer to the bud. Ultimately, it would be nice if we had > >> a process that prevented any breaking patch from landing, but we're > >> not there yet. > > > > A quick tweak to test_profile to print out what it got for output > > shows that some changes to io.py led to a different tracing. > > Thanks Brett. Do we just need to regen the expected output then? Nope, it's still failing. I think there might be a slight issue with how it is generating. I am going to work on this for another half hour before I need to take off for an appointment. I will send another update before I take off. -Brett From brett at python.org Fri Feb 29 21:39:38 2008 From: brett at python.org (Brett Cannon) Date: Fri, 29 Feb 2008 12:39:38 -0800 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote: > > > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw > > wrote: > >> > > >> I'll see what can be done to fix this particular problem, and try to > >> be more attentive to buildbot failure notices when they happen so we > >> can nip it closer to the bud. Ultimately, it would be nice if we had > >> a process that prevented any breaking patch from landing, but we're > >> not there yet. > > > > A quick tweak to test_profile to print out what it got for output > > shows that some changes to io.py led to a different tracing. > > Thanks Brett. Do we just need to regen the expected output then? Ah, the joys of expected output. =) So here is the problem, and it is probably going to mean test_profile needs to be rewritten... When you run ``./python Lib/test/test_profile.py -r`` it runs the profile and rewrites the file with the golden output. That's great and all, but there was a slight difference with that golden output and what was being generated by ``./python Lib/test/test_profile.py``. You know what that difference was? An extra write call in io.py. Turns out that when the golden output is generated that all streams have their buffers empty. But when unittest prints its whole "__test__ .. ok" output it has stuff in a buffer since the "ok" has not been written yet (in other words sys.stdout has not been fully flushed yet while the test is running). If you change the print() call in test_profile.regenerate_expected_output() to ``print('Regenerating %s ... ' % filename, end='')`` then sys.stdout gets a flush() call during the profiling for the golden output and everyone is happy. But guess what happens when you run ``./python Lib/test/regrtest.py test_profile``? That's right, there is no extra flush call! regrtest.py outputs a single line, runs the test, and then prints another line when it knows the result of the test.. Actually, there is one less flush call in the regrtest.py case than in the generated stuff, so I suspect regrtest or someone is doing an explicit flush or something. In other words, having the golden output have any I/O in it is making it impossible to test. What would probably be best is to just strip out any and all lines that do not directly involve profilee.py and only compare those lines. I have no clue if a simple 'contains' check will suffice because I don't know if the stat numbers are consistent enough. Anyway, I am out of time. Hopefully someone can finish up this work some time today. -Brett From guido at python.org Fri Feb 29 22:15:32 2008 From: guido at python.org (Guido van Rossum) Date: Fri, 29 Feb 2008 13:15:32 -0800 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Fri, Feb 29, 2008 at 12:39 PM, Brett Cannon wrote: > On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > > > On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote: > > > > > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw > > > wrote: > > >> > > > > >> I'll see what can be done to fix this particular problem, and try to > > >> be more attentive to buildbot failure notices when they happen so we > > >> can nip it closer to the bud. Ultimately, it would be nice if we had > > >> a process that prevented any breaking patch from landing, but we're > > >> not there yet. > > > > > > A quick tweak to test_profile to print out what it got for output > > > shows that some changes to io.py led to a different tracing. > > > > Thanks Brett. Do we just need to regen the expected output then? > > Ah, the joys of expected output. =) > > So here is the problem, and it is probably going to mean test_profile > needs to be rewritten... > > When you run ``./python Lib/test/test_profile.py -r`` it runs the > profile and rewrites the file with the golden output. That's great and > all, but there was a slight difference with that golden output and > what was being generated by ``./python Lib/test/test_profile.py``. You > know what that difference was? An extra write call in io.py. > > Turns out that when the golden output is generated that all streams > have their buffers empty. But when unittest prints its whole "__test__ > .. ok" output it has stuff in a buffer since the "ok" has not been > written yet (in other words sys.stdout has not been fully flushed yet > while the test is running). If you change the print() call in > test_profile.regenerate_expected_output() to ``print('Regenerating %s > ... ' % filename, end='')`` then sys.stdout gets a flush() call during > the profiling for the golden output and everyone is happy. > > But guess what happens when you run ``./python Lib/test/regrtest.py > test_profile``? That's right, there is no extra flush call! > regrtest.py outputs a single line, runs the test, and then prints > another line when it knows the result of the test.. Actually, there is > one less flush call in the regrtest.py case than in the generated > stuff, so I suspect regrtest or someone is doing an explicit flush or > something. > > In other words, having the golden output have any I/O in it is making > it impossible to test. What would probably be best is to just strip > out any and all lines that do not directly involve profilee.py and > only compare those lines. I have no clue if a simple 'contains' check > will suffice because I don't know if the stat numbers are consistent > enough. > > Anyway, I am out of time. Hopefully someone can finish up this work > some time today. Good analysis. Can't we insert a flush somewhere before the profiler is started so that the starting point is always with empty output buffers? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Fri Feb 29 23:38:38 2008 From: brett at python.org (Brett Cannon) Date: Fri, 29 Feb 2008 14:38:38 -0800 Subject: [Python-3000] test_profile failing In-Reply-To: References: Message-ID: On Fri, Feb 29, 2008 at 1:15 PM, Guido van Rossum wrote: > > On Fri, Feb 29, 2008 at 12:39 PM, Brett Cannon wrote: > > On Fri, Feb 29, 2008 at 11:57 AM, Barry Warsaw wrote: > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA1 > > > > > > > > > On Feb 29, 2008, at 2:44 PM, Brett Cannon wrote: > > > > > > > On Fri, Feb 29, 2008 at 11:22 AM, Barry Warsaw > > > > wrote: > > > >> > > > > > > >> I'll see what can be done to fix this particular problem, and try to > > > >> be more attentive to buildbot failure notices when they happen so we > > > >> can nip it closer to the bud. Ultimately, it would be nice if we had > > > >> a process that prevented any breaking patch from landing, but we're > > > >> not there yet. > > > > > > > > A quick tweak to test_profile to print out what it got for output > > > > shows that some changes to io.py led to a different tracing. > > > > > > Thanks Brett. Do we just need to regen the expected output then? > > > > Ah, the joys of expected output. =) > > > > So here is the problem, and it is probably going to mean test_profile > > needs to be rewritten... > > > > When you run ``./python Lib/test/test_profile.py -r`` it runs the > > profile and rewrites the file with the golden output. That's great and > > all, but there was a slight difference with that golden output and > > what was being generated by ``./python Lib/test/test_profile.py``. You > > know what that difference was? An extra write call in io.py. > > > > Turns out that when the golden output is generated that all streams > > have their buffers empty. But when unittest prints its whole "__test__ > > .. ok" output it has stuff in a buffer since the "ok" has not been > > written yet (in other words sys.stdout has not been fully flushed yet > > while the test is running). If you change the print() call in > > test_profile.regenerate_expected_output() to ``print('Regenerating %s > > ... ' % filename, end='')`` then sys.stdout gets a flush() call during > > the profiling for the golden output and everyone is happy. > > > > But guess what happens when you run ``./python Lib/test/regrtest.py > > test_profile``? That's right, there is no extra flush call! > > regrtest.py outputs a single line, runs the test, and then prints > > another line when it knows the result of the test.. Actually, there is > > one less flush call in the regrtest.py case than in the generated > > stuff, so I suspect regrtest or someone is doing an explicit flush or > > something. > > > > In other words, having the golden output have any I/O in it is making > > it impossible to test. What would probably be best is to just strip > > out any and all lines that do not directly involve profilee.py and > > only compare those lines. I have no clue if a simple 'contains' check > > will suffice because I don't know if the stat numbers are consistent > > enough. > > > > Anyway, I am out of time. Hopefully someone can finish up this work > > some time today. > > Good analysis. Can't we insert a flush somewhere before the profiler > is started so that the starting point is always with empty output > buffers? Flushing both stdout and stderr just before the logging call still doesn't fix it for running the test through regrtest. There is an extra call to 'exec' (not sure if it is just a method on some object named that or the actual built-in) triggering a call to IOBase.flush() that is only being triggered when the test is run under regrtest. Looking in regrtest the only exec() call is when you hunt for refleaks, not general testing so I am not sure where it is coming from. -Brett From scott+python-dev at scottdial.com Fri Feb 29 09:57:47 2008 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 29 Feb 2008 03:57:47 -0500 Subject: [Python-3000] [Python-Dev] Code freeze? In-Reply-To: <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> References: <47C7105A.20908@cheimes.de> <00E337B6-FFC8-43C8-BA9F-142725954320@python.org> <87hcfsloqf.fsf@uwakimon.sk.tsukuba.ac.jp> <476595E7-1350-40F1-8E64-EB4EB64F69DB@python.org> Message-ID: <47C7C90B.5040707@scottdial.com> Barry Warsaw wrote: >> Alterntaively, I guess you could just suggest that people check the >> buildbot page for their platforms before downloading .... > > Yes, good idea. I'm only going to cut source tarballs for the alphas. > I apologize for having doubt in your plan, and I can certainly appreciate the work you will be doing as release manager. But.. I don't understand who these alpha releases are supposed to be for, and who they will serve. When I first saw this plan, I thought "ok, you're the man.." But now I wonder even more if they are only going to be source tarballs. Who is the intended audience of these alphas? It seems like cutting only source tarballs is targeting developers, and if that is the case, I wonder if you have misplaced your motivations. I'm not sure what developer outside of the core community would want to work with something missing key features and released fairly arbitrarily. Ok, I grant you that it's a monthly cycle, but there are no feature milestones involved, so it's fairly arbitrary from a utility standpoint; it's not clear to me that we are even worrying about all of the buildbots being green before releasing. More to the point, I don't know why a developer wouldn't just checkout from SVN in any case. Certainly if they are going to help root out bugs, then we would like them to be using the trunk if possible. I fear that once an alpha is 2 weeks old, we will start saying "please check if its still a problem on the trunk." A mild concern is how this effects checkins with individuals either trying to meet a deadline or even wait until after an alpha release to checkin a large change. I'm not sure this will happen, but having releases scheduled without feature milestones attached certainly changes the environment for work to be done in. I guess I am hoping to understand better how these alphas serve us. Again, I apologize if that sounded judgmental, but I wanted to make sure that releasing alphas was the best plan. And again, I appreciate your enthusiasm and effort. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu