From robert.kern at gmail.com Mon Jun 1 18:44:06 2009 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 01 Jun 2009 11:44:06 -0500 Subject: [Python-Dev] Indentation oddness... In-Reply-To: <4A21E541.1020003@canterbury.ac.nz> References: <1A472770E042064698CB5ADC83A12ACD02915C13@TK5EX14MBXC118.redmond.corp.microsoft.com> <4A21E541.1020003@canterbury.ac.nz> Message-ID: On 2009-05-30 21:02, Greg Ewing wrote: > Robert Kern wrote: > >> The 'single' mode, which is used for the REPL, is a bit different than >> 'exec', which is used for modules. This difference lets you insert >> "blank" lines of whitespace into a function definition without exiting >> the definition. > > All that means is that the REPL needs to keep reading > lines until it gets a completely blank one. I don't > see why the compiler has to treat the source any > differently once the REPL has decided how much text > to feed it. Not true. The REPL will keep reading lines until compile(code,'','single') passes without a SyntaxError. You can put in blank lines when line continuation is implicit, like in the middle of a list. This is the reason that there is a 'single' mode in the first place, to determine when you've stopped typing. It's easier to add the grammar rule that a block does not end with a line of whitespace to the compiler than to implement all of the context-specific special cases for pure empty lines outside of the compiler. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From dinov at microsoft.com Mon Jun 1 18:50:19 2009 From: dinov at microsoft.com (Dino Viehland) Date: Mon, 1 Jun 2009 16:50:19 +0000 Subject: [Python-Dev] syntax warnings don't go through warnings.warn? Message-ID: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> I'm just a little surprised by this - Is there a reason why syntax warnings are special and untrappable via warnings.warn? >>> import warnings >>> def mywarn(*args): print 'xx', args ... >>> warnings.warn = mywarn >>> compile("def f():\n a = 1\n global a\n", "", "exec") :3: SyntaxWarning: name 'a' is assigned to before global declaration at 012DB410, file "", line 1> -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew at matthewwilkes.co.uk Mon Jun 1 19:29:08 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Mon, 1 Jun 2009 18:29:08 +0100 Subject: [Python-Dev] syntax warnings don't go through warnings.warn? In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk> On 1 Jun 2009, at 17:50, Dino Viehland wrote: > I?m just a little surprised by this - Is there a reason why syntax > warnings are special and untrappable via warnings.warn? Why should this work? From the docs... "Python programmers issue warnings by calling the warn() function defined in this module. (C programmers use PyErr_WarnEx; see Exception Handling for details)." Check out the warnings.catch_warnings context manager, but if you have any further questions please direct them to the normal Python mailing list, this is for development _of_ Python only. Matthew From fuzzyman at voidspace.org.uk Mon Jun 1 19:42:24 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 01 Jun 2009 18:42:24 +0100 Subject: [Python-Dev] syntax warnings don't go through warnings.warn? In-Reply-To: <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk> References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk> Message-ID: <4A241300.4060200@voidspace.org.uk> Matthew Wilkes wrote: > > On 1 Jun 2009, at 17:50, Dino Viehland wrote: > >> I?m just a little surprised by this - Is there a reason why syntax >> warnings are special and untrappable via warnings.warn? > > Why should this work? From the docs... "Python programmers issue > warnings by calling the warn() function defined in this module. (C > programmers use PyErr_WarnEx; see Exception Handling for details)." > > Check out the warnings.catch_warnings context manager, but if you have > any further questions please direct them to the normal Python mailing > list, this is for development _of_ Python only. Dino is developing Python - he's one of the core developers of IronPython and I suspect he is asking whether this is intentional, and IronPython should implement the same behaviour, or whether it is a bug. Michael > > Matthew > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From robert.kern at gmail.com Mon Jun 1 19:44:22 2009 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 01 Jun 2009 12:44:22 -0500 Subject: [Python-Dev] syntax warnings don't go through warnings.warn? In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> Message-ID: On 2009-06-01 11:50, Dino Viehland wrote: > I?m just a little surprised by this - Is there a reason why syntax > warnings are special and untrappable via warnings.warn? > > >>> import warnings > > >>> def mywarn(*args): print 'xx', args > > ... > > >>> warnings.warn = mywarn > > >>> compile("def f():\n a = 1\n global a\n", "", "exec") > > :3: SyntaxWarning: name 'a' is assigned to before global declaration > > at 012DB410, file "", line 1> symtable.c uses PyErr_WarnExplicit() to provide file and line number information. You need to stub warnings.warn_explicit(). >>> import warnings >>> def mywarn_explicit(*args): ... print 'xx', args ... >>> warnings.warn_explicit = mywarn_explicit >>> compile("def f():\n a = 1\n global a\n", "", "exec") xx ("name 'a' is assigned to before global declaration", , '', 3, None, None) at 0x39e9f8, file "", line 1> -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From dinov at microsoft.com Mon Jun 1 19:44:45 2009 From: dinov at microsoft.com (Dino Viehland) Date: Mon, 1 Jun 2009 17:44:45 +0000 Subject: [Python-Dev] syntax warnings don't go through warnings.warn? In-Reply-To: <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk> References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk> Message-ID: <1A472770E042064698CB5ADC83A12ACD0296DAA3@TK5EX14MBXC118.redmond.corp.microsoft.com> I work on IronPython so I am asking a question about development of Python. In particular my goal is to make sure that IronPython is behaving the same as CPython. If the normal discussion list is more appropriate for these questions I'd be happy to take it there. But before I do that please consider nt.tempnam - a built-in function. This goes through the normal warning mechanism but the parser doesn't. I would assume that this would go through PyErr_WarnEx (sorry, I can't actually look at the CPython code) given that tempnam is implemented in C. Why wouldn't the parser also go through PyErr_WarnEx? And warnings.catch_warnings doesn't work w/ parse warnings either so I'm not sure what the point of bringing that up is. -----Original Message----- From: Matthew Wilkes [mailto:matthew at matthewwilkes.co.uk] Sent: Monday, June 01, 2009 10:29 AM To: Dino Viehland Cc: Python-Dev Subject: Re: [Python-Dev] syntax warnings don't go through warnings.warn? On 1 Jun 2009, at 17:50, Dino Viehland wrote: > I'm just a little surprised by this - Is there a reason why syntax > warnings are special and untrappable via warnings.warn? Why should this work? From the docs... "Python programmers issue warnings by calling the warn() function defined in this module. (C programmers use PyErr_WarnEx; see Exception Handling for details)." Check out the warnings.catch_warnings context manager, but if you have any further questions please direct them to the normal Python mailing list, this is for development _of_ Python only. Matthew From matthew at matthewwilkes.co.uk Mon Jun 1 19:49:04 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Mon, 1 Jun 2009 18:49:04 +0100 Subject: [Python-Dev] syntax warnings don't go through warnings.warn? In-Reply-To: <4A241300.4060200@voidspace.org.uk> References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk> <4A241300.4060200@voidspace.org.uk> Message-ID: <02BF8B44-5285-4574-8CE1-734BD579E3F1@matthewwilkes.co.uk> On 1 Jun 2009, at 18:42, Michael Foord wrote: > Dino is developing Python - he's one of the core developers of > IronPython Ah, sorry, I'm bad with names, don't always pick up on who is who! > and I suspect he is asking whether this is intentional, and > IronPython should implement the same behaviour, or whether it is a > bug. The docs do seem to be clear though, warnings.warn creates a warning, but there are multiple other ways to do it, it's a convenience method. Matt From dinov at microsoft.com Mon Jun 1 19:58:33 2009 From: dinov at microsoft.com (Dino Viehland) Date: Mon, 1 Jun 2009 17:58:33 +0000 Subject: [Python-Dev] syntax warnings don't go through warnings.warn? In-Reply-To: <02BF8B44-5285-4574-8CE1-734BD579E3F1@matthewwilkes.co.uk> References: <1A472770E042064698CB5ADC83A12ACD0296CD4E@TK5EX14MBXC118.redmond.corp.microsoft.com> <2BB78789-E545-4C3A-9652-5DF473C8A4A7@matthewwilkes.co.uk> <4A241300.4060200@voidspace.org.uk> <02BF8B44-5285-4574-8CE1-734BD579E3F1@matthewwilkes.co.uk> Message-ID: <1A472770E042064698CB5ADC83A12ACD0296DC13@TK5EX14MBXC118.redmond.corp.microsoft.com> Ahh, ok, now I think I see... there is clearly some other mechanism that this warning is going through that we're not handling correctly. I'll try and track it down. Thanks! -----Original Message----- From: Matthew Wilkes [mailto:matthew at matthewwilkes.co.uk] Sent: Monday, June 01, 2009 10:49 AM To: Michael Foord Cc: Dino Viehland; Python-Dev Subject: Re: [Python-Dev] syntax warnings don't go through warnings.warn? On 1 Jun 2009, at 18:42, Michael Foord wrote: > Dino is developing Python - he's one of the core developers of > IronPython Ah, sorry, I'm bad with names, don't always pick up on who is who! > and I suspect he is asking whether this is intentional, and > IronPython should implement the same behaviour, or whether it is a > bug. The docs do seem to be clear though, warnings.warn creates a warning, but there are multiple other ways to do it, it's a convenience method. Matt From python at rcn.com Mon Jun 1 20:32:03 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 1 Jun 2009 11:32:03 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr Message-ID: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> In http://bugs.python.org/issue3959 , Clay McClure is raising some objections to the new ipaddr.py module. JP Calderone shares his concerns. I think they were the only commenters not directly affiliated with one of the competing projects. The issues they've raised seem serious, but I don't know enough about the subject to make a meaningful comment. Am hoping python-dev participants can provide some informed judgments. At issue is whether the module has some deep conceptual flaws that would warrant pulling it out of the 3.1 release. Also at issue is whether the addition was too rushed (see David Moss's comment on the tracker and later comments by Antoine Pitrou). Does anyone here know if Clay's concern about subnets vs netmasks in accurate and whether it affects the usability of the module? Raymond From guido at python.org Mon Jun 1 20:45:01 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 1 Jun 2009 11:45:01 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> Message-ID: I haven't read the bug, but given the extensive real-life use that ipaddr.py has seen at Google before inclusion into the stdlib, I think "deep conceptual flaws" must be an overstatement. There may be real differences of opinion about the politically correct way to view subnets and netmasks, but I don't doubt that the module as it stands is usable enough to keep it in the stdlib. Nothing's perfect. I think we should just stick to "sorry, too late, try again for 3.2". We've done that with plenty of more important flaws that were discovered on the verge of a release, and I don't recall ever regretting it. We can always add more features to the module in 3.2. --Guido On Mon, Jun 1, 2009 at 11:32 AM, Raymond Hettinger wrote: > In http://bugs.python.org/issue3959 , Clay McClure is raising some > objections to the new ipaddr.py module. ?JP Calderone shares his concerns. > ?I think they were the only commenters not directly affiliated with one of > the competing projects. ?The issues they've raised seem serious, but I don't > know enough about the subject to make a meaningful comment. > > Am hoping python-dev participants can provide some informed judgments. ?At > issue is whether the module has some deep conceptual flaws that would > warrant pulling it out of the 3.1 release. ?Also at issue is whether the > addition was too rushed (see David Moss's comment on the tracker and later > comments by Antoine Pitrou). > > Does anyone here know if Clay's concern about subnets vs netmasks in > accurate and whether it affects the usability of the module? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rdmurray at bitdance.com Mon Jun 1 21:11:45 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 1 Jun 2009 15:11:45 -0400 (EDT) Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> Message-ID: On Mon, 1 Jun 2009 at 11:32, Raymond Hettinger wrote: > Does anyone here know if Clay's concern about subnets vs netmasks in accurate > and whether it affects the usability of the module? I can't speak to usability of the module, not having looked at it yet, but as far as I know from 10+ years of experience programming Cisco and other routers, a host address can always also be considered as a subnet consisting of one address. The netmask is 255.255.255.255, and is called a "host mask". That said, address+hostmask is not always the way you _think_ about an individual address; it depends on the context. FWIW I hate dealing with non-subnet-aligned IP address ranges whenever they come up. But it is true that they do come up. --David From martin at v.loewis.de Mon Jun 1 21:16:01 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 01 Jun 2009 21:16:01 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> Message-ID: <4A2428F1.7030708@v.loewis.de> Raymond Hettinger wrote: > Also at issue is whether the > addition was too rushed (see David Moss's comment on the tracker and > later comments by Antoine Pitrou). That comment was from January 2009; it had two aspects a) is ipaddr getting special treatment just because it was contributed by Google? and b) shouldn't alternatives be considered? For a), the answer was always a clear "no". That the code comes from Google didn't affect inclusion at all. For b), there have been several attempts to approach alternatives since. For example, both netaddr people and ipaddr people tried to merge the projects, unsuccessfully. Both Duncan McGreggor and David Moss eventually spoke in favor of including ipaddr. So I think this particular issue was resolved. As for Clay McLure's issue: I feel it's primarily a matter of taste. I see nothing morally wrong in using the same class for hosts and networks, i.e. representing a host as a network of size 1. I can understand why people dislike that, but I don't see why it would stop people from doing with the library what they want to do. A number of other features might be missing from the library, for example, as Clay complains, there is no support for ranges of addresses (or, more generally, arbitrary sets). While I understand that there are good applications for such a data structure, I also think it can be added when contributed (and I would think "range of addresses" is still too narrow, for some applications I would envision - such as computing consistency of BGP tables and whois databases). Regards, Martin From mcguire at google.com Tue Jun 2 03:54:21 2009 From: mcguire at google.com (Jake McGuire) Date: Mon, 1 Jun 2009 18:54:21 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A2428F1.7030708@v.loewis.de> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> Message-ID: <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. L?wis" wrote: > As for Clay McLure's issue: I feel it's primarily a matter of taste. > I see nothing morally wrong in using the same class for hosts and > networks, i.e. representing a host as a network of size 1. I can > understand why people dislike that, but I don't see why it would stop > people from doing with the library what they want to do. > To the extent that Clay is having issues, it's because ipaddr.py is poorly documented, has potentially confusing comments, and he became confused. Lesser issues are that ipaddr.py doesn't work the way he wants and that ip addressing is inherently subtle. Looking at the code in detail shows that ipaddr.IP/IPv4/IPv6 objects always represent *networks*. He wants one particular address out of that network, and that requires using __getitem__ or using IP.ip_ext. Neither is particularly intuitive. >>> import ipaddr >>> ip = ipaddr.IPv4('10.33.11.17') >>> ip IPv4('10.33.11.17/32') >>> ip[0] '10.33.11.17' >>> ip.ip_ext '10.33.11.17' >>> This feels much more like poor documentation than wide-ranging conceptual flaws. I could put this in the tracker, but I'm not sure if that's appropriate. -jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From jake at youtube.com Tue Jun 2 04:47:46 2009 From: jake at youtube.com (Jake McGuire) Date: Mon, 1 Jun 2009 19:47:46 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> Message-ID: <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> On Mon, Jun 1, 2009 at 6:54 PM, Jake McGuire wrote: > On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. L?wis" wrote: > >> As for Clay McLure's issue: I feel it's primarily a matter of taste. >> I see nothing morally wrong in using the same class for hosts and >> networks, i.e. representing a host as a network of size 1. I can >> understand why people dislike that, but I don't see why it would stop >> people from doing with the library what they want to do. >> > > To the extent that Clay is having issues, it's because ipaddr.py is poorly > documented, has potentially confusing comments, and he became confused. > Lesser issues are that ipaddr.py doesn't work the way he wants and that ip > addressing is inherently subtle. > Sorry for the spam, I wrote my last message before reading the entire discussion surrounding the two libraries and trying to imagine using both ipaddr and netaddr. Clay is basically correct. The ipaddr.py API is missing important features, and it would probably be a mistake to add it to the python standard library if that means we'd have to maintain compatibility for the indefinite future. Like all largeish python projects, we wrote a library to do IP address manipulation. In our case, it's a whopping 64 lines long. While I wasn't aware of ipaddr.py or netaddr at the time, the API we created is much closer to netaddr's. Migrating our code to ipaddr would require significant work and is unlikely to happen. In fact, if I was starting a new project from scratch with similar requirements, I'd probably write my own library instead of using ipaddr. -jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From clay at daemons.net Tue Jun 2 07:06:47 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 01:06:47 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr Message-ID: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> On Mon, Jun 1, 2009 at 11:32 AM, Guido van Rossum wrote: > I haven't read the bug, but given the extensive real-life use that > ipaddr.py has seen at Google before inclusion into the stdlib, I think > "deep conceptual flaws" must be an overstatement. When the users of the library are also the authors of the library, it is not surprising that the library enjoys "extensive real-life use." The real test of a library is not how well it succeeds at one installation, but how well it meets the needs of the larger user base. Having read the code and the author's comments, it seems to me that networking concepts not employed at Google have been neglected. While some of these features are easily added to ipaddr, their omission exposes a narrow view of the problem domain that has resulted in more fundamental problems in the library, such as the conflation of addresses and networks. > I think we should just stick to "sorry, too late, try again for 3.2". > We've done that with plenty of more important flaws that were > discovered on the verge of a release, and I don't recall ever > regretting it. We can always add more features to the module in 3.2. My concerns are not so much about adding features as they are about changing the API. Addressing the concerns that I and others have raised requires making backwards-incompatible changes. Doing that now, before ipaddr enjoys widespread deployment as part of the stdlib, seems prudent. Removing ipaddr from 3.1 appears to me less painful than fixing apps when ipaddr's API changes in 3.2. If this were a core feature that many developers were anxiously awaiting, I could understand the desire to release and iterate. But is there really a pressing need for an IP library in the stdlib? Until a satisfactory library is available for inclusion in the stdlib, the few developers that do require such functionality can easily enough download a library that meets their needs. Clay From martin at v.loewis.de Tue Jun 2 07:38:43 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 02 Jun 2009 07:38:43 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> Message-ID: <4A24BAE3.8070809@v.loewis.de> > Clay is basically correct. The ipaddr.py API is missing important > features, and it would probably be a mistake to add it to the python > standard library if that means we'd have to maintain compatibility for > the indefinite future. >From a maintenance point of view, these two statements don't really relate. Yes, adding it to the standard library means that compatibility must be maintained (not for the indefinite future, but in a very strong sense). But no, that doesn't mean that it cannot have new features. Adding new features would not have to break compatibility, and, in many real-world cases of existing libraries, didn't. For the net-vs-host issue, I think a backwards-compatible solution is possible: just give the IP() function an option parameter that makes it reject a netmask during parsing. > Like all largeish python projects, we wrote a library to do IP address > manipulation. In our case, it's a whopping 64 lines long. [...] > In fact, if I was starting a new project from scratch with similar > requirements, I'd probably write my own library instead of using ipaddr. That was my feeling as well when ipaddr was first offered. It's just not an important library, and people will continue to roll their own for some time. OTOH, with ipaddr in the standard library, people will also start contributing extensions that make it support their use cases, so it should grow a wider application area than it currently supports. Regards, Martin From martin at v.loewis.de Tue Jun 2 07:47:28 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 02 Jun 2009 07:47:28 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> Message-ID: <4A24BCF0.5030800@v.loewis.de> > If this were a core feature that many developers were anxiously > awaiting, I could understand the desire to release and iterate. But is > there really a pressing need for an IP library in the stdlib? You should have asked this question a few month ago. Now, release mechanics make it difficult to remove the library, except if a severe problem was uncovered - which you haven't been able to demonstrate. I don't believe that your issue "hosts and nets are represented with the same class" is severe: it is very well possible to use the IP function to represent individual hosts (including having a string representation that doesn't print a netmask). The only possibly-missing feature is support for rejecting host strings that include a mask on parsing; that can be added in a backwards-compatible way as an optional parameter to the IP function (as discussed on the tracker). Regards, Martin From clay at daemons.net Tue Jun 2 07:52:31 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 01:52:31 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A24BAE3.8070809@v.loewis.de> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> Message-ID: <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> On Tue, Jun 2, 2009 at 1:38 AM, "Martin v. L?wis" wrote: > For the net-vs-host issue, I think a backwards-compatible solution > is possible: just give the IP() function an option parameter that > makes it reject a netmask during parsing. That doesn't solve much. IPv4 objects still always use CIDR notation when coerced to strings, meaning that IP addresses will always be rendered with a trailing "/32". Such notation is unacceptable in real-world applications that (correctly) distinguish between address and network. > That was my feeling as well when ipaddr was first offered. It's just > not an important library, and people will continue to roll their own > for some time. OTOH, with ipaddr in the standard library, people will > also start contributing extensions that make it support their use cases, > so it should grow a wider application area than it currently supports. That being the case, why not delay its inclusion until we can be sure that it in fact represents a good base upon which to build? Clay From clay at daemons.net Tue Jun 2 08:01:28 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 02:01:28 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A24BCF0.5030800@v.loewis.de> References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> <4A24BCF0.5030800@v.loewis.de> Message-ID: <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com> On Tue, Jun 2, 2009 at 1:47 AM, "Martin v. L?wis" wrote: >> If this were a core feature that many developers were anxiously >> awaiting, I could understand the desire to release and iterate. But is >> there really a pressing need for an IP library in the stdlib? > > You should have asked this question a few month ago. Now, release > mechanics make it difficult to remove the library, except if a severe > problem was uncovered - which you haven't been able to demonstrate. This argument makes no sense. The feasibility of removing the library does not depend on the severity of the issues found within it. Either it is hard to remove the library, or it is easy. If it's hard to remove, it doesn't get any easier because I discover a fatal flaw. I've actually provided several examples of where the library fails when used in common scenarios, yet your solution involves incremental hacks that don't fix the underlying problems. You write as if you have a vested interest in releasing the library -- why? > I don't believe that your issue "hosts and nets are represented with the > same class" is severe: it is very well possible to use the IP function > to represent individual hosts (including having a string representation > that doesn't print a netmask). The only possibly-missing feature is > support for rejecting host strings that include a mask on parsing; that > can be added in a backwards-compatible way as an optional parameter to > the IP function (as discussed on the tracker). There are other missing features, but again, my concern is not about missing features: it's about a quirky API that conflates concepts in the problem domain, leading to subtle bugs and breaking the principle of least surprise. Clay From martin at v.loewis.de Tue Jun 2 08:08:56 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 02 Jun 2009 08:08:56 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> Message-ID: <4A24C1F8.7080905@v.loewis.de> Clay McClure wrote: > On Tue, Jun 2, 2009 at 1:38 AM, "Martin v. L?wis" wrote: > >> For the net-vs-host issue, I think a backwards-compatible solution >> is possible: just give the IP() function an option parameter that >> makes it reject a netmask during parsing. > > That doesn't solve much. IPv4 objects still always use CIDR notation > when coerced to strings, meaning that IP addresses will always be > rendered with a trailing "/32". That's not true: py> x = ipaddr.IP("30.40.50.60") py> print(x.ip_ext_full) 30.40.50.60 > Such notation is unacceptable in > real-world applications that (correctly) distinguish between address > and network. So use a different notation that is also supported by the library. >> That was my feeling as well when ipaddr was first offered. It's just >> not an important library, and people will continue to roll their own >> for some time. OTOH, with ipaddr in the standard library, people will >> also start contributing extensions that make it support their use cases, >> so it should grow a wider application area than it currently supports. > > That being the case, why not delay its inclusion until we can be sure > that it in fact represents a good base upon which to build? Because we *are* sure that it in fact represents a good base upon which to build. Regards, Martin From martin at v.loewis.de Tue Jun 2 08:15:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 02 Jun 2009 08:15:42 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com> References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> <4A24BCF0.5030800@v.loewis.de> <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com> Message-ID: <4A24C38E.1050609@v.loewis.de> > This argument makes no sense. The feasibility of removing the library > does not depend on the severity of the issues found within it. Either > it is hard to remove the library, or it is easy. If it's hard to > remove, it doesn't get any easier because I discover a fatal flaw. We could remove it, but then what we have wouldn't really be a release candidate anymore, so the release would get delayed. > I've actually provided several examples of where the library fails > when used in common scenarios, yet your solution involves incremental > hacks that don't fix the underlying problems. You write as if you have > a vested interest in releasing the library -- why? I have a vested interest in releasing Python 3.1, and in sticking to decisions that have been made by other committers. I trust these fellow committers, so I defend their decisions. I personally have no plans for using this library, or any other IP address library (at least not in any application I plan to write soon). At the moment, I'm struggling more with IP address libraries in Perl. > There are other missing features, but again, my concern is not about > missing features: it's about a quirky API that conflates concepts in > the problem domain, leading to subtle bugs and breaking the principle > of least surprise. I believe the API appears more quirky to you than it actually is, probably because you don't have understood it fully (but I might be wrong with that, and there might be a different reason). Regards, Martin From rdmurray at bitdance.com Tue Jun 2 15:22:33 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 2 Jun 2009 09:22:33 -0400 (EDT) Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> Message-ID: On Mon, 1 Jun 2009 at 18:54, Jake McGuire wrote: > On Mon, Jun 1, 2009 at 12:16 PM, "Martin v. L?wis" wrote: >> As for Clay McLure's issue: I feel it's primarily a matter of taste. >> I see nothing morally wrong in using the same class for hosts and >> networks, i.e. representing a host as a network of size 1. I can >> understand why people dislike that, but I don't see why it would stop >> people from doing with the library what they want to do. > > To the extent that Clay is having issues, it's because ipaddr.py is poorly > documented, has potentially confusing comments, and he became confused. > Lesser issues are that ipaddr.py doesn't work the way he wants and that ip > addressing is inherently subtle. > > Looking at the code in detail shows that ipaddr.IP/IPv4/IPv6 objects always > represent *networks*. He wants one particular address out of that network, > and that requires using __getitem__ or using IP.ip_ext. Neither is > particularly intuitive. > >>>> import ipaddr >>>> ip = ipaddr.IPv4('10.33.11.17') >>>> ip > IPv4('10.33.11.17/32') >>>> ip[0] > '10.33.11.17' Actually I find that very intuitive if I understand that the objects are always networks. >>>> ip.ip_ext > '10.33.11.17' That's not intuitive :) > This feels much more like poor documentation than wide-ranging conceptual > flaws. > > I could put this in the tracker, but I'm not sure if that's appropriate. I would say yes, put it in the tracker. --David From clay at daemons.net Tue Jun 2 16:08:18 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 10:08:18 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> Message-ID: <8657ee3f0906020708i3d9330adx2c032d8521653153@mail.gmail.com> On Tue, Jun 2, 2009 at 9:22 AM, R. David Murray wrote: >>>>> ip[0] >> >> '10.33.11.17' > > Actually I find that very intuitive if I understand that the objects > are always networks. I suspect the authors would disagree with you on this point, since they insist that host routes and host addresses are the same thing, but assuming you are right, two flaws are immediately apparent: * The ipaddr classes are poorly named. Since they model networks, they should be named appropriately: IPNet or some such. * The ipaddr authors have overlooked IP addresses, which also deserve first-class treatment in any meaningful IP address library. It is called ipaddr, after all, and not ipnet. >>>>> ip.ip_ext >> >> '10.33.11.17' > > That's not intuitive :) No, nor is the rest of the library intuitive. This is one of the reasons I've called it "quirky". Clay From clay at daemons.net Tue Jun 2 18:25:39 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 12:25:39 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A24C38E.1050609@v.loewis.de> References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> <4A24BCF0.5030800@v.loewis.de> <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com> <4A24C38E.1050609@v.loewis.de> Message-ID: <8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com> On Tue, Jun 2, 2009 at 2:15 AM, "Martin v. L?wis" wrote: > We could remove it, but then what we have wouldn't really be a release > candidate anymore, so the release would get delayed. How long do release candidates soak in the field before being accepted? I'm curious if an exception could be made in this case, given that you have admitted that ipaddr is not an important library. The chances of a problem being introduced due to its removal are vanishingly small. No other components of the stdlib depend on ipaddr, and the few (approximately zero?) developers who will have started writing applications depending on ipaddr due to its inclusion in the release candidate could simply download the library from Google. > I believe the API appears more quirky to you than it actually is, > probably because you don't have understood it fully (but I might > be wrong with that, and there might be a different reason). I believe the API is quirky because: * It tries to represent distinct domain model concepts in a single class * Its methods and properties are strangely named * Methods and properties that should return domain model objects return native types instead * It cannot correctly parse output from netstat, nor can it correctly pass values to ifconfig You seem comfortable with these quirks, but then you're not planning to write software with this library. Developers who do intend to write meaningful network applications do seem concerned, yet we're ignored. If you consult the issue notes, you'll see objections of the type I just mentioned were raised months ago, yet no work has been done to correct them. The only changes that I can see were related to pedantic style issues: camel case and indentation. Clay From clay at daemons.net Tue Jun 2 18:26:58 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 12:26:58 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A24C1F8.7080905@v.loewis.de> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> Message-ID: <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. L?wis" wrote: >> That doesn't solve much. IPv4 objects still always use CIDR notation >> when coerced to strings, meaning that IP addresses will always be >> rendered with a trailing "/32". > > That's not true: > > py> x = ipaddr.IP("30.40.50.60") > py> print(x.ip_ext_full) > 30.40.50.60 Thankfully the authors have provided this obscure and strangely-named method to get at the correct string representation of an IP address, but sadly their __str__ method -- which is the Pythonic way to get string representations of objects -- fails in this regard because they have only one class representing two distinct concepts. I could probably make ipaddr work in my application; that is not the issue (at least in my mind). The issue is that I shouldn't have to work around design flaws in the library when they're simple enough to fix before the library is bundled with the stdlib. I do not see the utility of including ipaddr when my choice as a user is to either use its strangely-named methods instead of obvious Pythonic conventions, or to write my own library. >> Such notation is unacceptable in >> real-world applications that (correctly) distinguish between address >> and network. > > So use a different notation that is also supported by the library. I'm not referring to my software here -- I'm referring to applications like ifconfig that expect addresses to be formatted properly. If the default string representation produced by the ipaddr library does not match the canonical representation expected by software that has existed as long as IP itself, that indicates to me the library is doing something wrong. > Because we *are* sure that it in fact represents a good base upon which > to build. You certainly seem convinced. I could not disagree more. Clay From martin at v.loewis.de Tue Jun 2 19:34:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 02 Jun 2009 19:34:11 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com> References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> <4A24BCF0.5030800@v.loewis.de> <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com> <4A24C38E.1050609@v.loewis.de> <8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com> Message-ID: <4A256293.9000204@v.loewis.de> >> We could remove it, but then what we have wouldn't really be a release >> candidate anymore, so the release would get delayed. > > How long do release candidates soak in the field before being accepted? For this release, the release schedule is defined in PEP 375 > I'm curious if an exception could be made in this case, given that you > have admitted that ipaddr is not an important library. This would be need to be decided by the release manager. However, given that Guido has already pronounced on this issue, Benjamin is unlikely to change anything. > You seem comfortable with these quirks, but then you're not planning > to write software with this library. Developers who do intend to write > meaningful network applications do seem concerned, yet we're ignored. I don't hear a public outcry - only a single complainer. Regards, Martin From exarkun at divmod.com Tue Jun 2 19:56:00 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 2 Jun 2009 13:56:00 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A256293.9000204@v.loewis.de> Message-ID: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> On Tue, 02 Jun 2009 19:34:11 +0200, "\"Martin v. L?wis\"" wrote: > [snip] > >> You seem comfortable with these quirks, but then you're not planning >> to write software with this library. Developers who do intend to write >> meaningful network applications do seem concerned, yet we're ignored. > >I don't hear a public outcry - only a single complainer. Clay repeatedly pointed out that other people have objected to ipaddr and been ignored. It's really, really disappointing to see you continue to ignore not only them, but the repeated attempts Clay has made to point them out. I don't have time to argue this issue, but I agree with essentially everything Clay has said in this thread, and I commented about these problems on the ticket months ago, before ipaddr was added. Jean-Paul From python at rcn.com Tue Jun 2 20:01:55 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 2 Jun 2009 11:01:55 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com><4A24BCF0.5030800@v.loewis.de><8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com><4A24C38E.1050609@v.loewis.de> <8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com> Message-ID: > We could remove it, but then what we have wouldn't really be a release > candidate anymore, so the release would get delayed. Not true. There is a second release candidate scheduled on June 13th. Removing the module involves doing "svn remove" on two files and updating Misc/NEWS. Yesterday, there was a conversation on IRC (including the RM) where it was discussed. So, in the unlikely event that Guido changes his mind, there is still time to do so. Raymond From casevh at gmail.com Tue Jun 2 20:16:15 2009 From: casevh at gmail.com (Case Vanhorsen) Date: Tue, 2 Jun 2009 11:16:15 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A256293.9000204@v.loewis.de> References: <8657ee3f0906012206l11a82e9cl8af9fc4a7f587348@mail.gmail.com> <4A24BCF0.5030800@v.loewis.de> <8657ee3f0906012301u39b70faat4a939251a0619486@mail.gmail.com> <4A24C38E.1050609@v.loewis.de> <8657ee3f0906020925i338f5376y949a0f6a68001db7@mail.gmail.com> <4A256293.9000204@v.loewis.de> Message-ID: <99e0b9530906021116m3af1107aya36d7a0998792ba6@mail.gmail.com> On Tue, Jun 2, 2009 at 10:34 AM, "Martin v. L?wis" wrote: >>> We could remove it, but then what we have wouldn't really be a release >>> candidate anymore, so the release would get delayed. >> >> How long do release candidates soak in the field before being accepted? > > For this release, the release schedule is defined in PEP 375 > >> I'm curious if an exception could be made in this case, given that you >> have admitted that ipaddr is not an important library. > > This would be need to be decided by the release manager. However, given > that Guido has already pronounced on this issue, Benjamin is unlikely to > change anything. > >> You seem comfortable with these quirks, but then you're not planning >> to write software with this library. Developers who do intend to write >> meaningful network applications do seem concerned, yet we're ignored. > > I don't hear a public outcry - only a single complainer. I normally just lurk on python-dev, but I will comment on this thread. I manage several large IP address spaces and I've written my own IP address tools in the past. The comments on the thread motivated me to look at the ipaddr module. I fully agree with Clay's comments. I would not use the module as it stands. I apologize for lurking too much and not commenting earlier. casevh > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com > From larry.bugbee at boeing.com Tue Jun 2 20:25:38 2009 From: larry.bugbee at boeing.com (Bugbee, Larry) Date: Tue, 2 Jun 2009 11:25:38 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: Message-ID: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com> > The chances of a problem being introduced due to its removal > are vanishingly small. But that provides little consolation to the user who sees it in the standard library, is not aware to this discussion, and builds it into his app. Changes to the lib later may cause subtle but significant effects. ...perhaps undetected for a while. > > > I don't hear a public outcry - only a single complainer. > > > > Clay repeatedly pointed out that other people have objected > > to ipaddr and been ignored. It's really, really disappointing > > to see you continue to ignore not only them, but the repeated > > attempts Clay has made to point them out. > > I don't have time to argue this issue, but I agree with > essentially everything Clay has said in this thread.... I too agree. If it is not ready, it is not ready. Please don't create problems for others. Remove the lib until it is ready. Larry From p.f.moore at gmail.com Tue Jun 2 22:02:19 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 2 Jun 2009 21:02:19 +0100 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com> References: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com> Message-ID: <79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com> 2009/6/2 Bugbee, Larry : >> > > I don't hear a public outcry - only a single complainer. >> > >> > Clay repeatedly pointed out that other people have objected >> > to ipaddr and been ignored. ?It's really, really disappointing >> > to see you continue to ignore not only them, but the repeated >> > attempts Clay has made to point them out. >> >> I don't have time to argue this issue, but I agree with >> essentially everything Clay has said in this thread.... > > I too agree. ?If it is not ready, it is not ready. ?Please don't create > problems for others. ?Remove the lib until it is ready. I don't write the sort of code that requires this module very often, so I guess I class as an example of a casual user who would assume the stdlib module was "best practice", and expect to use it naively to avoid the more obvious "gotchas". I've just now read the documentation for the first time (after reading this thread) and for what it's worth here are my thoughts: * I'd expect separate classes for "an IP address" and "a subnet" - I've no problem with that expectation being wrong, but I'd like some documentation as to *why* a single class is appropriate. (More generally, the documentation seems pretty terse). Seeing "/32" all over the place in the examples reinforces my feeling. * I'm sorry, but ip_ext is a hopeless name. It makes no sense to me. My first thought was "why not just use ip?" until I realised that (contrary to the expectations of a naive user) an IP address is an integer and that (uncommon???) usage gets to use the "ip" property name. But still, why not "ip_str" at least? * IP('1.2.3.4') vs IP('1.2.3.0/255.255.255.0') - it seems entirely sane for the former to have a .ip/.ip_str/.ip_ext property, but bizarre for the latter to. Explain the principles all you like, it still seems peculiar. * To be honest, I'd prefer to have the _ext versions be named without a suffix, and the currently unsuffixed versions have an _int suffix. That may reflect my naive expectations, and experts may well expect the integer forms to be the more easily accessible, but is the library intended for expert or non-expert users? (That's not a rhetorical question). The people I work with (DBAs) deal with IP addresses all the time, but almost certainly aren't even aware that they aren't strings. If I did a poll, I guess most wouldn't even know that the components were restricted to 0-255! I can't imagine them being comfortable with this module. Reading the documentation, I'd probably end up assuming it was for experts, and writing my own code rather than using it - which is probably a sign of failure for the module. Simple example. If I want to scan all the IP addresses on my network (my IP address is 192.168.1.101) I'd probably write: for i in range(253): ip = '192.168.1.' + str(i+1) ... - and to heck with generality. Even after reading the documentation, I've *no idea* how I would use the ipaddr module to do this better. Let alone in as few lines. My requirements certainly aren't important enough to drive the design of the stdlib - and it's possible that these issues are *precisely* the sort of things that can be fixed with documentation and backward-compatible changes - but I also think that a bit more time to address such things would be reasonable. And I also apologise for not checking the module out sooner. Blame an assumption that my trivial needs would "obviously" be covered... Paul. From rdmurray at bitdance.com Tue Jun 2 22:53:16 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 2 Jun 2009 16:53:16 -0400 (EDT) Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> Message-ID: On Tue, 2 Jun 2009 at 12:26, Clay McClure wrote: > On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. L?wis" wrote: >> py> x = ipaddr.IP("30.40.50.60") >> py> print(x.ip_ext_full) >> 30.40.50.60 > > Thankfully the authors have provided this obscure and strangely-named > method to get at the correct string representation of an IP address, > but sadly their __str__ method -- which is the Pythonic way to get > string representations of objects -- fails in this regard because they > have only one class representing two distinct concepts. Having thought more about this, I will agree with you that it would be useful to have an address-without-netmask class. I'm thinking about the cases where having an attached netmask is not particularly helpful, for example in DNS entries. But there are only two reasons I've so far come up with why this would be useful: to have a different default output format, and to have a way to encode an IP so that I can have my code raise an error if I try to use it in a context where a netmask is required. (Note, however, that even a DNS entry can conceptually be considered to be a host route.) IMO, not having such a class is an inconvenience, not a show stopper, especially since it seems like one could be added without breaking backward compatibility. I also don't particularly like the names of the ipaddr attributes for accessing the IP address-without-netmask or the netmask-without-ip-address, etc; but again, I don't consider that a show stopper. So I'm not in favor of pulling ipaddr from 3.1, and it's too late in the release cycle to change anything. I wish you had brought this energy to bear earlier, when changes could have been made. Reality is what it is, though, and now we should work on making improvements for the next release. I see in the ticket that the netaddr folks were going to propose improvements so that they could build netaddr on top of ipaddr, but I guess that didn't happen (yet?). I have no association with Google, by the way, and I do intend to use ipaddr in upcoming code, and have hacked my own address manipulation stuff in previous code. >>> Such notation is unacceptable in >>> real-world applications that (correctly) distinguish between address >>> and network. >> >> So use a different notation that is also supported by the library. > > I'm not referring to my software here -- I'm referring to applications > like ifconfig that expect addresses to be formatted properly. If the > default string representation produced by the ipaddr library does not > match the canonical representation expected by software that has > existed as long as IP itself, that indicates to me the library is > doing something wrong. I don't understand why you are trying to use ifconfig as an example. It is actually a counter example to your thesis: when working with an IP address intended for consumption by ifconfig, you had best be using a datatype that includes a netmask for that IP, or your code is going to be broken. What's more, modern versions of ifconfig _do_ accept the default output format of ipaddr. So this does the Right Thing: myip = ipaddr.IP('192.168.1.1/26') system('ifconfig eth0 {}'.format(myip)) If you use your ip-address-only class with ifconfig, you will wind up using the classful default netmask, which is only going to be correct by luck. Hmm. I think there is a conceptual divide here. You have said you think about IP addresses and networks as separate objects, so I wonder if you would be pulling the netmask for ifconfig out of a separate network object? On the other hand I, a network professional, think about an IP address paired with a netmask as a fundamental object. Very rarely do I use an IP address in isolation (without a netmask), and in many of _those_ cases there is an implied netmask of 255.255.255.255. Networks to me are closely related objects, defined by their "network number" (the zero of the subnet, which is an IP address and normally not used as a host address) and the mask. So to me ipaddr's use of a single datatype makes reasonable sense. I would, as I said, above, find an ip-without-netmask data type to be also useful (but a lot less important!) I would also find a subclass that was network-only (rejects anything but the zero of the subnet for the IP) useful. But I think both of those can be implemented fairly trivially as subclasses of the existing ipaddr objects. --David PS: I've looked briefly at netaddr, and while I could probably work with it by adding some equally trivial support code, I don't think it would serve my needs as well as ipaddr will. In particular, this is _very_ troubling: >>> import netaddr >>> ip = netaddr.IP('192.168.1.1/26') >>> ip IP('192.168.1.1/26') >>> ip2 = netaddr.IP('192.168.1.1/27') >>> ip2 IP('192.168.1.1/27') >>> ip == ip2 True The docs say the netmask is accepted only for "user convenience", but to me the netmask is an integral part of the data entity I want to manipulate. Nor can I express ip-address-with-netmask using the CIDR data type, since it will not accept anything but the zero of the network as the IP. In short, netaddr's object model does not match my desired model, while ipaddr's is a lot closer to my desired model. From mcguire at google.com Tue Jun 2 22:11:22 2009 From: mcguire at google.com (Jake McGuire) Date: Tue, 2 Jun 2009 13:11:22 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> Message-ID: <77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com> On Tue, Jun 2, 2009 at 9:26 AM, Clay McClure wrote: > On Tue, Jun 2, 2009 at 2:08 AM, "Martin v. L?wis" wrote: > >>> That doesn't solve much. IPv4 objects still always use CIDR notation >>> when coerced to strings, meaning that IP addresses will always be >>> rendered with a trailing "/32". >> >> That's not true: >> >> py> x = ipaddr.IP("30.40.50.60") >> py> print(x.ip_ext_full) >> 30.40.50.60 > > Thankfully the authors have provided this obscure and strangely-named > method to get at the correct string representation of an IP address, > but sadly their __str__ method -- which is the Pythonic way to get > string representations of objects -- fails in this regard because they > have only one class representing two distinct concepts. The minimal demonstration of the problem of representing networks and addresses using the same class: >>> container = [1, 2, 3, 4] >>> for item in container: ... print "%s in %s: %s" % (item, container, item in container) ... 1 in [1, 2, 3, 4]: True 2 in [1, 2, 3, 4]: True 3 in [1, 2, 3, 4]: True 4 in [1, 2, 3, 4]: True >>> import ipaddr >>> container = ipaddr.IP('192.168.1.0/24') >>> for item in container: ... print "%s in %s: %s" % (item, container, item in container) ... Traceback (most recent call last): File "", line 2, in File "ipaddr.py", line 438, in __contains__ return self.network <= other.ip and self.broadcast >= other.broadcast AttributeError: 'str' object has no attribute 'ip' >>> -jake From rdmurray at bitdance.com Tue Jun 2 23:27:47 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 2 Jun 2009 17:27:47 -0400 (EDT) Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com> References: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com> <79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com> Message-ID: On Tue, 2 Jun 2009 at 21:02, Paul Moore wrote: > * I'd expect separate classes for "an IP address" and "a subnet" - > I've no problem with that expectation being wrong, but I'd like some > documentation as to *why* a single class is appropriate. (More > generally, the documentation seems pretty terse). Seeing "/32" all > over the place in the examples reinforces my feeling. Yeah, the documentation needs a lot of work. Since this is a subject area I know something about, hopefully I can make time to contribute something. > * I'm sorry, but ip_ext is a hopeless name. It makes no sense to me. My guess is that it is "ip external representation", as opposed to the internal representation as an integer. I agree that the name is terrible. > My first thought was "why not just use ip?" until I realised that > (contrary to the expectations of a naive user) an IP address is an > integer and that (uncommon???) usage gets to use the "ip" property > name. But still, why not "ip_str" at least? Agreed; unless you are talking to C code, I'd think you'd want the string representation. This seems like an odd design decision. > * IP('1.2.3.4') vs IP('1.2.3.0/255.255.255.0') - it seems entirely > sane for the former to have a .ip/.ip_str/.ip_ext property, but > bizarre for the latter to. Explain the principles all you like, it > still seems peculiar. It may seem peculiar, but IMO it is correct. The ip is the zero of the network, and is just as valid an IP as an ip inside the network with the same netmask. > * To be honest, I'd prefer to have the _ext versions be named without > a suffix, and the currently unsuffixed versions have an _int suffix. I agree. I wish I'd looked at this back when it was put in, but I was a lot busier with other things then :) > That may reflect my naive expectations, and experts may well expect > the integer forms to be the more easily accessible, but is the library > intended for expert or non-expert users? (That's not a rhetorical > question). The people I work with (DBAs) deal with IP addresses all > the time, but almost certainly aren't even aware that they aren't > strings. If I did a poll, I guess most wouldn't even know that the > components were restricted to 0-255! I can't imagine them being > comfortable with this module. If they don't know that, and they only work with IP addresses in the most trivial form, then I don't think they would need any of the services this library provides. That doesn't mean the library is for "experts", but it does mean it's for people who need to manipulate IP addresses within the context of networks. (If all you need to do is move IP addresses around, just keep them as strings.) Hmm. Except I suppose they could use the input validation checking... I think we've already agreed that adding a flag to IP saying 'no netmask' is a good idea...if the object created that way would by default output without a netmask, then the trivial needs of your DBA's would probably be met. > Reading the documentation, I'd probably end up assuming it was for > experts, and writing my own code rather than using it - which is > probably a sign of failure for the module. > > Simple example. If I want to scan all the IP addresses on my network > (my IP address is 192.168.1.101) I'd probably write: > > for i in range(253): > ip = '192.168.1.' + str(i+1) > ... > > - and to heck with generality. > > Even after reading the documentation, I've *no idea* how I would use > the ipaddr module to do this better. Let alone in as few lines. net = ipaddr.IP('192.168.1.0/24'): for i in range(253): ip = net[i] ... So, that's one example that needs to be added to the docs. I'd have liked to write that as: for ip in ipaddr.IP('192.168.1.0/24')[:253]: ... but apparently it doesn't support slicing (time for an RFE :) > My requirements certainly aren't important enough to drive the design > of the stdlib - and it's possible that these issues are *precisely* > the sort of things that can be fixed with documentation and > backward-compatible changes - but I also think that a bit more time to > address such things would be reasonable. If they are documentation and backward compatible changes (and I believe they are), why not get the thing into the field so more people can provide feedback and RFEs? > And I also apologise for not checking the module out sooner. Blame an > assumption that my trivial needs would "obviously" be covered... Yeah, me too. --David From martin at v.loewis.de Tue Jun 2 23:32:59 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Tue, 02 Jun 2009 23:32:59 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: <4A259A8B.1090109@v.loewis.de> > Clay repeatedly pointed out that other people have objected to ipaddr and > been ignored. It's really, really disappointing to see you continue to > ignore not only them, but the repeated attempts Clay has made to point > them out. [I meant to stop discussing here, but I just want comment on this remark] I had seen objections from Victor Stinner, which I did not fully understand, but he seemed to be saying that he is ok with including ipaddr. I had also seen objections from David Moss, which he then seems to have withdrawn. I did not take your message (msg78675 in the tracker) as an objection - you just seemed to express a preference to use netaddr instead. You said it had minor quirks, and some of them have to be removed - but I would not infer that the library should be exclude because of these minor quirks. > I don't have time to argue this issue, but I agree with essentially > everything Clay has said in this thread, and I commented about these > problems on the ticket months ago, before ipaddr was added. I now understand (but honestly didn't understand before) that you are objecting to ipaddr's inclusion, and that you would prefer its removal at this point. Regards, Martin From clay at daemons.net Wed Jun 3 00:25:10 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 18:25:10 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> Message-ID: <8657ee3f0906021525h74d28aeet614cda2e4c481d9d@mail.gmail.com> On Tue, Jun 2, 2009 at 4:53 PM, R. David Murray wrote: > Having thought more about this, I will agree with you that it would > be useful to have an address-without-netmask class. Not only useful, but necessary. It seems there are few people on this list who understand IP well enough to realize how distorted ipaddr actually is. > (Note, however, that even a DNS entry can conceptually be considered to > be a host route.) Not at all. A host route and a host address (as represented in DNS) are fundamentally different concepts. Please see my recent post to ipaddr-py-dev for a refresher on these concepts: http://groups.google.com/group/ipaddr-py-dev/t/94d54fe581d24e72 > So I'm not in favor of pulling ipaddr from 3.1, and it's too late in > the release cycle to change anything. I'm not sure why you say that when others have said that another release candidate is planned, and that removing ipaddr is essentially trivial to do. > I wish you had brought this energy to bear earlier, when changes could > have been made. ?Reality is what it is, though, and now we should work > on making improvements for the next release. ?I see in the ticket that > the netaddr folks were going to propose improvements so that they could > build netaddr on top of ipaddr, but I guess that didn't happen (yet?). I don't think that can happen, actually. If I was a netaddr committer (which I'm not), I would find it hard to salvage anything reusable from ipaddr. It is certainly simpler and clearer to start over with an object model that actually makes sense. > I have no association with Google, by the way, and I do intend to use > ipaddr in upcoming code, and have hacked my own address manipulation > stuff in previous code. Sorry, I wasn't aware of that. My mistake. Regardless, I find that your understanding of IP is similar to that of ipaddr's authors, which is to say imprecise. > I don't understand why you are trying to use ifconfig as an example. Because it's an obvious real world example that explains why these two strings are not equivalent: 192.168.1.1 and 192.168.1.1/32 You and others continue to suggest that those strings are equivalent, yet ifconfig is a tool that has been around for thirty years that clearly demonstrates that those strings are not equivalent. If what you say is true, I should be able to pass either string to ifconfig and get the same result. That is not the case, because the strings are not equivalent, because a host route is not the same thing as a host address. > So this does the Right Thing: > > ? ?myip = ipaddr.IP('192.168.1.1/26') > ? ?system('ifconfig eth0 {}'.format(myip)) Sure, but shouldn't this also do the right thing? address = ipaddr.IP('192.168.1.1') netmask = ipaddr.IP('255.255.255.192') system("ifconfig eth0 %s/%s" % (address, netmask)) It doesn't. > Hmm. ?I think there is a conceptual divide here. ?You have said you > think about IP addresses and networks as separate objects, so I wonder > if you would be pulling the netmask for ifconfig out of a separate > network object? Of course, because addresses don't have masks; networks do. This command: ifconfig en0 192.168.1.1/24 is shorthand for operator convenience. What's going on behind the scenes is quite a lot different than it looks. First, ifconfig computes a network address by masking the supplied interface address with the supplied network mask. It is then able to configure a route for the proper network: 192.168.1.0/24. The fact that "192.168.1.1/24" appears in the command does *not* mean that the address 192.168.1.1 has a mask of /24. That is absurd. Addresses don't have masks; networks do. That's why they're called netmasks. > On the other hand I, a network professional, think about an IP address > paired with a netmask as a fundamental object. The IT industry is unique among engineering disciplines in that formal training and licensing are typically not required for IT professionals. Whereas concepts like resistance, current, and voltage have very specific meanings to electrical engineers, the IT vernacular is not so precise. Since formal training is rare, and what little is available is often high-priced and vendor-specific, IT professionals tend to learn their trade from trade books, word of mouth, and hands-on experience. As a result, IT professionals tend to have a good working knowledge of how technology applies to their particular job, but may not have an appreciation of the more theoretical aspects of the discipline. What this means in practice is that your experience as a network professional may not resemble the experiences of other network professionals. That you tend to think of addresses as having masks is probably not universal, or even particularly common, among network professionals. Some electrical engineers probably think of voltage as pressure, and that may be a useful abstraction, but I would be surprised to see a voltmeter calibrated in pascals. What are we to do? How do we arrive at a common understanding of our domain? We should consult the canonical sources of truth: RFC-791, and the BSD IP implementation. In neither of those will you see that an IP address has a mask. If this were any other problem domain, I think it should be obvious that the design of the library is flawed. But given that this is IP, a subject that many people think they understand but actually don't, the design flaws are obscured by ambiguity. Guido's earlier comment about "political correctness" underscores this point. This is not simply a case of me preferring my way of thinking about IP addresses to the way the ipaddr authors think about IP addresses. I'm simply stating a fact that, were it not true, the Internet would not function: addresses and networks are not the same thing. To represent them in the same class is a mistake. > But I think both of > those can be implemented fairly trivially as subclasses of the existing > ipaddr objects. Yes, I could certainly see BaseIPAddress and BaseIPNetwork classes inheriting from BaseIP, with the ipaddr.IP function selectively return an object of either type depending on what the user passes in the constructor. If they include a mask, it's a network. If they don't, it's an address. I think that particular change might be backwards compatible, but I'm not sure. None the less, other changes that I think are important (returning IP objects instead of strings and ints for some properties; renaming poorly named methods) do change the API and thus are not backwards compatible without adding lots of cruft. Better to fix those now, in my opinion. > In short, netaddr's object model does not match my desired model, while > ipaddr's is a lot closer to my desired model. I'm not advocating netaddr. The decision of which library to use has already been made; I'm not debating that point. I'm merely suggesting that we pull ipaddr from the release until such time that it can be evolved to have a more agreeable API. ipaddr might be usable for you (probably because your limited understanding of IP matches the ipaddr developers'), but we've already heard from a handful of others on this list who would rather roll their own library than suffer through the quirks in ipaddr's current implementation. Clay From solipsis at pitrou.net Wed Jun 3 01:18:09 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 2 Jun 2009 23:18:09 +0000 (UTC) Subject: [Python-Dev] Issues with Py3.1's new ipaddr References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> <8657ee3f0906021525h74d28aeet614cda2e4c481d9d@mail.gmail.com> Message-ID: Clay McClure daemons.net> writes: > > Not only useful, but necessary. It seems there are few people on this > list who understand IP well enough to realize how distorted ipaddr > actually is. Not having myself enough knowledge about IP routing and administration (although I did happen to dissect BGP and IS-IS packets in an earlier life), I'm, however, inclined to trust Jean-Paul's word when it comes to network programming. IMHO, we should delay ipaddr's official inclusion in the stdlib until the conceptual issues are solved. Regards Antoine. From drkjam at gmail.com Wed Jun 3 01:50:53 2009 From: drkjam at gmail.com (DrKJam) Date: Wed, 3 Jun 2009 00:50:53 +0100 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> <8657ee3f0906021525h74d28aeet614cda2e4c481d9d@mail.gmail.com> Message-ID: <538a660a0906021650p7a5bfa65j53194db40d023c86@mail.gmail.com> I've just subscribed to python-dev again after being pointed towards this thread (thanks Raymond). I'd be the first to accept failings and oddities in the interface of my own library. Since its release netaddr has taken its own interesting set of twists and turns. However, all along I have tried to be responsive and accommodating with regard to my users needs and requests for new features and bug fixes. There has been a lot of useful feedback which I have faithfully incorporated into netaddr's codebase. It would be a shame to see all the hard work go to waste on Y.A.P.I.M. :- http://code.google.com/p/netaddr/wiki/YetAnotherPythonIPModule There is a veritable graveyard of stuff out there! Some good, some not so good. The netaddr/ipaddr-py merge went awry and our projects decided to remain separate. I don't see any benefit in raking over those old coals; it's all water under the bridge. That being said, based on the passion in this thread, the decision to include ipaddr-py into the stdlib seems to be proving too hasty and contentious for some. It really might be worth taking a step back and taking heed of those concerns. Having had quite a while to think about this over the last few months, this is what I would ideally like to see. A sensible discussion from a *broad* base of network admins, sysadmins and developers using Python on the formulation of a reasonable functional specification and design for a brand new library (without the associated baggage and vested interests of any particular implementation). This would be an opportunity for us to nail our respective problems and differences while simultaneously bringing together most of the Python community behind code that WE WILL ACTUALLY USE. If this is going in the stdlib surely that is doubly important? If possible, I would particularly like to see input from Victor Stinner, the current IPy maintainer. There as some wrinkles and failings in IPy's interface and implementation but its actually not as bad as I first thought having actually spent some time implementing its interface (mostly successfully) as a facade on top of netaddr. See the netaddr repo if you are interested and *no* it is not supported code ;-) Would this actually be feasible or am I just a hopeless optimist? Should we formulate a PEP even though calls for that have so far been rejected? Possibly PEPs don't apply to libraries? Whoever overseas this would need to be someone with a fairly neutral perspective on all of this. Dave M. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrewm at object-craft.com.au Wed Jun 3 02:13:08 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Wed, 3 Jun 2009 10:13:08 +1000 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: On 03/06/2009, at 3:56 AM, Jean-Paul Calderone wrote: > On Tue, 02 Jun 2009 19:34:11 +0200, "\"Martin v. L?wis\"" > wrote: >> [snip] >> >>> You seem comfortable with these quirks, but then you're not planning >>> to write software with this library. Developers who do intend to >>> write >>> meaningful network applications do seem concerned, yet we're >>> ignored. >> >> I don't hear a public outcry - only a single complainer. > > Clay repeatedly pointed out that other people have objected to > ipaddr and > been ignored. It's really, really disappointing to see you continue > to > ignore not only them, but the repeated attempts Clay has made to point > them out. > > I don't have time to argue this issue, but I agree with essentially > everything Clay has said in this thread, and I commented about these > problems on the ticket months ago, before ipaddr was added. Indeed... "Me too" - I've been quietly concerned with these issues, but have have not said anything as Clay's postings pretty much cover it (and swine flu response is trumping all my other priorities). From guido at python.org Wed Jun 3 04:39:36 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 2 Jun 2009 19:39:36 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: Well, it sounds like the current incarnation of the ipaddr module is widely loathed, even if it also has some fans. Since it is still available as a 3rd party module, and hasn't been available in other releases except 3.1 beta and rc1, I expect few users will be inconvenienced if we withdraw it at this point. Benjamin, what would be involved in removing it? I suppose there's the module itself, some unit tests, and some docs. (I'm not asking you to remove it yet -- but I'm asking to look into the consequences, so that we can be sure to do the right thing before releasing 3.1 final.) I'm disappointed in the process -- it's as if nobody really reviewed the API until it was released with rc1, and this despite there being a significant discussion about its inclusion and alternatives months ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me in the behind, and I can honestly say that I don't know whether /8 means to look only at the first 8 bits or whether it means to mask off the last 8 bits.) I hope we can learn from this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg at krypto.org Wed Jun 3 04:41:20 2009 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 2 Jun 2009 19:41:20 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> <77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com> Message-ID: <52dc1c820906021941r7bf5ad25l1433dc9ed33ec3a4@mail.gmail.com> On Tue, Jun 2, 2009 at 1:11 PM, Jake McGuire wrote: > The minimal demonstration of the problem of representing networks and > addresses using the same class: fwiw, that (hosts vs networks in the same class) is not what you are demonstrating below. What you demonstrate is that its silly for iteration over a network to return strings rather than IP objects. Regardless, it is a good example of a problem with the API. ipaddr could be changed to yield IPv4 /32 or IPv6 /128 objects when iterating over it instead of strings and this example would work properly. >>>> container = [1, 2, 3, 4] >>>> for item in container: > ... ? print "%s in %s: %s" % (item, container, item in container) > ... > 1 in [1, 2, 3, 4]: True > 2 in [1, 2, 3, 4]: True > 3 in [1, 2, 3, 4]: True > 4 in [1, 2, 3, 4]: True >>>> import ipaddr >>>> container = ipaddr.IP('192.168.1.0/24') >>>> for item in container: > ... ? print "%s in %s: %s" % (item, container, item in container) > ... > Traceback (most recent call last): > ?File "", line 2, in > ?File "ipaddr.py", line 438, in __contains__ > ? ?return self.network <= other.ip and self.broadcast >= other.broadcast > AttributeError: 'str' object has no attribute 'ip' >>>> Regardless, after reading the many different opinions on this thread especially including those of other python developers it sounds like we should not include ipaddr in 3.1. This example is IMHO one good reason to not include ipaddr in the standard library as it stands. It sounds like we have a 3.1rc2 scheduled for June 13th. At this point based on the multitude of other developer comments in these threads and barring strong arguments in favor of including it as it stands I will remove it because there seem to be quite a number of people with objections to it as an API in its current form. I've filed http://bugs.python.org/issue6182 as a release blocker to track its removal (or not) based on any further discussion in these python-dev threads. I do believe an API for an ip and network address manipulation library can be worked out but during the 3.1 release candidate phase is not the time to hastily do that and choose one so removal due to disagreement seems the best option. Would someone volunteer write up a proposal (PEP) for such with a goal of having it completed within the next few months? (I know people have suggested various other libraries, that counts; i have not taken the time to look at them). -gps From greg at krypto.org Wed Jun 3 04:43:49 2009 From: greg at krypto.org (Gregory P. Smith) Date: Tue, 2 Jun 2009 19:43:49 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com> On Tue, Jun 2, 2009 at 7:39 PM, Guido van Rossum wrote: > > I'm disappointed in the process -- it's as if nobody really reviewed > the API until it was released with rc1, and this despite there being a > significant discussion about its inclusion and alternatives months > ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me > in the behind, and I can honestly say that I don't know whether /8 > means to look only at the first 8 bits or whether it means to mask off > the last 8 bits.) > > I hope we can learn from this. Yep, thats a fair summary. Removal will be trivial. Should it only be removed from py3k branch or also from trunk pending a decision as to if the library is reworked or if something else entirely is adopted? From aahz at pythoncraft.com Wed Jun 3 04:44:58 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 2 Jun 2009 19:44:58 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: <20090603024458.GA22308@panix.com> On Tue, Jun 02, 2009, Guido van Rossum wrote: > > I hope we can learn from this. I'm not thrilled with adding more process just because we had a problem here, and the only obvious solution I see is to require a PEP every time a module is added. Based on what I've seen of this discussion so far, I think that cure would at this time be worse than the disease. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ my-python-code-runs-5x-faster-this-month-thanks-to-dumping-$2K- on-a-new-machine-ly y'rs - tim From benjamin at python.org Wed Jun 3 04:53:36 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 2 Jun 2009 21:53:36 -0500 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: <1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com> 2009/6/2 Guido van Rossum : > Benjamin, what would be involved in removing it? I suppose there's the > module itself, some unit tests, and some docs. (I'm not asking you to > remove it yet -- but I'm asking to look into the consequences, so that > we can be sure to do the right thing before releasing 3.1 final.) As Raymond and Gregory have pointed out in this thread, the library is quite independent as it stands now in the stlib, so should be trivial to remove. Nothing else should be affected. -- Regards, Benjamin From benjamin at python.org Wed Jun 3 04:55:44 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 2 Jun 2009 21:55:44 -0500 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <20090603024458.GA22308@panix.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603024458.GA22308@panix.com> Message-ID: <1afaf6160906021955x7f0c6dd5x490f85e4b2701617@mail.gmail.com> 2009/6/2 Aahz : > On Tue, Jun 02, 2009, Guido van Rossum wrote: >> >> I hope we can learn from this. > > I'm not thrilled with adding more process just because we had a problem > here, and the only obvious solution I see is to require a PEP every time > a module is added. ?Based on what I've seen of this discussion so far, I > think that cure would at this time be worse than the disease. I don't think he's suggesting adding more process yet, just in the diagnosis phase. -- Regards, Benjamin From clay at daemons.net Wed Jun 3 04:58:09 2009 From: clay at daemons.net (Clay McClure) Date: Tue, 2 Jun 2009 22:58:09 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <52dc1c820906021941r7bf5ad25l1433dc9ed33ec3a4@mail.gmail.com> References: <2AD0D64C97C84D06AABB8C1292B5EE64@RaymondLaptop1> <4A2428F1.7030708@v.loewis.de> <77c780b40906011854t5afec054x718b2f59ee4e097c@mail.gmail.com> <77c780b40906011947x56ef79famb5b12891efe4963f@mail.gmail.com> <4A24BAE3.8070809@v.loewis.de> <8657ee3f0906012252t7dd1ccefq79b56afcd3c57e55@mail.gmail.com> <4A24C1F8.7080905@v.loewis.de> <8657ee3f0906020926n42b41837k5afeb7d8333d063b@mail.gmail.com> <77c780b40906021311y5520a7ffheb559f3f88f375f7@mail.gmail.com> <52dc1c820906021941r7bf5ad25l1433dc9ed33ec3a4@mail.gmail.com> Message-ID: <8657ee3f0906021958x6dcbf40ap3c9ba6ced09651c8@mail.gmail.com> On Tue, Jun 2, 2009 at 10:41 PM, Gregory P. Smith wrote: > ipaddr could be changed to yield IPv4 /32 or IPv6 /128 objects when > iterating over it instead of strings and this example would work > properly. I have opened an issue in the ipaddr bug tracker that I think addresses this issue. There are a few methods and properties in the ipaddr.IP classes that return native types but should return IP objects: http://code.google.com/p/ipaddr-py/issues/detail?id=21 > It sounds like we have a 3.1rc2 scheduled for June 13th. ?At this > point based on the multitude of other developer comments in these > threads and barring strong arguments in favor of including it as it > stands I will remove it because there seem to be quite a number of > people with objections to it as an API in its current form. Thank you. > Would someone volunteer write up a proposal (PEP) for such with a goal > of having it completed within the next few months? ?(I know people > have suggested various other libraries, that counts; i have not taken > the time to look at them). I am happy to take a stab at that. Thanks, Clay From benjamin at python.org Wed Jun 3 04:58:33 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 2 Jun 2009 21:58:33 -0500 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com> Message-ID: <1afaf6160906021958r623f12fbjebefa963ba639a9@mail.gmail.com> 2009/6/2 Gregory P. Smith : > Should it only be removed from py3k branch or also from trunk pending > a decision as to if the library is reworked or if something else > entirely is adopted? I think it should disappear from the whole python tree for the moment. Even the unstable trunk is not a good place for a module pending uncertain, future changes. -- Regards, Benjamin From andrewm at object-craft.com.au Wed Jun 3 06:55:49 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Wed, 3 Jun 2009 14:55:49 +1000 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: On 03/06/2009, at 12:39 PM, Guido van Rossum wrote: > I'm disappointed in the process -- it's as if nobody really reviewed > the API until it was released with rc1, and this despite there being a > significant discussion about its inclusion and alternatives months > ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me > in the behind, and I can honestly say that I don't know whether /8 > means to look only at the first 8 bits or whether it means to mask off > the last 8 bits.) > > I hope we can learn from this. When including third-party modules into the standard library, we've generally only included them after they have broad acceptance in the community. In this case, however, it seems that while the ipaddr module had acceptance within Google, it had not had much exposure to the broader python community. I think if anyone other than Guido had proposed adding the module to the standard library, we would not have even considered it until it had spent some time standing on it's own two feet. From stephen at xemacs.org Wed Jun 3 07:31:11 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 03 Jun 2009 14:31:11 +0900 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <20090603024458.GA22308@panix.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603024458.GA22308@panix.com> Message-ID: <87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp> Aahz writes: > On Tue, Jun 02, 2009, Guido van Rossum wrote: > > > > I hope we can learn from this. > > I'm not thrilled with adding more process just because we had a problem > here, and the only obvious solution I see is to require a PEP every time > a module is added. Based on what I've seen of this discussion so far, I > think that cure would at this time be worse than the disease. +1 A couple of people commented that they didn't say anything because they were really busy. I don't think there's much you can do about that, unless the time machine can be used to unmutate the swine flu! FWIW I also agree with Martin's assessment of the thread in the tracker that it looked like there was only one person strongly opposed. Mostly an unfortunate combination of circumstances. One thing I would recommend is that while inclusion is not a matter of voting, people who are recognized as domain experts on Python-Dev *should* try to add their "+1" or "-1" early. Especially if they don't expect to have time to participate actively in discussion. After all, they can always change their assessment based on either changes or as a response to a persuasive lobby, precisely because it's not a vote. From stefan_ml at behnel.de Wed Jun 3 09:08:51 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 03 Jun 2009 09:08:51 +0200 Subject: [Python-Dev] Serious regression in doctest in Py3.1rc1 Message-ID: Hi, I can't currently file a bug report on this, but I was told by Lisandro Dalc?n that there is a serious problem with the doctest module in Py3.1rc1. In Cython, we use doctests to test the compiler in that we compile a Python/Cython module with doctests into a C module and then run doctest on the imported extension module. >From the error report it seems to me that doctest is now trying to read the module itself through linecache for some reason, which horribly fails for a binary module. Could someone please look into this? I'll open up a bug report tomorrow unless someone beats me to it. Thanks, Stefan From p.f.moore at gmail.com Wed Jun 3 09:51:54 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 3 Jun 2009 08:51:54 +0100 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603024458.GA22308@panix.com> <87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <79990c6b0906030051l3aac84c6v5f68803096554d2d@mail.gmail.com> 2009/6/3 Stephen J. Turnbull : > Aahz writes: > > ?> On Tue, Jun 02, 2009, Guido van Rossum wrote: > ?> > > ?> > I hope we can learn from this. > ?> > ?> I'm not thrilled with adding more process just because we had a problem > ?> here, and the only obvious solution I see is to require a PEP every time > ?> a module is added. ?Based on what I've seen of this discussion so far, I > ?> think that cure would at this time be worse than the disease. > > +1 [...] > > One thing I would recommend is that while inclusion is not a matter of > voting, people who are recognized as domain experts on Python-Dev > *should* try to add their "+1" or "-1" early. ?Especially if they > don't expect to have time to participate actively in discussion. > > After all, they can always change their assessment based on either > changes or as a response to a persuasive lobby, precisely because it's > not a vote. FWIW, I'd add some points: 1. Publishing the documentation somewhere prominent would help. I don't know if that happened here, but it makes it a *lot* easier for people to "have a quick look". Downloading a zip file, unpacking the docs and opening a HTML file (or worse still, building it first!) can be enough of a barrier to stop people who are pressed for time from commenting. (Once the module was included, it gets into the online Python docs, hence I could read them and comment). 2. Encouraging a clear +1/-1 from people, in addition to discussion on specific points, would clarify things. I believe Martin commented that he hadn't realised that one of the opposing comments was a strong enough objection to count as a -1. 3. Discussion should happen on python-dev, not on the tracker. (Some people may object to this, I know). I saw the call for input to the tracker item, and thought "that's not a module I'm likely to need, I'll leave it to the experts" and did nothing more. When the discussion flared up on python-dev, on the other hand, I kept skimming the discussion, and when I saw something that seemed at my level, I felt encouraged to comment. Also, seeing that there *was* disagreement encouraged me to comment - if the experts aren't agreeing, maybe my non-expert view might be helpful input on usability/intuitiveness. But I agree, let's not add more process if a bit of focus in the discussion is enough. Paul. From p.f.moore at gmail.com Wed Jun 3 10:02:25 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 3 Jun 2009 09:02:25 +0100 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com> <79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com> Message-ID: <79990c6b0906030102m5549f543j8bd75592d26e82c4@mail.gmail.com> 2009/6/2 R. David Murray : > On Tue, 2 Jun 2009 at 21:02, Paul Moore wrote: >> Simple example. If I want to scan all the IP addresses on my network >> (my IP address is 192.168.1.101) I'd probably write: >> >> ? for i in range(253): >> ? ? ? ip = '192.168.1.' + str(i+1) >> ? ? ? ... >> >> - and to heck with generality. >> >> Even after reading the documentation, I've *no idea* how I would use >> the ipaddr module to do this better. Let alone in as few lines. > > ? ?net = ipaddr.IP('192.168.1.0/24'): > ? ?for i in range(253): > ? ? ? ?ip = net[i] > ? ? ? ?... > > So, that's one example that needs to be added to the docs. > > I'd have liked to write that as: > > ? ?for ip in ipaddr.IP('192.168.1.0/24')[:253]: > ? ? ? ?... > > but apparently it doesn't support slicing (time for an RFE :) Given that what I *want* to do is to skip the "special" cases of 0 and 255, would the following work? net = ipaddr.IP('192.168.1.101/255.255.255.0') # Note 1 for ip in net: if ip.ip = ip.broadcast or ip.is_multicast(): continue ... That would be what I mean by the module helping me to avoid "gotchas" - like assuming 0 and 255 are the "special" multicast and broadcast (or whatever they are) addresses that I shouldn't be testing in my port scanner. I'd like a .is_broadcast() method. Does that expose my lack of understanding, or is it a sensible thing to add? Note 1 - by the way, I use this form because I don't understand how the /24 notation works. I can get the subnet mask from ipconfig, so I use that. Ideally, I'd rather just put my IP address and have the module work out the "obvious" subnet (at my level of use, it's always 255.255.255.0 for 192.168 addresses) but I guess that's not actually a well-defined idea. Paul. From mike at pennington.net Wed Jun 3 10:42:47 2009 From: mike at pennington.net (Mike Pennington) Date: Wed, 03 Jun 2009 03:42:47 -0500 Subject: [Python-Dev] Issues with Py3.1's new ipaddr Message-ID: <4A263787.5010203@pennington.net> Raymond solicited a comment from me about the design of ipaddr. By way of full disclosure, I have a small competing project called pynet. That said, I test drove ipaddr for about 30 minutes and so far like the big-picture API design quite a bit. I'll specifically address Clay's concern about hosts vs networks, because this issue is important to me; I've been in the network engineering field for over 15 years, worked on Cisco's product development team, and held a CCIE (consider it the equivalent of a CPA for network engineers) for 10 years... Clay seems to object to ipaddr's IP object because it is not the same as the object model used in the BSD ip stack. Indeed, I'm one of the raving fans of what BSD has done for the quality of ip networking, but let's also consider their requirements. BSD must approach ip networking from a host perspective, it is the consumer of individual IP packets and their payloads. ipaddr's whole point of existence is really driven towards the manipulation of potentially massive lists of ip addresses. This is no small difference in requirements, and I believe ipaddr's different approach makes their code much simpler for the tasks it needs to do. Incorporating host addresses as a special case of a /32 IPv4 network or /128 IPv6 network makes a lot of sense to me, in fact, I also chose this same object model. Perl's NetAddr::IP does this too, it is considered the gold standard for perl's address manipulation module. Whether python includes ipaddr now, later, or uses another module entirely does not bother me. Whatever is included should have a very stable API, and major bugs should be worked out. Documentation should be good enough for the average consumer, and if anything this is where ipaddr to be lacking a bit. I hope that python does include something to manipulate IPv4 and IPv6 address blocks in the future, since this is a big hole is python's batteries-included philosophy. However, I'd need more time at the wheel of ipaddr before I could comment whether this truly is ready for inclusion in stdlib. All the best, \m From amauryfa at gmail.com Wed Jun 3 13:26:13 2009 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 3 Jun 2009 13:26:13 +0200 Subject: [Python-Dev] Serious regression in doctest in Py3.1rc1 In-Reply-To: References: Message-ID: Hello, 2009/6/3 Stefan Behnel : > Hi, > > I can't currently file a bug report on this, but I was told by Lisandro > Dalc?n that there is a serious problem with the doctest module in Py3.1rc1. > In Cython, we use doctests to test the compiler in that we compile a > Python/Cython module with doctests into a C module and then run doctest on > the imported extension module. > > >From the error report it seems to me that doctest is now trying to read the > module itself through linecache for some reason, which horribly fails for a > binary module. > > Could someone please look into this? I'll open up a bug report tomorrow > unless someone beats me to it. I don't have the time either, but the problem looks very similar to http://bugs.python.org/issue4050 The fix was to replace: file = inspect.getsourcefile(object) or inspect.getfile(object) was replaced with file = inspect.getsourcefile(object) -- Amaury Forgeot d'Arc From barry at python.org Wed Jun 3 14:31:50 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 3 Jun 2009 08:31:50 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <52dc1c820906021943o390bffefk74cbcbc1e760b387@mail.gmail.com> Message-ID: On Jun 2, 2009, at 10:43 PM, Gregory P. Smith wrote: > Should it only be removed from py3k branch or also from trunk pending > a decision as to if the library is reworked or if something else > entirely is adopted? I think it should be removed from trunk if it's removed from the py3k branch. Nothing goes into Python 2.7 that isn't already in Python 3.1. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From barry at python.org Wed Jun 3 14:37:00 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 3 Jun 2009 08:37:00 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: <15494D8B-AEF9-4204-9B85-19AD23325F95@python.org> On Jun 2, 2009, at 10:39 PM, Guido van Rossum wrote: > I hope we can learn from this. One crazy thought: let's use the Cheeseshop. When I search for 'ipaddr' I get three hits, with Google's module at the top with a score of '8'. I really don't know what that means but I'm guessing it means that module is "two times better" than the next highest score of 4 for ipaddresslib. It would be really nice if say the Cheeseshop had a voting feature. Use PEP 10 voting to get a rough estimate of a module's popularity (download counts alone might not tell you everything). Then at least you can get a rough idea of how generally popular a module is in the wider community. Also, a module should have to live on its own two feet for while on Cheeseshop before being considered for inclusion in the stdlib. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From ncoghlan at gmail.com Wed Jun 3 14:49:18 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 03 Jun 2009 22:49:18 +1000 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <79990c6b0906030102m5549f543j8bd75592d26e82c4@mail.gmail.com> References: <9418DB6C0B9D434190E54A78E931C3D1097E0E69@XCH-NW-7V1.nw.nos.boeing.com> <79990c6b0906021302u8986b6fq6cdd4acb136947f9@mail.gmail.com> <79990c6b0906030102m5549f543j8bd75592d26e82c4@mail.gmail.com> Message-ID: <4A26714E.2000209@gmail.com> Paul Moore wrote: > Note 1 - by the way, I use this form because I don't understand how > the /24 notation works. I can get the subnet mask from ipconfig, so I > use that. It's just a shorthand way of writing IPv4 net masks based on their binary form: /8 = 8 ones followed by 24 zeroes = 255.0.0.0 /16 = 16 ones followed by 16 zeroes = 255.255.0.0 /24 = 24 ones followed by 8 zeroes = 255.255.255.0 /30 = 30 ones followed by 2 zeroes = 255.255.255.252 /32 = 32 ones followed by no zeroes = 255.255.255.255 It's particularly convenient when you're dividing subnets up into chunks that don't align with a byte boundary in the IPv4 address (e.g. /27 can easily be recognised as giving a subnet containing 2**5 = 32 hosts, but the subnet size is significantly less obvious when written using the equivalent 255.255.255.224 netmask). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From davide.alberani at gmail.com Wed Jun 3 16:35:39 2009 From: davide.alberani at gmail.com (Davide Alberani) Date: Wed, 3 Jun 2009 16:35:39 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <538a660a0906021650p7a5bfa65j53194db40d023c86@mail.gmail.com> Message-ID: <20090603143539.GA3657@gmail.com> > I've just subscribed to python-dev again after being pointed towards > this thread (thanks Raymond). The same for me, thanks. :-) I'm the author of IPLib [1]; I don't consider myself an expert on the subject and my code dates back to Python 1.6 times (last updated in 2005). Moreover, while it works for me and its almost non-existent user base, I assume it can't be used for anything but as a source of ideas. > There is a veritable graveyard of stuff out there! Some good, > some not so good. The fact is that most of the times a programmer writes yet another Python IP module to cover only a very limited and usually simple aspect of IP manipulation that he needs at the given time (a conversion to/from decimal, a check for inclusion in a CIDR, ...) > A sensible discussion from a *broad* base of network admins, > sysadmins and developers using Python on the formulation of a > reasonable functional specification and design for a brand new > library To me, this makes a lot of sense: check which are the most used modules and ask the users. Generically speaking, I tend to agree with Clay, as I always had looked at IP addresses, netmasks and CIDR blocks as separate concepts, but again: I'm not an expert. +++ [1] http://erlug.linux.it/~da/soft/iplib/ Only supports IPv4; its main use is to convert amongst notations, but can be used to check if an IP (or another subnet) is included in a CIDR block and to gather some basic information about a CIDR. -- Davide Alberani [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ From rdmurray at bitdance.com Wed Jun 3 17:16:01 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 3 Jun 2009 11:16:01 -0400 (EDT) Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <4A263787.5010203@pennington.net> References: <4A263787.5010203@pennington.net> Message-ID: On Wed, 3 Jun 2009 at 03:42, Mike Pennington wrote: > That said, I test drove ipaddr for about 30 minutes and so far like the > big-picture API design quite a bit. I'll specifically address Clay's concern > about hosts vs networks, because this issue is important to me; I've been in > the network engineering field for over 15 years, worked on Cisco's product > development team, and held a CCIE (consider it the equivalent of a CPA for > network engineers) for 10 years... > > Clay seems to object to ipaddr's IP object because it is not the same as the > object model used in the BSD ip stack. Indeed, I'm one of the raving fans of > what BSD has done for the quality of ip networking, but let's also consider > their requirements. BSD must approach ip networking from a host perspective, > it is the consumer of individual IP packets and their payloads. ipaddr's > whole point of existence is really driven towards the manipulation of > potentially massive lists of ip addresses. This is no small difference in > requirements, and I believe ipaddr's different approach makes their code much > simpler for the tasks it needs to do. Incorporating host addresses as a > special case of a /32 IPv4 network or /128 IPv6 network makes a lot of sense > to me, in fact, I also chose this same object model. Perl's NetAddr::IP does > this too, it is considered the gold standard for perl's address manipulation > module. I think this hits the nail on the head. Rather than network engineers having a less precise understanding of IP, what we have is two different sets of domain requirements. Network engineers deal with networks, with IPs being a necessary special case. Others deal with host addresses, with networks as an additional data type. Both approaches are valid, but lead to different design decisions. I don't see any reason why both needs cannot be met by a common API, but I'm wondering if any existing package is going to incorporate both approaches satisfactorily. As another poster said, each package that gets written solves the problems that the particular author(s) needed solved. Since it seems clear that ipaddr does not address the needs of the "other half" of the user base, and that its API considered on its own does have design flaws that would be difficult to fix in a backward compatible fashion (eg: returning strings from __getitem__), I withdraw my support for keeping it in 3.1. --David From janssen at parc.com Wed Jun 3 17:31:07 2009 From: janssen at parc.com (Bill Janssen) Date: Wed, 3 Jun 2009 08:31:07 PDT Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <4A263787.5010203@pennington.net> Message-ID: <31390.1244043067@parc.com> R. David Murray wrote: > Both approaches are valid, but lead to different design decisions. > I don't see any reason why both needs cannot be met by a common API, > but I'm wondering if any existing package is going to incorporate both > approaches satisfactorily. As another poster said, each package that gets > written solves the problems that the particular author(s) needed solved. I wonder if part of the problem is the name of the module. Just from "ipaddr", I'd expect it to deal with host addresses (what I think of as an IP address) and would probably approach its use with the wrong expectations. I could see frustration and irritation following from that. If the module was called "networks" instead of "ipaddr", it might help. Bill From nas at arctrix.com Wed Jun 3 19:13:19 2009 From: nas at arctrix.com (Neil Schemenauer) Date: Wed, 3 Jun 2009 17:13:19 +0000 (UTC) Subject: [Python-Dev] Issues with Py3.1's new ipaddr References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <15494D8B-AEF9-4204-9B85-19AD23325F95@python.org> Message-ID: Barry Warsaw wrote: > It would be really nice if say the Cheeseshop had a voting feature. > Use PEP 10 voting to get a rough estimate of a module's popularity > (download counts alone might not tell you everything). Then at least > you can get a rough idea of how generally popular a module is in the > wider community. Also, a module should have to live on its own two > feet for while on Cheeseshop before being considered for inclusion in > the stdlib. Better yet would be something like Debian's popularity-contest mechanism (it provides opt-in voting of packages based on what is installed on your machine). popularity-contest runs from a cron job. Maybe when Python is installed it could ask if you want to submit package statistics. If so, installing a package with distutils would submit the name and version number to a central server. If we knew which batteries were most popular we could make sure they are included in the package. ;-) Neil From guido at python.org Wed Jun 3 19:19:25 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 3 Jun 2009 10:19:25 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <15494D8B-AEF9-4204-9B85-19AD23325F95@python.org> Message-ID: On Wed, Jun 3, 2009 at 10:13 AM, Neil Schemenauer wrote: > Barry Warsaw wrote: >> It would be really nice if say the Cheeseshop had a voting feature. >> Use PEP 10 voting to get a rough estimate of a module's popularity >> (download counts alone might not tell you everything). ?Then at least >> you can get a rough idea of how generally popular a module is in the >> wider community. ?Also, a module should have to live on its own two >> feet for while on Cheeseshop before being considered for inclusion in >> the stdlib. > > Better yet would be something like Debian's popularity-contest > mechanism (it provides opt-in voting of packages based on what is > installed on your machine). ?popularity-contest runs from a cron > job. ?Maybe when Python is installed it could ask if you want to > submit package statistics. ?If so, installing a package with > distutils would submit the name and version number to a central > server. > > If we knew which batteries were most popular we could make sure they > are included in the package. ;-) Whoa. Are you all suddenly trying to turn Python into a democracy? I'm outta here if that ever happens (even if I were voted BDFL by a majority :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From glyph at divmod.com Wed Jun 3 19:41:51 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 03 Jun 2009 17:41:51 -0000 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> Message-ID: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> On 02:39 am, guido at python.org wrote: >I'm disappointed in the process -- it's as if nobody really reviewed >the API until it was released with rc1, and this despite there being a >significant discussion about its inclusion and alternatives months >ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me >in the behind, and I can honestly say that I don't know whether /8 >means to look only at the first 8 bits or whether it means to mask off >the last 8 bits.) > >I hope we can learn from this. As he pointed out to Martin, Jean-Paul voiced objections several months ago which are similar to the ones which are now being discussed. To be fair, he didn't unambiguously say "... and therefore don't include this library"; he simply suggested that netaddr was superior in some ways and that perhaps some documentation could illuminate why ipaddr was better. I've been frustrated with similar aspects of Python's development process in the past. The biggest problem I can see is that it's too hard to follow the discussion, and catch oneself up on the discussion thus far. It's also difficult to refer back to posts much earlier in the history of an email discussion, and that frequently needs to be done when someone jumps into the middle of a discussion. The way Twisted dealt with this particular issue was to move *all* discussions relevant to a particular feature into the ticket for that feature. If discussion starts up on the mailing list, within a few messages of it starting, someone on the dev team will pipe up and say "Hey, here's the ticket for this, could you add a link to this discussion and a summary". Once on a ticket, the phraseology and typesetting used by our core team has reached a very precise consensus. Like the feature? "Merge this patch" or "Land this branch". Don't like it? "Thanks for your patch, but:", followed by a list of enumerated feedback. The required reactions to such feedback are equally well understood. Even if a comment isn't a full, formal code review, it still follows a similar style. This system is possibly too simplistic for the more wide-ranging development of Python; although Twisted has its share of enthusiastic discussions, there is rarely the level of controversy I see here on python-dev, so I can't recommend it as such. I can say that keeping ticket discussions on tickets is generally a good idea, though, especially in a system like roundup or trac where it's easy to say "see message 7" rather than repeating yourself. It seems that there is a habit of occasionally using ASF-style +1/+0/-0/-1 markers, but it's inconsistently applied. More importantly, nobody ever actually adds them up, so it's not entirely clear when they should be used. To go back to JP's original comments though: what was the right thing for him to do, back in January, when he had these concerns? Should he have said "I am therefore -1 on this inclusion"? Should he have been discussing this on the mailing list rather than the tracker? Should he have kept coming back to the ticket and answering every single message reinforcing his original conclusions? I honestly don't think it's very clear what one is "officially" supposed to do. Without a clear expectation that one should say "No" to features that are problematic, it seems gratuitously mean to do so; so, it's nicer to just say "here's what I found" with the hopes that someone will evaluate that feedback. Unfortunately it seems like the winning strategy here is just to keep flogging a dead horse until it's a dead horse hamburger; reply and reply and reply until everybody gets sick of talking about it. Repeat your original points in every post so that nobody will miss them. I think everyone is ill-served by this discussion format. Certainly when I voice my own objections or support for something, I'd like to just stop by, add a note for the committers to take into account when considering the issue, and then go away. So, here are my recommendations: 1. Use the tracker for discussing tickets, so that it's easy to refer back to a previous point in the discussion, and so that people working on those tickets can easily find your commentary. 2. Use the mailing list for drawing attention to these discussions if they are of general interest, especially if the discussion is time- critical. In this case, an announcement "You have six weeks to review ipaddr now until its inclusion is permanent, anyone interested please look at issue 3959." 3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at the top of your message, so that it's easy for newcomers to the discussion to get a general feel. Of course, this won't prevent all meandering discussions, or discussions that are too late to the party, but I do think it will help. However, I think before everyone just starts doing this, even *more* important is my meta-suggestion: let's agree on how and when feedback is useful, and when it will be considered, so that it's not just a contest to overflow Guido's inbox. The opinion of the committers who will be considering feedback is much more important than mine :). From glyph at divmod.com Wed Jun 3 19:45:07 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 03 Jun 2009 17:45:07 -0000 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <20090603024458.GA22308@panix.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603024458.GA22308@panix.com> Message-ID: <20090603174507.12555.1277667142.divmod.xquotient.12295@weber.divmod.com> On 02:44 am, aahz at pythoncraft.com wrote: >On Tue, Jun 02, 2009, Guido van Rossum wrote: >> >>I hope we can learn from this. > >I'm not thrilled with adding more process just because we had a problem >here, and the only obvious solution I see is to require a PEP every >time >a module is added. Based on what I've seen of this discussion so far, >I >think that cure would at this time be worse than the disease. I thought the solution that I just proposed was pretty obvious ;-). But in all seriousness, even if an improvement looks nothing like what I just proposed, it seems like a simple failure of imagination to say that nothing could make this situation better. From glyph at divmod.com Wed Jun 3 19:51:35 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 03 Jun 2009 17:51:35 -0000 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <79990c6b0906030051l3aac84c6v5f68803096554d2d@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603024458.GA22308@panix.com> <87ws7thoio.fsf@uwakimon.sk.tsukuba.ac.jp> <79990c6b0906030051l3aac84c6v5f68803096554d2d@mail.gmail.com> Message-ID: <20090603175135.12555.1469649106.divmod.xquotient.12300@weber.divmod.com> On 07:51 am, p.f.moore at gmail.com wrote: >2009/6/3 Stephen J. Turnbull : >>One thing I would recommend is that while inclusion is not a matter of >>voting, people who are recognized as domain experts on Python-Dev >>*should* try to add their "+1" or "-1" early. ?Especially if they >>don't expect to have time to participate actively in discussion. >2. Encouraging a clear +1/-1 from people, in addition to discussion on >specific points, would clarify things. I believe Martin commented that >he hadn't realised that one of the opposing comments was a strong >enough objection to count as a -1. +1 :-) From p.f.moore at gmail.com Wed Jun 3 19:42:48 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 3 Jun 2009 18:42:48 +0100 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> Message-ID: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> 2009/6/3 : > So, here are my recommendations: > > ?1. Use the tracker for discussing tickets, so that it's easy to refer back > to a previous point in the discussion, and so that people working on those > tickets can easily find your commentary. > ?2. Use the mailing list for drawing attention to these discussions if they > are of general interest, especially if the discussion is time- critical. ?In > this case, an announcement "You have six weeks to review ipaddr now until > its inclusion is permanent, anyone interested please look at issue 3959." > ?3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at the > top of your message, so that it's easy for newcomers to the discussion to > get a general feel. Mostly, I agree, but I definitely disagree, I'm afraid, on the use of the tracker for discussions. To keep track of discussions on a ticket, I have to personally keep a list of the tickets I'm interested in, check back regularly to see if there's anything new, and keep a mental note of where I've read up to so I know what's new. RSS would make this simpler, certainly, but I'm not sure about how I'd use it (it's not how I currently use RSS, so I'd have to mess round with my current setup to make it appropriate). Email is delivered to me by default - I get anything new in my python-dev folder, and I can skip or read the discussion as I choose. I don't have to take action just to monitor things. (In other words, the default is for people to see the discussions, rather than the other way around. Paul. From glyph at divmod.com Wed Jun 3 20:04:53 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 03 Jun 2009 18:04:53 -0000 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <15494D8B-AEF9-4204-9B85-19AD23325F95@python.org> Message-ID: <20090603180453.12555.597041055.divmod.xquotient.12306@weber.divmod.com> On 05:19 pm, guido at python.org wrote: >On Wed, Jun 3, 2009 at 10:13 AM, Neil Schemenauer >wrote: >>Barry Warsaw wrote: >>>It would be really nice if say the Cheeseshop had a voting feature. >>>Use PEP 10 voting to get a rough estimate of a module's popularity >>>(download counts alone might not tell you everything). ?Then at least >>>you can get a rough idea of how generally popular a module is in the >>>wider community. ?Also, a module should have to live on its own two >>>feet for while on Cheeseshop before being considered for inclusion in >>>the stdlib. >> >>Better yet would be something like Debian's popularity-contest >>mechanism (it provides opt-in voting of packages based on what is >>installed on your machine). ?popularity-contest runs from a cron >>job. ?Maybe when Python is installed it could ask if you want to >>submit package statistics. ?If so, installing a package with >>distutils would submit the name and version number to a central >>server. >> >>If we knew which batteries were most popular we could make sure they >>are included in the package. ;-) > >Whoa. Are you all suddenly trying to turn Python into a democracy? I'm >outta here if that ever happens (even if I were voted BDFL by a >majority :-). I'm sure that what Barry and Neil are recommending is the same as what I did later: a way to give our most beloved regent accurate data on the populace's current mood. Certainly we wouldn't be discussing our plans for a democratic coup out in the open like this! Clearly, that would be foolish ;-). From glyph at divmod.com Wed Jun 3 20:12:36 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Wed, 03 Jun 2009 18:12:36 -0000 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> Message-ID: <20090603181236.12555.1355794514.divmod.xquotient.12321@weber.divmod.com> On 05:42 pm, p.f.moore at gmail.com wrote: >2009/6/3 : >>So, here are my recommendations: >> >>?1. Use the tracker for discussing tickets, so that it's easy to refer >>back >>to a previous point in the discussion, and so that people working on >>those >>tickets can easily find your commentary. >>?2. Use the mailing list for drawing attention to these discussions if >>they >>are of general interest, especially if the discussion is time- >>critical. ?In >>this case, an announcement "You have six weeks to review ipaddr now >>until >>its inclusion is permanent, anyone interested please look at issue >>3959." >>?3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself >>at the >>top of your message, so that it's easy for newcomers to the discussion >>to >>get a general feel. > >Mostly, I agree, but I definitely disagree, I'm afraid, on the use of >the tracker for discussions. To keep track of discussions on a ticket, >I have to personally keep a list of the tickets I'm interested in, >check back regularly to see if there's anything new, and keep a mental >note of where I've read up to so I know what's new. RSS would make >this simpler, certainly, but I'm not sure about how I'd use it (it's >not how I currently use RSS, so I'd have to mess round with my current >setup to make it appropriate). > >Email is delivered to me by default - I get anything new in my >python-dev folder, and I can skip or read the discussion as I choose. >I don't have to take action just to monitor things. (In other words, >the default is for people to see the discussions, rather than the >other way around. A good point, but there are a couple of technical solutions to this problem, which, according to http://wiki.python.org/moin/TrackerDocs/, have already been implemented. If you want to get email about new issues, subscribe to new-bugs- announce at mail.python.org. If you want to know about every message on every issue, subscribe to python-bugs-list at mail.python.org. But, frankly, I think it's a bad idea to subscribe to python-bugs-list for announcements. The whole point here is that there is simply too much going on in python development for anyone to reasonably keep track of at a low level. Guido himself has complained on numerous occasions of being too busy to monitor things closely. A better model is to subscribe to new-bugs-announce and selectively pay attention to the bugs which are interesting to you; and, when a discussion you're involved in gets interesting and becomes of more general interest, raise it on python-dev. (On the other hand, if you want to subscribe to get your own personal searchable archive, then by all means.) From python at rcn.com Wed Jun 3 19:39:26 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 3 Jun 2009 10:39:26 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <15494D8B-AEF9-4204-9B85-19AD23325F95@python.org> Message-ID: [GvR] > Whoa. Are you all suddenly trying to turn Python into a democracy? Arthur: I am your king! Woman: Well I didn't vote for you! Arthur: You don't vote for kings. Woman: Well how'd you become king then? [Angelic music plays...] Arthur: The Lady of the Lake, her arm clad in the purest shimmering silmite held aloft Excalibur from the bosom of the water, signifying by divine providence that I, Arthur, was to carry Excalibur. THAT is why I am your king! Dennis interrupting: Listen, strange women lyin' in ponds distributin' swords is no basis for a system of government! Supreme executive power derives from a mandate from the masses, not from some farcical aquatic ceremony! ----------------------- Dennis: Oh, but you can't expect to wield supreme executive power just because some watery tart threw a sword at you! ----------------------- Dennis: Oh but if I went 'round sayin' I was Emperor, just because some moistened bint lobbed a scimitar at me, they'd put me away! ----------------------- Dennis: Help! Help! I'm being oppressed! Violence inherent in the system! Violence inherent in the system! From clay at daemons.net Wed Jun 3 20:25:28 2009 From: clay at daemons.net (Clay McClure) Date: Wed, 3 Jun 2009 14:25:28 -0400 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: References: <4A263787.5010203@pennington.net> Message-ID: <8657ee3f0906031125v31d2c10ev25b64675dea9fc13@mail.gmail.com> On Wed, Jun 3, 2009 at 11:16 AM, R. David Murray wrote: > I think this hits the nail on the head. ?Rather than network engineers > having a less precise understanding of IP, what we have is two different > sets of domain requirements. ?Network engineers deal with networks, with > IPs being a necessary special case. ?Others deal with host addresses, > with networks as an additional data type. It has been brought to my attention that my use of the word "imprecise" may have been construed as an insult. For that I apologize. That was not my intent. I am sure that your understanding of IP in your domain (network engineering) is as good or better than mine in my domain (UNIX administration). That is not the point I was arguing. Had you said that thinking of addresses as having netmasks is a useful mental model, I would have agreed with you wholeheartedly. To me, this is similar to thinking of voltage as pressure. Instead, you said (or at least implied) that addresses in fact do have netmasks, which I think is technically an imprecise way of describing how the technology works. To me, that would be like building a voltmeter calibrated in pascals. As another poster has commented, I think the name of the module is the source of some confusion. While I see the validity of your use case, that is not the use case I had in mind for a module named "ipaddr". In any case, I think with some enhanced documentation and maybe some class name changes, we can clarify ipaddr's API and strive to make it support both use cases. Your continued input will be invaluable as we move forward with the PEP. Clay From mcguire at google.com Wed Jun 3 20:05:36 2009 From: mcguire at google.com (Jake McGuire) Date: Wed, 3 Jun 2009 11:05:36 -0700 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> Message-ID: <77c780b40906031105o63b9131n3609d29a5bc7f189@mail.gmail.com> On Wed, Jun 3, 2009 at 10:41 AM, wrote: > > On 02:39 am, guido at python.org wrote: >> >> I'm disappointed in the process -- it's as if nobody really reviewed >> the API until it was released with rc1, and this despite there being a >> significant discussion about its inclusion and alternatives months >> ago. (Don't look at me -- I wouldn't recognize a netmask if it bit me >> in the behind, and I can honestly say that I don't know whether /8 >> means to look only at the first 8 bits or whether it means to mask off >> the last 8 bits.) >> >> I hope we can learn from this. > > As he pointed out to Martin, Jean-Paul voiced objections several months ago > which are similar to the ones which are now being discussed. ?To be fair, he > didn't unambiguously say "... and therefore don't include this library"; he > simply suggested that netaddr was superior in some ways and that perhaps > some documentation could illuminate why ipaddr was better. The thing that stands out about the earlier tracker/mailing list discussions is how very few people affirmatively wanted ipaddr added to the standard library. Most people thought it sounded ok in principle, didn't care, or thought it was not a great idea but didn't feel like arguing about it. -jake From nad at acm.org Wed Jun 3 20:53:31 2009 From: nad at acm.org (Ned Deily) Date: Wed, 03 Jun 2009 11:53:31 -0700 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <20090603181236.12555.1355794514.divmod.xquotient.12321@weber.divmod.com> Message-ID: In article <20090603181236.12555.1355794514.divmod.xquotient.12321 at weber.divmod.com >glyph at divmod.com wrote: > If you want to get email about new issues, subscribe to new-bugs- > announce at mail.python.org. If you want to know about every message on > every issue, subscribe to python-bugs-list at mail.python.org. > > But, frankly, I think it's a bad idea to subscribe to python-bugs-list > for announcements. The whole point here is that there is simply too > much going on in python development for anyone to reasonably keep track > of at a low level. Guido himself has complained on numerous occasions > of being too busy to monitor things closely. A better model is to > subscribe to new-bugs-announce and selectively pay attention to the bugs > which are interesting to you; and, when a discussion you're involved in > gets interesting and becomes of more general interest, raise it on > python-dev. Another option: if you are more comfortable with managing information flow via usenet newsgroups than via email lists, gmane.org provides mail-to-nntp gateways and archiving of most of the major python mailing lists, including this one, the bugs list, and code checkins. It also supports posting via your news reader, rss feeds, searching, and several web interfaces. http://dir.gmane.org/gmane.comp.python.bugs http://dir.gmane.org/gmane.comp.python.cvs http://dir.gmane.org/gmane.comp.python.devel http://dir.gmane.org/index.php?prefix=gmane.comp.python -- Ned Deily, nad at acm.org From fuzzyman at voidspace.org.uk Wed Jun 3 20:56:44 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 03 Jun 2009 19:56:44 +0100 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> Message-ID: <4A26C76C.9020506@voidspace.org.uk> Paul Moore wrote: > 2009/6/3 : > >> So, here are my recommendations: >> >> 1. Use the tracker for discussing tickets, so that it's easy to refer back >> to a previous point in the discussion, and so that people working on those >> tickets can easily find your commentary. >> 2. Use the mailing list for drawing attention to these discussions if they >> are of general interest, especially if the discussion is time- critical. In >> this case, an announcement "You have six weeks to review ipaddr now until >> its inclusion is permanent, anyone interested please look at issue 3959." >> 3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at the >> top of your message, so that it's easy for newcomers to the discussion to >> get a general feel. >> > > Mostly, I agree, but I definitely disagree, I'm afraid, on the use of > the tracker for discussions. To keep track of discussions on a ticket, > I have to personally keep a list of the tickets I'm interested in, > check back regularly to see if there's anything new, Not true - if you are added as nosy on a tracker item (which happens when you make a comment or you can do yourself) then you get emailed about new comments. The email contains the body of the comment so you can follow discussions completely by email only going to the tracker to add responses. Michael > and keep a mental > note of where I've read up to so I know what's new. RSS would make > this simpler, certainly, but I'm not sure about how I'd use it (it's > not how I currently use RSS, so I'd have to mess round with my current > setup to make it appropriate). > > Email is delivered to me by default - I get anything new in my > python-dev folder, and I can skip or read the discussion as I choose. > I don't have to take action just to monitor things. (In other words, > the default is for people to see the discussions, rather than the > other way around. > > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From martin at v.loewis.de Wed Jun 3 21:08:07 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 03 Jun 2009 21:08:07 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> Message-ID: <4A26CA17.901@v.loewis.de> > To go back to JP's original comments though: what was the right thing > for him to do, back in January, when he had these concerns? Should he > have said "I am therefore -1 on this inclusion"? Should he have been > discussing this on the mailing list rather than the tracker? Should he > have kept coming back to the ticket and answering every single message > reinforcing his original conclusions? I honestly don't think it's very > clear what one is "officially" supposed to do. To me, it's fairly clear: what the committer needs to get is guidance in any action to take. In most cases, the set of possible actions comes down to three: a) reject-as-is b) commit-as-is c) commit-with-changes (specify changes to make) [d) take no action at this point, until certain preconditions are met] For d), it is common to request, to the submitter, resubmit-with-changes, then the code needs to be reevaluated when the submitter claims to have implemented the requested changes. In the specific case, JP didn't propose an action to take, hence it wasn't clear (to me) whom his comment was directed to; I understood it as "the module has these minor flaws, they should be fixed at some point", which means "commit, then change later". This is what happened. Regards, Martin From martin at v.loewis.de Wed Jun 3 21:12:49 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 03 Jun 2009 21:12:49 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <77c780b40906031105o63b9131n3609d29a5bc7f189@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <77c780b40906031105o63b9131n3609d29a5bc7f189@mail.gmail.com> Message-ID: <4A26CB31.20804@v.loewis.de> > The thing that stands out about the earlier tracker/mailing list > discussions is how very few people affirmatively wanted ipaddr added > to the standard library. Most people thought it sounded ok in > principle, didn't care, or thought it was not a great idea but didn't > feel like arguing about it. I specifically thought "manipulating IP addresses is a frequent task, and probably everybody's requirements will be different. So starting with this library is as good as starting with any other - we can add use cases as we go". I was in favor of ipaddr because his author offered to maintain it. I then didn't have the time to review it myself, and was happy that others picked it up. Regards, Martin From jnoller at gmail.com Wed Jun 3 21:08:33 2009 From: jnoller at gmail.com (Jesse Noller) Date: Wed, 3 Jun 2009 15:08:33 -0400 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A26C76C.9020506@voidspace.org.uk> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A26C76C.9020506@voidspace.org.uk> Message-ID: <4222a8490906031208j1ee7cea6p404eec07457c1497@mail.gmail.com> On Wed, Jun 3, 2009 at 2:56 PM, Michael Foord wrote: > Paul Moore wrote: >> >> 2009/6/3 ?: >> >>> >>> So, here are my recommendations: >>> >>> ?1. Use the tracker for discussing tickets, so that it's easy to refer >>> back >>> to a previous point in the discussion, and so that people working on >>> those >>> tickets can easily find your commentary. >>> ?2. Use the mailing list for drawing attention to these discussions if >>> they >>> are of general interest, especially if the discussion is time- critical. >>> ?In >>> this case, an announcement "You have six weeks to review ipaddr now until >>> its inclusion is permanent, anyone interested please look at issue 3959." >>> ?3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at >>> the >>> top of your message, so that it's easy for newcomers to the discussion to >>> get a general feel. >>> >> >> Mostly, I agree, but I definitely disagree, I'm afraid, on the use of >> the tracker for discussions. To keep track of discussions on a ticket, >> I have to personally keep a list of the tickets I'm interested in, >> check back regularly to see if there's anything new, > > Not true - if you are added as nosy on a tracker item (which happens when > you make a comment or you can do yourself) then you get emailed about new > comments. The email contains the body of the comment so you can follow > discussions completely by email only going to the tracker to add responses. > > Michael > You can also directly reply to tracker issues via email, which is how I do most of my responses. It's cool. From martin at v.loewis.de Wed Jun 3 21:15:18 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 03 Jun 2009 21:15:18 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A26C76C.9020506@voidspace.org.uk> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A26C76C.9020506@voidspace.org.uk> Message-ID: <4A26CBC6.6000207@v.loewis.de> > Not true - if you are added as nosy on a tracker item (which happens > when you make a comment or you can do yourself) then you get emailed > about new comments. The email contains the body of the comment so you > can follow discussions completely by email only going to the tracker to > add responses. Actually, one of roundup's big advantages over many competitors is that you can also respond by email; many contributors actually do that. You only have to remember that this isn't really email, so you can usually omit salute and signature. Regards, Martin From tjreedy at udel.edu Wed Jun 3 22:37:19 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 03 Jun 2009 16:37:19 -0400 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> Message-ID: glyph at divmod.com wrote: > So, here are my recommendations: > > 1. Use the tracker for discussing tickets, so that it's easy to refer > back to a previous point in the discussion, and so that people working > on those tickets can easily find your commentary. > 2. Use the mailing list for drawing attention to these discussions if > they are of general interest, especially if the discussion is time- > critical. In this case, an announcement "You have six weeks to review > ipaddr now until its inclusion is permanent, anyone interested please > look at issue 3959." > 3. If you have an opinion, put your +1/+0/-0/-1 on a line by itself at > the top of your message, so that it's easy for newcomers to the > discussion to get a general feel. I watched and was greatly impressed by the video demo of Google's new Wave collaborative communication system. I believe it would/will help with some of the chronic problems we (and others) have. Someone already added a click-on Yes/No/Maybe client-side widget (for group outings) that tabulates on the wave who made each vote, with votes changeable. From solipsis at pitrou.net Wed Jun 3 22:40:55 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 3 Jun 2009 20:40:55 +0000 (UTC) Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> Message-ID: Terry Reedy udel.edu> writes: > > I watched and was greatly impressed by the video demo of Google's new > Wave collaborative communication system. I believe it would/will help > with some of the chronic problems we (and others) have. I really don't think technical systems are an answer to social issues. It's a flaw of the engineering mindset. (even when there's "Google" engraved on it ;-)) cheers Antoine. From g.brandl at gmx.net Wed Jun 3 23:08:20 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 03 Jun 2009 23:08:20 +0200 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > 2009/6/2 Guido van Rossum : >> Benjamin, what would be involved in removing it? I suppose there's the >> module itself, some unit tests, and some docs. (I'm not asking you to >> remove it yet -- but I'm asking to look into the consequences, so that >> we can be sure to do the right thing before releasing 3.1 final.) > > As Raymond and Gregory have pointed out in this thread, the library is > quite independent as it stands now in the stlib, so should be trivial > to remove. Nothing else should be affected. Don't forget the what's new :) Georg From python at rcn.com Wed Jun 3 23:58:29 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 3 Jun 2009 14:58:29 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com> Message-ID: <7796C6C0FA624BBE83F5DDBB31D1E3A0@RaymondLaptop1> [GvR] >>> Benjamin, what would be involved in removing it? I suppose there's the >>> module itself, some unit tests, and some docs. (I'm not asking you to >>> remove it yet -- but I'm asking to look into the consequences, so that >>> we can be sure to do the right thing before releasing 3.1 final.) [Benjamin Peterson] >> As Raymond and Gregory have pointed out in this thread, the library is >> quite independent as it stands now in the stlib, so should be trivial >> to remove. Nothing else should be affected. Guido, have you made a firm decision to remove ipaddr.py from 3.1? The guys on IRC are chomping at the bit. Raymond From guido at python.org Thu Jun 4 01:31:05 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 3 Jun 2009 16:31:05 -0700 Subject: [Python-Dev] Issues with Py3.1's new ipaddr In-Reply-To: <7796C6C0FA624BBE83F5DDBB31D1E3A0@RaymondLaptop1> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <1afaf6160906021953l3157a361oa0956514ac9f0ae2@mail.gmail.com> <7796C6C0FA624BBE83F5DDBB31D1E3A0@RaymondLaptop1> Message-ID: On Wed, Jun 3, 2009 at 2:58 PM, Raymond Hettinger wrote: > [GvR] >>>> >>>> Benjamin, what would be involved in removing it? I suppose there's the >>>> module itself, some unit tests, and some docs. (I'm not asking you to >>>> remove it yet -- but I'm asking to look into the consequences, so that >>>> we can be sure to do the right thing before releasing 3.1 final.) > > [Benjamin Peterson] >>> >>> As Raymond and Gregory have pointed out in this thread, the library is >>> quite independent as it stands now in the stlib, so should be trivial >>> to remove. Nothing else should be affected. > > Guido, have you made a firm decision to remove ipaddr.py from 3.1? > The guys on IRC are chomping at the bit. I don't make firm decisions any more (this is something that comes with old age). Let them remove it. If it means Python will never have such functionality, because the users can't agree on the correct semantic model, so be it. It's pretty advanced stuff, some people are happy to write their own based on the RFCs, others will gladly download one of the existing contenders (ipaddr included). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ben+python at benfinney.id.au Thu Jun 4 01:41:08 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Jun 2009 09:41:08 +1000 Subject: [Python-Dev] Google Wave as a developer communication tool (was: Issues with process and discussions (Re: Issues with Py3.1's new ipaddr)) References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> Message-ID: <87vdncq417.fsf_-_@benfinney.id.au> Terry Reedy writes: > I watched and was greatly impressed by the video demo of Google's new > Wave collaborative communication system. I believe it would/will help > with some of the chronic problems we (and others) have. I watched that too. It appears to be heavily reliant on *very* fast internet access for participants in a wave. That's far from universal in the Python community, let alone the internet at large. It also appears to be heavily reliant on the wave's existence at a single point of failure (the hosting server): if that one point becomes unreliable, all participants are hosed. Neither of these problems exist with email (or NNTP). -- \ ?Well, my brother says Hello. So, hooray for speech therapy.? | `\ ?Emo Philips | _o__) | Ben Finney From janssen at parc.com Thu Jun 4 02:17:38 2009 From: janssen at parc.com (Bill Janssen) Date: Wed, 3 Jun 2009 17:17:38 PDT Subject: [Python-Dev] Google Wave as a developer communication tool (was: Issues with process and discussions (Re: Issues with Py3.1's new ipaddr)) In-Reply-To: <87vdncq417.fsf_-_@benfinney.id.au> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> Message-ID: <44142.1244074658@parc.com> Ben Finney wrote: > Terry Reedy writes: > > > I watched and was greatly impressed by the video demo of Google's new > > Wave collaborative communication system. I believe it would/will help > > with some of the chronic problems we (and others) have. > > I watched that too. It appears to be heavily reliant on *very* fast > internet access for participants in a wave. That's far from universal in > the Python community, let alone the internet at large. And on HTML 5. Those of us using IE might be hosed, but perhaps Python developers all use Firefox? (I use Camino.) > It also appears to be heavily reliant on the wave's existence at a > single point of failure (the hosting server): if that one point becomes > unreliable, all participants are hosed. Well, they've got this server synchronization protocol. Might mitigate both problems. I'm thinking about building a Python Wave server to run in UpLib, if I can find the time. Bill From greg.ewing at canterbury.ac.nz Thu Jun 4 02:23:19 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 04 Jun 2009 12:23:19 +1200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A26C76C.9020506@voidspace.org.uk> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A26C76C.9020506@voidspace.org.uk> Message-ID: <4A2713F7.3090904@canterbury.ac.nz> Michael Foord wrote: > if you are added as nosy on a tracker item (which happens > when you make a comment or you can do yourself) then you get emailed > about new comments. That's good, but... > only going to the tracker to add responses. is not so good. If the goal is to ensure that all previous discussion on a given issue is reliably recorded, then email replies ought to be archived under the relevant ticket automatically. For me at least, having to go somewhere special to post a reply would be too error-prone. I get a large number of Python-related messages in my inbox from a number of sources, and I don't usually pay much attention to exactly where they're coming from. I just hit reply-all and trust that it will go somewhere sensible. -- Greg From stephen at xemacs.org Thu Jun 4 02:46:46 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 04 Jun 2009 09:46:46 +0900 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> Message-ID: <87skighll5.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > Terry Reedy udel.edu> writes: > > > > I watched and was greatly impressed by the video demo of Google's new > > Wave collaborative communication system. I believe it would/will help > > with some of the chronic problems we (and others) have. > > I really don't think technical systems are an answer to social > issues. It's a flaw of the engineering mindset. That depends on the definition of "problem". For example, if the problem is "half our people detest web interfaces and want to discuss issues by email", then "Roundup: nosy list" *is* a technical system that is an answer. I agree that Terry wasn't particularly specific about what impressed him and how it would help for what problem. But rather than just say "technology is not a universal answer", we should ask "what problem does this address?" (Personally, I'm satisfied from the example Terry gave that he had the "summarizing the opinions of those whose opinions we respect in this domain" problem in mind, and I think there *are* technical solutions to that. Terry?) From rdmurray at bitdance.com Thu Jun 4 03:02:09 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 3 Jun 2009 21:02:09 -0400 (EDT) Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A2713F7.3090904@canterbury.ac.nz> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A26C76C.9020506@voidspace.org.uk> <4A2713F7.3090904@canterbury.ac.nz> Message-ID: On Thu, 4 Jun 2009 at 12:23, Greg Ewing wrote: > Michael Foord wrote: > >> if you are added as nosy on a tracker item (which happens when you make a >> comment or you can do yourself) then you get emailed about new comments. > > That's good, but... > >> only going to the tracker to add responses. > > is not so good. If the goal is to ensure that all previous > discussion on a given issue is reliably recorded, then > email replies ought to be archived under the relevant > ticket automatically. As Martin sad, replies work. Michael was mistaken in thinking you have to go to the tracker to post a followup (I initially did not know I could just hit reply either). You can even open new tickets via email, but I've never tried that. --David From glyph at divmod.com Thu Jun 4 04:48:02 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Thu, 04 Jun 2009 02:48:02 -0000 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A26CA17.901@v.loewis.de> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <4A26CA17.901@v.loewis.de> Message-ID: <20090604024802.12555.604416129.divmod.xquotient.12376@weber.divmod.com> On 3 Jun, 07:08 pm, martin at v.loewis.de wrote: >>To go back to JP's original comments though: what was the right thing >>for him to do, back in January, when he had these concerns? >To me, it's fairly clear: what the committer needs to get is guidance >in >any action to take. In most cases, the set of possible actions comes >down to three: >a) reject-as-is >b) commit-as-is >c) commit-with-changes (specify changes to make) >[d) take no action at this point, until certain preconditions are met] > >For d), it is common to request, to the submitter, >resubmit-with-changes, then the code needs to be reevaluated when the >submitter claims to have implemented the requested changes. Is there a document which lists these things, and explains how it is desirable to communicate them? I recently updated Twisted's equivalent document, adding minutae like which buttons to click on in our issue tracker, since that seems obvious to me but apparently wasn't obvious to a lot of new contributors. >In the specific case, JP didn't propose an action to take, hence it >wasn't clear (to me) whom his comment was directed to; I understood >it as "the module has these minor flaws, they should be fixed at some >point", which means "commit, then change later". This is what happened. My reading of it suggests that he was saying "netaddr appears to be superior in every way, so python should include that instead. But, if someone is insisting on ipaddr here are the things that could change about it". The important thing here is that interpretation of the comment is required, so I can definitely see how you saw it the way you did. There is no "-1" in his comment, and there's no documentation (that I'm aware of) which says that a "-1" is required, or how it will be used or interpreted. From tjreedy at udel.edu Thu Jun 4 07:43:32 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 04 Jun 2009 01:43:32 -0400 Subject: [Python-Dev] Google Wave as a developer communication tool In-Reply-To: <87vdncq417.fsf_-_@benfinney.id.au> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> Message-ID: Ben Finney wrote: > Terry Reedy writes: > >> I watched and was greatly impressed by the video demo of Google's new >> Wave collaborative communication system. I believe it would/will help >> with some of the chronic problems we (and others) have. Example: if PEPs were waves, then responses could either be entered as live edits (with permission) or comments immediately following the relevant text (as with email/newsgroups) visible to all. Much easier than current situation. Edits are marked in red shading for those who have previously seen document. Many have complained that it is hard to read multiple versions of a PEP since so much is new and there is no indication of what is new to the particular reader. > I watched that too. It appears to be heavily reliant on *very* fast > internet access for participants in a wave. That's far from universal in > the Python community, let alone the internet at large. Even a slow connection would make participation in PEPs better than today. And being able to get a version of the docs that hi-lites "what's new for you" would also be very nice. > It also appears to be heavily reliant on the wave's existence at a > single point of failure (the hosting server): if that one point becomes > unreliable, all participants are hosed. We have that problem already with the tracker, which does occasionally go down for a bit. And the svn host? (One reason to move to distributed system.) > Neither of these problems exist with email (or NNTP). But do for an email list, like this one. Or a wiki. Terry Jan Reedy From tjreedy at udel.edu Thu Jun 4 09:12:37 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 04 Jun 2009 03:12:37 -0400 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <87skighll5.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87skighll5.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Stephen J. Turnbull wrote: > Antoine Pitrou writes: > > Terry Reedy udel.edu> writes: > > > > > > I watched and was greatly impressed by the video demo of Google's new > > > Wave collaborative communication system. I believe it would/will help > > > with some of the chronic problems we (and others) have. > > > > I really don't think technical systems are an answer to social > > issues. It's a flaw of the engineering mindset. Well, its true that telephones did not cure people of dishonesty, but it did make communication-at-a-distance, truthful or otherwise, easier. > That depends on the definition of "problem". For example, if the > problem is "half our people detest web interfaces and want to discuss > issues by email", then "Roundup: nosy list" *is* a technical system > that is an answer. And for me, it is very helpful. > I agree that Terry wasn't particularly specific about what impressed > him and how it would help for what problem. Right. A quick three lines plus a few more. There were several things that Wave brings together. I will suggest specific experiments when it is actually available for use. > But rather than just say > "technology is not a universal answer", we should ask "what problem > does this address?" As I said in my response to Ben Finney, two problems addressed are 1. Reading a document that has been edited since you read it last. The old method of in-place deltas -- typically strikeout for deletions and some other special marking for additions -- has stood that test of time. Many word processors do this, but Waves improve on them by individualizing the markings for the particular reader and then removing them once read. 2. Multiple people editing a document. One solution is the "You're it" method, whether informal by passing a doc around either on paper or electronicly or formal by VCS checkout. Another is edit in isolation, merge, and resolve conflicts. Waves allow real-time simultaneous editing and merging via micro-deltas, so conflicts are immediately apparent. This would generally work better with text docs than with code, which needs to be frequently frozen to run, but code might even be workable for side-by-side pair or sprint programming. > (Personally, I'm satisfied from the example Terry > gave that he had the "summarizing the opinions of those whose opinions > we respect in this domain" problem in mind, and I think there *are* > technical solutions to that. Terry?) The current 'summarize opinion' system is somewhat haphazard and informal, with no auto-tabulation. It seems to have broken-down a bit for the ipaddr issue, at least until the last few days when the best solution was to remove and defer. Better would have been either not add and defer or add with generally supported revision. I speculate that if there had been a proto-pep wave (though not then possible) with a vote widget with buttons such as Accept as is, Probably accept after revision, Don't know, and Reject, we might have reached a better outcome sooner. Terry Jan Reedy From ben+python at benfinney.id.au Thu Jun 4 10:21:14 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Jun 2009 18:21:14 +1000 Subject: [Python-Dev] Google Wave as a developer communication tool References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> Message-ID: <87tz2wo1dx.fsf@benfinney.id.au> Terry Reedy writes: > Ben Finney wrote: > > I watched [the Google Wave presentation] too. It appears to be > > heavily reliant on *very* fast internet access for participants in a > > wave. That's far from universal in the Python community, let alone > > the internet at large. > > Even a slow connection would make participation in PEPs better than > today. How can you know that? A slow link doesn't punish email or NNTP communication the way an interactive web application does. Why would Google Wave be any less punitive to low-bandwidth users than existing live web applications? I would not want to put money against Google technologists giving lower priority to the needs of the majority of internet users without fast connections. > > It also appears to be heavily reliant on the wave's existence at a > > single point of failure (the hosting server): if that one point > > becomes unreliable, all participants are hosed. > > We have that problem already with the tracker, which does occasionally > go down for a bit. And the svn host? (One reason to move to > distributed system.) Right. These are all reasons for moving toward distributed systems; Python has chosen to do so already with its VCS. Why would the choice of a new communications technology not take this into consideration? > > Neither of these problems exist with email (or NNTP). > > But do for an email list, like this one. Or a wiki. No. None of mailing list, NNTP, or wiki are heavily punitive to low-bandwidth links. -- \ ?Creativity can be a social contribution, but only in so far as | `\ society is free to use the results.? ?Richard Stallman | _o__) | Ben Finney From ben+python at benfinney.id.au Thu Jun 4 10:50:27 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 04 Jun 2009 18:50:27 +1000 Subject: [Python-Dev] Google Wave as a developer communication tool References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> <87tz2wo1dx.fsf@benfinney.id.au> Message-ID: <87ljo8o018.fsf@benfinney.id.au> Ben Finney writes: > Terry Reedy writes: > > Even a slow connection would make participation in PEPs better than > > today. > > How can you know that? A slow link doesn't punish email or NNTP > communication the way an interactive web application does. Strike that; reverse it. Should be ?A slow link is not punished by email or NNTP communication the way it is by an interactive web application?. -- \ ?I went over to the neighbor's and asked to borrow a cup of | `\ salt. ?What are you making?? ?A salt lick.?? ?Steven Wright | _o__) | Ben Finney From ncoghlan at gmail.com Thu Jun 4 12:07:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 04 Jun 2009 20:07:05 +1000 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> Message-ID: <4A279CC9.7080507@gmail.com> Paul Moore wrote: > Mostly, I agree, but I definitely disagree, I'm afraid, on the use of > the tracker for discussions. To keep track of discussions on a ticket, > I have to personally keep a list of the tickets I'm interested in, > check back regularly to see if there's anything new, and keep a mental > note of where I've read up to so I know what's new. Others have already noted that Roundup not only emails you with new comments, but also lets you reply via email. That happens for anyone on the nosy list for an issue (commenting on an issue automatically adds you to the nosy list). In addition, you can fairly easily create a saved query to show you all the open tickets that you are on the nosy list for. (Although I created and saved my query for that so long ago that I don't recall the exact details on how to go about doing that). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From p.f.moore at gmail.com Thu Jun 4 12:20:56 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 4 Jun 2009 11:20:56 +0100 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A279CC9.7080507@gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> Message-ID: <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> 2009/6/4 Nick Coghlan : > Paul Moore wrote: >> Mostly, I agree, but I definitely disagree, I'm afraid, on the use of >> the tracker for discussions. To keep track of discussions on a ticket, >> I have to personally keep a list of the tickets I'm interested in, >> check back regularly to see if there's anything new, and keep a mental >> note of where I've read up to so I know what's new. > > Others have already noted that Roundup not only emails you with new > comments, but also lets you reply via email. That happens for anyone on > the nosy list for an issue (commenting on an issue automatically adds > you to the nosy list). > > In addition, you can fairly easily create a saved query to show you all > the open tickets that you are on the nosy list for. (Although I created > and saved my query for that so long ago that I don't recall the exact > details on how to go about doing that). Agreed, and thanks to all for the pointers to features I wasn't aware of. I could still argue that there are downsides (need to take action to set myself as nosy on an issue, possibly setting up a new mail filter, housekeeping cruft, the fact that people don't quote in the same way on tracker items) that make tracker discussions less attractive to me, but it's very personal things. Let's leave it as merely that I wouldn't have contributed to the ipaddr discussion if it had been solely on the tracker (not even if pointers to and reminders of the tracker item were posted to python-dev). My input wasn't very valuable, so maybe that doesn't matter much. Maybe the key point is that keeping discussion on the tracker introduces a barrier to participation by lurkers, FWIW. Paul. From aahz at pythoncraft.com Thu Jun 4 14:23:00 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 4 Jun 2009 05:23:00 -0700 Subject: [Python-Dev] Google Wave as a developer communication tool In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> Message-ID: <20090604122300.GA3768@panix.com> On Thu, Jun 04, 2009, Terry Reedy wrote: > > Example: if PEPs were waves, then responses could either be entered as > live edits (with permission) or comments immediately following the > relevant text (as with email/newsgroups) visible to all. Much easier > than current situation. Edits are marked in red shading for those who > have previously seen document. Many have complained that it is hard to > read multiple versions of a PEP since so much is new and there is no > indication of what is new to the particular reader. It sounds like Wave requires a high-powered browser, similar to Google Maps. That makes me -1 because I want to continue using Lynx. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Given that C++ has pointers and typecasts, it's really hard to have a serious conversation about type safety with a C++ programmer and keep a straight face. It's kind of like having a guy who juggles chainsaws wearing body armor arguing with a guy who juggles rubber chickens wearing a T-shirt about who's in more danger." --Roy Smith, c.l.py, 2004.05.23 From jaraco at jaraco.com Thu Jun 4 15:02:53 2009 From: jaraco at jaraco.com (Jason R. Coombs) Date: Thu, 4 Jun 2009 09:02:53 -0400 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio Message-ID: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> I just wanted to share my experience with the mercurial checkout. I cloned http://code.python.org/hg/branches/py3k to continue work on http://bugs.python.org/issue1578269 but I found that when I click on PC/VS8.0/pcbuild.sln, nothing happens. This appears to be due to a bug/limitation in vslauncher in that it doesn't recognize LF as a line separator. vslauncher is the default association for sln files and its purpose is to parse out the .sln file and launch it with the appropriate Visual Studio version based on the header. What makes matters worse is if vslauncher fails to recognize the format, it does nothing, so it just appears as if the file fails to launch anything. It seems that within the hg repository, everything has been converted to LF for line endings. I suspect this is because HG provides no integrated support for line-ending conversions and because the hg to svn bridge is probably running on a Unix OS. So converting the pcbuild.sln file to CRLF line endings resolved the problem and the file would launch normally. Also, without conversion, it was possible to open the .sln file in Visual Studio explicitly. I wanted to share this with the community in case anyone else runs into this issue. Also, if there's a recommended procedure for addressing this issue (and others that might arise due to non-native line endings), I'd be interested to hear it. Regards, Jason From phd at phd.pp.ru Thu Jun 4 15:30:31 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu, 4 Jun 2009 17:30:31 +0400 Subject: [Python-Dev] Mercurial and linefeeds In-Reply-To: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> Message-ID: <20090604133031.GH30429@phd.pp.ru> On Thu, Jun 04, 2009 at 09:02:53AM -0400, Jason R. Coombs wrote: > It seems that within the hg repository, everything has been converted to LF for line endings. I suspect this is because HG provides no integrated support for line-ending conversions and because the hg to svn bridge is probably running on a Unix OS. http://www.selenic.com/mercurial/wiki/FAQ#FAQ.2BAC8-TechnicalDetails.What_about_Windows_line_endings_vs._Unix_line_endings.3F Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From dirkjan at ochtman.nl Thu Jun 4 15:30:51 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 4 Jun 2009 15:30:51 +0200 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> Message-ID: On Thu, Jun 4, 2009 at 3:02 PM, Jason R. Coombs wrote: > I wanted to share this with the community in case anyone else runs into this issue. ?Also, if there's a recommended procedure for addressing this issue (and others that might arise due to non-native line endings), I'd be interested to hear it. Mercurial comes with a win32text extension that helps resolve issues like these. It allows you to specify in the .hg/hgrc that .sln files should always be expanded to CRLF. Cheers, Dirkjan From rdmurray at bitdance.com Thu Jun 4 15:49:46 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 4 Jun 2009 09:49:46 -0400 (EDT) Subject: [Python-Dev] Mercurial and linefeeds In-Reply-To: <20090604133031.GH30429@phd.pp.ru> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <20090604133031.GH30429@phd.pp.ru> Message-ID: On Thu, 4 Jun 2009 at 17:30, Oleg Broytmann wrote: > On Thu, Jun 04, 2009 at 09:02:53AM -0400, Jason R. Coombs wrote: >> It seems that within the hg repository, everything has been converted to LF for line endings. I suspect this is because HG provides no integrated support for line-ending conversions and because the hg to svn bridge is probably running on a Unix OS. > > http://www.selenic.com/mercurial/wiki/FAQ#FAQ.2BAC8-TechnicalDetails.What_about_Windows_line_endings_vs._Unix_line_endings.3F Linked from that is: http://www.selenic.com/mercurial/wiki/Win32TextExtension which says: "The filter specification can however not be managed by the repository..." (I'm not clear what is being filtered on, but it looks like it might be the _content_ of the files.) The above statement seems like a really odd design choice, with the potential for causing considerable developer coordination headaches. Any Mercurial boffins want to talk about how this works in practice? --David From dirkjan at ochtman.nl Thu Jun 4 16:12:05 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 4 Jun 2009 16:12:05 +0200 Subject: [Python-Dev] Mercurial and linefeeds In-Reply-To: References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <20090604133031.GH30429@phd.pp.ru> Message-ID: On Thu, Jun 4, 2009 at 3:49 PM, R. David Murray wrote: > The above statement seems like a really odd design choice, with the > potential for causing considerable developer coordination headaches. > > Any Mercurial boffins want to talk about how this works in practice? I'm guessing that's just because it uses Mercurial's built-in configuration file chain (.hg/hgrc, ~/.hgrc and /etc/mercurial/hgrc). Those aren't transmitted with repositories because they can contain potentially security-sensitive information (like triggers for executing interesting hooks and information on what users to trust). It would certainly be possible to write a modified win32text-like extension that takes its cues from a versioned file (like the .hgtags and .hgsigs files currently supported). I'm not sure why the original author didn't go that way, though. Cheers, Dirkjan From p.f.moore at gmail.com Thu Jun 4 16:20:56 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 4 Jun 2009 15:20:56 +0100 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> Message-ID: <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> 2009/6/4 Dirkjan Ochtman : > On Thu, Jun 4, 2009 at 3:02 PM, Jason R. Coombs wrote: >> I wanted to share this with the community in case anyone else runs into this issue. ?Also, if there's a recommended procedure for addressing this issue (and others that might arise due to non-native line endings), I'd be interested to hear it. > > Mercurial comes with a win32text extension that helps resolve issues > like these. It allows you to specify in the .hg/hgrc that .sln files > should always be expanded to CRLF. Silly question, but given that Mercurial itself does *no* conversion, and CRLF is the only valid form for that file, why shouldn't the file be checked into the repository with CRLF endings, and that's the end of it (apart from misconfigured win32text setups on the client)? Ideally, the history should be imported with CRLF throughout, which is a task for the migration process. If that's too hard because of limitations in the available conversion tools, then maybe the limitation has to be accepted that pre-conversion copies of the file have incorrect line endings and a manual fiox is committed right after the switchover. Doesn't help the current Mercurial mirrors, of course, but at least this report alerts us to a consideration for the final switchover. Essentially, pcbuild.sln is a binary file, and should be treated as such. Maybe it's an error in the Subversion setup that it's treated as text at all... Paul. From phd at phd.pp.ru Thu Jun 4 16:29:40 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Thu, 4 Jun 2009 18:29:40 +0400 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> Message-ID: <20090604142940.GA15804@phd.pp.ru> On Thu, Jun 04, 2009 at 03:20:56PM +0100, Paul Moore wrote: > Essentially, pcbuild.sln is a binary file, and should be treated as > such. Maybe it's an error in the Subversion setup that it's treated as > text at all... Subversion has a built-in notion of eol-conversion (don't know if it was used for this particular file). Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From alexander.belopolsky at gmail.com Thu Jun 4 17:03:22 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 4 Jun 2009 11:03:22 -0400 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> Message-ID: On Thu, Jun 4, 2009 at 6:20 AM, Paul Moore wrote: ... > I could still argue that there are downsides (need to take action to > set myself as nosy on an issue, possibly setting up a new mail filter, > housekeeping cruft, the fact that people don't quote in the same way > on tracker items) that make tracker discussions less attractive to me, > but it's very personal things. How hard would it be to set up a bot that will subscribe to python-dev and archive messages that have [issueXXXX] in the subject under appropriate roundup ticket? This way if discussion started on roundup, it would only take adding python-dev in CC to bring it to the larger python-dev audience and adding issue number to the python-dev thread subject will be all it takes to archive new messages on roundup. This would not solve a problem of linking threads that start on python-dev before a ticket is opened, but with some proper conventions this problem can be solved as well. From dirkjan at ochtman.nl Thu Jun 4 17:03:28 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 4 Jun 2009 17:03:28 +0200 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> Message-ID: On Thu, Jun 4, 2009 at 4:20 PM, Paul Moore wrote: > Silly question, but given that Mercurial itself does *no* conversion, > and CRLF is the only valid form for that file, why shouldn't the file > be checked into the repository with CRLF endings, and that's the end > of it (apart from misconfigured win32text setups on the client)? That's certainly a valid setup, and I wasn't suggesting otherwise. I have no experience with .sln files, and I haven't really used win32text either; I was just commenting on the general facilities of Mercurial wrt these kinds of issues. > Ideally, the history should be imported with CRLF throughout, which is > a task for the migration process. If that's too hard because of > limitations in the available conversion tools, then maybe the > limitation has to be accepted that pre-conversion copies of the file > have incorrect line endings and a manual fiox is committed right after > the switchover. Importing it with CRLF throughout would certainly be possible. I'll add a section to my draft PEP. > Essentially, pcbuild.sln is a binary file, and should be treated as > such. Maybe it's an error in the Subversion setup that it's treated as > text at all... Yes, it certainly seems that way. Cheers, Dirkjan From alexander.belopolsky at gmail.com Thu Jun 4 17:17:11 2009 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Thu, 4 Jun 2009 11:17:11 -0400 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090604150814.GB17981@idyll.org> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> <20090604150814.GB17981@idyll.org> Message-ID: Alternatively, one could create a python-dev profile on roundup and encourage people to add python-dev to the nosy list when discussion needs to be broaden. This way replies to all will be posted in both places. Once discussion gets too specialized, someone can remove python-dev from the "nosies." No new code is required to set this up as far as I can tell. On Thu, Jun 4, 2009 at 11:08 AM, C. Titus Brown wrote: > On Thu, Jun 04, 2009 at 11:03:22AM -0400, Alexander Belopolsky wrote: > -> On Thu, Jun 4, 2009 at 6:20 AM, Paul Moore wrote: > -> ... > -> > I could still argue that there are downsides (need to take action to > -> > set myself as nosy on an issue, possibly setting up a new mail filter, > -> > housekeeping cruft, the fact that people don't quote in the same way > -> > on tracker items) that make tracker discussions less attractive to me, > -> > but it's very personal things. > -> > -> How hard would it be to set up a bot that will subscribe to python-dev > -> and archive messages that have [issueXXXX] in the subject under > -> appropriate roundup ticket? ?This way if discussion started on > -> roundup, it would only take adding python-dev in CC to bring it to the > -> larger python-dev audience and adding issue number to the python-dev > -> thread subject will be all it takes to archive new messages on > -> roundup. ?This would not solve a problem of linking threads that start > -> on python-dev before a ticket is opened, but with some proper > -> conventions this problem can be solved as well. > > In my experience this kind automated stuff is too fragile to work > reliably & specifically over time, which is what we would want -- "fire > and forget". > > Is there a good python-dev archive search mechanism (other than to > google "python-dev " ;) out there? ?Wouldn't that help? > > cheers, > --titus > -- > C. Titus Brown, ctb at msu.edu > From ctb at msu.edu Thu Jun 4 17:08:14 2009 From: ctb at msu.edu (C. Titus Brown) Date: Thu, 4 Jun 2009 08:08:14 -0700 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> Message-ID: <20090604150814.GB17981@idyll.org> On Thu, Jun 04, 2009 at 11:03:22AM -0400, Alexander Belopolsky wrote: -> On Thu, Jun 4, 2009 at 6:20 AM, Paul Moore wrote: -> ... -> > I could still argue that there are downsides (need to take action to -> > set myself as nosy on an issue, possibly setting up a new mail filter, -> > housekeeping cruft, the fact that people don't quote in the same way -> > on tracker items) that make tracker discussions less attractive to me, -> > but it's very personal things. -> -> How hard would it be to set up a bot that will subscribe to python-dev -> and archive messages that have [issueXXXX] in the subject under -> appropriate roundup ticket? This way if discussion started on -> roundup, it would only take adding python-dev in CC to bring it to the -> larger python-dev audience and adding issue number to the python-dev -> thread subject will be all it takes to archive new messages on -> roundup. This would not solve a problem of linking threads that start -> on python-dev before a ticket is opened, but with some proper -> conventions this problem can be solved as well. In my experience this kind automated stuff is too fragile to work reliably & specifically over time, which is what we would want -- "fire and forget". Is there a good python-dev archive search mechanism (other than to google "python-dev " ;) out there? Wouldn't that help? cheers, --titus -- C. Titus Brown, ctb at msu.edu From martin at v.loewis.de Thu Jun 4 17:50:34 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 04 Jun 2009 17:50:34 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090604024802.12555.604416129.divmod.xquotient.12376@weber.divmod.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <4A26CA17.901@v.loewis.de> <20090604024802.12555.604416129.divmod.xquotient.12376@weber.divmod.com> Message-ID: <4A27ED4A.6080002@v.loewis.de> > Is there a document which lists these things, and explains how it is > desirable to communicate them? I recently updated Twisted's equivalent > document, adding minutae like which buttons to click on in our issue > tracker, since that seems obvious to me but apparently wasn't obvious to > a lot of new contributors. I'm not quite sure whether this is what you are asking for, but there is http://www.python.org/dev/workflow/ > My reading of it suggests that he was saying "netaddr appears to be > superior in every way, so python should include that instead. But, if > someone is insisting on ipaddr here are the things that could change > about it". The important thing here is that interpretation of the > comment is required, so I can definitely see how you saw it the way you > did. There is no "-1" in his comment, and there's no documentation > (that I'm aware of) which says that a "-1" is required, or how it will > be used or interpreted. Well, there is http://www.python.org/dev/peps/pep-0010/ I personally use them sometimes, but wouldn't insist on others using them. Regards, Martin From dirkjan at ochtman.nl Thu Jun 4 17:53:39 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Thu, 4 Jun 2009 17:53:39 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090604150814.GB17981@idyll.org> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> <20090604150814.GB17981@idyll.org> Message-ID: On Thu, Jun 4, 2009 at 5:08 PM, C. Titus Brown wrote: > In my experience this kind automated stuff is too fragile to work > reliably & specifically over time, which is what we would want -- "fire > and forget". You could a python-dev account in the tracker and allow people to nosy it. > Is there a good python-dev archive search mechanism (other than to > google "python-dev " ;) out there? ?Wouldn't that help? Like http://python.markmail.org/? Cheers, Dirkjan From martin at v.loewis.de Thu Jun 4 17:54:23 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 04 Jun 2009 17:54:23 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <20090604150814.GB17981@idyll.org> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> <20090604150814.GB17981@idyll.org> Message-ID: <4A27EE2F.4080600@v.loewis.de> > Is there a good python-dev archive search mechanism (other than to > google "python-dev " ;) out there? Wouldn't that help? I would add site:mail.python.org into the google query, but apart from that: what's wrong with google search? Regards, Martin From martin at v.loewis.de Thu Jun 4 17:55:52 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 04 Jun 2009 17:55:52 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> <20090604150814.GB17981@idyll.org> Message-ID: <4A27EE88.7080208@v.loewis.de> Alexander Belopolsky wrote: > Alternatively, one could create a python-dev profile on roundup and > encourage people to add python-dev to the nosy list when discussion > needs to be broaden. This way replies to all will be posted in both > places. Once discussion gets too specialized, someone can remove > python-dev from the "nosies." No new code is required to set this up > as far as I can tell. That would certainly be possible. I'm not sure whether it *should* be done, though. If anybody actually wants that feature, please create a ticket in the meta tracker. Regards, Martin From martin at v.loewis.de Thu Jun 4 18:01:03 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 04 Jun 2009 18:01:03 +0200 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A279CC9.7080507@gmail.com> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> Message-ID: <4A27EFBF.7070006@v.loewis.de> > In addition, you can fairly easily create a saved query to show you all > the open tickets that you are on the nosy list for. (Although I created > and saved my query for that so long ago that I don't recall the exact > details on how to go about doing that). It's fairly easy. Start a search, put your account name into "Nosy list member", and put "My nosy" (or some such) into "Query name**". Then hit search, and watch "My nosy" appear under "Your queries". Regards, Martin From tjreedy at udel.edu Thu Jun 4 19:15:16 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 04 Jun 2009 13:15:16 -0400 Subject: [Python-Dev] Google Wave as a developer communication tool In-Reply-To: <87tz2wo1dx.fsf@benfinney.id.au> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> <87tz2wo1dx.fsf@benfinney.id.au> Message-ID: Ben Finney wrote: > Terry Reedy writes: > >> Ben Finney wrote: >>> I watched [the Google Wave presentation] too. It appears to be >>> heavily reliant on *very* fast internet access for participants in a >>> wave. That's far from universal in the Python community, let alone >>> the internet at large. >> Even a slow connection would make participation in PEPs better than >> today. > > How can you know that? A slow link doesn't punish email or NNTP > communication the way an interactive web application does. Why would > Google Wave be any less punitive to low-bandwidth users than existing > live web applications? I am not sure of what 'live web application' you have in mind to compare to ;-). A general answer is text stream versus binary blob stream. A wave is more like a souped-up mailing-list or newsgroup thread or wiki page than an online game. You open your wave client and PEPxxx.wave is marked as having new content. At your leisure, you open it (or perhaps you have marked it 'download updates in background'). That that takes longer with a slow connection is no different than with other text streams. If you type in a comment, even 1200 baud upsteam is fast enough. Where do you get 'punitive' in this? >>> It also appears to be heavily reliant on the wave's existence at a >>> single point of failure (the hosting server): if that one point >>> becomes unreliable, all participants are hosed. >> We have that problem already with the tracker, which does occasionally >> go down for a bit. And the svn host? (One reason to move to >> distributed system.) > > Right. These are all reasons for moving toward distributed systems; > Python has chosen to do so already with its VCS. Why would the choice of > a new communications technology not take this into consideration? It should. The wave protocol includes server-to-server mirroring. A wave can be shared across multiple servers. >>> Neither of these problems exist with email (or NNTP). >> But do for an email list, like this one. Or a wiki. The second of 'these problems' was 'single point of failure'. That *can* apply to email and wiki. > No. None of mailing list, NNTP, or wiki are heavily punitive to > low-bandwidth links. I would not expect that to be much more true of a text-only wave. It is binary content (and embeded ad links) that made broadband really necessary for web surfing. Terry Jan Reedy From mario.danic at gmail.com Thu Jun 4 19:17:23 2009 From: mario.danic at gmail.com (Mario) Date: Thu, 4 Jun 2009 19:17:23 +0200 Subject: [Python-Dev] Roundup improvements use-cases Message-ID: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com> Hello, in the last few weeks I have been working on defining use-cases which will lead the improvement of the Roundup tracker. As this is very important, I would like your valuable input in form of comments, criticism and advices. Use-cases in-lined: ============== Daniel is a great hacker, but the only time he has time to hack is when his child is asleep. And his child wakes up every now and then, pretty often actually. So when it starts crying, Daniel commits fix to that silly bug that has been hunting us for ages, but forgets to close down the bug so it stays open. Looking over bug tracker, Jane finds the same bug and starts working on the same bug. Wasted time. Daniel should be able to format commit message in such a way that would automatically close the bug. Maria is a student, and got involved in FOSS recently. With all the studies she has to do, she would prefer not to spend her time changing bug status. She should be able to format commit message in such a way that would change bug status to something other then the current status. Technical talk: USE CASE A: Integrate issue property editing with Mercurial, including: * USE CASE A1: Allow users to close issues via tokens embedded into commit messages. * USE CASE A2: Allow users to change issue status via tokens embedded into commit messages. Ronny is about to fix a torny bug, so he has a public branch for that. He is making great progress, but the issue that tracks the bug only contains a very out-of-date patch, prompting other developers to try to fix the same bug. Ronny should be able to tell Roundup about where his code lives, so users can get up-to-date patches/diffs automatically. This also allows other users to know all the code that changed to fix a given bug. Technical talk: Integrate branch and rich patch handling into Roundup * USE CASE B: Track all changesets that relate to a given issue: o USE CASE B1: Using tokens embedded into commit messages. (Post commit) o USE CASE B2: Using named branches, bookmarks. (Pre or post commit) o USE CASE B3: Using patchsets, bundles or whatchacallit for fat Mercurial patches. (Pre or post commit) Brett wants to fix a couple of related issues and has a local Mercurial branch for that. He would like his commit messages to include useful information for when his patch/branch lands in the Python repository. Besides the Mercurial->Roundup integration, a Roundup->Mercurial one that would allow one to fetch issue details and existing patches/branches with metadata would make Brett's work more valuable. Technical talk: USE CASE C: Add a CLI interface for Roundup so VCSs can query the tracker. * USE CASE C1: Automatically fetch issue data. * USE CASE C2: Pre-format output for greater usefulness in commit messages: o USE CASE C2.1: Setting issue properties. o USE CASE C2.2: Grouping changesets. * USE CASE C3: Fetch branch information for local cloning. * USE CASE C4: Add a Mercurial extension to exercize the CLI client. USE CASE D: Antoine is merging lots of branches and applying lots of patches to his local branch of the Python repository. He goes to each issue with a patch/branch and tells people whether their patches apply cleanly, just to avoid merge issues in the main branch. While he could use Mercurial Patch Queues (mq), Roundup would need to learn to both listen to and to submit patches to mq in order to completely replace Antoine's work with automated tools. Having a quick 'check if this patch applies cleanly' button would make triaging issues much easier. USE CASE E: David is checking the python-commits list for avoiding bad code from landing and nesting in the Python code base. He only sees the patches: while informative, it requires a bit of mental gymanstics to see how it would merge with the surrounding code. Whenever he thinks some code has tabs and spaces or lines that are too long, he runs pylint and reindent.py locally. He can only raise concerns about the code after it lands on the main repository. It should be easier to see the code changes in context. Having a way to check the code for mistakes in the tracker would make David's life a lot easier. USE CASE F: Van Lindberg is concerned about code submissions from non-core developers: how can the PSF re-license this code in the future without talking to each contributor, whether the PSF is safe from litigation based on copyrights of these contributions and related questions are always bugging him. While the PSF has Contributor Agreements exactly for these cases, it would be great to have the issue tracker and the VCS talk to each other, so they could ask contributors to sign (or declare to have already signed) the CAs when necessary. USE CASE G: Use Transplant/Patch Branch to generate patches from branches linked from Roundup. USE CASE J: Integrate the code/commits navigation interface with Roundup, so changesets, branches, etc., can be easily linked/browsed (starting) from the Roundup UI and issues can be created/linked to commits (starting) from the navigation tool UI. USE CASE K: For a given issue, add per patched file links for RSS logs (see http://selenic.com/hg/rss-log/tip/mercurial/hgweb/hgweb_mod.py). USE CASE M: Besides links to files, allow adding links to files at given versions/tags/branchs, links to tarballs and easy to clone links to branches and repositories. USE CASE V: Handle small branches (and maybe suggest using them for small patches?) generated using the convert extension with --filemap. ============== Improvements to the use cases will follow on the wiki page: http://www.roundup-tracker.org/cgi-bin/moin.cgi/GSoC09 Thanks for your attention and time. Respectfully, Mario From tjreedy at udel.edu Thu Jun 4 19:24:26 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 04 Jun 2009 13:24:26 -0400 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A27EFBF.7070006@v.loewis.de> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <4A27EFBF.7070006@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> In addition, you can fairly easily create a saved query to show you all >> the open tickets that you are on the nosy list for. (Although I created >> and saved my query for that so long ago that I don't recall the exact >> details on how to go about doing that). > > It's fairly easy. Start a search, put your account name into "Nosy list > member", and put "My nosy" (or some such) into "Query name**". Then hit > search, and watch "My nosy" appear under "Your queries". Done. Wow. Thanks. tjr From ctb at msu.edu Thu Jun 4 19:34:00 2009 From: ctb at msu.edu (C. Titus Brown) Date: Thu, 4 Jun 2009 10:34:00 -0700 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) In-Reply-To: <4A27EE2F.4080600@v.loewis.de> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> <20090604150814.GB17981@idyll.org> <4A27EE2F.4080600@v.loewis.de> Message-ID: <20090604173400.GB25702@idyll.org> On Thu, Jun 04, 2009 at 05:54:23PM +0200, "Martin v. L?wis" wrote: -> > Is there a good python-dev archive search mechanism (other than to -> > google "python-dev " ;) out there? Wouldn't that help? -> -> I would add site:mail.python.org into the google query, but apart from -> that: what's wrong with google search? It generally points you towards the right messages in the python-dev archives, but those are kept by month and the threading doesn't always work well. Sometimes conversations span many months, and threading can be an immense help for understanding the discussion. Something like MarkMail (as Dirkjan mentioned) may have a better interface. I'll give it a try. thanks, --titus -- C. Titus Brown, ctb at msu.edu From nad at acm.org Thu Jun 4 20:48:07 2009 From: nad at acm.org (Ned Deily) Date: Thu, 04 Jun 2009 11:48:07 -0700 Subject: [Python-Dev] Issues with process and discussions (Re: Issues with Py3.1's new ipaddr) References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <79990c6b0906031042y190e05d6obbf675200b171666@mail.gmail.com> <4A279CC9.7080507@gmail.com> <79990c6b0906040320m22c79a56jb3d3286f330bcb68@mail.gmail.com> <20090604150814.GB17981@idyll.org> <4A27EE2F.4080600@v.loewis.de> <20090604173400.GB25702@idyll.org> Message-ID: In article <20090604173400.GB25702 at idyll.org>, "C. Titus Brown" wrote: > Something like MarkMail (as Dirkjan mentioned) may have a better > interface. I'll give it a try. Or http://search.gmane.org/ with group gmane.comp.python.devel -- Ned Deily, nad at acm.org From van.lindberg at gmail.com Thu Jun 4 20:52:12 2009 From: van.lindberg at gmail.com (VanL) Date: Thu, 04 Jun 2009 13:52:12 -0500 Subject: [Python-Dev] Roundup improvements use-cases In-Reply-To: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com> References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com> Message-ID: Mario wrote: > USE CASE F: > > Van Lindberg is concerned about code submissions from non-core > developers: how can the PSF re-license this code in the future without > talking to each contributor, whether the PSF is safe from litigation > based on copyrights of these contributions and related questions are > always bugging him. While the PSF has Contributor Agreements exactly > for these cases, it would be great to have the issue tracker and the > VCS talk to each other, so they could ask contributors to sign (or > declare to have already signed) the CAs when necessary. +1. This is exactly right. From seefeld at sympatico.ca Thu Jun 4 22:10:41 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Thu, 04 Jun 2009 16:10:41 -0400 Subject: [Python-Dev] [Roundup-devel] Roundup improvements use-cases In-Reply-To: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com> References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com> Message-ID: <4A282A41.8040701@sympatico.ca> Hi Mario, thanks for sending these use-cases out. Let me give some feedback: Mario wrote: > Technical talk: USE CASE A: Integrate issue property editing with > Mercurial, including: > > * USE CASE A1: Allow users to close issues via tokens embedded > into commit messages. > * USE CASE A2: Allow users to change issue status via tokens > embedded into commit messages. > This is the most fundamental one, and it is implementable without even touching Roundup itself. All that is required is a 'post-commit-hook' (whatever that is spelled for a particular VCS) that sends a mail to a tracker, if some tokens were found in the commit message. While in theory this is all up for customization, I think it would be nice to set up some conventions to guide users in how to do this (the 'token grammar', mostly), just for convenience. > Ronny is about to fix a torny bug, so he has a public branch for that. > He is making great progress, but the issue that tracks the bug only > contains a very out-of-date patch, prompting other developers to try > to fix the same bug. Ronny should be able to tell Roundup about where > his code lives, so users can get up-to-date patches/diffs > automatically. This also allows other users to know all the code that > changed to fix a given bug. > > Technical talk: Integrate branch and rich patch handling into Roundup > > * USE CASE B: Track all changesets that relate to a given issue: > o USE CASE B1: Using tokens embedded into commit messages. > (Post commit) > o USE CASE B2: Using named branches, bookmarks. (Pre or post commit) > o USE CASE B3: Using patchsets, bundles or whatchacallit for > fat Mercurial patches. (Pre or post commit) > Hmm, I have difficulties mapping your prosaic description of the use case above to the 'technical talk'. The latter almost sounds like an implementation strategy to me. This is a very interesting use case, driving us at the core of the 'VCS integration'. We need to customize a schema to provide a notion for 'repository' and 'branch', and also for 'patch'. Then we need to somehow endow the patch type with knowledge of what it means to be 'applied' to a branch, etc. (more on this below). > Brett wants to fix a couple of related issues and has a local > Mercurial branch for that. He would like his commit messages to > include useful information for when his patch/branch lands in the > Python repository. Besides the Mercurial->Roundup integration, a > Roundup->Mercurial one that would allow one to fetch issue details and > existing patches/branches with metadata would make Brett's work more > valuable. > > Technical talk: USE CASE C: Add a CLI interface for Roundup so VCSs > can query the tracker. > > * USE CASE C1: Automatically fetch issue data. > * USE CASE C2: Pre-format output for greater usefulness in commit messages: > o USE CASE C2.1: Setting issue properties. > o USE CASE C2.2: Grouping changesets. > * USE CASE C3: Fetch branch information for local cloning. > * USE CASE C4: Add a Mercurial extension to exercize the CLI client. > This set of use cases ('C') is what I have most difficulties with. It clearly is about 'VCS <-> Roundup integration', but it's in the other direction. And thus, this doesn't seem to concern Roundup itself, or does it ? Presumably, everything needed already exists. We now have an XMLRPC interface, so it is possible to write (remote) Roundup clients. I think writing such a 'Roundup client library' would be useful, but I think somewhat out of scope with respect to this GSoC project. > USE CASE D: > > Antoine is merging lots of branches and applying lots of patches to > his local branch of the Python repository. He goes to each issue with > a patch/branch and tells people whether their patches apply cleanly, > just to avoid merge issues in the main branch. While he could use > Mercurial Patch Queues (mq), Roundup would need to learn to both > listen to and to submit patches to mq in order to completely replace > Antoine's work with automated tools. Having a quick 'check if this > patch applies cleanly' button would make triaging issues much easier. > Let's drill down here a little, as this may be more than one use-case: D1: provide an interface through which users can attempt to apply a submitted patch to a given branch. This requires a 'working copy' of the code (Roundup may be configured to create that if it doesn't exist yet), and a simple interface to the 'patch' tool. If we create a new 'Patch' type, it may include metadata that help Roundup to figure out what repo / branch URL to use to fetch the code from against which the patch is to be applied against. D2: While D1 is about applying patches, this may not be enough to figure out whether the patch is good. The user may also want to attempt to build the code (and even run tests). While I believe this to be outside the realm of a bug / patch tracker, I can see good use in integration with external build tools. In fact, it is a use case such as this that I had in mind when alluding to bridging with buildbot. I think it would be nice to think a little more about it, without actually implementing it. Just making it possible for future work in this direction ought to be enough. > USE CASE E: > > David is checking the python-commits list for avoiding bad code from > landing and nesting in the Python code base. He only sees the patches: > while informative, it requires a bit of mental gymanstics to see how > it would merge with the surrounding code. Whenever he thinks some code > has tabs and spaces or lines that are too long, he runs pylint and > reindent.py locally. He can only raise concerns about the code after > it lands on the main repository. It should be easier to see the code > changes in context. Having a way to check the code for mistakes in the > tracker would make David's life a lot easier. > Yes, if we create a new 'Patch' type, we also want to add to the templating module to provide rendering functions that can render the patch in different ways (diffs between the target branch, with and without the patch applied, etc.) Again, 'checking the code for mistakes' can be a big task, and so goes into the D2 case above. > USE CASE F: > > Van Lindberg is concerned about code submissions from non-core > developers: how can the PSF re-license this code in the future without > talking to each contributor, whether the PSF is safe from litigation > based on copyrights of these contributions and related questions are > always bugging him. While the PSF has Contributor Agreements exactly > for these cases, it would be great to have the issue tracker and the > VCS talk to each other, so they could ask contributors to sign (or > declare to have already signed) the CAs when necessary. > Sounds good. Nothing new here (besides what we already described in the use cases above). > USE CASE G: Use Transplant/Patch Branch to generate patches from > branches linked from Roundup. > This sounds like accessing a VCS-specific GUI through Roundup. While I can see this being useful, I'm not sure how much this relates directly to Roundup. > USE CASE J: > > Integrate the code/commits navigation interface with Roundup, so > changesets, branches, etc., can be easily linked/browsed (starting) > from the Roundup UI and issues can be created/linked to commits > (starting) from the navigation tool UI. > Similarly here. > USE CASE K: For a given issue, add per patched file links for RSS logs > (see http://selenic.com/hg/rss-log/tip/mercurial/hgweb/hgweb_mod.py). > This sounds like a small addition to case A above: Once I collect (references to) changesets in an issue, I can use them to backlink into a repository UI. > USE CASE M: Besides links to files, allow adding links to files at > given versions/tags/branchs, links to tarballs and easy to clone links > to branches and repositories. > Right. Easy enough once our schema knows about 'repo', 'branch' and 'revision' (We still need a per VCS definition of how to construct such a link from this tuple. > USE CASE V: > > Handle small branches (and maybe suggest using them for small > patches?) generated using the convert extension with --filemap. > I have no idea what this is about, sorry. :-) > ============== > > Improvements to the use cases will follow on the wiki page: > http://www.roundup-tracker.org/cgi-bin/moin.cgi/GSoC09 > Great, thanks. I suggest that we keep discussions here on the list(s), and then capture any outcome on that (and related) wiki page(s). > Thanks for your attention and time. > Thanks for working on this ! Stefan -- ...ich hab' noch einen Koffer in Berlin... From amk at amk.ca Thu Jun 4 22:44:09 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 4 Jun 2009 16:44:09 -0400 Subject: [Python-Dev] Google Wave as a developer communication tool In-Reply-To: References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> <87tz2wo1dx.fsf@benfinney.id.au> Message-ID: <20090604204409.GA15415@amk-desktop.matrixgroup.net> On Thu, Jun 04, 2009 at 01:15:16PM -0400, Terry Reedy wrote: > marked as having new content. At your leisure, you open it (or perhaps > you have marked it 'download updates in background'). That that takes > longer with a slow connection is no different than with other text > streams. If you type in a comment, even 1200 baud upsteam is fast > enough. Where do you get 'punitive' in this? I visited my parents at Christmas. They live in a rural area, don't have great phone lines, and my dad's computer ends up getting about a 21Kb modem connection (not even 38.4!). I discovered that great swaths of the Web 2.0 world were effectively unusable for me; after login, the Twitter home page took 3 minutes to display. Logging into my bank was a tortuous 10 minute process. I never succeeded in getting into Facebook at all. Many pages don't render until they're completely downloaded. The little AJAXy update that adds an extra second or two on a fast connection becomes shockingly painful on a slow connection. The JS files are organized Unless Google Wave is written with attention to such slow connections (which I doubt -- such users are pretty rare, after all), I would assume it will be unusable. The bandwidth is enough for a character by character stream, but web browser apps impose many overheads atop that stream. --amk From jake at youtube.com Thu Jun 4 23:04:00 2009 From: jake at youtube.com (Jake McGuire) Date: Thu, 4 Jun 2009 14:04:00 -0700 Subject: [Python-Dev] Google Wave as a developer communication tool In-Reply-To: <20090604204409.GA15415@amk-desktop.matrixgroup.net> References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> <87tz2wo1dx.fsf@benfinney.id.au> <20090604204409.GA15415@amk-desktop.matrixgroup.net> Message-ID: <77c780b40906041404k737802few774556ca294a3646@mail.gmail.com> Google Wave is also still in tightly restricted beta. Gmail went through a long invite-only period. Until we have an idea of how long it will be until basically all python developers who want a Wave account can get one, it doesn't make sense to talk about using it for python development, IMO. -jake From db3l.net at gmail.com Thu Jun 4 23:23:24 2009 From: db3l.net at gmail.com (David Bolen) Date: Thu, 04 Jun 2009 17:23:24 -0400 Subject: [Python-Dev] Google Wave as a developer communication tool References: <20090602175600.22176.619128085.divmod.quotient.1116@henry.divmod.com> <20090603174151.12555.1359697426.divmod.xquotient.12290@weber.divmod.com> <87vdncq417.fsf_-_@benfinney.id.au> <20090604122300.GA3768@panix.com> Message-ID: Aahz writes: > It sounds like Wave requires a high-powered browser, similar to Google > Maps. That makes me -1 because I want to continue using Lynx. I'm not sure - I think you can implement your own choices at different points. What's interesting to me so far is less the current UI/flashiness (drag 'n drop photos, real-time keystrokes, etc...) they are showing, than the openness of the protocol and implementation, opening the door to all sorts of potential uses and/or integrations with existing systems. Other than convenience for interactive use, and as a reference client, I'm not sure how critical a browser really is to the system. Fundamentally, wave seems to be rich, versioned (or at least time tracked), content management system with well defined server to server and server to client communication protocols, including conflict resolution. They even brought up VCS-like concepts (currently unimplemented) of having waves branch and evolve independently, fold changes back into a parent wave and then re-distribute down to the other children over time, which points to a reasonable central data structure. For example, if you jump to about 1:08:13 in the presentation you can see a more textual display implemented by an independent wave server/client - almost newsreader like. I presume such a client has to make similar decisions as to display of non-textual data as Lynx has to with the web. Or if you embed the wave in a web page, it could still be handled by text browsers (the blog integration at 20:43 for example) or perhaps with RSS. Server robots could help too - a robot participating in the wave could produce a more text-friendly reader version, perhaps published elsewhere on the web or in a different system. They demoed, for example, integrating with Google Code's bug tracking back end at 1:02:30 where comments in the wave were reflected into the bug comments (though the reverse direction wasn't implemented yet), or even the twitter integration at 57:53. -- David From dalcinl at gmail.com Thu Jun 4 23:44:06 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 4 Jun 2009 18:44:06 -0300 Subject: [Python-Dev] Serious regression in doctest in Py3.1rc1 In-Reply-To: References: Message-ID: http://bugs.python.org/issue6195 (with patch) On Wed, Jun 3, 2009 at 8:26 AM, Amaury Forgeot d'Arc wrote: > Hello, > > 2009/6/3 Stefan Behnel : >> Hi, >> >> I can't currently file a bug report on this, but I was told by Lisandro >> Dalc?n that there is a serious problem with the doctest module in Py3.1rc1. >> In Cython, we use doctests to test the compiler in that we compile a >> Python/Cython module with doctests into a C module and then run doctest on >> the imported extension module. >> >> >From the error report it seems to me that doctest is now trying to read the >> module itself through linecache for some reason, which horribly fails for a >> binary module. >> >> Could someone please look into this? I'll open up a bug report tomorrow >> unless someone beats me to it. > > I don't have the time either, but the problem looks very similar to > http://bugs.python.org/issue4050 > The fix was to replace: > ? ? file = inspect.getsourcefile(object) or inspect.getfile(object) > was replaced with > ? ? file = inspect.getsourcefile(object) > > -- > Amaury Forgeot d'Arc > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From ben+python at benfinney.id.au Fri Jun 5 01:52:39 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 05 Jun 2009 09:52:39 +1000 Subject: [Python-Dev] Roundup improvements use-cases References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com> <4A282A41.8040701@sympatico.ca> Message-ID: <874ouvo8u0.fsf@benfinney.id.au> Stefan Seefeld writes: > While in theory [implementing a hook for closing bugs via VCS commit > messages] is all up for customization, I think it would be nice to set > up some conventions to guide users in how to do this (the 'token > grammar', mostly), just for convenience. A reference implementation that I think works quite well is the system used by the Debian project for detecting, from the changelog entries, which bugs to close when a particular release of a package is uploaded. Note also that this system doesn't just close the bug, it makrs it as fixed in a specific version of the package: the package version that included the close request in its changelog entry. It would be valuable if analogous behaviour could occur with Roundup, which requires allowing for it in the specification of the workflow. > This is a very interesting use case, driving us at the core of the > 'VCS integration'. We need to customize a schema to provide a notion > for 'repository' and 'branch', and also for 'patch'. Then we need to > somehow endow the patch type with knowledge of what it means to be > 'applied' to a branch, etc. (more on this below). All this would apply to the above workflow for ?mark that bug as fixed by this specific branch?. (good sigmonster, have a cookie) -- \ ?That's all very good in practice, but how does it work in | `\ *theory*?? ?anonymous | _o__) | Ben Finney From ben+python at benfinney.id.au Fri Jun 5 02:06:07 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 05 Jun 2009 10:06:07 +1000 Subject: [Python-Dev] Roundup improvements use-cases References: <79957db20906041017u67232014r21f87ddea5304d50@mail.gmail.com> <4A282A41.8040701@sympatico.ca> Message-ID: <87zlcnmtn4.fsf@benfinney.id.au> Stefan Seefeld writes: > This set of use cases ('C') is what I have most difficulties with. It > clearly is about 'VCS <-> Roundup integration', but it's in the other > direction. And thus, this doesn't seem to concern Roundup itself, or > does it ? Presumably, everything needed already exists. We now have an > XMLRPC interface, so it is possible to write (remote) Roundup clients. I > think writing such a 'Roundup client library' would be useful, but I > think somewhat out of scope with respect to this GSoC project. Note that there are several attempts in the wild at ?BTS database as versioned data in the VCS repository?, allowing the tightest integration between the two and ensuring that VCS commits and BTS updates are inherently tied to each other. An article at Linux Weekly News describes these systems. It might be instructive for those looking to tightly integrate VCS and BTS to seek inspiration from these designs. -- \ ?The userbase for strong cryptography declines by half with | `\ every additional keystroke or mouseclick required to make it | _o__) work.? ?Carl Ellison | Ben Finney From solipsis at pitrou.net Fri Jun 5 09:19:51 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 5 Jun 2009 07:19:51 +0000 (UTC) Subject: [Python-Dev] py3k buildbots Message-ID: Hello Only one of the py3k buildbots seems up: http://www.python.org/dev/buildbot/3.x.stable/ cheers Antoine. From theller at ctypes.org Fri Jun 5 11:26:34 2009 From: theller at ctypes.org (Thomas Heller) Date: Fri, 05 Jun 2009 11:26:34 +0200 Subject: [Python-Dev] py3k buildbots In-Reply-To: References: Message-ID: Antoine Pitrou schrieb: > Hello > > Only one of the py3k buildbots seems up: > http://www.python.org/dev/buildbot/3.x.stable/ Maybe they are waiting for the snakebite network ;-) (what's up with it, anyway?). I've restarted mine (x86 osx.5), but it isn't in the stable list... -- Thanks, Thomas From ncoghlan at gmail.com Fri Jun 5 11:31:03 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 05 Jun 2009 19:31:03 +1000 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> Message-ID: <4A28E5D7.2030402@gmail.com> Dirkjan Ochtman wrote: >> Essentially, pcbuild.sln is a binary file, and should be treated as >> such. Maybe it's an error in the Subversion setup that it's treated as >> text at all... > > Yes, it certainly seems that way. Except it isn't a binary file - it's a text file with CRLF line endings. Why would we throw away the chance to get meaningful diffs when Subversion can easily get the line endings correct? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From p.f.moore at gmail.com Fri Jun 5 15:14:52 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 5 Jun 2009 14:14:52 +0100 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: <4A28E5D7.2030402@gmail.com> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> <4A28E5D7.2030402@gmail.com> Message-ID: <79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com> 2009/6/5 Nick Coghlan : > Dirkjan Ochtman wrote: >>> Essentially, pcbuild.sln is a binary file, and should be treated as >>> such. Maybe it's an error in the Subversion setup that it's treated as >>> text at all... >> >> Yes, it certainly seems that way. > > Except it isn't a binary file - it's a text file with CRLF line endings. > Why would we throw away the chance to get meaningful diffs when > Subversion can easily get the line endings correct? Fair point - I hadn't thought of it in that context. I wonder if Mercurial can treat it as a specialised form of binary file with custom diff/merge behaviour (which just happens to be "treat as text")? Of course, if that also requires client-side configuration, it's no better solution than win32text. I've a feeling I've seen discussions of versioned metadata (things like file properties) in Mercurial at some point in the past, but my Google skills are failing me at the moment... Paul. From status at bugs.python.org Fri Jun 5 18:07:16 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 5 Jun 2009 18:07:16 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090605160716.09A8578632@psf.upfronthosting.co.za> ACTIVITY SUMMARY (05/29/09 - 06/05/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2207 open (+37) / 15822 closed (+27) / 18029 total (+64) Open issues with patches: 868 Average duration of open issues: 653 days. Median duration of open issues: 403 days. Open Issues Breakdown open 2178 (+37) pending 29 ( +0) Issues Created Or Reopened (66) _______________________________ string module requires bytes type for maketrans, but calling met 05/31/09 CLOSED http://bugs.python.org/issue5675 reopened rhettinger IDLE - an extension to clear the shell window 05/29/09 http://bugs.python.org/issue6143 created serwy [IDLE] UnicodeDecodeError when invoking force-open-completions 05/29/09 http://bugs.python.org/issue6144 created jamesie distutils.extension.read_setup_file misinterprets -C switch 05/29/09 http://bugs.python.org/issue6145 created donlorenzo patch markup error in Doc/library/rlcompleter.rst 05/29/09 CLOSED http://bugs.python.org/issue6146 created Trundle patch multithreading.Pool.map() crashes Windows computer 05/30/09 CLOSED http://bugs.python.org/issue6147 created ac.james Help well execute code it is called on 05/30/09 CLOSED http://bugs.python.org/issue6148 created imiers1 WeakValueDictionary constructor ported to Python 3.0 incorrectly 05/30/09 CLOSED http://bugs.python.org/issue6149 created zzzeek patch test_unicode fails in wide unicode build 05/30/09 CLOSED http://bugs.python.org/issue6150 created hagen Make PyDescr_COMMON conform to standard C 05/30/09 http://bugs.python.org/issue6151 created alexandre.vassalotti patch, patch, needs review Parallel regression testing 05/31/09 http://bugs.python.org/issue6152 created pitrou patch email parsing - Rare Failure 05/31/09 CLOSED http://bugs.python.org/issue6153 created WinstonEwert Python 3.1 rc1 will not build on OS X 10.5.7 with macports libin 05/31/09 http://bugs.python.org/issue6154 created brett.cannon patch "logging" example uses unavailable cPickle module 05/31/09 CLOSED http://bugs.python.org/issue6155 created mnewman Error compiling valid regex 05/31/09 http://bugs.python.org/issue6156 created Eloff patch Tkinter.Text: changes for bbox, debug, and edit methods. 05/31/09 http://bugs.python.org/issue6157 created gpolo patch test_aifc failing on 32bit windows in python 3.1 rc 1 05/31/09 CLOSED http://bugs.python.org/issue6158 created rhettinger patch Tkinter.PanedWindow: docstring fixes, change in paneconfigure an 05/31/09 http://bugs.python.org/issue6159 created gpolo patch Tkinter.Spinbox: fix for bbox and removed some uninteresting ret 06/01/09 http://bugs.python.org/issue6160 created gpolo patch httplib: HTTPResponse not storing response body 06/01/09 CLOSED http://bugs.python.org/issue6161 created Guthur What should happen to signals when the main thread dies? 06/01/09 http://bugs.python.org/issue6162 created eggy [HP-UX] ld: Unrecognized argument: +s -L 06/01/09 http://bugs.python.org/issue6163 created srid patch [AIX] Patch to correct the AIX C/C++ linker argument used for 'r 06/01/09 http://bugs.python.org/issue6164 created srid patch strftime incorrectly processes DST flag when passed time touples 06/01/09 http://bugs.python.org/issue6165 created jonathan.cervidae encoding error for 'setup.py --author' when read via subprocess 06/01/09 http://bugs.python.org/issue6166 created srid Tkinter.Scrollbar: the activate method needs to return a value, 06/01/09 http://bugs.python.org/issue6167 created gpolo patch Missing Shell menu in Linux IDLE 06/01/09 http://bugs.python.org/issue6168 created sirgimp Important comparison bug in ipaddr 06/01/09 CLOSED http://bugs.python.org/issue6169 created shields Mac 'make frameworkinstall' error: [...]/Python.framework/Versio 06/02/09 http://bugs.python.org/issue6170 created srid Class Browser selection in Ubuntu 06/02/09 http://bugs.python.org/issue6171 created sirgimp 'make framework...' fails on Mac ([...]/bin/pythonw3.1: No such 06/02/09 CLOSED http://bugs.python.org/issue6172 created srid Minor typo in socket.py 06/02/09 CLOSED http://bugs.python.org/issue6173 created ptn patch What's new in 2.6, wrong indentation in code sample 06/02/09 CLOSED http://bugs.python.org/issue6174 created joe.amenta patch inet_aton documentation kind of lies 06/02/09 CLOSED http://bugs.python.org/issue6175 created dsvensson Reference to a wrong version of flock's documentation 06/02/09 CLOSED http://bugs.python.org/issue6176 created pakal unittest.main testRunner default argument changed from None 06/02/09 CLOSED http://bugs.python.org/issue6177 created michael.foord patch, easy Core error in Py_EvalFrameEx 2.6.2 06/02/09 http://bugs.python.org/issue6178 created tsavannah Documentation of for statement in Reference says that range() re 06/02/09 CLOSED http://bugs.python.org/issue6179 created MLModel Tkinter.Entry: fix for xview and some doc clarifications 06/02/09 http://bugs.python.org/issue6180 created gpolo patch Tkinter.Listbox several minor issues 06/02/09 http://bugs.python.org/issue6181 created gpolo patch Remove ipaddr library from py3k before 3.1rc2 06/03/09 CLOSED http://bugs.python.org/issue6182 created gregory.p.smith test_time fails on VC6 06/03/09 CLOSED http://bugs.python.org/issue6183 created ocean-city patch py3k build gets spurious errors from libinstall target compile o 06/03/09 CLOSED http://bugs.python.org/issue6184 reopened nad 2to3 misses buffer slicing 06/03/09 CLOSED http://bugs.python.org/issue6185 created joe.amenta patch test_thread occasionally reports unhandled exceptions on OS X 06/03/09 http://bugs.python.org/issue6186 created nad Improvement in doc of "Extending and Embedding the Python Interp 06/03/09 http://bugs.python.org/issue6187 created blubdiebla Error Evaluating float(x) ** float(y) 06/03/09 CLOSED http://bugs.python.org/issue6188 created Zero subprocess module not compatible with python 2.5 06/03/09 http://bugs.python.org/issue6189 created itsadok patch Tutorial 3.1.2 (Strings) has duplicate lines 06/03/09 CLOSED http://bugs.python.org/issue6190 created cofi patch HTMLParser attribute parsing - 2 test cases when it fails 06/04/09 http://bugs.python.org/issue6191 reopened momat add disable_nagle_algorithm to SocketServer.TCPServer 06/04/09 http://bugs.python.org/issue6192 created krisvale patch, patch, easy urllib: ... IOError: ... unknown url type: 'c' 06/04/09 CLOSED http://bugs.python.org/issue6193 created srid fcntl footnote about O_SHLOCK and O_EXLOCK is misleading 06/04/09 http://bugs.python.org/issue6194 created r.david.murray easy Serious regression in doctest in Py3.1rc1 06/04/09 http://bugs.python.org/issue6195 created dalcinl patch tarfile.extractall(readaccess=True) 06/04/09 http://bugs.python.org/issue6196 created srid test__locale fails with RADIXCHAR on Windows 06/05/09 CLOSED http://bugs.python.org/issue6197 created abbeyj patch test_float fails on Windows 06/05/09 CLOSED http://bugs.python.org/issue6198 created abbeyj test_unittest fails on Windows 06/05/09 CLOSED http://bugs.python.org/issue6199 created abbeyj patch test_winreg fails 06/05/09 http://bugs.python.org/issue6201 created abbeyj Obsolete default file encoding "mac-roman" on OS X, not influenc 06/05/09 http://bugs.python.org/issue6202 created nad 3.x locale does not default to C, contrary to the documentation 06/05/09 http://bugs.python.org/issue6203 created nad Missing reference in section 4.6 to chapter on classes 06/05/09 http://bugs.python.org/issue6204 created cofi patch sdist doesn't include data_files 06/05/09 http://bugs.python.org/issue6205 created purpleidea Correct a trivial typo introduced by r73238. 06/05/09 http://bugs.python.org/issue6206 created vshenoy patch Simple For-Loops 06/05/09 http://bugs.python.org/issue6207 created gabrielkfl Function-level import in os triggering an threaded import deadlo 06/04/09 http://bugs.python.org/issue1590864 reopened brett.cannon Issues Now Closed (67) ______________________ Load tests from path (patch included) 615 days http://bugs.python.org/issue1207 michael.foord patch S_unpack_from() Read Access Violation 421 days http://bugs.python.org/issue2590 amaury.forgeotdarc Py3k warn on unicode escapes in raw strings 401 days http://bugs.python.org/issue2713 amaury.forgeotdarc patch example code does not work 392 days http://bugs.python.org/issue2742 fogus patch time.strftime() always decodes result with UTF-8 356 days http://bugs.python.org/issue3061 loewis patch pkg-config support 285 days http://bugs.python.org/issue3585 loewis patch, needs review base64.encodestring does not actually accept strings 288 days http://bugs.python.org/issue3613 georg.brandl patch Python 2.6 can't read sets pickled with Python 3.0 283 days http://bugs.python.org/issue3675 alexandre.vassalotti patch traceback.format_exception_only() misplaces the caret for certai 283 days http://bugs.python.org/issue3684 georg.brandl patch bsddb not completely removed 271 days http://bugs.python.org/issue3791 georg.brandl patch, patch select.epoll calling register with the same fd fails 261 days http://bugs.python.org/issue3848 r.david.murray patch Add Google's ipaddr.py to the stdlib 251 days http://bugs.python.org/issue3959 belopolsky Long jumps with frame_setlineno 178 days http://bugs.python.org/issue4547 amaury.forgeotdarc patch, needs review Refcount error and file descriptor leaks in pwd, grp modules 142 days http://bugs.python.org/issue4873 loewis patch Subversion problem with PythonLauncher after building 3.0 or 3.1 109 days http://bugs.python.org/issue5267 ronaldoussoren profile and cProfile do not report C functions called with keywo 99 days http://bugs.python.org/issue5330 pitrou patch TypeError when '\x00' in docstring 98 days http://bugs.python.org/issue5373 alexandre.vassalotti patch Locale-based date formatting crashes on non-ASCII data 65 days http://bugs.python.org/issue5562 loewis test_memoryio fails for py3k on windows 65 days http://bugs.python.org/issue5645 jaraco patch OS X Installer: do not install obsolete documentation within Pyt 64 days http://bugs.python.org/issue5648 ronaldoussoren string module requires bytes type for maketrans, but calling met 0 days http://bugs.python.org/issue5675 georg.brandl Segfault when loading not recompiled module 51 days http://bugs.python.org/issue5735 amaury.forgeotdarc patch, needs review xmlrpclib loads invalid documents 49 days http://bugs.python.org/issue5767 georg.brandl patch test_asynchat fails on Mac OSX 46 days http://bugs.python.org/issue5798 MrJean1 patch strftime fails in non UTF-8 locale 27 days http://bugs.python.org/issue5903 loewis Multi-with patch 26 days http://bugs.python.org/issue5922 Merwok patch PyList_GetSlice does not indicate negative ranges dont work as i 27 days http://bugs.python.org/issue5967 georg.brandl patch Test discovery for unittest 18 days http://bugs.python.org/issue6001 michael.foord patch, patch, needs review test_distutils leaves a 'foo' file behind in the cwd 16 days http://bugs.python.org/issue6022 rpetrov Python fails to build with Subversion 1.7 7 days http://bugs.python.org/issue6094 benjamin.peterson patch Encoded surrogate characters on command line not escaped in sys. 5 days http://bugs.python.org/issue6097 loewis patch Expanding arrays inside other arrays 5 days http://bugs.python.org/issue6100 tjreedy When the package has non-ascii path and .pyc file, we cannot imp 5 days http://bugs.python.org/issue6102 tjreedy IDLE rendering issue with oriental characters on OSX 7 days http://bugs.python.org/issue6109 benjamin.peterson Fix O(n**2) performance problem in socket._fileobject 7 days http://bugs.python.org/issue6117 krisvale patch, patch, easy, needs review Typo in email.base64mime 1 days http://bugs.python.org/issue6139 georg.brandl patch missing first argument on subprocess.Popen w/ executable 0 days http://bugs.python.org/issue6141 r.david.murray patch markup error in Doc/library/rlcompleter.rst 0 days http://bugs.python.org/issue6146 georg.brandl patch multithreading.Pool.map() crashes Windows computer 2 days http://bugs.python.org/issue6147 benjamin.peterson Help well execute code it is called on 0 days http://bugs.python.org/issue6148 georg.brandl WeakValueDictionary constructor ported to Python 3.0 incorrectly 0 days http://bugs.python.org/issue6149 pitrou patch test_unicode fails in wide unicode build 1 days http://bugs.python.org/issue6150 loewis email parsing - Rare Failure 2 days http://bugs.python.org/issue6153 amaury.forgeotdarc "logging" example uses unavailable cPickle module 0 days http://bugs.python.org/issue6155 georg.brandl test_aifc failing on 32bit windows in python 3.1 rc 1 0 days http://bugs.python.org/issue6158 loewis patch httplib: HTTPResponse not storing response body 0 days http://bugs.python.org/issue6161 Guthur Important comparison bug in ipaddr 0 days http://bugs.python.org/issue6169 gregory.p.smith 'make framework...' fails on Mac ([...]/bin/pythonw3.1: No such 0 days http://bugs.python.org/issue6172 ronaldoussoren Minor typo in socket.py 0 days http://bugs.python.org/issue6173 marketdickinson patch What's new in 2.6, wrong indentation in code sample 2 days http://bugs.python.org/issue6174 georg.brandl patch inet_aton documentation kind of lies 2 days http://bugs.python.org/issue6175 georg.brandl Reference to a wrong version of flock's documentation 2 days http://bugs.python.org/issue6176 georg.brandl unittest.main testRunner default argument changed from None 0 days http://bugs.python.org/issue6177 michael.foord patch, easy Documentation of for statement in Reference says that range() re 0 days http://bugs.python.org/issue6179 benjamin.peterson Remove ipaddr library from py3k before 3.1rc2 2 days http://bugs.python.org/issue6182 rhettinger test_time fails on VC6 0 days http://bugs.python.org/issue6183 ocean-city patch py3k build gets spurious errors from libinstall target compile o 1 days http://bugs.python.org/issue6184 benjamin.peterson 2to3 misses buffer slicing 1 days http://bugs.python.org/issue6185 benjamin.peterson patch Error Evaluating float(x) ** float(y) 0 days http://bugs.python.org/issue6188 marketdickinson Tutorial 3.1.2 (Strings) has duplicate lines 0 days http://bugs.python.org/issue6190 georg.brandl patch urllib: ... IOError: ... unknown url type: 'c' 0 days http://bugs.python.org/issue6193 r.david.murray test__locale fails with RADIXCHAR on Windows 0 days http://bugs.python.org/issue6197 ocean-city patch test_float fails on Windows 0 days http://bugs.python.org/issue6198 eric.smith test_unittest fails on Windows 0 days http://bugs.python.org/issue6199 michael.foord patch SRE: (?flag:...) is not supported 2912 days http://bugs.python.org/issue433028 georg.brandl Update pickle docs to describe format of persistent IDs 1838 days http://bugs.python.org/issue956303 alexandre.vassalotti easy hide tests from TestProgram 1408 days http://bugs.python.org/issue1244929 michael.foord patch Top Issues Most Discussed (10) ______________________________ 33 Python 3.1 rc1 will not build on OS X 10.5.7 with macports libi 5 days open http://bugs.python.org/issue6154 32 improved allocation of PyUnicode objects 12 days open http://bugs.python.org/issue1943 27 Add Google's ipaddr.py to the stdlib 251 days closed http://bugs.python.org/issue3959 17 Mac 'make frameworkinstall' error: [...]/Python.framework/Versi 4 days open http://bugs.python.org/issue6170 15 pydoc reports misleading failure if target module raises an Imp 113 days open http://bugs.python.org/issue5230 14 Make pickle generated by Python 3.x compatible with 2.x and vic 8 days open http://bugs.python.org/issue6137 10 Distutils doesn't remove .pyc files 7 days open http://bugs.python.org/issue6142 10 LOAD_CONST followed by LOAD_ATTR can be optimized to just be a 9 days open http://bugs.python.org/issue6133 9 test_float fails on Windows 0 days closed http://bugs.python.org/issue6198 8 test_asynchat fails on Mac OSX 46 days closed http://bugs.python.org/issue5798 From db3l.net at gmail.com Fri Jun 5 21:38:32 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 05 Jun 2009 15:38:32 -0400 Subject: [Python-Dev] py3k buildbots References: Message-ID: Antoine Pitrou writes: > Only one of the py3k buildbots seems up: > http://www.python.org/dev/buildbot/3.x.stable/ Strange - everything looks good on my buildbot end (XP-4), including an established TCP session back to dinsdale. Not sure why the master thinks it's offline. Although I'm pretty sure I've seen this a handful of times in the past too. The fact that it also shows my FreeBSD buildbot offline (which I rarely have to touch compared to the Windows version and they're on the same host machine) seems to imply some wierd communication breakdown. I've restarted my buildbots. -- David From theller at ctypes.org Fri Jun 5 21:52:46 2009 From: theller at ctypes.org (Thomas Heller) Date: Fri, 05 Jun 2009 21:52:46 +0200 Subject: [Python-Dev] py3k buildbots In-Reply-To: References: Message-ID: David Bolen schrieb: > Antoine Pitrou writes: > >> Only one of the py3k buildbots seems up: >> http://www.python.org/dev/buildbot/3.x.stable/ > > Strange - everything looks good on my buildbot end (XP-4), including > an established TCP session back to dinsdale. Not sure why the master > thinks it's offline. Although I'm pretty sure I've seen this a handful > of times in the past too. > > The fact that it also shows my FreeBSD buildbot offline (which I > rarely have to touch compared to the Windows version and they're on > the same host machine) seems to imply some wierd communication > breakdown. > > I've restarted my buildbots. Well, here is an excerpt from the twistd.log of my osx buildbot, in case someone want to look at it. I restarted it this morning, at 2009/06/05 11:21 +0200: 2009/06/01 20:20 +0200 [Broker,client] SlaveBuilder._ackFailed: SlaveBuilder.sendUpdate 2009/06/01 20:20 +0200 [Broker,client] SlaveBuilder._ackFailed: SlaveBuilder.sendUpdate 2009/06/01 20:20 +0200 [Broker,client] SlaveBuilder._ackFailed: SlaveBuilder.sendUpdate 2009/06/01 20:20 +0200 [Broker,client] lost remote 2009/06/01 20:20 +0200 [Broker,client] lost remote 2009/06/01 20:20 +0200 [Broker,client] lost remote 2009/06/01 20:20 +0200 [Broker,client] lost remote 2009/06/01 20:20 +0200 [Broker,client] lost remote step 2009/06/01 20:20 +0200 [Broker,client] stopCommand: halting current command 2009/06/01 20:20 +0200 [Broker,client] command interrupted, killing pid 21579 2009/06/01 20:20 +0200 [Broker,client] trying os.kill(-pid, 9) 2009/06/01 20:20 +0200 [Broker,client] signal 9 sent successfully 2009/06/01 20:20 +0200 [Broker,client] will retry in 3 seconds 2009/06/01 20:20 +0200 [Broker,client] Stopping factory 2009/06/01 20:20 +0200 [-] Unhandled Error Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/log.py", line 48, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/log.py", line 33, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) --- --- File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/internet/selectreactor.py", line 139, in _doReadOrWrite why = getattr(selectable, method)() File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/internet/abstract.py", line 113, in doWrite l = self.writeSomeData(self.dataBuffer) File "/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/twisted/internet/process.py", line 879, in writeSomeData return os.write(self.fd, data) exceptions.OSError: [Errno 5] Input/output error 2009/06/01 20:20 +0200 [-] command finished with signal 9, exit code None 2009/06/01 20:20 +0200 [-] SlaveBuilder.commandComplete None 2009/06/01 20:20 +0200 [-] Starting factory 2009/06/05 11:21 +0200 [-] Received SIGTERM, shutting down. 2009/06/05 11:21 +0200 [Broker,client] Stopping factory 2009/06/05 11:21 +0200 [-] Main loop terminated. 2009/06/05 11:21 +0200 [-] Server Shut Down. 2009/06/05 11:21 +0200 [-] Log opened. 2009/06/05 11:21 +0200 [-] twistd 2.5.0 (/System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python 2.5.1) starting up 2009/06/05 11:21 +0200 [-] reactor class: 2009/06/05 11:21 +0200 [-] Loading buildbot.tac... -- Thanks, Thomas From dirkjan at ochtman.nl Fri Jun 5 22:48:15 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Fri, 5 Jun 2009 22:48:15 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial Message-ID: So, a while ago Martin von L?wis asked who would champion the migration to Mercurial, and I volunteered. He asked me to produce a PEP outlining which steps to take, which I've been working on. The PEP is numbered 385, and is just about ready for your review. I'd really welcome any sort of feedback or questions about Mercurial and surrounding tools relevant to the migration project (I'm happy to answer other questions as well, but maybe python-dev is not the appropriate forum for that -- instead, try mercurial at selenic.com and/or #mercurial on freenode). http://www.python.org/dev/peps/pep-0385/ In particular, you may want to review (a) branches that you care about and whether they would be migrated under the current proposal, (b) non-release tags that you think are useful and (c) the author map (check if it has a correct email address for you and people you know). Thanks to Antoine, Benjamin, Georg and Martin for early feedback. Cheers, Dirkjan From benjamin at python.org Fri Jun 5 22:58:19 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 5 Jun 2009 15:58:19 -0500 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: References: Message-ID: <1afaf6160906051358n25a833f0vc1be58a3598be88d@mail.gmail.com> 2009/6/5 Dirkjan Ochtman : > > In particular, you may want to review (a) branches that you care about > and whether they would be migrated under the current proposal, (b) > non-release tags that you think are useful and (c) the author map > (check if it has a correct email address for you and people you know). Thanks very much for doing this! The PEP should talk about tracker migration. ie. how to handle specifying revisions on multiple clones if we choose that path. -- Regards, Benjamin From solipsis at pitrou.net Sat Jun 6 00:35:59 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 5 Jun 2009 22:35:59 +0000 (UTC) Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial References: Message-ID: Dirkjan Ochtman ochtman.nl> writes: > > http://www.python.org/dev/peps/pep-0385/ ? [cloned branches] makes it easier to distinguish branches, at the expense of requiring more disk space on the client. ? This is a bit misleading. Actually, by separating branches into distinct repositories, you can save quite a bit of repository space. It really depends on the structure of the project and your own workflow. ? This reflects that the default branch in hg is called 'default' instead of Subversion's 'trunk', and reflects the proposed new tag format. ? If we use several distinct repositories, the default branch should instead mirror the repository name (like what is done in the current hg mirrors: "trunk", "py3k", etc.). Regards Antoine. From techtonik at gmail.com Sat Jun 6 01:59:24 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Sat, 6 Jun 2009 02:59:24 +0300 Subject: [Python-Dev] Roundup keywords for bug tracking Message-ID: It is impossible to edit roundup keywords and this takes away the flexibility in selecting bugs related to a module/function/test or some other aspect of development. For example, I need to gather all subprocess bugs in one query and things that won't be fixed in deprecated os.popen() into another. In Trac I would use "subprocess" and "os.popen" keywords. On ohloh I would add similar tags (if bugs were software) without, but I can't do anything about Python roundup. Is there any reason for such restriction? --anatoly t. From tjreedy at udel.edu Sat Jun 6 02:42:14 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 05 Jun 2009 20:42:14 -0400 Subject: [Python-Dev] Roundup keywords for bug tracking In-Reply-To: References: Message-ID: anatoly techtonik wrote: > It is impossible to edit roundup keywords and this takes away the > flexibility in selecting bugs related to a module/function/test or > some other aspect of development. For example, I need to gather all > subprocess bugs in one query At the moment, search for 'subprocess' in text and component = library returns 53 open issues. A quick scan of the titles suggests that about 40 are for the subprocess module. So changing the search to 'subprocess' in title might be better. and things that won't be fixed in > deprecated os.popen() into another. In Trac I would use "subprocess" > and "os.popen" keywords. On ohloh I would add similar tags (if bugs > were software) without, but I can't do anything about Python roundup. > Is there any reason for such restriction? If every module and built-in and keyword were added to the drop-down list, it would be pretty long. But I agree that better sorting could help. G. Polo, for one, made his own app to download canned search results (for tk and idle) and store them in a local db, where he could add annotations. tjr From ajaksu at gmail.com Sat Jun 6 03:07:44 2009 From: ajaksu at gmail.com (Daniel Diniz) Date: Fri, 5 Jun 2009 22:07:44 -0300 Subject: [Python-Dev] Roundup keywords for bug tracking In-Reply-To: References: Message-ID: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com> anatoly techtonik wrote: > > It is impossible to edit roundup keywords and this takes away the > flexibility in selecting bugs related to a module/function/test or > some other aspect of development. For example, I need to gather all > subprocess bugs in one query and things that won't be fixed in > deprecated os.popen() into another. In Trac I would use "subprocess" > and "os.popen" keywords. On ohloh I would add similar tags (if bugs > were software) without, but I can't do anything about Python roundup. > Is there any reason for such restriction? Well, keywords are used as a very restricted set of tags, so only users in the Developer group can create them. We've discussed free form issue tags that any user can create or edit in #python-dev and tracker-discuss[0]. I'm pretty sure they'd cover your use-case. I've submitted a patch to Rietveld[1], but it seems I never filled it in the meta-tracker, oopsie. If you (or anyone else) want to test-drive the tags feature, I can create an account in the experimental tracker[2] (which needs some attention anyway). I should be able to submit the patch to the meta-tracker during the weekend. Also, if you would like to bookmark arbitrary sets of issues, the bookmarklet and form in http://static.bot.bio.br/tool.html may be of help. You can paste the ids into the search page's ID field and create a query for a given (static) set of issues. Regards, Daniel [0] http://mail.python.org/pipermail/tracker-discuss/2009-April/002099.html [1] http://codereview.appspot.com/40100/show [2] http://bot.bio.br/python-dev-exp/issue5 From skippy.hammond at gmail.com Sat Jun 6 03:20:54 2009 From: skippy.hammond at gmail.com (Mark Hammond) Date: Sat, 06 Jun 2009 11:20:54 +1000 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: <79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> <4A28E5D7.2030402@gmail.com> <79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com> Message-ID: <4A29C476.6060101@gmail.com> Paul Moore wrote: > 2009/6/5 Nick Coghlan : >> Dirkjan Ochtman wrote: >>>> Essentially, pcbuild.sln is a binary file, and should be treated as >>>> such. Maybe it's an error in the Subversion setup that it's treated as >>>> text at all... >>> Yes, it certainly seems that way. >> Except it isn't a binary file - it's a text file with CRLF line endings. >> Why would we throw away the chance to get meaningful diffs when >> Subversion can easily get the line endings correct? > > Fair point - I hadn't thought of it in that context. I wonder if > Mercurial can treat it as a specialised form of binary file with > custom diff/merge behaviour (which just happens to be "treat as > text")? Of course, if that also requires client-side configuration, > it's no better solution than win32text. > > I've a feeling I've seen discussions of versioned metadata (things > like file properties) in Mercurial at some point in the past, but my > Google skills are failing me at the moment... I've looked into this recently. IIUC: * There has been some limited support expressed for having win32text look in a normal versioned file for hints about the current repo. * Once concern here was the fact that such config files are somewhat 'untrusted', but the hg config parsing code has recently been reimplemented to support the concept of 'trusted' versus 'untrusted' options. I'm not sure what this means in practice though. * There isn't currently much activity on win32text. None of the core hg devs seem to use Windows, so while they are receptive to Windows patches, it might be necessary to be quite noisy to get attention. I recently submitted some changes to the filtering for Windows which were accepted without any angst, and the ability to use a versioned file for per-repo rules is something I'd like too. I believe that if a few Windows users got together and agreed on both semantics and implementation we could get such an enhancement landed in the core of hg... Cheers, Mark From p.f.moore at gmail.com Sat Jun 6 14:25:11 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 6 Jun 2009 13:25:11 +0100 Subject: [Python-Dev] Mercurial, linefeeds, and Visual Studio In-Reply-To: <4A29C476.6060101@gmail.com> References: <8B473FAE8A08C34C9F5666FD4B0A87B67E2DDA50B9@hornigold.jaraco.com> <79990c6b0906040720m1d31f2fuff863262b891928a@mail.gmail.com> <4A28E5D7.2030402@gmail.com> <79990c6b0906050614k6cfc4fdbmf4285367c8e447fa@mail.gmail.com> <4A29C476.6060101@gmail.com> Message-ID: <79990c6b0906060525ga75ab3bmf2220169b92c34ba@mail.gmail.com> 2009/6/6 Mark Hammond : > Paul Moore wrote: >> >> 2009/6/5 Nick Coghlan : >>> >>> Dirkjan Ochtman wrote: >>>>> >>>>> Essentially, pcbuild.sln is a binary file, and should be treated as >>>>> such. Maybe it's an error in the Subversion setup that it's treated as >>>>> text at all... >>>> >>>> Yes, it certainly seems that way. >>> >>> Except it isn't a binary file - it's a text file with CRLF line endings. >>> Why would we throw away the chance to get meaningful diffs when >>> Subversion can easily get the line endings correct? >> >> Fair point - I hadn't thought of it in that context. I wonder if >> Mercurial can treat it as a specialised form of binary file with >> custom diff/merge behaviour (which just happens to be "treat as >> text")? Of course, if that also requires client-side configuration, >> it's no better solution than win32text. >> >> I've a feeling I've seen discussions of versioned metadata (things >> like file properties) in Mercurial at some point in the past, but my >> Google skills are failing me at the moment... > > I've looked into this recently. ?IIUC: > > * There has been some limited support expressed for having win32text look in > a normal versioned file for hints about the current repo. > > * Once concern here was the fact that such config files are somewhat > 'untrusted', but the hg config parsing code has recently been reimplemented > to support the concept of 'trusted' versus 'untrusted' options. ?I'm not > sure what this means in practice though. > > * There isn't currently much activity on win32text. ?None of the core hg > devs seem to use Windows, so while they are receptive to Windows patches, it > might be necessary to be quite noisy to get attention. > > I recently submitted some changes to the filtering for Windows which were > accepted without any angst, and the ability to use a versioned file for > per-repo rules is something I'd like too. ?I believe that if a few Windows > users got together and agreed on both semantics and implementation we could > get such an enhancement landed in the core of hg... I'm willing to help on this, but I don't personally use win32text (my approach is to have all files always in Unix line-ending format, which every tool I use handles fine). So take any design suggestions I might make with a pinch of salt :-) Paul. From g.brandl at gmx.net Sat Jun 6 19:43:44 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 06 Jun 2009 19:43:44 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: References: Message-ID: Antoine Pitrou schrieb: > Dirkjan Ochtman ochtman.nl> writes: >> >> http://www.python.org/dev/peps/pep-0385/ > > ? [cloned branches] makes it easier to distinguish branches, at the expense of > requiring more disk space on the client. ? > > This is a bit misleading. Actually, by separating branches into distinct > repositories, you can save quite a bit of repository space. It really depends on > the structure of the project and your own workflow. If you follow the "strict-subset" policy (which I would strongly recommend, from my experience) you'll end up with both branches in the trunk repo anyway. > ? This reflects that the default branch in hg is called 'default' instead of > Subversion's 'trunk', and reflects the proposed new tag format. ? > > If we use several distinct repositories, the default branch should instead > mirror the repository name (like what is done in the current hg mirrors: > "trunk", "py3k", etc.). +1. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Sat Jun 6 19:45:11 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 06 Jun 2009 19:45:11 +0200 Subject: [Python-Dev] Roundup keywords for bug tracking In-Reply-To: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com> References: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com> Message-ID: Daniel Diniz schrieb: > anatoly techtonik wrote: >> >> It is impossible to edit roundup keywords and this takes away the >> flexibility in selecting bugs related to a module/function/test or >> some other aspect of development. For example, I need to gather all >> subprocess bugs in one query and things that won't be fixed in >> deprecated os.popen() into another. In Trac I would use "subprocess" >> and "os.popen" keywords. On ohloh I would add similar tags (if bugs >> were software) without, but I can't do anything about Python roundup. >> Is there any reason for such restriction? > > Well, keywords are used as a very restricted set of tags, so only > users in the Developer group can create them. We've discussed free > form issue tags that any user can create or edit in #python-dev and > tracker-discuss[0]. I'm pretty sure they'd cover your use-case. I've > submitted a patch to Rietveld[1], but it seems I never filled it in > the meta-tracker, oopsie. > > If you (or anyone else) want to test-drive the tags feature, I can > create an account in the experimental tracker[2] (which needs some > attention anyway). I should be able to submit the patch to the > meta-tracker during the weekend. I'm not sure it will get well tested on the experimental tracker. If it basically works, and no one has any real objections, I'd just add it to the live tracker and try out how and if it is used. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ctb at msu.edu Sun Jun 7 05:16:09 2009 From: ctb at msu.edu (C. Titus Brown) Date: Sat, 6 Jun 2009 20:16:09 -0700 Subject: [Python-Dev] Snakebite (was: Re: py3k buildbots) In-Reply-To: References: Message-ID: <20090607031609.GC8905@idyll.org> On Fri, Jun 05, 2009 at 11:26:34AM +0200, Thomas Heller wrote: -> Antoine Pitrou schrieb: -> > Hello -> > -> > Only one of the py3k buildbots seems up: -> > http://www.python.org/dev/buildbot/3.x.stable/ -> -> Maybe they are waiting for the snakebite network ;-) (what's up with it, anyway?). We're still getting the machines set up. It turns out delivering power and A/C to a wide variety of architectures is more complicated than it may sound ;) --titus -- C. Titus Brown, ctb at msu.edu From tjreedy at udel.edu Sun Jun 7 05:21:06 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 06 Jun 2009 23:21:06 -0400 Subject: [Python-Dev] Status of 2.7 and 3.2 Message-ID: 2.7 Once upon a time, the plan was to come out with 2.6 and 3.0, and then after the usual interval, 2.7 and 3.1. As it turned out, 3.0 came out 3 months after 2.6, but, as it typical of x.0 releases, had some flaws leading to 3.1 now just 6 month later. I have thought that 2.7 was now to come out instead with 3.2 and would include backported 3.2 new features. Others expect 2.7 to come out soon after 3.1 and to only contain new 3.1 features. So Guido or someone, please clarify: is 2.7 to be the counterpart of 3.1 or 3.2? Data: On the tracker, new features are being assigned to 2.7 and 3.2. PEP 373 Python 2.7 Release Schedule says zilch: Release Schedule: Not yet finalized Possible features for 2.7: Nothing here 3.2 At some point, 3.x will become the "trunk" branch. It seems to me that this should be done with 3.2 as part of the transition to Mercurial. A. As long as 2.x is 'trunk', some people will view 3.x as 'experimental'. That was true for 3.0, but (much?) less so for 3.1. Is there any known reason why 3.2 should not soon be considered and treated as the main development version, to become the main production version? B. All new features will go into 3.2. Only some will be backported to 2.x. So it seems that the flow should be to develop for 3.2 and maybe backport thereafter. 2.final Is there any thought of making 2.7 be 2.final? A. To my mind, the main reason to add features to 2.x is to make transition to 3.x easier, rather than to discourage transition to 3.x. B. Do we really want to encourage library developers to put their 'upgrade to a new version' energy into 2.x to 2.x+1 upgrades, rather than a 2.x to 3.y upgrades? C. Some people are sticking with some version of 2.x because they want a stable version with minimal disturbance. Such people might have preferred, for instance, getting 2.5.5 in April instead of the latest 2.6 release. Instead people 2.5 fixes are being told "Sorry. 2.5 is out of Maintenance phase and into SecurityFix only phase." Once 2.x is put in feature freeze, micro bugfix releases can appear for years, as long as bugs are found and patches submitted and committed. It should gradually become truly rock solid. Terry Jan Reedy From martin at v.loewis.de Sun Jun 7 09:37:29 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 07 Jun 2009 09:37:29 +0200 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: References: Message-ID: <4A2B6E39.3090405@v.loewis.de> > I have thought that 2.7 was now to come out instead with 3.2 and would > include backported 3.2 new features. Others expect 2.7 to come out soon > after 3.1 and to only contain new 3.1 features. So Guido or someone, > please clarify: is 2.7 to be the counterpart of 3.1 or 3.2? Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between April 2010 and October 2010. I think it's too early to speculate about a release schedule for 3.2. > 3.2 > > At some point, 3.x will become the "trunk" branch. It seems to me that > this should be done with 3.2 as part of the transition to Mercurial. I'm not sure that the concept of a "trunk" branch still exists in Mercurial. PEP 385 apparently doesn't have resolved the branch strategy for Mercurial yet. With cloned branches, I think the concept of a "trunk" becomes irrelevant. > A. As long as 2.x is 'trunk', some people will view 3.x as > 'experimental'. That was true for 3.0, but (much?) less so for 3.1. Is > there any known reason why 3.2 should not soon be considered and treated > as the main development version, to become the main production version? > > B. All new features will go into 3.2. Only some will be backported to > 2.x. So it seems that the flow should be to develop for 3.2 and maybe > backport thereafter. What about bug fixes? How will they flow? > 2.final > > Is there any thought of making 2.7 be 2.final? Yes. Whether or not a 2.8 release will happen hasn't been decided yet, but 2.7 may well be the last 2.x release. > B. Do we really want to encourage library developers to put their > 'upgrade to a new version' energy into 2.x to 2.x+1 upgrades, rather > than a 2.x to 3.y upgrades? IMO, it's much up to the contributors. If the regular committers want to continue to work on 2.x, and a release manager is found to create releases, we can continue to release 2.8, and perhaps 2.9. However, I think at this point, it is too early to discuss that - 2.7 is still many months ahead. Regards, Martin From stephen at xemacs.org Sun Jun 7 14:25:05 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 07 Jun 2009 21:25:05 +0900 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2B6E39.3090405@v.loewis.de> References: <4A2B6E39.3090405@v.loewis.de> Message-ID: <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp> "Martin v. L?wis" writes: > I'm not sure that the concept of a "trunk" branch still exists in > Mercurial. PEP 385 apparently doesn't have resolved the branch strategy > for Mercurial yet. With cloned branches, I think the concept of a > "trunk" becomes irrelevant. "Trunk" exists as a technical restriction in CVS, and maybe to some extent in Subversion. Of course that restriction is lifted by DVCSes. But really "trunk" is a social concept. Most projects have a trunk. Even the anarchy known as GNU Arch converged on Tom Lord's version, just as the storm of turbulance known as "Linux kernel development" does on Linus's. Python's 2.x/py3k division is a tour de force; I still can't believe my eyes that you've pulled it off. Consider Perl 6, LaTeX 3, and Zope 3, or Linux 3, for that matter. The first 3 are all facing the "what is trunk?" issue, in the case of Zope several years after the "point oh" release, and AFAIK there are no current plans for Linux 3 (a microkernel architecture, maybe?<1.01 wink>). Of course the issues faced by LaTeX, Zope, and the kernel are quite different from Python's, and I don't know enough about Perl internals to compare. So I think "trunk" does matter. But it's not entirely in the power of the Python committers, not even the BDFL, to determine what "trunk" is. >>>>> Terry Reedy writes: > > A. As long as 2.x is 'trunk', some people will view 3.x as > > 'experimental'. That was true for 3.0, but (much?) less so for 3.1. Is > > there any known reason why 3.2 should not soon be considered and treated > > as the main development version, to become the main production version? Yes. User/developers may prefer strongly prefer the stability of 2.x. That's the problem of being a successful product, you lose agility almost by definition. > > B. All new features will go into 3.2. Only some will be backported to > > 2.x. So it seems that the flow should be to develop for 3.2 and maybe > > backport thereafter. > > What about bug fixes? How will they flow? This has to be ad hoc, at least at first. Some bugs will be uncovered in each version. The solutions will often not be the same in the different versions. Many developers will be downstream, and only willing to contribute the patch for the version they use. > > B. Do we really want to encourage library developers to put their > > 'upgrade to a new version' energy into 2.x to 2.x+1 upgrades, rather > > than a 2.x to 3.y upgrades? > > IMO, it's much up to the contributors. I think so, too. Terry's word "encourage" is appropriate here, though, and at some point that question will need to be answered. I think he's right to raise it early. From tseaver at palladion.com Sun Jun 7 15:02:41 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sun, 07 Jun 2009 09:02:41 -0400 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A2B6E39.3090405@v.loewis.de> <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Stephen J. Turnbull wrote: > "Martin v. L?wis" writes: > > > I'm not sure that the concept of a "trunk" branch still exists in > > Mercurial. PEP 385 apparently doesn't have resolved the branch strategy > > for Mercurial yet. With cloned branches, I think the concept of a > > "trunk" becomes irrelevant. > > "Trunk" exists as a technical restriction in CVS, and maybe to some > extent in Subversion. Of course that restriction is lifted by DVCSes. > > But really "trunk" is a social concept. Most projects have a trunk. > Even the anarchy known as GNU Arch converged on Tom Lord's version, > just as the storm of turbulance known as "Linux kernel development" > does on Linus's. > > Python's 2.x/py3k division is a tour de force; I still can't believe > my eyes that you've pulled it off. Consider Perl 6, LaTeX 3, and Zope > 3, or Linux 3, for that matter. The first 3 are all facing the "what > is trunk?" issue, in the case of Zope several years after the "point > oh" release, and AFAIK there are no current plans for Linux 3 (a > microkernel architecture, maybe?<1.01 wink>). Of course the issues > faced by LaTeX, Zope, and the kernel are quite different from > Python's, and I don't know enough about Perl internals to compare. FWIW, it seems to be consensus in the Zope community that "Zope3" was a misnomer, reflecting a vision for the software (serving as the replacement / successor to Zope2) which it didn't grow to fulfill. At this point, the "reusable" bits now form a collection of separately-released packages referred to as the "Zope Toolkit,"[1] which is intended to be used by Zope2, Grok, and the folks still running the "pure" Zope3 appserver. A lot of the work involved in this effort has been in disentagling the dependencies among the various packages, precisely so that they can be used piecemeal, without pulling in all of the stack. [1] http://docs.zope.org/zopetoolkit/ Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKK7px+gerLs4ltQ4RAoMnAJ9WTL8zo3+bSop+SAXr+akp85tMZQCfS5oC y14j64VJPjgxDQzCb1p/iZo= =RCZa -----END PGP SIGNATURE----- From ncoghlan at gmail.com Sun Jun 7 15:32:20 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 07 Jun 2009 23:32:20 +1000 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2B6E39.3090405@v.loewis.de> References: <4A2B6E39.3090405@v.loewis.de> Message-ID: <4A2BC164.6050201@gmail.com> Martin v. L?wis wrote: >> I have thought that 2.7 was now to come out instead with 3.2 and would >> include backported 3.2 new features. Others expect 2.7 to come out soon >> after 3.1 and to only contain new 3.1 features. So Guido or someone, >> please clarify: is 2.7 to be the counterpart of 3.1 or 3.2? > > Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between > April 2010 and October 2010. I think it's too early to speculate about > a release schedule for 3.2. I think Terry's underlying question is more basic than that. It can also be phrased as: "Is 2.7 allowed to have new features which did not appear in 3.1?" In previous discussions, a general policy has been articulated that having released Python 3.0, any new features should appear in a 3.x release before they appear in a 2.x release. Following that policy (which I think is actually a good one) means there are certain consequences for the two possible answers to the above question: A. "No." This answer means that 2.7 will only contain new features that are part of 3.1. If 2.7 sticks to a normal 18-24 month release cycle the only activities on the trunk for the next 12 months or so should be backports from 3.1 and bug fixes. New features added to the py3k branch for 3.2 should not be backported until after 2.7 is released. The other alternative in this case is to release 2.7 earlier than normal, but that creates problems in terms of the absolute duration of maintenance branch support for 2.6. B. "Yes." This answer means that the 3.1 to 3.2 development cycle will need to be truncated by roughly 6 months so that 3.2 can be released before 2.7 with any new features of interest. The 3.2 and 2.7 releases should then occur within a few months of each other. Releasing 2.7 early doesn't seem like a good idea to me and neither does putting the trunk into the equivalent of a feature freeze for 12 months or more. So I'm in favour of the idea of a paired 3.2/2.7 release late next year. I don't think that's a novel idea though - I'm pretty sure it was suggested (and met with general approval) when the idea of a short release cycle for 3.1 was first brought up. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Sun Jun 7 18:55:43 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 07 Jun 2009 18:55:43 +0200 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2BC164.6050201@gmail.com> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> Message-ID: <4A2BF10F.3040105@v.loewis.de> Nick Coghlan wrote: > Martin v. L?wis wrote: >>> I have thought that 2.7 was now to come out instead with 3.2 and would >>> include backported 3.2 new features. Others expect 2.7 to come out soon >>> after 3.1 and to only contain new 3.1 features. So Guido or someone, >>> please clarify: is 2.7 to be the counterpart of 3.1 or 3.2? >> Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between >> April 2010 and October 2010. I think it's too early to speculate about >> a release schedule for 3.2. > > I think Terry's underlying question is more basic than that. It can also > be phrased as: > > "Is 2.7 allowed to have new features which did not appear in 3.1?" I don't think this is the question Terry meant to ask, and the answer to this question is a clear "Certainly, but you are asking the wrong question." The question that Terry really meant to ask is "why is there no release candidate for 2.7 already, when 3.1 is going to be released RSN?", and that question arises from the (incorrect) understanding that 2.x releases and 3.x releases will be lock-step. The question that you posed should have been phrased as "Is 2.7 allowed to have new features which don't also appear on the 3k branch?", to which the answer is a clear "No." > In previous discussions, a general policy has been articulated that > having released Python 3.0, any new features should appear in a 3.x > release before they appear in a 2.x release. I don't think that the policy has been articulated in exactly that way, and it certainly wasn't implemented for 2.6/3.0. 2.6 was released with features that had not been released in any 3.x release, just because 3.x had not been released at all. > A. "No." > > This answer means that 2.7 will only contain new features that are > part of 3.1. If 2.7 sticks to a normal 18-24 month release cycle the > only activities on the trunk for the next 12 months or so should be > backports from 3.1 and bug fixes. New features added to the py3k branch > for 3.2 should not be backported until after 2.7 is released. The other > alternative in this case is to release 2.7 earlier than normal, but that > creates problems in terms of the absolute duration of maintenance branch > support for 2.6. I think this makes it clear that the answer can't possibly be "No". Not allowing addition of new features even though the release is still 12 months away would be unreasonable, IMO - what's the point of freezing development now, and where should new features to 2.x be added if not to the trunk? Likewise, releasing 2.7 now is completely unacceptable. > B. "Yes." > > This answer means that the 3.1 to 3.2 development cycle will need to > be truncated by roughly 6 months so that 3.2 can be released before 2.7 > with any new features of interest. The 3.2 and 2.7 releases should then > occur within a few months of each other. Assuming the policy "release new features for 2.x only after they got released for 3.x". I don't think such a policy actually exists. Regards, Martin From rdmurray at bitdance.com Sun Jun 7 19:51:04 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 7 Jun 2009 13:51:04 -0400 (EDT) Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2BF10F.3040105@v.loewis.de> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> Message-ID: On Sun, 7 Jun 2009 at 18:55, "Martin v. L?wis" wrote: >> B. "Yes." >> >> This answer means that the 3.1 to 3.2 development cycle will need to >> be truncated by roughly 6 months so that 3.2 can be released before 2.7 >> with any new features of interest. The 3.2 and 2.7 releases should then >> occur within a few months of each other. > > Assuming the policy "release new features for 2.x only after they got > released for 3.x". I don't think such a policy actually exists. I'm neutral on time frames, but I think that it _should_ be a policy that new features only get released to the 2.x branch after they have been released in the 3.x branch. Or, rather, I though that policy was implicit in the idea that we weren't _automatically_ backporting features, specifically in order to encourage 3.x adoption. So if 2.7 is going to come out before 3.2, you'll have to convince me that having new features in 2.7 that aren't in 3.1 is a good idea :) Note that I'm not advocating feature freezing the trunk, but rather having 3.2 come out before or in sync with 2.7. --David From martin at v.loewis.de Sun Jun 7 21:21:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 07 Jun 2009 21:21:11 +0200 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> Message-ID: <4A2C1327.9020909@v.loewis.de> > I'm neutral on time frames, but I think that it _should_ be a policy > that new features only get released to the 2.x branch after they have > been released in the 3.x branch. Or, rather, I though that policy was > implicit in the idea that we weren't _automatically_ backporting features, > specifically in order to encourage 3.x adoption. I think "backporting" is an entirely different issue, let alone "automatic" backporting. What *is* policy (AFAIU) is "any feature in the trunk must also exist in the py3k branch". IMO, this is sufficient to encourage 3.x adoption, and it is a policy that is silent wrt. to release date ordering. By the policy you propose, we could not have released 2.6 in October 2008, which we really really wanted to because Apple wanted us to. > So if 2.7 is going to > come out before 3.2, you'll have to convince me that having new features > in 2.7 that aren't in 3.1 is a good idea :) I don't see why the precise ordering of release dates matters at all. It seems you are happy with having 3.2 be released "around the same time" as 2.7. What is "the same time"? 3 months difference? 6 months difference? It certainly wouldn't be a year. Regards, Martin From barry at python.org Sun Jun 7 22:31:44 2009 From: barry at python.org (Barry Warsaw) Date: Sun, 7 Jun 2009 16:31:44 -0400 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2BF10F.3040105@v.loewis.de> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> Message-ID: On Jun 7, 2009, at 12:55 PM, Martin v. L?wis wrote: > Assuming the policy "release new features for 2.x only after they got > released for 3.x". I don't think such a policy actually exists. The policy, as I remember it, can be summed up: don't innovate new features in the 2.x branch. Meaning, add it to 3.x first and then backport if you want. I don't believe a new feature has to be in a / released/ version of 3.x first for it to show up in the next 2.x release. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From rdmurray at bitdance.com Sun Jun 7 22:40:31 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 7 Jun 2009 16:40:31 -0400 (EDT) Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2C1327.9020909@v.loewis.de> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> <4A2C1327.9020909@v.loewis.de> Message-ID: On Sun, 7 Jun 2009 at 21:21, "Martin v. L?wis" wrote: >> I'm neutral on time frames, but I think that it _should_ be a policy >> that new features only get released to the 2.x branch after they have >> been released in the 3.x branch. Or, rather, I though that policy was >> implicit in the idea that we weren't _automatically_ backporting features, >> specifically in order to encourage 3.x adoption. > > I think "backporting" is an entirely different issue, let alone > "automatic" backporting. > > What *is* policy (AFAIU) is "any feature in the trunk must also exist in > the py3k branch". IMO, this is sufficient to encourage 3.x adoption, and > it is a policy that is silent wrt. to release date ordering. You are right, my use of the term backport was imprecise. My impression at pycon was that Guido (and others) wanted a stronger policy than "make sure new features in trunk are also in 3.x". I heard this as "put new features in 3.x, not 2.x, to encourage 3.x adoption," but leaving the decision to the individual developers on whether or not to also "backport" them to 2.x. (The quotes around backport being that if you know you are going to put it into both it is currently easier to do trunk first.) As we have discovered since, this tends to get modified by wanting to ease transition from 2.x to 3.x by providing some of the features in 2.x (I'm thinking specifically of the distutils discussion.) How I got from that to release date ordering was by hearing that as "new features should be in 3.x first, and only maybe in 2.x". Clearly that was just my interpretation :) > By the policy you propose, we could not have released 2.6 in October > 2008, which we really really wanted to because Apple wanted us to. I don't think the 2.6 release date is relevant to this discussion, since 3.x hadn't been released at all at that point. What I want to avoid is people writing new code for 2.7 instead of 3.1 because they want to take advantage of a nifty new feature that 3.1 doesn't have. >> So if 2.7 is going to >> come out before 3.2, you'll have to convince me that having new features >> in 2.7 that aren't in 3.1 is a good idea :) > > I don't see why the precise ordering of release dates matters at all. > It seems you are happy with having 3.2 be released "around the same > time" as 2.7. What is "the same time"? 3 months difference? 6 months > difference? It certainly wouldn't be a year. No more than three months, I'd say, but that's just a gut level feeling, not backed by a specific argument. That said, I also have a gut level feeling that it is better to have the new features come out in 3.2 _first_. But I'm not presenting that as anything more than my feeling, and I'm happy to go with whatever consensus develops. FWIW, Benjamin has said the he personally has no problem with 3.2's release cycle being shorter than "normal". --David From fuzzyman at voidspace.org.uk Sun Jun 7 22:55:19 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 07 Jun 2009 21:55:19 +0100 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> <4A2C1327.9020909@v.loewis.de> Message-ID: <4A2C2937.8080502@voidspace.org.uk> R. David Murray wrote: > [snip...] >> By the policy you propose, we could not have released 2.6 in October >> 2008, which we really really wanted to because Apple wanted us to. > > I don't think the 2.6 release date is relevant to this discussion, > since 3.x hadn't been released at all at that point. What I want to > avoid is people writing new code for 2.7 instead of 3.1 because they > want to take advantage of a nifty new feature that 3.1 doesn't have. > But 3.1 is in feature freeze (py3k trunk) and it is not possible to check-in code for 3.2. Following this policy it would mean a feature freeze on trunk for an indefinite period of time. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From benjamin at python.org Sun Jun 7 22:57:25 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 7 Jun 2009 15:57:25 -0500 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2C2937.8080502@voidspace.org.uk> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> <4A2C1327.9020909@v.loewis.de> <4A2C2937.8080502@voidspace.org.uk> Message-ID: <1afaf6160906071357s293cb591n629b8f41c4b5f2ac@mail.gmail.com> 2009/6/7 Michael Foord : > R. David Murray wrote: >> >> [snip...] >>> >>> By the policy you propose, we could not have released 2.6 in October >>> 2008, which we really really wanted to because Apple wanted us to. >> >> I don't think the 2.6 release date is relevant to this discussion, >> since 3.x hadn't been released at all at that point. ?What I want to >> avoid is people writing new code for 2.7 instead of 3.1 because they >> want to take advantage of a nifty new feature that 3.1 doesn't have. >> > But 3.1 is in feature freeze (py3k trunk) and it is not possible to check-in > code for 3.2. Following this policy it would mean a feature freeze on trunk > for an indefinite period of time. And that's what we want to avoid, so we are discussing 3.2. -- Regards, Benjamin From tjreedy at udel.edu Sun Jun 7 23:30:23 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 07 Jun 2009 17:30:23 -0400 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2BC164.6050201@gmail.com> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> Message-ID: Nick Coghlan wrote: > Martin v. L?wis wrote: >>> I have thought that 2.7 was now to come out instead with 3.2 and would >>> include backported 3.2 new features. Others expect 2.7 to come out soon >>> after 3.1 and to only contain new 3.1 features. So Guido or someone, >>> please clarify: is 2.7 to be the counterpart of 3.1 or 3.2? >> Neither, nor. 2.7 will be released 18..24 months after 2.6, i.e. between >> April 2010 and October 2010. I think it's too early to speculate about >> a release schedule for 3.2. > > I think Terry's underlying question is more basic than that. It can also > be phrased as: > > "Is 2.7 allowed to have new features which did not appear in 3.1?" That is the issue that came up in a Python list discussion when two people, including one who I expect to know as much as me about this, 'corrected' my 'yes' to "No, 2.7 will come out in a few months after 3.1." So there clearly is confusion on this and related issues (as I already see in the responses on this thread ;-). The answer makes some difference in how issues are handled on the tracker, which I occasionally help with. > In previous discussions, a general policy has been articulated that > having released Python 3.0, any new features should appear in a 3.x > release before they appear in a 2.x release. Following that policy > (which I think is actually a good one) means there are certain > consequences for the two possible answers to the above question: > > A. "No." > > This answer means that 2.7 will only contain new features that are > part of 3.1. If 2.7 sticks to a normal 18-24 month release cycle the > only activities on the trunk for the next 12 months or so should be > backports from 3.1 and bug fixes. New features added to the py3k branch > for 3.2 should not be backported until after 2.7 is released. The other > alternative in this case is to release 2.7 earlier than normal, but that > creates problems in terms of the absolute duration of maintenance branch > support for 2.6. I agree, but this is what two people are expecting. > B. "Yes." > > This answer means that the 3.1 to 3.2 development cycle will need to > be truncated by roughly 6 months so that 3.2 can be released before 2.7 > with any new features of interest. The 3.2 and 2.7 releases should then > occur within a few months of each other. > > Releasing 2.7 early doesn't seem like a good idea to me and neither does > putting the trunk into the equivalent of a feature freeze for 12 months > or more. So I'm in favour of the idea of a paired 3.2/2.7 release late > next year. This is what I have been expecting, > > I don't think that's a novel idea though - I'm pretty sure it was > suggested (and met with general approval) when the idea of a short > release cycle for 3.1 was first brought up. I presume because it has been stated before. In addition to the question above, I am also trying to provoke thought on the nature and purpose of 2.7. Backporting features 'if someone feels like it' seems pretty haphazard. For someone wanting to maintain compatibility across multiple 2.x releases, a random new features may be nearly useless. Terry Jan Reedy From fuzzyman at voidspace.org.uk Sun Jun 7 23:32:49 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 07 Jun 2009 22:32:49 +0100 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> Message-ID: <4A2C3201.4060007@voidspace.org.uk> Terry Reedy wrote: > [snip...] >> >> I don't think that's a novel idea though - I'm pretty sure it was >> suggested (and met with general approval) when the idea of a short >> release cycle for 3.1 was first brought up. > > I presume because it has been stated before. > > In addition to the question above, I am also trying to provoke thought > on the nature and purpose of 2.7. Backporting features 'if someone > feels like it' seems pretty haphazard. For someone wanting to > maintain compatibility across multiple 2.x releases, a random new > features may be nearly useless. > The "What's new in Python 2.7" list is already very impressive. :-) Michael > Terry Jan Reedy > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From rdmurray at bitdance.com Sun Jun 7 23:37:35 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 7 Jun 2009 17:37:35 -0400 (EDT) Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2C2937.8080502@voidspace.org.uk> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> <4A2C1327.9020909@v.loewis.de> <4A2C2937.8080502@voidspace.org.uk> Message-ID: On Sun, 7 Jun 2009 at 21:55, Michael Foord wrote: > R. David Murray wrote: >> [snip...] >> > By the policy you propose, we could not have released 2.6 in October >> > 2008, which we really really wanted to because Apple wanted us to. >> >> I don't think the 2.6 release date is relevant to this discussion, >> since 3.x hadn't been released at all at that point. What I want to >> avoid is people writing new code for 2.7 instead of 3.1 because they >> want to take advantage of a nifty new feature that 3.1 doesn't have. >> > But 3.1 is in feature freeze (py3k trunk) and it is not possible to check-in > code for 3.2. Following this policy it would mean a feature freeze on trunk > for an indefinite period of time. As I said on IRC, that's not what I'm advocating. I'm advocating having 3.2 come out before 2.7, so that the new post-3.1 features get released to the public in 3.2 first, before appearing in 2.7. (Although "first" could alternatively mean the same day, if the community were up to for that). But like I said I'm not attached to the idea, just advocating it ;) --David From python at rcn.com Mon Jun 8 00:23:49 2009 From: python at rcn.com (Raymond Hettinger) Date: Sun, 7 Jun 2009 18:23:49 -0400 (EDT) Subject: [Python-Dev] Status of 2.7 and 3.2 Message-ID: <20090607182349.BID99621@ms19.lnh.mail.rcn.net> How about we just continue to improve both branches, doing forward or backports as appropriate. No need to develop a policy of crippling one branch on the theory that it will make the other seem more attractive. Besides, if 2.7 and 3.2 get released within a few months of each other, any inversion of incentives will be temporary and fleeting. Most likely people's decisions on switching to 3.x will be dominated by other factors such as the availability of third-party modules or other dependencies. IIRC, Benjamin's current merge procedures flow from the trunk to the py3k branch. Probably, it is best to continue with that practice lest we muck-up his merge/block entries. Raymond From ncoghlan at gmail.com Mon Jun 8 04:21:11 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 08 Jun 2009 12:21:11 +1000 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2BF10F.3040105@v.loewis.de> References: <4A2B6E39.3090405@v.loewis.de> <4A2BC164.6050201@gmail.com> <4A2BF10F.3040105@v.loewis.de> Message-ID: <4A2C7597.1050103@gmail.com> Martin v. L?wis wrote: > I don't think that the policy has been articulated in exactly that > way, and it certainly wasn't implemented for 2.6/3.0. 2.6 was released > with features that had not been released in any 3.x release, just > because 3.x had not been released at all. I don't think the idea of "3.x first" was raised explicitly until the discussions about having a short release cycle for 3.1 came up. Before then there was an assumption that 3.1 and 2.7 would come out at roughly the same time. >> B. "Yes." >> >> This answer means that the 3.1 to 3.2 development cycle will need to >> be truncated by roughly 6 months so that 3.2 can be released before 2.7 >> with any new features of interest. The 3.2 and 2.7 releases should then >> occur within a few months of each other. > > Assuming the policy "release new features for 2.x only after they got > released for 3.x". I don't think such a policy actually exists. As RDM already mentioned, the potential problem such a policy is intended to address is the idea of being able to write 2.7 code that can't be forward ported to 3.x because no version with a corresponding feature set has been released yet. My last sentence above was actually deliberately ambiguous as to whether 2.7 or 3.2 ends up being released first - so long as the releases happen within a few months of each other and any new features that appear in 2.7 also appear in 3.2, then the migration path for different 2.x versions remains clear (i.e 2.6-only code can be migrated to 3.1, but 2.7 code may need to be migrated to 3.2 instead if it uses features that aren't in 3.1). The exact order and timing of 3.2/2.7 will no doubt depend on any specific issues relating to the two releases around the time that they come out. But in the interests of maintaining a coherent migration path from 2.x to 3.x, it makes sense to plan to release them at roughly the same time. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Jun 8 04:28:45 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 08 Jun 2009 12:28:45 +1000 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <20090607182349.BID99621@ms19.lnh.mail.rcn.net> References: <20090607182349.BID99621@ms19.lnh.mail.rcn.net> Message-ID: <4A2C775D.7030205@gmail.com> Raymond Hettinger wrote: > How about we just continue to improve both branches, doing forward or > backports as appropriate. No need to develop a policy of crippling > one branch on the theory that it will make the other seem more > attractive. > > Besides, if 2.7 and 3.2 get released within a few months of each > other, any inversion of incentives will be temporary and fleeting. > Most likely people's decisions on switching to 3.x will be dominated > by other factors such as the availability of third-party modules or > other dependencies. > > IIRC, Benjamin's current merge procedures flow from the trunk to the > py3k branch. Probably, it is best to continue with that practice > lest we muck-up his merge/block entries. Agreed. >From Terry's later response, I think the main confusion here was whether or not 2.7 was going to be getting released hot on the heels of 3.1, and the answer to that is "No, 2.7 won't be released until 18-24 months have passed since the release of 2.6, as with previous 2.x releases". Releasing 3.2 around the same time as 2.7 to allow for easier migration from the latest 2.x release to the latest 3.x release will likely make sense, but doesn't actually need to be a 100% definite plan at this point. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From fwierzbicki at gmail.com Mon Jun 8 16:29:14 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Mon, 8 Jun 2009 10:29:14 -0400 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: References: Message-ID: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> At PyCon, we discussed moving Jython's svn repository to Python's with Martin von L?wis. I would think that Jython would live in Python's hg repository in the same way as stackless and distutils. Has the parallel project strategy been determined? Will they be separate repositories, separate "forests", something else? Also, Martin suggested we migrate to Python's svn and then go along for the svn->hg ride. Does that still make sense now that some planning has been done? -Frank From techtonik at gmail.com Mon Jun 8 17:07:13 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 8 Jun 2009 18:07:13 +0300 Subject: [Python-Dev] Roundup keywords for bug tracking In-Reply-To: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com> References: <2d75d7660906051807s3ca051fcw5af5841e99cd707a@mail.gmail.com> Message-ID: On Sat, Jun 6, 2009 at 4:07 AM, Daniel Diniz wrote: > anatoly techtonik wrote: >> >> It is impossible to edit roundup keywords and this takes away the >> flexibility in selecting bugs related to a module/function/test or >> some other aspect of development. For example, I need to gather all >> subprocess bugs in one query and things that won't be fixed in >> deprecated os.popen() into another. In Trac I would use "subprocess" >> and "os.popen" keywords. On ohloh I would add similar tags (if bugs >> were software) without, but I can't do anything about Python roundup. >> Is there any reason for such restriction? > > Well, keywords are used as a very restricted set of tags, so only > users in the Developer group can create them. We've discussed free > form issue tags that any user can create or edit in #python-dev and > tracker-discuss[0]. I'm pretty sure they'd cover your use-case. I've > submitted a patch to Rietveld[1], but it seems I never filled it in > the meta-tracker, oopsie. >From [0] discussion it seems that tags are planned to be a replacement for component or keywords field, but in my vision they should be just tags that doesn't have any specific meaning or administration interface. Autocomplete with ajax lookup is nice, but no drop-down lists etc. I made some comments in Rietveld at [1], but was unable to test it live, because [2] is offline. > If you (or anyone else) want to test-drive the tags feature, I can > create an account in the experimental tracker[2] (which needs some > attention anyway). I should be able to submit the patch to the > meta-tracker during the weekend. Hope this went well. I would definitely like to see how far this feature from how I imagine it, but b.p.o. deployment could be a better alternative for a real testing. > Also, if you would like to bookmark arbitrary sets of issues, the > bookmarklet and form in http://static.bot.bio.br/tool.html may be of > help. You can paste the ids into the search page's ID field and create > a query for a given (static) set of issues. Seems like it can come in handy. Thanks. -- anatoly t. From g.brandl at gmx.net Mon Jun 8 17:27:35 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 08 Jun 2009 17:27:35 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> Message-ID: Frank Wierzbicki schrieb: > At PyCon, we discussed moving Jython's svn repository to Python's with > Martin von L?wis. I would think that Jython would live in Python's hg > repository in the same way as stackless and distutils. Has the > parallel project strategy been determined? Will they be separate > repositories, separate "forests", something else? It should definitely be a separate repository, repositories tend to be a smaller unit in hg than in SVN, due to the fact that partial clones/checkouts are not supported. > Also, Martin suggested we migrate to Python's svn and then go along > for the svn->hg ride. Does that still make sense now that some > planning has been done? I don't think so. It should not matter from which repository the conversion is done, especially since there is no shared history at all. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From dirkjan at ochtman.nl Mon Jun 8 17:32:46 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Mon, 8 Jun 2009 17:32:46 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> Message-ID: On Mon, Jun 8, 2009 at 16:29, Frank Wierzbicki wrote: > At PyCon, we discussed moving Jython's svn repository to Python's with > Martin von L?wis. ?I would think that Jython would live in Python's hg > repository in the same way as stackless and distutils. ?Has the > parallel project strategy been determined? ?Will they be separate > repositories, separate "forests", something else? I think they should just be separate repositories. The svn.python.org "repository" is more like a collection of actual repositories than a repository in itself. > Also, Martin suggested we migrate to Python's svn and then go along > for the svn->hg ride. ?Does that still make sense now that some > planning has been done? I'd say migrating to Python's svn doesn't make a whole lot of sense at this point, but I'll leave that to Martin (since he has to do the work). For the conversion, I can just as well take the Jython repo from your current server. I've started a svnsync job with your repo so I can run some test conversions. It's a relatively small repository, so it shouldn't be much of a problem. Cheers, Dirkjan From techtonik at gmail.com Mon Jun 8 19:50:26 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Mon, 8 Jun 2009 20:50:26 +0300 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: References: Message-ID: On Sun, Jun 7, 2009 at 6:21 AM, Terry Reedy wrote: > > I have thought that 2.7 was now to come out instead with 3.2 and would > include backported 3.2 new features. ?Others expect 2.7 to come out soon > after 3.1 and to only contain new 3.1 features. ?So Guido or someone, please > clarify: is 2.7 to be the counterpart of 3.1 or 3.2? Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing in Windows I would vote for more time for stabilizating things - not adding new features. Long awaited subprocess as replacement for os.popen() AFAIK is still incapable to asynchronously communicate with spawned processes on Windows. I would call this feature as critical even on 2.6 As a release testcase - try porting pyexpect module to this platform. Absence of native curses/console/readline module also makes Python one-way unix shell language while many users expect it to be crossplatform. -- anatoly t. From solipsis at pitrou.net Mon Jun 8 20:17:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 8 Jun 2009 18:17:32 +0000 (UTC) Subject: [Python-Dev] Status of 2.7 and 3.2 References: Message-ID: anatoly techtonik gmail.com> writes: > > Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing > in Windows I would vote for more time for stabilizating things - not > adding new features. Long awaited subprocess as replacement for > os.popen() AFAIK is still incapable to asynchronously communicate with > spawned processes on Windows. I would call this feature as critical > even on 2.6 As a release testcase - try porting pyexpect module to > this platform. Absence of native curses/console/readline module also > makes Python one-way unix shell language while many users expect it to > be crossplatform. As always, patches and proposals are welcome! However, as far as the above issues are concerned, it seems to be less a matter of time between releases than of motivation to get things done (tm). I don't think a knowledgeable and determined Windows programmer would have much trouble solving each of them. Regards Antoine. From fwierzbicki at gmail.com Mon Jun 8 20:19:10 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Mon, 8 Jun 2009 14:19:10 -0400 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> Message-ID: <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> On Mon, Jun 8, 2009 at 11:32 AM, Dirkjan Ochtman wrote: > I'd say migrating to Python's svn doesn't make a whole lot of sense at > this point, but I'll leave that to Martin (since he has to do the > work). For the conversion, I can just as well take the Jython repo > from your current server. I've started a svnsync job with your repo so > I can run some test conversions. It's a relatively small repository, > so it shouldn't be much of a problem. Great! Thanks for giving this a try! For my part I'm fine either way (and I agree that Martin should decide based on what is best for him). -Frank From g.brandl at gmx.net Mon Jun 8 20:46:16 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 08 Jun 2009 20:46:16 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> Message-ID: Frank Wierzbicki schrieb: > On Mon, Jun 8, 2009 at 11:32 AM, Dirkjan Ochtman wrote: >> I'd say migrating to Python's svn doesn't make a whole lot of sense at >> this point, but I'll leave that to Martin (since he has to do the >> work). For the conversion, I can just as well take the Jython repo >> from your current server. I've started a svnsync job with your repo so >> I can run some test conversions. It's a relatively small repository, >> so it shouldn't be much of a problem. > Great! Thanks for giving this a try! > > For my part I'm fine either way (and I agree that Martin should decide > based on what is best for him). For CPython, we still need SVN, e.g. for the website repo, and several others like it that probably won't get converted (yet), and also for reference since we won't convert every old branch to hg. If that isn't necessary for Jython, I see no reason for moving into svn.python.org first. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Mon Jun 8 21:51:04 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 08 Jun 2009 21:51:04 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> Message-ID: <4A2D6BA8.2060906@v.loewis.de> >> Also, Martin suggested we migrate to Python's svn and then go along >> for the svn->hg ride. Does that still make sense now that some >> planning has been done? > > I don't think so. It should not matter from which repository the conversion > is done, especially since there is no shared history at all. Also, I would not merge the Jython SVN into the /projects repository, but instead set up a separate repository (http://svn.python.org/jython, and svn+ssh://jython at svn.python.org/). As discussed at PyCon, we should collect SSH keys for Jython committers in advance of the svn switchover; we would have to do that either way (also for hg), so now might be the right time to start doing it. As for the repository organization: I have now opinion (or, rather, no experience with hg). Regards, Martin From martin at v.loewis.de Mon Jun 8 21:57:05 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 08 Jun 2009 21:57:05 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> Message-ID: <4A2D6D11.4090707@v.loewis.de> > For my part I'm fine either way (and I agree that Martin should decide > based on what is best for him). See my other message. We need to collect SSH keys, and I don't mind moving the Jython repository. OTOH, if the Jython repository gets converted into hg right away, it's certainly (a little) less work. Of course, the Jython committers would need to relocate their svn sandboxes now, and then completely abandon them and re-checkout from hg in a few months, so if you can wait a few months until the hg conversion is ready, it's also less work for your committers. FWIW, I really think that PEP 385 should really grow a timeline pretty soon. Are we going to switch this year, next year, or 2011? Regards, Martin From martin at v.loewis.de Mon Jun 8 22:01:17 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 08 Jun 2009 22:01:17 +0200 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: References: Message-ID: <4A2D6E0D.6050009@v.loewis.de> > Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing > in Windows I would vote for more time for stabilizating things - not > adding new features. Long awaited subprocess as replacement for > os.popen() AFAIK is still incapable to asynchronously communicate with > spawned processes on Windows. I would call this feature as critical > even on 2.6 As a release testcase - try porting pyexpect module to > this platform. Absence of native curses/console/readline module also > makes Python one-way unix shell language while many users expect it to > be crossplatform. I am not quite sure whether you are for new features or not. Your first sentence ("vote for ... not adding new features") seems to suggest that you would not like to see new features, and your last sentence ("absence of native curses/console/readline module") suggests that you *do* want to see new features (namely, a native curses module, and a native readline module). Which one is it? Regards, Martin From alexandre at peadrop.com Mon Jun 8 22:04:27 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Mon, 8 Jun 2009 16:04:27 -0400 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: <4A2D6D11.4090707@v.loewis.de> References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> <4A2D6D11.4090707@v.loewis.de> Message-ID: On Mon, Jun 8, 2009 at 3:57 PM, "Martin v. L?wis" wrote: > FWIW, I really think that PEP 385 should really grow a timeline > pretty soon. Are we going to switch this year, next year, or 2011? > +1 -- Alexandre From dirkjan at ochtman.nl Mon Jun 8 22:29:32 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Mon, 8 Jun 2009 22:29:32 +0200 Subject: [Python-Dev] Draft PEP 385: Migrating from svn to Mercurial In-Reply-To: <4A2D6D11.4090707@v.loewis.de> References: <4dab5f760906080729gfb4ff90tdba68d800ac32ab6@mail.gmail.com> <4dab5f760906081119nfc13950p1201c54dbfc70a8d@mail.gmail.com> <4A2D6D11.4090707@v.loewis.de> Message-ID: On Mon, Jun 8, 2009 at 21:57, "Martin v. L?wis" wrote: > See my other message. We need to collect SSH keys, and I don't mind > moving the Jython repository. OTOH, if the Jython repository gets > converted into hg right away, it's certainly (a little) less work. Yeah, I guess if you move it to a separate repo on the svn.python.org server that might be nice. But it's not a big deal either way. > FWIW, I really think that PEP 385 should really grow a timeline > pretty soon. Are we going to switch this year, next year, or 2011? Sorry, I should've been clearer. I fully intend to complete the conversion in a few months, say by October 1st or sooner. That would be between 3.1 and whatever the next release will end up being, I hope? It's just hard to be very specific at this point. BTW, I tried my hand at rewriting the revlog for the manifest, as described in the PEP, and it made the full conversion *much* (as in 70%) smaller. I've also been looking at what branches we should keep and will ask for definitive feedback on that point soonish. Cheers, Dirkjan From fabiofz at gmail.com Mon Jun 8 23:59:19 2009 From: fabiofz at gmail.com (Fabio Zadrozny) Date: Mon, 8 Jun 2009 18:59:19 -0300 Subject: [Python-Dev] Cannot set PYTHONPATH with big paths with Python 3.0 and 3.1 Message-ID: Hi all, I've reported bug http://bugs.python.org/issue5924 some time ago and I think it's a release blocker -- it seems easy to fix, but I don't have time to actually submit a patch, so, I'd like to draw attention to it, especially as a release candidate is already out. Cheers, Fabio From martin at v.loewis.de Tue Jun 9 07:13:41 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 09 Jun 2009 07:13:41 +0200 Subject: [Python-Dev] Cannot set PYTHONPATH with big paths with Python 3.0 and 3.1 In-Reply-To: References: Message-ID: <4A2DEF85.4070108@v.loewis.de> > I've reported bug http://bugs.python.org/issue5924 some time ago and I > think it's a release blocker -- it seems easy to fix, but I don't have > time to actually submit a patch, so, I'd like to draw attention to it, > especially as a release candidate is already out. In absence of a patch, it can't be a release blocker, IMO. It's not a really critical bug (just an annoying one), as it is possible to work around it (e.g. by setting sys.path in sitecustomize, or by moving all modules into a single directory so that sys.path doesn't have to be that long). If nobody else has time to submit a patch, either, Python 3.1 will get released with this bug. Regards, Martin From solipsis at pitrou.net Tue Jun 9 12:48:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 9 Jun 2009 10:48:14 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Cannot_set_PYTHONPATH_with_big_paths_with_?= =?utf-8?q?Python_3=2E0=09and_3=2E1?= References: <4A2DEF85.4070108@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > > I've reported bug http://bugs.python.org/issue5924 some time ago and I > > think it's a release blocker -- it seems easy to fix, but I don't have > > time to actually submit a patch, so, I'd like to draw attention to it, > > especially as a release candidate is already out. > > In absence of a patch, it can't be a release blocker, IMO. I think we've had lots of issues filed as release blockers while they still didn't have a patch. As for whether this particular bug deserves to be a blocker, I don't know. It is certainly annoying, however. Regards Antoine. From amauryfa at gmail.com Tue Jun 9 12:58:08 2009 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 9 Jun 2009 12:58:08 +0200 Subject: [Python-Dev] Cannot set PYTHONPATH with big paths with Python 3.0 and 3.1 In-Reply-To: References: <4A2DEF85.4070108@v.loewis.de> Message-ID: Hi, 2009/6/9 Antoine Pitrou : > Martin v. L?wis v.loewis.de> writes: >> In absence of a patch, it can't be a release blocker, IMO. > > I think we've had lots of issues filed as release blockers while they still > didn't have a patch. > As for whether this particular bug deserves to be a blocker, I don't know. It is > certainly annoying, however. FYI, I just submitted a patch. It is simple enough to be considered for inclusion for rc2. -- Amaury Forgeot d'Arc From techtonik at gmail.com Tue Jun 9 12:58:50 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Tue, 9 Jun 2009 13:58:50 +0300 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <4A2D6E0D.6050009@v.loewis.de> References: <4A2D6E0D.6050009@v.loewis.de> Message-ID: On Mon, Jun 8, 2009 at 11:01 PM, "Martin v. L?wis" wrote: > > I am not quite sure whether you are for new features or not. Your > first sentence ("vote for ... not adding new features") seems to > suggest that you would not like to see new features, and your last > sentence ("absence of native curses/console/readline module") > suggests that you *do* want to see new features (namely, a native > curses module, and a native readline module). > > Which one is it? I would like to see new features in Python, but only if they are cross-platform. Unfortunately, I do not possess C skills to make this happen, nor do I have deep understanding of Microsoft Visual Studio .project files to port Makefiles and GCC options even when codebase is available for windows. The level to make a contribution in this case is too high. The lack of free time makes it impossible to close the gap in one step leaving remnants of work-in-process that will make it harder to continue in future than to start from scratch. Perhaps the necessity to make it in one huge step could be compensated by incremental solution approach and development process if there was obvious centralized place to organize efforts AND provide clear visibility into progress made so far, initial plan, plan deviation and current status. Mailing lists are good for discussions, but that's all - information becomes outdated, text-too-much, no prompt reply often stops the progress. Perhaps I shift my problem from lack-of-skill into lack-of-tools direction, but being amazed by efforts people put into supporting this mailing list I most of the time unable to reply to emails I get mostly because replies require time for testing and proving facts. There is no definite proposal to solve problems of enabling OpenID or SSO for python.org, of porting curses to windows, of testing subprocessing etc., but there is an idea that some things could be given more visibility AND priority to allow people to see the big picture and focus on outstanding problems. Even though most people here know about big-things-to-fix, these things doesn't standout from the pile of issues in roundup. The thing I miss the most is ability to gather all information relevant to one problem in one place. This includes timeline with commits, branches, relevant issues, issue updates, relevant wiki edits, current focus URLs, _filtered_ threads and refactored comments. The problem is to ensure that this information is up to date and provide easy way/instruction how to bring it up to date in case something is broken. It is not necessary to meet the bus to experience the effect of bus factor. -- anatoly t. From kristjan at ccpgames.com Tue Jun 9 15:49:42 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 9 Jun 2009 13:49:42 +0000 Subject: [Python-Dev] xmlrpc and http/1.1 Message-ID: <930F189C8A437347B80DF2C156F7EC7F057F9DD5D4@exchis.ccp.ad.local> Hello there. I've been doing some work on xmlrpc for CCP in the last weeks. I'm trying to communicate the results of this back to the python trunk. I've had the following issues open for a while now: http://bugs.python.org/issue6096 http://bugs.python.org/issue6099 I would appreciate some comments on this. Moving forward, I'd like to add support for the "gzip" content encoding for XMLRPC requests and responses. K -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Tue Jun 9 16:52:19 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 9 Jun 2009 16:52:19 +0200 Subject: [Python-Dev] Tags from closed branch heads? Message-ID: So ca8d05e1f1d1 changed the default for repo.heads() to closed=False, so that calls to heads() by default will not return closed heads. Unfortunately, this also means that any tags from those heads will not be considered anymore. That seems inadvertent at best, and may be should be reverted. Conceptually, it seems wrong that deleting a branch would also delete all the tags from it. I'd like to propose this change: diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py --- a/mercurial/localrepo.py +++ b/mercurial/localrepo.py @@ -320,7 +320,7 @@ def _hgtagsnodes(self): last = {} ret = [] - for node in reversed(self.heads()): + for node in reversed(self.heads(closed=True)): c = self[node] rev = c.rev() try: Is that something we can agree on? Cheers, Dirkjan From kalman.gergely at duodecad.hu Tue Jun 9 16:46:54 2009 From: kalman.gergely at duodecad.hu (=?ISO-8859-1?Q?K=E1lm=E1n_Gergely?=) Date: Tue, 09 Jun 2009 16:46:54 +0200 Subject: [Python-Dev] python sendmsg()/recvmsg() implementation Message-ID: <4A2E75DE.9060508@duodecad.hu> Hello, my name is Greg. I've just started using python after many years of C programming, and I'm also new to the list. I wanted to clarify this first, so that maybe I will get a little less beating for my stupidity :) I use python3 and Linux on arch x86 (production will be on x86_64, though this shouldn't matter much). The application that I'm presently working on is a network server. It would use separate processes to accept the connections, and to do the work (much like how apache prefork does). One process accept()s on the original socket and the received socket (client socket) will be read for a request. After the request is received and parsed this process (the controller) will choose one from its' children that is most capable of handling the said request. It would then pass the file descriptor through a socketpair to the appropriate children and go handle the next client. All works fine and smooth, but I realized that I need sendmsg()/recvmsg() to pass the FD. Since these are not implemented in the python socket module, and Linux has no other way to do this, I'm stuck. Fell flat on my face, too :) Browsing the net I've found a patch to the python core (http://bugs.python.org/issue1194378), dated 2005. First of all, I would like to ask you guys, whether you know of any way of doing this FD passing magic, or that you know of any 3rd party module / patch / anything that can do this for me. Since I'm fairly familiar with C and (not that much, but I feel the power) python, I would take the challenge of writing it, given that the above code is still somewhat usable. If all else fails I would like to have your help to guide me through this process. Thanks Kalman Gergely From exarkun at divmod.com Tue Jun 9 17:10:26 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 9 Jun 2009 11:10:26 -0400 Subject: [Python-Dev] python sendmsg()/recvmsg() implementation In-Reply-To: <4A2E75DE.9060508@duodecad.hu> Message-ID: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com> On Tue, 09 Jun 2009 16:46:54 +0200, K?lm?n Gergely wrote: >Hello, my name is Greg. > >I've just started using python after many years of C programming, and I'm >also new to the list. I wanted to clarify this >first, so that maybe I will get a little less beating for my stupidity :) > Welcome! > > [snip] > >Browsing the net I've found a patch to the python core >(http://bugs.python.org/issue1194378), dated 2005. > >First of all, I would like to ask you guys, whether you know of any way of >doing this FD passing magic, or that you know >of any 3rd party module / patch / anything that can do this for me. Aside from the patch in the tracker, there are several implementations of these APIs as third-party extension modules. > >Since I'm fairly familiar with C and (not that much, but I feel the power) >python, I would take the challenge of writing >it, given that the above code is still somewhat usable. If all else fails I >would like to have your help to guide me through >this process. > What would be great is if you could take the patch in the tracker and get it into shape so that it is suitable for inclusion. This would involve three things, I think: 1. Write unit tests for the functionality (since the patch itself provides none) 2. Update the patch so that it again applies cleanly against trunk 3. Add documentation for the new APIs Once this is done, you can get a committer to look at it and either provide more specific feedback or apply it. Thanks, Jean-Paul From dirkjan at ochtman.nl Tue Jun 9 17:15:58 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 9 Jun 2009 17:15:58 +0200 Subject: [Python-Dev] Tags from closed branch heads? In-Reply-To: References: Message-ID: Sorry about that, got dev-lists mixed up in my head somewhere... On Tue, Jun 9, 2009 at 16:52, Dirkjan Ochtman wrote: > So ca8d05e1f1d1 changed the default for repo.heads() to closed=False, > so that calls to heads() by default will not return closed heads. > Unfortunately, this also means that any tags from those heads will not > be considered anymore. That seems inadvertent at best, and may be > should be reverted. Conceptually, it seems wrong that deleting a > branch would also delete all the tags from it. I'd like to propose > this change: > > diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py > --- a/mercurial/localrepo.py > +++ b/mercurial/localrepo.py > @@ -320,7 +320,7 @@ > ? ? def _hgtagsnodes(self): > ? ? ? ? last = {} > ? ? ? ? ret = [] > - ? ? ? ?for node in reversed(self.heads()): > + ? ? ? ?for node in reversed(self.heads(closed=True)): > ? ? ? ? ? ? c = self[node] > ? ? ? ? ? ? rev = c.rev() > ? ? ? ? ? ? try: > > Is that something we can agree on? > > Cheers, > > Dirkjan > From nestornissen at gmail.com Wed Jun 10 04:58:47 2009 From: nestornissen at gmail.com (Nestor) Date: Tue, 9 Jun 2009 22:58:47 -0400 Subject: [Python-Dev] Rewrite os.popen using subprocess.Popen? Message-ID: <4b57b0700906091958k294e9ea7u65fc45a94e272f31@mail.gmail.com> pydoc.py uses os.popen once on line 1359. According to the documentation that function is deprecated since Python 2.6. Does it make sense to rewrite that line using the newer subprocess instead? I am asking because os.popen stopped working for me in Python 3.1 but I don't know if it is worth investing time in a module that is deprecated when the newer one does work. http://bugs.python.org/issue6236 From josiah.carlson at gmail.com Wed Jun 10 05:36:19 2009 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Tue, 9 Jun 2009 20:36:19 -0700 Subject: [Python-Dev] python sendmsg()/recvmsg() implementation In-Reply-To: <4A2E75DE.9060508@duodecad.hu> References: <4A2E75DE.9060508@duodecad.hu> Message-ID: On Tue, Jun 9, 2009 at 7:46 AM, K?lm?n Gergely wrote: > Hello, my name is Greg. > > I've just started using python after many years of C programming, and I'm > also new to the list. I wanted to clarify this > first, so that maybe I will get a little less beating for my stupidity :) > > I use python3 and Linux on arch x86 (production will be on x86_64, though > this shouldn't matter much). > > The application that I'm presently working on is a network server. It would > use separate processes to accept the > connections, and to do the work (much like how apache prefork does). One > process accept()s on the original socket and > the received socket (client socket) will be read for a request. After the > request is received and parsed this process (the > controller) will choose one from its' children that is most capable of > handling the said request. It would then pass the > file descriptor through a socketpair to the appropriate children and go > handle the next client. All works fine and smooth, > but I realized that I need sendmsg()/recvmsg() to pass the FD. Since these > are not implemented in the python socket > module, and Linux has no other way to do this, I'm stuck. Fell flat on my > face, too :) > > Browsing the net I've found a patch to the python core > (http://bugs.python.org/issue1194378), dated 2005. > > First of all, I would like to ask you guys, whether you know of any way of > doing this FD passing magic, or that you know > of any 3rd party module / patch / anything that can do this for me. IIRC, this is already implemented in the multiprocessing package, which comes standard with Python 2.6 and 3.0 . It looks like the test is disabled in test/test_multiprocessing.py , and re-enabling it (on Windows) produces errors that make me think it's more an issue with the tests than with multiprocessing itself. Dig into it, see if you can get the tests to pass :) - Josiah > Since I'm fairly familiar with C and (not that much, but I feel the power) > python, I would take the challenge of writing > it, given that the above code is still somewhat usable. If all else fails I > would like to have your help to guide me through > this process. > > Thanks > > Kalman Gergely > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/josiah.carlson%40gmail.com > From cylix at solace.info Thu Jun 11 03:38:57 2009 From: cylix at solace.info (Frederick Reeve) Date: Wed, 10 Jun 2009 20:38:57 -0500 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 Message-ID: <20090610203857.00a019fa@cylix> Greetings, As I'm sure you all know there are currently two implementations of the io module one in python and one much faster implementation in C. As I recall the python version was used in python3 and the C version is now used by default in python3.1x. The behavior of the two is different in some ways especially regarding io.BufferedReader.peek(). I wrote an email to the authors of the new C code last Friday. I also sent a copy of it to the python list for comments. I was directed by Antoine Pitrou that I should possibly bring up what I had asked there here or as a bug report. I elected to write here because I am not sure it constitutes a bug. In my former email I stated I was willing to submit patches if the old behavior was desired back and the code author was fine with the changes but didn't want to implement them. Antoine said this, "If people need more sophisticated semantics, I'm open to changing peek() to accommodate it." Antoine: If I do wrong quoting you are free to chastise me. So my basic question is: The behavior of io.BufferedReader.peek() has changed; is that change something that should: stay as is, revert or be different entirely? Here are the two behaviors: The python version of io.BufferedReader.peek() behaves as: If the buffer holds less than requested (upto buffersize) read from the raw stream the difference or up to EOF into the buffer. Return requested number of bytes from the start of the buffer. This may advance the raw stream but not the local stream. This version can guarantee a peek of one chunk (4096 bytes here). The C version behaves as: If the buffer holds 0 bytes fill from the raw stream or up to EOF. Return what is in the buffer. This may advance the raw stream but not the local stream. This version cannot guarantee a peek of over 1 byte if random length reads are being used at all and not tracked. Neither case limits what is possible, though, in my opinion, one makes it easier to accomplish certain things and is more efficient in those cases. Take the following two parser examples: s = io.BufferedReader wrapped stream with no negative seek in most cases. f = output file handler or such. python version work flow: are = re.compile(b'(\r\n|\r|\n)') while True: d = s.peek(4096) # chunk size or so. found = are.search(d) if found: w = d[:found.start()] s.seek(f.write(w)) p = s.peek(74) if p.startswith(multipart_boundary): s.seek(len(multipart_boundary)) # other code containing more possible splits # across boundaries continue w = d[found.start():found.end()] s.seek(f.write(w)) continue f.write(d) #more code continue C version work flow: old = b'' are = re.compile(b'(\r\n|\r|\n)') while True: d = old if old != b'' else s.read1(4096) found = are.search(d) if found: w = d[:found.start()] f.write(w) w = d[found.start():] p = w if len(w) >= 74 else w + s.read(73) if p.startswith(multipart_boundary): # Other code containing more possible splits # across boundaries and joins to p. old = ??? continue f.write(d[found.start():found.end()]) old = dd[found.end():] + p continue old = b'' f.write(d) #more code continue These two examples are not real code but get the point across and are based off code I put into a multipart parser. The former written for python3. I later tried running that parser on 3.1 after the new io layer and found it broken. Then rewrote it to the new interface. That rewrite is represented in the latter some what. This is only one example. Others may vary, of course. Peek seems to me to have little use outside of parsers. Thus I used parsers as an example. My opinion is that it would be better to have a peek function similar to the the python implementation in C like as follows: peek(n): If n is less than 0, None, or not set; return buffer contents with out advancing stream position. If the buffer is empty read a full chunk and return the buffer. Otherwise return exactly n bytes up to _chunk size_(not contents) with out advancing the stream position. If the buffer contents is less than n, buffer an additional chunk from the "raw" stream before hand. If EOF is encountered during any raw read then return as much as we can up to n. (maybe I should write that in code form??) This allows us to obtain the behavior of the current C io implementation easily and would give us the old python implementation's behavior when n is given. The basis for this is: 1. Code reduction and Simplicity Looking at the examples, the code reduction should be obvious. The logic needed to maintain a bytestring of the variously required lengths, so that it may be checked, would not be necessary. The need to hold a bytestring to the next iteration would be done away with as well. Other pieces of data handling would also be simpler. 2. Speed It would require less handling in the "slower" interpreter if we would use the buffer in the buffered reader. Also, all that logic mentioned in 1 is moved to the faster C code or done away with. There is very little necessity for peek outside of parsers, so speed in read-through and random reads would not have to be affected. I have other reasons and arguments, but I want to know what every one else thinks. This will most likely show me what I have missed or am not seeing, if anything. Please I have babbled enough. Thanks so much for the consideration. Frederick Reeve From kristjan at ccpgames.com Thu Jun 11 17:33:22 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Thu, 11 Jun 2009 15:33:22 +0000 Subject: [Python-Dev] xmlrpc improvements Message-ID: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> Hello there. I've been trying to, in the last weeks, to pass on to the trunk the improvements I've made to XMLRPC. I've created several issues in order to do these changes incrementally but have got no comments. Perhaps it is best to show the whole thing in context, then. I?ve gathered all the changes in a single patch, here: http://codereview.appspot.com/73041/show I've also created a corresponding issue in the issue tracker: http://bugs.python.org/issue6267 I'd really like to get this stuff in. The performance gains allowing http1.1 and gzip for xmlrpc are quite significant. Also, there are bugfixes in there. Cheers, Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.barrett at asgard.com Thu Jun 11 22:48:09 2009 From: david.barrett at asgard.com (David A. Barrett) Date: Thu, 11 Jun 2009 16:48:09 -0400 Subject: [Python-Dev] Adding key and reverse args to bisect Message-ID: <4A316D89.80702@Asgard.COM> I propose adding the key parameter to the bisect.bisect routines. This would allow it to be used on lists with an ordering other than the one "natural" to the contained objects. (and anywhere else it makes sense in the bisect module). Would this be easy enough to do? It looks like the main difficulty may have to do with the C implementation. I've also noticed that sort appears far faster; is the C implementation working as expected? It may be nice to have the reverse parameter as well, but I'm not sure how that could be implemented right off the bat. David A. Barrett - - - - Here's an example I've coded up for my own use (reverse hasn't been implemented here): def bisect_right(a, x, lo=0, hi=None, key=lambda x: x, reverse=False): """A keyed version of bisect.bisect_bisect_right: return i: for all e in a[:i] e <= x < f, for all f in a[i:] """ if hi is None: hi = len(a) if reverse: while lo < hi: mid = (lo+hi)//2 if key(a[mid]) < key(x): hi = mid else: lo = mid+1 else: while lo < hi: mid = (lo+hi)//2 if key(x) < key(a[mid]): hi = mid else: lo = mid+1 return lo def bisect_left(a, x, lo=0, hi=None, key=lambda x: x, reverse=False): """A keyed version of bisect.bisect_bisect_left: return i: for all e in a[:i] e < x <= f, for all f in a[i:] """ if hi is None: hi = len(a) if reverse: while lo < hi: mid = (lo+hi)//2 if key(x) < key(a[mid]): lo = mid+1 else: hi = mid else: while lo < hi: mid = (lo+hi)//2 if key(a[mid]) < key(x): lo = mid+1 else: hi = mid return lo From python at rcn.com Thu Jun 11 23:33:03 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 11 Jun 2009 14:33:03 -0700 Subject: [Python-Dev] Adding key and reverse args to bisect References: <4A316D89.80702@Asgard.COM> Message-ID: [David A. Barrett] >I propose adding the key parameter > to the bisect.bisect routines. This > would allow it to be used on lists with > an ordering other than the one "natural" > to the contained objects. Algorithmically, the bisect routines are the wrong place to do key lookups. If you do many calls to bisect(), each one will make multiple calls to the key function, potentially repeating calls that were made on previous searches. Instead, it is better to search a list of precomputed keys to find the index of the record in question. >>> data = [('red', 5), ('blue', 1), ('yellow', 8), ('black', 0)] >>> data.sort(key=lambda r: r[1]) # key function called exactly len(data) times >>> keys = [r[1] for r in data] >>> data[bisect_left(keys, 0)] ('black', 0) >>> data[bisect_left(keys, 1)] ('blue', 1) >>> data[bisect_left(keys, 5)] ('red', 5) >>> data[bisect_left(keys, 8)] ('yellow', 8) Raymond From aahz at pythoncraft.com Fri Jun 12 01:01:54 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 11 Jun 2009 16:01:54 -0700 Subject: [Python-Dev] Adding key and reverse args to bisect In-Reply-To: <4A316D89.80702@Asgard.COM> References: <4A316D89.80702@Asgard.COM> Message-ID: <20090611230154.GA20007@panix.com> On Thu, Jun 11, 2009, David A. Barrett wrote: > > I propose adding the key parameter to the bisect.bisect routines. > This would allow it to be used on lists with an ordering other than > the one "natural" to the contained objects. Raymond addressed your actual question, but please post suggestions like this to python-ideas, that's the best place for them. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "If you don't know what your program is supposed to do, you'd better not start writing it." --Dijkstra From hfuerstenau at gmx.net Fri Jun 12 17:18:18 2009 From: hfuerstenau at gmx.net (=?UTF-8?B?SGFnZW4gRsO8cnN0ZW5hdQ==?=) Date: Fri, 12 Jun 2009 16:18:18 +0100 Subject: [Python-Dev] Iterator version of contextlib.nested Message-ID: <4A3271BA.2070904@gmx.net> contextlib.nested has recently been deprecated on grounds of being unnecessary now that the with statement accepts multiple context managers. However, as has been mentioned before (http://mail.python.org/pipermail/python-dev/2009-May/089359.html), that doesn't cover the case of a variable number of context managers, i.e. with contextlib.nested(*list_of_managers) as list_of_results: or with contexlib.nested(*iterator_of_managers): It was suggested that in these use cases a custom context manager should be implemented. However, it seems that such an implementation would be an almost exact copy of the present code for "nested". I'm proposing to add an iterator version of "nested" to contextlib (possibly called "inested"), which takes an iterable of context managers instead of a variable number of parameters. The implementation could be taken over from the present "nested", only changing "def nested(*managers)" to "def inested(managers)". This has the advantage that an iterator can be passed to "inested", so that each context managers is created in the context of all previous ones, which was one of the reasons for introducing the multi-with statement in the first place. "contextlib.inested" would therefore be the generalization of the multi-with statement to a variable number of managers (and "contextlib.nested" would stay deprecated). - Hagen From status at bugs.python.org Fri Jun 12 18:07:24 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 12 Jun 2009 18:07:24 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090612160724.26F897839F@psf.upfronthosting.co.za> ACTIVITY SUMMARY (06/05/09 - 06/12/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2225 open (+39) / 15869 closed (+26) / 18094 total (+65) Open issues with patches: 882 Average duration of open issues: 652 days. Median duration of open issues: 402 days. Open Issues Breakdown open 2197 (+39) pending 28 ( +0) Issues Created Or Reopened (68) _______________________________ Issue with RotatingFileHandler logging handler on Windows 06/08/09 CLOSED http://bugs.python.org/issue4749 reopened r.david.murray add disable_nagle_algorithm to SocketServer.TCPServer 06/08/09 http://bugs.python.org/issue6192 reopened pitrou patch, patch, easy test_float fails on Windows 06/05/09 CLOSED http://bugs.python.org/issue6198 reopened eric.smith path separator output ignores shell's path separator: / instead 06/05/09 http://bugs.python.org/issue6208 created ThurnerRupert compilation error in std. lib. module shutil (Python 3.1rc1, pla 06/05/09 CLOSED http://bugs.python.org/issue6209 created dpodbori Exception Chaining missing method for suppressing context 06/05/09 http://bugs.python.org/issue6210 created poke [Tutorial] Section 4.7.2 has a wrong description of an example 06/05/09 CLOSED http://bugs.python.org/issue6211 created cofi patch piped input 06/05/09 CLOSED http://bugs.python.org/issue6212 created rtmq Incremental encoder incompatibility between 2.x and py3k 06/05/09 http://bugs.python.org/issue6213 created pitrou test__locale broken on trunk 06/05/09 CLOSED http://bugs.python.org/issue6214 created pitrou Backport the IO lib to trunk 06/05/09 http://bugs.python.org/issue6215 created pitrou patch Raise Unicode KEEPALIVE_SIZE_LIMIT from 9 to 32? 06/06/09 http://bugs.python.org/issue6216 created tjreedy Add _io._TextIOWrapper.errors 06/06/09 CLOSED http://bugs.python.org/issue6217 created pjenvey patch Make io.BytesIO and io.StringIO picklable. 06/06/09 http://bugs.python.org/issue6218 created alexandre.vassalotti patch, patch nested list value change 06/06/09 CLOSED http://bugs.python.org/issue6219 created pushkarparanjpe typo: opne in "doanddont" 06/06/09 CLOSED http://bugs.python.org/issue6220 created cate Windows buildbot failure in test_winreg 06/06/09 CLOSED http://bugs.python.org/issue6221 created pitrou 2to3 except fixer failed in certain case 06/06/09 CLOSED http://bugs.python.org/issue6222 created bhy Make _PyUnicode_AsString as public API 06/06/09 CLOSED http://bugs.python.org/issue6223 created bhy References to JPython 06/06/09 CLOSED http://bugs.python.org/issue6224 created thijs Fixing several minor bugs in Tkinter.Canvas and one in Misc._con 06/06/09 http://bugs.python.org/issue6225 created gpolo patch Inconsistent 'file' vs 'stream' kwarg in pprint, other stdlibs 06/06/09 http://bugs.python.org/issue6226 created pjenvey doctest_aliases doesn't test duplicate removal 06/07/09 http://bugs.python.org/issue6227 created abbeyj patch round() error 06/07/09 CLOSED http://bugs.python.org/issue6228 created steve21 Installation python on mac 06/07/09 CLOSED http://bugs.python.org/issue6229 created kostonstyle ElementTree.Element and cElementTree.Element have slightly diffe 06/07/09 http://bugs.python.org/issue6230 created Neil Muller patch ElementInclude may drop text 06/07/09 http://bugs.python.org/issue6231 created Neil Muller patch Improve test coverage of ElementTree and cElementTree 06/07/09 http://bugs.python.org/issue6232 created Neil Muller ElementTree (py3k) doesn't properly encode characters that can't 06/07/09 http://bugs.python.org/issue6233 created Neil Muller patch cgi.FieldStorage is broken when given POST data 06/07/09 http://bugs.python.org/issue6234 created efosmark \d missing from effects of re.ASCII flag 06/08/09 CLOSED http://bugs.python.org/issue6235 created MLModel os.popen causes illegal seek on AIX in Python 3.1rc 06/08/09 http://bugs.python.org/issue6236 created nestor Build errors when using LDFLAGS="-Wl,--no-undefined" 06/08/09 http://bugs.python.org/issue6237 created fundawang string.ljust documentation is missing optional fillchar descript 06/08/09 CLOSED http://bugs.python.org/issue6238 created jaraco c_char_p return value returns string, not bytes 06/08/09 http://bugs.python.org/issue6239 created georg.brandl API to get source encoding as defined by PEP 263 06/08/09 CLOSED http://bugs.python.org/issue6240 created srid Better type checking for the arguments of io.StringIO 06/08/09 http://bugs.python.org/issue6241 created alexandre.vassalotti patch Fix reference leak in io.StringIO 06/08/09 http://bugs.python.org/issue6242 created alexandre.vassalotti patch getkey() can segfault in combination with curses.ungetch() 06/08/09 http://bugs.python.org/issue6243 created Trundle patch Support for tcl 8.6 06/08/09 http://bugs.python.org/issue6244 created misc patch Add "intel" universal architecture on OSX 06/08/09 http://bugs.python.org/issue6245 created ronaldoussoren patch, needs review Python debugger ignores exception if instructed to return from g 06/08/09 http://bugs.python.org/issue6246 created inducer should we include argparse 06/09/09 CLOSED http://bugs.python.org/issue6247 reopened rickysarraf TCP Sockets not closed by TCPServer and StreamRequestHandler 06/09/09 http://bugs.python.org/issue6248 created krisvale patch, patch, needs review Error Prompt 06/10/09 CLOSED http://bugs.python.org/issue6249 created SonMarvin Python compiles dead code 06/10/09 http://bugs.python.org/issue6250 created abbeyj patch c++ extension module implementation guide/example in extending/e 06/10/09 http://bugs.python.org/issue6251 created subgeometer logging midnight rotation 06/10/09 CLOSED http://bugs.python.org/issue6252 created SanityIO optparse.OptionParser.get_usage uses wrong formatter 06/10/09 http://bugs.python.org/issue6253 created ccx patch tarfile unnecessarily requires seekable files 06/10/09 CLOSED http://bugs.python.org/issue6254 created johnsonm PyInt_FromSize_t is undocumented. 06/10/09 http://bugs.python.org/issue6255 created naoki Wrong stacklevel in warning for contextlib.nested 06/10/09 CLOSED http://bugs.python.org/issue6256 created hagen patch Idle terminates on source save while debugging 06/10/09 http://bugs.python.org/issue6257 created andyharrington distributions built with bdist_msi on 64-bit Windows fail to ins 06/10/09 http://bugs.python.org/issue6258 created jaraco ctypes pointer arithmetic 06/10/09 http://bugs.python.org/issue6259 created theller patch os.utime should allow None values for ATIME or MTIME 06/10/09 CLOSED http://bugs.python.org/issue6260 created purpleidea Clarify behaviour of random.uniform 06/10/09 CLOSED http://bugs.python.org/issue6261 created marketdickinson VS 2008 binaries 06/11/09 CLOSED http://bugs.python.org/issue6262 created bellenot syntax error in get_msvcr 06/11/09 CLOSED http://bugs.python.org/issue6263 created tarek Misleading instructions for building extensions with mingw 06/11/09 CLOSED http://bugs.python.org/issue6264 created bdew cElementTree & ElementTree use different exceptions for XML Erro 06/11/09 http://bugs.python.org/issue6265 created Neil Muller cElementTree.iterparse & ElementTree.iterparse return differentl 06/11/09 http://bugs.python.org/issue6266 created Neil Muller Cumulative patch to http and xmlrpc 06/11/09 http://bugs.python.org/issue6267 created krisvale Seeking to the beginning of a text file a second time will retur 06/11/09 http://bugs.python.org/issue6268 created eggy threading documentation makes no mention of the GIL 06/11/09 http://bugs.python.org/issue6269 created segfaulthunter patch Menu deletecommand fails if command is already deleted 06/11/09 http://bugs.python.org/issue6270 created gregcouch patch mmap: don't close file description if fd=-1 06/12/09 http://bugs.python.org/issue6271 created haypo patch Upgrading xml.etree to ElementTree 1.3 06/12/09 http://bugs.python.org/issue6272 created orsenthil Issues Now Closed (52) ______________________ Improve the hackish runtime_library_dirs support for gcc 653 days http://bugs.python.org/issue1032 alexandre.vassalotti patch Builtin round function is sometimes inaccurate for floats 50 days http://bugs.python.org/issue1869 marketdickinson patch Set up "supported"-only buildbot waterfall view 452 days http://bugs.python.org/issue2376 exarkun "No windows home dir" doc error 384 days http://bugs.python.org/issue2922 r.david.murray subprocess (Replacing popen) - add a warning / hint 382 days http://bugs.python.org/issue2947 r.david.murray patch, easy __newobj__ pickle feature is not documented 272 days http://bugs.python.org/issue3816 alexandre.vassalotti easy ctypes documentation 208 days http://bugs.python.org/issue4309 mnewman Issue with RotatingFileHandler logging handler on Windows 3 days http://bugs.python.org/issue4749 vsajip Using LDFLAGS='-rpath=\$$LIB:/some/other/path' ./configure break 121 days http://bugs.python.org/issue5201 tarek patch test_fdopen fails with vs2005, release build on Windows 2000 70 days http://bugs.python.org/issue5623 krisvale patch "No such file or directory" with framework build under MacOS 10. 48 days http://bugs.python.org/issue5809 ronaldoussoren When setting complete PYTHONPATH on Python 3.x, paths in the PYT 36 days http://bugs.python.org/issue5924 amaury.forgeotdarc patch Make pickle generated by Python 3.x compatible with 2.x and vice 12 days http://bugs.python.org/issue6137 rhettinger patch Python 3.1 rc1 will not build on OS X 10.5.7 with macports libin 8 days http://bugs.python.org/issue6154 brett.cannon patch fcntl footnote about O_SHLOCK and O_EXLOCK is misleading 4 days http://bugs.python.org/issue6194 georg.brandl easy Serious regression in doctest in Py3.1rc1 8 days http://bugs.python.org/issue6195 r.david.murray patch tarfile.extractall(readaccess=True) 1 days http://bugs.python.org/issue6196 lars.gustaebel test_float fails on Windows 4 days http://bugs.python.org/issue6198 eric.smith test_winreg fails 5 days http://bugs.python.org/issue6201 amaury.forgeotdarc Obsolete default file encoding "mac-roman" on OS X, not influenc 2 days http://bugs.python.org/issue6202 nad patch Missing reference in section 4.6 to chapter on classes 1 days http://bugs.python.org/issue6204 georg.brandl patch Correct a trivial typo introduced by r73238. 1 days http://bugs.python.org/issue6206 georg.brandl patch Simple For-Loops 0 days http://bugs.python.org/issue6207 pitrou compilation error in std. lib. module shutil (Python 3.1rc1, pla 0 days http://bugs.python.org/issue6209 benjamin.peterson [Tutorial] Section 4.7.2 has a wrong description of an example 1 days http://bugs.python.org/issue6211 georg.brandl patch piped input 0 days http://bugs.python.org/issue6212 georg.brandl test__locale broken on trunk 1 days http://bugs.python.org/issue6214 ezio.melotti Add _io._TextIOWrapper.errors 1 days http://bugs.python.org/issue6217 benjamin.peterson patch nested list value change 0 days http://bugs.python.org/issue6219 rhettinger typo: opne in "doanddont" 0 days http://bugs.python.org/issue6220 ezio.melotti Windows buildbot failure in test_winreg 1 days http://bugs.python.org/issue6221 benjamin.peterson 2to3 except fixer failed in certain case 0 days http://bugs.python.org/issue6222 benjamin.peterson Make _PyUnicode_AsString as public API 0 days http://bugs.python.org/issue6223 benjamin.peterson References to JPython 0 days http://bugs.python.org/issue6224 georg.brandl round() error 0 days http://bugs.python.org/issue6228 marketdickinson Installation python on mac 0 days http://bugs.python.org/issue6229 benjamin.peterson \d missing from effects of re.ASCII flag 0 days http://bugs.python.org/issue6235 georg.brandl string.ljust documentation is missing optional fillchar descript 0 days http://bugs.python.org/issue6238 georg.brandl API to get source encoding as defined by PEP 263 0 days http://bugs.python.org/issue6240 benjamin.peterson should we include argparse 3 days http://bugs.python.org/issue6247 bethard Error Prompt 0 days http://bugs.python.org/issue6249 georg.brandl logging midnight rotation 1 days http://bugs.python.org/issue6252 vsajip tarfile unnecessarily requires seekable files 2 days http://bugs.python.org/issue6254 lars.gustaebel Wrong stacklevel in warning for contextlib.nested 0 days http://bugs.python.org/issue6256 rhettinger patch os.utime should allow None values for ATIME or MTIME 2 days http://bugs.python.org/issue6260 purpleidea Clarify behaviour of random.uniform 1 days http://bugs.python.org/issue6261 rhettinger VS 2008 binaries 1 days http://bugs.python.org/issue6262 loewis syntax error in get_msvcr 0 days http://bugs.python.org/issue6263 tarek Misleading instructions for building extensions with mingw 1 days http://bugs.python.org/issue6264 bdew IDLE / PyOS_InputHook 2108 days http://bugs.python.org/issue798058 gpolo import on Windows: please call SetErrorMode first 1664 days http://bugs.python.org/issue1069410 srid logging module / wrong bytecode? 876 days http://bugs.python.org/issue1633605 annacoder Top Issues Most Discussed (10) ______________________________ 31 Issue with RotatingFileHandler logging handler on Windows 3 days closed http://bugs.python.org/issue4749 12 should we include argparse 3 days closed http://bugs.python.org/issue6247 10 3.x locale does not default to C, contrary to the documentation 7 days open http://bugs.python.org/issue6203 8 Make pickle generated by Python 3.x compatible with 2.x and vic 12 days closed http://bugs.python.org/issue6137 8 Make logging configuration files easier to use 15 days open http://bugs.python.org/issue6136 7 Serious regression in doctest in Py3.1rc1 8 days closed http://bugs.python.org/issue6195 7 GCC detection for runtime_library_dirs when ccache is used 1403 days open http://bugs.python.org/issue1254718 5 tarfile unnecessarily requires seekable files 2 days closed http://bugs.python.org/issue6254 5 Add "intel" universal architecture on OSX 4 days open http://bugs.python.org/issue6245 5 typo: opne in "doanddont" 0 days closed http://bugs.python.org/issue6220 From facundobatista at gmail.com Fri Jun 12 23:25:30 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 12 Jun 2009 18:25:30 -0300 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() Message-ID: In a long lived process at work, we started leaking file descriptors. The problem were that subprocess.Popen._execute_child() method creates two files descriptors through a call to os.pipe(), and after some work it closes them. But an os.read() on the descriptor was interrupted (EINTR), so an exception was raised, and the descriptors were not closed... leakage! This problem is easy to fix (I have a patch that fixes it, all tests pass ok, see http://bugs.python.org/issue6274). So, why this mail? I just think that the fix is ugly... it's not elegant. It has the following structure: errpipe_read, errpipe_write = os.pipe() try: try: ..... ..... ..... ..... ..... ..... finally: os.close(errpipe_write) ..... ..... ..... finally: os.close(errpipe_read) I just don't like a huge try/finally... but as FDs are just ints, do you think is there a better way to handle it? Thank you!! Regards, -- . ? ?Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From omega_force2003 at yahoo.com Sat Jun 13 01:01:43 2009 From: omega_force2003 at yahoo.com (OMEGA RED) Date: Fri, 12 Jun 2009 16:01:43 -0700 (PDT) Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER Message-ID: <958091.40012.qm@web34401.mail.mud.yahoo.com> ? TO ALL, ? ? I AM HAPPY TO SAY THAT I AM NOW IN THE PROCESS OF DEVELOPING A WEBSITE FOR THE STATED PURPOSE OF COMBINING SOME OF THE BEST MINDS IN THE WORLD.? THIS PROJECT WILL BE A MEANS TO DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE VARIANT OF THE PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER.? WHICH MEANS THAT EVERYTHING ABOUT THE DESIGN, DEVELOPMENT, AND CONSTRUCTION OF THE ENGINE WILL BE COMPLETELY AVAILABLE TO THE PUBLIC AND TO ANY ENGINE MANUFACTURE FREE OF ANY REQUIREMENTS.? ? THIS ENGINE WILL BE CALLED THE PHOENIX NexT F-200 ENGINE.? ? THIS ENGINE WILL ALSO BE A POTENTIAL THIRD AND FINAL ALTERNATIVE ENGINE FOR THE JOINT STRIKE FIGHTER.? ? THIS ENGINE WILL BE STATED AS BEING A COMPETITOR WITH BOTH THE F-135 AND THE F-136 ENGINE, WHICH ARE PROVIDED BY BOTH PRATT-WHITNEY AND GENERAL ELECTRIC/ ROLLS ROYCE.? ? ? THIS ENGINE WILL UTILIZE PYTHON FOR THE MAJOR PORTION OF THE SIMULATION, RESEARCH AND DEVELOPMENT STAGES FOR THE DEVELOPMENT OF THE ENGINE.? THE CONSTRUCTION OF THE MATHEMATICAL PROCESSING OF THE REQUIRED NONLINEAR CONTROLS WILL BE BASED ON A SPECIALIZED VERSION OF GROUP THEORY FOR THE DESIGN AND CONSTRUCTION OF THE NONLINEAR CONTROL SYSTEMS.? ? ? THE PURPOSE OF THE DEVELOPMENT OF THE ENGINE WILL BE TO CONSTRUCT AN ENGINE FOR THE F-35 THAT WILL BE AS RELIABLE AND CAPABLE AS THE F-135 AND THE F-136.? ? HOWEVER, THE PHOENIX NexT ENGINE WILL ALSO BE STATED AS BEING ONLY AROUND $20,000 TO $110,000 DOLLARS TO BEING PURCHASED BY AN OPERATOR OF THE F-35, WHICH WILL BE AN INSIGNIFICANT COST COMPARED TO THE COST OF BOTH THE F-135 AND THE F-136 ENGINE (10 MILLION DOLLARS PER UNIT).? ? HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ?? I BELIEVE THAT WE ALL?CAN SUCCEED WHERE BOTH PRATT-WHITNEY AND GE/ROLLS ROYCE HAVE FAILED TO DEVELOP AN INEXPENSIVE AND RELIABLE ENGINE FOR THE F-35. ? THANKS,? ? W108DAB -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew at matthewwilkes.co.uk Sat Jun 13 01:53:44 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Sat, 13 Jun 2009 00:53:44 +0100 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER In-Reply-To: <958091.40012.qm@web34401.mail.mud.yahoo.com> References: <958091.40012.qm@web34401.mail.mud.yahoo.com> Message-ID: <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> On 13 Jun 2009, at 00:01, OMEGA RED wrote: > DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE VARIANT OF THE > PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER You're unlikely to find many people who want to help use open-source to facilitate murder. > HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ? Nobody here. You're off topic, this list is for development of Python, not pet projects. Matt From guido at python.org Sat Jun 13 02:00:52 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Jun 2009 17:00:52 -0700 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER In-Reply-To: <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> References: <958091.40012.qm@web34401.mail.mud.yahoo.com> <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> Message-ID: On Fri, Jun 12, 2009 at 4:53 PM, Matthew Wilkes wrote: > > On 13 Jun 2009, at 00:01, OMEGA RED wrote: > >> DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE VARIANT OF THE >> PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER > > You're unlikely to find many people who want to help use open-source to > facilitate murder. That's a rather presumptuous statement. Despite the poster's use of SHOUTING I don't see a reason to tell them they should use proprietary software just because you disagree with their politics. (Hey, I disagree with Eric Raymond's *and* Richard Stallman's politics. :-) >> HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ? > > Nobody here. ?You're off topic, this list is for development of Python, not > pet projects. True. Comp.lang.python might be a better place. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From benjamin at python.org Sat Jun 13 02:05:06 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 12 Jun 2009 19:05:06 -0500 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER In-Reply-To: References: <958091.40012.qm@web34401.mail.mud.yahoo.com> <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> Message-ID: <1afaf6160906121705r2e2acac2n3568a78469317f51@mail.gmail.com> 2009/6/12 Guido van Rossum : > On Fri, Jun 12, 2009 at 4:53 PM, Matthew > Wilkes wrote: >> On 13 Jun 2009, at 00:01, OMEGA RED wrote: >>> HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ? >> >> Nobody here. ?You're off topic, this list is for development of Python, not >> pet projects. > > True. Comp.lang.python might be a better place. Actually, I think the Python community might be better off if he went to comp.lang.perl. -- Regards, Benjamin From matthew at matthewwilkes.co.uk Sat Jun 13 02:09:24 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Sat, 13 Jun 2009 01:09:24 +0100 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER In-Reply-To: References: <958091.40012.qm@web34401.mail.mud.yahoo.com> <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> Message-ID: <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> On 13 Jun 2009, at 01:00, Guido van Rossum wrote: > That's a rather presumptuous statement. Despite the poster's use of > SHOUTING I don't see a reason to tell them they should use proprietary > software just because you disagree with their politics Oh, I didn't mean they should use proprietary software, just that in my experience the kind of people who are active in open source are quite anti-war, green, etc. There are notable exceptions, but I know people who worry that their work will have military applications, and who turn down projects because of it. You're more likely to get people who are interested in aviation with some programming background (universities do teach coding to engineers, afterall), than the other way around. Matt From lists at cheimes.de Sat Jun 13 02:06:22 2009 From: lists at cheimes.de (Christian Heimes) Date: Sat, 13 Jun 2009 02:06:22 +0200 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: References: Message-ID: <4A32ED7E.4070504@cheimes.de> Facundo Batista wrote: > I just don't like a huge try/finally... but as FDs are just ints, do > you think is there a better way to handle it? How about a nice 'n shiny context wrapper for the pipe: import os class Pipe(object): def __enter__(self): self.read, self.write = os.pipe() return self.read, self.write def __exit__(self, *args): try: os.close(self.read) finally: # make sure that write is closed even if # self.read can't be closed os.close(self.write) with Pipe() as (read, write): print read, write Christian PS and nit pick: File descriptor are opaque resource handlers which just happened to be ints. They should be treated as magic cookies. From lists at cheimes.de Sat Jun 13 02:06:13 2009 From: lists at cheimes.de (Christian Heimes) Date: Sat, 13 Jun 2009 02:06:13 +0200 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: References: Message-ID: <4A32ED75.7060803@cheimes.de> Facundo Batista wrote: > I just don't like a huge try/finally... but as FDs are just ints, do > you think is there a better way to handle it? How about a nice 'n shiny context wrapper for the pipe: import os class Pipe(object): def __enter__(self): self.read, self.write = os.pipe() return self.read, self.write def __exit__(self, *args): try: os.close(self.read) finally: # make sure that write is closed even if # self.read can't be closed os.close(self.write) with Pipe() as (read, write): print read, write Christian PS and nit pick: File descriptor are opaque resource handlers which just happened to be ints. They should be treated as magic cookies. From python at rcn.com Sat Jun 13 02:17:50 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 12 Jun 2009 17:17:50 -0700 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE F-35 JOINT STRIKE FIGHTER References: <958091.40012.qm@web34401.mail.mud.yahoo.com><4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> Message-ID: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> [Matthew Wilkes] > Oh, I didn't mean they should use proprietary software, just that in > my experience the kind of people who are active in open source are > quite anti-war, green, etc. There are notable exceptions, but I know > people who worry that their work will have military applications, and > who turn down projects because of it. I question the whole notion of using open source in military weapons. It seems like a rather basic violation of operational security. Perhaps your enemies will exploit your bugs instead of nicely reporting them and submitting patches on SourceForge ;-) Raymond From guido at python.org Sat Jun 13 02:19:30 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 12 Jun 2009 17:19:30 -0700 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE F-35 JOINT STRIKE FIGHTER In-Reply-To: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> References: <958091.40012.qm@web34401.mail.mud.yahoo.com> <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> Message-ID: On Fri, Jun 12, 2009 at 5:17 PM, Raymond Hettinger wrote: > > [Matthew Wilkes] >> >> Oh, I didn't mean they should use proprietary software, just that in ?my >> experience the kind of people who are active in open source are ?quite >> anti-war, green, etc. ?There are notable exceptions, but I know ?people who >> worry that their work will have military applications, and ?who turn down >> projects because of it. > > I question the whole notion of using open source in military weapons. > It seems like a rather basic violation of operational security. ?Perhaps > your enemies will exploit your bugs instead of nicely reporting them > and submitting patches on SourceForge ;-) Eric Raymond would argue that it's probably the other way around -- proprietary software doesn't have enough eyeballs to make it safe. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From cylix at solace.info Sat Jun 13 02:37:07 2009 From: cylix at solace.info (Frederick Reeve) Date: Fri, 12 Jun 2009 19:37:07 -0500 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090610203857.00a019fa@cylix> References: <20090610203857.00a019fa@cylix> Message-ID: <20090612193707.24f5563c@cylix> Greetings, I feel the need to point out I made a mistake. When I wrote my last email I said the behavior had changed python3-3.1. This seems not to be the case.. I had made that assumption because I had written code based on the looking at the code in _pyio.py as well as the python3 documentation (http://docs.python.org/3.0/library/io.html#io.BufferedReader) which seems to be wrong on that point or I miss understand. Anyway I'm sorry about that. The other point still stands though. I would like to see peek changed. I am willing to write and submit changes but don't want to unless others agree this is a good idea. So I put forth the implementation at the bottom of this email. If its bad or you don't see the point I may try to clarify but if nobody things its good, please just tell me I'm waisting your time, and I will go away. I also apologize my last email was so long. peek(n): If n is less than 0, None, or not set; return buffer contents with out advancing stream position. If the buffer is empty read a full chunk and return the buffer. Otherwise return exactly n bytes up to _chunk size_(not contents) with out advancing the stream position. If the buffer contents is less than n, buffer an additional chunk from the "raw" stream before hand. If EOF is encountered during any raw read then return as much as we can up to n. (maybe I should write that in code form??) Thanks Frederick Reeve From ncoghlan at gmail.com Sat Jun 13 03:01:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 13 Jun 2009 11:01:49 +1000 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <4A3271BA.2070904@gmx.net> References: <4A3271BA.2070904@gmx.net> Message-ID: <4A32FA7D.1060906@gmail.com> Hagen F?rstenau wrote: > I'm proposing to add an iterator version of "nested" to contextlib > (possibly called "inested"), which takes an iterable of context managers > instead of a variable number of parameters. The implementation could be > taken over from the present "nested", only changing "def > nested(*managers)" to "def inested(managers)". > > This has the advantage that an iterator can be passed to "inested", so > that each context managers is created in the context of all previous > ones, which was one of the reasons for introducing the multi-with > statement in the first place. "contextlib.inested" would therefore be > the generalization of the multi-with statement to a variable number of > managers (and "contextlib.nested" would stay deprecated). The semantic change actually needed to make nested() more equivalent to the multi-with statement is for it to accept zero-argument callables that create context managers as arguments rather than pre-created context managers. Rather than changing the name of the function, this could be done by inspecting the first argument for an "__enter__" method. If it has one, use the old semantics (and issue a DeprecationWarning as in 3.1). Otherwise, use the proposed new semantics. However, the semantic equivalence still won't be complete as nested() currently has no way of matching the multi-with behaviour of allowing outer context managers to suppress exceptions raised by the constructors or __enter__ methods of inner context managers. Attempting to do so will result in a RuntimeError as the contextlib.contextmanager wrapper complains that the generator implementing nested() didn't yield. The further enhancement needed to address that would be to tweak nested() to raise a custom exception in that case (e.g. ContextSkipped) and provide an "allowskip" context manager that just catches and suppresses that specific exception: i.e. @contextmanager def allowskip(): try: yield except ContextSkipped: pass with allowskip(), nested(*cm_factories): # Do something I suggest putting an RFE on the tracker for this. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From phillip.sitbon+python-dev at gmail.com Sat Jun 13 03:10:15 2009 From: phillip.sitbon+python-dev at gmail.com (Phillip Sitbon) Date: Fri, 12 Jun 2009 18:10:15 -0700 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE F-35 JOINT STRIKE FIGHTER In-Reply-To: References: <958091.40012.qm@web34401.mail.mud.yahoo.com> <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> Message-ID: <536685ea0906121810m367ab103tf8b1b3e4980b0e71@mail.gmail.com> >> I question the whole notion of using open source in military weapons. >> It seems like a rather basic violation of operational security. ?Perhaps >> your enemies will exploit your bugs instead of nicely reporting them >> and submitting patches on SourceForge ;-) > > Eric Raymond would argue that it's probably the other way around -- > proprietary software doesn't have enough eyeballs to make it safe. :-) > I guess in some cases it wouldn't matter if it were open source: http://online.wsj.com/article/SB124027491029837401.html I'm not sure of the seriousness (or mental stability) of Mr. "OMEGA RED", but I definitely got a chuckle from this :) - Phillip From ncoghlan at gmail.com Sat Jun 13 03:30:29 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 13 Jun 2009 11:30:29 +1000 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE F-35 JOINT STRIKE FIGHTER In-Reply-To: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> References: <958091.40012.qm@web34401.mail.mud.yahoo.com><4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> Message-ID: <4A330135.8060803@gmail.com> Raymond Hettinger wrote: > > [Matthew Wilkes] >> Oh, I didn't mean they should use proprietary software, just that in >> my experience the kind of people who are active in open source are >> quite anti-war, green, etc. There are notable exceptions, but I know >> people who worry that their work will have military applications, and >> who turn down projects because of it. > > I question the whole notion of using open source in military weapons. > It seems like a rather basic violation of operational security. Perhaps > your enemies will exploit your bugs instead of nicely reporting them > and submitting patches on SourceForge ;-) As Guido said, even the military are aware that there are major problems with the idea of security through obscurity. Plus most defence forces around the world are just as interested in saving a few bucks on software costs as any other organisation, particularly if they can reduce their reliance on a foreign software vendor in the process. As to the existence of open source developers that are willing to work with the military... as Matthew said, while they may not be particularly common, I can definitely say that they're around ;) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From pjenvey at underboss.org Sat Jun 13 03:15:09 2009 From: pjenvey at underboss.org (Philip Jenvey) Date: Fri, 12 Jun 2009 18:15:09 -0700 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE F-35 JOINT STRIKE FIGHTER In-Reply-To: <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> References: <958091.40012.qm@web34401.mail.mud.yahoo.com><4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> Message-ID: On Jun 12, 2009, at 5:17 PM, Raymond Hettinger wrote: > > [Matthew Wilkes] >> Oh, I didn't mean they should use proprietary software, just that >> in my experience the kind of people who are active in open source >> are quite anti-war, green, etc. There are notable exceptions, but >> I know people who worry that their work will have military >> applications, and who turn down projects because of it. > > I question the whole notion of using open source in military weapons. > It seems like a rather basic violation of operational security. > Perhaps > your enemies will exploit your bugs instead of nicely reporting them > and submitting patches on SourceForge ;-) FYI Python (Jython) is already used in production of the F-35. There was a talk @ PyCon '08 about it: http://us.pycon.org/2008/conference/schedule/event/27/ http://www.youtube.com/watch?v=zgE55z_RNgQ -- Philip Jenvey From ncoghlan at gmail.com Sat Jun 13 04:24:13 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 13 Jun 2009 12:24:13 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090612193707.24f5563c@cylix> References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix> Message-ID: <4A330DCD.4080608@gmail.com> Frederick Reeve wrote: > The other point still stands though. I would like to see peek > changed. I am willing to write and submit changes but don't want to > unless others agree this is a good idea. So I put forth the > implementation at the bottom of this email. If its bad or you don't > see the point I may try to clarify but if nobody things its good, > please just tell me I'm waisting your time, and I will go away. I > also apologize my last email was so long. > > peek(n): If n is less than 0, None, or not set; return buffer > contents with out advancing stream position. If the buffer is empty > read a full chunk and return the buffer. Otherwise return exactly n > bytes up to _chunk size_(not contents) with out advancing the stream > position. If the buffer contents is less than n, buffer an > additional chunk from the "raw" stream before hand. If EOF is > encountered during any raw read then return as much as we can up to > n. (maybe I should write that in code form??) I would phrase this suggestion as users having a reasonable expectation that the following invariant should hold for a buffered stream: f.peek(n) == f.read(n) Since the latter method will perform as many reads of the underlying stream as necessary to reach the requested number of bytes (or EOF), then so should the former. However, the default value for n for peek() should remain at 1 to remain consistent with the current documented behaviour. If this invariant was implemented, I would also suggest adding a "peek1" method to parallel "read1". Note that the current behaviour I get from Python 3.1 is for it to return the *entire* buffer, no matter what number I pass to it: (current Py3k head) >>> f = open('setup.py', 'rb') >>> len(f.peek(10)) 4096 >>> len(f.peek(1)) 4096 >>> len(f.peek(4095)) 4096 >>> len(f.peek(10095)) 4096 That's an outright bug - I've promoted an existing issue about this [1] to a release blocker and sent it to Benjamin to have another look at. Cheers, Nick. [1] http://bugs.python.org/issue5811 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From cs at zip.com.au Sat Jun 13 07:12:22 2009 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 13 Jun 2009 15:12:22 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A330DCD.4080608@gmail.com> Message-ID: <20090613051222.GA23544@cskk.homeip.net> On 13Jun2009 12:24, Nick Coghlan wrote: | Frederick Reeve wrote: | > The other point still stands though. I would like to see peek | > changed. I am willing to write and submit changes but don't want to | > unless others agree this is a good idea. So I put forth the | > implementation at the bottom of this email. If its bad or you don't | > see the point I may try to clarify but if nobody things its good, | > please just tell me I'm waisting your time, and I will go away. I | > also apologize my last email was so long. | > | > peek(n): If n is less than 0, None, or not set; return buffer | > contents with out advancing stream position. If the buffer is empty | > read a full chunk and return the buffer. Otherwise return exactly n | > bytes up to _chunk size_(not contents) with out advancing the stream | > position. If the buffer contents is less than n, buffer an | > additional chunk from the "raw" stream before hand. If EOF is | > encountered during any raw read then return as much as we can up to | > n. (maybe I should write that in code form??) | | I would phrase this suggestion as users having a reasonable expectation | that the following invariant should hold for a buffered stream: | | f.peek(n) == f.read(n) | | Since the latter method will perform as many reads of the underlying | stream as necessary to reach the requested number of bytes (or EOF), | then so should the former. I disagree. If that were that case, why have peek() at all? I realise that it doesn't move the logical position, but it does mean that peek(huge_number) imposes a requirement to grow the stream buffer arbitrarily. A peek that does at most one raw read has the advantage that it can pick up data outside the buffer but lurking in the OS buffer, yet to be obtained. Those data are free, if they're present. (Of course, if they're absent peek() wil still block). Since (if the OS buffer is also empty) even a peek(1) can block, maybe we should canvas peek()'s common use cases. Naively (never having used peek()), my own desire would normally be for a peek(n, block=False) a bit like Queue.get(). Then I could be sure not to block if I wanted to avoid it, even on a blocking stream, yet still obtain unread buffered data if present. So: what do people use peek() for, mostly? Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ We're in the business of putting goo on a substrate. - overhead by WIRED at the Intelligent Printing conference Oct2006 From ncoghlan at gmail.com Sat Jun 13 08:59:02 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 13 Jun 2009 16:59:02 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090613051222.GA23544@cskk.homeip.net> References: <20090613051222.GA23544@cskk.homeip.net> Message-ID: <4A334E36.6000705@gmail.com> Cameron Simpson wrote: > On 13Jun2009 12:24, Nick Coghlan wrote: > | I would phrase this suggestion as users having a reasonable expectation > | that the following invariant should hold for a buffered stream: > | > | f.peek(n) == f.read(n) > | > | Since the latter method will perform as many reads of the underlying > | stream as necessary to reach the requested number of bytes (or EOF), > | then so should the former. > > I disagree. If that were that case, why have peek() at all? I realise > that it doesn't move the logical position, but it does mean that > peek(huge_number) imposes a requirement to grow the stream buffer > arbitrarily. > > A peek that does at most one raw read has the advantage that it can pick up > data outside the buffer but lurking in the OS buffer, yet to be obtained. > Those data are free, if they're present. (Of course, if they're absent > peek() wil still block). Note my suggestion later that if the above invariant were to be adopted then a peek1() method should be added to parallel read1(). However, from what Benjamin has said, a more likely invariant is going to be: preview = f.peek(n) f.read(n).startswith(preview) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From eric.pruitt at gmail.com Sat Jun 13 03:55:30 2009 From: eric.pruitt at gmail.com (Eric Pruitt) Date: Fri, 12 Jun 2009 20:55:30 -0500 Subject: [Python-Dev] Status of 2.7 and 3.2 Message-ID: <171e8a410906121855v17f211dred8f1c4cbdf96f3c@mail.gmail.com> I am in the process of implementing a number of often requested features and proposed patches in the subprocess module for my Google Summer of Code 2009 project. For information on my progress, check out my blog located at * http://subdev.blogspot.com/* . Any comments and suggestions are greatly appreciated. > Just my 0.02 cents, but struggling with all warts of 2.5 subprocessing > > in Windows I would vote for more time for stabilizating things - not > > adding new features. Long awaited subprocess as replacement for > > os.popen() AFAIK is still incapable to asynchronously communicate with > > spawned processes on Windows. I would call this feature as critical > > even on 2.6 As a release testcase - try porting pyexpect module to > > this platform. Absence of native curses/console/readline module also > > makes Python one-way unix shell language while many users expect it to > > be crossplatform. > > I am not quite sure whether you are for new features or not. Your > first sentence ("vote for ... not adding new features") seems to > suggest that you would not like to see new features, and your last > sentence ("absence of native curses/console/readline module") > suggests that you *do* want to see new features (namely, a native > curses module, and a native readline module). > > Which one is it? > > Regards, > Martin > -------------- next part -------------- An HTML attachment was scrubbed... URL: From regebro at gmail.com Sat Jun 13 11:05:12 2009 From: regebro at gmail.com (Lennart Regebro) Date: Sat, 13 Jun 2009 11:05:12 +0200 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A2B6E39.3090405@v.loewis.de> <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <319e029f0906130205s337b77f4t50ef992cf8d75412@mail.gmail.com> 2009/6/7 Stephen J. Turnbull : > Python's 2.x/py3k division is a tour de force; I still can't believe > my eyes that you've pulled it off. Well, It's not pulled off until Python 3 has surpassed Python 2 in usage. That's still a long way away. I'm not familiar with the other examples except Zope 3, which is a completely different thing, so it may very well be that Python did it way better. But still, it's not pulled off yet. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 From stephen at xemacs.org Sat Jun 13 13:58:13 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 13 Jun 2009 20:58:13 +0900 Subject: [Python-Dev] Status of 2.7 and 3.2 In-Reply-To: <319e029f0906130205s337b77f4t50ef992cf8d75412@mail.gmail.com> References: <4A2B6E39.3090405@v.loewis.de> <87hbysfcym.fsf@uwakimon.sk.tsukuba.ac.jp> <319e029f0906130205s337b77f4t50ef992cf8d75412@mail.gmail.com> Message-ID: <87fxe4wdka.fsf@uwakimon.sk.tsukuba.ac.jp> Lennart Regebro writes: > 2009/6/7 Stephen J. Turnbull : > > Python's 2.x/py3k division is a tour de force; I still can't believe > > my eyes that you've pulled it off. > > Well, It's not pulled off until Python 3 has surpassed Python 2 in > usage. I'm referring only to the management of the dual-trunk workflow, not to whether Python3 will be the preferred path for the future. That workflow has its problems, but still it seems very successful in keeping what needs to be synced, synced, and what needs to be different, different. From solipsis at pitrou.net Sat Jun 13 14:16:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 13 Jun 2009 12:16:14 +0000 (UTC) Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix> <4A330DCD.4080608@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > > Since the latter method will perform as many reads of the underlying > stream as necessary to reach the requested number of bytes (or EOF), > then so should the former. How do you propose to implement this while staying compatible with 1) unseekable raw streams 2) the expectation that peek() doesn't advance the logical file pointer? > Note that the current behaviour I get from Python 3.1 is for it to > return the *entire* buffer, no matter what number I pass to it: > [...] > > That's an outright bug - I've promoted an existing issue about this [1] > to a release blocker and sent it to Benjamin to have another look at. The original docstring for peek() says: """Returns buffered bytes without advancing the position. The argument indicates a desired minimal number of bytes; we do at most one raw read to satisfy it. We never return more than self.buffer_size. """ In that light, I'm not sure it's a bug -- although it can certainly look unexpected at first sight. Regards Antoine. From solipsis at pitrou.net Sat Jun 13 14:33:46 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 13 Jun 2009 12:33:46 +0000 (UTC) Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix> Message-ID: Frederick Reeve solace.info> writes: > > peek(n): > If n is less than 0, None, or not set; return buffer contents with out > advancing stream position. If the buffer is empty read a full chunk and > return the buffer. Otherwise return exactly n bytes up to _chunk > size_(not contents) with out advancing the stream position. If the > buffer contents is less than n, buffer an additional chunk from the > "raw" stream before hand. If EOF is encountered during any raw read > then return as much as we can up to n. (maybe I should write that in > code form??) This proposal looks reasonable to me. Please note that it's too late for 3.1 anyway - we're in release candidate phase. Once you have a patch, you can post it on the bug tracker. Regards Antoine. From solipsis at pitrou.net Sat Jun 13 14:35:12 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 13 Jun 2009 12:35:12 +0000 (UTC) Subject: [Python-Dev] Status of 2.7 and 3.2 References: <171e8a410906121855v17f211dred8f1c4cbdf96f3c@mail.gmail.com> Message-ID: Eric Pruitt gmail.com> writes: > > I am in the process of implementing a > number of often requested features and proposed patches in the subprocess > module for my Google Summer of Code 2009 project. For information on > my progress, check out my blog located at?http://subdev.blogspot.com/. Please note there's already a Mercurial mirror of the Python trunk: http://code.python.org/hg/trunk/ (I'm replying here since your blog disallows anonymous comments) Antoine. From facundobatista at gmail.com Sat Jun 13 14:40:49 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Sat, 13 Jun 2009 09:40:49 -0300 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: <4A32ED75.7060803@cheimes.de> References: <4A32ED75.7060803@cheimes.de> Message-ID: On Fri, Jun 12, 2009 at 9:06 PM, Christian Heimes wrote: > How about a nice 'n shiny context wrapper for the pipe: I'll do this! Thank you for the suggestion! BTW, as this is a good way of avoiding the FD leakage, should this context wrapper for pipe() be in the stdlib? Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From hfuerstenau at gmx.net Sat Jun 13 14:45:56 2009 From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=) Date: Sat, 13 Jun 2009 13:45:56 +0100 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <4A32FA7D.1060906@gmail.com> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> Message-ID: <4A339F84.2070400@gmx.net> > The semantic change actually needed to make nested() more equivalent to > the multi-with statement is for it to accept zero-argument callables > that create context managers as arguments rather than pre-created > context managers. It seems to me that both passing callables which return managers and passing a generator which yields managers achieve about the same thing. Are you proposing the former just to avoid introducing a new interface? > Rather than changing the name of the function, this could be done by > inspecting the first argument for an "__enter__" method. If it has one, > use the old semantics (and issue a DeprecationWarning as in 3.1). > Otherwise, use the proposed new semantics. I guess this is much too late for 3.1, but could we then at least un-deprecate "contextlib.nested" for now? As it is, you get a DeprecationWarning for something like with contextlib.nested(*my_managers): without any good way to get rid of it. - Hagen From benjamin at python.org Sat Jun 13 16:46:28 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 13 Jun 2009 09:46:28 -0500 Subject: [Python-Dev] [RELEASED] Python 3.1 Release Candidate 2 Message-ID: <1afaf6160906130746v2d951486v7a52d7b838070189@mail.gmail.com> On behalf of the Python development team, I'm happy to announce the second release candidate of Python 3.1. Python 3.1 focuses on the stabilization and optimization of the features and changes that Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. Other features include an ordered dictionary implementation, a condensed syntax for nested with statements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/dev/py3k/whatsnew/3.1.html or Misc/NEWS in the Python distribution. This is a release candidate, and as such, we do not recommend use in production environments. However, please take this opportunity to test the release with your libraries or applications. This will hopefully discover bugs before the final release and allow you to determine how changes in 3.1 might impact you. If you find things broken or incorrect, please submit a bug report at http://bugs.python.org For more information and downloadable distributions, see the Python 3.1 website: http://www.python.org/download/releases/3.1/ See PEP 375 for release schedule details: http://www.python.org/dev/peps/pep-0375/ Enjoy, -- Benjamin Benjamin Peterson benjamin at python.org Release Manager (on behalf of the entire python-dev team and 3.1's contributors) From pebarrett at gmail.com Sat Jun 13 19:02:47 2009 From: pebarrett at gmail.com (Paul Barrett) Date: Sat, 13 Jun 2009 13:02:47 -0400 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FORTHE F-35 JOINT STRIKE FIGHTER In-Reply-To: <4A330135.8060803@gmail.com> References: <958091.40012.qm@web34401.mail.mud.yahoo.com> <4B23F475-2DDE-4929-8993-B568922F5A5B@matthewwilkes.co.uk> <7CFD8BD5-784E-4927-8A7C-8422472B441E@matthewwilkes.co.uk> <73492FDFAB444938B5185B04F8E39DF4@RaymondLaptop1> <4A330135.8060803@gmail.com> Message-ID: <40e64fa20906131002i61c44668of03092456828b830@mail.gmail.com> The other benefit of the military using open source software is that is can save the taxpayers money over the short and long term. For some projects it is a small percentage of the total cost. However, for others it can be significant portion of the cost, so don't discount its use or its benefit. Saving time and money is a good thing. Cheers, Paul On Fri, Jun 12, 2009 at 9:30 PM, Nick Coghlan wrote: > Raymond Hettinger wrote: >> >> [Matthew Wilkes] >>> Oh, I didn't mean they should use proprietary software, just that in >>> my experience the kind of people who are active in open source are >>> quite anti-war, green, etc. ?There are notable exceptions, but I know >>> people who worry that their work will have military applications, and >>> who turn down projects because of it. >> >> I question the whole notion of using open source in military weapons. >> It seems like a rather basic violation of operational security. ?Perhaps >> your enemies will exploit your bugs instead of nicely reporting them >> and submitting patches on SourceForge ;-) > > As Guido said, even the military are aware that there are major problems > with the idea of security through obscurity. Plus most defence forces > around the world are just as interested in saving a few bucks on > software costs as any other organisation, particularly if they can > reduce their reliance on a foreign software vendor in the process. > > As to the existence of open source developers that are willing to work > with the military... as Matthew said, while they may not be particularly > common, I can definitely say that they're around ;) > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/pebarrett%40gmail.com > From zooko at zooko.com Sat Jun 13 18:58:16 2009 From: zooko at zooko.com (Zooko Wilcox-O'Hearn) Date: Sat, 13 Jun 2009 10:58:16 -0600 Subject: [Python-Dev] ctime: I don't think that word means what you think it means. Message-ID: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com> The stat module uses the "st_ctime" slot to hold two kinds of values which are semantically different and which are frequently confused with one another. It chooses which kind of value to put in there based on platform -- Windows gets the file creation time and all other platforms get the "ctime". The only sane way to use this API is then to switch on platform: if platform.system() == "Windows": metadata["creation time"] = s.st_ctime else: metadata["unix ctime"] = s.st_ctime (That is an actual code snippet from the Allmydata-Tahoe project.) Many or even most programmers incorrectly think that unix ctime is file creation time, so instead of using the sane idiom above, they write the following: metadata["ctime"] = s.st_ctime thus passing on the confusion to the users of their metadata, who may not be able to tell on which platform this metadata was created. This is the situation we have found ourselves in for the Allmydata-Tahoe project -- we now have a bunch of "ctime" values stored in our filesystem and no way to tell which kind they were. More and more filesystems such as ZFS and Mac HFS+ apparently offer creation time nowadays. I propose the following changes: 1. Add a "st_crtime" field which gets populated on filesystems (Windows, ZFS, Mac) which can do so. That is hopefully not too controversial and we could proceed to do so even if the next proposal gets bogged down: 2. Add a "st_unixctime" field which gets populated *only* by the unix ctime and never by any other value (even on Windows, where the unix ctime *is* available even though nobody cares about it), and deprecate the hopelessly ambiguous "st_ctime" field. You may be interested in http://allmydata.org/trac/tahoe/ticket/628 ("mtime" and "ctime": I don't think that word means what you think it means.) where the Allmydata-Tahoe project is carefully unpicking the mess we made for ourselves by confusing ctime with file-creation time. This is ticket http://bugs.python.org/issue5720 . Regards, Zooko From omega_force2003 at yahoo.com Sat Jun 13 21:35:57 2009 From: omega_force2003 at yahoo.com (OMEGA RED) Date: Sat, 13 Jun 2009 12:35:57 -0700 (PDT) Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER Message-ID: <331154.6420.qm@web34401.mail.mud.yahoo.com> Guido, ? ? The reason why I use a code name is that it would be dangerous to reveal my true name to the world since there are those who are radical liberals who would like to think that there is no need for a U.S. military and that people such as myself should be eliminated.? Think of the U.S. military as the heroes that they truly are?to protect its citizens from attack from other entities.? The U.S. military is here to serve and protect people such as yourself, not to act as the terrorist organization as?bin laden would like?you to think.? ? If any of you need any further evidence of the need of the U.S. military or the F-35 for that matter, then I would suggest that you all take a nice long trip to Manhattan island in New York City and take some time to notice the fact that there appears to be a large hole where many?buildings used to be located.? ? ? Second, the biggest concern that I see in utilizing the python language in safety-critical systems is that it is a dynamically type language.? Unlike Ada which is safety statically type language.? These were just brainstorming ideas, but if you do not like those that brainstorm regarding the further development of python, then I guess I will be moving forward to some other language that is more capable to assist in the development of safety?critical systems.? These were just brainstorming ideas.? ? The idea being is to research onto the idea of developing a statically?type variant of the python language?for the development of software for safety critical systems.? Is this possible?? I am also wondering if it was possible to interface Ada programs with Python, similar to the method that has been utilized to interface FORTRAN with Python.? Is this also possible?? ? ? ? Thank You.? ? ? W108dab ? ? ? ? ? --- On Sat, 6/13/09, Guido van Rossum wrote: From: Guido van Rossum Subject: Re: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER To: "OMEGA RED" omega_force2003 at yahoo.com Date: Saturday, June 13, 2009, 11:58 AM I'm sorry, but I have so far three problems with your proposals. (1) You use a code name. (2) Your first message was ALL CAPS. (3) You don't seem to know much about programming language design, or you wouldn't propose such a vague non-starter as "adopt the safety-critical charasteristics of the Ada programming language an incorporate them into Python." This is my last message to you. --Guido On Sat, Jun 13, 2009 at 7:27 AM, OMEGA RED wrote: > Yes, > > I think that python developers would be great in the development of the > FADEC controller as an open-source alternative to the propreitary FADEC > controller software to the propulsion system for the F-35.? The FADEC > controller is Full Authority Flight Digital Electronic Controller. > > I know that the python community could never possibly create the mechanical > aspects of this proposed engine, but that does not mean that we cannot > create something else. > > I believe that what could be done is to create a safety-critical version of > python.? Such as adopt the safety-critical charasteristics of the Ada > programming language an incorporate them into python. > > How about that as a project?? Create the python equivalent of Ada? > > Thank You, > > w108dab > > > Subject: Re: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR > THE F-35 JOINT STRIKE FIGHTER > To: "OMEGA RED" > Date: Saturday, June 13, 2009, 6:16 AM > > > I'm reasonably confused here,?are you searching for Python developers? > > > > > On Fri, Jun 12, 2009 at 7:01 PM, OMEGA RED > wrote: >> >> >> TO ALL, >> >> >> I AM HAPPY TO SAY THAT I AM NOW IN THE PROCESS OF DEVELOPING A WEBSITE FOR >> THE STATED PURPOSE OF COMBINING SOME OF THE BEST MINDS IN THE WORLD.? THIS >> PROJECT WILL BE A MEANS TO DEVELOP THE FIRST AND ONLY COMPLETELY OPEN SOURCE >> VARIANT OF THE PROPULSION ENGINE FOR THE F-35 JOINT STRIKE FIGHTER.? WHICH >> MEANS THAT EVERYTHING ABOUT THE DESIGN, DEVELOPMENT, AND CONSTRUCTION OF THE >> ENGINE WILL BE COMPLETELY AVAILABLE TO THE PUBLIC AND TO ANY ENGINE >> MANUFACTURE FREE OF ANY REQUIREMENTS. >> >> THIS ENGINE WILL BE CALLED THE PHOENIX NexT F-200 ENGINE. >> >> THIS ENGINE WILL ALSO BE A POTENTIAL THIRD AND FINAL ALTERNATIVE ENGINE >> FOR THE JOINT STRIKE FIGHTER. >> >> THIS ENGINE WILL BE STATED AS BEING A COMPETITOR WITH BOTH THE F-135 AND >> THE F-136 ENGINE, WHICH ARE PROVIDED BY BOTH PRATT-WHITNEY AND GENERAL >> ELECTRIC/ ROLLS ROYCE. >> >> THIS ENGINE WILL UTILIZE PYTHON FOR THE MAJOR PORTION OF THE SIMULATION, >> RESEARCH AND DEVELOPMENT STAGES FOR THE DEVELOPMENT OF THE ENGINE.? THE >> CONSTRUCTION OF THE MATHEMATICAL PROCESSING OF THE REQUIRED NONLINEAR >> CONTROLS WILL BE BASED ON A SPECIALIZED VERSION OF GROUP THEORY FOR THE >> DESIGN AND CONSTRUCTION OF THE NONLINEAR CONTROL SYSTEMS. >> >> >> THE PURPOSE OF THE DEVELOPMENT OF THE ENGINE WILL BE TO CONSTRUCT AN >> ENGINE FOR THE F-35 THAT WILL BE AS RELIABLE AND CAPABLE AS THE F-135 AND >> THE F-136. >> >> HOWEVER, THE PHOENIX NexT ENGINE WILL ALSO BE STATED AS BEING ONLY AROUND >> $20,000 TO $110,000 DOLLARS TO BEING PURCHASED BY AN OPERATOR OF THE F-35, >> WHICH WILL BE AN INSIGNIFICANT COST COMPARED TO THE COST OF BOTH THE F-135 >> AND THE F-136 ENGINE (10 MILLION DOLLARS PER UNIT). >> >> HOW MANY OUT THERE WANT TO HELP IN THIS ENDEAVOR ?? I BELIEVE THAT WE >> ALL?CAN SUCCEED WHERE BOTH PRATT-WHITNEY AND GE/ROLLS ROYCE HAVE FAILED TO >> DEVELOP AN INEXPENSIVE AND RELIABLE ENGINE FOR THE F-35. >> >> THANKS, >> >> W108DAB >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/josepharmbruster%40gmail.com >> > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: From arcriley at gmail.com Sat Jun 13 21:52:53 2009 From: arcriley at gmail.com (Arc Riley) Date: Sat, 13 Jun 2009 15:52:53 -0400 Subject: [Python-Dev] FINAL PROPULSION OPEN SOURCE ENGINE VARIANT FOR THE F-35 JOINT STRIKE FIGHTER In-Reply-To: <331154.6420.qm@web34401.mail.mud.yahoo.com> References: <331154.6420.qm@web34401.mail.mud.yahoo.com> Message-ID: Enough is enough guys. As entertaining as this thread has been, shouldn't we be focused on the 3.1 release? Don't feed the trolls. Ok so one wandered in, but nobody needed to respond and it can only get worse from here. Please just flag the offending address(es) for moderation and ask them politely to keep their posts to this list on-topic for core Python development. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sun Jun 14 02:33:35 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 14 Jun 2009 12:33:35 +1200 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix> <4A330DCD.4080608@gmail.com> Message-ID: <4A34455F.1030106@canterbury.ac.nz> Antoine Pitrou wrote: > The original docstring for peek() says: > > ...we > do at most one raw read to satisfy it. > > In that light, I'm not sure it's a bug It may be behaving according to the docs, but is that behaviour useful? Seems to me that if you're asking for n bytes, then it's because you're doing some kind of parsing that requires lookahead, and nothing less than n bytes will do. I think it would be more useful if the "at most one raw read" part were dropped. That would give it the kind of deterministic behaviour generally expected when dealing with buffered streams. -- Greg From solipsis at pitrou.net Sun Jun 14 02:35:16 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 14 Jun 2009 00:35:16 +0000 (UTC) Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix> <4A330DCD.4080608@gmail.com> <4A34455F.1030106@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > I think it would be more useful if the "at most one > raw read" part were dropped. That would give it the > kind of deterministic behaviour generally expected > when dealing with buffered streams. As I already told Nick: please propose an implementation scheme. Antoine. From greg.ewing at canterbury.ac.nz Sun Jun 14 02:40:53 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 14 Jun 2009 12:40:53 +1200 Subject: [Python-Dev] ctime: I don't think that word means what you think it means. In-Reply-To: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com> References: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com> Message-ID: <4A344715.7010604@canterbury.ac.nz> Zooko Wilcox-O'Hearn wrote: > 1. Add a "st_crtime" field which gets populated on filesystems > (Windows, ZFS, Mac) which can do so. "crtime" looks rather too similar to "ctime" for my liking. People who think that the "c" in "ctime" means "creation" are still likely to confuse them. Why not give it a more explicit name, such as "st_creationtime"? -- Greg From ncoghlan at gmail.com Sun Jun 14 03:22:37 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 14 Jun 2009 11:22:37 +1000 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <4A339F84.2070400@gmx.net> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> Message-ID: <4A3450DD.6040905@gmail.com> Hagen F?rstenau wrote: > I guess this is much too late for 3.1, but could we then at least > un-deprecate "contextlib.nested" for now? As it is, you get a > DeprecationWarning for something like > > with contextlib.nested(*my_managers): > > without any good way to get rid of it. I actually almost asked for that to be changed to a PendingDeprecationWarning when it was first added - Benjamin, do you mind if I downgrade this warning to a pending one post rc2? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From jeremy.cowles at gmail.com Sun Jun 14 04:45:06 2009 From: jeremy.cowles at gmail.com (Jeremy Cowles) Date: Sat, 13 Jun 2009 19:45:06 -0700 Subject: [Python-Dev] Distributed computing & sending the interpreter Message-ID: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com> Hi all, I apologize if this question is misplaced, I wasn't sure which list to post it in. I'm working on a distributed computing project (PyMW and BOINC) where we are sending Python scripts to client machines. Currently, we make two very unlikely assumptions: that Python 2.5 is installed and that the interpreter is available on the PATH. We are looking at our options to remove this assumption and the two most likely are redistributing the entire Python installation (for each supported platform) and embedding the interpreter in a custom executable and sending select parts of the standard library with it (to reduce size). It seems like we would have to pre-compile for each different platform, which is a pain and sending the entire installation could be tricky. I am looking for alternatives, comments and suggestions. Any help would be greatly appreciated. Thanks, Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Sun Jun 14 05:44:23 2009 From: aahz at pythoncraft.com (Aahz) Date: Sat, 13 Jun 2009 20:44:23 -0700 Subject: [Python-Dev] Distributed computing & sending the interpreter In-Reply-To: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com> References: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com> Message-ID: <20090614034422.GA15064@panix.com> On Sat, Jun 13, 2009, Jeremy Cowles wrote: > > I apologize if this question is misplaced, I wasn't sure which list to > post it in. > > I'm working on a distributed computing project (PyMW and BOINC) where > we are sending Python scripts to client machines. Currently, we make > two very unlikely assumptions: that Python 2.5 is installed and that > the interpreter is available on the PATH. Definitely the wrong list -- python-dev is for people working on the core interpreter and libraries. comp.lang.python would be the standard place, but you might also find some good advice on the new list concurrency-sig. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer From jeremy.cowles at gmail.com Sun Jun 14 05:48:35 2009 From: jeremy.cowles at gmail.com (Jeremy Cowles) Date: Sat, 13 Jun 2009 20:48:35 -0700 Subject: [Python-Dev] Distributed computing & sending the interpreter In-Reply-To: <20090614034422.GA15064@panix.com> References: <373cf0740906131945o6f17d70dnaa54b2d2b6c85833@mail.gmail.com> <20090614034422.GA15064@panix.com> Message-ID: <373cf0740906132048u3f0989dfx546b689ddf5ca919@mail.gmail.com> > > Definitely the wrong list -- python-dev is for people working on the core > interpreter and libraries. comp.lang.python would be the standard place, > but you might also find some good advice on the new list concurrency-sig. Sorry for the list pollution, I will repost it there. Thanks, Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Sun Jun 14 07:16:18 2009 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 14 Jun 2009 15:16:18 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A34455F.1030106@canterbury.ac.nz> Message-ID: <20090614051618.GA19302@cskk.homeip.net> On 14Jun2009 12:33, Greg Ewing wrote: > Antoine Pitrou wrote: >> The original docstring for peek() says: >> >> ...we >> do at most one raw read to satisfy it. >> >> In that light, I'm not sure it's a bug > > It may be behaving according to the docs, but is that > behaviour useful? > > Seems to me that if you're asking for n bytes, then it's > because you're doing some kind of parsing that requires > lookahead, and nothing less than n bytes will do. > > I think it would be more useful if the "at most one > raw read" part were dropped. That would give it the > kind of deterministic behaviour generally expected > when dealing with buffered streams. Is it possible to access the buffer? I see nothing in the docs. People seem to want peek() to be "read() without moving the read offset", which it almost seems to be. Nick and Greg both want it to really be that, and thus do enough raw reads to get "n" bytes; Nick wants a peek1() like read1(), too. It has a pleasing feel to me, too. But ... For myself, I'd expect more often to want to see if there's stuff in the buffer _without_ doing any raw reads at all. A peek0(n), if you will: Read and return up to n bytes without calling on the raw stream. It feels like peek is trying to span both extremes and doesn't satisfy either really well. If peek gets enhanced to act like read in terms of the amount of data returned, should there not be a facility to examine buffered data without raw reads? Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Being on a Beemer and not having a wave returned by a Sportster is like having a clipper ship's hailing not returned by an orphaned New Jersey solid waste barge. - OTL From cs at zip.com.au Sun Jun 14 07:26:47 2009 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 14 Jun 2009 15:26:47 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090614051618.GA19302@cskk.homeip.net> Message-ID: <20090614052647.GA23072@cskk.homeip.net> On 14Jun2009 15:16, I wrote: | Is it possible to access the buffer? I see nothing in the docs. I've just found getvalue() in IOBase. Forget I said anything. It seems to be my day for that kind of post:-( -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ These are but a few examples of what can happen when the human mind is employed to learn, to probe, to question as opposed to merely keeping the ears from touching. - rec.humor.funny 90.07.16 From hfuerstenau at gmx.net Sun Jun 14 10:18:28 2009 From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=) Date: Sun, 14 Jun 2009 09:18:28 +0100 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <4A3450DD.6040905@gmail.com> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> Message-ID: <4A34B254.6050907@gmx.net> > I actually almost asked for that to be changed to a > PendingDeprecationWarning when it was first added - Benjamin, do you > mind if I downgrade this warning to a pending one post rc2? I'm not sure what that would buy us. For the use case I mentioned it would be just as annoying to get a PendingDeprecationWarning. But if the warning was completely removed now, nested could still get deprecated in 3.2 as soon as some better mechanism for a variable number of managers has been provided. - Hagen From steve at pearwood.info Sun Jun 14 10:59:41 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 14 Jun 2009 18:59:41 +1000 Subject: [Python-Dev] ctime: I don't think that word means what you think it means. In-Reply-To: <4A344715.7010604@canterbury.ac.nz> References: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com> <4A344715.7010604@canterbury.ac.nz> Message-ID: <200906141859.42328.steve@pearwood.info> On Sun, 14 Jun 2009 10:40:53 am Greg Ewing wrote: > Zooko Wilcox-O'Hearn wrote: > > 1. Add a "st_crtime" field which gets populated on filesystems > > (Windows, ZFS, Mac) which can do so. > > "crtime" looks rather too similar to "ctime" for my > liking. People who think that the "c" in "ctime" > means "creation" are still likely to confuse them. > > Why not give it a more explicit name, such > as "st_creationtime"? Speaking as somebody who thought the c in ctime meant creation, I'm +1 on this proposal with Greg's modification. -- Steven D'Aprano From Scott.Daniels at Acm.Org Sun Jun 14 14:49:12 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Sun, 14 Jun 2009 05:49:12 -0700 Subject: [Python-Dev] ctime: I don't think that word means what you think it means. In-Reply-To: <200906141859.42328.steve@pearwood.info> References: <69132116-861D-4362-A30D-35BCC45A6772@zooko.com> <4A344715.7010604@canterbury.ac.nz> <200906141859.42328.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > On Sun, 14 Jun 2009 10:40:53 am Greg Ewing wrote: >> Zooko Wilcox-O'Hearn wrote: >>> 1. Add a "st_crtime" field which gets populated on filesystems >>> (Windows, ZFS, Mac) which can do so. >> "crtime" looks rather too similar to "ctime" for my >> liking. People who think that the "c" in "ctime" >> means "creation" are still likely to confuse them. >> >> Why not give it a more explicit name, such >> as "st_creationtime"? My this bike-shed looks a bit greenish. How about: creation_time or, at least, st_creation_time > Speaking as somebody who thought the c in ctime meant creation, I'm +1 > on this proposal with Greg's modification. --Scott David Daniels Scott.Daniels at Acm.Org From benjamin at python.org Sun Jun 14 16:21:32 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 14 Jun 2009 09:21:32 -0500 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090614052647.GA23072@cskk.homeip.net> References: <20090614051618.GA19302@cskk.homeip.net> <20090614052647.GA23072@cskk.homeip.net> Message-ID: <1afaf6160906140721i27610ab5j245a1d8678a73c21@mail.gmail.com> 2009/6/14 Cameron Simpson : > On 14Jun2009 15:16, I wrote: > | Is it possible to access the buffer? I see nothing in the docs. > > I've just found getvalue() in IOBase. Forget I said anything. > It seems to be my day for that kind of post:-( Where are you seeing this? Only BytesIO and StringIO have a getvalue() method. -- Regards, Benjamin From benjamin at python.org Sun Jun 14 16:24:46 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 14 Jun 2009 09:24:46 -0500 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <4A3450DD.6040905@gmail.com> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> Message-ID: <1afaf6160906140724i72aa9d47mdffa5b24a052a67f@mail.gmail.com> 2009/6/13 Nick Coghlan : > Hagen F?rstenau wrote: >> I guess this is much too late for 3.1, but could we then at least >> un-deprecate "contextlib.nested" for now? As it is, you get a >> DeprecationWarning for something like >> >> with contextlib.nested(*my_managers): >> >> without any good way to get rid of it. > > I actually almost asked for that to be changed to a > PendingDeprecationWarning when it was first added - Benjamin, do you > mind if I downgrade this warning to a pending one post rc2? Yes, I think that's a good idea. It will also help people who have to use contextlib.nested() for backwards compatibility. -- Regards, Benjamin From mrs at mythic-beasts.com Sun Jun 14 18:09:36 2009 From: mrs at mythic-beasts.com (Mark Seaborn) Date: Sun, 14 Jun 2009 17:09:36 +0100 (BST) Subject: [Python-Dev] CPython in the web browser under Native Client Message-ID: <20090614.170936.846958184.mrs@localhost.localdomain> I have been doing some work to extend Google's Native Client [1] to support dynamic linking [2]. For those who haven't heard of it, Native Client is a sandboxing system for running a subset of x86 code. It is proposed as a way of running native code inside web apps. One of my aims has been to get CPython working in the web browser under Native Client without having to modify CPython. I recently got to the point where modules from the Python standard library are importable under Native Client, including (as a demonstration) the Sqlite extension module. Sqlite also requires no modification - it builds straight from the Debian package. I've written a simple REPL to demonstrate Python running in the browser. There are some screenshots on my blog [3]. I haven't implemented accessing the DOM from Python yet - that's another project for later. :-) Mark [1] http://code.google.com/p/nativeclient/ [2] http://plash.beasts.org/wiki/NativeClient [3] http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html From mrs at mythic-beasts.com Sun Jun 14 17:42:55 2009 From: mrs at mythic-beasts.com (Mark Seaborn) Date: Sun, 14 Jun 2009 16:42:55 +0100 (BST) Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: References: Message-ID: <20090614.164255.343165597.mrs@localhost.localdomain> Facundo Batista wrote: > errpipe_read, errpipe_write = os.pipe() > try: > try: > ..... > ..... > ..... > ..... > ..... > ..... > finally: > os.close(errpipe_write) > ..... > ..... > ..... > finally: > os.close(errpipe_read) > > I just don't like a huge try/finally... but as FDs are just ints, do > you think is there a better way to handle it? I use a convenience function like this, so that GC takes care of the FDs: def make_pipe(): read_fd, write_fd = os.pipe() return os.fdopen(read_fd, "r"), os.fdopen(write_fd, "w") Mark From sanxiyn at gmail.com Sun Jun 14 21:09:25 2009 From: sanxiyn at gmail.com (Seo Sanghyeon) Date: Mon, 15 Jun 2009 04:09:25 +0900 Subject: [Python-Dev] Exception for setting attributes of built-in type Message-ID: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> Exception for setting attributes of built-in type differs between CPython and IronPython. This is not purely theoretical, as zope.interface tries to set Implements declaration as __implemented__ attribute of built-in type object, and excepts TypeError. Python 2.6.1 >>> object.flag = True TypeError: can't set attributes of built-in/extension type 'object' IronPython 2.6 >>> object.flag = True AttributeError: 'object' object has no attribute 'flag' I was surprised that CPython raises TypeError. Library Reference seems to mention it here: exception AttributeError Raised when an attribute reference or assignment fails. (When an object does not support attribute references or attribute assignments at all, TypeError is raised.) http://docs.python.org/library/exceptions.html What does it mean that "an object does not support attribute references or attribute assignments at all"? -- Seo Sanghyeon From tjreedy at udel.edu Mon Jun 15 00:40:56 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 14 Jun 2009 18:40:56 -0400 Subject: [Python-Dev] Exception for setting attributes of built-in type In-Reply-To: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> Message-ID: Seo Sanghyeon wrote: > Exception for setting attributes of built-in type differs between > CPython and IronPython. This is not purely theoretical, as > zope.interface tries to set Implements declaration as __implemented__ > attribute of built-in type object, and excepts TypeError. > > Python 2.6.1 >>>> object.flag = True > TypeError: can't set attributes of built-in/extension type 'object' > > IronPython 2.6 >>>> object.flag = True > AttributeError: 'object' object has no attribute 'flag' > > I was surprised that CPython raises TypeError. Library Reference seems > to mention it here: > > exception AttributeError > Raised when an attribute reference or assignment fails. (When an > object does not support attribute references or attribute assignments > at all, TypeError is raised.) > http://docs.python.org/library/exceptions.html > > What does it mean that "an object does not support attribute > references or attribute assignments at all"? I see it as slightly ambiguous: 1. It neither supports references nor assignments. 2. It either does not support reference or it does not support assignments. 1 could hardly apply any more, certainly to built-ins since everything now has attributes. 2 covers object, so if that is meant, then TypeError is appropriate. Or were you unclear about 'at all', which means 'never'? Terry Jan Reedy From python at mrabarnett.plus.com Mon Jun 15 00:42:56 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 14 Jun 2009 23:42:56 +0100 Subject: [Python-Dev] Exception for setting attributes of built-in type In-Reply-To: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> Message-ID: <4A357CF0.2050604@mrabarnett.plus.com> Seo Sanghyeon wrote: > Exception for setting attributes of built-in type differs between > CPython and IronPython. This is not purely theoretical, as > zope.interface tries to set Implements declaration as __implemented__ > attribute of built-in type object, and excepts TypeError. > > Python 2.6.1 >>>> object.flag = True > TypeError: can't set attributes of built-in/extension type 'object' > > IronPython 2.6 >>>> object.flag = True > AttributeError: 'object' object has no attribute 'flag' > > I was surprised that CPython raises TypeError. Library Reference seems > to mention it here: > > exception AttributeError > Raised when an attribute reference or assignment fails. (When an > object does not support attribute references or attribute assignments > at all, TypeError is raised.) > http://docs.python.org/library/exceptions.html > > What does it mean that "an object does not support attribute > references or attribute assignments at all"? > Here's my guess: Some objects have a fixed (and possibly empty) set of attributes. Attempting to add a new attribute or get a non-existent attribute raises an AttributeError. Other objects, such as types (are there any that aren't types?), don't have attributes at all. Attempting to add or get an attribute raises a TypeError. From guido at python.org Mon Jun 15 01:04:21 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 14 Jun 2009 16:04:21 -0700 Subject: [Python-Dev] CPython in the web browser under Native Client In-Reply-To: <20090614.170936.846958184.mrs@localhost.localdomain> References: <20090614.170936.846958184.mrs@localhost.localdomain> Message-ID: Wow. I'm impressed. On Sun, Jun 14, 2009 at 9:09 AM, Mark Seaborn wrote: > I have been doing some work to extend Google's Native Client [1] to > support dynamic linking [2]. ?For those who haven't heard of it, > Native Client is a sandboxing system for running a subset of x86 code. > It is proposed as a way of running native code inside web apps. > > One of my aims has been to get CPython working in the web browser > under Native Client without having to modify CPython. > > I recently got to the point where modules from the Python standard > library are importable under Native Client, including (as a > demonstration) the Sqlite extension module. ?Sqlite also requires no > modification - it builds straight from the Debian package. > > I've written a simple REPL to demonstrate Python running in the > browser. ?There are some screenshots on my blog [3]. ?I haven't > implemented accessing the DOM from Python yet - that's another project > for later. :-) > > Mark > > [1] http://code.google.com/p/nativeclient/ > [2] http://plash.beasts.org/wiki/NativeClient > [3] http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Jun 15 01:19:18 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 14 Jun 2009 16:19:18 -0700 Subject: [Python-Dev] Exception for setting attributes of built-in type In-Reply-To: <4A357CF0.2050604@mrabarnett.plus.com> References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> Message-ID: On Sun, Jun 14, 2009 at 3:42 PM, MRAB wrote: > Seo Sanghyeon wrote: >> >> Exception for setting attributes of built-in type differs between >> CPython and IronPython. This is not purely theoretical, as >> zope.interface tries to set Implements declaration as __implemented__ >> attribute of built-in type object, and excepts TypeError. >> >> Python 2.6.1 >>>>> >>>>> object.flag = True >> >> TypeError: can't set attributes of built-in/extension type 'object' >> >> IronPython 2.6 >>>>> >>>>> object.flag = True >> >> AttributeError: 'object' object has no attribute 'flag' >> >> I was surprised that CPython raises TypeError. Library Reference seems >> to mention it here: >> >> exception AttributeError >> Raised when an attribute reference or assignment fails. (When an >> object does not support attribute references or attribute assignments >> at all, TypeError is raised.) >> http://docs.python.org/library/exceptions.html >> >> What does it mean that "an object does not support attribute >> references or attribute assignments at all"? >> > Here's my guess: > > Some objects have a fixed (and possibly empty) set of attributes. > Attempting to add a new attribute or get a non-existent attribute raises > an AttributeError. > > Other objects, such as types (are there any that aren't types?), don't > have attributes at all. Attempting to add or get an attribute raises a > TypeError. This particular error comes (grep tells me :-) from type_setattro() in Objects/typeobject.c. It makes a specific check whether the type object whose attribute you want to set is a "built-in type" (this is done by checking the HEAPTYPE flag, which is never set for built-in types and always for user-defined types). For built-in types it disallows setting attributes. This is a pure policy issue: it prevents different 3rd party modules to make incompatible modifications of built-in types. In general, CPython isn't always consistent in raising AttributeError and TypeError when it comes to such policy issues: there are various places that raise TypeError in typeobject.c (and probably elsewhere) that simply forbid setting a specific attribute (another example is __name__). Given how poorly Python exceptions are specified in general, I don't want to hold IronPython to this standard (nor CPython :-). The reason this breaks zope.interfaces in IronPython is probably just that zope.interfaces was primarily written/tested against CPython. Going back to the phrase quoted from the reference manual, it's probably referring to the possibility that a type's tp_getattro slot is NULL; but even so it's wrong: PyObject_GetAttr() raises AttributeError in this case. Uselessly y'rs, -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Jun 15 01:48:39 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 15 Jun 2009 11:48:39 +1200 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090614051618.GA19302@cskk.homeip.net> References: <20090614051618.GA19302@cskk.homeip.net> Message-ID: <4A358C57.3060004@canterbury.ac.nz> Cameron Simpson wrote: > For myself, I'd expect more often to want to see if there's stuff in the > buffer _without_ doing any raw reads at all. What uses do you have in mind for that? -- Greg From cs at zip.com.au Mon Jun 15 02:34:07 2009 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 15 Jun 2009 10:34:07 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <1afaf6160906140721i27610ab5j245a1d8678a73c21@mail.gmail.com> Message-ID: <20090615003407.GA12475@cskk.homeip.net> On 14Jun2009 09:21, Benjamin Peterson wrote: | 2009/6/14 Cameron Simpson : | > On 14Jun2009 15:16, I wrote: | > | Is it possible to access the buffer? I see nothing in the docs. | > | > I've just found getvalue() in IOBase. Forget I said anything. | > It seems to be my day for that kind of post:-( | | Where are you seeing this? Only BytesIO and StringIO have a getvalue() method. I had thought I'd traced it by class inheritance. But I got BytesIO and IOBase confused. So: no getvalue then. So probably there is a case for peek0(), which never does a raw read. Thoughts? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ I was gratified to be able to answer promptly and I did. I said I didn't know. - Mark Twain From cs at zip.com.au Mon Jun 15 03:00:46 2009 From: cs at zip.com.au (Cameron Simpson) Date: Mon, 15 Jun 2009 11:00:46 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A358C57.3060004@canterbury.ac.nz> Message-ID: <20090615010046.GA30997@cskk.homeip.net> On 15Jun2009 11:48, Greg Ewing wrote: >> For myself, I'd expect more often to want to see if there's stuff in the >> buffer _without_ doing any raw reads at all. > > What uses do you have in mind for that? It seems like whenever I want to do some kind of opportunistic but non-blocking stuff with a remote service (eg something I have a packetising connection to, such as a TCP stream) I want to be able to see if there's "more stuff" to gather up before issuing a "flush" operation. And I want to be able to do that in a non-blocking way, much as a Queue has a non-blocking get() method. As an example, I've got the occasional protocol handler where it has to make a remote query. To avoid deadlock, the stream must be flushed after write()ing the query packet. However, flushing on every such packet is horribly wasteful if you know you have a bunch of them (for example, the caller is asking a bunch of questions). It is far far more efficient to write() each packet without flushes, keep the knowledge that a flush is needed, and flush when there's nothing more pending. That way the lower layer has maximum opportunity to pack data into packets. All that presumes another thread reading responses, which is how I generally write this stuff anyway, otherwise a full buffer will deadlock too. So your dispatch thread inner loop looks like this: # single consumer, so Q.empty() implies ok to Q.get() needFlush = False while not Q.empty(): P=Q.get() if P.needFlush: needFlush = True out.write(P.serialise()) if needFlush: out.flush() In this scheme, there _are_ packets that don't need a flush, because nobody is waiting on their response. Anyway, if I were reading from an IO object instead of a Queue I'd want to poll for "buffer empty". If there isn't an empty buffer I know there will be a packet worth of data coming immediately and I can pick it up with regular read()s, just as I'm doing with Q.get() above. But if the buffer is empty I can drop out of the "pick it all up now" loop and flush(). Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ If you take something apart and put it back together again enough times, you will eventually have enough parts left over to build a second one. - Ayse Sercan From python at rcn.com Mon Jun 15 04:48:18 2009 From: python at rcn.com (Raymond Hettinger) Date: Sun, 14 Jun 2009 19:48:18 -0700 Subject: [Python-Dev] Iterator version of contextlib.nested References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com><4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> Message-ID: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> FWIW, I think resurrecting contextlib.nested() is a bad idea. Part of the justification for the new with-statement syntax was that nested() doesn't have a way to finalize the constructors if one of them fails. It is a pitfall for the unwary. And now that we have the new with-statement syntax, it mostly just represents a second-way-to-do-it (a second way that has has the stated pitfall). The new statement was not designed to support passing in tuples of context-managers. This issue was raised while the new with-statement was being designed and it was intentionally left-out (in part, because the use cases were questionable and in-part because there were other ways to do it such as adding __enter__ and __exit__ to tuple). I suggest a PEP for 2.7 and 3.2 for building-out the with-statement to support tuples of context managers (perhaps modeled after the syntax for except-statements which allows either individual exceptions or tuples of exceptions). The reason I suggest a PEP is that use cases need to be fully thought out and the failing constructor problem needs to be dealt with. IMO, this represents doing-it-the-right-way instead of preserving a construct that is known to be problematic. Leaving it in will enshrine it. Better to just provide our new syntax that correctly handles the common case and then wait to carefully think through how to add support for passed-in nested cm's if in-fact those turn-out to have reasonable use cases. Raymond From benjamin at python.org Mon Jun 15 04:58:30 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 14 Jun 2009 21:58:30 -0500 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> Message-ID: <1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com> 2009/6/14 Raymond Hettinger : > FWIW, I think resurrecting contextlib.nested() is a bad idea. > Part of the justification for the new with-statement syntax was > that nested() doesn't have a way to finalize the constructors > if one of them fails. ? It is a pitfall for the unwary. ?And now > that we have the new with-statement syntax, it mostly just > represents a second-way-to-do-it (a second way that has > has the stated pitfall). I don't consider changing a DeprecationWarning to a PendingDeprecationWarning "resurrecting" its target. Fully deprecating a feature in the same version that we add its replacement will just make more difficulties for cross-version libraries. .... > I suggest a PEP for 2.7 and 3.2 for building-out the > with-statement to support tuples of context managers > (perhaps modeled after the syntax for except-statements > which allows either individual exceptions or tuples of > exceptions). I think the question of extending the syntax later is orthogonal to the issue of the DeprecationWarning. -- Regards, Benjamin From hfuerstenau at gmx.net Mon Jun 15 10:11:51 2009 From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=) Date: Mon, 15 Jun 2009 09:11:51 +0100 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com><4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> Message-ID: <4A360247.5060407@gmx.net> > Part of the justification for the new with-statement syntax was > that nested() doesn't have a way to finalize the constructors > if one of them fails. I think the problem was a little bit more subtle: nested() gets passed managers, so their __init__()s should all have run when the first context is entered. The only problem comes up when the __exit__() of an outer manager tries to suppress an exception raised by the __enter__() of an inner one. This is a limited defect in that it doesn't affect the common situation where no __exit__() tries to suppress any exceptions. (In a quick glance over the std library I couldn't find a single instance of an exception-suppressing __exit__().). > And now > that we have the new with-statement syntax, it mostly just > represents a second-way-to-do-it (a second way that has > has the stated pitfall). So the functionalities of nested() and multi-with overlap in the common use cases, and each has its own limitation in an uncommon one. I agree that this situation is unfortunate, but I think introducing support for one uncommon case and removing it for another is not the way to go in 3.1. That's why I think nested() should stay un-deprecated until there is a replacement which handles a superset of its use cases. > The new statement was not designed to support passing in > tuples of context-managers. This issue was raised while > the new with-statement was being designed and it was > intentionally left-out (in part, because the use cases were > questionable FWIW, my use case (which made me notice the DeprecationWarning in the first place) is in a command dispatch function, which looks at the command to be executed and pre-processes its arguments in a uniform way. Part of that pre-processing is entering contexts of context manager before passing them along (and exiting them when the command finishes or raises an exception). > and in-part because there were other ways > to do it such as adding __enter__ and __exit__ to tuple). Which has not been done for 3.1. Granted, you could subclass tuple and add them yourself, but then you would mostly be copying what's already implemented in nested(). > I suggest a PEP for 2.7 and 3.2 for building-out the > with-statement to support tuples of context managers That sounds like a good idea. > IMO, this represents doing-it-the-right-way instead of preserving a > construct that is known to be problematic. > Leaving it in will enshrine it. I don't see the problem with deprecating it only after a completely suitable replacement is found. Why would it be any harder to deprecate nested() in 3.2? - Hagen From ncoghlan at gmail.com Mon Jun 15 12:47:45 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 15 Jun 2009 20:47:45 +1000 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <4A34B254.6050907@gmx.net> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> Message-ID: <4A3626D1.5000402@gmail.com> Hagen F?rstenau wrote: >> I actually almost asked for that to be changed to a >> PendingDeprecationWarning when it was first added - Benjamin, do you >> mind if I downgrade this warning to a pending one post rc2? > > I'm not sure what that would buy us. For the use case I mentioned it > would be just as annoying to get a PendingDeprecationWarning. But if the > warning was completely removed now, nested could still get deprecated in > 3.2 as soon as some better mechanism for a variable number of managers > has been provided. Unlike a full DeprecationWarning, a PendingDeprecationWarning is ignored by default. You have to switch them on explicitly via code or a command line switch in order to see them. Pending warnings give the hint that we intend to get rid of something, but either don't have clear replacements for some legitimate use cases (as in the case of contextlib.nested) or have some other reason for not generating a warning by default (e.g. we may not have cleared all uses out of the standard library yet). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From hfuerstenau at gmx.net Mon Jun 15 12:56:47 2009 From: hfuerstenau at gmx.net (=?ISO-8859-1?Q?Hagen_F=FCrstenau?=) Date: Mon, 15 Jun 2009 11:56:47 +0100 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <4A3626D1.5000402@gmail.com> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <4A3626D1.5000402@gmail.com> Message-ID: <4A3628EF.4010405@gmx.net> > Unlike a full DeprecationWarning, a PendingDeprecationWarning is ignored > by default. You have to switch them on explicitly via code or a command > line switch in order to see them. Sorry, I should have made myself more familiar with the warnings mechanism before writing. In that case I'm fine with a PendingDeprecationWarning. :-) - Hagen From ncoghlan at gmail.com Mon Jun 15 13:13:03 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 15 Jun 2009 21:13:03 +1000 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com><4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> Message-ID: <4A362CBF.7070609@gmail.com> Raymond Hettinger wrote: > I suggest a PEP for 2.7 and 3.2 for building-out the > with-statement to support tuples of context managers > (perhaps modeled after the syntax for except-statements > which allows either individual exceptions or tuples of > exceptions). The reason I suggest a PEP is that use > cases need to be fully thought out and the failing > constructor problem needs to be dealt with. > IMO, this represents doing-it-the-right-way instead of preserving a > construct that is known to be problematic. > Leaving it in will enshrine it. Better to just provide our new > syntax that correctly handles the common case and then > wait to carefully think through how to add support for passed-in > nested cm's if in-fact those turn-out to have reasonable > use cases. I agree that the current incarnation of contextlib.nested needs to go - it isn't really salvagable in its current form. However, I don't think we should generate a warning for it by default until we provide a new mechanism for handling a variable number of context managers - PendingDeprecationWarning seems a much better fit. A 2.7/3.2 PEP can then address the two main issues with the current approach: 1. The __init__ calls for the inner context managers occur outside the scope of the outer context managers. Some form of lazy evaluation would be needed to deal with that. 2. If an inner __enter__ call raises an exception that is suppressed by an outer __exit__ call then the body of with statement should be skipped rather than raising RuntimeError. This means either new syntax with semantics along the lines of the previously rejected PEP 377 or else a custom exception and a second context manager that suppresses it. Personally, I don't think new syntax for the PEP 377 semantics is warranted for the same reason that PEP 377 itself was rejected - it complicates the statement definition significantly for a really obscure corner case. Better to come up with a new and improved version of nested that eliminates (or better handles) the quirks and leave the statement definition alone. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Jun 15 13:18:54 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 15 Jun 2009 21:18:54 +1000 Subject: [Python-Dev] CPython in the web browser under Native Client In-Reply-To: <20090614.170936.846958184.mrs@localhost.localdomain> References: <20090614.170936.846958184.mrs@localhost.localdomain> Message-ID: <4A362E1E.3060007@gmail.com> Mark Seaborn wrote: > [3] http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html Very cool! Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Jun 15 13:27:24 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 15 Jun 2009 21:27:24 +1000 Subject: [Python-Dev] Exception for setting attributes of built-in type In-Reply-To: References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> Message-ID: <4A36301C.2050100@gmail.com> Guido van Rossum wrote: > In general, CPython isn't always consistent in raising AttributeError > and TypeError when it comes to such policy issues: there are various > places that raise TypeError in typeobject.c (and probably elsewhere) > that simply forbid setting a specific attribute (another example is > __name__). We're pretty inconsistent when it comes to looking up special methods as well - those that are looked up through dedicated slots in abstract.c usually raise TypeError, while those that are looked up via a PyType method usually raise AttributeError. I'm not sure that there is any practical way to handle that other than for applications that care about cross-implementation compatibility in this area to catch a (TypeError, AttributeError) tuple rather than one or the other. CHeers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From kalman.gergely at duodecad.hu Mon Jun 15 13:47:55 2009 From: kalman.gergely at duodecad.hu (=?UTF-8?B?S8OhbG3DoW4gR2VyZ2VseQ==?=) Date: Mon, 15 Jun 2009 13:47:55 +0200 Subject: [Python-Dev] python sendmsg()/recvmsg() implementation In-Reply-To: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com> References: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com> Message-ID: <4A3634EB.2020800@duodecad.hu> Jean-Paul Calderone wrote: > On Tue, 09 Jun 2009 16:46:54 +0200, K?lm?n Gergely > wrote: >> Hello, my name is Greg. >> >> I've just started using python after many years of C programming, and >> I'm also new to the list. I wanted to clarify this >> first, so that maybe I will get a little less beating for my >> stupidity :) >> > > Welcome! > >> >> [snip] >> >> Browsing the net I've found a patch to the python core >> (http://bugs.python.org/issue1194378), dated 2005. >> >> First of all, I would like to ask you guys, whether you know of any >> way of doing this FD passing magic, or that you know >> of any 3rd party module / patch / anything that can do this for me. > > Aside from the patch in the tracker, there are several implementations of > these APIs as third-party extension modules. > >> >> Since I'm fairly familiar with C and (not that much, but I feel the >> power) python, I would take the challenge of writing >> it, given that the above code is still somewhat usable. If all else >> fails I would like to have your help to guide me through >> this process. >> > > What would be great is if you could take the patch in the tracker and get > it into shape so that it is suitable for inclusion. This would involve > three things, I think: > > 1. Write unit tests for the functionality (since the patch itself > provides > none) > 2. Update the patch so that it again applies cleanly against trunk > 3. Add documentation for the new APIs > > Once this is done, you can get a committer to look at it and either > provide > more specific feedback or apply it. > > Thanks, > > Jean-Paul > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/synapse%40jasmin.hu Hello again So, after a little cleanup I've managed to integrate the code into socketmodule.c/h. It works fine now, though I needed to add it to Lib/socket.py, otherwise it wouldn't show up in the socket module (I've searched for recvfrom and added it). I've also cleaned up the code a little, fixed some codingstyle issues (which might still exist). Since I am not a python core developer the patch might still be in a pretty outdated state. I'd like someone to look it over and direct me to some documentation (the ones I've found so far were pretty sketchy), for it to be acceptable for inclusion. The sanity of the code is what matters to me the most. I've looked it over though and found it in a sound state, but I guess you guys might have a different opinion about that ;) With writing the test cases, some documentation would be nice. I've attached the patch generated with svn diff for revision 73434, and the test code that I use to pass a file descriptor between processes. It works just fine :). Thanks Kalman Gergely -------------- next part -------------- A non-text attachment was scrubbed... Name: sendmsg.diff Type: text/x-patch Size: 19805 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sendfd.py Type: text/x-python Size: 1353 bytes Desc: not available URL: From ncoghlan at gmail.com Mon Jun 15 16:32:51 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 16 Jun 2009 00:32:51 +1000 Subject: [Python-Dev] python sendmsg()/recvmsg() implementation In-Reply-To: <4A3634EB.2020800@duodecad.hu> References: <20090609151026.22176.614294214.divmod.quotient.3795@henry.divmod.com> <4A3634EB.2020800@duodecad.hu> Message-ID: <4A365B93.7020501@gmail.com> K?lm?n Gergely wrote: > Since I am not a python core developer the patch might still be in a > pretty outdated state. I'd like someone to look it over > and direct me to some documentation (the ones I've found so far were > pretty sketchy), for it to be acceptable for inclusion. > The sanity of the code is what matters to me the most. I've looked it > over though and found it in a sound state, but I guess > you guys might have a different opinion about that ;) > > With writing the test cases, some documentation would be nice. Most unit tests these days are written based on either doctest or unittest. When adding new features to an existing module, it is usually best to follow the testing style already used for the rest of that module (in this case, that should be test/test_socket.py). The relevant question in the dev FAQ gives good pointers: http://www.python.org/dev/faq/#how-to-test-a-patch > I've attached the patch generated with svn diff for revision 73434, and > the test code that I use to pass a file descriptor > between processes. It works just fine :). Uploading files to the tracker is generally the best option - patches tend to get lost if they're just sent to the mailing list. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Mon Jun 15 16:48:33 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 15 Jun 2009 07:48:33 -0700 Subject: [Python-Dev] Exception for setting attributes of built-in type In-Reply-To: References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> Message-ID: On Sun, Jun 14, 2009 at 4:19 PM, Guido van Rossum wrote: > In general, CPython isn't always consistent in raising AttributeError > and TypeError when it comes to such policy issues: there are various > places that raise TypeError in typeobject.c (and probably elsewhere) > that simply forbid setting a specific attribute (another example is > __name__). I should add that this policy is also forced somewhat by the existence of the "multiple interpreters in one address space" feature, which is used e.g. by mod_python. This feature attempts to provide isolation between interpreters to the point that each one can have a completely different set of modules loaded and can be working on a totally different application. The implementation of CPython shares built-in types between multiple interpreters (and it wouldn't be easy to change this); if you were able to modify a built-in type from one interpreter, all other interpreters would see that same modification. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dinov at microsoft.com Mon Jun 15 17:54:51 2009 From: dinov at microsoft.com (Dino Viehland) Date: Mon, 15 Jun 2009 15:54:51 +0000 Subject: [Python-Dev] [IronPython] Exception for setting attributes of built-in type In-Reply-To: References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com> Guido wrote: > I should add that this policy is also forced somewhat by the existence > of the "multiple interpreters in one address space" feature, which is > used e.g. by mod_python. This feature attempts to provide isolation > between interpreters to the point that each one can have a completely > different set of modules loaded and can be working on a totally > different application. The implementation of CPython shares built-in > types between multiple interpreters (and it wouldn't be easy to change > this); if you were able to modify a built-in type from one > interpreter, all other interpreters would see that same modification. IronPython is in the exact same boat here - we share built-in types Across multiple Python engines as well. From fuzzyman at voidspace.org.uk Mon Jun 15 18:10:08 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 15 Jun 2009 17:10:08 +0100 Subject: [Python-Dev] [IronPython] Exception for setting attributes of built-in type In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com> References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> <1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com> Message-ID: <4A367260.2010400@voidspace.org.uk> Dino Viehland wrote: > Guido wrote: > >> I should add that this policy is also forced somewhat by the existence >> of the "multiple interpreters in one address space" feature, which is >> used e.g. by mod_python. This feature attempts to provide isolation >> between interpreters to the point that each one can have a completely >> different set of modules loaded and can be working on a totally >> different application. The implementation of CPython shares built-in >> types between multiple interpreters (and it wouldn't be easy to change >> this); if you were able to modify a built-in type from one >> interpreter, all other interpreters would see that same modification. >> > > IronPython is in the exact same boat here - we share built-in types > Across multiple Python engines as well. > > > And indeed it is needed - if you are working with multiple interpreters (engines in IronPython) you don't want isinstance(something, dict) to fail because it is a dictionary from a different interpreter... Michael > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From guido at python.org Mon Jun 15 19:10:59 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 15 Jun 2009 10:10:59 -0700 Subject: [Python-Dev] [IronPython] Exception for setting attributes of built-in type In-Reply-To: <4A367260.2010400@voidspace.org.uk> References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> <1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com> <4A367260.2010400@voidspace.org.uk> Message-ID: On Mon, Jun 15, 2009 at 9:10 AM, Michael Foord wrote: > Dino Viehland wrote: >> Guido wrote: >>> I should add that this policy is also forced somewhat by the existence >>> of the "multiple interpreters in one address space" feature, which is >>> used e.g. by mod_python. This feature attempts to provide isolation >>> between interpreters to the point that each one can have a completely >>> different set of modules loaded and can be working on a totally >>> different application. The implementation of CPython shares built-in >>> types between multiple interpreters (and it wouldn't be easy to change >>> this); if you were able to modify a built-in type from one >>> interpreter, all other interpreters would see that same modification. >> IronPython is in the exact same boat here - we share built-in types >> Across multiple Python engines as well. > And indeed it is needed - if you are working with multiple interpreters > (engines in IronPython) you don't want isinstance(something, dict) to fail > because it is a dictionary from a different interpreter... Ah, but that suggests you have sharing between different interpreters. If you're doing that, perhaps you shouldn't be using multiple interpreters, but instead multiple threads? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Mon Jun 15 19:20:41 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 15 Jun 2009 18:20:41 +0100 Subject: [Python-Dev] [IronPython] Exception for setting attributes of built-in type In-Reply-To: References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> <1A472770E042064698CB5ADC83A12ACD0324503A@TK5EX14MBXC116.redmond.corp.microsoft.com> <4A367260.2010400@voidspace.org.uk> Message-ID: <4A3682E9.3000405@voidspace.org.uk> Guido van Rossum wrote: > On Mon, Jun 15, 2009 at 9:10 AM, Michael Foord wrote: > >> Dino Viehland wrote: >> >>> Guido wrote: >>> >>>> I should add that this policy is also forced somewhat by the existence >>>> of the "multiple interpreters in one address space" feature, which is >>>> used e.g. by mod_python. This feature attempts to provide isolation >>>> between interpreters to the point that each one can have a completely >>>> different set of modules loaded and can be working on a totally >>>> different application. The implementation of CPython shares built-in >>>> types between multiple interpreters (and it wouldn't be easy to change >>>> this); if you were able to modify a built-in type from one >>>> interpreter, all other interpreters would see that same modification. >>>> > > >>> IronPython is in the exact same boat here - we share built-in types >>> Across multiple Python engines as well. >>> > > >> And indeed it is needed - if you are working with multiple interpreters >> (engines in IronPython) you don't want isinstance(something, dict) to fail >> because it is a dictionary from a different interpreter... >> > > Ah, but that suggests you have sharing between different interpreters. > If you're doing that, perhaps you shouldn't be using multiple > interpreters, but instead multiple threads? > > Well, in our use case we use multiple engines to provide an isolated execution context for every document (the Resolver One spreadsheet written in IronPython). Each of these has their own calculation thread as well - but the engine per document structure is nice and clean and means each document can have its own set of modules loaded without affecting the other documents (although they share a core set of modules). Once we move these engines into their own app domains we can completely isolate each document and apply separate security permissions to each one. That might mean each document effectively paying the not-insubstantial startup time hit and we haven't begun to look at how to mitigate that. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From benjamin at python.org Mon Jun 15 21:09:00 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 15 Jun 2009 14:09:00 -0500 Subject: [Python-Dev] Exception for setting attributes of built-in type In-Reply-To: <4A36301C.2050100@gmail.com> References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> <4A36301C.2050100@gmail.com> Message-ID: <1afaf6160906151209h10138643ob957e1235176ba45@mail.gmail.com> 2009/6/15 Nick Coghlan : > Guido van Rossum wrote: >> In general, CPython isn't always consistent in raising AttributeError >> and TypeError when it comes to such policy issues: there are various >> places that raise TypeError in typeobject.c (and probably elsewhere) >> that simply forbid setting a specific attribute (another example is >> __name__). > > We're pretty inconsistent when it comes to looking up special methods as > well - those that are looked up through dedicated slots in abstract.c > usually raise TypeError, while those that are looked up via a PyType > method usually raise AttributeError. What's a PyType method? -- Regards, Benjamin From python at rcn.com Mon Jun 15 21:17:55 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 15 Jun 2009 12:17:55 -0700 Subject: [Python-Dev] Iterator version of contextlib.nested References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> <1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com> Message-ID: > I don't consider changing a DeprecationWarning to a > PendingDeprecationWarning "resurrecting" its target. Seems like resurrection to me. Pending warnings are hidden by default, so someone has to go look for it (and no one does this). The problem with the nested() construct is not so much that it duplicates the new with-statement; the problem is that it is a bug factory when used as advertised. The sole justification for keeping it around is that it handles an obscure use case (one that isn't even shown in its documentation or examples). I'm not opposing the idea to change the DeprecationWarning to a PendingDeprecationWarning, but I don't think we're doing the users any favors by hiding the warning message. Raymond P.S. If you switch to PendingDeprecationWarning, the example in the docs should probably be switched to show the one valid use case (passing in a prepackaged nest of context managers). Right now, the current example just shows the hazardous pattern that is much better served by the new with-statement syntax. From benjamin at python.org Mon Jun 15 21:29:22 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 15 Jun 2009 14:29:22 -0500 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> <1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com> Message-ID: <1afaf6160906151229g226c67acpa957ed4087a2dfde@mail.gmail.com> 2009/6/15 Raymond Hettinger : > P.S. ?If you switch to PendingDeprecationWarning, the example > in the docs should probably be switched to show the one valid > use case (passing in a prepackaged nest of context managers). > Right now, the current example just shows the hazardous pattern > that is much better served by the new with-statement syntax. +1 I think this should be done anyway. -- Regards, Benjamin From tjreedy at udel.edu Mon Jun 15 22:36:13 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 15 Jun 2009 16:36:13 -0400 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> <1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com> Message-ID: Raymond Hettinger wrote: > P.S. If you switch to PendingDeprecationWarning, the example > in the docs should probably be switched to show the one valid > use case (passing in a prepackaged nest of context managers). It could even suggest that it only be used for this, since it may disappear, and that other uses should use the new syntax. That would give people the best chance of writing future-proof code when they can. From p.f.moore at gmail.com Mon Jun 15 22:41:11 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 15 Jun 2009 21:41:11 +0100 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> <1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com> Message-ID: <79990c6b0906151341k2224a02eu202ad7bac6f90774@mail.gmail.com> 2009/6/15 Terry Reedy : > Raymond Hettinger wrote: > >> P.S. ?If you switch to PendingDeprecationWarning, the example >> in the docs should probably be switched to show the one valid >> use case (passing in a prepackaged nest of context managers). > > It could even suggest that it only be used for this, since it may disappear, > and that other uses should use the new syntax. ?That would give people the > best chance of writing future-proof code when they can. And if the warning is *not* changed to a PendingDeprecationWarning, the documentation could also note the necessary warnings call needed to let the example code run warning-free. Paul. From p.f.moore at gmail.com Mon Jun 15 22:46:40 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 15 Jun 2009 21:46:40 +0100 Subject: [Python-Dev] Is the py3k mercurial mirror broken? Message-ID: <79990c6b0906151346w7e104b2el59280d4dc529c391@mail.gmail.com> I get the following: >hg version Mercurial Distributed SCM (version 1.2) Copyright (C) 2005-2008 Matt Mackall and others This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. >hg clone http://code.python.org/hg/branches/py3k python-3k requesting all changes adding changesets transaction abort! rollback completed ** unknown exception encountered, details follow ** report bug details to http://www.selenic.com/mercurial/bts ** or mercurial at selenic.com ** Mercurial Distributed SCM (version 1.2) ** Extensions loaded: Traceback (most recent call last): File "hg", line 27, in File "mercurial\dispatch.pyc", line 16, in run File "mercurial\dispatch.pyc", line 25, in dispatch File "mercurial\dispatch.pyc", line 41, in _runcatch File "mercurial\dispatch.pyc", line 372, in _dispatch File "mercurial\dispatch.pyc", line 247, in runcommand File "mercurial\dispatch.pyc", line 417, in _runcommand File "mercurial\dispatch.pyc", line 377, in checkargs File "mercurial\dispatch.pyc", line 371, in File "mercurial\util.pyc", line 718, in check File "mercurial\commands.pyc", line 603, in clone File "mercurial\hg.pyc", line 215, in clone File "mercurial\localrepo.pyc", line 2149, in clone File "mercurial\localrepo.pyc", line 1492, in pull File "mercurial\localrepo.pyc", line 2016, in addchangegroup File "mercurial\revlog.pyc", line 1192, in addgroup File "mercurial\changegroup.pyc", line 31, in chunkiter File "mercurial\changegroup.pyc", line 15, in getchunk File "mercurial\util.pyc", line 1674, in read File "mercurial\httprepo.pyc", line 18, in zgenerator zlib.error: Error -3 while decompressing: incorrect header check Paul. From benjamin at python.org Mon Jun 15 22:49:11 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 15 Jun 2009 15:49:11 -0500 Subject: [Python-Dev] Is the py3k mercurial mirror broken? In-Reply-To: <79990c6b0906151346w7e104b2el59280d4dc529c391@mail.gmail.com> References: <79990c6b0906151346w7e104b2el59280d4dc529c391@mail.gmail.com> Message-ID: <1afaf6160906151349t5328bd67lcf25bee88a4eeb99@mail.gmail.com> There seem to be some issues with Subversion, so that's probably affecting hg. 2009/6/15 Paul Moore : > I get the following: > >>hg version > Mercurial Distributed SCM (version 1.2) > > Copyright (C) 2005-2008 Matt Mackall and others > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > >>hg clone http://code.python.org/hg/branches/py3k python-3k > requesting all changes > adding changesets > transaction abort! > rollback completed > ** unknown exception encountered, details follow > ** report bug details to http://www.selenic.com/mercurial/bts > ** or mercurial at selenic.com > ** Mercurial Distributed SCM (version 1.2) > ** Extensions loaded: > Traceback (most recent call last): > ?File "hg", line 27, in > ?File "mercurial\dispatch.pyc", line 16, in run > ?File "mercurial\dispatch.pyc", line 25, in dispatch > ?File "mercurial\dispatch.pyc", line 41, in _runcatch > ?File "mercurial\dispatch.pyc", line 372, in _dispatch > ?File "mercurial\dispatch.pyc", line 247, in runcommand > ?File "mercurial\dispatch.pyc", line 417, in _runcommand > ?File "mercurial\dispatch.pyc", line 377, in checkargs > ?File "mercurial\dispatch.pyc", line 371, in > ?File "mercurial\util.pyc", line 718, in check > ?File "mercurial\commands.pyc", line 603, in clone > ?File "mercurial\hg.pyc", line 215, in clone > ?File "mercurial\localrepo.pyc", line 2149, in clone > ?File "mercurial\localrepo.pyc", line 1492, in pull > ?File "mercurial\localrepo.pyc", line 2016, in addchangegroup > ?File "mercurial\revlog.pyc", line 1192, in addgroup > ?File "mercurial\changegroup.pyc", line 31, in chunkiter > ?File "mercurial\changegroup.pyc", line 15, in getchunk > ?File "mercurial\util.pyc", line 1674, in read > ?File "mercurial\httprepo.pyc", line 18, in zgenerator > zlib.error: Error -3 while decompressing: incorrect header check > > Paul. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/benjamin%40python.org > -- Regards, Benjamin From ncoghlan at gmail.com Mon Jun 15 23:31:06 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 16 Jun 2009 07:31:06 +1000 Subject: [Python-Dev] Exception for setting attributes of built-in type In-Reply-To: <1afaf6160906151209h10138643ob957e1235176ba45@mail.gmail.com> References: <5b0248170906141209y4398defanacfd0bb62504e550@mail.gmail.com> <4A357CF0.2050604@mrabarnett.plus.com> <4A36301C.2050100@gmail.com> <1afaf6160906151209h10138643ob957e1235176ba45@mail.gmail.com> Message-ID: <4A36BD9A.3010301@gmail.com> Benjamin Peterson wrote: > 2009/6/15 Nick Coghlan : >> Guido van Rossum wrote: >>> In general, CPython isn't always consistent in raising AttributeError >>> and TypeError when it comes to such policy issues: there are various >>> places that raise TypeError in typeobject.c (and probably elsewhere) >>> that simply forbid setting a specific attribute (another example is >>> __name__). >> We're pretty inconsistent when it comes to looking up special methods as >> well - those that are looked up through dedicated slots in abstract.c >> usually raise TypeError, while those that are looked up via a PyType >> method usually raise AttributeError. > > What's a PyType method? I was misremembering - for some reason I thought: a) _PyObject_LookupSpecial was a PyType method b) That it raised AttributeError itself instead of letting the caller decide what error to raise It's still the case that (e.g.) special_lookup() in ceval.c raises an AttributeError, as do special method lookups from Python of the form "type(obj).__method__(obj)". Whether CPython raises TypeError or AttributeError when it comes to special methods is really pretty arbitrary rather than a matter of any grand design. Cheers, Nick. P.S. If anyone feels like digging into the archives, the last time I can recall this topic coming up is when I was concerned about the original with statement implementation raising AttributeError rather than TypeError when it couldn't find an __enter__ or __exit__ method. Guido chimed in then (as now) to say that either exception was fine. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Jun 15 23:44:32 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 16 Jun 2009 07:44:32 +1000 Subject: [Python-Dev] Iterator version of contextlib.nested In-Reply-To: <79990c6b0906151341k2224a02eu202ad7bac6f90774@mail.gmail.com> References: <4A3271BA.2070904@gmx.net> <4A32FA7D.1060906@gmail.com> <4A339F84.2070400@gmx.net> <4A3450DD.6040905@gmail.com> <4A34B254.6050907@gmx.net> <022349E0DE2F46F3B68B0050A7608EF4@RaymondLaptop1> <1afaf6160906141958s3bb76c11o7c60c3a2ceb98e9a@mail.gmail.com> <79990c6b0906151341k2224a02eu202ad7bac6f90774@mail.gmail.com> Message-ID: <4A36C0C0.4090704@gmail.com> Paul Moore wrote: > 2009/6/15 Terry Reedy : >> Raymond Hettinger wrote: >> >>> P.S. If you switch to PendingDeprecationWarning, the example >>> in the docs should probably be switched to show the one valid >>> use case (passing in a prepackaged nest of context managers). >> It could even suggest that it only be used for this, since it may disappear, >> and that other uses should use the new syntax. That would give people the >> best chance of writing future-proof code when they can. > > And if the warning is *not* changed to a PendingDeprecationWarning, > the documentation could also note the necessary warnings call needed > to let the example code run warning-free. I think I like that option even better than downgrading the warning. I created http://bugs.python.org/issue6288 to make it clear to Benjamin when I have finished updating the docs. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From greg.ewing at canterbury.ac.nz Tue Jun 16 01:21:03 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 16 Jun 2009 11:21:03 +1200 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090615010046.GA30997@cskk.homeip.net> References: <20090615010046.GA30997@cskk.homeip.net> Message-ID: <4A36D75F.5070805@canterbury.ac.nz> Cameron Simpson wrote: > It seems like whenever I want to do some kind of opportunistic but > non-blocking stuff with a remote service Do you actually do this with buffered streams? I find it's better to steer well clear of buffered I/O objects when doing non-blocking stuff, because they don't play well with other things like select(). Anyhow, I wouldn't be opposed to having a way of looking into the buffer, but it should be a separate API -- preferably with a better name than peek0(), which gives no clue at all about what it does differently from peek(). -- Greg From cs at zip.com.au Tue Jun 16 02:43:27 2009 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 16 Jun 2009 10:43:27 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A36D75F.5070805@canterbury.ac.nz> Message-ID: <20090616004327.GA15853@cskk.homeip.net> On 16Jun2009 11:21, Greg Ewing wrote: > Cameron Simpson wrote: >> It seems like whenever I want to do some kind of opportunistic but >> non-blocking stuff with a remote service > > Do you actually do this with buffered streams? Sure, in C, python and perl quite happily. In some circumstances. Provided you're careful about when to fflush() it can all go quite smoothly. It's certainly not applicable to everything. > I find > it's better to steer well clear of buffered I/O objects > when doing non-blocking stuff, because they don't play > well with other things like select(). Ah, the non-blockingness. Well that's the rub. I normally avoid non-blocking requirements by using threads, so that the thread gathering from the stream can block. Then the consumer can poll a Queue from the worker thread, etc. I really don't like select(/poll/epoll etc) much; aside from select's scaling issues with lots of files (hence poll/epoll) there are high performance things where having an event loop using select is a good way to go, but I generally prefer using threads and/or generators to make the code clear (single flow of control, single task for the block of code, etc) if there's no reason not to. > Anyhow, I wouldn't be opposed to having a way of looking > into the buffer, but it should be a separate API -- > preferably with a better name than peek0(), which gives > no clue at all about what it does differently from peek(). Indeed, though arguably read1() is a lousy name too, on the same basis. My itch is that peek() _feels_ like it should be "look into the buffer" but actually can block and/or change the buffer. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ You can't wait for inspiration. You have to go after it with a club. - Jack London From python at mrabarnett.plus.com Tue Jun 16 03:18:22 2009 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 16 Jun 2009 02:18:22 +0100 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090616004327.GA15853@cskk.homeip.net> References: <20090616004327.GA15853@cskk.homeip.net> Message-ID: <4A36F2DE.3010007@mrabarnett.plus.com> Cameron Simpson wrote: > On 16Jun2009 11:21, Greg Ewing wrote: >> Cameron Simpson wrote: >>> It seems like whenever I want to do some kind of opportunistic but >>> non-blocking stuff with a remote service >> Do you actually do this with buffered streams? > > Sure, in C, python and perl quite happily. In some circumstances. > Provided you're careful about when to fflush() it can all go quite > smoothly. It's certainly not applicable to everything. > >> I find >> it's better to steer well clear of buffered I/O objects >> when doing non-blocking stuff, because they don't play >> well with other things like select(). > > Ah, the non-blockingness. Well that's the rub. I normally avoid > non-blocking requirements by using threads, so that the thread gathering > from the stream can block. Then the consumer can poll a Queue from the > worker thread, etc. > > I really don't like select(/poll/epoll etc) much; aside from select's > scaling issues with lots of files (hence poll/epoll) there are high > performance things where having an event loop using select is a good > way to go, but I generally prefer using threads and/or generators > to make the code clear (single flow of control, single task for the > block of code, etc) if there's no reason not to. > >> Anyhow, I wouldn't be opposed to having a way of looking >> into the buffer, but it should be a separate API -- >> preferably with a better name than peek0(), which gives >> no clue at all about what it does differently from peek(). > > Indeed, though arguably read1() is a lousy name too, on the same basis. > My itch is that peek() _feels_ like it should be "look into the buffer" > but actually can block and/or change the buffer. > Can block, but not if you don't want it too. You might just want to see what, if anything, is currently available, up to n bytes. From cs at zip.com.au Tue Jun 16 04:03:11 2009 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 16 Jun 2009 12:03:11 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A36F2DE.3010007@mrabarnett.plus.com> Message-ID: <20090616020311.GA31250@cskk.homeip.net> On 16Jun2009 02:18, MRAB wrote: >> My itch is that peek() _feels_ like it should be "look into the buffer" >> but actually can block and/or change the buffer. >> > Can block, but not if you don't want it too. You might just want to see > what, if anything, is currently available, up to n bytes. Am I missing something? In the face of an _empty_ buffer (which I can't tell from outside) how do I prevent peek() blocking? More generally, if I go peek(n) and if n > bytes_in_buffer_right_now and the raw stream would block if a raw read is done? My concerns would go away if I could probe the buffer content size; then I could ensure peek(n) chose n <= the content size. If that's not enough, my problem - I can choose to read-and-block or go away and come back later. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ If all around you is darkness and you feel you're contending in vain, then the light at the end of the tunnel is the front of an oncoming train. From cs at zip.com.au Tue Jun 16 05:20:53 2009 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 16 Jun 2009 13:20:53 +1000 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: <20090614.164255.343165597.mrs@localhost.localdomain> Message-ID: <20090616032053.GA15938@cskk.homeip.net> On 14Jun2009 16:42, Mark Seaborn wrote: | I use a convenience function like this, so that GC takes care of the FDs: | | def make_pipe(): | read_fd, write_fd = os.pipe() | return os.fdopen(read_fd, "r"), os.fdopen(write_fd, "w") Not guarrenteed to be timely. The try/except at least closes things as control passes out of the relevant scope. I don't think all pythons do immediate ref-counted GC. But it's very neat! -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Trust the computer industry to shorten Year 2000 to Y2K. It was this thinking that caused the problem in the first place. - Mark Ovens From python at mrabarnett.plus.com Tue Jun 16 13:14:45 2009 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 16 Jun 2009 12:14:45 +0100 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090616020311.GA31250@cskk.homeip.net> References: <20090616020311.GA31250@cskk.homeip.net> Message-ID: <4A377EA5.4090804@mrabarnett.plus.com> Cameron Simpson wrote: > On 16Jun2009 02:18, MRAB wrote: >>> My itch is that peek() _feels_ like it should be "look into the buffer" >>> but actually can block and/or change the buffer. >>> >> Can block, but not if you don't want it too. You might just want to see >> what, if anything, is currently available, up to n bytes. > > Am I missing something? > > In the face of an _empty_ buffer (which I can't tell from outside) how > do I prevent peek() blocking? More generally, if I go peek(n) and if n > > bytes_in_buffer_right_now and the raw stream would block if a raw read > is done? > > My concerns would go away if I could probe the buffer content size; > then I could ensure peek(n) chose n <= the content size. If that's not > enough, my problem - I can choose to read-and-block or go away and come > back later. I was thinking along the lines of: def peek(self, size=None, block=True) If 'block' is True then return 'size' bytes, unless the end of the file/stream is reached; if 'block' is False then return up to 'size' bytes, without blocking. The blocking form might impose a limit to how much can be peeked (the maximum size of the buffer), or it might enlarge the buffer as necessary. From steve at pearwood.info Tue Jun 16 13:19:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 16 Jun 2009 21:19:38 +1000 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: <20090616032053.GA15938@cskk.homeip.net> References: <20090616032053.GA15938@cskk.homeip.net> Message-ID: <200906162119.39786.steve@pearwood.info> On Tue, 16 Jun 2009 01:20:53 pm Cameron Simpson wrote: > I don't think all > pythons do immediate ref-counted GC. Jython and IronPython don't. I don't know about PyPy, CLPython, Unladen Swallow, etc. -- Steven D'Aprano From fuzzyman at voidspace.org.uk Tue Jun 16 13:21:29 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 16 Jun 2009 12:21:29 +0100 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: <200906162119.39786.steve@pearwood.info> References: <20090616032053.GA15938@cskk.homeip.net> <200906162119.39786.steve@pearwood.info> Message-ID: <4A378039.5050904@voidspace.org.uk> Steven D'Aprano wrote: > On Tue, 16 Jun 2009 01:20:53 pm Cameron Simpson wrote: > > >> I don't think all >> pythons do immediate ref-counted GC. >> > > Jython and IronPython don't. I don't know about PyPy, CLPython, Unladen > Swallow, etc. > > > PyPy doesn't, Unladen Swallow won't. Michael -- http://www.ironpythoninaction.com/ From lukepadawan at gmail.com Tue Jun 16 15:10:40 2009 From: lukepadawan at gmail.com (Lucas P Melo) Date: Tue, 16 Jun 2009 10:10:40 -0300 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090616004327.GA15853@cskk.homeip.net> References: <20090616004327.GA15853@cskk.homeip.net> Message-ID: <4A3799D0.6010001@gmail.com> Cameron Simpson wrote: > Indeed, though arguably read1() is a lousy name too, on the same basis. > My itch is that peek() _feels_ like it should be "look into the buffer" > but actually can block and/or change the buffer. > I guess all the buffer operations should be transparent to the user if he wants it to be like that, since not so many people want to have a tight control over this kind of detail. I think of peek() as an operation that allows me to peek what's going to show up in the future without affecting further read()s. This kind of behavior is expected by users without prior knowledge of the inner workings of buffered IO. So, if an user _really_ wants to take a look at what's to come without affecting the buffer, we could allow that by doing something like this: peek(5, change_buffer=False) This is an alternative to the peek0(). But I am ok wih the peek0() too. From lukepadawan at gmail.com Tue Jun 16 15:14:08 2009 From: lukepadawan at gmail.com (Lucas P Melo) Date: Tue, 16 Jun 2009 10:14:08 -0300 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A377EA5.4090804@mrabarnett.plus.com> References: <20090616020311.GA31250@cskk.homeip.net> <4A377EA5.4090804@mrabarnett.plus.com> Message-ID: <4A379AA0.7030409@gmail.com> MRAB wrote: > I was thinking along the lines of: > > def peek(self, size=None, block=True) I think this is fine too. :) > > If 'block' is True then return 'size' bytes, unless the end of the > file/stream is reached; if 'block' is False then return up to 'size' > bytes, without blocking. The blocking form might impose a limit to how > much can be peeked (the maximum size of the buffer), or it might enlarge > the buffer as necessary. I guess the limit wouldn't be a problem to someone that chose to block further reads. From metawilm at gmail.com Tue Jun 16 16:19:09 2009 From: metawilm at gmail.com (Willem Broekema) Date: Tue, 16 Jun 2009 16:19:09 +0200 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: <4A378039.5050904@voidspace.org.uk> References: <20090616032053.GA15938@cskk.homeip.net> <200906162119.39786.steve@pearwood.info> <4A378039.5050904@voidspace.org.uk> Message-ID: On Tue, Jun 16, 2009 at 1:21 PM, Michael Foord wrote: > Steven D'Aprano wrote: >> On Tue, 16 Jun 2009 01:20:53 pm Cameron Simpson wrote: >>> I don't think all pythons do immediate ref-counted GC. >> >> Jython and IronPython don't. I don't know about PyPy, CLPython, Unladen >> Swallow, etc. > > PyPy doesn't, Unladen Swallow won't. Also most Lisp implementations, thus CLPython, use a tracing GC. - Willem From devin.c.cook at gmail.com Tue Jun 16 16:09:11 2009 From: devin.c.cook at gmail.com (Devin Cook) Date: Tue, 16 Jun 2009 10:09:11 -0400 Subject: [Python-Dev] SSL Certificate Validation Message-ID: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> Hi all, I have a few questions about validating SSL certificates. From what I gather, this validation occurs in the OpenSSL code called from _ssl.c. Is this correct? Also, I have looked through the docs and code, but haven't been able to figure out exactly what is included in certificate "validation". Is it just validating the chain? Does it check the NotBefore and NotAfter dates? Does it check that the host the socket is connected to is the same as what's given in the CN field in the certificate? Where I'm going with this is I think all this checking needs to be part of certificate validation in the ssl module. If it isn't yet, I'd be happy to work on a patch for it. Please let me know what you think. Thanks! -Devin Cook -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrs at mythic-beasts.com Tue Jun 16 20:04:44 2009 From: mrs at mythic-beasts.com (Mark Seaborn) Date: Tue, 16 Jun 2009 19:04:44 +0100 (BST) Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: <20090616032053.GA15938@cskk.homeip.net> References: <20090614.164255.343165597.mrs@localhost.localdomain> <20090616032053.GA15938@cskk.homeip.net> Message-ID: <20090616.190444.343167081.mrs@localhost.localdomain> Cameron Simpson wrote: > On 14Jun2009 16:42, Mark Seaborn wrote: > | I use a convenience function like this, so that GC takes care of the FDs: > | > | def make_pipe(): > | read_fd, write_fd = os.pipe() > | return os.fdopen(read_fd, "r"), os.fdopen(write_fd, "w") > > Not guarrenteed to be timely. The try/except at least closes things as > control passes out of the relevant scope. I don't think all pythons > do immediate ref-counted GC. Yep. I don't mind closing FDs explicitly when it's easy to do so in a try..finally, but it's not always simple. There are two different problems with non-prompt closing of FDs: * Whether an FD has been closed is sometimes externally observable. e.g. Pipes and sockets notify the other end of the connection. Open file and directory FDs prevent filesystems from being unmounted. * FDs use up space in the process's FD table. The second problem could be dealt with by running the GC when we get EMFILE, or before any calls that allocate FDs when the FD table is almost full, just as the GC runs when we "run out" of memory. I wonder if this proposal could help: 'GC & The "Expensive Object" Problem' http://www.eros-os.org/pipermail/e-lang/1999-May/002590.html Mark From Scott.Daniels at Acm.Org Tue Jun 16 20:20:48 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Tue, 16 Jun 2009 11:20:48 -0700 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A377EA5.4090804@mrabarnett.plus.com> References: <20090616020311.GA31250@cskk.homeip.net> <4A377EA5.4090804@mrabarnett.plus.com> Message-ID: MRAB wrote: > I was thinking along the lines of: > def peek(self, size=None, block=True) > If 'block' is True then return 'size' bytes, unless the end of the > file/stream is reached; if 'block' is False then return up to 'size' > bytes, without blocking.... I tend to prefer zero-ish defaults, how about: def peek(self, size=None, nonblocking=False) We still don't have "at most one read" code, but something a bit like data = obj.peek(size=desired, nonblocking=True) if len(data) < desired: data = obj.peek(size=wanted, nonblocking=False) might suffice. --Scott David Daniels Scott.Daniels at Acm.Org From v+python at g.nevcal.com Tue Jun 16 21:08:04 2009 From: v+python at g.nevcal.com (Glenn Linderman) Date: Tue, 16 Jun 2009 12:08:04 -0700 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: References: <20090616020311.GA31250@cskk.homeip.net> <4A377EA5.4090804@mrabarnett.plus.com> Message-ID: <4A37ED94.4060403@g.nevcal.com> On approximately 6/16/2009 11:20 AM, came the following characters from the keyboard of Scott David Daniels: > MRAB wrote: >> I was thinking along the lines of: >> def peek(self, size=None, block=True) >> If 'block' is True then return 'size' bytes, unless the end of the >> file/stream is reached; if 'block' is False then return up to 'size' >> bytes, without blocking.... > > I tend to prefer zero-ish defaults, how about: > def peek(self, size=None, nonblocking=False) No, no, no! Double negatives are extremely easy to not code correctly. The lack of ease of not understanding of code containing double negatives quadruples, at least! Not so differently, I'm sure my sentences here are not easy to understand because I put the inverse logic in them in the places that are not the usual. -- Glenn -- http://nevcal.com/ =========================== A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking From solipsis at pitrou.net Tue Jun 16 21:17:09 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 16 Jun 2009 19:17:09 +0000 (UTC) Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 References: <20090616020311.GA31250@cskk.homeip.net> <4A377EA5.4090804@mrabarnett.plus.com> Message-ID: Scott David Daniels Acm.Org> writes: > > MRAB wrote: > > I was thinking along the lines of: > > def peek(self, size=None, block=True) > > If 'block' is True then return 'size' bytes, unless the end of the > > file/stream is reached; if 'block' is False then return up to 'size' > > bytes, without blocking.... > > I tend to prefer zero-ish defaults, how about: > def peek(self, size=None, nonblocking=False) Since blocking and non-blocking are already used to refer to different types of raw streams, another verb should be found for this option. Antoine. From martin at v.loewis.de Tue Jun 16 21:23:18 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 16 Jun 2009 21:23:18 +0200 Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> Message-ID: <4A37F126.5010008@v.loewis.de> > I have a few questions about validating SSL certificates. From what I > gather, this validation occurs in the OpenSSL code called from _ssl.c. > Is this correct? This question is really off-topic for python-dev. As a python-dev poster, you should do research upfront, and only post on what you consider facts. > Where I'm going with this is I think all this checking needs to be part > of certificate validation in the ssl module. If it isn't yet, I'd be > happy to work on a patch for it. Please let me know what you think. I think you need to familiarize yourself much more with OpenSSL. Regards, Martin From jnoller at gmail.com Tue Jun 16 21:42:59 2009 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 16 Jun 2009 15:42:59 -0400 Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <4A37F126.5010008@v.loewis.de> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <4A37F126.5010008@v.loewis.de> Message-ID: <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com> On Tue, Jun 16, 2009 at 3:23 PM, "Martin v. L?wis" wrote: >> I have a few questions about validating SSL certificates. From what I >> gather, this validation occurs in the OpenSSL code called from _ssl.c. >> Is this correct? > > This question is really off-topic for python-dev. As a python-dev > poster, you should do research upfront, and only post on what you > consider facts. Martin, I told him to ask his question about _ssl internals on python-dev as he is new, and looking to work on some of the internals/make a patch for core. I didn't think that asking internals questions was a faux pas for the list, especially as he's looking to submit a patch to core. >> Where I'm going with this is I think all this checking needs to be part >> of certificate validation in the ssl module. If it isn't yet, I'd be >> happy to work on a patch for it. Please let me know what you think. > > I think you need to familiarize yourself much more with OpenSSL. I don't think that's called for, he is attempting to familiarize himself and simply inquiring about some of the internals. I'm sure he'll know plenty by the time the patch is more fully formed. -jesse From martin at v.loewis.de Tue Jun 16 22:14:35 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 16 Jun 2009 22:14:35 +0200 Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <4A37F126.5010008@v.loewis.de> <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com> Message-ID: <4A37FD2B.1000904@v.loewis.de> >> This question is really off-topic for python-dev. As a python-dev >> poster, you should do research upfront, and only post on what you >> consider facts. > > Martin, I told him to ask his question about _ssl internals on > python-dev as he is new, and looking to work on some of the > internals/make a patch for core. I didn't think that asking internals > questions was a faux pas for the list, especially as he's looking to > submit a patch to core. Hmm. For somebody new to Python, I'm fairly skeptical that the SSL module is the best starting point. >>> Where I'm going with this is I think all this checking needs to be part >>> of certificate validation in the ssl module. If it isn't yet, I'd be >>> happy to work on a patch for it. Please let me know what you think. >> I think you need to familiarize yourself much more with OpenSSL. > > I don't think that's called for, he is attempting to familiarize > himself and simply inquiring about some of the internals. I'm sure > he'll know plenty by the time the patch is more fully formed. But I really do believe that this is what he need to do next: familiarize himself with OpenSSL. There is a lot of APIs in that library, and it takes a while (i.e.: several months) to get productive, in particular since OpenSSL doesn't have the most intuitive API. >From "I want to know what features it currently has" to "I can contribute new features" is really a looong way here. To give a little more guidance: find out what SSL_CTX_use_certificate_chain_file and SSL_CTX_set_verify do. Finding that out is really out of scope of python-dev, since it has nothing to do with Python. Regards, Martin From devin.c.cook at gmail.com Tue Jun 16 23:31:23 2009 From: devin.c.cook at gmail.com (Devin Cook) Date: Tue, 16 Jun 2009 17:31:23 -0400 Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <4A37FD2B.1000904@v.loewis.de> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <4A37F126.5010008@v.loewis.de> <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com> <4A37FD2B.1000904@v.loewis.de> Message-ID: <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com> > But I really do believe that this is what he need to do next: > familiarize himself with OpenSSL. There is a lot of APIs in that > library, and it takes a while (i.e.: several months) to get > productive, in particular since OpenSSL doesn't have the most > intuitive API. Well, I realized this as soon as I looked at the _ssl.c code... I was just hoping?that someone would be able to give me a quick clarification on exactly what gets validated. If it's just the chain (which is what I suspect), I would like to submit a patch that does the rest of the validation (that a browser typically does: CN/hostname, NotBefore, NotAfter, etc.) in the ssl module. I was also hoping to find out what the consensus is about this: mainly, *should* that verification be done in the ssl module? Maybe this verification should somehow be done in OpenSSL, which would mean that I need to do a LOT more reading and go pester their mailing list instead. This is for issue 6273 ( http://bugs.python.org/issue6273 ). In your reply to that issue, it seemed to me like you were saying that these things were not getting checked in the ssl module (and, therefore, not in OpenSSL either): > I find the patch incomplete, for formal and semantical reasons: > a) it doesn't come with documentation or test suite changes, and > b) it doesn't implement the typical certificate checks that browsers > do, beyond validating that the certificate is valid - e.g. also > validating that the certificate is issued to the host you are trying > to connect to. I would like to do validation of server certificates in a project I'm working on, and I figured it would be better to be proactive and try to help create a patch than to just sit back and complain about it. It seems to me that this is a bug that you can't do peer certificate validation in httplib. If this isn't the place to ask these kinds of questions, I apologise. I can take the discussion elsewhere if I need to. Thanks, -Devin From janssen at parc.com Tue Jun 16 23:45:14 2009 From: janssen at parc.com (Bill Janssen) Date: Tue, 16 Jun 2009 14:45:14 PDT Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> Message-ID: <26228.1245188714@parc.com> Devin Cook wrote: > Also, I have looked through the docs and code, but haven't been able to > figure out exactly what is included in certificate "validation". Is it just > validating the chain? Does it check the NotBefore and NotAfter dates? I believe so, but you'll have to check the OpenSSL code. > Does it check that the host the socket is connected to is the same as > what's given in the CN field in the certificate? No. That, in general, doesn't work very well. The IETF working group on this is considering deprecating putting a hostname in the CN field at all, and just adding hostnames via the subjectAltName extension. The problem that's come up is that many computers don't have fixed IP addresses, and even with that the hostname is part of a different mapping of hostnames to IP addresses, which can also vary. I think that when the https: protocol scheme was written, it seemed like a good idea, but conventions on the Internet have changed a lot since then. > Where I'm going with this is I think all this checking needs to be part of > certificate validation in the ssl module. I don't think so. I put in hooks to let you do this in user code if you need to. See the archives for more discussion on this -- I'm not going to rehash it again. (This is really a question for OpenSSL mailing lists, or perhaps python-list.) Bill From jnoller at gmail.com Wed Jun 17 00:09:58 2009 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 16 Jun 2009 18:09:58 -0400 Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <4A37F126.5010008@v.loewis.de> <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com> <4A37FD2B.1000904@v.loewis.de> <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com> Message-ID: <4222a8490906161509x44c843bel9ca8fb14814ce6a8@mail.gmail.com> On Tue, Jun 16, 2009 at 5:31 PM, Devin Cook wrote: >> But I really do believe that this is what he need to do next: >> familiarize himself with OpenSSL. There is a lot of APIs in that >> library, and it takes a while (i.e.: several months) to get >> productive, in particular since OpenSSL doesn't have the most >> intuitive API. > > Well, I realized this as soon as I looked at the _ssl.c code... I was > just hoping?that someone would be able to give me a quick > clarification on exactly what gets validated. If it's just the chain > (which is what I suspect), I would like to submit a patch that does > the rest of the validation (that a browser typically does: > CN/hostname, NotBefore, NotAfter, etc.) in the ssl module. I was also > hoping to find out what the consensus is about this: mainly, *should* > that verification be done in the ssl module? Maybe this verification > should somehow be done in OpenSSL, which would mean that I need to do > a LOT more reading and go pester their mailing list instead. > > This is for issue 6273 ( http://bugs.python.org/issue6273 ). In your > reply to that issue, it seemed to me like you were saying that these > things were not getting checked in the ssl module (and, therefore, not > in OpenSSL either): > Also my initial bug report "client-side cert support" was a big fat typo on my part. face-palm'dly yours, jesse From martin at v.loewis.de Wed Jun 17 00:16:29 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 17 Jun 2009 00:16:29 +0200 Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <4A37F126.5010008@v.loewis.de> <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com> <4A37FD2B.1000904@v.loewis.de> <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com> Message-ID: <4A3819BD.4000301@v.loewis.de> > If this isn't the place to ask these kinds of questions, I apologise. > I can take the discussion elsewhere if I need to. It really depends on what "these questions" are. If your question is "I have this patch, is it correct?", then the question is entirely appropriate. If it is "I just have barely looked at the API, can somebody please explain it all to me?", then this isn't appropriate for this list, and probably not appropriate elsewhere: anybody answering this question could just as well fix the original problem right away. So please do try to find the answer for yourself, with the (little) direction I gave. If you find that it takes a lot of effort, then you'll probably have to accept the bug as-is, and live with it. FWIW, I actually don't know the answer for sure, either, so I would have to research this myself, too. In any case, _ssl.c is *not* the place where any of the certificate validation actually happens - nor does it happen elsewhere in the Python source code, IIUC. Regards, Martin From greg.ewing at canterbury.ac.nz Wed Jun 17 00:55:26 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 17 Jun 2009 10:55:26 +1200 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090616004327.GA15853@cskk.homeip.net> References: <20090616004327.GA15853@cskk.homeip.net> Message-ID: <4A3822DE.2040503@canterbury.ac.nz> Cameron Simpson wrote: > I normally avoid > non-blocking requirements by using threads, so that the thread gathering > from the stream can block. If you have a thread dedicated to reading from that stream, then I don't see why you need to peek into the buffer. Just have it loop reading a packet at a time and put each completed packet in the queue. If several packets arrive at once, it'll loop around that many times before blocking. > arguably read1() is a lousy name too, on the same basis. Certainly. > My itch is that peek() _feels_ like it should be "look into the buffer" > but actually can block and/or change the buffer. My problem with the idea of looking into the buffer is that it crosses levels of abstraction. A buffered stream is supposed to behave the same way as a deterministic non-buffered stream, with the buffer being an internal optimisation detail that doesn't exist as far as the outside world is concerned. Sometimes it's pragmatic to break the abstraction, but it should be made very obvious when you're doing that. So I'd prefer something like peek_buffer() to make it perfectly clear what's being done. Anything else such as peek() that doesn't explicitly mention the buffer should fit into the abstraction properly. -- Greg From greg.ewing at canterbury.ac.nz Wed Jun 17 01:04:37 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 17 Jun 2009 11:04:37 +1200 Subject: [Python-Dev] Functions that steal references (Re: [pygame] [patch] minor memory leaks...) In-Reply-To: <4A36F1EE.9040704@telus.net> References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net> Message-ID: <4A382505.8010101@canterbury.ac.nz> Lenard Lindstrom wrote: > I assumed that since PyModule_AddObject is documented as > stealing a reference, it always stole a reference. But in reality it > only does so conditionally, when it succeeds. As an aside, is this a general feature of functions that steal references, or is PyModule_AddObject an oddity? I ask because I've been thinking about adding features to Pyrex for dealing with stolen references, and it could be important to know things like this. Also, if it's an oddity, it would be a good idea to mention this behaviour in the docs. -- Greg From benjamin at python.org Wed Jun 17 01:05:08 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 16 Jun 2009 18:05:08 -0500 Subject: [Python-Dev] Functions that steal references (Re: [pygame] [patch] minor memory leaks...) In-Reply-To: <4A382505.8010101@canterbury.ac.nz> References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net> <4A382505.8010101@canterbury.ac.nz> Message-ID: <1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com> 2009/6/16 Greg Ewing : > Lenard Lindstrom wrote: > >> I assumed that since PyModule_AddObject is documented as stealing a >> reference, it always stole a reference. But in reality it only does so >> conditionally, when it succeeds. > > As an aside, is this a general feature of functions > that steal references, or is PyModule_AddObject an > oddity? IIRC, It's an oddity. > > I ask because I've been thinking about adding features > to Pyrex for dealing with stolen references, and it > could be important to know things like this. > > Also, if it's an oddity, it would be a good idea > to mention this behaviour in the docs. Yes, indeed. -- Regards, Benjamin From lists at cheimes.de Wed Jun 17 01:24:27 2009 From: lists at cheimes.de (Christian Heimes) Date: Wed, 17 Jun 2009 01:24:27 +0200 Subject: [Python-Dev] Functions that steal references (Re: [pygame] [patch] minor memory leaks...) In-Reply-To: <1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com> References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net> <4A382505.8010101@canterbury.ac.nz> <1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > 2009/6/16 Greg Ewing : >> Lenard Lindstrom wrote: >> >>> I assumed that since PyModule_AddObject is documented as stealing a >>> reference, it always stole a reference. But in reality it only does so >>> conditionally, when it succeeds. >> As an aside, is this a general feature of functions >> that steal references, or is PyModule_AddObject an >> oddity? > > IIRC, It's an oddity. But it is a convenient oddity nonetheless. Christian From solipsis at pitrou.net Wed Jun 17 01:38:49 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 16 Jun 2009 23:38:49 +0000 (UTC) Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 References: <20090616004327.GA15853@cskk.homeip.net> <4A3822DE.2040503@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > Anything else such as peek() that doesn't explicitly > mention the buffer should fit into the abstraction > properly. peek() doesn't "fit into the abstraction" since it doesn't even exist on raw streams. While buffered and non-buffered streams have a reasonably similar API, expecting them to behave the same in all circumstances is IMO unrealistic. Antoine. From greg.ewing at canterbury.ac.nz Wed Jun 17 02:25:22 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 17 Jun 2009 12:25:22 +1200 Subject: [Python-Dev] Functions that steal references (Re: [pygame] [patch] minor memory leaks...) In-Reply-To: References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net> <4A382505.8010101@canterbury.ac.nz> <1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com> Message-ID: <4A3837F2.4070802@canterbury.ac.nz> Christian Heimes wrote: > But it is a convenient oddity nonetheless. What's convenient about it? Seems to me it's the opposite, since you can't just bail out if it fails, but have to decref the reference you thought it was going to take care of for you. -- Greg From cs at zip.com.au Wed Jun 17 03:27:19 2009 From: cs at zip.com.au (Cameron Simpson) Date: Wed, 17 Jun 2009 11:27:19 +1000 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A3822DE.2040503@canterbury.ac.nz> Message-ID: <20090617012719.GA19974@cskk.homeip.net> On 17Jun2009 10:55, Greg Ewing wrote: > Cameron Simpson wrote: >> I normally avoid >> non-blocking requirements by using threads, so that the thread gathering >> from the stream can block. > > If you have a thread dedicated to reading from that > stream, then I don't see why you need to peek into > the buffer. Just have it loop reading a packet at a > time and put each completed packet in the queue. > If several packets arrive at once, it'll loop around > that many times before blocking. Yes, this is true. But people not using threads, or at any rate not dedicating a thread to the reading task, don't have such luxury. Are we disputing the utility of being able to ask "how much might I read/peek without blocking"? Or disputing the purpose of peek, which feels to me like it should/might ask that question, but doesn't. [...] >> My itch is that peek() _feels_ like it should be "look into the buffer" >> but actually can block and/or change the buffer. > > My problem with the idea of looking into the buffer > is that it crosses levels of abstraction. A buffered > stream is supposed to behave the same way as a > deterministic non-buffered stream, with the buffer > being an internal optimisation detail that doesn't > exist as far as the outside world is concerned. > > Sometimes it's pragmatic to break the abstraction, > but it should be made very obvious when you're doing > that. So I'd prefer something like peek_buffer() to > make it perfectly clear what's being done. > > Anything else such as peek() that doesn't explicitly > mention the buffer should fit into the abstraction > properly. It's perfectly possible, even reasonable, to avoid talking about the buffer at all. I'd be happy not to mention the buffer. For example, one can readily imagine the buffered stream being capable of querying its input raw stream if there's "available now" data. The raw stream can sometimes know if a read of a given size would block, or be able to ask what size read will not block. As a concrete example, the UNIX FIONREAD ioctl can generally query a file descriptor for instantly-available data (== in the OS buffer). I've also used UNIXen where your can fstat() a pipe and use the st_size field to test for available unread data in the pipe buffer. Raw streams which can't do that would return 0 (== can't guarentee any non-blocking data) unless the stream itself also had a buffer of its own and it wasn't empty. So I would _want_ the spec for available_data() (new lousy name) to talk about "data available without blocking", allowing the implementation to use data in the IO buffer and/or to query the raw stream, etc. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ For those who understand, NO explanation is needed, for those who don't understand, NO explanation will be given! - Davey D From cylix at solace.info Wed Jun 17 04:21:40 2009 From: cylix at solace.info (Frederick Reeve) Date: Tue, 16 Jun 2009 21:21:40 -0500 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: References: <20090610203857.00a019fa@cylix> <20090612193707.24f5563c@cylix> Message-ID: <20090616212140.5f291873@cylix> On Sat, 13 Jun 2009 12:33:46 +0000 (UTC) Antoine Pitrou wrote: > This proposal looks reasonable to me. Please note that it's too late for 3.1 > anyway - we're in release candidate phase. Once you have a patch, you can post > it on the bug tracker. Thanks I will do that. Sometime in the next couple of weeks. Gratefully Frederick From hrvoje.niksic at avl.com Wed Jun 17 10:16:15 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 17 Jun 2009 10:16:15 +0200 Subject: [Python-Dev] Functions that steal references (Re: [pygame] [patch] minor memory leaks...) In-Reply-To: <31501613.234006.1245194699078.JavaMail.xicrypt@atgrzls001> References: <4A36AF6C.2000503@amberfisharts.com> <4A36F1EE.9040704@telus.net> <4A382505.8010101@canterbury.ac.nz> <1afaf6160906161605j5e8ec66dgb5ae740abf21de0a@mail.gmail.com> <31501613.234006.1245194699078.JavaMail.xicrypt@atgrzls001> Message-ID: <4A38A64F.1080005@avl.com> Christian Heimes wrote: >>>> I assumed that since PyModule_AddObject is documented as stealing a >>>> reference, it always stole a reference. But in reality it only does so >>>> conditionally, when it succeeds. >>> As an aside, is this a general feature of functions >>> that steal references, or is PyModule_AddObject an >>> oddity? >> >> IIRC, It's an oddity. > > But it is a convenient oddity nonetheless. Stealing references is sometimes convenient, but Greg was referring to functions that steal references *conditionally*, which is indeed an oddity. Most functions and macros that steal references do so unconditionally, typically because they can't fail anyway. Conditional stealing of references requires very careful thinking on the side of callers that care about not leaking references in the face of exceptions. See http://bugs.python.org/issue1782 for an example. From zeal_goswami at yahoo.com Wed Jun 17 12:27:13 2009 From: zeal_goswami at yahoo.com (abhishek goswami) Date: Wed, 17 Jun 2009 15:57:13 +0530 (IST) Subject: [Python-Dev] Python script language or not Message-ID: <350154.15071.qm@web94907.mail.in2.yahoo.com> Hi, I have very basic question about Python that do we consider pyhton as script language. I searched in google but it becomes more confusion for me. After some analysis I came to know that Python support oops . Can anyone clarify me. Please let me know also it is right forum or not. I was looking for Python forum but i did not find any forum. Do we have forum like linux forum so it will be very easy for me to ask this question in forum and i will not ask repeated question becuase i am sure this question may be raised before Abhishek Goswami Chennai Phone No -0996227099 Love Cricket? Check out live scores, photos, video highlights and more. Click here http://cricket.yahoo.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dirkjan at ochtman.nl Wed Jun 17 12:52:05 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Wed, 17 Jun 2009 12:52:05 +0200 Subject: [Python-Dev] Python script language or not In-Reply-To: <350154.15071.qm@web94907.mail.in2.yahoo.com> References: <350154.15071.qm@web94907.mail.in2.yahoo.com> Message-ID: On Wed, Jun 17, 2009 at 12:27, abhishek goswami wrote: > Can anyone clarify me. Please let me know also it is right forum or not. This is not the right forum. This mailing list is about developing the CPython interpreter. For general questions, you may want to try the comp.lang.python newsgroup. Cheers, Dirkjan From tav at espians.com Wed Jun 17 16:13:29 2009 From: tav at espians.com (tav) Date: Wed, 17 Jun 2009 15:13:29 +0100 Subject: [Python-Dev] CPython in the web browser under Native Client In-Reply-To: <20090614.170936.846958184.mrs@localhost.localdomain> References: <20090614.170936.846958184.mrs@localhost.localdomain> Message-ID: Hey Mark, > http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html Really glad to see that you carried on with this -- great work!! I guess the elders will simply say that it's history repeating itself, but Grails 2.0 is looking promising! We can finally give that upstart Javascript a run for its money (despite ES5 looking nice...) -- not to mention Google Wave/Opera Unite/etc. Now, the real blocker is accessing the DOM from Python. I haven't looked into the source code deeply, but perhaps Kroll could be of help: * http://github.com/jhaynie/kroll/tree/master I was able to do the following with Titanium Desktop (which uses it) to access the DOM from Python:
Hello
World
More relevant links: * http://www.appcelerator.com/ * http://github.com/appcelerator/ Anyways, I'm sure others might have a better idea of integrating Python and the DOM. Well done again! -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From tjreedy at udel.edu Wed Jun 17 19:26:50 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 17 Jun 2009 13:26:50 -0400 Subject: [Python-Dev] Python script language or not In-Reply-To: References: <350154.15071.qm@web94907.mail.in2.yahoo.com> Message-ID: Dirkjan Ochtman wrote: > On Wed, Jun 17, 2009 at 12:27, abhishek goswami wrote: >> Can anyone clarify me. Please let me know also it is right forum or not. > > This is not the right forum. This mailing list is about developing the > CPython interpreter. > > For general questions, you may want to try the comp.lang.python newsgroup. Or the python-list mailing list, or the gmane.comp.python.general newsgroup mirror at gmane.org (free). From janssen at parc.com Wed Jun 17 21:14:35 2009 From: janssen at parc.com (Bill Janssen) Date: Wed, 17 Jun 2009 12:14:35 PDT Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <26228.1245188714@parc.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <26228.1245188714@parc.com> Message-ID: <36924.1245266075@parc.com> Bill Janssen wrote: > > Does it check that the host the socket is connected to is the same as > > what's given in the CN field in the certificate? > > No. That, in general, doesn't work very well. The IETF working group > on this is considering deprecating putting a hostname in the CN field at > all, and just adding hostnames via the subjectAltName extension. The > problem that's come up is that many computers don't have fixed IP > addresses, and even with that the hostname is part of a different > mapping of hostnames to IP addresses, which can also vary. Incidentally, the current working draft on this seems to be at . Bill From janssen at parc.com Wed Jun 17 21:15:57 2009 From: janssen at parc.com (Bill Janssen) Date: Wed, 17 Jun 2009 12:15:57 PDT Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <4A3819BD.4000301@v.loewis.de> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <4A37F126.5010008@v.loewis.de> <4222a8490906161242p305e2268q550cf48d1d82f884@mail.gmail.com> <4A37FD2B.1000904@v.loewis.de> <8847f980906161431n2f9043afk27a43194ac3721f5@mail.gmail.com> <4A3819BD.4000301@v.loewis.de> Message-ID: <36941.1245266157@parc.com> Martin v. L?wis wrote: > FWIW, I actually don't know the answer for sure, either, so I would have > to research this myself, too. In any case, _ssl.c is *not* the place > where any of the certificate validation actually happens - nor does it > happen elsewhere in the Python source code, IIUC. Strictly speaking, that's right. It's all done by OpenSSL. Bill From devin.c.cook at gmail.com Wed Jun 17 23:00:01 2009 From: devin.c.cook at gmail.com (Devin Cook) Date: Wed, 17 Jun 2009 17:00:01 -0400 Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <36924.1245266075@parc.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <26228.1245188714@parc.com> <36924.1245266075@parc.com> Message-ID: <8847f980906171400p3656304o70559dc2a946c789@mail.gmail.com> Ok, thanks for all the feedback. Just for clarity, I'll summarize everything as I understand it: * OpenSSL does the all validation of the certificate itself. (http://openssl.org/docs/apps/verify.html) * httplib should have a way to enable validation of the certificate. * httplib should have a way to enable checking of the reference identity. (that complies with section 3 of this draft: http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00) * The reference identity checking (and cert validation, I assume) shouldn't be automatic. (per Bill) Does that sound about right? I'll try to work up a patch tonight implementing this. -Devin From janssen at parc.com Wed Jun 17 23:57:48 2009 From: janssen at parc.com (Bill Janssen) Date: Wed, 17 Jun 2009 14:57:48 PDT Subject: [Python-Dev] SSL Certificate Validation In-Reply-To: <8847f980906171400p3656304o70559dc2a946c789@mail.gmail.com> References: <8847f980906160709q405ad950qac0c56b304562d3@mail.gmail.com> <26228.1245188714@parc.com> <36924.1245266075@parc.com> <8847f980906171400p3656304o70559dc2a946c789@mail.gmail.com> Message-ID: <40972.1245275868@parc.com> I think if you check the issue tracker, there's already a patch for this somewhere, IIRC. Bill Devin Cook wrote: > Ok, thanks for all the feedback. Just for clarity, I'll summarize > everything as I understand it: > > * OpenSSL does the all validation of the certificate itself. > (http://openssl.org/docs/apps/verify.html) > * httplib should have a way to enable validation of the certificate. > * httplib should have a way to enable checking of the reference > identity. (that complies with section 3 of this draft: > http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00) > * The reference identity checking (and cert validation, I assume) > shouldn't be automatic. (per Bill) > > Does that sound about right? I'll try to work up a patch tonight > implementing this. > > -Devin From greg.ewing at canterbury.ac.nz Thu Jun 18 03:48:31 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 18 Jun 2009 13:48:31 +1200 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <20090617012719.GA19974@cskk.homeip.net> References: <20090617012719.GA19974@cskk.homeip.net> Message-ID: <4A399CEF.2070202@canterbury.ac.nz> Cameron Simpson wrote: > But people not using threads, or at any rate not > dedicating a thread to the reading task, don't have such luxury. But without a dedicated thread you need to use select() or poll(), and then buffering causes other headaches. > Are we disputing the utility of being able to ask "how much might I > read/peek without blocking"? I'm saying that I don't see how I would make use of such a thing, so I probably wouldn't mind if it didn't exist. > Or disputing the purpose of peek, which > feels to me like it should/might ask that question, but doesn't. I think what I'm saying is that there are two distinct use cases being talked about for a peek-like operation, and different people seem to have different ideas on which one should be mapped to the name "peek". So perhaps they should both be given more-explicit names. > It's perfectly possible, even reasonable, to avoid talking about the > buffer at all. I'd be happy not to mention the buffer. Even if you don't mention it explicitly, its existence shows through in the fact that there is an arbitrary limit on the amount you can peek ahead, and that limit needs to be documented so that people can write correct programs. This is true of both kinds of peeking, so I concede that they both break the abstraction. However I think the non-blocking peek breaks it more than the blocking one, because it also brings non-deterministic behaviour. -- Greg From drnlmuller+python at gmail.com Thu Jun 18 15:22:13 2009 From: drnlmuller+python at gmail.com (Neil Muller) Date: Thu, 18 Jun 2009 15:22:13 +0200 Subject: [Python-Dev] ElementTree issues in python 3.1 rc2 Message-ID: The following issues in ElementTree still exist in the latest release candidate & all have small patches to fix the issue. Any chance of getting these in before the final release? http://bugs.python.org/issue6231 http://bugs.python.org/issue6233 http://bugs.python.org/issue2746 -- Neil Muller drnlmuller at gmail.com From solipsis at pitrou.net Thu Jun 18 15:27:26 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 18 Jun 2009 13:27:26 +0000 (UTC) Subject: [Python-Dev] ElementTree issues in python 3.1 rc2 References: Message-ID: Neil Muller gmail.com> writes: > > The following issues in ElementTree still exist in the latest release > candidate & all have small patches to fix the issue. Any chance of > getting these in before the final release? > > http://bugs.python.org/issue6231 > http://bugs.python.org/issue6233 > http://bugs.python.org/issue2746 I've set issue 6233 as a release blocker, since it's an annoying regression compared to trunk. As a side note, I think we have a process problem with externally maintained modules such as etree and json, especially when the maintainer isn't very reactive to bug reports. Regards Antoine. From lukepadawan at gmail.com Thu Jun 18 16:54:18 2009 From: lukepadawan at gmail.com (Lucas P Melo) Date: Thu, 18 Jun 2009 11:54:18 -0300 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A399CEF.2070202@canterbury.ac.nz> References: <20090617012719.GA19974@cskk.homeip.net> <4A399CEF.2070202@canterbury.ac.nz> Message-ID: <4A3A551A.6060304@gmail.com> Greg Ewing wrote: > Even if you don't mention it explicitly, its > existence shows through in the fact that there > is an arbitrary limit on the amount you can > peek ahead, and that limit needs to be documented > so that people can write correct programs. > > This is true of both kinds of peeking, so I > concede that they both break the abstraction. > > However I think the non-blocking peek breaks it > more than the blocking one, because it also > brings non-deterministic behaviour. It depends on the point of view. For example, someone is writing a program that must read from any kind of file descriptor and generate the derivation tree of the text read based on some context-free grammar. The problem is that the chosen method to accomplish it would read 2 symbols (bytes) ahead and this guy is using peek() to grab these 2 bytes. The program will seem to work correctly most of the time, but on the 4095th byte read, he would grab 1 byte at most using peek() (despite the existence of say 10k bytes ahead). The blocking definition of peek() would create this hard-to-spot bug. Thus, for him, this behavior would seem non-deterministic. On the other hand, if someone is conscious about the number of raw reads, he would definitely be willing to look into the documentation for any parameters that match his special needs. That's why the non-blocking behavior should be the default one while the blocking behavior should be accessible by choice. From eric.pruitt at gmail.com Thu Jun 18 17:30:29 2009 From: eric.pruitt at gmail.com (Eric Pruitt) Date: Thu, 18 Jun 2009 10:30:29 -0500 Subject: [Python-Dev] Popen asynchronous input for Windows Message-ID: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com> Hello, I am looking for alternatives to Josiah Carlson's asynchronous I/O patch for subprocess.Popen. While his patch seems to work well, it relies on pywin32 which is not part of the standard Python library. If I cannot find an alternative, I will be using cTypes with the parts of Mark Hammond's code that I need, license permitting. Any suggestions are greatly appreciated. Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at cheimes.de Thu Jun 18 20:32:27 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 18 Jun 2009 20:32:27 +0200 Subject: [Python-Dev] Popen asynchronous input for Windows In-Reply-To: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com> References: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com> Message-ID: <4A3A883B.5040508@cheimes.de> Eric Pruitt schrieb: > Hello, > > I am looking for alternatives to Josiah Carlson's asynchronous I/O patch for > subprocess.Popen. While his patch seems to work well, it relies on pywin32 > which is not part of the standard Python library. If I cannot find an > alternative, I will be using cTypes with the parts of Mark Hammond's code > that I need, license permitting. Any suggestions are greatly appreciated. The subprocess module several wrappers for win32 APIs in Modules/_subprocess.c. You could add the necessary functions. Christian From facundobatista at gmail.com Thu Jun 18 22:07:02 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Thu, 18 Jun 2009 17:07:02 -0300 Subject: [Python-Dev] Avoiding file descriptors leakage in subprocess.Popen() In-Reply-To: References: <4A32ED75.7060803@cheimes.de> Message-ID: On Sat, Jun 13, 2009 at 9:40 AM, Facundo Batista wrote: >> How about a nice 'n shiny context wrapper for the pipe: > > I'll do this! > > Thank you for the suggestion! Boo, I can not take this approach, neither the previous one. The reason is very specific for subprocess.py... after creating the FDs, it forks(), and the behaviour of when closing which descriptors are different for the parent and child processes. If I take Christian approach, the test just hangs. One drawback of the "with" usage is that it closes both FDs at the same time... and in this case this may be a problem, as they're used and closed by different processes in different times... don't know. I also tried the approach of wrapping the FDs with a file object... this *almost* work... but in the case of a problem in the child process, when a traceback should be written through the pipe to the parent, nothing happens (it seems that the GC just closes the object and the info is not transferred). So, I'll stick to the code I submitted to the bug, because even if it's ugly it works. -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From eric.pruitt at gmail.com Thu Jun 18 22:29:44 2009 From: eric.pruitt at gmail.com (Eric Pruitt) Date: Thu, 18 Jun 2009 15:29:44 -0500 Subject: [Python-Dev] Popen asynchronous input for Windows In-Reply-To: <4A3A883B.5040508@cheimes.de> References: <171e8a410906180830u3ef49ba8j3dc384f329763167@mail.gmail.com> <4A3A883B.5040508@cheimes.de> Message-ID: <171e8a410906181329j5f981266p5a61b4a9f4008357@mail.gmail.com> Thanks for the lead. I have the pywin32 source code and have found the files that appear to contain the code I need inside of some "*.i" files. After a bit of Googling and paying attention to the blatantly obvious *.cpp files, I realized Hammond's code is written in C++ whereas Python uses C. I am familiar with neither language very well but if I can't find a work-around for Windows asynchronous I/O, I will be learning a bit of both of them. Please feel free to suggest any other ideas that don't rely on my sparse knowledge of C and C++. Eric On Thu, Jun 18, 2009 at 13:32, Christian Heimes wrote: > Eric Pruitt schrieb: > > Hello, > > > > I am looking for alternatives to Josiah Carlson's asynchronous I/O patch > for > > subprocess.Popen. While his patch seems to work well, it relies on > pywin32 > > which is not part of the standard Python library. If I cannot find an > > alternative, I will be using cTypes with the parts of Mark Hammond's code > > that I need, license permitting. Any suggestions are greatly appreciated. > > The subprocess module several wrappers for win32 APIs in > Modules/_subprocess.c. You could add the necessary functions. > > Christian > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Jun 19 02:50:27 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 19 Jun 2009 12:50:27 +1200 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A3A551A.6060304@gmail.com> References: <20090617012719.GA19974@cskk.homeip.net> <4A399CEF.2070202@canterbury.ac.nz> <4A3A551A.6060304@gmail.com> Message-ID: <4A3AE0D3.2090708@canterbury.ac.nz> Lucas P Melo wrote: > The problem is that the chosen > method to accomplish it would read 2 symbols (bytes) ahead and this guy > is using peek() to grab these 2 bytes. The program will seem to work > correctly most of the time, but on the 4095th byte read, he would grab 1 > byte at most using peek() That's exactly why I think the blocking version should keep reading until the requested number of bytes is available (or the buffer is full or EOF occurs). In other words, the blocking version should be fully deterministic given knowledge of the buffer size. -- Greg From benjamin at python.org Fri Jun 19 04:17:44 2009 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 18 Jun 2009 21:17:44 -0500 Subject: [Python-Dev] draft pep: backwards compatibility Message-ID: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Backwards compatibility seems to be an issue that arises a lot here. I think we all have an idea of it is, but we need some hard place to point to. So here's my attempt: PEP: 387 Title: Backwards Compatibility Policy Version: $Revision$ Last-Modified: $Date$ Author: Benjamin Peterson Status: Draft Type: Process Content-Type: text/x-rst Created: 18-Jun-2009 Abstract ======== This PEP outlines Python's backwards compatibility policy. Rationale ========= As one of the most used programming languages today [#tiobe]_, the Python core language and its standard library play a critcal role in thousands of applications and libraries. This is fantastic; it is probably one of a language designer's most wishful dreams. However, it means the development team must be very careful not to break this existing 3rd party code with new releases. Backwards Compatibility Rules ============================= This policy applys to all public APIs. These include the C-API, the standard library, and the core language including syntax and operation as defined by the reference manual. This is the basic policy for backwards compatibility: * The behavior of an API *must* not change between any two consecutive releases. * A feature cannot be removed without notice between any two consecutive releases. * Addition of a feature which breaks 3rd party libraries or applications should have a large benefit to breakage ratio, and/or the incompatibility should be trival to fix in broken code. Making Incompatible Changes =========================== It's a fact: design mistakes happen. Thus it is important to be able to change APIs or remove misguided features. This is accomplished through a gradual process over several releases: 1. Discuss the change. Depending on the size of the incompatibility, this could be on the bug tracker, python-dev, python-list, or the appropriate SIG. A PEP or similar document may be written. Hopefully users of the affected API will pipe up to comment. 2. Add a warning [#warnings_]_. If behavior is changing, a the API may gain a new function or method to perform the new behavior; old usage should raise the warning. If an API is being removed, simply warn whenever it is entered. DeprecationWarning is the usual warning category to use, but PendingDeprecationWarning may be used in special cases were the old and new versions of the API will coexist for many releases. 3. Wait for a release. 4. See if there's any feedback. Users not involved in the original discussions may comment now after seeing the warning. Perhaps reconsider. 5. The behavior change or feature removal may now be made default or permanent in the next release. Remove the old version and warning. References ========== .. [#tiobe] TIOBE Programming Community Index http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html .. [#warnings] The warnings module http://docs.python.org/library/warnings.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From solipsis at pitrou.net Fri Jun 19 10:23:01 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 19 Jun 2009 08:23:01 +0000 (UTC) Subject: [Python-Dev] draft pep: backwards compatibility References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > This policy applys to all public APIs. applies? > * The behavior of an API *must* not change between any two consecutive > releases. > > * A feature cannot be removed without notice between any two consecutive > releases. By induction, this would mean no API could change and no feature could be removed without notice between any N consecutive releases. Do you really mean it? > * Addition of a feature which breaks 3rd party libraries or applications > should > have a large benefit to breakage ratio, and/or the incompatibility should > be > trival to fix in broken code. There is always the possibility that a new feature breaks existing code, for example because it relies on a similarly named attribute, or on some obscure internal condition. I think this should be qualified so that it only applies when e.g. a fair number of third-party apps or libraries are broken. From p.f.moore at gmail.com Fri Jun 19 10:29:23 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 19 Jun 2009 09:29:23 +0100 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <79990c6b0906190129p5c930fe3l1d93162219e0bbbf@mail.gmail.com> 2009/6/19 Benjamin Peterson : > Backwards compatibility seems to be an issue that arises a lot here. I > think we all have an idea of it is, but we need some hard place to > point to. So here's my attempt: Nice :-) A general point - it's probably worth clarifying that you're referring to major releases (2.6 -> 2.7 etc.) here. Minor releases have a strict bugfixes-only policy. > applications and libraries. ?This is fantastic; it is probably one of a language > designer's most wishful dreams. That's a slightly odd wording. I'm not sure I can think of a better one, though... > This policy applys to all public APIs. ?These include the C-API, the standard ... applies ... > * The behavior of an API *must* not change between any two consecutive releases. With the exception of compatibility warnings as described below. In practice, I think APIs *do* change, so presumably there must have been a pair of releases between which the change happened. I see what you're trying to say, but I think you overstated things a little. Paul. From glyph at divmod.com Fri Jun 19 11:31:30 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 19 Jun 2009 09:31:30 -0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> On 02:17 am, benjamin at python.org wrote: >Backwards compatibility seems to be an issue that arises a lot here. I >think we all have an idea of it is, but we need some hard place to >point to. So here's my attempt: >PEP: 387 >Title: Backwards Compatibility Policy Thanks for getting started on this. This is indeed a bikeshed that there would be less discussion around if there were a clearer PEP to point to. However, this draft PEP points to the problem rather than the solution. In order to really be useful this needs to answer a whole slew of very very specific questions, because the real problem is that everybody thinks they know what they mean but in fact we all have slightly different definitions of these words. I've taken a stab at this myself in the past while defining a policy for Twisted, which is here: . I think we might be a bit stricter than Python, but I believe our attempt to outline these terms specifically is worth taking a look at. The questions that follow might seem satirical or parodic but I am completely serious and each of these terms really does have a variable definition. >* The behavior of an API *must* not change between any two consecutive >releases. What is "behavior"? Software is composed of behavior. If no behavior changes, then nothing can ever happen. What is an "API"? Are you talking about a public name in a Python module? A name covered in documentation? A name covered by a specific piece of documentation? Is "subclassing" considered an API, or just invoking? What about isinstance()? What about repr()? Are string formats allowed to change? For example, certain warnings have caused the Twisted test suite to fail because with the introduction of warnings at the C level it becomes progressively harder for a Python process to control its own stderr stream. I can't remember the specifics right now, but on more than one occasion a python upgrade caused our test suite to fail because the expectation was that a process would be able to successfully terminate without putting anything on any FD but stdout. In order for any changes to be possible, there needs to be a clearly labeled mine-field: don't touch or depend on these things in your Python application or it *will* break every time Python is released. What are "consecutive" releases? If we have releases 1, 2, 3, 4, and behavior can't change 1->2, 2->3, or 3->4, then no change may ever be made. What does "must not" mean here? My preference would be "once there is agreement that an incompatible change has occurred, there should be an immediate revert with no discussion". Unless of course the change has already been released. >* A feature cannot be removed without notice between any two >consecutive > releases. Presumably removal of a feature is a change in behavior, so isn't this redundant with the previous point? >* Addition of a feature which breaks 3rd party libraries or >applications should > have a large benefit to breakage ratio, and/or the incompatibility >should be > trival to fix in broken code. How do you propose to measure the benefit to breakage ratio? How do you propose to define "trivial" to fix? Most projects, including Python, Twisted, Django, and Zope, have an ever-increasing bug count, which means that trivial issues fall off the radar pretty easily. One of the big problems here in detecting and fixing "trivial" changes is that they can occur in test code as well. So if the answer is "run your tests", that's not much help if you can't call the actual APIs whose behavior has changed. The standard library would need to start providing a lot more verified test stubs for things like I/O in order for this to be feasible. >Making Incompatible Changes >=========================== > >It's a fact: design mistakes happen. Thus it is important to be able >to change >APIs or remove misguided features. This is accomplished through a >gradual >process over several releases: > >1. Discuss the change. Depending on the size of the incompatibility, >this could > be on the bug tracker, python-dev, python-list, or the appropriate >SIG. A > PEP or similar document may be written. Hopefully users of the >affected API > will pipe up to comment. There are a very large number of users of Python, the large percentage of whom do not read python-dev. A posting on python-list is likely to provoke an unproductive pile-on. I suggest a dedicated list, which should ideally be low traffic, "python-incompatible-announce", or something like that, so that *everyone* who maintains Python software can subscribe to it, even if they're not otherwise interested in Python development. >2. Add a warning [#warnings_]_. If behavior is changing, a the API may >gain a > new function or method to perform the new behavior; old usage should >raise > the warning. If an API is being removed, simply warn whenever it is >entered. > DeprecationWarning is the usual warning category to use, but > PendingDeprecationWarning may be used in special cases were the old >and new > versions of the API will coexist for many releases. Why is PendingDeprecationWarning the special case? As outlined in the Twisted compatibility proposal, I'd prefer it if every warning went through multiple phases: pending, deprecated, gone: so that careful developers could release new versions of their software to quash noisy warnings *before* the new version of Python that actually made them user-visible was released. >3. Wait for a release. How does this affect the parallel 2.x/3.x release cycle? >4. See if there's any feedback. Users not involved in the original >discussions > may comment now after seeing the warning. Perhaps reconsider. At this point, many developers may already have been porting over to the new, non-deprecated API. I realize that there may be no way around that, but it's worth considering. Thanks for tackling this thorny problem. Good luck! From solipsis at pitrou.net Fri Jun 19 11:21:49 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 19 Jun 2009 09:21:49 +0000 (UTC) Subject: [Python-Dev] draft pep: backwards compatibility References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> Message-ID: divmod.com> writes: > > In order for any changes to be possible, there needs to be a clearly > labeled mine-field: don't touch or depend on these things in your Python > application or it *will* break every time Python is released. I think the "frozen area" should be defined positively (explicit public APIs and explicitly guaranteed behaviour) rather than negatively (an explicit "mine field"). Otherwise, we will forget to put some important things in the minefield and get bitten later when we need to change those things in a backwards-incompatible way. For example, I think it was wrong to change the name of a test-skipping unittest method just so that it wouldn't clash with a method created by Twisted subclassing unittest (besides, self.skip() was much nicer than self.skipTest() ;-)). Just because some class is public shouldn't prevent us to add new public methods or attributes to it. Regards Antoine. From glyph at divmod.com Fri Jun 19 13:26:34 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 19 Jun 2009 11:26:34 -0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> Message-ID: <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> On 09:21 am, solipsis at pitrou.net wrote: > divmod.com> writes: >> >>In order for any changes to be possible, there needs to be a clearly >>labeled mine-field: don't touch or depend on these things in your >>Python >>application or it *will* break every time Python is released. > >I think the "frozen area" should be defined positively (explicit public >APIs and >explicitly guaranteed behaviour) rather than negatively (an explicit >"mine >field"). Otherwise, we will forget to put some important things in the >minefield >and get bitten later when we need to change those things in a >backwards-incompatible way. This is a false dichotomy; for core developers, the list needs to be exhaustive. Everything that can change needs to be described as either compatible or incompatible. However, the reason I say that the list needs to be phrased as a whitelist is that we have to assume *all* people writing Python code, who ever want to be able to upgrade Python, need to *completely* understand the list of things which they should not depend on. Every piece of documentation they read and every API they find, the assumption is that it's going to be compatible and it's not something they need to worry about. It's important to remember that this PEP has a "public" (python application programmers) and "private" (python core developer) interfaces, too ;-). >For example, I think it was wrong to change the name of a test-skipping >unittest >method just so that it wouldn't clash with a method created by Twisted >subclassing unittest (besides, self.skip() was much nicer than >self.skipTest() >;-)). Just because some class is public shouldn't prevent us to add new >public >methods or attributes to it. I think it would be wrong to have a blanket prohibition on such changes, by which I mean additions of names to public namespaces. Twisted's own compatibility possibility would not forbid a similar change. But in that specific case I think it was the right thing to do. Like it or not, a lot of people use Twisted, a lot of people run tests with Trial, and we beat stdlib unittest to the punch on the 'skip' testing feature by a good 5 years. We caught the change well before the release, we reported it and discussed it. IMHO this is the best way to deal with incompatible changes, especially in the case of superclasses, given Python's subtle and complex inheritance semantics. It's not feasible to provide a policy that somehow prohibits subclasses from adding names which may eventually be used on a superclass. Projects which notice test failures with new versions of Python should report them so that the features can be adjusted to be compatible, assuming the project in question hasn't done anything in egregious violation of the compatibility policy (like invoking a private method). This lets users, system administrators, and application authors upgrade components individually, without worrying about the components further down the stack flaking out on them. It also provides a feedback loop for the compatibility policy: if there are things that initially seem reasonable, but projects report compatibility issues when they are changed, they might need to be added later. From solipsis at pitrou.net Fri Jun 19 13:11:11 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 19 Jun 2009 11:11:11 +0000 (UTC) Subject: [Python-Dev] draft pep: backwards compatibility References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> Message-ID: divmod.com> writes: > > This is a false dichotomy; for core developers, the list needs to be > exhaustive. Everything that can change needs to be described as either > compatible or incompatible. How do you enumerate "everything that can change"? It does not look like a finite set to me (but perhaps I'm wrong); and certainly not like a set of a size reasonable enough to be enumerated in a human-readable way :) From fuzzyman at voidspace.org.uk Fri Jun 19 13:13:39 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 19 Jun 2009 12:13:39 +0100 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> Message-ID: <4A3B72E3.3070106@voidspace.org.uk> glyph at divmod.com wrote: > [snip...] >> For example, I think it was wrong to change the name of a >> test-skipping unittest >> method just so that it wouldn't clash with a method created by Twisted >> subclassing unittest (besides, self.skip() was much nicer than >> self.skipTest() >> ;-)). Just because some class is public shouldn't prevent us to add >> new public >> methods or attributes to it. > > I think it would be wrong to have a blanket prohibition on such > changes, by which I mean additions of names to public namespaces. > Twisted's own compatibility possibility would not forbid a similar > change. But in that specific case I think it was the right thing to > do. Like it or not, a lot of people use Twisted, a lot of people run > tests with Trial, and we beat stdlib unittest to the punch on the > 'skip' testing feature by a good 5 years. We caught the change well > before the release, we reported it and discussed it. Just to note that Twisted (along with Bazaar) are one of the few 'good citizens' in the Python community running their tests on Python trunk. Both projects have caught incompatibilities *before* release of new versions which is greatly preferable to discovering them after a release. Thanks for this. Michael Foordt > > IMHO this is the best way to deal with incompatible changes, > especially in the case of superclasses, given Python's subtle and > complex inheritance semantics. It's not feasible to provide a policy > that somehow prohibits subclasses from adding names which may > eventually be used on a superclass. > > Projects which notice test failures with new versions of Python should > report them so that the features can be adjusted to be compatible, > assuming the project in question hasn't done anything in egregious > violation of the compatibility policy (like invoking a private > method). This lets users, system administrators, and application > authors upgrade components individually, without worrying about the > components further down the stack flaking out on them. It also > provides a feedback loop for the compatibility policy: if there are > things that initially seem reasonable, but projects report > compatibility issues when they are changed, they might need to be > added later. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From fuzzyman at voidspace.org.uk Fri Jun 19 13:14:55 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 19 Jun 2009 12:14:55 +0100 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> Message-ID: <4A3B732F.5080808@voidspace.org.uk> Antoine Pitrou wrote: > divmod.com> writes: > >> This is a false dichotomy; for core developers, the list needs to be >> exhaustive. Everything that can change needs to be described as either >> compatible or incompatible. >> > > How do you enumerate "everything that can change"? It does not look like a > finite set to me (but perhaps I'm wrong); and certainly not like a set of a size > reasonable enough to be enumerated in a human-readable way :) > > And this is why expressing a finite list of things we guarantee won't change is a virtually impossible task - unless one of you is volunteering to write an official spec for Python and its libraries... :-) (Something that would not be bad IMO - just a long and difficult task, *especially* if you include the library along with language semantics and APIs). Michael > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From fuzzyman at voidspace.org.uk Fri Jun 19 13:16:30 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 19 Jun 2009 12:16:30 +0100 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <4A3B738E.3040309@voidspace.org.uk> Benjamin Peterson wrote: > Backwards compatibility seems to be an issue that arises a lot here. I > think we all have an idea of it is, but we need some hard place to > point to. So here's my attempt: > Should this PEP include a note about features which require new syntax, particularly new keywords? The policy (AFAICT) is that if new keywords are created they are enabled with a future import (with a warning?) in the version they are introduced and then enabled by default in the next version. All the best, Michael Foord > > PEP: 387 > Title: Backwards Compatibility Policy > Version: $Revision$ > Last-Modified: $Date$ > Author: Benjamin Peterson > Status: Draft > Type: Process > Content-Type: text/x-rst > Created: 18-Jun-2009 > > > Abstract > ======== > > This PEP outlines Python's backwards compatibility policy. > > > Rationale > ========= > > As one of the most used programming languages today [#tiobe]_, the Python core > language and its standard library play a critcal role in thousands of > applications and libraries. This is fantastic; it is probably one of a language > designer's most wishful dreams. However, it means the development team must be > very careful not to break this existing 3rd party code with new releases. > > > Backwards Compatibility Rules > ============================= > > This policy applys to all public APIs. These include the C-API, the standard > library, and the core language including syntax and operation as defined by the > reference manual. > > This is the basic policy for backwards compatibility: > > * The behavior of an API *must* not change between any two consecutive releases. > > * A feature cannot be removed without notice between any two consecutive > releases. > > * Addition of a feature which breaks 3rd party libraries or applications should > have a large benefit to breakage ratio, and/or the incompatibility should be > trival to fix in broken code. > > > Making Incompatible Changes > =========================== > > It's a fact: design mistakes happen. Thus it is important to be able to change > APIs or remove misguided features. This is accomplished through a gradual > process over several releases: > > 1. Discuss the change. Depending on the size of the incompatibility, this could > be on the bug tracker, python-dev, python-list, or the appropriate SIG. A > PEP or similar document may be written. Hopefully users of the affected API > will pipe up to comment. > > 2. Add a warning [#warnings_]_. If behavior is changing, a the API may gain a > new function or method to perform the new behavior; old usage should raise > the warning. If an API is being removed, simply warn whenever it is entered. > DeprecationWarning is the usual warning category to use, but > PendingDeprecationWarning may be used in special cases were the old and new > versions of the API will coexist for many releases. > > 3. Wait for a release. > > 4. See if there's any feedback. Users not involved in the original discussions > may comment now after seeing the warning. Perhaps reconsider. > > 5. The behavior change or feature removal may now be made default or permanent > in the next release. Remove the old version and warning. > > > References > ========== > > .. [#tiobe] TIOBE Programming Community Index > > http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html > > .. [#warnings] The warnings module > > http://docs.python.org/library/warnings.html > > > Copyright > ========= > > This document has been placed in the public domain. > > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From solipsis at pitrou.net Fri Jun 19 13:22:10 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 19 Jun 2009 11:22:10 +0000 (UTC) Subject: [Python-Dev] draft pep: backwards compatibility References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> <4A3B732F.5080808@voidspace.org.uk> Message-ID: Michael Foord voidspace.org.uk> writes: > > And this is why expressing a finite list of things we guarantee won't > change is a virtually impossible task - unless one of you is > volunteering to write an official spec for Python and its libraries... Well, we *can* enumerate a list of things that are guaranteed not to change - it's just that the list will be "completely incomplete", and the rest will rely on application of common sense rather than formal guarantees. And it actually more or less exists: it's Python's official documentation, combined with a bit of common sense when interpreting its written text. Regards Antoine. From lukepadawan at gmail.com Fri Jun 19 13:29:43 2009 From: lukepadawan at gmail.com (Lucas P Melo) Date: Fri, 19 Jun 2009 08:29:43 -0300 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A3AE0D3.2090708@canterbury.ac.nz> References: <20090617012719.GA19974@cskk.homeip.net> <4A399CEF.2070202@canterbury.ac.nz> <4A3A551A.6060304@gmail.com> <4A3AE0D3.2090708@canterbury.ac.nz> Message-ID: <4A3B76A7.7060701@gmail.com> Greg Ewing wrote: > That's exactly why I think the blocking version should > keep reading until the requested number of bytes is > available (or the buffer is full or EOF occurs). Do you mean that the blocking version should keep waiting for new bytes until they show up? This would not be acceptable, since the program would hang forever most of the time (no changes to the buffer would ever occur in this situation when there's only the main thread running). Am I understanding this correctly: * The blocking version would not do any raw reads. * The non-blocking version would do. From glyph at divmod.com Fri Jun 19 14:20:49 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 19 Jun 2009 12:20:49 -0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> Message-ID: <20090619122049.12555.1502244749.divmod.xquotient.12951@weber.divmod.com> On 11:11 am, solipsis at pitrou.net wrote: > divmod.com> writes: >> >>This is a false dichotomy; for core developers, the list needs to be >>exhaustive. Everything that can change needs to be described as >>either >>compatible or incompatible. > >How do you enumerate "everything that can change"? It does not look >like a >finite set to me (but perhaps I'm wrong); and certainly not like a set >of a size >reasonable enough to be enumerated in a human-readable way :) Sorry. You're right. To be clear, I mean, "everything that can change" as classified by some arbitrary classification system yet to be defined :-). For example, the number of different programmatic entities in Python is pretty small: you've got classes, methods, modules, and constants. The ways you can change them is also not too huge: you can add to them, remove them, or rename them. I realize that given Python's flexibility when loading code, any system of such classification is going to have edge cases, but I doubt those are going to matter that much. From glyph at divmod.com Fri Jun 19 14:25:34 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 19 Jun 2009 12:25:34 -0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <4A3B72E3.3070106@voidspace.org.uk> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <20090619112634.12555.1573592942.divmod.xquotient.12922@weber.divmod.com> <4A3B72E3.3070106@voidspace.org.uk> Message-ID: <20090619122534.12555.2014907760.divmod.xquotient.12955@weber.divmod.com> On 11:13 am, fuzzyman at voidspace.org.uk wrote: >Just to note that Twisted (along with Bazaar) are one of the few 'good >citizens' in the Python community running their tests on Python trunk. >Both projects have caught incompatibilities *before* release of new >versions which is greatly preferable to discovering them after a >release. Thanks for this. And thank *you* for being so responsive to the issues we've reported as a result of that. It's definitely an incentive to remain engaged in core development :). From benjamin at python.org Fri Jun 19 15:56:12 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 19 Jun 2009 08:56:12 -0500 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <1afaf6160906190656x2b2243cerbe6e85d75973b209@mail.gmail.com> 2009/6/19 Antoine Pitrou : > Benjamin Peterson python.org> writes: >> >> This policy applys to all public APIs. > > applies? Yes, thanks. > >> * The behavior of an API *must* not change between any two consecutive >> releases. >> >> * A feature cannot be removed without notice between any two consecutive >> ? releases. > > By induction, this would mean no API could change and no feature could be > removed without notice between any N consecutive releases. Do you really mean > it? No, I'll reword. > >> * Addition of a feature which breaks 3rd party libraries or applications >> should >> ? have a large benefit to breakage ratio, and/or the incompatibility should >> be >> ? trival to fix in broken code. > > There is always the possibility that a new feature breaks existing code, for > example because it relies on a similarly named attribute, or on some obscure > internal condition. I think this should be qualified so that it only applies > when e.g. a fair number of third-party apps or libraries are broken. I add a few examples. -- Regards, Benjamin From benjamin at python.org Fri Jun 19 16:09:44 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 19 Jun 2009 09:09:44 -0500 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> Message-ID: <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> 2009/6/19 : > > On 02:17 am, benjamin at python.org wrote: >> >> Backwards compatibility seems to be an issue that arises a lot here. I >> think we all have an idea of it is, but we need some hard place to >> point to. So here's my attempt: > >> PEP: 387 >> Title: Backwards Compatibility Policy > > Thanks for getting started on this. ?This is indeed a bikeshed that there > would be less discussion around if there were a clearer PEP to point to. > ?However, this draft PEP points to the problem rather than the solution. ?In > order to really be useful this needs to answer a whole slew of very very > specific questions, because the real problem is that everybody thinks they > know what they mean but in fact we all have slightly different definitions > of these words. I don't expect this document at all to be the definitive policy all at once. I expect it will evolve and be filled in as it is applied in real life. > > I've taken a stab at this myself in the past while defining a policy for > Twisted, which is here: > . ?I think we might > be a bit stricter than Python, but I believe our attempt to outline these > terms specifically is worth taking a look at. Yes, that's helpful. Thanks. > > The questions that follow might seem satirical or parodic but I am > completely serious and each of these terms really does have a variable > definition. And will always no matter what we do. It's how natural language works. >> >> * The behavior of an API *must* not change between any two consecutive >> releases. > > What is "behavior"? ?Software is composed of behavior. ?If no behavior > changes, then nothing can ever happen. I mean that if you pass X and Y into a function and get Z in 2.6, then you should be able to get Z from passing X and Y in 2.7 even if there's a new argument that returns Z' if you pass True to it. (Obviously, this is somewhat simplified, but I hope it helps.) >> >> * A feature cannot be removed without notice between any two consecutive >> ?releases. > > Presumably removal of a feature is a change in behavior, so isn't this > redundant with the previous point? I wanted to qualify differently from explicit behavior change as I defined above in this message. >> >> * Addition of a feature which breaks 3rd party libraries or applications >> should >> ?have a large benefit to breakage ratio, and/or the incompatibility should >> be >> ?trival to fix in broken code. > > How do you propose to measure the benefit to breakage ratio? ?How do you > propose to define "trivial" to fix? ?Most projects, including Python, > Twisted, Django, and Zope, have an ever-increasing bug count, which means > that trivial issues fall off the radar pretty easily. Well, if you're tests aren't failing from it, is it an incompatibility? > > One of the big problems here in detecting and fixing "trivial" changes is > that they can occur in test code as well. ?So if the answer is "run your > tests", that's not much help if you can't call the actual APIs whose > behavior has changed. ?The standard library would need to start providing a > lot more verified test stubs for things like I/O in order for this to be > feasible. >> >> Making Incompatible Changes >> =========================== >> >> It's a fact: design mistakes happen. ?Thus it is important to be able to >> change >> APIs or remove misguided features. ?This is accomplished through a gradual >> process over several releases: >> >> 1. Discuss the change. ?Depending on the size of the incompatibility, this >> could >> ?be on the bug tracker, python-dev, python-list, or the appropriate SIG. >> ?A >> ?PEP or similar document may be written. ?Hopefully users of the affected >> API >> ?will pipe up to comment. > > There are a very large number of users of Python, the large percentage of > whom do not read python-dev. ?A posting on python-list is likely to provoke > an unproductive pile-on. ?I suggest a dedicated list, which should ideally > be low traffic, "python-incompatible-announce", or something like that, so > that *everyone* who maintains Python software can subscribe to it, even if > they're not otherwise interested in Python development. And that won't generate a pile-on? >> ?DeprecationWarning is the usual warning category to use, but >> ?PendingDeprecationWarning may be used in special cases were the old and >> new >> ?versions of the API will coexist for many releases. > > Why is PendingDeprecationWarning the special case? It should be used for things like string exceptions which are deprecated but are not removed for a while. > > As outlined in the Twisted compatibility proposal, I'd prefer it if every > warning went through multiple phases: pending, deprecated, gone: so that > careful developers could release new versions of their software to quash > noisy warnings *before* the new version of Python that actually made them > user-visible was released. >> >> 3. Wait for a release. > > How does this affect the parallel 2.x/3.x release cycle? I just added something about this. >> >> 4. See if there's any feedback. ?Users not involved in the original >> discussions >> ?may comment now after seeing the warning. ?Perhaps reconsider. > > At this point, many developers may already have been porting over to the > new, non-deprecated API. ?I realize that there may be no way around that, > but it's worth considering. > > Thanks for tackling this thorny problem. ?Good luck! I think I'll need it. Thanks. :) -- Regards, Benjamin From solipsis at pitrou.net Fri Jun 19 16:15:23 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 19 Jun 2009 14:15:23 +0000 (UTC) Subject: [Python-Dev] draft pep: backwards compatibility References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > I mean that if you pass X and Y into a function and get Z in 2.6, then > you should be able to get Z from passing X and Y in 2.7 even if > there's a new argument that returns Z' if you pass True to it. Well, except if returning Z rather than Z' was a bug. Regards Antoine. From rdmurray at bitdance.com Fri Jun 19 16:30:20 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 19 Jun 2009 10:30:20 -0400 (EDT) Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> Message-ID: On Fri, 19 Jun 2009 at 14:15, Antoine Pitrou wrote: > Benjamin Peterson python.org> writes: >> >> I mean that if you pass X and Y into a function and get Z in 2.6, then >> you should be able to get Z from passing X and Y in 2.7 even if >> there's a new argument that returns Z' if you pass True to it. > > Well, except if returning Z rather than Z' was a bug. I'm pretty sure there have been cases of keeping buggy behavior in point releases for backward compatibility reasons. I think the decision has depended on the nature, severity, and age of the bug, and the estimated likelihood that code in the wild would break if the bug were fixed. --David From status at bugs.python.org Fri Jun 19 18:07:03 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 19 Jun 2009 18:07:03 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090619160703.F35CD782C1@psf.upfronthosting.co.za> ACTIVITY SUMMARY (06/12/09 - 06/19/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2239 open (+26) / 15895 closed (+14) / 18134 total (+40) Open issues with patches: 886 Average duration of open issues: 653 days. Median duration of open issues: 404 days. Open Issues Breakdown open 2210 (+25) pending 29 ( +1) Issues Created Or Reopened (40) _______________________________ Add client side certificate support to httplib 06/12/09 CLOSED http://bugs.python.org/issue6273 created jnoller patch subprocess.Popen() may leak file descriptors 06/12/09 http://bugs.python.org/issue6274 created facundobatista patch let unittest.assertRaises() return the exception object caught 06/14/09 http://bugs.python.org/issue6275 reopened krisvale patch, patch, easy, needs review Fix contextlib.nested DeprecationWarning for test_signal. 06/13/09 CLOSED http://bugs.python.org/issue6276 created vshenoy patch Add description of new syntax of with to 2.7 whatsnew document. 06/13/09 CLOSED http://bugs.python.org/issue6277 created vshenoy patch http.server, BaseHTTPRequestHandler write string error 06/13/09 CLOSED http://bugs.python.org/issue6278 created System32 datamodel documentation confuses staticmethod with classmethod 06/13/09 CLOSED http://bugs.python.org/issue6279 created segfaulthunter patch calendar.timegm() belongs in time module, next to time.gmtime() 06/13/09 http://bugs.python.org/issue6280 created zooko Bug in hashlib 06/13/09 http://bugs.python.org/issue6281 created Eloff In tarfile, compression level cannot be specified 06/14/09 CLOSED http://bugs.python.org/issue6282 created Tzigi Cannot build extension in amd64 using msvc9compiler 06/14/09 http://bugs.python.org/issue6283 created alexsh C/API PyErr_AsUnicode() 06/14/09 http://bugs.python.org/issue6284 created ideasman42 patch Silent abort on XP help document display 06/14/09 http://bugs.python.org/issue6285 created scott_daniels distutils upload command doesn't work with http proxy 06/15/09 http://bugs.python.org/issue6286 created tarek distutils.core.setup does not document the 'license' meta-data 06/15/09 CLOSED http://bugs.python.org/issue6287 created mattsmart Update contextlib.nested docs in light of deprecation 06/17/09 http://bugs.python.org/issue6288 reopened ncoghlan compile() raises SyntaxError with confusing message for some dec 06/15/09 CLOSED http://bugs.python.org/issue6289 created exarkun cPickle can misread data type 06/15/09 http://bugs.python.org/issue6290 created ac.james TypeError: b2a_base64() argument 1 must be bytes or buffer, not 06/16/09 CLOSED http://bugs.python.org/issue6291 created milivojm Fix tests to work with -OO 06/16/09 http://bugs.python.org/issue6292 created collinwinter patch, easy Have regrtest.py echo back sys.flags 06/16/09 http://bugs.python.org/issue6293 created collinwinter patch, easy Improve shutdown exception ignored message 06/17/09 http://bugs.python.org/issue6294 created tjreedy easy curses doc : getch is blocking by default 06/17/09 CLOSED http://bugs.python.org/issue6295 created nojhan Native (and default) tarfile support for setup.py sdist in distu 06/17/09 http://bugs.python.org/issue6296 created michael.foord Added Misc/python.pc to 'distclean' Rule 06/17/09 http://bugs.python.org/issue6297 created jcsalterego patch http://python.org/download says Python 2.4.5, but I think it mea 06/17/09 CLOSED http://bugs.python.org/issue6298 created zooko pyexpat build failure on Solaris 10 for 2.6.1/2.6.2 06/17/09 http://bugs.python.org/issue6299 created enchanter encode and decode should accept 'errors' as a keyword argument 06/18/09 http://bugs.python.org/issue6300 created r.david.murray easy Error in tutorial section 7.2 06/18/09 CLOSED http://bugs.python.org/issue6301 created melbourne email.header.decode_header data types are inconsistent and incor 06/18/09 http://bugs.python.org/issue6302 created r.david.murray print does not appear in the 3.x documentation index 06/18/09 CLOSED http://bugs.python.org/issue6303 created r.david.murray Confusing error message when passing bytes to print with file po 06/18/09 http://bugs.python.org/issue6304 created r.david.murray islice doesn't accept large stop values 06/18/09 http://bugs.python.org/issue6305 created thomasguest filecmp.cmp can not compare two files from different OS with the 06/18/09 http://bugs.python.org/issue6306 created higer AttributeError in traceback.print_last() 06/18/09 CLOSED http://bugs.python.org/issue6307 created complex termios fix for QNX breaks HP-UX 06/18/09 http://bugs.python.org/issue6308 created haubi Tix needs TCL package require statement 06/19/09 http://bugs.python.org/issue6309 created eantelman Windows "App Paths" key is not checked when installed for curren 06/19/09 http://bugs.python.org/issue6310 created techtonik virtual memory error with archivemail 06/19/09 http://bugs.python.org/issue6311 created helgekraak httplib fails with HEAD requests to pages with "transfer-encodin 06/19/09 http://bugs.python.org/issue6312 created ezio.melotti Issues Now Closed (29) ______________________ linecache .updatecache fails on utf8 encoded files 544 days http://bugs.python.org/issue1685 amaury.forgeotdarc patch, easy test_urllib2_localnet fails on MacOS X 10.4.11 (Intel) 332 days http://bugs.python.org/issue3407 orsenthil Ill-formed surrogates not treated as errors during encoding/deco 295 days http://bugs.python.org/issue3672 hippietrail patch Local variables not freed when Exception raises in function call 76 days http://bugs.python.org/issue5641 georg.brandl Unexpected universal newline behavior (newline duplication) in W 16 days http://bugs.python.org/issue6127 alexandre.vassalotti patch subprocess module not compatible with python 2.5 15 days http://bugs.python.org/issue6189 amaury.forgeotdarc patch sdist doesn't include data_files 10 days http://bugs.python.org/issue6205 purpleidea doctest_aliases doesn't test duplicate removal 8 days http://bugs.python.org/issue6227 amaury.forgeotdarc patch Python compiles dead code 6 days http://bugs.python.org/issue6250 loewis patch PyInt_FromSize_t is undocumented. 7 days http://bugs.python.org/issue6255 georg.brandl distributions built with bdist_msi on 64-bit Windows fail to ins 3 days http://bugs.python.org/issue6258 loewis mmap: don't close file description if fd=-1 2 days http://bugs.python.org/issue6271 ocean-city patch Add client side certificate support to httplib 0 days http://bugs.python.org/issue6273 jnoller patch Fix contextlib.nested DeprecationWarning for test_signal. 6 days http://bugs.python.org/issue6276 georg.brandl patch Add description of new syntax of with to 2.7 whatsnew document. 3 days http://bugs.python.org/issue6277 rhettinger patch http.server, BaseHTTPRequestHandler write string error 2 days http://bugs.python.org/issue6278 amaury.forgeotdarc datamodel documentation confuses staticmethod with classmethod 1 days http://bugs.python.org/issue6279 segfaulthunter patch In tarfile, compression level cannot be specified 1 days http://bugs.python.org/issue6282 Tzigi distutils.core.setup does not document the 'license' meta-data 0 days http://bugs.python.org/issue6287 tarek compile() raises SyntaxError with confusing message for some dec 0 days http://bugs.python.org/issue6289 benjamin.peterson TypeError: b2a_base64() argument 1 must be bytes or buffer, not 0 days http://bugs.python.org/issue6291 r.david.murray curses doc : getch is blocking by default 0 days http://bugs.python.org/issue6295 georg.brandl http://python.org/download says Python 2.4.5, but I think it mea 0 days http://bugs.python.org/issue6298 georg.brandl Error in tutorial section 7.2 1 days http://bugs.python.org/issue6301 pjenvey print does not appear in the 3.x documentation index 0 days http://bugs.python.org/issue6303 r.david.murray AttributeError in traceback.print_last() 0 days http://bugs.python.org/issue6307 r.david.murray current directory in sys.path handles symlinks badly 1665 days http://bugs.python.org/issue1074015 marko enhance unittest to define tests that *should* fail 1253 days http://bugs.python.org/issue1399935 benjamin.peterson patch keyword and topic help broken in Pythonwin IDE 1126 days http://bugs.python.org/issue1489051 georg.brandl Top Issues Most Discussed (10) ______________________________ 18 io.BufferedReader.peek(): Documentation differs from Implementa 58 days open http://bugs.python.org/issue5811 13 Backport the IO lib to trunk 14 days open http://bugs.python.org/issue6215 9 Add "daemon" argument to threading.Thread constructor 31 days open http://bugs.python.org/issue6064 8 Error in tutorial section 7.2 1 days closed http://bugs.python.org/issue6301 8 Update contextlib.nested docs in light of deprecation 2 days open http://bugs.python.org/issue6288 8 Python compiles dead code 6 days closed http://bugs.python.org/issue6250 7 Native (and default) tarfile support for setup.py sdist in dist 2 days open http://bugs.python.org/issue6296 6 Add description of new syntax of with to 2.7 whatsnew document. 3 days closed http://bugs.python.org/issue6277 6 let unittest.assertRaises() return the exception object caught 5 days open http://bugs.python.org/issue6275 6 path separator output ignores shell's path separator: / instead 14 days open http://bugs.python.org/issue6208 From tseaver at palladion.com Fri Jun 19 19:47:22 2009 From: tseaver at palladion.com (Tres Seaver) Date: Fri, 19 Jun 2009 13:47:22 -0400 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Antoine Pitrou wrote: > There is always the possibility that a new feature breaks existing code, for > example because it relies on a similarly named attribute, or on some obscure > internal condition. I think this should be qualified so that it only applies > when e.g. a fair number of third-party apps or libraries are broken. I think the recent asyncore changes are a good test case here: they broke major consumers of the package (Zope, supervisord), but it wasn't obvious that the breakage would occur, because the API of the package wasn't clear: the apps broken by the change were forced to rely on stuff that the subsequent maintainer considered "implementation details." Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKO88q+gerLs4ltQ4RAmmkAJ4ghgszBWGruCqONtouAXp82G+blgCgySsL 4ywgpU5137D5isiJ8+d6KtM= =OksB -----END PGP SIGNATURE----- From glyph at divmod.com Fri Jun 19 20:44:28 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 19 Jun 2009 18:44:28 -0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> Message-ID: <20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com> On 02:09 pm, benjamin at python.org wrote: >2009/6/19 : >> >>On 02:17 am, benjamin at python.org wrote: >>I've taken a stab at this myself in the past while defining a policy >>for >>Twisted >Yes, that's helpful. Thanks. Glad you found it useful. >>The questions that follow might seem satirical or parodic but I am >>completely serious and each of these terms really does have a variable >>definition. > >And will always no matter what we do. It's how natural language works. Well, one cannot proceed from the informal to the formal by formal means. I'm pretty sure we can nail down the definitions of these terms for the scope of Python core development. >>What is "behavior"? ?Software is composed of behavior. ?If no behavior >>changes, then nothing can ever happen. >I mean that if you pass X and Y into a function and get Z in 2.6, then >you should be able to get Z from passing X and Y in 2.7 even if >there's a new argument that returns Z' if you pass True to it. >(Obviously, this is somewhat simplified, but I hope it helps.) This definition only makes sense if the interesting thing about a function is its return value, and if the only sort of thing you have are functions. What about side-effects, or exceptional conditions? What about interactions with subclasses? (Changing a class in a library from old-style to new-style has serious repercussions, as MRO conflicts can result in applications that subclass it.) >>How do you propose to measure the benefit to breakage ratio? ?How do >>you >>propose to define "trivial" to fix? ?Most projects, including Python, >>Twisted, Django, and Zope, have an ever-increasing bug count, which >>means >>that trivial issues fall off the radar pretty easily. > >Well, if you're tests aren't failing from it, is it an incompatibility? Well, let's say the tests do fail from it. Right now, even Twisted trunk still doesn't *quite* support Python 2.6, but only on Windows, due to stricter checking of the 'mode' argument for files. But this failing test is just not that high priority, so our recommendation is "don't use python 2.6 yet on Windows, 2.5 works fine". My point is that triviality is a matter of perspective :). Eventually somebody will get around to it, but 2.6 has been out for a while now. >>There are a very large number of users of Python, the large percentage >>of >>whom do not read python-dev. ?A posting on python-list is likely to >>provoke >>an unproductive pile-on. ?I suggest a dedicated list, which should >>ideally >>be low traffic, "python-incompatible-announce", or something like >>that, so >>that *everyone* who maintains Python software can subscribe to it, >>even if >>they're not otherwise interested in Python development. > >And that won't generate a pile-on? Well, the etiquette for that specific list could be "keep this low- traffic unless you have a serious problem with this change". Or, better yet, make it an announce-only list: the pile-on can still happen on python-list, but only the results of the discussion will be announced on the incompatible-announce list. The point is, the notifications about compatibility are really important, and sometimes people need to offer feedback about them, but not everyone who needs to know about the changes needs to hear about the feedback. From python at rcn.com Fri Jun 19 21:06:27 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 19 Jun 2009 12:06:27 -0700 Subject: [Python-Dev] draft pep: backwards compatibility References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1> Not sure why we need yet another pep on the subject. Just update PEP 5 if needed. Also, I think there is a certain amount of wishful thinking here. Ideally, we could approve a tiny PEP with just a few bullet points and it would eliminate the need for all of the complicated decision making that we usually go through. Ideally, we could make all decisions in advance of seeing the facts for a given situation. ISTM, this pep is wishing away the actual complexity of making software design decisions. The policy for 2.x should probably be different than for 3.x. ISTM that 3.x has not been fully shaken out and that possibly many things will need to change as users start to report problems. The text vs bytes issue is lurking throughout the release. The JSON module in particular was affected by a half thought out port to 3.0. And yesterday on #python IRC, one developer reported the email package in 3.1 to be unusable and one of its maintainers characterized it as being in need of a serious overhaul (meaning major API changes). In the end, there is going to have to be some thoughtful balancing between making the needed changes and not hurting the existing users. I don't think a small, general purpose PEP like this one can wish that away. Another sticking point about preserving features across releases arises if the feature itself is hazardous in some way (like have a security hole or some such). The contextlib.nested() function was an example. It didn't ever really work as advertised for its intended purpose (it wasn't truly equivalent to two nested with-statements) and it presented users with the possibility of hard-to-spot bugs. The bugfix for it was to replace it with new syntax. Unfortunately, the new syntax didn't provide all of the functionality of the original. So, the question arises about whether this particular mine should be left on the battlefield. We resolved the question after a long and thoughtful discussion; I don't think that decision making process could have been solved in advance by a bullet-point in a general purpose process PEP. Raymond From benjamin at python.org Fri Jun 19 21:37:45 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 19 Jun 2009 14:37:45 -0500 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1> Message-ID: <1afaf6160906191237q67860926m8c1d3a8fa441d8e2@mail.gmail.com> 2009/6/19 Raymond Hettinger : > Not sure why we need yet another pep on the subject. ?Just update PEP 5 if > needed. Hmm. I didn't know about that one. > > Also, I think there is a certain amount of wishful thinking here. ?Ideally, > we could approve a tiny PEP with just a few bullet points and it would > eliminate the need for all of the complicated decision making that we > usually go through. ?Ideally, we could make all decisions in advance of > seeing the facts for a given situation. ?ISTM, this pep is wishing away the > actual complexity of making software design decisions. I think it should just serve as a backbone and basis for decisions. It will certainly not eliminate any complex thinking, but hopefully it will More importantly, it shows to the community what our policies are and what they should expect. Exceptions should be exceptional not the norm. > > The policy for 2.x should probably be different than for 3.x. ?ISTM that 3.x > has not been fully shaken out and that possibly many things will need to > change as users start to report problems. ?The text vs bytes issue is > lurking throughout the release. ?The JSON module in particular was affected > by a half thought out port to 3.0. ?And yesterday on #python IRC, one > developer reported the email package in 3.1 to be unusable and one of its > maintainers characterized it as being in need of a serious overhaul (meaning > major API changes). ?In the end, there is going to have to be some > thoughtful balancing between making the needed changes and not hurting the > existing users. ?I don't think a small, general purpose PEP like this one > can wish that away. That is very unfortunate. 2.7 may be the last 2.x release, so this PEP primarily applies to 3.x anyway. We could add a provision for experimental or unstable modules, but I don't think those should be released in the first place. > > Another sticking point about preserving features across releases arises if > the feature itself is hazardous in some way (like have a security hole or > some such). ?The contextlib.nested() function was an example. ?It didn't > ever really work as advertised for its intended purpose (it wasn't truly > equivalent to two nested with-statements) and it presented users with the > possibility of hard-to-spot bugs. ?The bugfix for it was to replace it with > new syntax. ?Unfortunately, the new syntax didn't provide all of the > functionality of the original. ?So, the question arises about whether this > particular mine should be left on the battlefield. ?We resolved the question > after a long and thoughtful discussion; I don't think that decision making > process could have been solved in advance by a bullet-point in a general > purpose process PEP. I would say that's pretty close to the procedure in the PEP actually: "Discuss the change. This may be on python-dev, the tracker...." -- Regards, Benjamin From g.brandl at gmx.net Fri Jun 19 22:13:12 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 19 Jun 2009 20:13:12 +0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > Backwards compatibility seems to be an issue that arises a lot here. I > think we all have an idea of it is, but we need some hard place to > point to. So here's my attempt: Yet another rather pointless bikeshed: if this becomes policy, maybe it should get a PEP number < 100, like PEP 5 or 6. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From benjamin at python.org Fri Jun 19 22:40:54 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 19 Jun 2009 15:40:54 -0500 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <1afaf6160906191340pbedf3abn2bccb185bdddf15b@mail.gmail.com> 2009/6/19 Georg Brandl : > Benjamin Peterson schrieb: >> Backwards compatibility seems to be an issue that arises a lot here. I >> think we all have an idea of it is, but we need some hard place to >> point to. So here's my attempt: > > Yet another rather pointless bikeshed: if this becomes policy, maybe it > should get a PEP number < 100, like PEP 5 or 6. The point is to avoid a little bike shedding later. -- Regards, Benjamin From g.brandl at gmx.net Fri Jun 19 22:58:04 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 19 Jun 2009 22:58:04 +0200 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> Message-ID: R. David Murray schrieb: > On Fri, 19 Jun 2009 at 14:15, Antoine Pitrou wrote: > >> Benjamin Peterson python.org> writes: >>> >>> I mean that if you pass X and Y into a function and get Z in 2.6, then >>> you should be able to get Z from passing X and Y in 2.7 even if >>> there's a new argument that returns Z' if you pass True to it. >> >> Well, except if returning Z rather than Z' was a bug. > > I'm pretty sure there have been cases of keeping buggy behavior in point > releases for backward compatibility reasons. I think the decision has > depended on the nature, severity, and age of the bug, and the estimated > likelihood that code in the wild would break if the bug were fixed. That is always a difficult issue. There are tons of issues in the tracker that would be quite easy to fix, but are not touched because nobody wants to take the blame if it is considered "not buggy enough" for an incompatible change. But they won't be closed either, because the current behavior clearly is wrong. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From benjamin at python.org Fri Jun 19 23:08:24 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 19 Jun 2009 16:08:24 -0500 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> <20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com> Message-ID: <1afaf6160906191408i49b346eegc9b40fae0cf69576@mail.gmail.com> 2009/6/19 : > On 02:09 pm, benjamin at python.org wrote: >> 2009/6/19 ?: >>> What is "behavior"? ?Software is composed of behavior. ?If no behavior >>> changes, then nothing can ever happen. > >> I mean that if you pass X and Y into a function and get Z in 2.6, then >> you should be able to get Z from passing X and Y in 2.7 even if >> there's a new argument that returns Z' if you pass True to it. >> (Obviously, this is somewhat simplified, but I hope it helps.) > > This definition only makes sense if the interesting thing about a function > is its return value, and if the only sort of thing you have are functions. > ?What about side-effects, or exceptional conditions? ?What about > interactions with subclasses? ?(Changing a class in a library from old-style > to new-style has serious repercussions, as MRO conflicts can result in > applications that subclass it.) I've added a little more about this to the PEP. See what you think. >>> >>> How do you propose to measure the benefit to breakage ratio? ?How do you >>> propose to define "trivial" to fix? ?Most projects, including Python, >>> Twisted, Django, and Zope, have an ever-increasing bug count, which means >>> that trivial issues fall off the radar pretty easily. >> >> Well, if you're tests aren't failing from it, is it an incompatibility? > > Well, let's say the tests do fail from it. ?Right now, even Twisted trunk > still doesn't *quite* support Python 2.6, but only on Windows, due to > stricter checking of the 'mode' argument for files. ?But this failing test > is just not that high priority, so our recommendation is "don't use python > 2.6 yet on Windows, 2.5 works fine". ?My point is that triviality is a > matter of perspective :). ?Eventually somebody will get around to it, but > 2.6 has been out for a while now. I understand. I think we will just have to provide guidelines for triviality and decide on a case by case basis. >>> There are a very large number of users of Python, the large percentage of >>> whom do not read python-dev. ?A posting on python-list is likely to >>> provoke >>> an unproductive pile-on. ?I suggest a dedicated list, which should >>> ideally >>> be low traffic, "python-incompatible-announce", or something like that, >>> so >>> that *everyone* who maintains Python software can subscribe to it, even >>> if >>> they're not otherwise interested in Python development. >> >> And that won't generate a pile-on? > > Well, the etiquette for that specific list could be "keep this low- traffic > unless you have a serious problem with this change". ?Or, better yet, make > it an announce-only list: the pile-on can still happen on python-list, but > only the results of the discussion will be announced on the > incompatible-announce list. I think that's a fine idea, but the PEP is dealing with policy as our mailing list infrastructure is now. -- Regards, Benjamin From aahz at pythoncraft.com Sat Jun 20 00:07:07 2009 From: aahz at pythoncraft.com (Aahz) Date: Fri, 19 Jun 2009 15:07:07 -0700 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <20090619220707.GA4300@panix.com> On Fri, Jun 19, 2009, Georg Brandl wrote: > > Yet another rather pointless bikeshed: if this becomes policy, maybe it > should get a PEP number < 100, like PEP 5 or 6. +1 -- I was debating whether to say something. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha From benjamin at python.org Sat Jun 20 00:15:31 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 19 Jun 2009 17:15:31 -0500 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <20090619220707.GA4300@panix.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619220707.GA4300@panix.com> Message-ID: <1afaf6160906191515x6b144085n64422aad18eade84@mail.gmail.com> 2009/6/19 Aahz : > On Fri, Jun 19, 2009, Georg Brandl wrote: >> >> Yet another rather pointless bikeshed: if this becomes policy, maybe it >> should get a PEP number < 100, like PEP 5 or 6. > > +1 -- I was debating whether to say something. Is that a +1 to numbering or bike shedding? -- Regards, Benjamin From glyph at divmod.com Sat Jun 20 00:40:36 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Fri, 19 Jun 2009 22:40:36 -0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1> Message-ID: <20090619224036.12555.889474637.divmod.xquotient.13046@weber.divmod.com> On 07:06 pm, python at rcn.com wrote: >Not sure why we need yet another pep on the subject. Just update PEP 5 >if needed. Hmm. This is a good point; it might make sense to have a more specific PEP for library compatibility as opposed to language compatibility though, since language compatibility is necessarily a vaguer concept. >Also, I think there is a certain amount of wishful thinking here. >Ideally, we could approve a tiny PEP with just a few bullet points and >it would eliminate the need for all of the complicated decision making >that we usually go through. Ideally, we could make all decisions in >advance of seeing the facts for a given situation. ISTM, this pep is >wishing away the actual complexity of making software design decisions. This is not about making design decisions. This is about how to treat the *output* of design decisions. A really important part of this PEP, as I alluded to previously, is the part that tells developers what *they* should be doing if they want their python code to function on the next release of the interpreter. Right now, the rule to write software that will not break with the next Python release is "read the minds of the python core committers and hope that you do not do anything with the stdlib that they don't like". Along with this there are several rules you can infer that are probably true most of the time: don't call stuff starting with "_", don't monkey- patch anything, don't use dynamic class replacement on objects from classes other than your own, don't depend on the depth of inheritance hierarchies (for example, no ".__bases__[0].__bases__[0]"), make sure your tests run without producing any DeprecationWarnings, be mindful of potential namespace conflicts when adding attributes to classes that inherit from other libraries. I don't think all these things are written down in one place though. >Another sticking point about preserving features across releases arises >if the feature itself is hazardous in some way (like have a security >hole or some such). The contextlib.nested() function was an example. >(...) >We resolved the question after a long and thoughtful discussion; I >don't think that decision making process could have been solved in >advance by a bullet-point in a general purpose process PEP. You are correct that nothing in Python's policy could have dictated how this problem could be resolved. However, the policy can most definitely state how to deal with code using contextlib.nested in the interim before it has been changed to use the new syntax: to wit, that contextlib.nested has to stick around, continue to do the (broken) thing that it did before, and emit a DeprecationWarning indicating the new syntax. The existing policy in PEP 5 already does this, but doesn't offer guidelines on *how* to do this for lots of different types of changes. For example, how do you issue a deprecation warning for a new parameter you want to require application code to accept? How do you change the name of a parameter, to wit, do you need to expect that all arguments can validly be used as keyword arguments? How do you determine an appropriate stack-depth, etc? From aahz at pythoncraft.com Sat Jun 20 00:26:30 2009 From: aahz at pythoncraft.com (Aahz) Date: Fri, 19 Jun 2009 15:26:30 -0700 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906191515x6b144085n64422aad18eade84@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619220707.GA4300@panix.com> <1afaf6160906191515x6b144085n64422aad18eade84@mail.gmail.com> Message-ID: <20090619222630.GA14341@panix.com> On Fri, Jun 19, 2009, Benjamin Peterson wrote: > 2009/6/19 Aahz : >> On Fri, Jun 19, 2009, Georg Brandl wrote: >>> >>> Yet another rather pointless bikeshed: if this becomes policy, maybe it >>> should get a PEP number < 100, like PEP 5 or 6. >> >> +1 -- I was debating whether to say something. > > Is that a +1 to numbering or bike shedding? +1 to changing the PEP number -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha From martin at v.loewis.de Sat Jun 20 18:27:22 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 20 Jun 2009 18:27:22 +0200 Subject: [Python-Dev] xmlrpc improvements In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> Message-ID: <4A3D0DEA.50002@v.loewis.de> > I?d really like to get this stuff in. The performance gains allowing > http1.1 and gzip for xmlrpc are quite significant. I think you really need to get Fredrik Lundh to comment on it. If he can't predict when he'll be able to review the changes, maybe he can accept releasing control of xmlrpclib. Regards, Martin From fredrik at pythonware.com Sat Jun 20 18:34:31 2009 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 20 Jun 2009 18:34:31 +0200 Subject: [Python-Dev] xmlrpc improvements In-Reply-To: <4A3D0DEA.50002@v.loewis.de> References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> <4A3D0DEA.50002@v.loewis.de> Message-ID: <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> 2009/6/20 "Martin v. L?wis" : >> I?d really ?like to get this stuff in. ?The performance gains allowing >> http1.1 and gzip for xmlrpc are quite significant. > > I think you really need to get Fredrik Lundh to comment on it. If he > can't predict when he'll be able to review the changes, maybe he can > accept releasing control of xmlrpclib. Pointer to the patch? From martin at v.loewis.de Sat Jun 20 18:57:20 2009 From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 20 Jun 2009 18:57:20 +0200 Subject: [Python-Dev] xmlrpc improvements In-Reply-To: <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> <4A3D0DEA.50002@v.loewis.de> <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> Message-ID: <4A3D14F0.7080100@v.loewis.de> Fredrik Lundh wrote: >> I think you really need to get Fredrik Lundh to comment on it. If he >> can't predict when he'll be able to review the changes, maybe he can >> accept releasing control of xmlrpclib. > > Pointer to the patch? http://bugs.python.org/issue6267 While I have your attention, please also review http://bugs.python.org/issue6233 Regards, Martin From glyph at divmod.com Sat Jun 20 19:55:28 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Sat, 20 Jun 2009 17:55:28 -0000 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906191408i49b346eegc9b40fae0cf69576@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <20090619093130.12555.864027812.divmod.xquotient.12879@weber.divmod.com> <1afaf6160906190709j260c1cc5l4b3e5cce4212742a@mail.gmail.com> <20090619184428.12555.1883394051.divmod.xquotient.12994@weber.divmod.com> <1afaf6160906191408i49b346eegc9b40fae0cf69576@mail.gmail.com> Message-ID: <20090620175528.12555.830517954.divmod.xquotient.13310@weber.divmod.com> On 19 Jun, 09:08 pm, benjamin at python.org wrote: >2009/6/19 : >>On 02:09 pm, benjamin at python.org wrote: >>>2009/6/19 ?: >>?What about side-effects, or exceptional conditions? ?What about >>interactions with subclasses? ?(Changing a class in a library from >>old-style >>to new-style has serious repercussions, as MRO conflicts can result in >>applications that subclass it.) >I've added a little more about this to the PEP. See what you think. Finally had a chance to take a look. It's a big improvement, certainly: much more specific. Although I think I have a few quibbles with the wording. For one thing, I don't like the use of the word "reasonable". Everybody thinks *their* exception to the rules is reasonable, but nobody else's is. Besides, if the BDFL thinks a change is really reasonable, do you think a PEP is going to stop him? :) For another, I think it's actually a bit too strict, as worded. The side-effects of a function includes the warnings that it emits; any method-call can have side-effects, so you can't change an algorithm *at all*. The only side-effects that I think are important are the ones that get invoked on objects that an application gets to inject somewhere, or inspect later via a public API. The word "releases" also needs to be qualified. Most importantly, minor-version and micro-version releases need to be described distinctly. Finally, the PEP should mention its participation in the universe of compatibility process PEPs. It should describe its relationship to at least some of PEP 3002, 291, 5, 6, 2, 3001, and 384. >>My point is that triviality is a matter of perspective :). >I understand. I think we will just have to provide guidelines for >triviality and decide on a case by case basis. A simple litmus test, or some examples, of triviality would be helpful. >>the pile-on can still happen on python-list, but >>only the results of the discussion will be announced on the >>incompatible-announce list. > >I think that's a fine idea, but the PEP is dealing with policy as our >mailing list infrastructure is now. Hmm... well, maybe everybody should just run their tests against Python trunk. The commits list is a reliable notification mechanism for potentially incompatible changes ;-). Perhaps it would be good to mention this specifically in the PEP? For example, "third party projects may request that an incompatible change be reverted, by providing evidence of tests failing on x.y+1 that passed on x.y, where the code in question does not rely on any details not specified as 'public' in the section above"? If more projects did this when there *was* a problem, then it would actually be a lot easier to break the policy with confidence. With an incompatible change, you could know, "we checked it in, and nobody complained". Whereas right now is nobody complains it's more likely that nobody is paying attention. From collinw at gmail.com Sat Jun 20 19:49:59 2009 From: collinw at gmail.com (Collin Winter) Date: Sat, 20 Jun 2009 10:49:59 -0700 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> Message-ID: <43aa6ff70906201049q26add809sa5893c470cc520d6@mail.gmail.com> On Thu, Jun 18, 2009 at 7:17 PM, Benjamin Peterson wrote: [snip] > Backwards Compatibility Rules > ============================= > > This policy applys to all public APIs. ?These include the C-API, the standard > library, and the core language including syntax and operation as defined by the > reference manual. > > This is the basic policy for backwards compatibility: > > * The behavior of an API *must* not change between any two consecutive releases. Is this intended to include performance changes? Clearly no-one will complain if things simply get faster, but I'm thinking about cases where, say, a function runs in half the time but uses double the memory (or vice versa). Collin From tjreedy at udel.edu Sat Jun 20 21:02:38 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 20 Jun 2009 15:02:38 -0400 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <20090619224036.12555.889474637.divmod.xquotient.13046@weber.divmod.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <8004914DB3B9496BB3DC1A1BFAF4D572@RaymondLaptop1> <20090619224036.12555.889474637.divmod.xquotient.13046@weber.divmod.com> Message-ID: glyph at divmod.com wrote: > On 07:06 pm, python at rcn.com wrote: >> Not sure why we need yet another pep on the subject. Just update PEP >> 5 if needed. I agree. The draft covers the same ground. Two PEPs on the same subject would be redundant where they agree but would create confusion where they do not. To the extent that the new PEP intends to change existing policy by severely curtailing language change, as it appears to, then that *idea* should be directly presented and discussed, perhaps on python-list, before worrying about wording (bikeshed) details. In other words, I think the discussion should have start out "Here is existing policy (PEP 5). I propose to change it like so..." or possibly "Here is existing policy in PEP 5. I believe it has defacto changed as evidenced by ... " In other words, discuss and decide whether the bikeshed should be re-painted before worrying about the exact shade. > Right now, the rule to write software that will not break with the next > Python release is "read the minds of the python core committers and hope > that you do not do anything with the stdlib that they don't like". A bit harsh ;-) > Along > with this there are several rules you can infer that are probably true > most of the time: don't call stuff starting with "_", don't monkey- > patch anything, don't use dynamic class replacement on objects from > classes other than your own, don't depend on the depth of inheritance > hierarchies (for example, no ".__bases__[0].__bases__[0]"), make sure > your tests run without producing any DeprecationWarnings, be mindful of > potential namespace conflicts when adding attributes to classes that > inherit from other libraries. I don't think all these things are > written down in one place though. This could be the core of a new PEP Keeeping up with Language Changes. I think that would be a good thing. Terry Jan Reedy From greg.ewing at canterbury.ac.nz Sun Jun 21 03:26:36 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 21 Jun 2009 13:26:36 +1200 Subject: [Python-Dev] io.BufferedReader.peek() Behaviour in python3.1 In-Reply-To: <4A3B76A7.7060701@gmail.com> References: <20090617012719.GA19974@cskk.homeip.net> <4A399CEF.2070202@canterbury.ac.nz> <4A3A551A.6060304@gmail.com> <4A3AE0D3.2090708@canterbury.ac.nz> <4A3B76A7.7060701@gmail.com> Message-ID: <4A3D8C4C.1030602@canterbury.ac.nz> Lucas P Melo wrote: > Am I understanding this correctly: > * The blocking version would not do any raw reads. No, the blocking version would keep doing raw reads until the buffer contains enough bytes. -- Greg From j at 3rdengine.com Sun Jun 21 19:36:40 2009 From: j at 3rdengine.com (Jerry Chen) Date: Sun, 21 Jun 2009 12:36:40 -0500 Subject: [Python-Dev] Binary Operator for New-Style String Formatting Message-ID: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> Hello all, For better or for worse, I have created a patch against the py3k trunk which introduces a binary operator '@' as an alternative syntax for the new string formatting system introduced by PEP 3101 ("Advanced String Formatting"). [1] For common cases, this syntax should be as simple and as elegant as its deprecated [2] predecessor ('%'), while also ensuring that more complex use cases do not suffer needlessly. I would just like to know whether this idea will float before submitting the patch on Roundup and going through the formal PEP process. This is my first foray into the internals of the Python core, and with any luck, I did not overlook any BDFL proclamations banning all new binary operators for string formatting. :-) QUICK EXAMPLES >>> "{} {} {}" @ (1, 2, 3) '1 2 3' >>> "foo {qux} baz" @ {"qux": "bar"} 'foo bar baz' One of the main complaints of a binary operator in PEP 3101 was the inability to mix named and unnamed arguments: The current practice is to use either a dictionary or a tuple as the second argument, but as many people have commented ... this lacks flexibility. To address this, a convention of having the last element of a tuple as the named arguments dictionary is introduced. >>> "{} {qux} {}" @ (1, 3, {"qux": "bar"}) '1 bar 3' Lastly, to print the repr() of a dictionary as an unnamed argument, one would have to append an additional dictionary so there is no ambiguity: >>> "{}" @ {"foo": "bar"} Traceback (most recent call last): File "", line 1, in IndexError: tuple index out of range >>> "{}" @ ({"foo": "bar"}, {}) "{'foo': 'bar'}" Admittedly, these workarounds are less than clean, but the understanding is the '@' syntax would indeed be an alternative, so one could easily fall back to the str.format() method or the format() function. IMPLEMENTATION Code-wise, the grammar was edited per PEP 306 [3], and a function was introduced in unicodeobject.c as PyUnicode_FormatPrime (in the mathematical sense of A and A' -- I didn't fully understand or want to intrude upon the *_FormatAdvanced namespace). The PyUnicode_FormatPrime function transforms the incoming arguments, i.e. the operands of the binary '@', and makes the appropriate do_string_format() call. Thus, I have reused as much code as possible. I have done my development with git by using two branches: 'master' and 'subversion', the latter of which can be used to run 'svn update' and merge back into master. This way my code changes and the official ones going into the Subversion repository can stay separate, meanwhile allowing 'svn diff' to produce an accurate patch at any given time. The code is available at: http://github.com/jcsalterego/py3k-atsign/ The SVN patch [4] or related commit [5] are good starting points. References: [1] http://www.python.org/dev/peps/pep-3101 [2] http://docs.python.org/3.0/whatsnew/3.0.html [3] http://www.python.org/dev/peps/pep-0306/ [4] http://github.com/jcsalterego/py3k-atsign/blob/master/py3k-atsign.diff [5] http://github.com/jcsalterego/py3k-atsign/commit/5c8bdf72d9252cea78af2b7809613f6530e25db4 Thanks, -- Jerry Chen From steven.bethard at gmail.com Sun Jun 21 20:08:42 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 21 Jun 2009 14:08:42 -0400 Subject: [Python-Dev] Binary Operator for New-Style String Formatting In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> Message-ID: On Sun, Jun 21, 2009 at 1:36 PM, Jerry Chen wrote: > QUICK EXAMPLES > > ? ?>>> "{} {} {}" @ (1, 2, 3) > ? ?'1 2 3' > > ? ?>>> "foo {qux} baz" @ {"qux": "bar"} > ? ?'foo bar baz' > > One of the main complaints of a binary operator in PEP 3101 was the > inability to mix named and unnamed arguments: > > ? ?The current practice is to use either a dictionary or a tuple as > ? ?the second argument, but as many people have commented ... this > ? ?lacks flexibility. The other reason an operator was a pain is the order of operations: >>> '{0}'.format(1 + 2) '3' >>> '%s' % 1 + 2 Traceback (most recent call last): File "", line 1, in TypeError: cannot concatenate 'str' and 'int' objects In general, I don't see any gain in introducing an operator for string formatting. What's the point? Maybe you save a few characters of typing, but it sure is easier to Google for "Python string format" than for "Python @". A big -1 from me. Steve -- Where did you get the preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From solipsis at pitrou.net Sun Jun 21 20:45:20 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 21 Jun 2009 18:45:20 +0000 (UTC) Subject: [Python-Dev] Binary Operator for New-Style String Formatting References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> Message-ID: Hello, > For better or for worse, I have created a patch against the py3k trunk > which introduces a binary operator '@' as an alternative syntax for > the new string formatting system introduced by PEP 3101 ("Advanced > String Formatting"). [1] While many people find the new format() tedious to adapt to, I don't think adding a third formatting syntax will help us. Especially given this annoyance: > Lastly, to print the repr() of a dictionary as an unnamed argument, > one would have to append an additional dictionary so there is no > ambiguity: > > >>> "{}" @ {"foo": "bar"} > Traceback (most recent call last): > File "", line 1, in > IndexError: tuple index out of range > > >>> "{}" @ ({"foo": "bar"}, {}) > "{'foo': 'bar'}" Regards Antoine. From rdmurray at bitdance.com Sun Jun 21 20:50:40 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 21 Jun 2009 14:50:40 -0400 (EDT) Subject: [Python-Dev] Binary Operator for New-Style String Formatting In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> Message-ID: On Sun, 21 Jun 2009 at 12:36, Jerry Chen wrote: > For better or for worse, I have created a patch against the py3k trunk > which introduces a binary operator '@' as an alternative syntax for > the new string formatting system introduced by PEP 3101 ("Advanced > String Formatting"). [1] It seems to me that this topic is more appropriate for python-ideas. That said, I'm -1 on it. The 'keywords as last item of tuple' reeks of code-smell to my nose, and I don't think you've addressed all of the reasons for why a method was chosen over an operator. Python has a tradition of having "one obvious way" to do something, so introducing an "alternative" syntax that you admit is sub-optimal does not seem to me to have enough benefit to justify breaking that design guideline. Congratulations on your first foray into the core, though :) --David From tjreedy at udel.edu Sun Jun 21 21:12:16 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 21 Jun 2009 15:12:16 -0400 Subject: [Python-Dev] Binary Operator for New-Style String Formatting In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> Message-ID: Jerry Chen wrote: > Hello all, > > For better or for worse, I have created a patch against the py3k trunk > which introduces a binary operator '@' as an alternative syntax for > the new string formatting system introduced by PEP 3101 ("Advanced > String Formatting"). [1] > > For common cases, this syntax should be as simple and as elegant as > its deprecated [2] predecessor ('%'), while also ensuring that more > complex use cases do not suffer needlessly. > > I would just like to know whether this idea will float before The place to float trial balloons is the python-ideas list. > submitting the patch on Roundup and going through the formal PEP > process. This is my first foray into the internals of the Python Even if this particular idea in not accepted, I hope you learned from and enjoyed the exercise and will try other forays. > core, and with any luck, I did not overlook any BDFL proclamations > banning all new binary operators for string formatting. :-) > > QUICK EXAMPLES > > >>> "{} {} {}" @ (1, 2, 3) The only advantage '@' over '.format' is fewer characters. I think it would be more useful to agitate to give 'format' a one char synonym such as 'f'. One disadvantage of using an actual tuple rather than an arg quasi-tuple is that people would have to remember the trailing comma when printing one thing. '{}' @ (1,) rather than '{}' @ (a) == '{}' @ a. [If you say, 'Oh, then accept the latter', then there is a problem when a is a tuple!] > '1 2 3' > > >>> "foo {qux} baz" @ {"qux": "bar"} > 'foo bar baz' > > One of the main complaints of a binary operator in PEP 3101 was the > inability to mix named and unnamed arguments: > > The current practice is to use either a dictionary or a tuple as > the second argument, but as many people have commented ... this > lacks flexibility. > > To address this, a convention of having the last element of a tuple > as the named arguments dictionary is introduced. > > >>> "{} {qux} {}" @ (1, 3, {"qux": "bar"}) > '1 bar 3' > > Lastly, to print the repr() of a dictionary as an unnamed argument, > one would have to append an additional dictionary so there is no > ambiguity: > > >>> "{}" @ {"foo": "bar"} > Traceback (most recent call last): > File "", line 1, in > IndexError: tuple index out of range > > >>> "{}" @ ({"foo": "bar"}, {}) > "{'foo': 'bar'}" This is another disadvantage -- to me a big one. Formatting is inherently an n-ary function who args are one format and an indefinite number of objects to plug in. Packaging the remaining args into an object to convert the function to binary is problematical, especially in Python with its mix of positional and named args. Even without that, there is possible confusion between a package as an arg in itself and a package as a container of multiple args. The % formatting problem with tuple puns was one of the reasons to seek a replacement. Terry Jan Reedy > > Admittedly, these workarounds are less than clean, but the > understanding is the '@' syntax would indeed be an alternative, so one > could easily fall back to the str.format() method or the format() > function. > > IMPLEMENTATION > > Code-wise, the grammar was edited per PEP 306 [3], and a > function was introduced in unicodeobject.c as PyUnicode_FormatPrime > (in the mathematical sense of A and A' -- I didn't fully understand or > want to intrude upon the *_FormatAdvanced namespace). > > The PyUnicode_FormatPrime function transforms the incoming arguments, > i.e. the operands of the binary '@', and makes the appropriate > do_string_format() call. Thus, I have reused as much code as > possible. > > I have done my development with git by using two branches: 'master' > and 'subversion', the latter of which can be used to run 'svn update' > and merge back into master. This way my code changes and the official > ones going into the Subversion repository can stay separate, meanwhile > allowing 'svn diff' to produce an accurate patch at any given time. > > The code is available at: > > http://github.com/jcsalterego/py3k-atsign/ > > The SVN patch [4] or related commit [5] are good starting points. > > References: > > [1] http://www.python.org/dev/peps/pep-3101 > [2] http://docs.python.org/3.0/whatsnew/3.0.html > [3] http://www.python.org/dev/peps/pep-0306/ > [4] http://github.com/jcsalterego/py3k-atsign/blob/master/py3k-atsign.diff > [5] http://github.com/jcsalterego/py3k-atsign/commit/5c8bdf72d9252cea78af2b7809613f6530e25db4 > > Thanks, From omega_force2003 at yahoo.com Sun Jun 21 21:22:30 2009 From: omega_force2003 at yahoo.com (omega_force2003 at yahoo.com) Date: Sun, 21 Jun 2009 12:22:30 -0700 (PDT) Subject: [Python-Dev] run time error anlysis of python source code Message-ID: <120485.30890.qm@web34403.mail.mud.yahoo.com> To All, ? ? It appears that one possibility of investigation into the development of a safety-critical variant of the python language would be to conduct run time error analysis of the source code that is responsible for producing the python language.? Therefore, I will now conduct these run time error analysis of the python source code as if the python environment itself was to be utilized as a FADEC controller?within an aircraft engine.? I have?already conducted some analysis already and it appears to?be some concern with memory management within Python.? I will redouble my efforts and determine if I am correct and as a promise.? I will give my findings to everyone to enjoy if they so want it.? I will also give the correct source if anyone would want it for their own purposes.? The source code that I will be evaluating is the one responsible for the newer variants of Python 3.0. ? I believe that I will name this new variant of the python language as ? Apocalypse Python !! ? I will also develop a web-page for the development and evolution of Apocalypse Python. If anyone has any questions, please let me know !!!? ? Thank You, ? David Blubaugh ? ? ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From j at 3rdengine.com Sun Jun 21 21:44:09 2009 From: j at 3rdengine.com (Jerry Chen) Date: Sun, 21 Jun 2009 14:44:09 -0500 Subject: [Python-Dev] Binary Operator for New-Style String Formatting In-Reply-To: References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> Message-ID: <1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com> Ah, the people have spoken! On Sun, Jun 21, 2009 at 2:12 PM, Terry Reedy wrote: > The place to float trial balloons is the python-ideas list. I'll put this one to rest, and as mentioned, will direct any future suggestions to python-ideas instead of here. Most of the arguments against my proposal state there is little gain and much to lose (in terms of clarity or an "obvious way" to go about string formatting) -- and, I agree. > The only advantage '@' over '.format' is fewer characters. > I think it would be more useful to agitate to give 'format' a one char > synonym such as 'f'. str.f() would be a great idea. > One disadvantage of using an actual tuple rather than an arg quasi-tuple is > that people would have to remember the trailing comma when printing one > thing. '{}' @ (1,) rather than '{}' @ (a) == '{}' @ a. [If you say, 'Oh, > then accept the latter', then there is a problem when a is a tuple!] My code transforms both '{}' @ (a) and '{}' @ a to '{}'.format(a), but the problem you speak of is probably an edge case I haven't quite wrapped my head around. For what it's worth, I spent a bit of time trying to work out the syntactical quirks, including adapting the format tests in Lib/test/test_unicode.py to this syntax and ensuring all the tests passed. In the end though, it seems to be an issue of usability and clarity. > Formatting is inherently an n-ary function who args are one format and an > indefinite number of objects to plug in. Packaging the remaining args into > an object to convert the function to binary is problematical, especially in > Python with its mix of positional and named args. Even without that, there > is possible confusion between a package as an arg in itself and a package as > a container of multiple args. The % formatting problem with tuple puns was > one of the reasons to seek a replacement. Also (from R. David Murray): > That said, I'm -1 on it. The 'keywords as last item of tuple' reeks > of code-smell to my nose, and I don't think you've addressed all of > the reasons for why a method was chosen over an operator. Python has a > tradition of having "one obvious way" to do something, so introducing an > "alternative" syntax that you admit is sub-optimal does not seem to me > to have enough benefit to justify breaking that design guideline. Well stated (and everyone else). Just one last note: I think my end goal here was to preserve the visual clarity and separation between format string and format parameters, as I much prefer: "%s %s %s" % (1, 2, 3) over "{0} {1} {2}".format(1, 2, 3) The former is a style I've grown accustomed to, and if % is indeed being slated for removal in Python 3.2, then I will miss it sorely (or... just get over it). Thanks to everyone who has provided constructive criticism and great arguments. Cheers, -- Jerry Chen From martin at v.loewis.de Sun Jun 21 23:02:12 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 21 Jun 2009 23:02:12 +0200 Subject: [Python-Dev] Binary Operator for New-Style String Formatting In-Reply-To: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> Message-ID: <4A3E9FD4.1000706@v.loewis.de> > For better or for worse, I have created a patch against the py3k trunk > which introduces a binary operator '@' as an alternative syntax for > the new string formatting system introduced by PEP 3101 ("Advanced > String Formatting"). [1] I'd like to join everybody else who said that this would be a change for the worse. This kind of syntax is one of the most prominent features of Perl. $@~ly/your/s!, Martin From fredrik at pythonware.com Sun Jun 21 23:28:57 2009 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 21 Jun 2009 23:28:57 +0200 Subject: [Python-Dev] xmlrpc improvements In-Reply-To: <4A3D14F0.7080100@v.loewis.de> References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> <4A3D0DEA.50002@v.loewis.de> <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> <4A3D14F0.7080100@v.loewis.de> Message-ID: <368a5cd50906211428t301f5599m460e8c2ec2c519a3@mail.gmail.com> On Sat, Jun 20, 2009 at 6:57 PM, "Martin v. L?wis" wrote: > Fredrik Lundh wrote: >>> I think you really need to get Fredrik Lundh to comment on it. If he >>> can't predict when he'll be able to review the changes, maybe he can >>> accept releasing control of xmlrpclib. >> >> Pointer to the patch? > > http://bugs.python.org/issue6267 The xmlrpclib.py changes looks ok. I'll leave it to other reviewers to check the rest. From eric at trueblade.com Sun Jun 21 23:40:09 2009 From: eric at trueblade.com (Eric Smith) Date: Sun, 21 Jun 2009 17:40:09 -0400 Subject: [Python-Dev] Binary Operator for New-Style String Formatting In-Reply-To: <1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com> References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> <1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com> Message-ID: <4A3EA8B9.3080108@trueblade.com> I'm against syntax for this, for all the reasons stated by others. Jerry Chen wrote: > Just one last note: I think my end goal here was to preserve the > visual clarity and separation between format string and format > parameters, as I much prefer: > > "%s %s %s" % (1, 2, 3) > > over > > "{0} {1} {2}".format(1, 2, 3) If it helps, in 3.1 and 2.7 this can be written as "{} {} {}".format(1, 2, 3) I'm not sure it helps for "visual clarity", but it definitely makes the typing easier for simple uses. > The former is a style I've grown accustomed to, and if % is indeed > being slated for removal in Python 3.2, then I will miss it sorely > (or... just get over it). I've basically come to accept that %-formatting can never go away, unfortunately. There are too many places where %-formatting is used, for example in logging Formatters. %-formatting either has to exist or it has to be emulated. Although if anyone has any suggestions for migrating uses like that, I'm interested. Eric. From fredrik at pythonware.com Sun Jun 21 23:39:26 2009 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sun, 21 Jun 2009 23:39:26 +0200 Subject: [Python-Dev] xmlrpc improvements In-Reply-To: <4A3D14F0.7080100@v.loewis.de> References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> <4A3D0DEA.50002@v.loewis.de> <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> <4A3D14F0.7080100@v.loewis.de> Message-ID: <368a5cd50906211439i4f9a22b7mb062df8a8a86c8d7@mail.gmail.com> On Sat, Jun 20, 2009 at 6:57 PM, "Martin v. L?wis" wrote: > While I have your attention, please also review > > http://bugs.python.org/issue6233 I'm pretty sure that fix is the wrong fix - afaik, _encode is used to encode tag/attribute names, and charrefs don't work in that context. From tjreedy at udel.edu Sun Jun 21 23:50:37 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 21 Jun 2009 17:50:37 -0400 Subject: [Python-Dev] run time error anlysis of python source code In-Reply-To: <120485.30890.qm@web34403.mail.mud.yahoo.com> References: <120485.30890.qm@web34403.mail.mud.yahoo.com> Message-ID: omega_force2003 at yahoo.com wrote: > It appears that one possibility of investigation into the development of > a safety-critical variant of the python language would be to conduct run > time error analysis of the source code that is responsible for producing > the python language. Therefore, I will now conduct these run time error > analysis of the python source code as if the python environment itself > was to be utilized as a FADEC controller within an aircraft engine. I > have already conducted some analysis already and it appears to be some > concern with memory management within Python. I will redouble my > efforts and determine if I am correct and as a promise. I will give my > findings to everyone to enjoy if they so want it. I will also give the > correct source if anyone would want it for their own purposes. The > source code that I will be evaluating is the one responsible for the > newer variants of Python 3.0. I hope you mean 3.1 ;-) 3.0 was basically a trial version of Py3. From tjreedy at udel.edu Sun Jun 21 23:54:40 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 21 Jun 2009 17:54:40 -0400 Subject: [Python-Dev] xmlrpc improvements In-Reply-To: <368a5cd50906211428t301f5599m460e8c2ec2c519a3@mail.gmail.com> References: <930F189C8A437347B80DF2C156F7EC7F057F9DDAF3@exchis.ccp.ad.local> <4A3D0DEA.50002@v.loewis.de> <368a5cd50906200934n5b15a1aboac83ad1d32f5c3bb@mail.gmail.com> <4A3D14F0.7080100@v.loewis.de> <368a5cd50906211428t301f5599m460e8c2ec2c519a3@mail.gmail.com> Message-ID: Fredrik Lundh wrote: > On Sat, Jun 20, 2009 at 6:57 PM, "Martin v. L?wis" wrote: >> Fredrik Lundh wrote: >>>> I think you really need to get Fredrik Lundh to comment on it. If he >>>> can't predict when he'll be able to review the changes, maybe he can >>>> accept releasing control of xmlrpclib. >>> Pointer to the patch? >> http://bugs.python.org/issue6267 > > The xmlrpclib.py changes looks ok. I'll leave it to other reviewers > to check the rest. Comment cc'ed to tracker From ajaksu at gmail.com Mon Jun 22 01:05:30 2009 From: ajaksu at gmail.com (Daniel Diniz) Date: Sun, 21 Jun 2009 20:05:30 -0300 Subject: [Python-Dev] run time error anlysis of python source code In-Reply-To: <120485.30890.qm@web34403.mail.mud.yahoo.com> References: <120485.30890.qm@web34403.mail.mud.yahoo.com> Message-ID: <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com> Hi David, wrote: > > It appears that one possibility of investigation into the development of a > safety-critical variant of the python language There is some interesting work related to a safety-critical variant of Python. > would be to conduct run time > error analysis of the source code that is responsible for producing the > python language. There's been some effort into this too, and the Coverity and Klocwork based fixes could also be of interest to you. > Therefore, I will now conduct these run time error > analysis of the python source code as if the python environment itself was > to be utilized as a FADEC controller?within an aircraft engine. Nice, what tools do you have available for this? Any papers that would be a good start on the topic? > I > have?already conducted some analysis already and it appears to?be some > concern with memory management within Python.? I will redouble my efforts > and determine if I am correct and as a promise.? I will give my findings to > everyone to enjoy if they so want it.? I will also give the correct source > if anyone would want it for their own purposes.? The source code that I will > be evaluating is the one responsible for the newer variants of Python 3.0. You want the py3k branch. BTW, take a look at Brett Cannon's work on Python security, as well as tav's. > I believe that I will name this new variant of the python language as > > Apocalypse Python !! Oh. Apocalypse Python !!, you say? Maybe something that conveys a security message or anything that doesn't relate to the end of the world could work better. > I will also develop a web-page for the development and evolution of > Apocalypse Python. Ah. Hm. David? Don't. I mean, read the mailing lists, take a look at open bug reports, read the community blogs. You'll get to know how things flow, you'll figure a nice way to relate your ideas to what people are discussing and past experiences. Make small contributions that bring us closer to... Apocalypse Python !!, then you'll have an easier time to push the idea of going the whole way towards it. Or just do it, dunno, if it makes you feel better go for it, we all have our own issues. That's why we have an issue tracker, it's soooo nice, wanna see it??? > If anyone has any questions, please let me know !!! OK, I promise. Daniel From fijall at gmail.com Mon Jun 22 01:09:06 2009 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 21 Jun 2009 17:09:06 -0600 Subject: [Python-Dev] run time error anlysis of python source code In-Reply-To: <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com> References: <120485.30890.qm@web34403.mail.mud.yahoo.com> <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com> Message-ID: <693bc9ab0906211609w78c831f6g516a3f749ab847b1@mail.gmail.com> On Sun, Jun 21, 2009 at 5:05 PM, Daniel Diniz wrote: > Hi David, > > wrote: >> >> It appears that one possibility of investigation into the development of a >> safety-critical variant of the python language > > There is some interesting work related to a safety-critical variant of Python. > >> would be to conduct run time >> error analysis of the source code that is responsible for producing the >> python language. > > There's been some effort into this too, and the Coverity and Klocwork > based fixes could also be of interest to you. > >> Therefore, I will now conduct these run time error >> analysis of the python source code as if the python environment itself was >> to be utilized as a FADEC controller?within an aircraft engine. > > Nice, what tools do you have available for this? Any papers that would > be a good start on the topic? > >> I >> have?already conducted some analysis already and it appears to?be some >> concern with memory management within Python.? I will redouble my efforts >> and determine if I am correct and as a promise.? I will give my findings to >> everyone to enjoy if they so want it.? I will also give the correct source >> if anyone would want it for their own purposes.? The source code that I will >> be evaluating is the one responsible for the newer variants of Python 3.0. > > You want the py3k branch. BTW, take a look at Brett Cannon's work on > Python security, as well as tav's. Is py3k branch even passing all tests on all buildbots all the time? I don't think svn head is the right thing to look at. Also, it's worth noting that most big libraries are 2.x compatible only. From solipsis at pitrou.net Mon Jun 22 01:28:06 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 21 Jun 2009 23:28:06 +0000 (UTC) Subject: [Python-Dev] run time error anlysis of python source code References: <120485.30890.qm@web34403.mail.mud.yahoo.com> <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com> <693bc9ab0906211609w78c831f6g516a3f749ab847b1@mail.gmail.com> Message-ID: Maciej Fijalkowski gmail.com> writes: > > Is py3k branch even passing all tests on all buildbots all the time? As much as other branches do (that is, not much, due to the flakiness of some of the tests and the lack of buildbot maintenance). > I don't think > svn head is the right thing to look at. Also, it's worth noting that > most big libraries are 2.x compatible only. For projects difficult enough that they won't be finished before a couple of years, I think it makes sense to target 3.x. Big libraries will hopefully migrate gradually. SQLAlchemy recently announced that their current development version is 100% py3k-compatible (http://www.sqlalchemy.org/news.html#item_1). From greg.ewing at canterbury.ac.nz Mon Jun 22 02:22:30 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 22 Jun 2009 12:22:30 +1200 Subject: [Python-Dev] run time error anlysis of python source code In-Reply-To: <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com> References: <120485.30890.qm@web34403.mail.mud.yahoo.com> <2d75d7660906211605u4d9cd1c9i6641de5cd2e77d79@mail.gmail.com> Message-ID: <4A3ECEC6.3020402@canterbury.ac.nz> Daniel Diniz wrote: > Apocalypse Python !!, you say? Maybe something that conveys a security > message or anything that doesn't relate to the end of the world could > work better. I guess the idea is meant to be that it's safe enough to use for something that would result in the end of the world if it failed !! Although personally, if something might cause the end of the world if it failed, I'd prefer not to attempt it in the first place !! -- Apocalypse Python !! - If you ever need to use it, you're in deep trouble... !! Greg !! From foom at fuhm.net Mon Jun 22 14:30:49 2009 From: foom at fuhm.net (James Y Knight) Date: Mon, 22 Jun 2009 08:30:49 -0400 Subject: [Python-Dev] Migration strategy for new-style string formatting [Was: Binary Operator for New-Style String Formatting] In-Reply-To: <4A3EA8B9.3080108@trueblade.com> References: <1c9b9de40906211036v2eb2c86ajbb14fa646d2d9014@mail.gmail.com> <1c9b9de40906211244v2561a758r12751695c8adedc1@mail.gmail.com> <4A3EA8B9.3080108@trueblade.com> Message-ID: <631E4C91-41C7-4196-854A-A7462841B319@fuhm.net> On Jun 21, 2009, at 5:40 PM, Eric Smith wrote: > I've basically come to accept that %-formatting can never go away, > unfortunately. There are too many places where %-formatting is used, > for example in logging Formatters. %-formatting either has to exist > or it has to be emulated. It'd possibly be helpful if there were builtin objects which forced the format style to be either newstyle or oldstyle, independent of whether % or format was called on it. E.g. x = newstyle_formatstr("{} {} {}") x % (1,2,3) == x.format(1,2,3) == "1 2 3" and perhaps, for symmetry: y = oldstyle_formatstr("%s %s %s") y.format(1,2,3) == x % (1,2,3) == "1 2 3" This allows the format string "style" decision is to be made external to the API actually calling the formatting function. Thus, it need not matter as much whether the logging API uses % or .format() internally -- that only affects the *default* behavior when a bare string is passed in. This could allow for a controlled staged towards the new format string format, with a long deprecation period for users to migrate: 1) introduce the above feature, and recommend in docs that people only ever use new-style format strings, wrapping the string in newstyle_formatstr() when necessary for passing to an API which uses % internally. 2) A long time later...deprecate str.__mod__; don't deprecate newstyle_formatstr.__mod__. 3) A while after that (maybe), remove str.__mod__ and replace all calls in Python to % (used as a formatting operator) with .format() so that the default is to use newstyle format strings for all APIs from then on. James From ziade.tarek at gmail.com Mon Jun 22 15:23:47 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 22 Jun 2009 15:23:47 +0200 Subject: [Python-Dev] PEP 376 Message-ID: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> Hello, We have polished out PEP 376 and its code prototype at Distutils-SIG. It seems to fullfill now all the requirements, so I am mailing it here again, for a new round of feedback, if needed. - the pep : http://svn.python.org/projects/peps/trunk/pep-0376.txt - the code prototype : http://bitbucket.org/tarek/pep376/src/tip/pkgutil.py Notice that if the PEP is accepted at this point, I will : - focus on making the code work as fast as possible, for directories browsing - work on the backport and the required patches for setuptools and pip at the same time, and see if I can get some beta-testers that are willing to switch to this new version to test it extensively before 2.7/3.2 are out. Regards Tarek -- Tarek Ziad? | http://ziade.org From solipsis at pitrou.net Mon Jun 22 16:59:24 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 22 Jun 2009 14:59:24 +0000 (UTC) Subject: [Python-Dev] PEP 376 References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> Message-ID: Hello, Tarek Ziad? gmail.com> writes: > > so I am mailing it here again, for a new round of feedback, if needed. > > - the pep : http://svn.python.org/projects/peps/trunk/pep-0376.txt Some comments: - the **MD5** hash of the file, encoded in hex. Notice that `pyc` and `pyo` generated files will not have a hash. Why the exception for pyc and pyo files? - `zlib` and `zlib-2.5.2.egg-info` are located in `site-packages` so the file paths are relative to it. Is it a general rule? That is, the paths in RECORD are always relative to its grandparent directory? The RECORD format ----------------- The `RECORD` file is a CSV file, composed of records, one line per installed file. The ``csv`` module is used to read the file, with the `excel` dialect, which uses these options to read the file: - field delimiter : `,` - quoting char : `"`. - line terminator : `\r\n` Wouldn't it be better to use the native line terminator on the current platform? (someone might want to edit or at least view the file) What is the character encoding for non-ASCII filenames? UTF-8? Are the RECORD file's contents somehow part of the DistributionMetadata? - ``DistributionDirectories``: manages ``EggInfoDirectory`` instances. What is an EggInfoDirectory ? A plural class name looks strange (I think it's the first time I see one in the CPython codebase). How about another name? (DistributionPool, DistributionMap, WorkingSet etc.). - ``get_egginfo_file(path, binary=False)`` -> file object Returns a file located under the `.egg-info` directory. Returns a ``file`` instance for the file pointed by ``path``. Is it always a file instance or just a file-like object? (zipped distributions come to mind). Is it opened read-only? - ``owner(path)`` -> ``Distribution`` instance or None If ``path`` is used by only one ``Distribution`` instance, returns it. Otherwise returns None. This is a bit confusing. If it returns None, it doesn't distinguish between the case where several Distributions refer to the path, and the case where no distributions refer to it, does it? Is there any reason to have this method while file_users(path) already exists? A new class called ``DistributionDirectories`` is created. It's a collection of ``DistributionDirectory`` and ``ZippedDistributionDirectory`` instances. The constructor takes one optional argument ``use_cache`` set to ``True`` by default. You forgot to describe the constructor's signature and what it does exactly. ``EggInfoDirectories`` also provides the following methods besides the ones from ``dict``:: What is EggInfoDirectories? - ``append(path)`` Creates an ``DistributionDirectory`` (or ``ZippedDistributionDirectory``) instance for ``path`` and adds it in the mapping. - ``load(paths)`` Creates and adds ``DistributionDirectory`` (or ``ZippedDistributionDirectory``) instances corresponding to ``paths``. Why are these methods named completely differently although they do almost the same thing? Besides, append() makes it look like ordering matters. Does it? (for a naming suggestion, e.g. load(path) and load_all(paths). Or, even simpler, load(*paths) or load(paths)) - ``get_file_users(path)`` -> Iterator of ``Distribution`` (or ``ZippedDistribution``) instances. This method is named file_users in another class. Perhaps the naming should be consistent? All these functions use the same global instance of ``DistributionDirectories`` to use the cache. Is the global instance available to users? >>> for path, hash, size in dist.get_installed_files():: ... print '%s %s %d %s' % (path, hash, size) There's one too many "%s" here. Thanks for your work! Antoine. From ziade.tarek at gmail.com Mon Jun 22 17:42:24 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 22 Jun 2009 17:42:24 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> Message-ID: <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> On Mon, Jun 22, 2009 at 4:59 PM, Antoine Pitrou wrote: > - the **MD5** hash of the file, encoded in hex. Notice that `pyc` and > `pyo` > generated files will not have a hash. > > Why the exception for pyc and pyo files? As in PEP 262, since they are produced automatically from a py file, checking the py file is enough to decide if the file has changed. > > - `zlib` and `zlib-2.5.2.egg-info` are located in `site-packages` so the > file > paths are relative to it. > > Is it a general rule? That is, the paths in RECORD are always relative to > its > grandparent directory? no, they can be located anywhere on the system. But when the paths are located in the same directory where the .egg-info directory is located, a relative path is used. (see the section before this example) I'll add an example that contains files located elswhere in the system (like a script and a data file) > > > The RECORD format > ----------------- > > The `RECORD` file is a CSV file, composed of records, one line per > installed file. The ``csv`` module is used to read the file, with > the `excel` dialect, which uses these options to read the file: > > - field delimiter : `,` > - quoting char : `"`. > - line terminator : `\r\n` > > Wouldn't it be better to use the native line terminator on the current > platform? (someone might want to edit or at least view the file) Good idea, I'll change that, > > > What is the character encoding for non-ASCII filenames? UTF-8? > Yes, there's a constant in Distutils, called PKG_INFO_ENCODING that will be used for the file generation. > > Are the RECORD file's contents somehow part of the DistributionMetadata? The DistributionMetadata correspond to the fields defined in PEP 345, e.g. written in the PKG-INFO file, which is mentioned in the RECORD file. We are reworking PEP 345 as well, to add some fields. What did you have in mind ? > > > - ``DistributionDirectories``: manages ``EggInfoDirectory`` instances. > > What is an EggInfoDirectory ? Typo (old name), fixing this.. > > > A plural class name looks strange (I think it's the first time I see one in > the CPython codebase). How about another name? (DistributionPool, > DistributionMap, WorkingSet etc.). Sure, WorkingSet is nice, it's the name used in setuptools, > > > - ``get_egginfo_file(path, binary=False)`` -> file object > > Returns a file located under the `.egg-info` directory. > > Returns a ``file`` instance for the file pointed by ``path``. > > Is it always a file instance or just a file-like object? (zipped > distributions > come to mind). Is it opened read-only? It's in read-only mode, either "r" either "rb" and in case of a zip file, it returns a file-like object using zipfile.ZipFile.open. > > - ``owner(path)`` -> ``Distribution`` instance or None > > If ``path`` is used by only one ``Distribution`` instance, returns it. > Otherwise returns None. > > This is a bit confusing. If it returns None, it doesn't distinguish between > the > case where several Distributions refer to the path, and the case where no > distributions refer to it, does it? > The idea of this API is to find out of a distribution "owns" a file, e.g. is the only distribution that uses it, so it can be safely removed. > Is there any reason to have this method while file_users(path) already > exists? > Its just a helper for uninstallers, but file_users() could probably be enough, I can remove "owns" if people find it confusing, > A new class called ``DistributionDirectories`` is created. It's a > collection of > ``DistributionDirectory`` and ``ZippedDistributionDirectory`` instances. > The constructor takes one optional argument ``use_cache`` set to ``True`` > by > default. > > You forgot to describe the constructor's signature and what it does > exactly. > I'll add that, > > ``EggInfoDirectories`` also provides the following methods besides the > ones > from ``dict``:: > > What is EggInfoDirectories? This is a DistributionDirectories, one more instance I forgot to rename, I'll fix that > > > - ``append(path)`` > > Creates an ``DistributionDirectory`` (or > ``ZippedDistributionDirectory``) > instance for ``path`` and adds it in the mapping. > > - ``load(paths)`` > > Creates and adds ``DistributionDirectory`` (or > ``ZippedDistributionDirectory``) instances corresponding to ``paths``. > > Why are these methods named completely differently although they do almost > the > same thing? Besides, append() makes it look like ordering matters. Does it? > (for a naming suggestion, e.g. load(path) and load_all(paths). Or, > even simpler, load(*paths) or load(paths)) > Right, I'll fix that, > > - ``get_file_users(path)`` -> Iterator of ``Distribution`` (or > ``ZippedDistribution``) instances. > > This method is named file_users in another class. Perhaps the naming should > be > consistent? Right, it used to be get_*, that's a typo. I'll fix it, > > > All these functions use the same global instance of > ``DistributionDirectories`` > to use the cache. > > Is the global instance available to users? No I didn't made it available to avoid concurrency problems, > > > >>> for path, hash, size in dist.get_installed_files():: > ... print '%s %s %d %s' % (path, hash, size) > > There's one too many "%s" here. > Fixing it too, > > > Thanks for your work! > Thanks for the feedback, I'll commit the fixes asap. Tarek -------------- next part -------------- An HTML attachment was scrubbed... URL: From pje at telecommunity.com Mon Jun 22 20:48:08 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 22 Jun 2009 14:48:08 -0400 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.co m> References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> Message-ID: <20090622184512.62DD43A4099@sparrow.telecommunity.com> At 05:42 PM 6/22/2009 +0200, Tarek Ziad? wrote: >Wouldn't it be better to use the native line terminator on the current >platform? (someone might want to edit or at least view the file) > > >Good idea, I'll change that, > As long as the file is always *read* with "U" mode, so that you can't mess it up, especially if the install is to a directory shared between platforms. >The idea of this API is to find out of a distribution "owns" a file, >e.g. is the only distribution >that uses it, so it can be safely removed. This could equally well be done by ``owners(path)``, returning a sequence of zero or more items. Any length <> 1 means the file can't be safely removed. Meanwhile, having the data about all the owners of a file would also be useful for tools that just want to inspect a directory's contents, for example, or to detect conflicts and overwrites. From benjamin at python.org Mon Jun 22 21:44:44 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 22 Jun 2009 14:44:44 -0500 Subject: [Python-Dev] draft pep: backwards compatibility In-Reply-To: <43aa6ff70906201049q26add809sa5893c470cc520d6@mail.gmail.com> References: <1afaf6160906181917m670663f5ucb21b8bc00c07735@mail.gmail.com> <43aa6ff70906201049q26add809sa5893c470cc520d6@mail.gmail.com> Message-ID: <1afaf6160906221244x208b1aa3v811483ab6955e038@mail.gmail.com> 2009/6/20 Collin Winter : > Is this intended to include performance changes? Clearly no-one will > complain if things simply get faster, but I'm thinking about cases > where, say, a function runs in half the time but uses double the > memory (or vice versa). I don't think we can say anything about those cases before hand. We'll have to decide on a case by case basis. -- Regards, Benjamin From benjamin at python.org Tue Jun 23 00:50:49 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 22 Jun 2009 17:50:49 -0500 Subject: [Python-Dev] final release procedure Message-ID: <1afaf6160906221550s5d860086p56da686c61ff2b8e@mail.gmail.com> Hi everyone, We're almost to the finish line on 3.1! Assuming that the last release blocker [1] gets ironed out and no more rear their ugly heads, I will go ahead. I'm going to freeze and tag the tree on Friday at about 15:00 UTC in order for the binaries to be built and uploaded. The release announcement should be the next day. [1] http://bugs.python.org/issue6233 -- Regards, Benjamin From kevin at bud.ca Tue Jun 23 03:41:21 2009 From: kevin at bud.ca (Kevin Teague) Date: Mon, 22 Jun 2009 18:41:21 -0700 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> Message-ID: > > > A plural class name looks strange (I think it's the first time I see > one in > the CPython codebase). How about another name? (DistributionPool, > DistributionMap, WorkingSet etc.). > > Sure, WorkingSet is nice, it's the name used in setuptools, > A WorkingSet and a DistributionDirectories (or whatever it gets named to) are different things though, no? A WorkingSet is "a collection of active distributions", where each distribution might come from different distribution directories: http://peak.telecommunity.com/DevCenter/PkgResources#workingset-objects Where as DistributionDirectories is a dictionary of locations where distributions are installed. The WorkingSet may be comprised of distributions from several different locations, and each location may contain the same or different versions of the same distribution. (as far as I understand things ...) I can't really think of a better name for a dict of distribution locations ... but then I'm not averse to a pluralized class name. Overall though, I think PEP 376 is starting to look very good! From tcr at freebits.de Tue Jun 23 08:47:36 2009 From: tcr at freebits.de (Tobias C. Rittweiler) Date: Tue, 23 Jun 2009 08:47:36 +0200 Subject: [Python-Dev] Adding syntax for units of measure Message-ID: <873a9rv43b.fsf@freebits.de> Hi! Has anyone added special syntax to allow writing numeric literals with physical units? So you can write 12m + 34cm, and would get 12.34m. My question is how would you modify the BNF the most sensible way to allow for this? The above example is simple, but think of 42 km/h. (For my purposes, modifying the BNF is perfectly reasonable, but if you can depict a good, and convenient!, way that would not result in modifying it, I'd like to hear it, too.) Thanks in advance for your input, -T. From tjreedy at udel.edu Tue Jun 23 09:00:26 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 23 Jun 2009 03:00:26 -0400 Subject: [Python-Dev] Adding syntax for units of measure In-Reply-To: <873a9rv43b.fsf@freebits.de> References: <873a9rv43b.fsf@freebits.de> Message-ID: Tobias C. Rittweiler wrote: > Has anyone added special syntax to allow writing numeric literals with > physical units? So you can write 12m + 34cm, and would get 12.34m. Python-dev is for concrete discussion of development of the next versions. Questions and speculative discussion should generally be directed to python-list and concrete proposals to python-ideas. There have been posts on this subject on python-list and you can search the archive. That list is mirrored as newsgroup gmane.comp.python.general at news.gmane.org, which I believe has its own searchable archive. tjr From ziade.tarek at gmail.com Tue Jun 23 10:29:38 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 23 Jun 2009 10:29:38 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: <20090622184512.62DD43A4099@sparrow.telecommunity.com> References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> <20090622184512.62DD43A4099@sparrow.telecommunity.com> Message-ID: <94bdd2610906230129kac2fcf9rd3ab107499b70821@mail.gmail.com> 2009/6/22 P.J. Eby : > At 05:42 PM 6/22/2009 +0200, Tarek Ziad? wrote: >> >> Wouldn't it be better to use the native line terminator on the current >> platform? (someone might want to edit or at least view the file) >> >> >> Good idea, I'll change that, >> > > As long as the file is always *read* with "U" mode, so that you can't mess > it up, especially if the install is to a directory shared between platforms. Adding that too, thanks; > > >> The idea of this API is to find out of a distribution "owns" a file, e.g. >> is the only distribution >> that uses it, so it can be safely removed. > > This could equally well be done by ``owners(path)``, returning a sequence of > zero or more items. ?Any length <> 1 means the file can't be safely removed. > ?Meanwhile, having the data about all the owners of a file would also be > useful for tools that just want to inspect a directory's contents, for > example, or to detect conflicts and overwrites. that's basically what "get_file_users" does except it's an iterator, roughly that would be : def get_file_owners(path): return list(get_file_users(paths)) So I am wondering if it worth having it.... -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Tue Jun 23 10:38:38 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 23 Jun 2009 10:38:38 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> Message-ID: <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com> On Tue, Jun 23, 2009 at 3:41 AM, Kevin Teague wrote: >> >> >> A plural class name looks strange (I think it's the first time I see one >> in >> the CPython codebase). How about another name? (DistributionPool, >> DistributionMap, WorkingSet etc.). >> >> Sure, WorkingSet is nice, it's the name used in setuptools, >> > > A WorkingSet and a DistributionDirectories (or whatever it gets named to) > are different things though, no? > > A WorkingSet is "a collection of active distributions", where each > distribution might come from different distribution directories: > > http://peak.telecommunity.com/DevCenter/PkgResources#workingset-objects > > Where as DistributionDirectories is a dictionary of locations where > distributions are installed. The WorkingSet may be comprised of > distributions from several different locations, and each location may > contain the same or different versions of the same distribution. DistributionDirectories can contain directories that are not located in the same parent directory, so I find it rather similar besides the "active" feature in Python doesn't exist (yet) In any case, maybe picking up a name that is not from setuptools will be less confusing for people that uses WorkingSet classes nowadays. What about using the same names used in Python's site module: "sitedir" is the name used for a directory we named DistributionDirectory. So what about : DistributionDirectory -> SiteDir DistributionDirectories -> SiteDirMap ++ Tarek From ncoghlan at gmail.com Tue Jun 23 11:57:53 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 23 Jun 2009 19:57:53 +1000 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com> References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com> Message-ID: <4A40A721.6090204@gmail.com> Tarek Ziad? wrote: > So what about : > > DistributionDirectory -> SiteDir > DistributionDirectories -> SiteDirMap 'site' has too many connections to existing concepts for my liking (site.py, sitesetup.py, site-packages). Something like DistributionDirectoryMap should cover it. You could probably get away with shortening "Directory" to "Dir" in the class names though: - Distribution - ZippedDistribution - DistributionDir - ZippedDistributionDir - DistributionDirMap (Shortening Distribution to Dist might also be a possibility, but I don't think that works well for the two basic classes, and if those use the long form then shortening it for the *Dir and *DirMap classes would just look odd) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From lists at cheimes.de Tue Jun 23 12:51:55 2009 From: lists at cheimes.de (Christian Heimes) Date: Tue, 23 Jun 2009 12:51:55 +0200 Subject: [Python-Dev] Adding syntax for units of measure In-Reply-To: <873a9rv43b.fsf@freebits.de> References: <873a9rv43b.fsf@freebits.de> Message-ID: Tobias C. Rittweiler schrieb: > Hi! > > Has anyone added special syntax to allow writing numeric literals with > physical units? So you can write 12m + 34cm, and would get 12.34m. > > My question is how would you modify the BNF the most sensible way to > allow for this? The above example is simple, but think of 42 km/h. You don't need special syntax in order to work with units. You just need to normalize all input to SI units like meter, second and meter per second: km=1000. m=1. dm=0.1 cm=0.01 mm=0.001 s= 1. h = 3600. m_s=m/s km_h=km/h length_in_mm = (12*m + 34*cm)/mm speed = 5*km_h For what purpose do you want to have physical units in Python syntax? Do you need to verify your formulas? Christian From g.brandl at gmx.net Tue Jun 23 19:40:28 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 23 Jun 2009 17:40:28 +0000 Subject: [Python-Dev] Adding syntax for units of measure In-Reply-To: <873a9rv43b.fsf@freebits.de> References: <873a9rv43b.fsf@freebits.de> Message-ID: Tobias C. Rittweiler schrieb: > Hi! > > Has anyone added special syntax to allow writing numeric literals with > physical units? So you can write 12m + 34cm, and would get 12.34m. > > My question is how would you modify the BNF the most sensible way to > allow for this? The above example is simple, but think of 42 km/h. > > (For my purposes, modifying the BNF is perfectly reasonable, but if you > can depict a good, and convenient!, way that would not result in > modifying it, I'd like to hear it, too.) Hi, normally you wouldn't add units to the language itself. When using them programmatically, it should be no effort to use a class that represents a quantity with unit. This can be made as easy as making "m" an object of that type, so that you only need to type "2*m" to get two meters. For the interactive shell, using a wrapper that allows simplified input is also a possibility, like IPython's "-profile physics" mode, or something like http://bitbucket.org/birkenfeld/phsh/ which allows you to write >>> `1 m` + `12 cm` 1.12 m cheers, Georg From forumer at smartmobili.com Tue Jun 23 20:37:34 2009 From: forumer at smartmobili.com (Vincent R.) Date: Tue, 23 Jun 2009 20:37:34 +0200 Subject: [Python-Dev] Python 3.0.1 and mingw Message-ID: <17a4b6870bdf20f0d2b794eb55ee8ba7@mail.smartmobili.com> Hi, I wanted to know if you have some patch to compile python 3.x on mingw platform because I found some but doesn't work very well : > make gcc -o python.exe \ Modules/python.o \ libpython3.0.a -lm Could not find platform independent libraries Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Fatal Python error: Py_Initialize: can't initialize sys standard streams ImportError: No module named encodings.utf_8 I have some questions about posixmodule.c, config.c and makesetup, I can see in posixmodule that you define a INITFUNC like this : #if (defined(_MSC_VER) || defined(__WATCOMC__) || defined(__BORLANDC__)) && !defined(__QNX__) #define INITFUNC PyInit_nt #define MODNAME "nt" #elif defined(PYOS_OS2) #define INITFUNC PyInit_os2 #define MODNAME "os2" #else #define INITFUNC PyInit_posix #define MODNAME "posix" #endif So first I tried to add || defined(____MINGW32____) to declare a PyInit_nt but after config.c was still using PyInit_posix. How does makesetup choose to include one function or another ? So finally python is using PyInit_posix... Any idea why I got this compilation error ? And after From aahz at pythoncraft.com Tue Jun 23 21:39:27 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 23 Jun 2009 12:39:27 -0700 Subject: [Python-Dev] Python 3.0.1 and mingw In-Reply-To: <17a4b6870bdf20f0d2b794eb55ee8ba7@mail.smartmobili.com> References: <17a4b6870bdf20f0d2b794eb55ee8ba7@mail.smartmobili.com> Message-ID: <20090623193927.GA25928@panix.com> On Tue, Jun 23, 2009, Vincent R. wrote: > > I wanted to know if you have some patch to compile python 3.x on mingw > platform because I found some but doesn't work very well : This question belongs on comp.lang.python, please re-post there. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha From sridharr at activestate.com Tue Jun 23 21:45:29 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Tue, 23 Jun 2009 12:45:29 -0700 Subject: [Python-Dev] PEP 376 In-Reply-To: <4A40A721.6090204@gmail.com> References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com> <4A40A721.6090204@gmail.com> Message-ID: <4A4130D9.5040108@activestate.com> On 09-06-23 02:57 AM, Nick Coghlan wrote: > Something like DistributionDirectoryMap should cover it. > > You could probably get away with shortening "Directory" to "Dir" in the > class names though: > > - Distribution > - ZippedDistribution > - DistributionDir > - ZippedDistributionDir > - DistributionDirMap 'Map' reminds me of an associated hash (or is it the actual map?). Hence I suggest: DistributionSet -srid From skip at pobox.com Wed Jun 24 06:07:02 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 23 Jun 2009 23:07:02 -0500 Subject: [Python-Dev] Adding syntax for units of measure In-Reply-To: References: <873a9rv43b.fsf@freebits.de> Message-ID: <19009.42598.902021.199401@montanaro.dyndns.org> >> Has anyone added special syntax to allow writing numeric literals with >> physical units? So you can write 12m + 34cm, and would get 12.34m. ... Georg> normally you wouldn't add units to the language itself. ... Georg> For the interactive shell, using a wrapper that allows simplified Georg> input is also a possibility, like IPython's "-profile physics" Georg> mode, or something like http://bitbucket.org/birkenfeld/phsh/ Georg> which allows you to write >>>> `1 m` + `12 cm` Georg> 1.12 m Also, check out the magnitude module (in PyPI). I use it to specify the units of the computation but allow users to input values using units which are meaningful to them. So, for example, if a value has units of time they could enter 1m or 60s and get the same internal value. Skip From pje at telecommunity.com Wed Jun 24 07:10:02 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 24 Jun 2009 01:10:02 -0400 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.co m> References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com> Message-ID: <20090624050718.C0DDD3A4099@sparrow.telecommunity.com> At 10:38 AM 6/23/2009 +0200, Tarek Ziad? wrote: >What about using the same names used in Python's site module: >"sitedir" is the name used for >a directory we named DistributionDirectory. No, a site dir is a Python-defined directory for site-installed packages, and/or a directory where .pth files are processed. Wrong connotations entirely, since packages may be installed to other directories, and typically are in e.g. shared hosting environments. DistributionDirectory is fine by me. DistributionDirectories sounds like what setuptools calls an Environment. From solipsis at pitrou.net Wed Jun 24 11:18:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 24 Jun 2009 09:18:35 +0000 (UTC) Subject: [Python-Dev] PEP 376 References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com> <4A40A721.6090204@gmail.com> <4A4130D9.5040108@activestate.com> Message-ID: Sridhar Ratnakumar activestate.com> writes: > > On 09-06-23 02:57 AM, Nick Coghlan wrote: > > Something like DistributionDirectoryMap should cover it. > > > > You could probably get away with shortening "Directory" to "Dir" in the > > class names though: > > > > - Distribution > > - ZippedDistribution > > - DistributionDir > > - ZippedDistributionDir > > - DistributionDirMap > > 'Map' reminds me of an associated hash (or is it the actual map?). Good thing, because it is a dict subclass if you read the PEP :) +1 with Nick's proposal. (we are good at finding bikesheds everywhere aren't we?) From pegasus2000 at email.it Wed Jun 24 17:09:21 2009 From: pegasus2000 at email.it (Filippo Battaglia) Date: Wed, 24 Jun 2009 17:09:21 +0200 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE DEVELOPERS Message-ID: <006801c9f4dd$c18294c0$011ea8c0@pegasus> I need help with an implementation of your interpreter under Nanodesktop PSPE/PSP. I'm working to a version of Stackless Python interpreter (2.5.2) called ndPython. It is able to work under PSPE and PSP and it shall be the most powerful interpreter ever realized before on these platforms. I need to know something about the C functions that are recalled by the interpreter when it executes a .pyc file. Our version of ndPython is very slow in execution respect to Carlos's StackLess Python (another product for PSP). We believe that the trouble is in any routine of our Nanodesktop libc that could be a bottleneck. But we don't know which can be the interested routine (string ? memory allocation ?) Can you tell me which modules provide to decode and execute a .pyc file ? The number of modules that are statically linked in the interpreter can affect its performance ? (for example, for a larger number of strcmp between the keywords). The execution of a .pyc file is a memory allocation intensive task ? I've already enable the PYMEM option and I've see that the speed is improved (but Carlos's StackLess Python remains faster). We've done these tests: http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20 from ndpsp import * x = time.clock () for i in range(1000): pass print >> clock, "1.", time.clock()-t We've the results: stackless: 0.0037279 ndPython: 0.1015625 We don't understand why ndPython is so slower. Please, help us. The product is complete, but we cannot release cause this problem. Thank you in advance. Filippo Battaglia -- Caselle da 1GB, trasmetti allegati fino a 3GB e in piu' IMAP, POP3 e SMTP autenticato? GRATIS solo con Email.it http://www.email.it/f Sponsor: Vieni in vacanza nei Riccione Family Hotels! Ti aspettano servizi per bambini, mini club e tanto divertimento! Prenota on-line la tua vacanza! Clicca qui: http://adv.email.it/cgi-bin/foclick.cgi?mid=9206&d=24-6 From tjreedy at udel.edu Wed Jun 24 18:10:15 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 24 Jun 2009 12:10:15 -0400 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE DEVELOPERS In-Reply-To: <006801c9f4dd$c18294c0$011ea8c0@pegasus> References: <006801c9f4dd$c18294c0$011ea8c0@pegasus> Message-ID: SUBJECT LINES IN ALL CAPS CREATE NEGATIVE IMPRESSION Filippo Battaglia wrote: [snip questions I cannot answer] Question about existing, older Python versions should be directed to python-list, at least for a first try. Stackless Python is not produced by PSF core developers. It has third-party alterations to the core that might affect the answers to your question. > http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20 You should mention that the above is Italian, not English. There once was, and maybe still is, a python discussion list in Italian. > from ndpsp import * x = time.clock () > for i in range(1000): pass print >> clock, "1.", time.clock()-t When you post code, cut and paste the actual code that you ran. The above code will not run. Line-endings are missing. t is not defined. Perhaps you meant 't = time.clock()'. Good luck with your project. Terry Jan Reedy From kristjan at ccpgames.com Wed Jun 24 18:23:57 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 24 Jun 2009 16:23:57 +0000 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE DEVELOPERS In-Reply-To: References: <006801c9f4dd$c18294c0$011ea8c0@pegasus> Message-ID: <930F189C8A437347B80DF2C156F7EC7F057FB5FB0E@exchis.ccp.ad.local> The OP was redirected here from the stackless list since his questions were not stackless specific. Stackless python is in sync with the latest 2.x, 3.x branches as well as the trunk so performance problems he may with performance in general are perhaps best resolved by this lot. Kristj?n > -----Original Message----- > From: python-dev-bounces+kristjan=ccpgames.com at python.org > [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf > Of Terry Reedy > Sent: 24. j?n? 2009 16:10 > To: python-dev at python.org > Subject: Re: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE > PYTHON CORE DEVELOPERS > Question about existing, older Python versions should be directed to > python-list, at least for a first try. Stackless Python is not produced > by PSF core developers. It has third-party alterations to the core that > might affect the answers to your question. > From dhburns at cherokeetel.net Wed Jun 24 17:59:17 2009 From: dhburns at cherokeetel.net (David H. Burns) Date: Wed, 24 Jun 2009 10:59:17 -0500 Subject: [Python-Dev] Python for Windows?? Message-ID: <4A424D55.6010404@cherokeetel.net> I downloaded what claims to be Python for Windows (3.01). The tutorial brags a lot about how easy it is to learn, but the tutorials and instruction seem to be for a Linux or Unix version. There are three executable programs in the Python directory and no indication which should be used to start Python. One opens a Dos-like window presumably for "command-line" entry. I can't make anything of it. I did a straight install on a XP system. Any help would be appreciated. Have a good day, David H. Burns From fuzzyman at voidspace.org.uk Wed Jun 24 19:13:32 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 24 Jun 2009 18:13:32 +0100 Subject: [Python-Dev] Python for Windows?? In-Reply-To: <4A424D55.6010404@cherokeetel.net> References: <4A424D55.6010404@cherokeetel.net> Message-ID: <4A425EBC.20007@voidspace.org.uk> David H. Burns wrote: > I downloaded what claims to be Python for Windows (3.01). The tutorial > brags a lot about how easy it is to learn, but the tutorials and > instruction seem to be for a Linux or Unix version. There are three > executable programs in the Python directory and no indication which > should be used to start Python. One opens a Dos-like window presumably > for "command-line" entry. I can't make anything of it. I'm afraid the mailing list you posted to is for the development *of* Python and not the use of Python. On the Python-tutor or Python-list mailing lists (or their google groups / gmane gateways if you prefer a web interface) you will find people willing and able to answer your questions. All the best, Michael Foord > > I did a straight install on a XP system. > > Any help would be appreciated. > > Have a good day, > > David H. Burns > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From solipsis at pitrou.net Wed Jun 24 19:41:39 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 24 Jun 2009 17:41:39 +0000 (UTC) Subject: [Python-Dev] ndPython References: <006801c9f4dd$c18294c0$011ea8c0@pegasus> Message-ID: Filippo Battaglia email.it> writes: > > http://www.psp-ita.com/forum/viewtopic.php?t=28323&start=20 > > from ndpsp import * > x = time.clock () > for i in range(1000): > pass > print >> clock, "1.", time.clock()-t First, it has been noted that your message would better be sent to python-list. You must provide a working code example that doesn't need any "ndpsp" import to work. Also, since it is about porting Python to a particular platform, please enlighten us as to what type of CPU it uses, what compiler, what optimization options were used etc. As a sidenote, a common way of timing small snippets of Python code is by using the "timeit" module. From the command line, you could write: $ python -m timeit "for i in range(1000): pass" 10000 loops, best of 3: 87.2 usec per loop If you don't spend at least a bit of time writing your message clearly and giving enough details, nobody will spend their unpaid voluntary time trying to understand what happens. Regards Antoine. From tjreedy at udel.edu Wed Jun 24 20:57:17 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 24 Jun 2009 14:57:17 -0400 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE DEVELOPERS In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F057FB5FB0E@exchis.ccp.ad.local> References: <006801c9f4dd$c18294c0$011ea8c0@pegasus> <930F189C8A437347B80DF2C156F7EC7F057FB5FB0E@exchis.ccp.ad.local> Message-ID: Kristj?n Valur J?nsson wrote: > The OP was redirected here from the stackless list since his > questions were not stackless specific. Ok. He should have said so. Third party directions are not always right. As it turns out, he did post to python-list but ignored the rather strong hint about not shouting in the subject line. > Stackless python is in sync with the latest 2.x, 3.x branches > as well as the trunk so performance > problems he may with performance in general are perhaps best resolved > by this lot. He said he is using Stackless with the obsolete 2.5.2 release, not even the final and best 2.5.4 release with whatever bug fixes and speedups it has. Switching to the latter, at least, would be my first suggestion. Terry From ziade.tarek at gmail.com Wed Jun 24 21:54:16 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 24 Jun 2009 21:54:16 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: References: <94bdd2610906220623k301e7924i5abc0a97d54741ba@mail.gmail.com> <94bdd2610906220842y475aad09l27f5be4b185718a4@mail.gmail.com> <94bdd2610906230138o186c5fa5h27ed7d857cd9284b@mail.gmail.com> <4A40A721.6090204@gmail.com> <4A4130D9.5040108@activestate.com> Message-ID: <94bdd2610906241254m60d39c2cs99d255dc5223af91@mail.gmail.com> On Wed, Jun 24, 2009 at 11:18 AM, Antoine Pitrou wrote: > Sridhar Ratnakumar activestate.com> writes: >> >> On 09-06-23 02:57 AM, Nick Coghlan wrote: >> > Something like DistributionDirectoryMap should cover it. >> > >> > You could probably get away with shortening "Directory" to "Dir" in the >> > class names though: >> > >> > ? - Distribution >> > ? - ZippedDistribution >> > ? - DistributionDir >> > ? - ZippedDistributionDir >> > ? - DistributionDirMap >> >> 'Map' reminds me of an associated hash (or is it the actual map?). > > Good thing, because it is a dict subclass if you read the PEP :) > > +1 with Nick's proposal. > I've changed using these names. Any other feedback with this PEP or we're good to go ? :) Regards Tarek From mail at timgolden.me.uk Wed Jun 24 22:05:10 2009 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 24 Jun 2009 21:05:10 +0100 Subject: [Python-Dev] Python for Windows?? In-Reply-To: <4A424D55.6010404@cherokeetel.net> References: <4A424D55.6010404@cherokeetel.net> Message-ID: <4A4286F6.5090102@timgolden.me.uk> David H. Burns wrote: > I downloaded what claims to be Python for Windows (3.01). The tutorial > brags a lot about how easy it is to learn, but the tutorials and > instruction seem to be for a Linux or Unix version. There are three > executable programs in the Python directory and no indication which > should be used to start Python. One opens a Dos-like window presumably > for "command-line" entry. I can't make anything of it. [as pointed out elsewhere, this is the wrong list for this question, but just to get you going...] The Python docs have a section on running Python on Windows. This is the online version: http://docs.python.org/using/windows.html The .chm version should be in c:\python31\doc\python31.chm TJG From horos11 at gmail.com Thu Jun 25 02:12:45 2009 From: horos11 at gmail.com (Edward Peschko) Date: Wed, 24 Jun 2009 17:12:45 -0700 Subject: [Python-Dev] trace module options. Message-ID: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com> All, I've been looking at the trace module, and although it looks useful, I'm surprised that there aren't a couple of features that I would have thought would be fairly basic. So, does trace support (for the --trace option): - indentation tracking stacklevel (where each function is prefixed by tabs equal to the number of stacklevels deep in the program) - output to something other than sys.stdout (eg. output to a file specified either by environmental variable or by parameter). - mult-threaded programs going to multiple output handles, especially in light of the above - fully qualified python modules in path: (eg: /path/to/module/my_module.py(1): print "HERE" instead of my_module.py(1): print "HERE". Ultimately, I'd like to be able to look at two runs of a program and be able to pinpoint the very first difference between thembased on the output of their trace runs. As it stands, I really can't do this. Of course I could program the above, but I was hoping to avoid duplicated effort if someone has already programmed options like this.. Ed From martin at v.loewis.de Thu Jun 25 09:30:54 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 25 Jun 2009 09:30:54 +0200 Subject: [Python-Dev] trace module options. In-Reply-To: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com> References: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com> Message-ID: <4A4327AE.7000907@v.loewis.de> > Of course I could program the above, but I was > hoping to avoid duplicated effort if someone has > already programmed options like this.. In this specific form, the question is off-topic for python-dev, and better asked on python-list (in particular since it has a wider audience). On-topic would have been a disussion of which of these features would be useful for the trace module to be built-in, how to best provide them, along with an offer to produce a patch. Regards, Martin From ncoghlan at gmail.com Thu Jun 25 12:11:10 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 25 Jun 2009 20:11:10 +1000 Subject: [Python-Dev] Python for Windows?? In-Reply-To: <4A4286F6.5090102@timgolden.me.uk> References: <4A424D55.6010404@cherokeetel.net> <4A4286F6.5090102@timgolden.me.uk> Message-ID: <4A434D3E.4090802@gmail.com> Tim Golden wrote: > The Python docs have a section on running Python > on Windows. This is the online version: > > http://docs.python.org/using/windows.html > > The .chm version should be in c:\python31\doc\python31.chm Which is more easily accessed via the start menu entry Python creates for you... (as is the interpreter itself for that matter) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From horos11 at gmail.com Thu Jun 25 14:54:21 2009 From: horos11 at gmail.com (Edward Peschko) Date: Thu, 25 Jun 2009 05:54:21 -0700 Subject: [Python-Dev] trace module options. In-Reply-To: <4A4327AE.7000907@v.loewis.de> References: <5cfa99000906241712n6bfbb167n8b8652522a9be7f0@mail.gmail.com> <4A4327AE.7000907@v.loewis.de> Message-ID: <5cfa99000906250554p67ab3038y5a3e6be3f9c6a48c@mail.gmail.com> > On-topic would have been a disussion of which of these features would > be useful for the trace module to be built-in, how to best provide them, > along with an offer to produce a patch. Ok, fair enough - which of the features listed (indentation matching stacklevel, output to a file (or files wrt threading, fully qualified output) should be builtin, which should be options, which should be left out, and based on the above, would a patch like this be accepted? Ed (ps - I'll also take it to python-list) From lanyjie at yahoo.com Thu Jun 25 17:40:58 2009 From: lanyjie at yahoo.com (Yingjie Lan) Date: Thu, 25 Jun 2009 08:40:58 -0700 (PDT) Subject: [Python-Dev] Syntax for parsing tuples/lists into C arrays Message-ID: <487033.84090.qm@web54208.mail.re2.yahoo.com> The current C API does not support directly parsing into C arrays. Parsing tuples into C arrays can be greatly facilitated with the introduction of a natural extension of the current parsing syntax. I'd like to propose a syntax extension to facilitate that kind of tasks, and would appreciate your feedbacks. The extension comes from the way strings are parsed in current C API: { const char * s; int i; PyArg_ParseTyple(args, "s#", &s, &i); } Consider for a moment the string as a sequence of char's, which by the example above is parsed into an array of char's. Note that in python, a tuple of some type (e.g. int's), is also a sequence of that type, This perspective provides a natural way to extend the the use of "#" symbol -- what is missing is how to signal a tuple as a sequence, without interfering with current fixed-length tuple parsing syntax. To signal a tuple as a sequence of a given type of objects, "(type:)" can be used. For example, "(i:)" signals a tuple of integers as array of integers. The ":" is chosen to denote possible repetition of the whole pattern inside the parentheses. Now consider 3 different scenarios of parsing tuples into arrays. Scenario (1) Unkown size. Here are some examples: { //parsing a tuple and its size int i, *t; PyArg_ParseTuple(args, "(i:)#", &t, &i); } { //parsing a matrix of i rows and j columns: int i, j, **t; PyArg_ParseTuple(args, "((i:):)##", &t, &i, &j); } Here is an informal explanation of "((i:):)##": the '#' can be seen as a peeling operator, which peels away the outer most "(...:)" that denote a "free sequence" (i.e. sequence of unknown size) with some pattern "...". For types like 's' or its similar types, pretend as if there is an invisible pair of parentheses around it. For example, "(s:)##" matches a tuple of strings of some fixed length. Here the first '#' peels away the parentheses, leaving only "s#", which is equivalent to "(*char*:)#" in concept, where "*char*" denote the character type (not exist in Python). As the second '#' is outside the tuple, it is shared by all strings inside, so they _must_ have the same length. If there is nothing to peel for a '#', it is considered a dangling '#', an error. Now consider some more interesting examples: { int i, *t, *v; PyArg_ParseTuple(args, "(ii:)#", &t, &v, &i); //args=(1,2,3,4,5,6) //t={1,3,5} //v={2,4,6} //i=3 } In that example, a pattern is provided inside the sequence "(ii:)", which can repeat itself multiple times. The repeated times is extracted by the '#' symbol. Below is another example: { int i,j, *t, **v; PyArg_ParseTuple(args, "(i(i:):)##", &t, &v, &i, &j); //args=(1,(3,),5,(7,)) //t={1,5} //v={{3},{7}} //i=2, j=1 } And another one: { int i,j, **t, **v; PyArg_ParseTuple(args, "((i:)(i:):)##", &t, &v, &i, &j); //args=((1,),(3,),(5,),(7,)) //t={{1},{5}} //v={{3},{7}} //i=2, j=1 } Note the second '#' is applied to both the free sequences inside the outer tuple. This indicates that all the inside tuples must be of the same length. What if they have their own lengths? We provide a separate matching '#' for each inside tuple: { int i,j,k, **t, **v; PyArg_ParseTuple(args, "((i:)(i:):)#(##)", &t, &v, &i, &j,&k); //args=((1,),(3,4),(5,),(7,8)) //t={{1},{5}} //v={{3,4},{7,8}} //i=2, j=1, k=2 } The syntax can also deal with more complicated situations, such as an irregular 2-dimensional array, where each row has a different row length: { int i, *j, **t; PyArg_ParseTuple(args, "((i:)#:)#", &t, &j, &i); for(a=0; a) //t={{{1,2},{1,2}},{{1,2}}} //i=2, k=2 //j={2,1} } This syntax can be used for parsing a tuple of strings of different lengths: { int i, *j; const char ** s; PyArg_ParseTuple(args, "(s#:)#", &s, &j, &i); //args=("ab", "cde") //s={"ab", "cde"} //j={2, 3} //i=2 } Scenario (2) If you know the size of a tuple before hand, use this syntax: "(*type*integer)", meaning there are exactly 'integer' number of *type* objects in the tuple. { int *t; // PyArg_ParseTuple(args, "(i5)", &t); //args = (1,2,3,4,5) //t = {1,2,3,4,5} } A complicated case: { int **t; PyArg_ParseTuple(args, "((i5)2)", &t); //args=((1,2,3,4,5),(5,6,7,8,9)) //t={{1,2,3,4,5},{5,6,7,8,9}} } Another complicated case: { int **t, **v, i; PyArg_ParseTuple(args, "(i3i2:)#", &t,&v,&i); //args=(1,2,3,4,5,6,7,8,9,10) //t={{1,2,3},{6,7,8}} //v={{4,5},{9,10}} //i=2 } One more thing: the difference between "(ii)" and "(i2)" is that the former yields two single integers, while the latter gives an array of length two. Scenario (3) Concatenating arrays while parsing. Sometimes you would like to have a matrix parsed into a single array. We can use a "(+" to signal that the arrays parsed from the elements in that tuple should be concatenated into a single array. { int i,j, *t; PyArg_ParseTuple(args, "(+(i))##", &t, &i, &j); //args=((1,2,3),(4,5,6)) //t={1,2,3,4,5,6} //i=2, j=3 } Another example: { int i, *t; PyArg_ParseTuple(args, "(+(i2))#", &t, &i); } Still another example, with strings: { int i, *j; const char * t; //string PyArg_ParseTuple(args, "(+(+s#2))#", &t, &j, &i); //args=(("ab","c"),("d","efg")) //t="abcdefg" //i=2 ////notice the size arrays are concatenated similarly //j={2,1,1,3} //without '+': j={{2,1},{1,3}} const char ** sa; //The '*' in place of '#' to discard the size info PyArg_ParseTuple(args, "(+(s*2))", &sa); //args=(("ab","c"),("d","efg")) //sa = {"ab","c","d","efg"} PyArg_ParseTuple(args, "((+s*2))", &sa); //args=(("ab","c"),("d","efg")) //sa = { "abc", "defg" } } And finally, an example with multi-item pattern: { int *t, *v, i; PyArg_ParseTuple(args, "(+i1i2:)#", &t, &v, &i); //args=(1,2,3,4,5,6) //t={1,4} //v={2,3,5,6} //i=2 } Epilogue: This syntax can be applied to lists: just use "[...:]" instead of "(...:)". It can also be used for building tuples or lists out of arrays. That's it. Comments? Yingjie From aahz at pythoncraft.com Thu Jun 25 19:49:22 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 25 Jun 2009 10:49:22 -0700 Subject: [Python-Dev] Syntax for parsing tuples/lists into C arrays In-Reply-To: <487033.84090.qm@web54208.mail.re2.yahoo.com> References: <487033.84090.qm@web54208.mail.re2.yahoo.com> Message-ID: <20090625174922.GB2323@panix.com> On Thu, Jun 25, 2009, Yingjie Lan wrote: > > The current C API does not support directly parsing into C > arrays. Parsing tuples into C arrays can be greatly facilitated > with the introduction of a natural extension of the current parsing > syntax. I'd like to propose a syntax extension to facilitate that kind > of tasks, and would appreciate your feedbacks. This should probably get reposted to python-ideas. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha From martin at v.loewis.de Thu Jun 25 20:40:47 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 25 Jun 2009 20:40:47 +0200 Subject: [Python-Dev] Syntax for parsing tuples/lists into C arrays In-Reply-To: <487033.84090.qm@web54208.mail.re2.yahoo.com> References: <487033.84090.qm@web54208.mail.re2.yahoo.com> Message-ID: <4A43C4AF.5000907@v.loewis.de> > Comments? I don't think there is any need to support that in the Python core. Instead, if you find such a functionality useful, feel free to implement it yourself, and share it with others. Regards, Martin From martin at v.loewis.de Thu Jun 25 21:30:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 25 Jun 2009 21:30:11 +0200 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE DEVELOPERS In-Reply-To: <006801c9f4dd$c18294c0$011ea8c0@pegasus> References: <006801c9f4dd$c18294c0$011ea8c0@pegasus> Message-ID: <4A43D043.5080406@v.loewis.de> > Can you tell me which modules provide to > decode and execute a .pyc file ? The bytecode loader in in marshal.c; the byte code interpreter is in ceval.c. Also consider frameobject.c and codeobject.c. HTH, Martin From barry at python.org Fri Jun 26 15:30:24 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 26 Jun 2009 09:30:24 -0400 Subject: [Python-Dev] [Python-checkins] r73569 - peps/trunk/pep-0101.txt In-Reply-To: <20090626124903.C330BE3B84@mail.wooz.org> References: <20090626124903.C330BE3B84@mail.wooz.org> Message-ID: <221034BF-8E59-4557-AEED-87A64E85A143@python.org> On Jun 26, 2009, at 8:49 AM, benjamin.peterson wrote: > Author: benjamin.peterson > Date: Fri Jun 26 14:48:55 2009 > New Revision: 73569 > > Log: > update release candidate shorthand > > Modified: > peps/trunk/pep-0101.txt > > Modified: peps/trunk/pep-0101.txt > = > = > = > = > = > = > = > = > ====================================================================== > --- peps/trunk/pep-0101.txt (original) > +++ peps/trunk/pep-0101.txt Fri Jun 26 14:48:55 2009 > @@ -66,7 +66,7 @@ > > We use the following conventions in the examples below. Where a > release > number is given, it is of the form X.YaZ, e.g. 2.6a3 for Python > 2.6 alpha > - 3, where "a" == alpha, "b" == beta, "c" == release candidate. > + 3, where "a" == alpha, "b" == beta, "rc" == release candidate. > > Final releases are named "releaseXY". The branch tag is > "releaseXY-maint" > because this will point to the long lived maintenance branch. > The fork I'm sure this has been discussed but I missed it. Why was this change made? If nothing else, it breaks many years of tradition. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From benjamin at python.org Fri Jun 26 15:35:00 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 26 Jun 2009 08:35:00 -0500 Subject: [Python-Dev] [Python-checkins] r73569 - peps/trunk/pep-0101.txt In-Reply-To: <221034BF-8E59-4557-AEED-87A64E85A143@python.org> References: <20090626124903.C330BE3B84@mail.wooz.org> <221034BF-8E59-4557-AEED-87A64E85A143@python.org> Message-ID: <1afaf6160906260635m3ca2bbbara439991adf70e0f0@mail.gmail.com> 2009/6/26 Barry Warsaw : > I'm sure this has been discussed but I missed it. ?Why was this change made? > ?If nothing else, it breaks many years of tradition. I assumed it was the tradition because all the 3.0 and 2.6 candidates had "rc". -- Regards, Benjamin From Scott.Daniels at Acm.Org Fri Jun 26 17:29:22 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Fri, 26 Jun 2009 08:29:22 -0700 Subject: [Python-Dev] [Python-checkins] r73569 - peps/trunk/pep-0101.txt In-Reply-To: <221034BF-8E59-4557-AEED-87A64E85A143@python.org> References: <20090626124903.C330BE3B84@mail.wooz.org> <221034BF-8E59-4557-AEED-87A64E85A143@python.org> Message-ID: While we are on that, I just noticed: http://www.python.org/download/releases/3.1/ Has downloads for 3.1rc2, but show checksums as if for 3.1rc1 The size and checksum is correct for python-3.1rc2.msi, distinct from that for python-3.1rc1.msi, but are labeled as rc1. The 32-bit .msi is the only one of the four I checked; I suspect the other three are similarly mislabeled. --Scott David Daniels Scott.Daniels at Acm.Org From status at bugs.python.org Fri Jun 26 18:07:17 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 26 Jun 2009 18:07:17 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090626160717.9D56E78617@psf.upfronthosting.co.za> ACTIVITY SUMMARY (06/19/09 - 06/26/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2242 open (+17) / 15925 closed (+16) / 18167 total (+33) Open issues with patches: 889 Average duration of open issues: 657 days. Median duration of open issues: 408 days. Open Issues Breakdown open 2212 (+17) pending 30 ( +0) Issues Created Or Reopened (33) _______________________________ test_with.py has a couple minor mistakes 06/19/09 CLOSED http://bugs.python.org/issue6313 created exarkun patch logging.basicConfig(level='DEBUG', ... 06/20/09 CLOSED http://bugs.python.org/issue6314 created alexl locale._build_localename(locale.getdefaultlocale()) returns 'C.m 06/20/09 CLOSED http://bugs.python.org/issue6315 created sjt format, str.format don't work well with datetime, date object 06/20/09 CLOSED http://bugs.python.org/issue6316 created falsetru winsound.PlaySound doesn't accept non-unicode string 06/21/09 http://bugs.python.org/issue6317 created ocean-city patch HTMLParser Attributes Containing Escaped Quotes 06/21/09 CLOSED http://bugs.python.org/issue6318 created ericryk bdist_msi runs out of memory for large packages 06/21/09 CLOSED http://bugs.python.org/issue6319 created bethard patch Standard string encodings should include GSM0.38 06/21/09 http://bugs.python.org/issue6320 created jwishnie Reload Python modules when running programs 06/22/09 http://bugs.python.org/issue6321 created samwyse Pdb breakpoints don't work on lines without bytecode 06/22/09 http://bugs.python.org/issue6322 created inducer Py3.1 pdb doesn't deal well with syntax errors 06/22/09 http://bugs.python.org/issue6323 created inducer patch, needs review "in" expression falls back to __iter__ before __getitem__ 06/22/09 http://bugs.python.org/issue6324 created afoglia robotparser doesn't handle URL's with query strings 06/23/09 http://bugs.python.org/issue6325 created skybrian Add a "swap" method to list 06/23/09 http://bugs.python.org/issue6326 created krisvale patch, patch [mimetext] long lines get cut with exclamation mark and newline 06/23/09 http://bugs.python.org/issue6327 created martijntje login() function failed in smtplib with message "argument 1 must 06/23/09 CLOSED http://bugs.python.org/issue6328 created hdvision Fix iteration for memoryviews 06/23/09 CLOSED http://bugs.python.org/issue6329 created rhettinger patch trunk does not build with --enable-unicode=ucs4 06/23/09 http://bugs.python.org/issue6330 created eric.smith easy Add unicode script info to the unicode database 06/23/09 http://bugs.python.org/issue6331 created doerwalter patch typo on man page warning control 06/23/09 CLOSED http://bugs.python.org/issue6332 created lieryan patch logging: ValueError: I/O operation on closed file 06/24/09 http://bugs.python.org/issue6333 created srid 3.0/3.1: Bad bug in range() computation (or possible Integer pro 06/24/09 CLOSED http://bugs.python.org/issue6334 created mfxmfx patch Add support for mingw 06/24/09 http://bugs.python.org/issue6335 created smartmobili patch nb_divide missing in docs 06/24/09 http://bugs.python.org/issue6336 created flub multiprocessing module: Double close of sys.stdin - ID: 2811568 06/24/09 CLOSED http://bugs.python.org/issue6337 created subdir patch Error message displayed on stderr when no C compiler is present 06/24/09 http://bugs.python.org/issue6338 created ehuss patch Some functional errors in turtle.py documentation (missing links 06/24/09 CLOSED http://bugs.python.org/issue6339 created gregorlingl replace tdemo_chaos.py 06/24/09 CLOSED http://bugs.python.org/issue6340 created gregorlingl io.path.ismount gives "local variable 'p' referenced before ass 06/25/09 CLOSED http://bugs.python.org/issue6341 created g.moralis io.path.ismount gives "local variable 'p' referenced before ass 06/25/09 CLOSED http://bugs.python.org/issue6342 created g.moralis TimedRotatingFileHandler permission denied rename failure on Win 06/25/09 http://bugs.python.org/issue6343 created dnagata mmap.read() crashes when passed a negative argument 06/25/09 http://bugs.python.org/issue6344 created amaury.forgeotdarc extra characters displayed when writing to sys.stdout.buffer 06/26/09 CLOSED http://bugs.python.org/issue6345 created metolone Issues Now Closed (31) ______________________ zlib.crc32() and adler32() return value 263 days http://bugs.python.org/issue1202 gregory.p.smith 64bit Tix HList class missing method implementation for info_bbox 627 days http://bugs.python.org/issue1230 gpolo xml/sax/expatreader.py raises AttributeError when run 202 days http://bugs.python.org/issue4490 amaury.forgeotdarc patch test_tcl testLoadTk fails if DISPLAY defined but connect fails, 105 days http://bugs.python.org/issue5450 gpolo patch pyexpat defines global symbol template_string 86 days http://bugs.python.org/issue5590 doko SimpleXMLRPCServer not suitable for HTTP/1.1 keep-alive 27 days http://bugs.python.org/issue6096 loewis patch, patch, easy, needs review HTTP/1.1 with keep-alive support for xmlrpclib.ServerProxy 27 days http://bugs.python.org/issue6099 loewis patch, patch, needs review Backport the IO lib to trunk 15 days http://bugs.python.org/issue6215 pitrou patch subprocess.Popen() may leak file descriptors 7 days http://bugs.python.org/issue6274 facundobatista patch Update contextlib.nested docs in light of deprecation 6 days http://bugs.python.org/issue6288 ncoghlan virtual memory error with archivemail 0 days http://bugs.python.org/issue6311 loewis test_with.py has a couple minor mistakes 0 days http://bugs.python.org/issue6313 benjamin.peterson patch logging.basicConfig(level='DEBUG', ... 2 days http://bugs.python.org/issue6314 vsajip locale._build_localename(locale.getdefaultlocale()) returns 'C.m 5 days http://bugs.python.org/issue6315 r.david.murray format, str.format don't work well with datetime, date object 0 days http://bugs.python.org/issue6316 falsetru HTMLParser Attributes Containing Escaped Quotes 0 days http://bugs.python.org/issue6318 georg.brandl bdist_msi runs out of memory for large packages 0 days http://bugs.python.org/issue6319 bethard patch login() function failed in smtplib with message "argument 1 must 0 days http://bugs.python.org/issue6328 r.david.murray Fix iteration for memoryviews 0 days http://bugs.python.org/issue6329 rhettinger patch typo on man page warning control 0 days http://bugs.python.org/issue6332 georg.brandl patch 3.0/3.1: Bad bug in range() computation (or possible Integer pro 1 days http://bugs.python.org/issue6334 mfxmfx patch multiprocessing module: Double close of sys.stdin - ID: 2811568 0 days http://bugs.python.org/issue6337 jnoller patch Some functional errors in turtle.py documentation (missing links 1 days http://bugs.python.org/issue6339 r.david.murray replace tdemo_chaos.py 1 days http://bugs.python.org/issue6340 r.david.murray io.path.ismount gives "local variable 'p' referenced before ass 0 days http://bugs.python.org/issue6341 r.david.murray io.path.ismount gives "local variable 'p' referenced before ass 0 days http://bugs.python.org/issue6342 amaury.forgeotdarc extra characters displayed when writing to sys.stdout.buffer 0 days http://bugs.python.org/issue6345 benjamin.peterson curselection() in Tkinter.py should return ints 1997 days http://bugs.python.org/issue869780 gpolo patch Add XML-RPC Fault Interoperability to XMLRPC libraries 1310 days http://bugs.python.org/issue1360243 loewis patch, easy Tix ComboBox entry is blank when not editable 944 days http://bugs.python.org/issue1600182 gpolo syslog syscall support for SysLogLogger 782 days http://bugs.python.org/issue1711603 LwarX patch Top Issues Most Discussed (10) ______________________________ 9 Native (and default) tarfile support for setup.py sdist in dist 9 days open http://bugs.python.org/issue6296 9 ElementTree (py3k) doesn't properly encode characters that can' 19 days open http://bugs.python.org/issue6233 9 Python 2.6 makes .pyc/.pyo bytecode files executable 37 days open http://bugs.python.org/issue6070 7 3.0/3.1: Bad bug in range() computation (or possible Integer pr 1 days closed http://bugs.python.org/issue6334 6 cElementTree.iterparse & ElementTree.iterparse return different 15 days open http://bugs.python.org/issue6266 5 Add a "swap" method to list 3 days open http://bugs.python.org/issue6326 5 Py3.1 pdb doesn't deal well with syntax errors 4 days open http://bugs.python.org/issue6323 5 os.popen causes illegal seek on AIX in Python 3.1rc 19 days open http://bugs.python.org/issue6236 5 test_tcl testLoadTk fails if DISPLAY defined but connect fails, 105 days closed http://bugs.python.org/issue5450 5 Document auto __ne__ generation; provide a use case for non-tri 215 days open http://bugs.python.org/issue4395 From fbattaglia at alice.it Fri Jun 26 17:45:15 2009 From: fbattaglia at alice.it (Filippo Battaglia) Date: Fri, 26 Jun 2009 17:45:15 +0200 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE Message-ID: <000c01c9f675$19917760$011ea8c0@pegasus> Thanks for your answers. Sorry for the title in upper case. I didn't want to create troubles. :) I've an important question for you: is it possible that a large python module, created using SWIG and with a hundred of routines, makes slower the execution (i.e. the job of ceval.c) of the Python interpreter ? We've observed that, if we don't import ndpsp.pyc at startup, the time of execution of a loop containing the pass instruction becomes near normal. How Python recalls the C functions in a C wrapper ? Thanks for your very important help. Filippo Battaglia From tjreedy at udel.edu Fri Jun 26 22:25:02 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 26 Jun 2009 16:25:02 -0400 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE In-Reply-To: <000c01c9f675$19917760$011ea8c0@pegasus> References: <000c01c9f675$19917760$011ea8c0@pegasus> Message-ID: Filippo Battaglia wrote: > Thanks for your answers. Sorry for the title in upper case. I didn't > want to create troubles. > :) > > I've an important question for you: is it > possible that a large python module, > created using SWIG and with a hundred > of routines, makes slower the execution > (i.e. the job of ceval.c) of the Python > interpreter ? If you were running on a PC with what is now considered to be very small memory, I would hypothesize that you had filled memory so that the interpreter or parts thereof were being swapped in and out of memory from and to disk. Is any thing like that possible with the PSP? Next, I would wonder whether any module, as part of its initialization, was doing anything 'unusual' with respect to its interaction with the interpreter. > > We've observed that, if we don't import > ndpsp.pyc at startup, the time of execution > of a loop containing the pass instruction > becomes near normal. What happens if you divide the imported stuff in half? Do both halves slow it down? Neither? Just one? The answer to that would be a start to answering whether the specific problem is quantitative or qualitative. Terry Jan Reedy From flormayer at aim.com Sat Jun 27 20:43:18 2009 From: flormayer at aim.com (Florian Mayer) Date: Sat, 27 Jun 2009 18:43:18 +0000 Subject: [Python-Dev] pthread sem PyThread_acquire_lock Message-ID: <4A466846.1050904@aim.com> Hello. As this is my first post I will try to introduce myself as requested in the welcome email. If you aren't interested in my person, just continue reading at the next paragraph. I'm a student from Vienna/Austria. I attend what would match high school in the United States. I have been writing Python for about 3 years now and have just began to dig into the implementation of CPython. Now to my real question. I have noticed that PyThread_acquire_lock swallows all signals with pthread using sems. Looking at the source code it appears that this is intended, but I cannot see the reason for that. It seems the pthread sem implementation is the only one doing so. Can any of you tell me the reason why it swallows signals? I have already prepared a patch that introduces a new _noswallow function if sem are used and uses this in threadmodule.c. But if there isn't any real reason for it to swallow exceptions, I think it would be wisest to change that. Best regards, Florian Mayer From martin at v.loewis.de Sat Jun 27 22:45:59 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 27 Jun 2009 22:45:59 +0200 Subject: [Python-Dev] pthread sem PyThread_acquire_lock In-Reply-To: <4A466846.1050904@aim.com> References: <4A466846.1050904@aim.com> Message-ID: <4A468507.1060503@v.loewis.de> > Now to my real question. I have noticed that PyThread_acquire_lock > swallows all signals with pthread using sems. Looking at the source code > it appears that this is intended, but I cannot see the reason for that. > It seems the pthread sem implementation is the only one doing so. Can > any of you tell me the reason why it swallows signals? Can you please explain what you mean by "swallowing"? What is the sequence of actions triggering the case you are talking about, what happens, and what do you expect to happen instead? Also, how would you fix this (in principle, not what the specific patch would look like)? Regards, Martin From flormayer at aim.com Sun Jun 28 00:50:18 2009 From: flormayer at aim.com (Florian Mayer) Date: Sat, 27 Jun 2009 22:50:18 +0000 Subject: [Python-Dev] pthread sem PyThread_acquire_lock In-Reply-To: <4A468507.1060503@v.loewis.de> References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de> Message-ID: <4A46A22A.40400@aim.com> Martin v. L?wis wrote: > Can you please explain what you mean by "swallowing"? The signals don't get through when acquiring the lock. > What is the sequence of actions triggering the case you are talking > about, what happens, and what do you expect to happen instead? Create a Lock object, call .acquire and try to press Ctrl-C, the keyboardinterrupt is swallowed > Also, how would you fix this (in principle, not what the specific > patch would look like)? Remove the loop that explicitly does this in a new function and use that one in threadmodule.c for lock.acquire. > > Regards, > Martin Best regards, Florian PS: Sorry for sending this message twice. From flormayer at aim.com Sun Jun 28 00:58:19 2009 From: flormayer at aim.com (Florian Mayer) Date: Sat, 27 Jun 2009 22:58:19 +0000 Subject: [Python-Dev] pthread sem PyThread_acquire_lock In-Reply-To: <4A468507.1060503@v.loewis.de> References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de> Message-ID: <4A46A40B.5080202@aim.com> Martin v. L?wis wrote: > Can you please explain what you mean by "swallowing"? The signals don't get through when acquiring the lock. > What is the sequence of actions triggering the case you are talking > about, what happens, and what do you expect to happen instead? Easiest way is to create a Queue.Queue and call .get() on it and try to Ctrl-C it. > Also, how would you fix this (in principle, not what the specific > patch would look like)? Remove the loop that explicitly does this in a new function and use that one in threadmodule.c for lock.acquire. > > Regards, > Martin Best regards, Florian PS: Excuse me, I messed up things once again. From benjamin at python.org Sat Jun 27 23:12:10 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 27 Jun 2009 16:12:10 -0500 Subject: [Python-Dev] [RELEASED] Python 3.1 final Message-ID: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> On behalf of the Python development team, I'm thrilled to announce the first production release of Python 3.1. Python 3.1 focuses on the stabilization and optimization of the features and changes that Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. Other features include an ordered dictionary implementation, a condensed syntax for nested with statements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/3.1/whatsnew/3.1.html or Misc/NEWS in the Python distribution. To download Python 3.1 visit: http://www.python.org/download/releases/3.1/ The 3.1 documentation can be found at: http://docs.python.org/3.1 Bugs can always be reported to: http://bugs.python.org Enjoy! -- Benjamin Peterson Release Manager benjamin at python.org (on behalf of the entire python-dev team and 3.1's contributors) From python at rcn.com Sun Jun 28 00:47:56 2009 From: python at rcn.com (Raymond Hettinger) Date: Sat, 27 Jun 2009 15:47:56 -0700 Subject: [Python-Dev] [RELEASED] Python 3.1 final References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> Message-ID: <0035D0ABC9D34D12B5D695005B39D9AC@RaymondLaptop1> > On behalf of the Python development team, I'm thrilled to announce the first > production release of Python 3.1. Sweet! Raymond From lists at cheimes.de Sun Jun 28 00:59:48 2009 From: lists at cheimes.de (Christian Heimes) Date: Sun, 28 Jun 2009 00:59:48 +0200 Subject: [Python-Dev] [RELEASED] Python 3.1 final In-Reply-To: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> Message-ID: <4A46A464.2090704@cheimes.de> Benjamin Peterson schrieb: > On behalf of the Python development team, I'm thrilled to announce the first > production release of Python 3.1. Gratulations! You did a fantastic job! :) Christian PS: I hope that I'm able to spare more of my work time on the development of Python 3.2 ... From arcriley at gmail.com Sun Jun 28 01:12:16 2009 From: arcriley at gmail.com (Arc Riley) Date: Sat, 27 Jun 2009 19:12:16 -0400 Subject: [Python-Dev] [RELEASED] Python 3.1 final In-Reply-To: <4A46A464.2090704@cheimes.de> References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> <4A46A464.2090704@cheimes.de> Message-ID: On Sat, Jun 27, 2009 at 6:59 PM, Christian Heimes wrote: > > Gratulations! You did a fantastic job! :) +1 ! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sun Jun 28 02:53:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 28 Jun 2009 10:53:49 +1000 Subject: [Python-Dev] [RELEASED] Python 3.1 final In-Reply-To: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> Message-ID: <4A46BF1D.2090002@gmail.com> Benjamin Peterson wrote: > On behalf of the Python development team, I'm thrilled to announce the first > production release of Python 3.1. Excellent news! Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From g.brandl at gmx.net Sun Jun 28 12:18:54 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 28 Jun 2009 10:18:54 +0000 Subject: [Python-Dev] [RELEASED] Python 3.1 final In-Reply-To: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> References: <1afaf6160906271412v3ca3ef9bo4efa4523db3a3685@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > On behalf of the Python development team, I'm thrilled to announce the first > production release of Python 3.1. Congrats! Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Mon Jun 29 09:39:32 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 29 Jun 2009 09:39:32 +0200 Subject: [Python-Dev] 2.4.7: Final 2.4 release Message-ID: <4A486FB4.9070008@v.loewis.de> Python 2.4 will become 5 years old this November. I plan to make the final security release this month or next month. If you want to see additional patches in Python 2.4, please let us now, or commit them yourself if you can. Remember that only security fixes can be considered for inclusion, and preferably only fixes that have already been applied to later releases of Python. Regards, Martin From asmodai at in-nomine.org Mon Jun 29 11:54:36 2009 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 29 Jun 2009 11:54:36 +0200 Subject: [Python-Dev] ndPython: I NEED TO TALK WITH ONE OF THE PYTHON CORE In-Reply-To: References: <000c01c9f675$19917760$011ea8c0@pegasus> Message-ID: <20090629095436.GH65921@nexus.in-nomine.org> -On [20090626 22:29], Terry Reedy (tjreedy at udel.edu) wrote: >If you were running on a PC with what is now considered to be very small >memory, I would hypothesize that you had filled memory so that the >interpreter or parts thereof were being swapped in and out of memory >from and to disk. Is any thing like that possible with the PSP? >From what I know it has either 32 or 64 MB of RAM, depending on the model. The storage comes in the form of a Memory Stick PRO Duo, but I doubt it is used in any form as a paging or swap file. At least playing on my own it only accesses it when I want to save game data. -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Friendship is love without wings... From ziade.tarek at gmail.com Mon Jun 29 19:57:56 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 29 Jun 2009 19:57:56 +0200 Subject: [Python-Dev] PEP 376 Message-ID: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> Hello, If no one objects, I'd like to push PEP 376 in the "accepted" status and go ahead with its implementation, with continuous feedback at Distutils-SIG as we did to build it. The next PEPs that are being discussed at Distutils-SIG are : - the new version of PEP 345 for the inclusion of fields like "installed_requires" (dependencies) - PEP 386, that talks about distribution version numbers (because it's needed to describe dependencies) While they are still under heavy discussion, I have good hope that they will both make it for 2.7 and 3.2. Regards Tarek -- Tarek Ziad? | http://ziade.org From phillip.sitbon+python-dev at gmail.com Mon Jun 29 21:33:00 2009 From: phillip.sitbon+python-dev at gmail.com (Phillip Sitbon) Date: Mon, 29 Jun 2009 12:33:00 -0700 Subject: [Python-Dev] pthread sem PyThread_acquire_lock In-Reply-To: <4A46A40B.5080202@aim.com> References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de> <4A46A40B.5080202@aim.com> Message-ID: <536685ea0906291233r1b3e8dc4ndf25e7f404a87e22@mail.gmail.com> I'll do my best to try and explain/contribute, but please feel free to correct anything I get wrong. I believe the "swallowing" he's referring to is the ignoring of errno EINTR. I don't think that's the correct place to handle signals to begin with- why not just use the signal module to deal with such a scenario? http://docs.python.org/dev/library/signal.html#module-signal AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers. There are many functions that don't return this value anymore, making them reentrant. I remember a number of years ago when it wasn't part of any standard to return EINTR or not, and so the only way to provide consistent behavior was to ignore it and loop. I'm not sure if that is still the case. A great example is reading from a socket. Whether or not it can be interrupted depends on the platform, so catching Ctrl+C often requires a timeout loop. Also, remember that signals are asynchronous in the sense that they are handled outside the normal execution flow of a program. Checking for EINTR probably isn't the best way to determine if a signal has been sent to the program. Cheers, Phillip On Sat, Jun 27, 2009 at 3:58 PM, Florian Mayer wrote: > > Martin v. L?wis wrote: >> >> Can you please explain what you mean by "swallowing"? > > The signals don't get through when acquiring the lock. >> >> What is the sequence of actions triggering the case you are talking >> about, what happens, and what do you expect to happen instead? > > Easiest way is to create a Queue.Queue and call .get() on it and try to Ctrl-C it. >> >> Also, how would you fix this (in principle, not what the specific >> patch would look like)? > > Remove the loop that explicitly does this in a new function and use that > one in threadmodule.c for lock.acquire. >> >> Regards, >> Martin > > Best regards, > Florian > > PS: Excuse me, I messed up things once again. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/phillip.sitbon%2Bpython-dev%40gmail.com From martin at v.loewis.de Mon Jun 29 22:45:09 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 29 Jun 2009 22:45:09 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> Message-ID: <4A4927D5.4040001@v.loewis.de> > If no one objects, I'd like to push PEP 376 in the "accepted" status > and go ahead with its implementation, > with continuous feedback at Distutils-SIG as we did to build it. I think this isn't quite the process. In the past, every PEP required BDFL pronouncement, which you should now seek. Regards, Martin From martin at v.loewis.de Mon Jun 29 23:28:07 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 29 Jun 2009 23:28:07 +0200 Subject: [Python-Dev] pthread sem PyThread_acquire_lock In-Reply-To: <536685ea0906291233r1b3e8dc4ndf25e7f404a87e22@mail.gmail.com> References: <4A466846.1050904@aim.com> <4A468507.1060503@v.loewis.de> <4A46A40B.5080202@aim.com> <536685ea0906291233r1b3e8dc4ndf25e7f404a87e22@mail.gmail.com> Message-ID: <4A4931E7.1020108@v.loewis.de> > AFAIK, ignoring EINTR doesn't preclude the calling of signal handlers. This is my understanding as well - so I don't think Python actually "swallows" the signal. > A great example is reading from a socket. Whether or not it can be > interrupted depends on the platform, so catching Ctrl+C often requires > a timeout loop. > > Also, remember that signals are asynchronous in the sense that they > are handled outside the normal execution flow of a program. Checking > for EINTR probably isn't the best way to determine if a signal has > been sent to the program. I think it would be reasonable to support "asynchronous" exceptions, and Python supports SIGINT fairly well most of the time. It might be possible to support keyboard interrupts throughout the system, but changing Python to do so could also cause incompatibilities. So any change must be done with greatest care, but simultaneously, should also try to arrange to cover all cases. Regards, Martin From jnoller at gmail.com Tue Jun 30 03:59:01 2009 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 29 Jun 2009 21:59:01 -0400 Subject: [Python-Dev] Anyone against having a loop option for regrtest? Message-ID: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com> Something that's been helping me squirrel out "wacky" and "fun" bugs in multiprocessing is running the tests in a loop - sometimes hundreds of times. Right now, I hack this up with a bash script, but I'm sitting here wondering if adding a "loop for x iterations" option to regrtest.py would be useful to others as well. Any thoughts? Does anyone hate this idea with the power of a thousand suns? -jesse From stephen at xemacs.org Tue Jun 30 04:23:52 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 30 Jun 2009 11:23:52 +0900 Subject: [Python-Dev] Anyone against having a loop option for regrtest? In-Reply-To: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com> References: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com> Message-ID: <87tz1ywjbb.fsf@uwakimon.sk.tsukuba.ac.jp> Jesse Noller writes: > Any thoughts? Does anyone hate this idea with the power of a thousand suns? If somebody has the power of 1000 Suns at their disposal, maybe they can contribute a few buildbots? Wishful-thinking-is-the-order-of-the-day-ly y'rs, From collinw at gmail.com Tue Jun 30 04:22:08 2009 From: collinw at gmail.com (Collin Winter) Date: Mon, 29 Jun 2009 19:22:08 -0700 Subject: [Python-Dev] Anyone against having a loop option for regrtest? In-Reply-To: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com> References: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com> Message-ID: <43aa6ff70906291922u4dc687bcm74bcbe38ba2d9a8f@mail.gmail.com> On Mon, Jun 29, 2009 at 6:59 PM, Jesse Noller wrote: > Something that's been helping me squirrel out "wacky" and "fun" bugs > in multiprocessing is running the tests in a loop - sometimes hundreds > of times. Right now, I hack this up with a bash script, but I'm > sitting here wondering if adding a "loop for x iterations" option to > regrtest.py would be useful to others as well. > > Any thoughts? Does anyone hate this idea with the power of a thousand suns? +1 for having this in regrtest. I've wished for this in the past, and ended up going the bash route, same as you. Collin From jnoller at gmail.com Tue Jun 30 04:41:49 2009 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 29 Jun 2009 22:41:49 -0400 Subject: [Python-Dev] Anyone against having a loop option for regrtest? In-Reply-To: <87tz1ywjbb.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4222a8490906291859o682a4aayb10cf8a7c798be06@mail.gmail.com> <87tz1ywjbb.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4222a8490906291941x593dd439u269d9791af17183a@mail.gmail.com> On Mon, Jun 29, 2009 at 10:23 PM, Stephen J. Turnbull wrote: > Jesse Noller writes: > > ?> Any thoughts? Does anyone hate this idea with the power of a thousand suns? > > If somebody has the power of 1000 Suns at their disposal, maybe they > can contribute a few buildbots? > > Wishful-thinking-is-the-order-of-the-day-ly y'rs, > > Sorry I had to take mine down. That being said, I need to rummage up a mac mini with a lot of ram and possibly fire up VMWare fusion/virtualbox on it to have an os/x/linux/windows buildbot mini farm. From 4vinoth at gmail.com Tue Jun 30 08:07:21 2009 From: 4vinoth at gmail.com (vin vin) Date: Tue, 30 Jun 2009 11:37:21 +0530 Subject: [Python-Dev] Flow of control - a new way - Idea In-Reply-To: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com> References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com> Message-ID: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com> HI all I am not the too technical guy, but thinking about the new way of controlling the flow instead of throwing an error. as of now if we need to break a control and go back, exceptions helps, but it is not a actual way. it would be great if we have a control over the frames execution, I mean A calls B which calls C which calls D at that point if we think to move directly to B (what error handler do if that B has the handler defined of the error), changing the frames instruction pointer to back to the B's position (in python code without raising an exception) is a really useful for this web applications. ----------------------------- How we can achieve this? a simple way? Actually python interpreter currently doing this. How? method *A* method *B* <== has exception handler try: except: method *C* method *D* <========= got exception here.. now exception handler checks that which caller methods has the exception handler provided for this exception. in our case method B has this. so, the instruction pointer has moved back to that method B with the exception. ------------------- We can have this same functionality* BUT WITHOUT THE USE OF EXCEPTION*. where it can help? so many places, & it gives the full power of execution control to the user. A simple usage of this is : Currently most of the WSGI frameworks breaking the flow if needed to go on another route, it simple raising the exception, But if any of the called modules handled those exceptions it fails the flow. and we can't guide the extension modules to don't use full try except as it is the pythons power. What you say? Please excuse me if we have this control already, (can u explain?) I hope, currently no languages gives such full control over coders. -- Thanks Vinoth -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jun 30 14:32:30 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 30 Jun 2009 22:32:30 +1000 Subject: [Python-Dev] PEP 376 In-Reply-To: <4A4927D5.4040001@v.loewis.de> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> Message-ID: <4A4A05DE.2040804@gmail.com> Martin v. L?wis wrote: >> If no one objects, I'd like to push PEP 376 in the "accepted" status >> and go ahead with its implementation, >> with continuous feedback at Distutils-SIG as we did to build it. > > I think this isn't quite the process. In the past, every PEP required > BDFL pronouncement, which you should now seek. Agreed. While Guido is highly likely to just accept the distutils-sig consensus on something like this, that doesn't eliminate the need for him to actually *say* that he is approving the PEP on that basis. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Tue Jun 30 14:36:20 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 30 Jun 2009 22:36:20 +1000 Subject: [Python-Dev] Flow of control - a new way - Idea In-Reply-To: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com> References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com> <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com> Message-ID: <4A4A06C4.1020205@gmail.com> vin vin wrote: > > HI all > > I am not the too technical guy, but thinking about the new way of > controlling the flow instead of throwing an error. This message is too speculative/tentative for python-dev (which is about concrete development of the next version of Python) or even python-ideas (which is the list for speculative proposals that may possibly be considered some day). I suggest taking this question to the general python list (python-list at python.org, also available as the newsgroup comp.lang.python). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ziade.tarek at gmail.com Tue Jun 30 14:38:51 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Jun 2009 14:38:51 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: <4A4A05DE.2040804@gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> Message-ID: <94bdd2610906300538r4c06b142k6c0b5c109e210958@mail.gmail.com> On Tue, Jun 30, 2009 at 2:32 PM, Nick Coghlan wrote: > Martin v. L?wis wrote: >>> If no one objects, I'd like to push PEP 376 in the "accepted" status >>> and go ahead with its implementation, >>> with continuous feedback at Distutils-SIG as we did to build it. >> >> I think this isn't quite the process. In the past, every PEP required >> BDFL pronouncement, which you should now seek. > > Agreed. While Guido is highly likely to just accept the distutils-sig > consensus on something like this, that doesn't eliminate the need for > him to actually *say* that he is approving the PEP on that basis. Sure, I didn't want to bypass the process, I was not really sure about the right move on this PEP since it was based on the summit decisions, I'll wait for Guido decision, thanks. Cheers Tarek -- Tarek Ziad? | http://ziade.org From daniel at stutzbachenterprises.com Tue Jun 30 15:44:09 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 30 Jun 2009 08:44:09 -0500 Subject: [Python-Dev] Flow of control - a new way - Idea In-Reply-To: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com> References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com> <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com> Message-ID: On Tue, Jun 30, 2009 at 1:07 AM, vin vin<4vinoth at gmail.com> wrote: > at that point if we think to move directly to B (what error handler do > if that B has the handler defined of the error), changing the frames > instruction pointer to back to the B's position (in python code without > raising an exception) is a really useful for this web applications. If I understand you correctly, the language feature you are looking for is called a "continuation", which was originally developed in the LISP/Scheme world (as "call/cc"). I'm not familiar with the details, but I believe you can get them in Stackless Python. Hope that helps, -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Jun 30 19:21:02 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 30 Jun 2009 10:21:02 -0700 Subject: [Python-Dev] PEP 376 In-Reply-To: <4A4A05DE.2040804@gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> Message-ID: On Tue, Jun 30, 2009 at 5:32 AM, Nick Coghlan wrote: > Martin v. L?wis wrote: >>> If no one objects, I'd like to push PEP 376 in the "accepted" status >>> and go ahead with its implementation, >>> with continuous feedback at Distutils-SIG as we did to build it. >> >> I think this isn't quite the process. In the past, every PEP required >> BDFL pronouncement, which you should now seek. > > Agreed. While Guido is highly likely to just accept the distutils-sig > consensus on something like this, that doesn't eliminate the need for > him to actually *say* that he is approving the PEP on that basis. Sure. :-) So what *is* the distutils-sig consensus? And is there consensus outside of it? (Remember the ipaddr debacle. It's easy for people to miss an important PEP.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Jun 30 19:44:58 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 30 Jun 2009 13:44:58 -0400 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.co m> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> Message-ID: <20090630174159.731CE3A4099@sparrow.telecommunity.com> At 07:57 PM 6/29/2009 +0200, Tarek Ziad? wrote: >Hello, > >If no one objects, I'd like to push PEP 376 in the "accepted" status >and go ahead with its implementation, >with continuous feedback at Distutils-SIG as we did to build it. I do have a question about the current draft... Do zipped distributions use EGG-INFO or a project-version.egg-info? This isn't spelled out in the PEP, although I get the general idea that the EGG-INFO format isn't supported, and thus the PEP and API do not support existing .egg files. This should probably be made clear, if that's the intention. From p.f.moore at gmail.com Tue Jun 30 20:37:11 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 30 Jun 2009 19:37:11 +0100 Subject: [Python-Dev] PEP 376 In-Reply-To: References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> Message-ID: <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> 2009/6/30 Guido van Rossum : > And is there consensus outside of it? (Remember the ipaddr debacle. > It's easy for people to miss an important PEP.) My impression (as someone who does read the distutils SIG, but in all honesty has very little practical interest in distutils internals): - It's very focused on distutils internals. If it has an impact on end users (as opposed to packagers), it's very hard to discern. The only hint of such a thing is the mention of an uninstall function. But it's only an API. So still no end user impact [1] :-( - The terminology and focus feels setuptools-inspired (my apologies if that's not the case). Expect pushback from setuptools haters... - It's quite dense for the casual reader not familiar with the terminology. I've never managed to read the whole thing through, personally. I'd suggest two things: - Add a section to the PEP describing the purely end user impact of the changes - Post that to python-list, with a note pointing to the PEP for people who care about distutils details If that gets no feedback, you've done as much as you can. Paul. [1] I'd actually like it if the PEP defined an uninstall command - something like "python -m distutils.uninstall packagename". It can be as minimalist as you like, but I'd like to see it present. From martin at v.loewis.de Tue Jun 30 21:18:55 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 30 Jun 2009 21:18:55 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> Message-ID: <4A4A651F.9000805@v.loewis.de> > - Post that to python-list, with a note pointing to the PEP for people > who care about distutils details If that hasn't been done before (I have not followed), it should be done right away. PEP 1 makes it a mandatory requirement for acceptance. Regards, Martin From ziade.tarek at gmail.com Tue Jun 30 21:19:07 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Jun 2009 21:19:07 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> Message-ID: <94bdd2610906301219h3300d610jaf8e376b14de3d42@mail.gmail.com> 2009/6/30 Guido van Rossum : > On Tue, Jun 30, 2009 at 5:32 AM, Nick Coghlan wrote: >> Martin v. L?wis wrote: >>>> If no one objects, I'd like to push PEP 376 in the "accepted" status >>>> and go ahead with its implementation, >>>> with continuous feedback at Distutils-SIG as we did to build it. >>> >>> I think this isn't quite the process. In the past, every PEP required >>> BDFL pronouncement, which you should now seek. >> >> Agreed. While Guido is highly likely to just accept the distutils-sig >> consensus on something like this, that doesn't eliminate the need for >> him to actually *say* that he is approving the PEP on that basis. > > Sure. :-) > > So what *is* the distutils-sig consensus? The consensus is to have one and only one way to install distributions in Python, and a way to retrieve information on installed distributions, and their files. > > And is there consensus outside of it? (Remember the ipaddr debacle. > It's easy for people to miss an important PEP.) I am not sure who you are thinking about, I am blogging a lot on the topic and I am trying to get key players involved in this, like os packagers etc. We've built this PEP in respect with existing tools like setuptools, etc, and I am sending mails at python-dev to make sure evereyone involved in python development has a chance to provide some feedback, > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- Tarek Ziad? | http://ziade.org From tjreedy at udel.edu Tue Jun 30 21:24:34 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 30 Jun 2009 15:24:34 -0400 Subject: [Python-Dev] Flow of control - a new way - Idea In-Reply-To: <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com> References: <6176a14d0906292300r437dee80g9a414ed434da9de5@mail.gmail.com> <6176a14d0906292307l70db1315vb6cf8c171b38a6eb@mail.gmail.com> Message-ID: vin vin wrote: Look up 'trampoline', but ask any further questions on python-list. From ziade.tarek at gmail.com Tue Jun 30 21:28:43 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Jun 2009 21:28:43 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> Message-ID: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com> On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore wrote: > - The terminology and focus feels setuptools-inspired (my apologies if > that's not the case). Expect pushback from setuptools haters... setuptools implemented *needed* features, like a way for developers to browse installed packages. We said during the summit at Pycon that we wanted this feature in Distutils, (Guido said so) So I worked in PEP 376 to introduce them. Now if the fact that we want to introduce the good ideas of setuptools into distutils, (problems Phillip resolved) will make people push it back *even* they are good idead, needed features, is something we need to fight against. > - It's quite dense for the casual reader not familiar with the > terminology. I've never managed to read the whole thing through, > personally. I'll try to add a definitions section, > > I'd suggest two things: > - Add a section to the PEP describing the purely end user impact of the changes > - Post that to python-list, with a note pointing to the PEP for people > who care about distutils details > Ok. > If that gets no feedback, you've done as much as you can. I hope it'll make it one day. If not, I don't understand the goal of the "Python Language Summit' > > Paul. > > [1] I'd actually like it if the PEP defined an uninstall command - > something like "python -m distutils.uninstall packagename". It can be > as minimalist as you like, but I'd like to see it present. it's already there: http://www.python.org/dev/peps/pep-0376/#adding-an-uninstall-function Regards Tarek -- Tarek Ziad? | http://ziade.org From ziade.tarek at gmail.com Tue Jun 30 21:35:50 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 30 Jun 2009 21:35:50 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: <20090630174159.731CE3A4099@sparrow.telecommunity.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <20090630174159.731CE3A4099@sparrow.telecommunity.com> Message-ID: <94bdd2610906301235g60bfea7dsb67a72a04213d8d0@mail.gmail.com> 2009/6/30 P.J. Eby : > At 07:57 PM 6/29/2009 +0200, Tarek Ziad? wrote: >> >> Hello, >> >> If no one objects, I'd like to push PEP 376 in the "accepted" status >> and go ahead with its implementation, >> with continuous feedback at Distutils-SIG as we did to build it. > > I do have a question about the current draft... ?Do zipped distributions use > EGG-INFO or a project-version.egg-info? ?This isn't spelled out in the PEP, > although I get the general idea that the EGG-INFO format isn't supported, > and thus the PEP and API do not support existing .egg files. ?This should > probably be made clear, if that's the intention. The intention is to have only one standard for the egg-info, I have explained it in this section: http://www.python.org/dev/peps/pep-0376/#egg-info-becomes-a-directory previous formats will not be supported but that won't break anything of course since the new APIs will work only over the distribution installed with the new version of distutils. > > -- Tarek Ziad? | http://ziade.org From martin at v.loewis.de Tue Jun 30 21:49:18 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 30 Jun 2009 21:49:18 +0200 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com> Message-ID: <4A4A6C3E.2090005@v.loewis.de> Tarek Ziad? wrote: > On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore wrote: >> - The terminology and focus feels setuptools-inspired (my apologies if >> that's not the case). Expect pushback from setuptools haters... > > setuptools implemented *needed* features, like a way for developers to browse > installed packages. That doesn't imply that setuptools itself is needed. I can browse installed packages with dpkg -l, or "add-and-remove programs". > Now if the fact that we want to introduce the good ideas of setuptools > into distutils, > (problems Phillip resolved) will make people push it back *even* they > are good idead, needed features, > is something we need to fight against. Assuming "we" have consensus before we start fighting against others. FWIW, I abstain from commenting on PEP 376. I don't need it, but it doesn't seem to hurt having it, especially since "egg-info" already managed to sneak in. > I hope it'll make it one day. If not, I don't understand the goal of > the "Python Language Summit' Just be patient. Both Python 2.7 and 3.2 are still many months ahead, so there is no urgency (AFAICT). Regards, Martin From Scott.Daniels at Acm.Org Tue Jun 30 22:06:07 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Tue, 30 Jun 2009 13:06:07 -0700 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com> Message-ID: Tarek Ziad? wrote: > On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore wrote: >> [1] I'd actually like it if the PEP defined an uninstall command - >> something like "python -m distutils.uninstall packagename". It can be >> as minimalist as you like, but I'd like to see it present. > > it's already there: > > http://www.python.org/dev/peps/pep-0376/#adding-an-uninstall-function That (at least as I read it) is a function, not a command. If it is a command, give an example of its use from the command line for us poor "don't want to research" people. If the following works: $ python setup.py uninstall some_package Then explicitly say so for us poor schlubs. --Scott David Daniels Scott.Daniels at Acm.Org From p.f.moore at gmail.com Tue Jun 30 22:11:44 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 30 Jun 2009 21:11:44 +0100 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <4A4927D5.4040001@v.loewis.de> <4A4A05DE.2040804@gmail.com> <79990c6b0906301137r3728c86atcca4cf2da61cdf7b@mail.gmail.com> <94bdd2610906301228s4ac0fb20id44bd144b82af600@mail.gmail.com> Message-ID: <79990c6b0906301311l4fd029dcq7ec2b4f427eb6bfb@mail.gmail.com> 2009/6/30 Tarek Ziad? : > On Tue, Jun 30, 2009 at 8:37 PM, Paul Moore wrote: >> - The terminology and focus feels setuptools-inspired (my apologies if >> that's not the case). Expect pushback from setuptools haters... > > setuptools implemented *needed* features, like a way for developers to browse > installed packages. No problem with any of that. Please understand that on the whole, I'm in favour of the setuptools features. (While I dislike intensely some of the setuptools "ecosystem", such as easy_install. In fact, one of my goals in wanting this PEP accepted is to ensure that I can get the "good bits" of setuptools, without having to buy into the easy_install, dependency management, automated download infrastructure that currently goes with it). > We said during the summit at Pycon that we wanted this feature in > Distutils, (Guido said so) "We" in this context denotes the people at the summit. Please remember that people who weren't there still have an opinion - and it may well differ. I'm not saying that it *does* differ, just that the view of the summit should not be viewed as conclusive (unless the way Python is developed is very different from how I understand it). > So I worked in PEP 376 to introduce them. And as with any PEP, that's a difficult and thankless task (I got a taste myself with PEP 302, and that was far easier), so just for the record, you have my thanks for doing this. > Now if the fact that we want to introduce the good ideas of setuptools > into distutils, > (problems Phillip resolved) will make people push it back *even* they > are good idead, needed features, > ?is something we need to fight against. But there *are* issues with setuptools. Some users have mentioned them on a number of occasions. I've raised a few myself. By all means introduce the good ideas into the core - but make sure people agree that the ideas *are* good. And remember - all I said was that people may react against the setuptools-style terminology. Not that they dislike the *idea*, just that they might dislike the way it's presented. After all, one common complaint with setuptools (and one that I agree with) is that its documentation is obscure to the point of being hostile. >> - It's quite dense for the casual reader not familiar with the >> terminology. I've never managed to read the whole thing through, >> personally. > > I'll try to add a definitions section, Thank you. I'll try to make the time to go through the PEP and comment more fully. >> [1] I'd actually like it if the PEP defined an uninstall command - >> something like "python -m distutils.uninstall packagename". It can be >> as minimalist as you like, but I'd like to see it present. > > it's already there: > > http://www.python.org/dev/peps/pep-0376/#adding-an-uninstall-function No, that defines an API, which as stated in the PEP, "allows a third-party application to use the uninstall function and make sure it's the only program that can remove a distribution it has previously installed". It does NOT define a standard uninstall command, to complement the standard install command. Paul. From ncoghlan at gmail.com Tue Jun 30 23:11:36 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 01 Jul 2009 07:11:36 +1000 Subject: [Python-Dev] PEP 376 In-Reply-To: <94bdd2610906301235g60bfea7dsb67a72a04213d8d0@mail.gmail.com> References: <94bdd2610906291057i3d2f533dqcfb3c986a5946835@mail.gmail.com> <20090630174159.731CE3A4099@sparrow.telecommunity.com> <94bdd2610906301235g60bfea7dsb67a72a04213d8d0@mail.gmail.com> Message-ID: <4A4A7F88.7080609@gmail.com> Tarek Ziad? wrote: > 2009/6/30 P.J. Eby : >> At 07:57 PM 6/29/2009 +0200, Tarek Ziad? wrote: >>> Hello, >>> >>> If no one objects, I'd like to push PEP 376 in the "accepted" >>> status and go ahead with its implementation, with continuous >>> feedback at Distutils-SIG as we did to build it. >> I do have a question about the current draft... Do zipped >> distributions use EGG-INFO or a project-version.egg-info? This >> isn't spelled out in the PEP, although I get the general idea that >> the EGG-INFO format isn't supported, and thus the PEP and API do >> not support existing .egg files. This should probably be made >> clear, if that's the intention. > > The intention is to have only one standard for the egg-info, I have > explained it in this section: > > http://www.python.org/dev/peps/pep-0376/#egg-info-becomes-a-directory > > > previous formats will not be supported but that won't break anything > of course since the new APIs will work only over the distribution > installed with the new version of distutils. To address PJE's question in the PEP, it may be worth expanding on this in the backwards compatibility section explaining how the new distutils metadata system avoids getting confused by the old pre-standardisation installation formats (e.g. it may be that the directory names and/or filenames all deliberately differ from current approaches precisely so they can coexist without interfering with each other) Also, I find the following paragraph (near the start of the section linked above) confusing: [PEP 376] > Notice that this change is based on the standard proposed by > EggFormats, although this standard proposes two ways to install > files: > > * A self-contained directory that can be zipped or left unzipped and > contains the distribution files and the .egg-info directory. > * A distinct .egg-info directory located in the site-packages directory. This is unclear as to what "this standard" is referring to (since the PEP itself is proposing a standard, but the sentence is also referring to the existing EggFormats de facto standard). If the PEP only supports the latter of the two options (which appears to be the case) it should say so explicitly. Another minor nit from the same section: [PEP 376] > Any '-' characters are currently replaced with '_'. This should say something like "Any '-' characters other than the one in 'egg-info' and the one separating the name from the version number are included in the replacement of non-alphanumeric characters with '_'" Other questions/comments: - What is a "local absolute path"? Absolute path I understand, relative path I understand, but "local absolute" is a novel term to me. - In DistributionDirMap constructor description: "is not not" -> "is not None" - as Paul Moore suggested, exposing a distutils based uninstall command at the command line may make sense (and if it doesn't, the PEP should say why not) Otherwise, from a non-expert point of view I actually find the PEP pretty readable and clear, and it all looks like a pretty sensible design to me. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia ---------------------------------------------------------------